Skip to content

feat(api): carry the capability ledger + footprint on a failed run's error - #136

Merged
ivarvong merged 1 commit into
mainfrom
error-carries-ledger
Jun 30, 2026
Merged

ivarvong merged 1 commit into
mainfrom
error-carries-ledger

Conversation

@ivarvong

Copy link
Copy Markdown
Collaborator

Why

The library gap we hit while building the sandbox example: to audit a failed run — what did the program touch before it crashed, and how much did it spend — you had to attach a :telemetry handler and correlate it back to the run. That's plumbing the library should make unnecessary. Auditing, policy-gating, and billing a failed run should be a value on the return, not a side channel.

What

{:error, %Pyex.Error{}} now carries:

  • runtime_spans — the capability ledger of every storage/IO op the program performed before it broke (the same unforgeable, host-rendered spans you get on success).
  • footprint — the resource usage at the point of failure (steps, compute, duration_ms, memory_bytes, output_bytes).

Both are []/nil for errors raised before execution (e.g. syntax) — correct, since nothing ran. On success the same is read off ctx via Pyex.Turn/Pyex.Ctx.runtime_spans, so the ledger is now first-class on every outcome.

The error path already computed both for the [:pyex, :run, :exception] telemetry event — this just stops discarding them. Consistent with the boundary we've held all along: the library exposes the value; the caller composes audit / policy / billing.

Payoff, demonstrated

examples/sandbox_server.exs drops its per-worker telemetry-capture handler entirely and reads the ledger straight off the error. Verified over curl — an erroring program still returns its db.set/db.get ledger + usage:

verdict: error  usage.steps: 3
── runtime · scope=pyex · 2 spans ──
━━━━━━━━     db.set   bytes=1 db.collection.name="k" db.operation.name="set" …
        ━━━━ db.get   hit=true …

Compatibility

Pre-release; additive struct fields. Existing {:error, %Pyex.Error{}} matchers are unaffected.

Gate

mix format ✅ · mix compile --warnings-as-errors ✅ · mix test (6185) ✅ · mix dialyzer ✅

🤖 Generated with Claude Code

https://claude.ai/code/session_019NokzcR7BiAigPgC78zpk9

…error

A failed `Pyex.run` now returns a `%Pyex.Error{}` that carries `runtime_spans`
(the capability ledger of what the program touched before it broke) and
`footprint` (its resource usage). Auditing, policy-gating, and billing a
*failed* run no longer requires attaching a telemetry handler — it's a value
on the return. Both are empty/nil for errors raised before execution (syntax),
which is correct: nothing ran, nothing was touched.

The error path already computed both for the [:pyex, :run, :exception] event;
this just stops throwing them away. On success the same is read off `ctx` via
`Pyex.Turn`/`Pyex.Ctx.runtime_spans`, so the ledger is now first-class on every
outcome. The library exposes the value; callers compose audit/policy/billing.

Simplifies examples/sandbox_server.exs accordingly: it reads the ledger
straight off the error instead of a per-worker telemetry-capture handler.

Pre-release; additive struct fields, non-breaking for `{:error, %Pyex.Error{}}`
matchers. Full suite + Dialyzer green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019NokzcR7BiAigPgC78zpk9
@ivarvong
ivarvong merged commit af4c078 into main Jun 30, 2026
18 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant