diff --git a/CHANGELOG.md b/CHANGELOG.md index 28bf45bf..909e42ce 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,70 @@ # Changelog +## 0.232.0 + +A child that DROPS now keeps its transcript, the receipt that says so has moved to where an +unavailable trace cannot hide it, and the thing itself is finally called what it is. + +**Renamed.** 0.229.0 shipped this as `NativeSession*`. "Native" answered a question nobody was +asking — native to what, as against what? — and the honest contrast is not native/non-native: it +is the HARNESS's own first-person session file (`.claude/projects/*/session.jsonl` and its Codex +/ OpenCode peers) against the SUPERVISOR's second-hand `trace` receipt of tool spans. So the +vocabulary is now `HarnessTranscript*` throughout: `HarnessTranscriptEvidence`, +`HarnessTranscriptArtifact`, `HarnessTranscriptFile`, `HARNESS_TRANSCRIPT_SCHEMA_VERSION`, +`captureHarnessTranscript`, and the field `harnessTranscript`. `src/runtime/harness-transcript.ts` +replaces `src/runtime/native-session-evidence.ts`. Nothing had time to depend on the old names. + +0.229.0 captured a sandbox child's harness transcript just before the settled result was built, +which covers the settled path and is the right seam for it. A child that settles `down` has no +result artifact for the capture to ride in, so its transcript died at destroy exactly as before +— and these are the children an operator most wants to read, because they are the ones that +failed. Measured 2026-09-15 on three capability-per-parameter pursuits under 0.226.0/0.227.0: 45 +children in one evening executed, reasoned, and settled `down` with nothing. One run lost 38 of +68 inside a ten-minute window (#1244). + +- The capture now also runs in `providerAsExecutor`'s failure path, at the last point the + environment is still live — the `finally` that destroys it runs after, so this is the only + moment a dropped child's session files can be read at all. +- `Executor` gains an optional `harnessTranscript()`, read on settle and valid after `execute` + resolves OR throws, exactly like the existing `metered()`. The scope carries the answer on + every settlement arm, done and down alike. +- **Breaking:** `WorkerTraceEvidence.nativeSession` is removed. It was declared in 0.229.0, + documented as a read path, and assigned by nothing — every settlement carried `undefined` + there, including successful ones. It also could not have worked on the path that needed it: + it sat inside the `status: 'available'` arm, and a child that drops has zero tool spans, so + its trace is `unavailable` and the field had no home. The receipt is now `harnessTranscript` on + the settlement itself, a SIBLING of `trace`, on `Settled` (both arms) and on the journal's + `settled` record. `ProviderLeafOut.nativeSession` (0.229.0's inline carrier) is removed too; see below. +- Four absences are now told apart instead of collapsing into one silence, because an operator + acts differently on each: `execution-never-started` (no environment was ever created — the + `maxUsd` admission refusal of #1240), `capture-did-not-run` (a box existed and the abort path + closed the stream before the capture could run), `executor-exposes-no-transcript` (a CLI + or in-process child that has no transcript by construction), and the capture's own reasons + (`unsupported-environment`, `enumeration-failed`, `no-transcript`, `unknown-harness`). + None of them is ever an empty artifact that reads as coverage. + +**The transcript is now its own blob, and the receipt is a pointer (#1248).** 0.229.0 inlined the +files as `content` strings inside the settled result, so every replay and resume of a child +re-read and re-parsed up to 16 MiB of transcript whether or not anyone wanted it, and a 132-child +fleet at the ceiling carried ~2.1 GiB inside its result blobs. Now the scope persists each capture +under its own content ref in the `ResultBlobStore` — the way the tool-span trace already is — and +the settlement carries `harnessTranscript: { status: 'available', transcriptRef, harness, +fileCount, totalBytes, skippedCount }`. Replay rehydrates a ref; the bytes are a disk number and +nobody pays for a transcript until they open it with the new `harnessTranscriptArtifact(evidence, +blobs)`. Consequences, all deliberate: + +- `ProviderLeafOut.nativeSession` is gone: the executor reports the in-memory + `HarnessTranscriptCapture` through its port and never touches storage, which restores the + "no blob sink in any provider or destroy site" property 0.229.0 set out to keep. +- The per-child bounds stay 2 MiB / 16 MiB / 1000 files, now enforced exactly (after each read, + so a child settles at or under 16 MiB instead of 16 MiB plus one file), and the module comment + carries the fleet multiplication so the next person sizing a fleet finds it. +- `transcript-persistence-failed` names a capture that existed in memory and never reached disk, + mirroring `trace-persistence-failed`. + +Settlements recorded before this release carry no `harnessTranscript` on the record, which is not +the same fact as a recorded `unavailable`, and stays absent rather than defaulting. + ## 0.231.1 Allow Sandbox 0.40.x beside Runtime without a consumer dependency override. @@ -27,8 +92,9 @@ Related system issue: SYS-011 — outcome: mitigates — proof: agent-runtime#12 teardown. Runtime reads the child's Claude Code / Codex / OpenCode session files out of the environment while it is still live and settles them with the result, so they reach a run record instead of being destroyed with the environment. Read them from - `ProviderLeafOut.nativeSession` on the settled artifact, or from - `WorkerTraceEvidence.nativeSession` on the receipt. The existing `trace` receipt is + `ProviderLeafOut.nativeSession` on the settled artifact. (This entry also named + `WorkerTraceEvidence.nativeSession`; that field was declared and never written by any code + path, and 0.232.0 removes it — see below.) The existing `trace` receipt is unchanged and still carries the supervisor's tool spans only, so `status: 'available'` there never meant the transcript survived; this is the field that says. An environment whose provider does not expose file reads reports a reason rather than an empty artifact, diff --git a/api-surface.json b/api-surface.json index 871e97b3..7a846592 100644 --- a/api-surface.json +++ b/api-surface.json @@ -916,7 +916,7 @@ "ExecRequest": "type 1ae7193c5ae7", "ExecResult": "type 88076a0a7f30", "ExecutionBindingReceipt": "type 65ebc4a19a73", - "Executor": "type 9302cd427d44", + "Executor": "type 921d3be70147", "ExecutorAccounting": "type 9902bc52553b", "ExecutorCancellation": "type a570eaf2765f", "ExecutorCancellationRequest": "type c9bf674c803e", @@ -953,6 +953,12 @@ "GraphResult": "type 9a2057bac283", "GroupOf": "type 40205210c0cb", "Handle": "type 2022f38d425c", + "HarnessTranscriptArtifact": "type be01cf3bec5f", + "HarnessTranscriptCapture": "type 0aa901e6d558", + "HarnessTranscriptEvidence": "type 90eb87444ae8", + "HarnessTranscriptFile": "type d04b19d418e7", + "HarnessTranscriptUnavailable": "type c85a2618af85", + "HarnessTranscriptUnavailableReason": "type efd68e08fb76", "HarnessUsage": "type 49b62b4cb06e", "HarvestCorpusOptions": "type 0a848c6cece4", "HarvestError": "value 24d31a2ce396", @@ -1033,9 +1039,6 @@ "NativeContextContinuationExecution": "type f08e18682191", "NativeContextContinuationHandle": "type dcfa1c093ab0", "NativeContextContinuationInput": "type 888bd32794fb", - "NativeSessionArtifact": "type 2ebadd0631dc", - "NativeSessionEvidence": "type 2f725b69d533", - "NativeSessionFile": "type d04b19d418e7", "NoProgressForOptions": "type edf172313df3", "NoWinnerError": "type adcf245eb125", "NodeExecutionIdentity": "type 76b27bc384d1", @@ -1096,7 +1099,7 @@ "PromptRegistry": "type 0170415f6590", "ProviderAsSandboxClientOptions": "type 5078ed438691", "ProviderExecutorOptions": "type a5f55b639c6c", - "ProviderLeafOut": "type 4fce055567a8", + "ProviderLeafOut": "type bb07fca9c162", "ProviderModelAttemptEvidence": "type 9c62f347cffb", "ProviderModelExecutionEvidence": "type d66858ac77d4", "ProviderPlacement": "type 5e7f4e08cc0a", @@ -1108,6 +1111,7 @@ "PublishOptions": "type 7eaa6111fd39", "QuestionEscalationOutcome": "type 8d7b45b7a120", "QuestionEscalationRecord": "type 074db62e6b90", + "ReadableEnvironment": "type cc52350c1576", "ReconnectRetainedInteractiveRunOptions": "type 4d5e7836f49d", "ReconnectRetainedRunOptions": "type 48ac412d7fab", "RecoverRetainedInteractiveRunOptions": "type 668404fb7d48", @@ -1205,13 +1209,13 @@ "SessionCapableBox": "type 9db08b40cab8", "SessionMessageLike": "type 726215634de7", "SessionTraceBox": "type b8d989093064", - "Settled": "type 64d1f703d9a3", + "Settled": "type 4694c79a44d1", "ShapeBudget": "type a1cb1a8d0995", "ShapeContext": "type a266c1b7e746", "ShapeRegistry": "type b567a5be55ec", "Shell": "type c676fe970f79", "ShotSpec": "type bf645fd74234", - "SpawnEvent": "type 52fbc9791642", + "SpawnEvent": "type e74c7b38c734", "SpawnForest": "type 7308f34da226", "SpawnForestEvent": "type 230e369c6548", "SpawnForestInDoubtNode": "type 4e4bddd6a7d9", @@ -1347,7 +1351,7 @@ "WorkerSteerAcknowledgement": "type ae4a55ff95a0", "WorkerSteerRequest": "type f52aab62492f", "WorkerToolTraceArtifact": "type 2dd5d4070071", - "WorkerTraceEvidence": "type 78d0bc00c694", + "WorkerTraceEvidence": "type 34812cbffcbf", "WorkerTraceResolver": "type 0e6d6b0af580", "WorkerTraceSeamCarrier": "type 61c70b089b56", "WorkerTraceUnavailableReason": "type 0dcb14d1071b", @@ -1396,6 +1400,7 @@ "cancelRun": "value 67d3c8656535", "cancelWorker": "value 93b2f7c7f8e4", "canonicalFindingEvent": "value 95036e8759ed", + "captureHarnessTranscript": "value a602c5424a3b", "captureWorkerTraceEvidence": "value 3f9a8366bce0", "chatTransportExecutor": "value f4890f34730e", "chatWorkerSeam": "value e61d1b275e53", @@ -1486,6 +1491,7 @@ "fsSurfaceReader": "value 50d9c6450932", "gateOnDeliverable": "value 3464e39aed65", "gitWorkspace": "value e6c2ccd429c9", + "harnessTranscriptArtifact": "value 5ccffcc259e2", "harnessUsageIsEmpty": "value 01877c3cb8c2", "harvestCorpus": "value a0edf1ecc26a", "harvestSurfaceDiffs": "value b832c832cb1e", @@ -1528,6 +1534,7 @@ "peerMailTools": "value 7067f2f655b0", "peerMailVerbNames": "value 373728f5643d", "pendingWaits": "value ac8d8460a665", + "persistHarnessTranscript": "value af83a8658324", "pickBestDelivered": "value 7e9dfe8b50d4", "pickChampion": "value 0a78daca150f", "pipeline": "value 85b24129e65b", diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index 43b00003..db741600 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -7,7 +7,7 @@ # Primitive catalog — the never-stale anti-reinvention inventory -> **GENERATED** from `@tangle-network/agent-runtime@0.231.1` and `@tangle-network/agent-eval@0.182.0` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. +> **GENERATED** from `@tangle-network/agent-runtime@0.232.0` and `@tangle-network/agent-eval@0.182.0` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. ## 1. agent-runtime — own public surface @@ -423,7 +423,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 167 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 947 exports. +Import from `@tangle-network/agent-runtime/kernel` — 954 exports. | Symbol | Kind | Summary | |---|---|---| @@ -454,6 +454,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 947 exports. | `cancelWorker` | function | Request the cancellation of ONE worker, idempotently, and return the operation's current | | `canDisplace` | function | The repair keep-best guard: a challenger displaces the incumbent only when it is | | `canonicalFindingEvent` | function | Producer-side cleanliness for the `finding` event. The findings payload is arbitrary analyst | +| `captureHarnessTranscript` | function | Read the harness transcript out of one LIVE environment. | | `captureWorkerTraceEvidence` | function | Collect and persist one executor's structured tool trace without changing its task outcome. | | `chatTransportExecutor` | function | Build one exact profile-driven chat executor through `createExecutor`. | | `chatWorkerSeam` | function | Session-owning worker factory for graph continuity. | @@ -532,6 +533,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 947 exports. | `fsSurfaceReader` | function | A {@link SurfaceReader} over the local filesystem, for worktree/local workers. Every path — | | `gateOnDeliverable` | function | Wrap an `Executor` so its settlement `valid` reflects the deliverable check, not the | | `gitWorkspace` | function | A `Workspace` over a git checkout: materialize an isolated worktree at `ref`, commit produced changes (conflict-aware), and read `head` — hooks disabled, identity pinned. | +| `harnessTranscriptArtifact` | function | Rehydrate the exact persisted transcript a receipt points at, or `undefined` when the receipt | | `harnessUsageIsEmpty` | function | True when a report states any spend at all. | | `harvestCorpus` | function | Batch the selected observation implementation over completed runs and retain its findings. | | `harvestSurfaceDiffs` | function | Re-read every mounted (and watched) surface and report the ones whose settled state differs from | @@ -571,6 +573,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 947 exports. | `patchDelivered` | function | Build the `DeliverableSpec`: `check(artifact)` runs the shared mechanical | | `peerMailTools` | function | The two tools ONE capability serves. `capabilityId` is closed over and `from` is not a parameter, | | `pendingWaits` | function | The waits a journaled tree shows as ARMED but never woken — what a resumed run re-arms with the | +| `persistHarnessTranscript` | function | Persist a capture under its own content ref and return the receipt a settlement carries. | | `pickBestDelivered` | function | The single argmax both the default finalizer and `finalizeBestDelivered` share: highest | | `pickChampion` | function | The champion pick over a means table. 'score' takes the best mean score (ties → | | `pipeline` | function | `pipeline(stages)` — run the stages in order, feeding each stage's `done` deliverable into the | @@ -940,6 +943,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 947 exports. | `ProvisionSupervisorConnection` | interface | Caller-supplied provider or Sandbox SDK connection for one supervisor run. | | `ProvisionSupervisorRequest` | interface | Input to the public Runtime supervisor provisioner. | | `QuestionEscalationRecord` | interface | The operator-facing artifact written for every `ask_parent`: which question left, whether | +| `ReadableEnvironment` | interface | The two optional environment reads the capture needs. Public because `captureHarnessTranscript` | | `ReconnectRetainedInteractiveRunOptions` | interface | Reconstruct one exact provider-owned native coding-agent process. | | `ReconnectRetainedRunOptions` | interface | Inputs sufficient to rebuild a control client in a new process. | | `RecoverRetainedInteractiveRunOptions` | interface | Recover a start after a pre-create crash or a lost provider response. | @@ -1117,6 +1121,9 @@ Import from `@tangle-network/agent-runtime/kernel` — 947 exports. | `FanoutWinnerSelector` | type | A winner-selection strategy: argmax/sort over the gathered child iterations (each output is the | | `FlatWidenGate` | type | The flat default `ScopeWidenGate` factory contract — never widens, keeping the R2 firewall | | `GroupOf` | type | The axis (matrix column) a record contributes to — default the scenario group. | +| `HarnessTranscriptCapture` | type | What the executor holds in memory between the read and the settle: the files, inline. | +| `HarnessTranscriptEvidence` | type | The durable receipt on a settlement: a content-addressed pointer to a persisted | +| `HarnessTranscriptUnavailableReason` | type | Why no transcript reached a record, from either the capture or the settle path. | | `InProcessOnPrompt` | type | The user callback: given a prompt and its round, produce the box's event | | `InteractiveWorkerEnvironment` | type | Environment fields supplied to every interactive worker after Runtime adds the exact profile. | | `LoopOptionsForDispatch` | type | runAgentRounds options minus the `ctx` (loopDispatch builds the ctx). | @@ -1209,7 +1216,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 947 exports. | `WorktreeCheckRunner` | type | The single shell-command-in-worktree runner seam (replaces the per-executor copies). | | `WorktreePatchArtifact` | type | Terminal artifact of one worktree-CLI run — the canonical worktree-harness result (the captured | -**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnInput`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CodeModeOptions`, `CoordinationAuthentication`, `CoordinationHttpAudit`, `CoordinationMcpHandle`, `CoordinationPublicAddress`, `CoordinationTransportOptions`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InPlaceCliExecutorOptions`, `InProcessSandboxClientOptions`, `IntentAudit`, `IsolatedCheckOptions`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NativeSessionArtifact`, `NativeSessionFile`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PeerMailbox`, `PeerMailboxOptions`, `PeerMailSendInput`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `WorkerSpawnRetryHooks`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `HarvestCorpusOptions`, `InboxMessage`, `IsolatedCheckResult`, `LoopTraceEvent`, `MakeWorkerAgent`, `NativeSessionEvidence`, `ObserveOptions`, `PeerMailOutcome`, `RepairStop`, `SandboxControlClient`, `UsageEvent`, `WorkspaceCommit`. +**Undocumented supporting types** (add a TSDoc line at the declaration to earn a table row): `AcquireOptions`, `AgentEnvironment`, `AgentEnvironmentCapabilities`, `AgentEnvironmentEvent`, `AgentEnvironmentProvider`, `AgentEnvironmentQuery`, `AgentEnvironmentSummary`, `AgentGraph`, `AgenticOptions`, `AgenticRunResult`, `AgenticTask`, `AgenticTool`, `AgentSession`, `AgentSessionRef`, `AgentTurnInput`, `AgentTurnResult`, `AllWorkersStalledOptions`, `AnalystRegistry`, `AnytimeReport`, `AnytimeStrategySummary`, `AnytimeTaskCurve`, `ArtifactHandle`, `AuditIntentInput`, `AuditIntentOptions`, `AuthoredHarness`, `AuthoredStrategy`, `AuthorStrategyOptions`, `BenchmarkConfig`, `BenchmarkLift`, `BenchmarkStrategySummary`, `BenchmarkTaskRow`, `BudgetPool`, `BusStats`, `ChampionPick`, `CheckpointRef`, `CheckpointRequest`, `CheckRunContext`, `CliWorktreeBridgeSeam`, `CodeModeOptions`, `CoordinationAuthentication`, `CoordinationHttpAudit`, `CoordinationMcpHandle`, `CoordinationPublicAddress`, `CoordinationTransportOptions`, `CopyOptions`, `CorpusReadbackOptions`, `CreateAgentEnvironmentInput`, `CreateTangleSandboxExactProcessProviderOptions`, `DefinedLeaderboard`, `DispatchReport`, `Driver`, `EvolutionArchiveNode`, `EvolutionAuthor`, `EvolutionBandInfo`, `EvolutionCandidate`, `EvolutionGeneration`, `EvolutionReport`, `ExecRequest`, `ExecResult`, `ExecutorResultMapping`, `ForkRequest`, `GitWorkspaceOptions`, `GraphResult`, `HarnessTranscriptArtifact`, `HarnessTranscriptFile`, `HarnessTranscriptUnavailable`, `HarvestFailure`, `HarvestReport`, `Inbox`, `InPlaceCliExecutorOptions`, `InProcessSandboxClientOptions`, `IntentAudit`, `IsolatedCheckOptions`, `Iteration`, `Leaderboard`, `LeaderboardOptions`, `LocalSandboxClientOptions`, `LoopDecisionPayload`, `LoopDispatchOptions`, `LoopEndedPayload`, `LoopIterationEndedPayload`, `LoopIterationStartedPayload`, `LoopPlanDescription`, `LoopResult`, `LoopSandboxPlacement`, `LoopStartedPayload`, `LoopTraceEmitter`, `LoopWinner`, `MaterializeLocalMcpOptions`, `McpEnvironmentOptions`, `McpToolDescriptor`, `NodeSnapshot`, `NoProgressForOptions`, `Observation`, `ObserveInput`, `OpenSandboxRunOptions`, `PairwiseOptions`, `PatchDeliverableOptions`, `PeerMailbox`, `PeerMailboxOptions`, `PeerMailSendInput`, `PlacementInfo`, `PlateauOptions`, `ProgressTrackerOptions`, `PromotionGateOptions`, `PromotionVerdict`, `PublishOptions`, `ReproductionCheck`, `ResolveSandboxClientOptions`, `ResourceRequest`, `RollingDispatchOptions`, `RunAgenticOptions`, `RunAgentRoundsOptions`, `SandboxRun`, `ShotSpec`, `SpawnOpts`, `StdioMcpConnection`, `StdioMcpServerSpec`, `SteerableSandboxArgs`, `Strategy`, `StrategyEvolutionConfig`, `StrategyResult`, `StreamAgentTurnOptions`, `StructuralRolloutConfig`, `SuperviseOptions`, `SuperviseSurfaceOptions`, `SupervisorAgentDeps`, `SupervisorOpts`, `SupervisorSpanOptions`, `SupervisorSpanRecorder`, `SurfaceScore`, `ToolSpec`, `ToolStepInput`, `TraceSource`, `TrajectoryAnalysis`, `UntrackedCopyStats`, `ValidationCtx`, `Validator`, `VerifierEnvironmentOptions`, `WatchTraceOptions`, `WaterfallCollector`, `WaterfallReport`, `WaterfallSpan`, `WorkerEvidenceInput`, `WorkerSpawnRetryHooks`, `Workspace`, `WorkspaceRequest`, `WorkspaceRun`, `WorktreeCliExecutorOptions`, `WorktreeFanoutOptions`, `AgentEnvironmentStatus`, `AgentSessionStatus`, `ChampionPolicy`, `EdgeDeliveryOutcome`, `GraphEdge`, `HarvestCorpusOptions`, `InboxMessage`, `IsolatedCheckResult`, `LoopTraceEvent`, `MakeWorkerAgent`, `ObserveOptions`, `PeerMailOutcome`, `RepairStop`, `SandboxControlClient`, `UsageEvent`, `WorkspaceCommit`. ### Analyst loop — trace findings on a running loop diff --git a/docs/api/runtime.md b/docs/api/runtime.md index ad2cb801..6b49964a 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -1022,7 +1022,7 @@ One flattened node with the journal tree that owns its records. ###### Inherited from -[`NodeSnapshot`](#nodesnapshot).[`status`](#status-16) +[`NodeSnapshot`](#nodesnapshot).[`status`](#status-17) ##### runtime @@ -4106,19 +4106,6 @@ Read a part's text from `part.text`; a retained frame's `delta` is only that fra How many streamed part updates the archive left out because a later frame superseded them. -##### nativeSession? - -> `optional` **nativeSession?**: [`NativeSessionEvidence`](#nativesessionevidence) - -**`Experimental`** - -The child's own harness transcript, read before the environment was destroyed. - -`events` above is the provider's stream and `trace` on the settlement is the supervisor's -tool spans; neither carries the harness's session files, which used to die with the -environment. Always present on the settled path: an environment that cannot be read says -so with a `reason` rather than being silently absent. #1214. - *** ### ProviderExecutorOptions @@ -4240,6 +4227,115 @@ Map the task while retaining the kernel's canonical prompt mapping by default. *** +### HarnessTranscriptFile + +#### Properties + +##### path + +> `readonly` **path**: `string` + +##### bytes + +> `readonly` **bytes**: `number` + +##### content + +> `readonly` **content**: `string` + +*** + +### HarnessTranscriptArtifact + +#### Properties + +##### schemaVersion + +> `readonly` **schemaVersion**: `1` + +##### harness + +> `readonly` **harness**: `string` + +##### files + +> `readonly` **files**: readonly [`HarnessTranscriptFile`](#harnesstranscriptfile)[] + +##### skipped + +> `readonly` **skipped**: readonly `object`[] + +Paths found but not read, with why — a gap named is a gap an operator can act on. + +*** + +### HarnessTranscriptUnavailable + +#### Properties + +##### status + +> `readonly` **status**: `"unavailable"` + +##### reason + +> `readonly` **reason**: [`HarnessTranscriptUnavailableReason`](#harnesstranscriptunavailablereason) + +##### skipped? + +> `readonly` `optional` **skipped?**: readonly `object`[] + +Present with `nothing-carried`: the paths that existed and why each was not read. + +*** + +### ReadableEnvironment + +The two optional environment reads the capture needs. Public because `captureHarnessTranscript` + is, so a BYO executor can satisfy it with any box that offers a bounded `read` and an `exec`. + +#### Properties + +##### read? + +> `readonly` `optional` **read?**: (`path`, `options?`) => `Promise`\<`string`\> + +###### Parameters + +###### path + +`string` + +###### options? + +###### signal? + +`AbortSignal` + +###### Returns + +`Promise`\<`string`\> + +##### exec? + +> `readonly` `optional` **exec?**: (`command`, `options?`) => `Promise`\<\{ `stdout?`: `string`; `exitCode?`: `number`; \}\> + +###### Parameters + +###### command + +`string` + +###### options? + +`Record`\<`string`, `unknown`\> + +###### Returns + +`Promise`\<\{ `stdout?`: `string`; `exitCode?`: `number`; \}\> + +*** + ### HarnessUsage One harness's own token-usage report for one turn, in the runtime's field names. @@ -4993,48 +5089,6 @@ Restrict/order the server's tools per task (e.g. the task's selected_tools). Def *** -### NativeSessionFile - -#### Properties - -##### path - -> `readonly` **path**: `string` - -##### bytes - -> `readonly` **bytes**: `number` - -##### content - -> `readonly` **content**: `string` - -*** - -### NativeSessionArtifact - -#### Properties - -##### schemaVersion - -> `readonly` **schemaVersion**: `1` - -##### harness - -> `readonly` **harness**: `string` - -##### files - -> `readonly` **files**: readonly [`NativeSessionFile`](#nativesessionfile)[] - -##### skipped - -> `readonly` **skipped**: readonly `object`[] - -Paths found but not read, with why — a gap named is a gap an operator can act on. - -*** - ### ObserveInput #### Properties @@ -6535,6 +6589,16 @@ Provider model evidence for every inference attempt owned by this node. Structured tool evidence captured before this settlement was journaled. +###### harnessTranscript? + +> `optional` **harnessTranscript?**: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +Whether the child's OWN harness transcript survived its environment, or why it did not. + A SIBLING of `trace`, not a field inside it: `trace` carries the supervisor's tool spans + (toolName, args, status, callId, startedAt === endedAt) and nothing the child said, and a + child with zero tool spans has an UNAVAILABLE trace — so a receipt nested inside the + available arm could never describe exactly the children that need it most (#1244). + ###### budgetViolation? > `optional` **budgetViolation?**: [`BudgetViolation`](#budgetviolation-3) @@ -21013,6 +21077,24 @@ executors omit it (returns `undefined`). [`Spend`](#spend) \| `undefined` +##### harnessTranscript()? + +> `optional` **harnessTranscript**(): [`HarnessTranscriptCapture`](#harnesstranscriptcapture) \| `undefined` + +The child's OWN harness transcript, read out of its environment while that environment was +still live. Read on settle, valid after `execute` resolves OR throws — the throw half is the +point: a child that drops produces no result artifact, so before this the only carrier was +the settled result and a dropped child's reasoning died with its box (#1244). + +An executor with no transcript to offer omits the method entirely; the settle path then +records `executor-exposes-no-transcript` rather than an empty artifact that reads as +coverage. Never throws. Returns the in-memory capture; the SCOPE persists it under its own +content ref and settles the receipt, so no executor ever learns about storage. + +###### Returns + +[`HarnessTranscriptCapture`](#harnesstranscriptcapture) \| `undefined` + *** ### ExecutorCancellationRequest @@ -26349,6 +26431,107 @@ mapper it already uses in the other direction, so a sandbox-shaped provider read *** +### HarnessTranscriptUnavailableReason + +> **HarnessTranscriptUnavailableReason** = `"unsupported-environment"` \| `"unknown-harness"` \| `"no-transcript"` \| `"enumeration-failed"` \| `"execution-never-started"` \| `"capture-did-not-run"` \| `"executor-exposes-no-transcript"` \| `"nothing-carried"` \| `"transcript-persistence-failed"` + +Why no transcript reached a record, from either the capture or the settle path. + +*** + +### HarnessTranscriptCapture + +> **HarnessTranscriptCapture** = \{ `status`: `"captured"`; `artifact`: [`HarnessTranscriptArtifact`](#harnesstranscriptartifact); `fileCount`: `number`; `totalBytes`: `number`; `skippedCount`: `number`; \} \| [`HarnessTranscriptUnavailable`](#harnesstranscriptunavailable) + +What the executor holds in memory between the read and the settle: the files, inline. + +Never journaled and never inside a result blob. The scope persists it under its own content +ref and records the [HarnessTranscriptEvidence](#harnesstranscriptevidence) receipt instead, so the settlement stays +small and a replay pays nothing for a transcript nobody opens. + +#### Union Members + +##### Type Literal + +\{ `status`: `"captured"`; `artifact`: [`HarnessTranscriptArtifact`](#harnesstranscriptartifact); `fileCount`: `number`; `totalBytes`: `number`; `skippedCount`: `number`; \} + +###### status + +> `readonly` **status**: `"captured"` + +###### artifact + +> `readonly` **artifact**: [`HarnessTranscriptArtifact`](#harnesstranscriptartifact) + +###### fileCount + +> `readonly` **fileCount**: `number` + +###### totalBytes + +> `readonly` **totalBytes**: `number` + +###### skippedCount + +> `readonly` **skippedCount**: `number` + +Non-zero when some transcript was found but deliberately not carried. + +*** + +[`HarnessTranscriptUnavailable`](#harnesstranscriptunavailable) + +*** + +### HarnessTranscriptEvidence + +> **HarnessTranscriptEvidence** = \{ `status`: `"available"`; `transcriptRef`: `string`; `harness`: `string`; `fileCount`: `number`; `totalBytes`: `number`; `skippedCount`: `number`; \} \| [`HarnessTranscriptUnavailable`](#harnesstranscriptunavailable) + +The durable receipt on a settlement: a content-addressed pointer to a persisted +[HarnessTranscriptArtifact](#harnesstranscriptartifact), or the exact reason there is none. A SIBLING of the tool-span +`trace` receipt, never nested inside it — a dropped child has zero tool spans and an +unavailable trace, and it is precisely the child whose transcript this exists to keep. + +#### Union Members + +##### Type Literal + +\{ `status`: `"available"`; `transcriptRef`: `string`; `harness`: `string`; `fileCount`: `number`; `totalBytes`: `number`; `skippedCount`: `number`; \} + +###### status + +> `readonly` **status**: `"available"` + +###### transcriptRef + +> `readonly` **transcriptRef**: `string` + +Content-addressed pointer to a persisted `HarnessTranscriptArtifact` in the run's blobs. + +###### harness + +> `readonly` **harness**: `string` + +###### fileCount + +> `readonly` **fileCount**: `number` + +###### totalBytes + +> `readonly` **totalBytes**: `number` + +###### skippedCount + +> `readonly` **skippedCount**: `number` + +Non-zero when some transcript was found but deliberately not carried. + +*** + +[`HarnessTranscriptUnavailable`](#harnesstranscriptunavailable) + +*** + ### HarvestCorpusOptions > **HarvestCorpusOptions** = [`ObserveOptions`](#observeoptions) & `object` @@ -26496,46 +26679,6 @@ runAgentRounds options minus the `ctx` (loopDispatch builds the ctx). *** -### NativeSessionEvidence - -> **NativeSessionEvidence** = \{ `status`: `"available"`; `artifact`: [`NativeSessionArtifact`](#nativesessionartifact); `fileCount`: `number`; `totalBytes`: `number`; `skippedCount`: `number`; \} \| \{ `status`: `"unavailable"`; `reason`: `"unsupported-environment"` \| `"unknown-harness"` \| `"no-transcript"` \| `"enumeration-failed"`; \} - -#### Union Members - -##### Type Literal - -\{ `status`: `"available"`; `artifact`: [`NativeSessionArtifact`](#nativesessionartifact); `fileCount`: `number`; `totalBytes`: `number`; `skippedCount`: `number`; \} - -###### status - -> `readonly` **status**: `"available"` - -###### artifact - -> `readonly` **artifact**: [`NativeSessionArtifact`](#nativesessionartifact) - -###### fileCount - -> `readonly` **fileCount**: `number` - -###### totalBytes - -> `readonly` **totalBytes**: `number` - -###### skippedCount - -> `readonly` **skippedCount**: `number` - -Non-zero when some transcript was found but deliberately not carried. - -*** - -##### Type Literal - -\{ `status`: `"unavailable"`; `reason`: `"unsupported-environment"` \| `"unknown-harness"` \| `"no-transcript"` \| `"enumeration-failed"`; \} - -*** - ### ObservationAnalysis > **ObservationAnalysis** = (`input`, `context`) => `Promise`\<`Pick`\<[`Observation`](#observation), `"findings"` \| `"report"` \| `"usage"`\>\> @@ -27881,7 +28024,7 @@ Why Runtime cannot provide structured tool-call evidence for one settled executi ### WorkerTraceEvidence -> **WorkerTraceEvidence** = \{ `status`: `"available"`; `traceRef`: `string`; `spanCount`: `number`; `nativeSession?`: [`NativeSessionEvidence`](#nativesessionevidence); \} \| \{ `status`: `"unavailable"`; `reason`: [`WorkerTraceUnavailableReason`](#workertraceunavailablereason); \} +> **WorkerTraceEvidence** = \{ `status`: `"available"`; `traceRef`: `string`; `spanCount`: `number`; \} \| \{ `status`: `"unavailable"`; `reason`: [`WorkerTraceUnavailableReason`](#workertraceunavailablereason); \} Durable proof of a worker's structured tool trace, or the exact reason it is unavailable. @@ -27889,7 +28032,7 @@ Durable proof of a worker's structured tool trace, or the exact reason it is una ##### Type Literal -\{ `status`: `"available"`; `traceRef`: `string`; `spanCount`: `number`; `nativeSession?`: [`NativeSessionEvidence`](#nativesessionevidence); \} +\{ `status`: `"available"`; `traceRef`: `string`; `spanCount`: `number`; \} ###### status @@ -27905,18 +28048,6 @@ Content-addressed pointer to a persisted `WorkerToolTraceArtifact`. > `readonly` **spanCount**: `number` -###### nativeSession? - -> `readonly` `optional` **nativeSession?**: [`NativeSessionEvidence`](#nativesessionevidence) - -The child's OWN harness transcript, read out of its environment before destroy. - -`traceRef` above points at the supervisor's tool spans: toolName, args, status, -callId, with startedAt === endedAt. It carries no assistant text, no reasoning and -no tool results, so `status: 'available'` on this object never meant the child's -session survived — it was destroyed with the environment. This says whether it did. -Absent on a settlement recorded before the capture existed. - *** ##### Type Literal @@ -28367,7 +28498,7 @@ recovery before a replacement can run. ### Settled -> **Settled**\<`Out`\> = \{ `kind`: `"done"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `out`: `Out`; `outRef`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} \| \{ `kind`: `"down"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `reason`: `string`; `outRef?`: `string`; `infra`: `boolean`; `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} +> **Settled**\<`Out`\> = \{ `kind`: `"done"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `out`: `Out`; `outRef`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} \| \{ `kind`: `"down"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `reason`: `string`; `outRef?`: `string`; `infra`: `boolean`; `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} A settled child, delivered by `scope.next()`. `seq` is the monotonic cursor order `next()` yielded this settlement (B2) — NOT wall-clock — and replay delivers strictly @@ -28383,7 +28514,7 @@ in `seq` order. `outRef` rehydrates `out` from the `ResultBlobStore` on replay. ##### Type Literal -\{ `kind`: `"done"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `out`: `Out`; `outRef`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} +\{ `kind`: `"done"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `out`: `Out`; `outRef`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} ###### kind @@ -28421,6 +28552,16 @@ Provider model evidence for every inference attempt owned by this node. Structured tool evidence captured before this settlement was journaled. +###### harnessTranscript? + +> `optional` **harnessTranscript?**: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +Whether the child's OWN harness transcript survived its environment, or why it did not. + A SIBLING of `trace`, not a field inside it: `trace` carries the supervisor's tool spans + (toolName, args, status, callId, startedAt === endedAt) and nothing the child said, and a + child with zero tool spans has an UNAVAILABLE trace — so a receipt nested inside the + available arm could never describe exactly the children that need it most (#1244). + ###### budgetViolation? > `optional` **budgetViolation?**: [`BudgetViolation`](#budgetviolation-3) @@ -28441,7 +28582,7 @@ Epoch ms parsed from the durable settlement record when available. ##### Type Literal -\{ `kind`: `"down"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `reason`: `string`; `outRef?`: `string`; `infra`: `boolean`; `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} +\{ `kind`: `"down"`; `handle`: [`Handle`](#handle-3)\<`Out`\>; `reason`: `string`; `outRef?`: `string`; `infra`: `boolean`; `trace`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `settledAt?`: `number`; `seq`: `number`; \} ###### kind @@ -28474,6 +28615,15 @@ True = infrastructure failure (excluded from merge `n` / equal-k), not a bad res Partial structured tool evidence captured before this failure was journaled. +###### harnessTranscript? + +> `optional` **harnessTranscript?**: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +The child's own harness transcript, read out of its environment at the last moment it + was live, or the named reason it could not be. This is the path the capture existed + for and never covered: a dropped child produces no result artifact, so before #1244 + its reasoning was destroyed with its box. + ###### providerModel? > `optional` **providerModel?**: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence) @@ -28500,7 +28650,7 @@ Epoch ms parsed from the durable settlement/cancellation record when available. ### SpawnEvent -> **SpawnEvent** = \{ `kind`: `"spawned"`; `id`: [`NodeId`](#nodeid-6); `parent?`: [`NodeId`](#nodeid-6); `label`: `string`; `key?`: `string`; `assignmentId?`: `string`; `budget`: [`Budget`](#budget-18); `runtime`: [`Runtime`](#runtime-7); `ownedTreeRoot?`: [`NodeId`](#nodeid-6); `identity?`: [`NodeExecutionIdentity`](#nodeexecutionidentity); `profileRef?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-input"`; `id`: [`NodeId`](#nodeid-6); `taskRef`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-admitted"`; `id`: [`NodeId`](#nodeid-6); `admission`: [`RetainedRunAdmission`](#retainedrunadmission); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-result"`; `outcome?`: `Pick`\<`AgentTurnResult`, `"success"` \| `"error"`\>; `id`: [`NodeId`](#nodeid-6); `outRef`: `string`; `spent`: [`Spend`](#spend); `verdict?`: `DefaultVerdict`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-bound"`; `id`: [`NodeId`](#nodeid-6); `binding`: [`ExecutionBindingReceipt`](#executionbindingreceipt); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"materialized"`; `id`: [`NodeId`](#nodeid-6); `receipt`: [`ProfileMaterializationReceipt`](#profilematerializationreceipt); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"settled"`; `id`: [`NodeId`](#nodeid-6); `status`: `"done"` \| `"down"`; `outRef?`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `infra?`: `boolean`; `reason?`: `string`; `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"cancelled"`; `id`: [`NodeId`](#nodeid-6); `reason`: `string`; `source?`: `string`; `infra?`: `boolean`; `spent?`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `outRef?`: `string`; `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"node-inputs-resolved"`; `id`: [`NodeId`](#nodeid-6); `node`: `string`; `instance`: `string`; `inputRef`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"edge-verdict"`; `id`: [`NodeId`](#nodeid-6); `edge`: `string`; `fired`: `boolean`; `sourceStatus`: `"done"` \| `"down"` \| `"invalid"`; `capped?`: `boolean`; `inputRef?`: `string`; `toInstance?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"join-state"`; `id`: [`NodeId`](#nodeid-6); `node`: `string`; `rule`: `"all"` \| `"any"` \| `"any_failed"` \| `"all_done"`; `satisfiedBy`: `ReadonlyArray`\<`string`\>; `consumedPending`: `ReadonlyArray`\<`string`\>; `instance`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"waiting"`; `id`: [`NodeId`](#nodeid-6); `parent?`: [`NodeId`](#nodeid-6); `label`: `string`; `spec`: [`WaitSpec`](#waitspec); `armedAt`: `number`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"woken"`; `id`: [`NodeId`](#nodeid-6); `by`: `"fired"` \| `"timeout"` \| `"cancelled"` \| `"expired"`; `outRef?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"metered"`; `id`: [`NodeId`](#nodeid-6); `spend`: [`Spend`](#spend); `accountingOnly?`: `true`; `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"progress"`; `id`: [`NodeId`](#nodeid-6); `spend`: [`Spend`](#spend); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"reconciled"`; `id`: [`NodeId`](#nodeid-6); `spent`: [`Spend`](#spend); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"teardown-unconfirmed"`; `id`: [`NodeId`](#nodeid-6); `label`: `string`; `runtime`: [`Runtime`](#runtime-7); `status`: [`NodeStatus`](#nodestatus); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"environment-teardown"`; `id`: [`NodeId`](#nodeid-6); `provider`: `string`; `environmentId`: `string`; `destroyed`: `boolean`; `detail?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"edge"`; `id`: [`NodeId`](#nodeid-6); `edge`: \{ `kind`: `"delegates"` \| `"analyzes"` \| `"data"`; `from`: `string`; `to`: `string`; `directive?`: `string`; `port?`: `string`; \}; `traversal`: `number`; `outcome`: `"delivered"` \| `"stripped"` \| `"empty"` \| `"unpropagated"`; `continuity?`: `"fresh"` \| `"resume"` \| `"steer"`; `bytes`: `number`; `reason?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"trace-unpropagated"`; `id`: [`NodeId`](#nodeid-6); `expectedTraceId`: `string`; `backend`: `string`; `reason`: `"no-env-channel"` \| `"no-worker-process"` \| `"caller-omitted"`; `seq`: `number`; `at`: `string`; \} +> **SpawnEvent** = \{ `kind`: `"spawned"`; `id`: [`NodeId`](#nodeid-6); `parent?`: [`NodeId`](#nodeid-6); `label`: `string`; `key?`: `string`; `assignmentId?`: `string`; `budget`: [`Budget`](#budget-18); `runtime`: [`Runtime`](#runtime-7); `ownedTreeRoot?`: [`NodeId`](#nodeid-6); `identity?`: [`NodeExecutionIdentity`](#nodeexecutionidentity); `profileRef?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-input"`; `id`: [`NodeId`](#nodeid-6); `taskRef`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-admitted"`; `id`: [`NodeId`](#nodeid-6); `admission`: [`RetainedRunAdmission`](#retainedrunadmission); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-result"`; `outcome?`: `Pick`\<`AgentTurnResult`, `"success"` \| `"error"`\>; `id`: [`NodeId`](#nodeid-6); `outRef`: `string`; `spent`: [`Spend`](#spend); `verdict?`: `DefaultVerdict`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"execution-bound"`; `id`: [`NodeId`](#nodeid-6); `binding`: [`ExecutionBindingReceipt`](#executionbindingreceipt); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"materialized"`; `id`: [`NodeId`](#nodeid-6); `receipt`: [`ProfileMaterializationReceipt`](#profilematerializationreceipt); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"settled"`; `id`: [`NodeId`](#nodeid-6); `status`: `"done"` \| `"down"`; `outRef?`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `infra?`: `boolean`; `reason?`: `string`; `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"cancelled"`; `id`: [`NodeId`](#nodeid-6); `reason`: `string`; `source?`: `string`; `infra?`: `boolean`; `spent?`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `outRef?`: `string`; `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"node-inputs-resolved"`; `id`: [`NodeId`](#nodeid-6); `node`: `string`; `instance`: `string`; `inputRef`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"edge-verdict"`; `id`: [`NodeId`](#nodeid-6); `edge`: `string`; `fired`: `boolean`; `sourceStatus`: `"done"` \| `"down"` \| `"invalid"`; `capped?`: `boolean`; `inputRef?`: `string`; `toInstance?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"join-state"`; `id`: [`NodeId`](#nodeid-6); `node`: `string`; `rule`: `"all"` \| `"any"` \| `"any_failed"` \| `"all_done"`; `satisfiedBy`: `ReadonlyArray`\<`string`\>; `consumedPending`: `ReadonlyArray`\<`string`\>; `instance`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"waiting"`; `id`: [`NodeId`](#nodeid-6); `parent?`: [`NodeId`](#nodeid-6); `label`: `string`; `spec`: [`WaitSpec`](#waitspec); `armedAt`: `number`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"woken"`; `id`: [`NodeId`](#nodeid-6); `by`: `"fired"` \| `"timeout"` \| `"cancelled"` \| `"expired"`; `outRef?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"metered"`; `id`: [`NodeId`](#nodeid-6); `spend`: [`Spend`](#spend); `accountingOnly?`: `true`; `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"progress"`; `id`: [`NodeId`](#nodeid-6); `spend`: [`Spend`](#spend); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"reconciled"`; `id`: [`NodeId`](#nodeid-6); `spent`: [`Spend`](#spend); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"teardown-unconfirmed"`; `id`: [`NodeId`](#nodeid-6); `label`: `string`; `runtime`: [`Runtime`](#runtime-7); `status`: [`NodeStatus`](#nodestatus); `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"environment-teardown"`; `id`: [`NodeId`](#nodeid-6); `provider`: `string`; `environmentId`: `string`; `destroyed`: `boolean`; `detail?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"edge"`; `id`: [`NodeId`](#nodeid-6); `edge`: \{ `kind`: `"delegates"` \| `"analyzes"` \| `"data"`; `from`: `string`; `to`: `string`; `directive?`: `string`; `port?`: `string`; \}; `traversal`: `number`; `outcome`: `"delivered"` \| `"stripped"` \| `"empty"` \| `"unpropagated"`; `continuity?`: `"fresh"` \| `"resume"` \| `"steer"`; `bytes`: `number`; `reason?`: `string`; `seq`: `number`; `at`: `string`; \} \| \{ `kind`: `"trace-unpropagated"`; `id`: [`NodeId`](#nodeid-6); `expectedTraceId`: `string`; `backend`: `string`; `reason`: `"no-env-channel"` \| `"no-worker-process"` \| `"caller-omitted"`; `seq`: `number`; `at`: `string`; \} Journaled spawn-tree events (B1/B2). `seq` is the cursor order; `at` is an ISO timestamp for human inspection only (NOT a replay input). @@ -28736,7 +28886,7 @@ Trusted runtime transformation from the authorized profile to actual wire bytes. ##### Type Literal -\{ `kind`: `"settled"`; `id`: [`NodeId`](#nodeid-6); `status`: `"done"` \| `"down"`; `outRef?`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `infra?`: `boolean`; `reason?`: `string`; `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} +\{ `kind`: `"settled"`; `id`: [`NodeId`](#nodeid-6); `status`: `"done"` \| `"down"`; `outRef?`: `string`; `verdict?`: `DefaultVerdict`; `spent`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `infra?`: `boolean`; `reason?`: `string`; `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} ###### kind @@ -28787,6 +28937,14 @@ journals written before this field existed remain replayable. Structured tool evidence. Optional only for journals written before trace capture. +###### harnessTranscript? + +> `optional` **harnessTranscript?**: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +Whether this child's harness transcript survived, or the named reason it did not. + Absent on journals written before the capture existed — which is not the same fact as + a recorded `unavailable`, and is why this stays optional rather than defaulting. + ###### budgetViolation? > `optional` **budgetViolation?**: [`BudgetViolation`](#budgetviolation-3) @@ -28805,7 +28963,62 @@ Present when the reconciled spend exceeded the reservation, on either status. ##### Type Literal -\{ `kind`: `"cancelled"`; `id`: [`NodeId`](#nodeid-6); `reason`: `string`; `source?`: `string`; `infra?`: `boolean`; `spent?`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `outRef?`: `string`; `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} +\{ `kind`: `"cancelled"`; `id`: [`NodeId`](#nodeid-6); `reason`: `string`; `source?`: `string`; `infra?`: `boolean`; `spent?`: [`Spend`](#spend); `providerModel?`: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence); `trace?`: [`WorkerTraceEvidence`](#workertraceevidence); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `outRef?`: `string`; `budgetViolation?`: [`BudgetViolation`](#budgetviolation-3); `seq`: `number`; `at`: `string`; \} + +###### kind + +> **kind**: `"cancelled"` + +###### id + +> **id**: [`NodeId`](#nodeid-6) + +###### reason + +> **reason**: `string` + +###### source? + +> `optional` **source?**: `string` + +###### infra? + +> `optional` **infra?**: `boolean` + +###### spent? + +> `optional` **spent?**: [`Spend`](#spend) + +###### providerModel? + +> `optional` **providerModel?**: [`ProviderModelExecutionEvidence`](#providermodelexecutionevidence) + +###### trace? + +> `optional` **trace?**: [`WorkerTraceEvidence`](#workertraceevidence) + +###### harnessTranscript? + +> `optional` **harnessTranscript?**: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +The child's harness transcript receipt, when the executor could still be read at the + cancel. The settle path writes it on this record exactly as on `settled`. + +###### outRef? + +> `optional` **outRef?**: `string` + +###### budgetViolation? + +> `optional` **budgetViolation?**: [`BudgetViolation`](#budgetviolation-3) + +###### seq + +> **seq**: `number` + +###### at + +> **at**: `string` *** @@ -29112,7 +29325,7 @@ without charging the same spend twice. ##### Type Literal -\{ `kind`: `"reconciled"`; `id`: [`NodeId`](#nodeid-6); `spent`: [`Spend`](#spend); `seq`: `number`; `at`: `string`; \} +\{ `kind`: `"reconciled"`; `id`: [`NodeId`](#nodeid-6); `spent`: [`Spend`](#spend); `harnessTranscript?`: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence); `seq`: `number`; `at`: `string`; \} ###### kind @@ -29134,6 +29347,15 @@ A retained child's reservation was reconciled at the child-work floor its execut > **spent**: [`Spend`](#spend) +###### harnessTranscript? + +> `optional` **harnessTranscript?**: [`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +The transcript receipt of a retained-pending child. This record is the ONLY durable home + it has: the node writes no `settled` record while its slot stays open, and these are the + #1244 children exactly — dropped mid-run with a live box the capture read. A later + terminal record for the node carries the same receipt forward. + ###### seq > **seq**: `number` @@ -31075,6 +31297,87 @@ through an experimental symbol. *** +### captureHarnessTranscript() + +> **captureHarnessTranscript**(`environment`, `harness`, `signal?`): `Promise`\<[`HarnessTranscriptCapture`](#harnesstranscriptcapture)\> + +Read the harness transcript out of one LIVE environment. + +Call this before the environment is destroyed — on the settled path that means before the +result is built, since the `finally` that destroys runs after. It never throws: a teardown +must not fail because evidence could not be collected, and every failure mode is a named +`reason` the settled receipt carries instead of an empty artifact that reads as coverage. + +#### Parameters + +##### environment + +[`ReadableEnvironment`](#readableenvironment) \| `undefined` + +##### harness + +`string` \| `undefined` + +##### signal? + +`AbortSignal` + +#### Returns + +`Promise`\<[`HarnessTranscriptCapture`](#harnesstranscriptcapture)\> + +*** + +### persistHarnessTranscript() + +> **persistHarnessTranscript**(`capture`, `blobs`): `Promise`\<[`HarnessTranscriptEvidence`](#harnesstranscriptevidence)\> + +Persist a capture under its own content ref and return the receipt a settlement carries. + +The scope calls this, not the executor: storage stays out of every provider and destroy site, +exactly as the tool-span trace is persisted by `captureWorkerTraceEvidence` and not by the +source that collected it. A capture that is already unavailable passes through untouched. + +#### Parameters + +##### capture + +[`HarnessTranscriptCapture`](#harnesstranscriptcapture) + +##### blobs + +`Pick`\<[`ResultBlobStore`](#resultblobstore), `"put"`\> + +#### Returns + +`Promise`\<[`HarnessTranscriptEvidence`](#harnesstranscriptevidence)\> + +*** + +### harnessTranscriptArtifact() + +> **harnessTranscriptArtifact**(`evidence`, `blobs`): `Promise`\<[`HarnessTranscriptArtifact`](#harnesstranscriptartifact) \| `undefined`\> + +Rehydrate the exact persisted transcript a receipt points at, or `undefined` when the receipt +says there is none. Throws only when the receipt claims a blob the store does not hold — that +is corruption, not absence, and must not read as "no transcript". + +#### Parameters + +##### evidence + +[`HarnessTranscriptEvidence`](#harnesstranscriptevidence) + +##### blobs + +`Pick`\<[`ResultBlobStore`](#resultblobstore), `"get"`\> + +#### Returns + +`Promise`\<[`HarnessTranscriptArtifact`](#harnesstranscriptartifact) \| `undefined`\> + +*** + ### decodeHarnessUsage() > **decodeHarnessUsage**(`event`, `harness?`): [`HarnessUsage`](#harnessusage) \| `undefined` diff --git a/docs/canonical-api.md b/docs/canonical-api.md index 3570a6fc..2ecb99b6 100644 --- a/docs/canonical-api.md +++ b/docs/canonical-api.md @@ -4,7 +4,7 @@ Generated signatures and the complete export list live in docs/api/. Run pnpm docs:freshness after editing this file. --> -> **Version 0.231.1.** +> **Version 0.232.0.** > [`docs/api/primitive-catalog.md`](./api/primitive-catalog.md) lists every export and import path. > `agent-eval` must satisfy `>=0.182.0 <0.183.0`. > `sandbox` must satisfy `>=0.36.4 <0.41.0`. diff --git a/package.json b/package.json index f2ebf77a..a266f53f 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@tangle-network/agent-runtime", - "version": "0.231.1", + "version": "0.232.0", "description": "Shared task-lifecycle skeleton for agents: a recursive loop kernel for chat turns, one-shot tasks, and multi-attempt loops, with trace capture and eval-gated self-improvement. Domain behavior lives in adapters; scoring and ship-gates in @tangle-network/agent-eval.", "homepage": "https://github.com/tangle-network/agent-runtime#readme", "repository": { diff --git a/src/durable/spawn-journal.ts b/src/durable/spawn-journal.ts index f156953c..d15c8442 100644 --- a/src/durable/spawn-journal.ts +++ b/src/durable/spawn-journal.ts @@ -1070,6 +1070,9 @@ export async function replaySpawnTree( : { providerModel: copyProviderModelEvidence(ev.providerModel) }), ...budgetViolationOf(ev), trace, + // A field the journal holds and replay drops is the #1214 bug again: a reader of the + // replayed tree would see no transcript where the record says there is one. + ...(ev.harnessTranscript === undefined ? {} : { harnessTranscript: ev.harnessTranscript }), ...settlementTime(ev.at), seq: ev.seq, }) @@ -1101,6 +1104,7 @@ export async function replaySpawnTree( : { providerModel: copyProviderModelEvidence(ev.providerModel) }), ...budgetViolationOf(ev), trace, + ...(ev.harnessTranscript === undefined ? {} : { harnessTranscript: ev.harnessTranscript }), ...settlementTime(ev.at), seq: ev.seq, }) diff --git a/src/runtime/environment-provider.test.ts b/src/runtime/environment-provider.test.ts index 6411308b..06cb0d1d 100644 --- a/src/runtime/environment-provider.test.ts +++ b/src/runtime/environment-provider.test.ts @@ -19,6 +19,7 @@ import { contentAddress, InMemoryResultBlobStore, InMemorySpawnJournal, + replaySpawnTree, } from '../durable/spawn-journal' import { type AgentEnvironment, @@ -33,6 +34,7 @@ import { providerAsSandboxClient, sandboxClientAsProvider, } from './environment-provider' +import { harnessTranscriptArtifact } from './harness-transcript' import { type ProviderPlacement, selectProviderPlacement } from './provider-placement' import { retainedCreateMaterial } from './retained-run-intent' import { collectAgentTurn, streamAgentTurn } from './stream-agent-turn' @@ -2051,6 +2053,171 @@ describe('environment provider adapters', () => { }) }) + // #1244. The capture PR (#1243) reads the transcript just before the settled result is built, + // so a child whose stream throws never reaches it and its reasoning dies with the box. Measured + // 2026-09-15: 45 children in one evening executed, reasoned, and settled `down` with nothing. + it('keeps a dropped child transcript that has no result artifact to ride in', async () => { + const transcript = '{"role":"assistant","text":"I proved the corner case"}' + const provider: AgentEnvironmentProvider = { + name: 'dropping-provider', + capabilities: () => fakeCapabilities(), + async create() { + return fakeEnvironment({ + exec: async () => ({ + stdout: '/root/.claude/projects/a/session.jsonl', + stderr: '', + exitCode: 0, + }), + read: async () => transcript, + stream: async function* (): AsyncIterable { + yield { type: 'message.part.updated', data: { delta: 'working' } } + // A platform 502 mid-stream: the exact shape that dropped 38 of 68 children on + // capability-per-parameter-cpp-glm-20260915c. + throw new Error('Platform key verification unavailable') + }, + }) as AgentEnvironment + }, + } + const spec: AgentSpec = { + profile: { name: 'worker', harness: 'claude-code' } as AgentProfile, + harness: null, + } + const ctx: ExecutorContext = { signal: new AbortController().signal, seams: {} } + const executor = providerAsExecutor(provider)(spec, ctx) + + await expect( + collect(executor.execute('task', ctx.signal) as AsyncIterable), + ).rejects.toThrow(/Platform key verification unavailable/u) + + // The executor threw and produced NO artifact, yet the reasoning survived. + const evidence = executor.harnessTranscript?.() + expect(evidence?.status).toBe('captured') + if (evidence?.status !== 'captured') return + expect(evidence.artifact.files.map((file) => file.content).join('')).toContain( + 'I proved the corner case', + ) + }) + + // The two absences are different facts and an operator acts differently on each: a child that + // never got a box has nothing to recover, a child that ran for twenty seconds does. + it('separates a child that never started from one whose transcript went unread', async () => { + const provider: AgentEnvironmentProvider = { + name: 'refusing-provider', + capabilities: () => fakeCapabilities(), + async create() { + throw new Error('budget pool refused unknown dollar cost under maxUsd') + }, + } + const spec: AgentSpec = { + profile: { name: 'worker', harness: 'claude-code' } as AgentProfile, + harness: null, + } + const ctx: ExecutorContext = { signal: new AbortController().signal, seams: {} } + const executor = providerAsExecutor(provider)(spec, ctx) + + await expect( + collect(executor.execute('task', ctx.signal) as AsyncIterable), + ).rejects.toThrow(/budget pool refused/u) + + // No environment was ever created (#1240), so this is an absence by construction and must + // never be reported as a transcript that merely went unread. + expect(executor.harnessTranscript?.()).toEqual({ + status: 'unavailable', + reason: 'execution-never-started', + }) + }) + + // End to end, #1244: the drop reaches the journal's settled record as a pointer that resolves. + it('journals a dropped child with a transcript pointer that resolves in the run blobs', async () => { + const provider: AgentEnvironmentProvider = { + name: 'dropping-provider', + capabilities: () => fakeCapabilities(), + async create() { + return fakeEnvironment({ + exec: async () => ({ + stdout: '/root/.claude/projects/a/session.jsonl', + stderr: '', + exitCode: 0, + }), + read: async () => '{"role":"assistant","text":"the corner certificate holds"}', + stream: async function* (): AsyncIterable { + yield { type: 'message.part.updated', data: { delta: 'working' } } + throw new Error('sidecar start failed') + }, + }) as AgentEnvironment + }, + } + const journal = new InMemorySpawnJournal() + await journal.beginTree('root', new Date(0).toISOString()) + const blobs = new InMemoryResultBlobStore() + const scope = createScope({ + parentId: 'root', + root: 'root', + journal, + blobs, + pool: createBudgetPool({ maxIterations: 2, maxTokens: 1_000 }, 0), + executors: createExecutorRegistry(), + seams: {}, + depth: 0, + signal: new AbortController().signal, + }) + const profile: AgentProfile = { + name: 'dropper', + harness: 'claude-code', + model: { provider: 'fixture', default: 'fixture/model' }, + } + expect( + scope.spawn( + Object.assign( + { name: 'dropper', act: async () => 'unused' }, + { + executorSpec: { + profile, + harness: null, + executorFactory: createExecutor({ backend: 'provider', provider }), + }, + }, + ), + 'task', + { label: 'dropper', budget: { maxIterations: 1, maxTokens: 1_000 } }, + ).ok, + ).toBe(true) + + const settled = await scope.next() + expect(settled).not.toBeNull() + if (settled === null) return + expect(settled.kind).toBe('down') + if (settled.kind !== 'down') return + // No result artifact, no tool spans — and still the receipt is there and resolves. + expect(settled.harnessTranscript?.status).toBe('available') + if (settled.harnessTranscript?.status !== 'available') return + const artifact = await harnessTranscriptArtifact(settled.harnessTranscript, blobs) + expect(artifact?.files.map((file) => file.content).join('')).toContain( + 'the corner certificate holds', + ) + // The durable record carries the same pointer, so replay and Lab read it without the process. + const record = (await journal.loadTree('root'))?.find( + (event) => event.kind === 'settled' && event.id === settled.handle.id, + ) + expect(record).toMatchObject({ + status: 'down', + harnessTranscript: { + status: 'available', + transcriptRef: settled.harnessTranscript.transcriptRef, + }, + }) + // And replay hands the receipt back: a field the journal holds and replay drops would be the + // #1214 bug one level down. + const replayed = await replaySpawnTree(journal, blobs, 'root') + expect(replayed.find((entry) => entry.handle.id === settled.handle.id)).toMatchObject({ + kind: 'down', + harnessTranscript: { + status: 'available', + transcriptRef: settled.harnessTranscript.transcriptRef, + }, + }) + }) + it('preserves a canonical provider billing receipt through provider execution', async () => { const provider: AgentEnvironmentProvider = { name: 'billed-provider', diff --git a/src/runtime/environment-provider.ts b/src/runtime/environment-provider.ts index e0dfb8d1..17966f30 100644 --- a/src/runtime/environment-provider.ts +++ b/src/runtime/environment-provider.ts @@ -1,5 +1,9 @@ import { randomUUID } from 'node:crypto' -import { captureNativeSessionEvidence, type NativeSessionEvidence } from './native-session-evidence' +import { + captureHarnessTranscript, + type HarnessTranscriptCapture, + harnessTranscriptUnavailable, +} from './harness-transcript' import { type ProviderPlacement, selectProviderPlacement } from './provider-placement' export type { ProviderPlacement } from './provider-placement' @@ -474,15 +478,6 @@ export interface ProviderLeafOut { events: AgentEnvironmentEvent[] /** How many streamed part updates the archive left out because a later frame superseded them. */ supersededPartUpdates?: number - /** - * The child's own harness transcript, read before the environment was destroyed. - * - * `events` above is the provider's stream and `trace` on the settlement is the supervisor's - * tool spans; neither carries the harness's session files, which used to die with the - * environment. Always present on the settled path: an environment that cannot be read says - * so with a `reason` rather than being silently absent. #1214. - */ - nativeSession?: NativeSessionEvidence } /** @@ -651,6 +646,19 @@ function createProviderExecutor( let artifact: ExecutorResult | undefined let retained: RetainedRunHandle | undefined let pending = false + // The child's own harness transcript, read out of the environment while it was still live. + // + // It is held HERE, on the executor, rather than only inside the settled result, because the + // children whose reasoning an operator most wants are the ones that never produce a result: + // a stream that throws lands in the `catch` below with no artifact to ride in (#1244). The + // scope pulls this after `execute` resolves OR throws, exactly as it pulls `metered()`. + // + // Seeded, never left undefined, so the two absences stay apart: a child killed at admission + // never had a box and keeps this seed, while a child whose box was created and then dropped + // reports `capture-did-not-run`. Reporting the second as the first would file a child that + // reasoned for twenty seconds as one that never ran — which is the #1240 population exactly. + let harnessTranscript: HarnessTranscriptCapture = + harnessTranscriptUnavailable('execution-never-started') const retention = retainedExecutorContext(ctx) // The stream destroys the environment on settle by default, so a later `teardown` would issue a // SECOND delete against a resource that is already gone. That second call is what the provider @@ -745,6 +753,14 @@ function createProviderExecutor( }, onEnvironment: (env) => { environment = env + // A box now exists, so `execution-never-started` has stopped being true. Until the + // capture reports, the honest answer is that nobody read it. + if ( + harnessTranscript.status === 'unavailable' && + harnessTranscript.reason === 'execution-never-started' + ) { + harnessTranscript = harnessTranscriptUnavailable('capture-did-not-run') + } // `create` resolved, so the environment identity the provider issued is now evidence. It // goes in `execution`, which the mid-run guard treats as per-attempt routing, and NOT in // `plan`, which the guard holds fixed across attempts. It used to be written to both, so a @@ -765,6 +781,9 @@ function createProviderExecutor( onArtifact: (next) => { artifact = next }, + onHarnessTranscript: (next) => { + harnessTranscript = next + }, onDestroyed: () => { destroyed = true }, @@ -892,6 +911,7 @@ function createProviderExecutor( return artifact }, traceSource: (): TraceSource => trace.source, + harnessTranscript: (): HarnessTranscriptCapture | undefined => harnessTranscript, } return attestRuntimeOwnedPendingExecutor(executor, runtime, plannedDeclaration, plannedBinding) } @@ -923,6 +943,9 @@ interface StreamProviderExecutorArgs { onPending: (pending: boolean) => void onEnvironment: (environment: AgentEnvironment) => void onArtifact: (artifact: ExecutorResult) => void + /** The harness transcript read out of the live environment, reported on the settled path AND + * on the drop path. One channel for both, so a reader never has to know which path ran. */ + onHarnessTranscript: (capture: HarnessTranscriptCapture) => void /** The environment was destroyed here, so `teardown` must not DELETE it a second time — the * double delete is what produced the 409 that used to fail a completed run. */ onDestroyed: () => void @@ -1061,16 +1084,19 @@ async function* streamProviderExecutor( // // It rides inside the settled result, so supervise blobs it under this child's outRef in // its own ResultBlobStore and replay rehydrates it. No destroy site learns about storage. - const nativeSession = await captureNativeSessionEvidence( - environment as Parameters[0], + const harnessTranscript = await captureHarnessTranscript( + environment as Parameters[0], args.profile.harness, // The run's linked abort, so a cancelled run stops mid-enumeration instead of reading // up to MAX_FILES out of an environment that is already being torn down. linked, ) + // Reported to the executor, NOT spliced into the result. The scope persists it under its + // own content ref and settles a receipt, so the result blob every replay rehydrates never + // carries the files, and the one channel serves the settled path and the drop path alike. + args.onHarnessTranscript(harnessTranscript) const result: ProviderLeafOut & SandboxOutcomeCarrier = { ...resultFromEvents(archive.events(), text), - nativeSession, ...(archive.superseded > 0 ? { supersededPartUpdates: archive.superseded } : {}), ...(explicitFailure ? { outcome: outcomeTracker.finish() } : {}), } @@ -1118,6 +1144,23 @@ async function* streamProviderExecutor( } catch (error) { failure = source.retained ? new RetainedExecutionPendingError(error) : error failed = true + // THE LAST POINT THE ENVIRONMENT IS STILL LIVE. The `finally` below destroys it for a + // non-retained source, and a retained one is released later by `releaseRetained` — either + // way nothing downstream can read it again. Measured 2026-09-15 on the + // capability-per-parameter pursuits: 45 children in one evening executed, reasoned, and + // settled `down` with no artifact for a transcript to ride in (#1244). These are the + // children an operator most wants to read, because they are the ones that failed. + // + // `linked` is already aborted when the drop was a cancellation; the capture then names + // every remaining path `aborted` rather than making doomed reads into a dying box. + // It never throws, so this cannot convert a stream failure into a teardown failure. + args.onHarnessTranscript( + await captureHarnessTranscript( + environment as Parameters[0], + args.profile.harness, + linked, + ), + ) } finally { if ( (!source.retained || (settled !== undefined && !failed)) && diff --git a/src/runtime/harness-transcript.test.ts b/src/runtime/harness-transcript.test.ts new file mode 100644 index 00000000..57a29170 --- /dev/null +++ b/src/runtime/harness-transcript.test.ts @@ -0,0 +1,311 @@ +import { describe, expect, it } from 'vitest' + +import { contentAddress } from '../durable/content-address' +import { InMemoryResultBlobStore } from '../durable/spawn-journal' +import { + captureHarnessTranscript, + harnessTranscriptArtifact, + persistHarnessTranscript, +} from './harness-transcript' + +function environment(files: Record, opts: { read?: boolean; exec?: boolean } = {}) { + const listing = Object.keys(files).join('\n') + return { + ...(opts.exec === false ? {} : { exec: async () => ({ stdout: listing, exitCode: 0 }) }), + ...(opts.read === false + ? {} + : { + read: async (path: string) => { + const content = files[path] + if (content === undefined) throw new Error(`no such file: ${path}`) + return content + }, + }), + } +} + +describe('captureHarnessTranscript', () => { + it('carries the assistant text the tool-span receipt never had', async () => { + const evidence = await captureHarnessTranscript( + environment({ + '/root/.claude/projects/a/session.jsonl': '{"role":"assistant","text":"the answer"}', + '/root/.claude/history.jsonl': '{"prompt":"the question"}', + }), + 'claude-code', + ) + expect(evidence.status).toBe('captured') + if (evidence.status !== 'captured') return + expect(evidence.fileCount).toBe(2) + expect(evidence.skippedCount).toBe(0) + // Inline in the artifact the executor settles with; supervise blobs it under the outRef. + expect(evidence.artifact.files.map((f) => f.content).join('')).toMatch(/the answer/u) + }) + + it('never reads a credential that sits inside a session tree', async () => { + const reads: string[] = [] + const evidence = await captureHarnessTranscript( + { + exec: async () => ({ + stdout: ['/root/.codex/sessions/rollout.jsonl', '/root/.codex/sessions/auth.json'].join( + '\n', + ), + exitCode: 0, + }), + read: async (path: string) => { + reads.push(path) + return '{"ok":true}' + }, + }, + 'codex', + ) + expect(evidence.status).toBe('captured') + if (evidence.status !== 'captured') return + expect(evidence.fileCount).toBe(1) + expect(evidence.skippedCount).toBe(1) + // The assertion that matters: the read never happened, not merely that it is absent. + expect(reads).toEqual(['/root/.codex/sessions/rollout.jsonl']) + }) + + it('reports a missing capability instead of an empty artifact that reads as coverage', async () => { + // agent-provider-tangle gates read behind capabilities.workspace.read && box.read. + expect(await captureHarnessTranscript(environment({}, { read: false }), 'codex')).toEqual({ + status: 'unavailable', + reason: 'unsupported-environment', + }) + // read takes one path and offers no listing, so enumeration needs exec. + expect(await captureHarnessTranscript(environment({}, { exec: false }), 'codex')).toEqual({ + status: 'unavailable', + reason: 'enumeration-failed', + }) + expect(await captureHarnessTranscript(environment({}), 'no-such-harness')).toEqual({ + status: 'unavailable', + reason: 'unknown-harness', + }) + expect(await captureHarnessTranscript(environment({}), 'codex')).toEqual({ + status: 'unavailable', + reason: 'no-transcript', + }) + }) + + it('produces an identical artifact for an identical transcript', async () => { + const files = { '/root/.codex/sessions/r.jsonl': '{"a":1}' } + const first = await captureHarnessTranscript(environment(files), 'codex') + const second = await captureHarnessTranscript(environment(files), 'codex') + expect(first.status).toBe('captured') + expect(second.status).toBe('captured') + if (first.status !== 'captured' || second.status !== 'captured') return + // Keeps supervise's contentRef over the settled result stable across a re-capture. + expect(first.artifact).toEqual(second.artifact) + }) + + it('stops reading when the run is aborted rather than draining a dying environment', async () => { + // Without the run's linked signal a cancelled run still enumerated and read up to + // MAX_FILES out of an environment the `finally` was already tearing down. + const controller = new AbortController() + const reads: string[] = [] + const evidence = await captureHarnessTranscript( + { + exec: async () => ({ + stdout: Array.from({ length: 50 }, (_, i) => `/root/.codex/sessions/r${i}.jsonl`).join( + '\n', + ), + exitCode: 0, + }), + read: async (path: string, options?: { signal?: AbortSignal }) => { + options?.signal?.throwIfAborted() + reads.push(path) + if (reads.length === 3) controller.abort() + return '{}' + }, + }, + 'codex', + controller.signal, + ) + expect(reads.length).toBe(3) + expect(evidence.status).toBe('captured') + if (evidence.status !== 'captured') return + // The reads that never happened are named, not silently missing. + expect(evidence.skippedCount).toBe(47) + }) + + it('refuses every credential shape, not just the lowercase ones', async () => { + const reads: string[] = [] + const denied = [ + '/root/.codex/sessions/.netrc', + '/root/.codex/sessions/ID_RSA', + '/root/.codex/sessions/server.key', + '/root/.codex/sessions/bundle.p12', + '/root/.codex/sessions/AUTH.JSON', + ] + await captureHarnessTranscript( + { + exec: async () => ({ + stdout: ['/root/.codex/sessions/r.jsonl', ...denied].join('\n'), + exitCode: 0, + }), + read: async (path: string) => { + reads.push(path) + return '{}' + }, + }, + 'codex', + ) + expect(reads).toEqual(['/root/.codex/sessions/r.jsonl']) + }) +}) + +describe('persistHarnessTranscript', () => { + it('settles a pointer and the files come back only when someone opens them', async () => { + const blobs = new InMemoryResultBlobStore() + const capture = await captureHarnessTranscript( + environment({ '/root/.codex/sessions/s.jsonl': '{"text":"the reasoning"}' }), + 'codex', + ) + const evidence = await persistHarnessTranscript(capture, blobs) + expect(evidence.status).toBe('available') + if (evidence.status !== 'available') return + // The receipt is small and names what it points at; the bytes are not on it (#1248). + expect(evidence).toMatchObject({ harness: 'codex', fileCount: 1, skippedCount: 0 }) + expect(JSON.stringify(evidence)).not.toContain('the reasoning') + const artifact = await harnessTranscriptArtifact(evidence, blobs) + expect(artifact?.files.map((file) => file.content).join('')).toContain('the reasoning') + }) + + it('names a capture that never reached disk instead of settling a dangling pointer', async () => { + const capture = await captureHarnessTranscript( + environment({ '/root/.codex/sessions/s.jsonl': '{}' }), + 'codex', + ) + const evidence = await persistHarnessTranscript(capture, { + put: async () => { + throw new Error('disk full') + }, + }) + expect(evidence).toEqual({ status: 'unavailable', reason: 'transcript-persistence-failed' }) + }) + + it('passes an unavailable capture through untouched', async () => { + const evidence = await persistHarnessTranscript( + { status: 'unavailable', reason: 'no-transcript' }, + new InMemoryResultBlobStore(), + ) + expect(evidence).toEqual({ status: 'unavailable', reason: 'no-transcript' }) + expect(await harnessTranscriptArtifact(evidence, new InMemoryResultBlobStore())).toBeUndefined() + }) +}) + +describe('captureHarnessTranscript bounds and absences', () => { + it('names files it found and did not carry instead of reporting no transcript', async () => { + const controller = new AbortController() + controller.abort() + const capture = await captureHarnessTranscript( + environment({ '/root/.codex/sessions/s.jsonl': '{"text":"was here"}' }), + 'codex', + controller.signal, + ) + // An abort before the reads is not "no transcript": the file existed and nobody read it. + expect(capture.status).toBe('unavailable') + if (capture.status !== 'unavailable') return + expect(capture.reason).toBe('nothing-carried') + expect(capture.skipped).toEqual([{ path: '/root/.codex/sessions/s.jsonl', reason: 'aborted' }]) + }) + + it('settles at or under 16 MiB, never 16 MiB plus one more file', async () => { + const twoMiB = 'x'.repeat(2 * 1024 * 1024) + const files: Record = {} + for (let i = 0; i < 9; i++) files[`/root/.codex/sessions/${i}.jsonl`] = twoMiB + const capture = await captureHarnessTranscript(environment(files), 'codex') + expect(capture.status).toBe('captured') + if (capture.status !== 'captured') return + expect(capture.totalBytes).toBe(16 * 1024 * 1024) + expect(capture.fileCount).toBe(8) + expect(capture.artifact.skipped).toEqual([ + { path: '/root/.codex/sessions/8.jsonl', reason: 'total-byte-budget-exhausted' }, + ]) + }) +}) + +describe('enumeration omissions are reported, never folded into no-transcript', () => { + // A box whose `find -printf` answers `sizepath` per file, as a real one does. + const boxWith = (listing: [number, string][], reads: string[]) => ({ + exec: async () => ({ + stdout: listing.map(([size, path]) => `${size}\t${path}`).join('\n'), + exitCode: 0, + }), + read: async (path: string) => { + reads.push(path) + return '{}' + }, + }) + + it('lists an oversized file as skipped without ever reading it', async () => { + const reads: string[] = [] + const capture = await captureHarnessTranscript( + boxWith( + [ + [2, '/root/.codex/sessions/small.jsonl'], + [3 * 1024 * 1024, '/root/.codex/sessions/huge.jsonl'], + ], + reads, + ), + 'codex', + ) + expect(capture.status).toBe('captured') + if (capture.status !== 'captured') return + expect(reads).toEqual(['/root/.codex/sessions/small.jsonl']) + expect(capture.skippedCount).toBe(1) + expect(capture.artifact.skipped).toEqual([ + { path: '/root/.codex/sessions/huge.jsonl', reason: 'file-exceeds-byte-bound' }, + ]) + }) + + it('names a listing that overflowed the file bound instead of reading as complete', async () => { + const reads: string[] = [] + const all = Array.from( + { length: 1001 }, + (_, i) => [2, `/root/.codex/sessions/${i}.jsonl`] as [number, string], + ) + const capture = await captureHarnessTranscript(boxWith(all, reads), 'codex') + expect(capture.status).toBe('captured') + if (capture.status !== 'captured') return + expect(capture.fileCount).toBe(1000) + expect(capture.artifact.skipped).toEqual([ + { path: '.codex/sessions .codex/history.jsonl', reason: 'enumeration-truncated-at-1000' }, + ]) + }) + + it('reports a transcript made only of files it could not carry', async () => { + const capture = await captureHarnessTranscript( + boxWith([[3 * 1024 * 1024, '/root/.codex/sessions/huge.jsonl']], []), + 'codex', + ) + expect(capture).toEqual({ + status: 'unavailable', + reason: 'nothing-carried', + skipped: [{ path: '/root/.codex/sessions/huge.jsonl', reason: 'file-exceeds-byte-bound' }], + }) + }) +}) + +describe('harnessTranscriptArtifact refuses a malformed blob', () => { + it('throws on an entry that is not a file rather than returning it as a transcript', async () => { + const blobs = new InMemoryResultBlobStore() + const bad = { schemaVersion: 1, harness: 'codex', files: [null], skipped: [] } + // Content-addressed like every real blob: the ref is right, the bytes are not a transcript. + const ref = contentAddress(bad) + await blobs.put(ref, bad) + await expect( + harnessTranscriptArtifact( + { + status: 'available', + transcriptRef: ref, + harness: 'codex', + fileCount: 1, + totalBytes: 0, + skippedCount: 0, + }, + blobs, + ), + ).rejects.toThrow(/no transcript artifact/u) + }) +}) diff --git a/src/runtime/harness-transcript.ts b/src/runtime/harness-transcript.ts new file mode 100644 index 00000000..6359e37c --- /dev/null +++ b/src/runtime/harness-transcript.ts @@ -0,0 +1,403 @@ +/** + * Read a sandbox child's own harness transcript out of its environment before the + * environment is destroyed, and persist it beside the tool-span `trace` receipt. + * + * Why this exists: Runtime destroys a child's environment at teardown and nothing read + * those files first, so a child's Claude Code / Codex / OpenCode session never reached a + * run record. The settled `trace` receipt covers the supervisor's own tool spans only — + * `toolName`, `args`, `status`, `callId`, with `startedAt === endedAt` — so it carries no + * assistant text, no reasoning, and no tool results, and `status: 'available'` never meant + * the transcript survived. See tangle-network/agent-runtime#1214. + * + * The port already has what this needs: `AgentEnvironment.read?(path, { signal })` + * returns a file's content as a string. Two constraints shape the design: + * + * - `read` is OPTIONAL and, on the Tangle provider, gated behind + * `capabilities.workspace.read && box.read`. An environment without it is reported as + * `unsupported` rather than silently producing an empty artifact — the whole point of + * #1214 is that a receipt must not claim coverage it does not have. + * - `read` takes one path and gives no directory listing, so enumeration goes through + * `exec`. An environment with `read` but no `exec` cannot be enumerated and is also + * reported, not guessed at. + * + * Storage is NOT the executor's business. The executor holds the in-memory capture and hands it + * to the scope through `Executor.harnessTranscript()`; the scope persists it under its own + * content ref with `persistHarnessTranscript` and settles the receipt. No provider or destroy + * site learns about storage, the result blob never carries the files, and replay rehydrates a + * pointer — the bytes are read only by `harnessTranscriptArtifact`, when someone opens them. + * + * Credentials are excluded by construction: enumeration lists only the transcript globs + * for the harness, and any path matching DENY is dropped even if a producer moved a + * credential file inside a session directory. The helper never reads `auth.json`, + * `.credentials.json`, `credentials`, or a dotenv file. + */ +import { contentAddress } from '../durable/content-address' +import { ValidationError } from '../errors' +import type { ResultBlobStore } from './supervise/types' + +/** Where each harness keeps the session files that hold the conversation. */ +const HARNESS_ROOTS: Readonly> = Object.freeze({ + 'claude-code': Object.freeze(['.claude/projects', '.claude/history.jsonl', '.claude/todos']), + codex: Object.freeze(['.codex/sessions', '.codex/history.jsonl']), + opencode: Object.freeze(['.local/share/opencode/storage', '.local/share/opencode/log']), + // Pi keeps a session tree per cwd; it had no entry, so a pi child captured nothing. + pi: Object.freeze(['.pi/agent/sessions']), +}) + +/** + * Never read, whatever a producer put inside a session tree. + * + * Case-insensitive and broader than the obvious names: `.netrc`, `*.key`, `*.p12`/`*.pfx` + * and an uppercase `ID_RSA` all slipped an earlier lowercase-only pattern. + */ +const DENY = + /(^|\/)(auth\.json|\.credentials\.json|credentials|\.netrc|\.env(\..*)?|secrets?(\.|$)|.*\.(pem|key|p12|pfx)|id_[a-z0-9_]+)$/iu + +/** + * Bounds per file and in total, PER CHILD. They multiply by fleet width, and the next person + * sizing a fleet must find that here: a 132-child pursuit (mech-interp-foundations-astra-20260911f, + * the widest retained on one operator host) at the ceiling is 132 x 16 MiB = 2.1 GiB of transcript + * for one run (#1248). + * + * That figure is a DISK number, not a replay number. The artifact is persisted under its own + * content ref in the `ResultBlobStore` and the settlement carries only the receipt, so replay and + * resume rehydrate a ref, never the files; nothing pays for a transcript until someone opens it. + * The bound is enforced exactly, after each read, so a child settles at or under MAX_TOTAL_BYTES. + */ +const MAX_FILE_BYTES = 2 * 1024 * 1024 +const MAX_TOTAL_BYTES = 16 * 1024 * 1024 +const MAX_FILES = 1_000 + +export const HARNESS_TRANSCRIPT_SCHEMA_VERSION = 1 as const + +export interface HarnessTranscriptFile { + readonly path: string + readonly bytes: number + readonly content: string +} + +export interface HarnessTranscriptArtifact { + readonly schemaVersion: typeof HARNESS_TRANSCRIPT_SCHEMA_VERSION + readonly harness: string + readonly files: readonly HarnessTranscriptFile[] + /** Paths found but not read, with why — a gap named is a gap an operator can act on. */ + readonly skipped: readonly { readonly path: string; readonly reason: string }[] +} + +/** Why no transcript reached a record, from either the capture or the settle path. */ +export type HarnessTranscriptUnavailableReason = + | 'unsupported-environment' + | 'unknown-harness' + | 'no-transcript' + | 'enumeration-failed' + /** No environment was ever created for this child, so there is no transcript and never + * was one. The admission refusal of #1240 is the measured case: a budget pool that + * refuses an unknown dollar cost kills the child before it runs. Distinct from + * `unsupported-environment`, which means a box existed and could not be read. */ + | 'execution-never-started' + /** An environment WAS created and the child ran, but the capture never executed — the + * deadline/abort path closes the stream with `iterator.return()`, which runs the + * generator's `finally` and skips its `catch`. Says only what is known: this child had a + * transcript and nobody read it. Never collapse it into `execution-never-started`; that + * would report a child that reasoned for twenty seconds as one that never ran. */ + | 'capture-did-not-run' + /** This executor implements no transcript port at all: a CLI, in-process, bridge, or + * sandbox-session executor that has no capture yet. An absence by construction, never a + * failure — and never a claim that the harness wrote nothing. */ + | 'executor-exposes-no-transcript' + /** Session files were found and none was carried: every path was refused, over budget, + * unreadable, or skipped by an abort. The `skipped` list beside this reason names each one, + * so "there was a transcript and it was not kept" never reads as "no transcript". */ + | 'nothing-carried' + /** The capture succeeded and the blob write did not. The transcript existed in memory and + * never reached disk; mirrors `trace-persistence-failed` on the tool-span receipt. */ + | 'transcript-persistence-failed' + +export interface HarnessTranscriptUnavailable { + readonly status: 'unavailable' + readonly reason: HarnessTranscriptUnavailableReason + /** Present with `nothing-carried`: the paths that existed and why each was not read. */ + readonly skipped?: readonly { readonly path: string; readonly reason: string }[] +} + +/** + * What the executor holds in memory between the read and the settle: the files, inline. + * + * Never journaled and never inside a result blob. The scope persists it under its own content + * ref and records the {@link HarnessTranscriptEvidence} receipt instead, so the settlement stays + * small and a replay pays nothing for a transcript nobody opens. + */ +export type HarnessTranscriptCapture = + | { + readonly status: 'captured' + readonly artifact: HarnessTranscriptArtifact + readonly fileCount: number + readonly totalBytes: number + /** Non-zero when some transcript was found but deliberately not carried. */ + readonly skippedCount: number + } + | HarnessTranscriptUnavailable + +/** + * The durable receipt on a settlement: a content-addressed pointer to a persisted + * {@link HarnessTranscriptArtifact}, or the exact reason there is none. A SIBLING of the tool-span + * `trace` receipt, never nested inside it — a dropped child has zero tool spans and an + * unavailable trace, and it is precisely the child whose transcript this exists to keep. + */ +export type HarnessTranscriptEvidence = + | { + readonly status: 'available' + /** Content-addressed pointer to a persisted `HarnessTranscriptArtifact` in the run's blobs. */ + readonly transcriptRef: string + readonly harness: string + readonly fileCount: number + readonly totalBytes: number + /** Non-zero when some transcript was found but deliberately not carried. */ + readonly skippedCount: number + } + | HarnessTranscriptUnavailable + +/** The two optional environment reads the capture needs. Public because `captureHarnessTranscript` + * is, so a BYO executor can satisfy it with any box that offers a bounded `read` and an `exec`. */ +export interface ReadableEnvironment { + readonly read?: (path: string, options?: { readonly signal?: AbortSignal }) => Promise + readonly exec?: ( + command: string, + options?: Record, + ) => Promise<{ readonly stdout?: string; readonly exitCode?: number }> +} + +function unavailable(reason: HarnessTranscriptUnavailableReason): HarnessTranscriptUnavailable { + return Object.freeze({ status: 'unavailable', reason }) +} + +/** The one place an absent transcript is spelled, so every settlement reads the same — the + * settle path needs it for the reasons only IT can know (an executor that offers no transcript, + * a child that never started), which the capture itself never sees. */ +export function harnessTranscriptUnavailable( + reason: HarnessTranscriptUnavailableReason, +): HarnessTranscriptUnavailable { + return unavailable(reason) +} + +interface Enumeration { + /** Paths to read, at most MAX_FILES of them. */ + readonly paths: readonly string[] + /** What the enumeration itself left out, named — never folded silently into "no transcript". */ + readonly omitted: readonly { readonly path: string; readonly reason: string }[] +} + +/** + * Enumerate candidate transcript paths inside the environment. + * + * `find` is given the roots and prints one `sizepath` line per file. A missing root is not + * an error: a child that never used a harness has no directory for it. A line with no size (an + * environment whose `find` lacks `-printf`) is read and bounded after the read, as before. + * + * Two omissions used to be silent, so `skippedCount` could read 0 on an incomplete capture: a + * `-size` filter dropped oversized files before they were ever listed, and `head` cut the listing + * at MAX_FILES with nothing saying so. Both are reported now: an oversized file is skipped by its + * listed size WITHOUT being read, and a listing that overflows carries one + * `enumeration-truncated` entry naming the bound. + */ +async function enumerate( + environment: ReadableEnvironment, + roots: readonly string[], + signal?: AbortSignal, +): Promise { + if (!environment.exec) return undefined + const quoted = roots.map((r) => `"$HOME/${r}"`).join(' ') + // One past the bound, so an exact overflow is observable rather than indistinguishable from + // a listing that happened to be full. + const command = `find ${quoted} -type f -printf '%s\\t%p\\n' 2>/dev/null | head -${MAX_FILES + 1}` + try { + const result = await environment.exec(command, signal ? { signal } : undefined) + const lines = (result.stdout ?? '') + .split('\n') + .map((l) => l.trim()) + .filter((l) => l.length > 0) + const omitted: { path: string; reason: string }[] = [] + const paths: string[] = [] + for (const line of lines.slice(0, MAX_FILES)) { + const tab = line.indexOf('\t') + const path = tab === -1 ? line : line.slice(tab + 1) + const size = tab === -1 ? undefined : Number(line.slice(0, tab)) + if (size !== undefined && Number.isFinite(size) && size > MAX_FILE_BYTES) { + omitted.push({ path, reason: 'file-exceeds-byte-bound' }) + continue + } + paths.push(path) + } + if (lines.length > MAX_FILES) { + omitted.push({ path: roots.join(' '), reason: `enumeration-truncated-at-${MAX_FILES}` }) + } + return { paths, omitted } + } catch { + return undefined + } +} + +/** + * Read the harness transcript out of one LIVE environment. + * + * Call this before the environment is destroyed — on the settled path that means before the + * result is built, since the `finally` that destroys runs after. It never throws: a teardown + * must not fail because evidence could not be collected, and every failure mode is a named + * `reason` the settled receipt carries instead of an empty artifact that reads as coverage. + */ +export async function captureHarnessTranscript( + environment: ReadableEnvironment | undefined, + harness: string | undefined, + signal?: AbortSignal, +): Promise { + if (!environment?.read) return unavailable('unsupported-environment') + // Narrow `harness` before use: the roots lookup alone does not, and an artifact must + // name the harness it came from. + if (harness === undefined) return unavailable('unknown-harness') + const roots = HARNESS_ROOTS[harness] + if (!roots) return unavailable('unknown-harness') + + const enumerated = await enumerate(environment, roots, signal) + if (enumerated === undefined) return unavailable('enumeration-failed') + const { paths } = enumerated + if (paths.length === 0 && enumerated.omitted.length === 0) return unavailable('no-transcript') + + const files: HarnessTranscriptFile[] = [] + // Seeded with what the enumeration itself left out, so an incomplete listing is never a + // receipt with skippedCount 0. + const skipped: { path: string; reason: string }[] = [...enumerated.omitted] + let total = 0 + for (const path of paths) { + // Stop on abort rather than attempting every remaining read and failing each: a + // cancelled run should leave the dying environment alone, not make MAX_FILES doomed + // calls into it. The remainder is still named in `skipped`. + if (signal?.aborted) { + skipped.push({ path, reason: 'aborted' }) + continue + } + if (DENY.test(path)) { + skipped.push({ path, reason: 'denied-credential-shaped-path' }) + continue + } + if (total >= MAX_TOTAL_BYTES) { + skipped.push({ path, reason: 'total-byte-budget-exhausted' }) + continue + } + try { + const content = await environment.read(path, signal ? { signal } : undefined) + const bytes = Buffer.byteLength(content, 'utf8') + if (bytes > MAX_FILE_BYTES) { + skipped.push({ path, reason: 'file-exceeds-byte-bound' }) + continue + } + // Checked AFTER the read so the ceiling is exact. A pre-read check let one child land at + // MAX_TOTAL_BYTES plus one more file, about 18 MB against a stated 16 (#1248). + if (total + bytes > MAX_TOTAL_BYTES) { + skipped.push({ path, reason: 'total-byte-budget-exhausted' }) + continue + } + files.push(Object.freeze({ path, bytes, content })) + total += bytes + } catch { + skipped.push({ path, reason: 'read-failed' }) + } + } + // Nothing read is two different facts: no session files at all (reported above), or files that + // every rule declined — including an abort that skipped them all. The second must name them. + if (files.length === 0) { + return Object.freeze({ + status: 'unavailable', + reason: 'nothing-carried', + skipped: Object.freeze(skipped), + }) + } + + const artifact: HarnessTranscriptArtifact = Object.freeze({ + schemaVersion: HARNESS_TRANSCRIPT_SCHEMA_VERSION, + harness, + files: Object.freeze(files), + skipped: Object.freeze(skipped), + }) + return Object.freeze({ + status: 'captured', + artifact, + fileCount: files.length, + totalBytes: total, + skippedCount: skipped.length, + }) +} + +/** + * Persist a capture under its own content ref and return the receipt a settlement carries. + * + * The scope calls this, not the executor: storage stays out of every provider and destroy site, + * exactly as the tool-span trace is persisted by `captureWorkerTraceEvidence` and not by the + * source that collected it. A capture that is already unavailable passes through untouched. + */ +export async function persistHarnessTranscript( + capture: HarnessTranscriptCapture, + blobs: Pick, +): Promise { + if (capture.status !== 'captured') return capture + const transcriptRef = contentAddress(capture.artifact) + try { + await blobs.put(transcriptRef, capture.artifact) + } catch { + return unavailable('transcript-persistence-failed') + } + return Object.freeze({ + status: 'available', + transcriptRef, + harness: capture.artifact.harness, + fileCount: capture.fileCount, + totalBytes: capture.totalBytes, + skippedCount: capture.skippedCount, + }) +} + +/** + * Rehydrate the exact persisted transcript a receipt points at, or `undefined` when the receipt + * says there is none. Throws only when the receipt claims a blob the store does not hold — that + * is corruption, not absence, and must not read as "no transcript". + */ +export async function harnessTranscriptArtifact( + evidence: HarnessTranscriptEvidence, + blobs: Pick, +): Promise { + if (evidence.status !== 'available') return undefined + const raw = await blobs.get(evidence.transcriptRef) + if (!isHarnessTranscriptArtifact(raw)) { + throw new ValidationError( + `harnessTranscriptArtifact: blob store has no transcript artifact for '${evidence.transcriptRef}'`, + ) + } + return raw +} + +/** Every entry, not just the arrays: a blob with `files: [null]` is corruption, not a transcript. */ +function isHarnessTranscriptArtifact(value: unknown): value is HarnessTranscriptArtifact { + if (value === null || typeof value !== 'object') return false + const artifact = value as Partial + return ( + artifact.schemaVersion === HARNESS_TRANSCRIPT_SCHEMA_VERSION && + typeof artifact.harness === 'string' && + Array.isArray(artifact.files) && + artifact.files.every( + (file) => + file !== null && + typeof file === 'object' && + typeof file.path === 'string' && + typeof file.content === 'string' && + Number.isSafeInteger(file.bytes) && + file.bytes >= 0, + ) && + Array.isArray(artifact.skipped) && + artifact.skipped.every( + (entry) => + entry !== null && + typeof entry === 'object' && + typeof entry.path === 'string' && + typeof entry.reason === 'string', + ) + ) +} diff --git a/src/runtime/index.ts b/src/runtime/index.ts index 25362b9e..962cee5b 100644 --- a/src/runtime/index.ts +++ b/src/runtime/index.ts @@ -205,6 +205,34 @@ export { sandboxClientAsProvider, type WorkspaceRequest, } from './environment-provider' +// Two substrates for the same "recursive agent decision" atom, both exported here (per +// docs/architecture.md): canonical = the reactive `Scope`/`Supervisor` + the personify +// combinators (budget-conserving, equal-k by construction — prefer for new recursive work); +// the round-synchronous `runAgentRounds` kernel = the path most benches still drive, with a +// caller-supplied `Driver` (fixed-shape or scripted) authoring the per-round topology. +// Recursive execution atom (the keystone): the open `Executor` runtime, the +// budget-conserving reactive `Scope`, the event-sourced `Supervisor`, and the spawn +// journal. Substrate types come from `./supervise/types`; the journal + blob store +// impls live in `../durable/spawn-journal`. +// +// Both pairs are exported: the in-memory stores (tests / scratch / a run that need not +// outlive its process) AND the file-backed stores that make a run RESUMABLE. Without the +// durable pair on the public surface a consumer cannot resume at all — and the one that +// tried wrote its own half-working copy, whose `loadTree` never read the file back. The +// replay readers ship with them, because a durable journal you cannot fold back into a +// tree is only a log. +export { + captureHarnessTranscript, + type HarnessTranscriptArtifact, + type HarnessTranscriptCapture, + type HarnessTranscriptEvidence, + type HarnessTranscriptFile, + type HarnessTranscriptUnavailable, + type HarnessTranscriptUnavailableReason, + harnessTranscriptArtifact, + persistHarnessTranscript, + type ReadableEnvironment, +} from './harness-transcript' // Per-harness usage decoders: the ONE registry of harnesses that report token usage only inside // their own event, read alongside the canonical usage events by `createSandboxUsageLedger`. Only // the composite is public, as with `decodeToolPart` — the registry and its entries stay internal @@ -263,27 +291,6 @@ export { type McpEnvironmentOptions, sanitizeMcpToolSchema, } from './mcp-environment' -// Two substrates for the same "recursive agent decision" atom, both exported here (per -// docs/architecture.md): canonical = the reactive `Scope`/`Supervisor` + the personify -// combinators (budget-conserving, equal-k by construction — prefer for new recursive work); -// the round-synchronous `runAgentRounds` kernel = the path most benches still drive, with a -// caller-supplied `Driver` (fixed-shape or scripted) authoring the per-round topology. -// Recursive execution atom (the keystone): the open `Executor` runtime, the -// budget-conserving reactive `Scope`, the event-sourced `Supervisor`, and the spawn -// journal. Substrate types come from `./supervise/types`; the journal + blob store -// impls live in `../durable/spawn-journal`. -// -// Both pairs are exported: the in-memory stores (tests / scratch / a run that need not -// outlive its process) AND the file-backed stores that make a run RESUMABLE. Without the -// durable pair on the public surface a consumer cannot resume at all — and the one that -// tried wrote its own half-working copy, whose `loadTree` never read the file back. The -// replay readers ship with them, because a durable journal you cannot fold back into a -// tree is only a log. -export type { - NativeSessionArtifact, - NativeSessionEvidence, - NativeSessionFile, -} from './native-session-evidence' export { observationFromRegistry } from './observation-registry' // The third-person observer: a worker's trace → trace-grounded findings, an // operator report, and durable corpus facts for the next run (the closed loop). diff --git a/src/runtime/native-session-evidence.test.ts b/src/runtime/native-session-evidence.test.ts deleted file mode 100644 index dd9eeb8c..00000000 --- a/src/runtime/native-session-evidence.test.ts +++ /dev/null @@ -1,149 +0,0 @@ -import { describe, expect, it } from 'vitest' - -import { captureNativeSessionEvidence } from './native-session-evidence' - -function environment(files: Record, opts: { read?: boolean; exec?: boolean } = {}) { - const listing = Object.keys(files).join('\n') - return { - ...(opts.exec === false ? {} : { exec: async () => ({ stdout: listing, exitCode: 0 }) }), - ...(opts.read === false - ? {} - : { - read: async (path: string) => { - const content = files[path] - if (content === undefined) throw new Error(`no such file: ${path}`) - return content - }, - }), - } -} - -describe('captureNativeSessionEvidence', () => { - it('carries the assistant text the tool-span receipt never had', async () => { - const evidence = await captureNativeSessionEvidence( - environment({ - '/root/.claude/projects/a/session.jsonl': '{"role":"assistant","text":"the answer"}', - '/root/.claude/history.jsonl': '{"prompt":"the question"}', - }), - 'claude-code', - ) - expect(evidence.status).toBe('available') - if (evidence.status !== 'available') return - expect(evidence.fileCount).toBe(2) - expect(evidence.skippedCount).toBe(0) - // Inline in the artifact the executor settles with; supervise blobs it under the outRef. - expect(evidence.artifact.files.map((f) => f.content).join('')).toMatch(/the answer/u) - }) - - it('never reads a credential that sits inside a session tree', async () => { - const reads: string[] = [] - const evidence = await captureNativeSessionEvidence( - { - exec: async () => ({ - stdout: ['/root/.codex/sessions/rollout.jsonl', '/root/.codex/sessions/auth.json'].join( - '\n', - ), - exitCode: 0, - }), - read: async (path: string) => { - reads.push(path) - return '{"ok":true}' - }, - }, - 'codex', - ) - expect(evidence.status).toBe('available') - if (evidence.status !== 'available') return - expect(evidence.fileCount).toBe(1) - expect(evidence.skippedCount).toBe(1) - // The assertion that matters: the read never happened, not merely that it is absent. - expect(reads).toEqual(['/root/.codex/sessions/rollout.jsonl']) - }) - - it('reports a missing capability instead of an empty artifact that reads as coverage', async () => { - // agent-provider-tangle gates read behind capabilities.workspace.read && box.read. - expect(await captureNativeSessionEvidence(environment({}, { read: false }), 'codex')).toEqual({ - status: 'unavailable', - reason: 'unsupported-environment', - }) - // read takes one path and offers no listing, so enumeration needs exec. - expect(await captureNativeSessionEvidence(environment({}, { exec: false }), 'codex')).toEqual({ - status: 'unavailable', - reason: 'enumeration-failed', - }) - expect(await captureNativeSessionEvidence(environment({}), 'no-such-harness')).toEqual({ - status: 'unavailable', - reason: 'unknown-harness', - }) - expect(await captureNativeSessionEvidence(environment({}), 'codex')).toEqual({ - status: 'unavailable', - reason: 'no-transcript', - }) - }) - - it('produces an identical artifact for an identical transcript', async () => { - const files = { '/root/.codex/sessions/r.jsonl': '{"a":1}' } - const first = await captureNativeSessionEvidence(environment(files), 'codex') - const second = await captureNativeSessionEvidence(environment(files), 'codex') - expect(first.status).toBe('available') - expect(second.status).toBe('available') - if (first.status !== 'available' || second.status !== 'available') return - // Keeps supervise's contentRef over the settled result stable across a re-capture. - expect(first.artifact).toEqual(second.artifact) - }) - - it('stops reading when the run is aborted rather than draining a dying environment', async () => { - // Without the run's linked signal a cancelled run still enumerated and read up to - // MAX_FILES out of an environment the `finally` was already tearing down. - const controller = new AbortController() - const reads: string[] = [] - const evidence = await captureNativeSessionEvidence( - { - exec: async () => ({ - stdout: Array.from({ length: 50 }, (_, i) => `/root/.codex/sessions/r${i}.jsonl`).join( - '\n', - ), - exitCode: 0, - }), - read: async (path: string, options?: { signal?: AbortSignal }) => { - options?.signal?.throwIfAborted() - reads.push(path) - if (reads.length === 3) controller.abort() - return '{}' - }, - }, - 'codex', - controller.signal, - ) - expect(reads.length).toBe(3) - expect(evidence.status).toBe('available') - if (evidence.status !== 'available') return - // The reads that never happened are named, not silently missing. - expect(evidence.skippedCount).toBe(47) - }) - - it('refuses every credential shape, not just the lowercase ones', async () => { - const reads: string[] = [] - const denied = [ - '/root/.codex/sessions/.netrc', - '/root/.codex/sessions/ID_RSA', - '/root/.codex/sessions/server.key', - '/root/.codex/sessions/bundle.p12', - '/root/.codex/sessions/AUTH.JSON', - ] - await captureNativeSessionEvidence( - { - exec: async () => ({ - stdout: ['/root/.codex/sessions/r.jsonl', ...denied].join('\n'), - exitCode: 0, - }), - read: async (path: string) => { - reads.push(path) - return '{}' - }, - }, - 'codex', - ) - expect(reads).toEqual(['/root/.codex/sessions/r.jsonl']) - }) -}) diff --git a/src/runtime/native-session-evidence.ts b/src/runtime/native-session-evidence.ts deleted file mode 100644 index bf8f03d1..00000000 --- a/src/runtime/native-session-evidence.ts +++ /dev/null @@ -1,206 +0,0 @@ -/** - * Read a sandbox child's own harness transcript out of its environment before the - * environment is destroyed, and persist it beside the tool-span `trace` receipt. - * - * Why this exists: Runtime destroys a child's environment at teardown and nothing read - * those files first, so a child's Claude Code / Codex / OpenCode session never reached a - * run record. The settled `trace` receipt covers the supervisor's own tool spans only — - * `toolName`, `args`, `status`, `callId`, with `startedAt === endedAt` — so it carries no - * assistant text, no reasoning, and no tool results, and `status: 'available'` never meant - * the transcript survived. See tangle-network/agent-runtime#1214. - * - * The port already has what this needs: `AgentEnvironment.read?(path, { signal })` - * returns a file's content as a string. Two constraints shape the design: - * - * - `read` is OPTIONAL and, on the Tangle provider, gated behind - * `capabilities.workspace.read && box.read`. An environment without it is reported as - * `unsupported` rather than silently producing an empty artifact — the whole point of - * #1214 is that a receipt must not claim coverage it does not have. - * - `read` takes one path and gives no directory listing, so enumeration goes through - * `exec`. An environment with `read` but no `exec` cannot be enumerated and is also - * reported, not guessed at. - * - * Storage is NOT this module's business. The executor returns the artifact inside the result - * it already settles with, and supervise puts that under the child's `outRef` in its own - * `ResultBlobStore`. No destroy site learns about storage, and replay rehydrates it for free. - * - * Credentials are excluded by construction: enumeration lists only the transcript globs - * for the harness, and any path matching DENY is dropped even if a producer moved a - * credential file inside a session directory. The helper never reads `auth.json`, - * `.credentials.json`, `credentials`, or a dotenv file. - */ -/** Where each harness keeps the session files that hold the conversation. */ -const HARNESS_ROOTS: Readonly> = Object.freeze({ - 'claude-code': Object.freeze(['.claude/projects', '.claude/history.jsonl', '.claude/todos']), - codex: Object.freeze(['.codex/sessions', '.codex/history.jsonl']), - opencode: Object.freeze(['.local/share/opencode/storage', '.local/share/opencode/log']), - // Pi keeps a session tree per cwd; it had no entry, so a pi child captured nothing. - pi: Object.freeze(['.pi/agent/sessions']), -}) - -/** - * Never read, whatever a producer put inside a session tree. - * - * Case-insensitive and broader than the obvious names: `.netrc`, `*.key`, `*.p12`/`*.pfx` - * and an uppercase `ID_RSA` all slipped an earlier lowercase-only pattern. - */ -const DENY = - /(^|\/)(auth\.json|\.credentials\.json|credentials|\.netrc|\.env(\..*)?|secrets?(\.|$)|.*\.(pem|key|p12|pfx)|id_[a-z0-9_]+)$/iu - -/** - * Bounds per file and in total. These are deliberately modest: the artifact rides inside the - * settled result that supervise blobs under one `outRef`, so an unbounded transcript would - * bloat every replay of that child, not just the capture. - */ -const MAX_FILE_BYTES = 2 * 1024 * 1024 -const MAX_TOTAL_BYTES = 16 * 1024 * 1024 -const MAX_FILES = 1_000 - -export const NATIVE_SESSION_SCHEMA_VERSION = 1 as const - -export interface NativeSessionFile { - readonly path: string - readonly bytes: number - readonly content: string -} - -export interface NativeSessionArtifact { - readonly schemaVersion: typeof NATIVE_SESSION_SCHEMA_VERSION - readonly harness: string - readonly files: readonly NativeSessionFile[] - /** Paths found but not read, with why — a gap named is a gap an operator can act on. */ - readonly skipped: readonly { readonly path: string; readonly reason: string }[] -} - -export type NativeSessionEvidence = - | { - readonly status: 'available' - readonly artifact: NativeSessionArtifact - readonly fileCount: number - readonly totalBytes: number - /** Non-zero when some transcript was found but deliberately not carried. */ - readonly skippedCount: number - } - | { - readonly status: 'unavailable' - readonly reason: - | 'unsupported-environment' - | 'unknown-harness' - | 'no-transcript' - | 'enumeration-failed' - } - -interface ReadableEnvironment { - readonly read?: (path: string, options?: { readonly signal?: AbortSignal }) => Promise - readonly exec?: ( - command: string, - options?: Record, - ) => Promise<{ readonly stdout?: string; readonly exitCode?: number }> -} - -type NativeSessionUnavailableReason = Extract< - NativeSessionEvidence, - { status: 'unavailable' } ->['reason'] - -function unavailable(reason: NativeSessionUnavailableReason): NativeSessionEvidence { - return Object.freeze({ status: 'unavailable', reason }) -} - -/** - * Enumerate candidate transcript paths inside the environment. - * - * `find` is given the roots and prints one path per line. A missing root is not an error: - * a child that never used a harness has no directory for it. - */ -async function enumerate( - environment: ReadableEnvironment, - roots: readonly string[], - signal?: AbortSignal, -): Promise { - if (!environment.exec) return undefined - const quoted = roots.map((r) => `"$HOME/${r}"`).join(' ') - const command = `find ${quoted} -type f -size -${Math.floor(MAX_FILE_BYTES / 1024)}k 2>/dev/null | head -${MAX_FILES}` - try { - const result = await environment.exec(command, signal ? { signal } : undefined) - return (result.stdout ?? '') - .split('\n') - .map((l) => l.trim()) - .filter((l) => l.length > 0) - } catch { - return undefined - } -} - -/** - * Read the harness transcript out of one LIVE environment. - * - * Call this before the environment is destroyed — on the settled path that means before the - * result is built, since the `finally` that destroys runs after. It never throws: a teardown - * must not fail because evidence could not be collected, and every failure mode is a named - * `reason` the settled receipt carries instead of an empty artifact that reads as coverage. - */ -export async function captureNativeSessionEvidence( - environment: ReadableEnvironment | undefined, - harness: string | undefined, - signal?: AbortSignal, -): Promise { - if (!environment?.read) return unavailable('unsupported-environment') - // Narrow `harness` before use: the roots lookup alone does not, and an artifact must - // name the harness it came from. - if (harness === undefined) return unavailable('unknown-harness') - const roots = HARNESS_ROOTS[harness] - if (!roots) return unavailable('unknown-harness') - - const paths = await enumerate(environment, roots, signal) - if (paths === undefined) return unavailable('enumeration-failed') - if (paths.length === 0) return unavailable('no-transcript') - - const files: NativeSessionFile[] = [] - const skipped: { path: string; reason: string }[] = [] - let total = 0 - for (const path of paths) { - // Stop on abort rather than attempting every remaining read and failing each: a - // cancelled run should leave the dying environment alone, not make MAX_FILES doomed - // calls into it. The remainder is still named in `skipped`. - if (signal?.aborted) { - skipped.push({ path, reason: 'aborted' }) - continue - } - if (DENY.test(path)) { - skipped.push({ path, reason: 'denied-credential-shaped-path' }) - continue - } - if (total >= MAX_TOTAL_BYTES) { - skipped.push({ path, reason: 'total-byte-budget-exhausted' }) - continue - } - try { - const content = await environment.read(path, signal ? { signal } : undefined) - const bytes = Buffer.byteLength(content, 'utf8') - if (bytes > MAX_FILE_BYTES) { - skipped.push({ path, reason: 'file-exceeds-byte-bound' }) - continue - } - files.push(Object.freeze({ path, bytes, content })) - total += bytes - } catch { - skipped.push({ path, reason: 'read-failed' }) - } - } - if (files.length === 0) return unavailable('no-transcript') - - const artifact: NativeSessionArtifact = Object.freeze({ - schemaVersion: NATIVE_SESSION_SCHEMA_VERSION, - harness, - files: Object.freeze(files), - skipped: Object.freeze(skipped), - }) - return Object.freeze({ - status: 'available', - artifact, - fileCount: files.length, - totalBytes: total, - skippedCount: skipped.length, - }) -} diff --git a/src/runtime/supervise/completion-gate.ts b/src/runtime/supervise/completion-gate.ts index 78ef5906..ff26b574 100644 --- a/src/runtime/supervise/completion-gate.ts +++ b/src/runtime/supervise/completion-gate.ts @@ -90,6 +90,7 @@ export function gateOnDeliverable( ...(inner.progress ? { progress: () => inner.progress?.() } : {}), ...(inner.traceSource ? { traceSource: () => inner.traceSource?.() } : {}), ...(inner.metered ? { metered: () => inner.metered?.() } : {}), + ...(inner.harnessTranscript ? { harnessTranscript: () => inner.harnessTranscript?.() } : {}), execute(task, signal) { const r = inner.execute(task, signal) if (isAsyncIterable(r)) { @@ -184,6 +185,7 @@ export function mapExecutorResult( ...(inner.traceSource ? { traceSource: () => inner.traceSource?.() } : {}), ...(inner.accounting ? { accounting: () => inner.accounting?.() } : {}), ...(inner.metered ? { metered: () => inner.metered?.() } : {}), + ...(inner.harnessTranscript ? { harnessTranscript: () => inner.harnessTranscript?.() } : {}), execute(task, signal) { const execution = inner.execute(task, signal) if (isAsyncIterable(execution)) { diff --git a/src/runtime/supervise/scope.ts b/src/runtime/supervise/scope.ts index b38ed883..5b2adc9f 100644 --- a/src/runtime/supervise/scope.ts +++ b/src/runtime/supervise/scope.ts @@ -33,6 +33,12 @@ import { import { contentAddress } from '../../durable/spawn-journal' import { ValidationError } from '../../errors' import { notifyRuntimeHookEvent, type RuntimeHooks } from '../../runtime-hooks' +import { + type HarnessTranscriptCapture, + type HarnessTranscriptEvidence, + harnessTranscriptUnavailable, + persistHarnessTranscript, +} from '../harness-transcript' import type { RetainedInteractiveAdmission } from '../retained-run-types' import type { Iteration } from '../types' import { cloneTokenUsage, zeroSpend } from '../util' @@ -427,6 +433,9 @@ type PreSeqSettled = spent: Spend providerModel?: import('./types').ProviderModelExecutionEvidence trace: WorkerTraceEvidence + /** Whether the child's own harness transcript survived its environment. Sibling of `trace`; + * see the `Settled` doc for why it cannot live inside the trace receipt. */ + harnessTranscript?: HarnessTranscriptEvidence /** A driver child's OWN-inference subtree total (from `Executor.metered()`) — journaled as a * `metered` event for this node, NOT reconciled (already debited live via `observe`). */ metered?: Spend @@ -437,6 +446,9 @@ type PreSeqSettled = infra: boolean outRef?: string trace: WorkerTraceEvidence + /** The dropped child's own harness transcript, or the named reason it is absent. The whole + * point of #1244: this path has no result artifact for a capture to ride in. */ + harnessTranscript?: HarnessTranscriptEvidence providerModel?: import('./types').ProviderModelExecutionEvidence /** A CRASHED driver child's partial OWN-inference subtree total — re-homed on the down path * too, so the journal matches the pool (which already debited it via `observe`). */ @@ -1369,6 +1381,9 @@ export function createScope(args: ScopeArgs): Scope { s.metered, s.providerModel, s.outRef, + // Re-wrapping a record the child already settled: its transcript receipt is + // established evidence and a persistence failure here does not unmake it. + s.harnessTranscript, ) } live.resolved = resolution @@ -2486,9 +2501,17 @@ async function appendReconciledFloor( id: NodeId, spent: Spend, at: string, + harnessTranscript?: HarnessTranscriptEvidence, ): Promise { const seq = await nextPerNodeSeq(journal, root, 'reconciled', id) - await appendAcknowledged(journal, root, { kind: 'reconciled', id, spent, seq, at }) + await appendAcknowledged(journal, root, { + kind: 'reconciled', + id, + spent, + ...(harnessTranscript ? { harnessTranscript } : {}), + seq, + at, + }) } /** A release receipt has the same per-child sequence discipline: outside the cursor namespace and @@ -2587,6 +2610,9 @@ async function finalizeSettlement( ...(settlement.providerModel ? { providerModel: settlement.providerModel } : {}), ...(child.budgetViolation ? { budgetViolation: child.budgetViolation } : {}), trace: settlement.trace, + ...(settlement.harnessTranscript + ? { harnessTranscript: settlement.harnessTranscript } + : {}), seq, at, }) @@ -2597,7 +2623,14 @@ async function finalizeSettlement( // retained child against a metered 10 (#1190). The floor is journaled in the settlement's // place, outside the cursor namespace, so the slot stays open and the ledgers agree. else if (settlement.reconciled !== undefined) - await appendReconciledFloor(args.journal, args.root, child.id, settlement.reconciled, at) + await appendReconciledFloor( + args.journal, + args.root, + child.id, + settlement.reconciled, + at, + settlement.harnessTranscript, + ) notifyRuntimeHookEvent( args.hooks, { @@ -2629,6 +2662,7 @@ async function finalizeSettlement( ...(settlement.providerModel ? { providerModel: settlement.providerModel } : {}), ...(child.budgetViolation ? { budgetViolation: child.budgetViolation } : {}), trace: settlement.trace, + ...(settlement.harnessTranscript ? { harnessTranscript: settlement.harnessTranscript } : {}), settledAt, seq, } @@ -2649,6 +2683,7 @@ async function finalizeSettlement( ...(settlement.providerModel ? { providerModel: settlement.providerModel } : {}), ...(child.budgetViolation ? { budgetViolation: child.budgetViolation } : {}), trace: settlement.trace, + ...(settlement.harnessTranscript ? { harnessTranscript: settlement.harnessTranscript } : {}), seq, at, }) @@ -2684,6 +2719,7 @@ async function finalizeSettlement( ...(settlement.providerModel ? { providerModel: settlement.providerModel } : {}), ...(child.budgetViolation ? { budgetViolation: child.budgetViolation } : {}), trace: settlement.trace, + ...(settlement.harnessTranscript ? { harnessTranscript: settlement.harnessTranscript } : {}), settledAt, seq, } @@ -2869,6 +2905,14 @@ async function runChild( traceEvidence ??= await captureWorkerTraceEvidence(live.readTraceSource, blobs, started) return traceEvidence } + // Same shape as the trace: read from the executor once per settle exit, persisted under its + // own content ref, and only the receipt travels on the record. Every arm below, done and + // down alike, carries it — the arms that lose it are the failure arms (#1244). + let transcriptEvidence: HarnessTranscriptEvidence | undefined + const persistTranscriptOnce = async (): Promise => { + transcriptEvidence ??= await persistHarnessTranscript(readHarnessTranscript(executor), blobs) + return transcriptEvidence + } const teardownOnce = async (grace: number | 'brutalKill' | 'infinity'): Promise => { if (teardownStarted) return teardownStarted = true @@ -2990,6 +3034,7 @@ async function runChild( // (done, aborted, crash) so the journal always matches what the pool already debited. const ownMetered = executor.metered?.() const trace = await captureTraceOnce() + const childHarnessTranscript = await persistTranscriptOnce() if (childAbort.signal.aborted && live.acceptedResult === undefined) { await teardownOnce(opts.shutdown ?? 'brutalKill') @@ -3000,6 +3045,7 @@ async function runChild( ownMetered, runtimeOwnedExecutorProviderEvidence(executor), retainedOutputRef, + childHarnessTranscript, ) } @@ -3015,6 +3061,8 @@ async function runChild( trace, ownMetered, runtimeOwnedExecutorProviderEvidence(executor), + undefined, + childHarnessTranscript, ), outRef, } @@ -3027,6 +3075,7 @@ async function runChild( ...(artifact.verdict ? { verdict: artifact.verdict } : {}), spent: live.spent, trace, + harnessTranscript: childHarnessTranscript, providerModel: runtimeOwnedExecutorProviderEvidence(executor), ...(ownMetered ? { metered: ownMetered } : {}), } @@ -3062,6 +3111,8 @@ async function runChild( trace, metered, runtimeOwnedExecutorProviderEvidence(executor), + undefined, + await persistTranscriptOnce(), ), outRef: accepted.outRef, } @@ -3073,6 +3124,7 @@ async function runChild( ...(accepted.verdict ? { verdict: accepted.verdict } : {}), spent: live.spent, trace, + harnessTranscript: await persistTranscriptOnce(), providerModel: runtimeOwnedExecutorProviderEvidence(executor), ...(metered ? { metered } : {}), } @@ -3128,6 +3180,9 @@ async function runChild( executor.metered?.(), undefined, retainedOutputRef, + // The #1244 population exactly: 45 of these in one evening, each of which executed and + // reasoned and left no artifact. This is the only record their transcript can reach. + await persistTranscriptOnce(), ), reconciled: live.spent, } @@ -3181,6 +3236,7 @@ async function runChild( executor.metered?.(), providerModel, retainedOutputRef, + await persistTranscriptOnce(), ) } finally { await closeRetainedWrites() @@ -3541,18 +3597,54 @@ function downRecord( metered?: Spend, providerModel?: import('./types').ProviderModelExecutionEvidence, outRef?: string, + harnessTranscript?: HarnessTranscriptEvidence, ): Extract { return { kind: 'down', reason, infra, trace, + ...(harnessTranscript ? { harnessTranscript } : {}), ...(providerModel ? { providerModel } : {}), ...(metered ? { metered } : {}), ...(outRef === undefined ? {} : { outRef }), } } +/** + * Read one executor's harness-transcript receipt without letting a broken port escape. + * + * The three answers this has to keep apart, because #1214 and #1244 are both about an artifact + * that reads as coverage without being coverage: + * - `available` the transcript survived; + * - a reason the CAPTURE produced a box existed and could not be read; + * - `executor-exposes-no-transcript` this runtime has no transcript to offer at all; + * - `execution-never-started` no environment was ever created (the executor's + * own seed, set before `create`); + * - `capture-did-not-run` the port answered nothing, so the capture was + * skipped rather than attempted and failed. + * + * Mirrors `readInteractiveSession` above: an absence is always a named reason, never `undefined`. + */ +function readHarnessTranscript(executor: { + harnessTranscript?: () => unknown +}): HarnessTranscriptCapture { + if (!executor.harnessTranscript) { + return harnessTranscriptUnavailable('executor-exposes-no-transcript') + } + let reported: unknown + try { + reported = executor.harnessTranscript() + } catch { + return harnessTranscriptUnavailable('executor-exposes-no-transcript') + } + if (reported === undefined) return harnessTranscriptUnavailable('capture-did-not-run') + const capture = reported as HarnessTranscriptCapture + if (capture.status === 'captured' && capture.artifact) return capture + if (capture.status === 'unavailable' && capture.reason) return capture + return harnessTranscriptUnavailable('executor-exposes-no-transcript') +} + /** The one place an absent interactive process is spelled, so every refusal reads the same. */ function noInteractiveSession( reason: WorkerInteractiveUnavailableReason, diff --git a/src/runtime/supervise/types.ts b/src/runtime/supervise/types.ts index c93f024f..ccd83832 100644 --- a/src/runtime/supervise/types.ts +++ b/src/runtime/supervise/types.ts @@ -35,7 +35,7 @@ import type { } from '@tangle-network/agent-interface' import type { BackendType } from '@tangle-network/sandbox' import type { RuntimeHooks } from '../../runtime-hooks' -import type { NativeSessionEvidence } from '../native-session-evidence' +import type { HarnessTranscriptCapture, HarnessTranscriptEvidence } from '../harness-transcript' import type { RetainedInteractiveRunHandle } from '../retained-interactive-types' import type { RetainedRunEffect } from '../retained-run-types' import type { LoopTokenUsage } from '../types' @@ -239,6 +239,18 @@ export interface Executor { * executors omit it (returns `undefined`). */ metered?(): Spend | undefined + /** + * The child's OWN harness transcript, read out of its environment while that environment was + * still live. Read on settle, valid after `execute` resolves OR throws — the throw half is the + * point: a child that drops produces no result artifact, so before this the only carrier was + * the settled result and a dropped child's reasoning died with its box (#1244). + * + * An executor with no transcript to offer omits the method entirely; the settle path then + * records `executor-exposes-no-transcript` rather than an empty artifact that reads as + * coverage. Never throws. Returns the in-memory capture; the SCOPE persists it under its own + * content ref and settles the receipt, so no executor ever learns about storage. + */ + harnessTranscript?(): HarnessTranscriptCapture | undefined } /** Why Runtime cannot provide structured tool-call evidence for one settled execution. */ @@ -260,16 +272,6 @@ export type WorkerTraceEvidence = /** Content-addressed pointer to a persisted `WorkerToolTraceArtifact`. */ readonly traceRef: string readonly spanCount: number - /** - * The child's OWN harness transcript, read out of its environment before destroy. - * - * `traceRef` above points at the supervisor's tool spans: toolName, args, status, - * callId, with startedAt === endedAt. It carries no assistant text, no reasoning and - * no tool results, so `status: 'available'` on this object never meant the child's - * session survived — it was destroyed with the environment. This says whether it did. - * Absent on a settlement recorded before the capture existed. - */ - readonly nativeSession?: NativeSessionEvidence } | { readonly status: 'unavailable' @@ -962,6 +964,12 @@ export type Settled = providerModel?: ProviderModelExecutionEvidence /** Structured tool evidence captured before this settlement was journaled. */ trace: WorkerTraceEvidence + /** Whether the child's OWN harness transcript survived its environment, or why it did not. + * A SIBLING of `trace`, not a field inside it: `trace` carries the supervisor's tool spans + * (toolName, args, status, callId, startedAt === endedAt) and nothing the child said, and a + * child with zero tool spans has an UNAVAILABLE trace — so a receipt nested inside the + * available arm could never describe exactly the children that need it most (#1244). */ + harnessTranscript?: HarnessTranscriptEvidence /** Present when the measured spend exceeded this child's reservation. */ budgetViolation?: BudgetViolation /** Epoch ms parsed from the durable settlement record when available. */ @@ -979,6 +987,11 @@ export type Settled = infra: boolean /** Partial structured tool evidence captured before this failure was journaled. */ trace: WorkerTraceEvidence + /** The child's own harness transcript, read out of its environment at the last moment it + * was live, or the named reason it could not be. This is the path the capture existed + * for and never covered: a dropped child produces no result artifact, so before #1244 + * its reasoning was destroyed with its box. */ + harnessTranscript?: HarnessTranscriptEvidence /** Partial provider model evidence survives an aborted or failed execution. */ providerModel?: ProviderModelExecutionEvidence /** Present when the spend reconciled for this child exceeded its reservation. */ @@ -1328,6 +1341,10 @@ export type SpawnEvent = reason?: string /** Structured tool evidence. Optional only for journals written before trace capture. */ trace?: WorkerTraceEvidence + /** Whether this child's harness transcript survived, or the named reason it did not. + * Absent on journals written before the capture existed — which is not the same fact as + * a recorded `unavailable`, and is why this stays optional rather than defaulting. */ + harnessTranscript?: HarnessTranscriptEvidence /** Present when the reconciled spend exceeded the reservation, on either status. */ budgetViolation?: BudgetViolation seq: number @@ -1342,6 +1359,9 @@ export type SpawnEvent = spent?: Spend providerModel?: ProviderModelExecutionEvidence trace?: WorkerTraceEvidence + /** The child's harness transcript receipt, when the executor could still be read at the + * cancel. The settle path writes it on this record exactly as on `settled`. */ + harnessTranscript?: HarnessTranscriptEvidence outRef?: string budgetViolation?: BudgetViolation seq: number @@ -1461,6 +1481,11 @@ export type SpawnEvent = kind: 'reconciled' id: NodeId spent: Spend + /** The transcript receipt of a retained-pending child. This record is the ONLY durable home + * it has: the node writes no `settled` record while its slot stays open, and these are the + * #1244 children exactly — dropped mid-run with a live box the capture read. A later + * terminal record for the node carries the same receipt forward. */ + harnessTranscript?: HarnessTranscriptEvidence seq: number at: string } diff --git a/src/runtime/supervise/worker-retry.ts b/src/runtime/supervise/worker-retry.ts index 06b5c0d7..c915588f 100644 --- a/src/runtime/supervise/worker-retry.ts +++ b/src/runtime/supervise/worker-retry.ts @@ -259,6 +259,7 @@ export function retryPreSpawnRefusals( ...(inner.progress ? { progress: () => inner.progress?.() } : {}), ...(inner.traceSource ? { traceSource: () => inner.traceSource?.() } : {}), ...(inner.metered ? { metered: () => inner.metered?.() } : {}), + ...(inner.harnessTranscript ? { harnessTranscript: () => inner.harnessTranscript?.() } : {}), ...(inner.interactive === undefined ? {} : { interactive: (): WorkerInteractiveSession => interactiveOf(inner) }), diff --git a/src/testing/fixtures/agent-improvement-proposal.json b/src/testing/fixtures/agent-improvement-proposal.json index 7699974d..8ad18e32 100644 --- a/src/testing/fixtures/agent-improvement-proposal.json +++ b/src/testing/fixtures/agent-improvement-proposal.json @@ -1,6 +1,6 @@ { "changedSurfaces": ["prompt"], - "digest": "sha256:6e82ad15ab3d4ce4f73913de01f1e13ebc68bb4b4d9120b43382f01954dd956e", + "digest": "sha256:201a9a98fbdc078d03c35314e4e03f34abaa36d7db394a11c4b1110bd036521c", "evaluation": { "decision": { "contributingChecks": [ @@ -4882,7 +4882,7 @@ ], "metadata": { "fixture": "agent-improvement-proposal", - "runtimeVersion": "0.231.1" + "runtimeVersion": "0.232.0" }, "objectives": [ { @@ -4993,8 +4993,8 @@ "baselineContentHash": "sha256:5c21ee53e513fc604cb09754e21c392b24a424da0ef37dbf8f1ee4a8a0b08f09", "candidateContentHash": "sha256:60fcbb1c728194bd51d7d19cb732d1c3f1881dce7e0a6266b41c8b98cfd65693", "kind": "agent-eval-loop", - "recordDigest": "sha256:e26e7e4ff57e2509e042c553c638f579680f438553fbede8b8d51c35b2c6be68", - "runId": "agent-runtime-0.231.1-proposal-fixture", + "recordDigest": "sha256:d26f696c1a140606d4e75b1ade7a03f08ce13f4fc4fdfe0527d2fbb4717e438d", + "runId": "agent-runtime-0.232.0-proposal-fixture", "schema": "agent-candidate-experiment" } }, @@ -5021,5 +5021,5 @@ ], "kind": "agent-improvement-proposal", "proposedAt": "2026-07-10T01:00:00.000Z", - "runId": "agent-runtime-0.231.1-proposal-fixture" + "runId": "agent-runtime-0.232.0-proposal-fixture" } diff --git a/src/testing/fixtures/agent-profile-improvement-proposal.json b/src/testing/fixtures/agent-profile-improvement-proposal.json index 65432b0f..1d92c27a 100644 --- a/src/testing/fixtures/agent-profile-improvement-proposal.json +++ b/src/testing/fixtures/agent-profile-improvement-proposal.json @@ -1,6 +1,6 @@ { "changedSurfaces": ["prompt", "skills"], - "digest": "sha256:d02c9c48865e63a968095846d85416c925186c770caca400680b2448988414f8", + "digest": "sha256:702dc1a85ce46804d67de7fd179bd69f6af0027dca7b6a99c46ab7f159bf7f5f", "evaluation": { "decision": { "contributingChecks": [ @@ -1715,7 +1715,7 @@ ], "metadata": { "fixture": "agent-profile-improvement-proposal", - "runtimeVersion": "0.231.1" + "runtimeVersion": "0.232.0" }, "objectives": [ { @@ -1826,7 +1826,7 @@ "baselineContentHash": "sha256:21c495a37c418c10bde64fbaa188beddeed31f1f051ea60a6a6582a9ee0db704", "candidateContentHash": "sha256:103f77bc8481601eef1ad5fe6ba84a40dffabc3a44f421f8c8559121edab84e9", "kind": "agent-eval-loop", - "recordDigest": "sha256:5cad90b609c3f39e1fe23ea5a140b10fb58221ee0b7aabf12b62a8e5c0e3b58c", + "recordDigest": "sha256:6a55eeeef02f261ce41fa35063839ec37d0b64815dae2517e9d92c75dd2585ce", "runId": "profile-improvement-1", "schema": "agent-profile-improvement-experiment" }