From 1117d11faa5343910b9f971b21be58f5a4bdf37e Mon Sep 17 00:00:00 2001 From: Raymond Chen <45910466+chenmingwei23@users.noreply.github.com> Date: Wed, 16 Sep 2026 13:59:36 +0000 Subject: [PATCH] docs(ledger): add an API-level reference for the ledger event contract The ledger's on-disk event contract is readable only by reading the emitter: which types exist, what each payload carries, which type closes which, and what a reader may assume are spread across session_ledger_emit.py and the store. Anything consuming a ledger has to guess field names and required-ness from the code that writes them. Adds docs/reference/ledger/ as five pages -- the concepts and guarantees (README), the shared envelope, the 26 session types on a fixed nine-part template, the crew types, and the read/write API with its error codes. docs/reference/ is a mirror directory that defaults to "do not author", so each page carries the named-exception mark and the index gains one marked row, following ledger-conductor-sequence.md. Field shapes come from session_ledger_emit.py as it stands on #11185. The summary table's Emitter column separates the 18 types an emitter writes today from the 8 whose emitter lands with that change. The crash-repair closer carries only turn and stop_reason, which is what store.py writes. tool/completed.status is an open set: the terminal frame passes the backend's own word through unmapped. Every entry point resolves the unit's path first, so exists, create and open all refuse bad_kind, invalid_id and bad_root before their own codes. test/test_docs_ledger_reference.py holds the pages to the code. Every documented type's domain is owned by the session kind in TYPE_OWNERSHIP; the summary table matches the subsections; every JSON example is one line, parses, names a documented type, names a session emitter, and survives the envelope reader. Two checks guard the parts a reader cannot verify by eye: a type marked as awaiting its emitter must have no caller in this build, so the mark cannot outlive the change that lands it, and a paragraph naming MAX_ENTRY_BYTES, MAX_REF_SPAN, MAX_PAGE_LIMIT or DEFAULT_PAGE_LIMIT must state that constant's current value. --- docs/reference/README.md | 1 + docs/reference/ledger/README.md | 153 ++++ docs/reference/ledger/crew-types.md | 149 +++ docs/reference/ledger/envelope.md | 164 ++++ docs/reference/ledger/errors.md | 262 ++++++ docs/reference/ledger/reading-and-writing.md | 270 ++++++ docs/reference/ledger/session-types.md | 912 +++++++++++++++++++ test/test_docs_ledger_reference.py | 276 ++++++ 8 files changed, 2187 insertions(+) create mode 100644 docs/reference/ledger/README.md create mode 100644 docs/reference/ledger/crew-types.md create mode 100644 docs/reference/ledger/envelope.md create mode 100644 docs/reference/ledger/errors.md create mode 100644 docs/reference/ledger/reading-and-writing.md create mode 100644 docs/reference/ledger/session-types.md create mode 100644 test/test_docs_ledger_reference.py diff --git a/docs/reference/README.md b/docs/reference/README.md index 249f03934f05..df6f92ccfe59 100644 --- a/docs/reference/README.md +++ b/docs/reference/README.md @@ -15,6 +15,7 @@ writing both marks, not editing the page and leaving the rule to look broken. |---|---| | [kiro-cli/](kiro-cli/README.md) | The `kiro-cli` documentation, taken from the 2.x line. kiro-cli is Kiro Crew's required agent backend, so its ACP surface, agent-spec schema, and MCP config shape are load-bearing here. | | [ledger-conductor-sequence.md](ledger-conductor-sequence.md) | **Named exception — local page, no upstream.** Kiro Crew's own end-to-end sequence for the work-ledger dispatch lifecycle. Preserve it across a re-fetch. | +| [ledger/](ledger/README.md) | **Named exception — local pages, no upstream.** API-style reference for the append-only ledger: envelope, session entry types, crew entry types, the reader and writer surface, and the error codes. Preserve it across a re-fetch. | Where Kiro Crew's own behavior differs from a mirrored page, Kiro Crew's docs win: this fork is KiroACP-only and does not use every capability the upstream CLI diff --git a/docs/reference/ledger/README.md b/docs/reference/ledger/README.md new file mode 100644 index 000000000000..872f6b10fb4a --- /dev/null +++ b/docs/reference/ledger/README.md @@ -0,0 +1,153 @@ +# Ledger reference + +**Local page, not a mirror.** This directory is Kiro Crew's own documentation and +has no upstream source, so a re-fetch of the `kiro-cli/` mirror must leave it +alone. It is marked as a named exception in [the Reference index](../README.md). + +A ledger is one append-only JSONL file per unit. A writer adds lines to the end +and never rewrites a line already there, so the file — not a context window, not +a summary — is the authority for what happened to that unit. Every line after the +first carries a writer-assigned `seq` that is contiguous from 1 within the file, +so a reader folds the lines in `seq` order and gets exactly one reading of the +unit's history. + +These pages say WHAT the format is. For WHY it is shaped this way, read +[`ledger-core.md`](../../system-specs/modules/ledger-core.md) and +[`session-ledger-emitter.md`](../../system-specs/modules/session-ledger-emitter.md). + +## Contents + +| Page | What it covers | +|---|---| +| [envelope.md](envelope.md) | File layout, header fields, the eight common entry fields, `ref` and `thread`. Kind-independent. | +| [session-types.md](session-types.md) | Every session ledger entry type, one subsection each, with fields and an example. | +| [crew-types.md](crew-types.md) | The crew ledger's type families and the two dispatch contracts. | +| [reading-and-writing.md](reading-and-writing.md) | Reader API, writer rules, ownership, and the fail-soft emitter. | +| [errors.md](errors.md) | Every `LedgerError` code, its trigger, and what a caller does about it. | + +## Glossary + +**ledger** — the storage layer (`kiro_crew.ledger`): a file format, the rules for +who may write what, and the read-back paths. "Append-only log" names the same +thing; these pages say *ledger* throughout. + +**unit** — the thing a ledger belongs to. One unit, one ledger. + +**kind** — which of the two unit types a ledger is: `crew` or `session`. A kind +decides which `type` domains may be written to the file and which `src` values may +write them. + +**session ledger** — a `session`-kind ledger, keyed by an ACP session id, +recording one session's turn lifecycle. [session-types.md](session-types.md) +covers it. + +**crew ledger** — a `crew`-kind ledger. Same envelope, different type domains, and +the place `thread` and guest-namespaced types are used. +[crew-types.md](crew-types.md) covers it. + +**entry** — one line after the header: an envelope plus a type-specific `data` +object. + +**segment** — one physical file in a unit's ledger. A unit's history may span +several, read oldest first. Every segment repeats the header. + +**opener / closer** — a pair of types where one records that something started and +the other that it ended. A closer is what makes an interrupted opener detectable. + +## What a ledger is not + +It is not the **SEL security audit log**, which records security-relevant +decisions for an auditor and answers a different question. + +It is not a **session transcript**. Transcripts are message-grain, are rewritten +by rotation and compaction, and are a best-effort shadow of what a session slot +held. A ledger is operation-grain and append-only. + +It is not the **per-turn usage row store**, which measures token and credit cost +per turn for reporting. + +It is not a **global event stream**. A day-sharded stream of lifecycle events +joined by a correlation key is a different shape: it has no per-unit `seq`, so it +cannot be folded into one unit's authoritative history. No emitter writes the same +fact to both. + +## The pointer model + +An entry cites another ledger with `ref`. A session's header points back at the +crew row that dispatched it with `thread`. Neither copies bytes: both name a +location, and a reader that wants the content goes and reads it. + +```mermaid +flowchart LR + subgraph crew["crew ledger — crews/<store name>/ledger.jsonl"] + D["seq 41 · crew/dispatch
data.target → a session"] + R["seq 58 · crew/report
thread: 41
ref: {unit: session, id: s-7f3a, from: 12, to: 40}"] + end + subgraph sess["session ledger — sessions/<store name>/ledger.jsonl"] + H["line 1 · header
thread: {crew: qa, seq: 41}"] + S["seq 12 … seq 40
the cited segment"] + end + R -- "ref cites the work" --> S + H -- "thread names the dispatch" --> D + R -. "thread groups the reply
onto the dispatch" .-> D +``` + +Two things are worth reading off that diagram. The `ref` is the one deliberate +cross-kind pointer: a crew entry cites a span of a session's file. The `thread` is +always within-file — `crew/report` threads onto `crew/dispatch` because both are +lines in the crew ledger — except on the session *header*, where `thread` is a +back-pointer naming a row in another unit and is spelled `{crew, seq}` rather than +a bare int. + +## Guarantees + +Each guarantee names the error a caller gets when it is enforced against them. + +**Append-only.** A writer adds lines. Nothing already written is rewritten in +place, so a reader that has folded up to `seq` N never has to re-read below N. + +**`seq` is contiguous from 1, and one writer owns a file.** Seq is assigned under +the unit's lock as `last_seq + 1`, read back from the file's own tail rather than +from a cached counter. A second process that tries to append gets +[`already_owned`](errors.md#already_owned) and appends nothing. A gap between two +segments mid-chain is damage, not a pruned log: +[`segment_gap`](errors.md#segment_gap). + +**Crash closers are deterministic.** Opening a session ledger with repair on +truncates a torn tail, drops a trailing orphan chunk run, and writes one closer +per still-open opener with a fixed value per type. The same file always repairs to +the same bytes. See [reading-and-writing.md](reading-and-writing.md#crash-repair). + +**An oversize entry is refused, never truncated.** The serialized line ceiling is +64 KiB (`MAX_ENTRY_BYTES`). A line over it raises +[`entry_too_large`](errors.md#entry_too_large) before any byte is written, so a +reader never meets a half-recorded fact. An oversize *body* is split into +`message/chunk` entries by the emitter instead of being clipped. + +**A `ref` spans at most 500 lines** (`MAX_REF_SPAN`). A citation is a pointer, not +a bulk export; a wider span raises [`bad_ref`](errors.md#bad_ref). + +**A reader is never handed an entry it cannot interpret.** A reader that declares +the types it knows gets [`unknown_entry_type`](errors.md#unknown_entry_type) on an +unknown entry, unless the writer marked that entry `ignorable`. Reconstruction +stops rather than silently skipping a line that may change the meaning of every +line after it. + +**Every file identifies its own unit.** Line 1 names the kind and the id, and each +segment repeats it. A header that names a different unit, or a version this build +does not read, is refused: [`bad_header`](errors.md#bad_header), +[`bad_segment`](errors.md#bad_segment), +[`unsupported_version`](errors.md#unsupported_version). + +**Agent file tools cannot reach `ledgers/`.** The whole root is a fenced leaf in +the shared sensitive-path floor, so a prompt-injected agent cannot read another +unit's history or forge a line into its own. The ledger's own code opens the files +directly and is unaffected. + +**Session emission is off by default.** The session emitter is inert unless +`KIROCREW_SESSION_LEDGER` is truthy. With the flag unset no ledger directory is +created and no emit path reaches storage. + +**The format is PRE-RELEASE.** Shapes may change while the flag is off by default. +Do not build a stored artifact that assumes a field will keep its name or its +presence rule until the flag ships on. diff --git a/docs/reference/ledger/crew-types.md b/docs/reference/ledger/crew-types.md new file mode 100644 index 000000000000..62eb072b0817 --- /dev/null +++ b/docs/reference/ledger/crew-types.md @@ -0,0 +1,149 @@ +# Crew ledger entry types + +**Local page, not a mirror.** Part of the [ledger reference](README.md), which is +marked as a named exception in [the Reference index](../README.md). + +> **No crew emitter exists.** The families and the two contracts on this page are +> the specified shape a crew writer must produce. The `crew/dispatch` and +> `crew/report` contracts are defined here and in +> [`ledger-core.md`](../../system-specs/modules/ledger-core.md); a validator that +> refuses a malformed one is pending. Treat every field rule below as the +> agreement, not as something the code enforces today. + +A crew ledger uses the same file layout, the same header rules and the same eight +envelope fields as a session ledger — [envelope.md](envelope.md) covers all of +that. Three things differ: which `type` domains may be written, which `src` values +may write them, and the fact that `thread` and `ref` are actually used. + +## The layering rule + +There is one base envelope, then two disjoint sets of types on top of it. A type +belongs to exactly one kind, and the kind of the ledger decides whether a type may +be written to it at all. Writing a crew-owned type into a session ledger is refused +with [`event_type_not_owned`](errors.md#event_type_not_owned), and the reverse is +refused the same way. + +The crew kind owns eight `type` domains: `member`, `activity`, `slot`, `patrol`, +`message`, `crew`, `item`, `memory`. + +Two overlaps between the kinds are deliberate rather than accidental. The +`message` domain exists in **both** kinds, carrying different `data` in each: +ownership answers "does this kind have such events", and a crew and a session both +do. And `ref` **crosses** kinds — a crew entry citing a span of a session's file is +the one intended cross-kind pointer. + +A type name never carries the writer's identity. Who wrote a line is `src`, not +part of `type`, so a dispatch written by one crew and a dispatch written by another +are the same type and a reader folds them together. The `app:/` guest +form is reserved for facts an app defines that no built-in domain covers. + +## Allowed `src` + +A crew entry is written by one of: + +| `src` | Who it is | +|---|---| +| `dashboard` | The dashboard, acting for the person using it. | +| `patrol` | A patrol pass. | +| `gateway` | The gateway itself. | +| `crew:` | A named crew, writing into its own crew ledger. | +| `app:` | A named app, writing only its own `app:/…` types, and only into a crew ledger. | + +`dashboard` and `patrol` are crew-side values: a session entry is written with +`gateway` or `acp` and nothing else. + +A guest's own name is its write permission. A `crew:` writer may write +crew-owned types into that crew's ledger; an `app:` writer is confined to its +own namespace. Writing outside it is +[`namespace_violation`](errors.md#namespace_violation), and an unparseable `src` is +[`bad_src`](errors.md#bad_src). + +## Families + +From the RFC's event-family model. Each row names the shape of a family rather than +an exhaustive type list, because a new action under an owned domain needs no +registry change. + +| Family | Types | What it points at | +|---|---|---| +| shipped | `member/*`, `activity/record`, `slot/*`, `patrol/*` | `slot/*` cites the session it shipped. | +| messages | `message/received`, `message/sent` | The redacted body is in the ledger; a permalink reply cites its anchor with `ref`. | +| tree | `crew/child-attached`, `crew/parent-attached`, and their `-detached` counterparts | — | +| dispatch | `crew/dispatch`, `crew/report` | A report cites the child's segment. | +| topics | `crew/topic-*`, `crew/forwarded`, `crew/run-state` | A topic cites its work session. | +| items | `item/phase`, `item/next`, `item/probe`, `item/verdict`, `crew/round-*` | A probe and a verdict cite their evidence. | +| knowledge | `crew/finding`, `crew/summary`, `crew/note-*`, `crew/link` | The segment covered. | +| memory | `memory/bound`, `memory/copied`, `memory/forgotten`, `memory/restored` | — | + +The chat surface is a fold over the `messages` family rather than a separate store. +A direct message is the entries with no `thread`; a thread page is the entries whose +`thread` is that anchor; a reply that quotes a line is a `message/sent` whose `ref` +names it. + +## `crew/dispatch` + +A crew handed one work item to a target. + +**Kind and `src`** — `crew`; written by the dispatching party, so `src` is +`dashboard`, `patrol`, `gateway` or `crew:`. + +**Pairing** — Opener. Answered by one or more [`crew/report`](#crewreport) entries +that thread onto its `seq`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `item` | string | required | The work item's id. | | +| `target` | object | required | Who the item went to. **A dispatch without a target is invalid.** | | +| `target.kind` | string | required | Which sort of target. | `session`, `crew` | +| `target.slot` | string | conditional | The slot key, required when `target.kind` is `session`. | | +| `target.name` | string | conditional | The crew name, required when `target.kind` is `crew`. | | +| `brief` | string | optional | The brief handed over. | | + +**Invariants** — `target` is required and its `kind` decides which of `slot` or +`name` must be present. The two forms are exclusive: a target names a session slot +or a crew, never both. + +```json +{"type":"crew/dispatch","seq":41,"time":1789000003000,"src":"crew:qa","data":{"item":"WI-4","target":{"kind":"session","slot":"dashboard:7"},"brief":"write the reference pages"}} +``` + +**Reader hint** — Group a work item's history by `item`, then order it by `seq`. The +dispatch is the anchor every report threads onto. + +## `crew/report` + +A dispatched party reported back on one work item. + +**Kind and `src`** — `crew`; written by the reporting party, so `src` is +`crew:` for a crew and `gateway` for a session's own report. + +**Pairing** — Closer in practice, though a `progress` status leaves the item open. +`thread` is the dispatch's `seq`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `item` | string | required | The work item's id, matching the dispatch. | | +| `status` | string | required | Where the item stands. | `done`, `blocked`, `failed`, `progress` | +| `credits` | number | optional | What the work cost. | | +| `summary` | string | optional | What was done. | | + +**Envelope requirements** — Unlike every other type documented here, `crew/report` +constrains the envelope as well as `data`: + +- **`ref` is required.** It cites the segment of the child's ledger holding the + work, so a report is never a claim without evidence. The span cap of 500 lines + (`MAX_REF_SPAN`) applies, so a long run is cited by its relevant span rather than + in full. +- **`thread`** is the `seq` of the `crew/dispatch` being answered. + +**Invariants** — A report without a `ref` is invalid. `status: "progress"` may +appear several times for one dispatch; a terminal status appears once. + +```json +{"type":"crew/report","seq":58,"time":1789000004000,"src":"crew:qa","thread":41,"ref":{"unit":"session","id":"s-7f3a","from":12,"to":40},"data":{"item":"WI-4","status":"done","credits":1.42,"summary":"six pages and a test"}} +``` + +**Reader hint** — Resolve the `ref` to read the work itself, and handle all four +statuses the resolution can answer with — the cited ledger may be gone or pruned +long after the report was written. See +[reading-and-writing.md](reading-and-writing.md#resolving-a-ref). diff --git a/docs/reference/ledger/envelope.md b/docs/reference/ledger/envelope.md new file mode 100644 index 000000000000..41b047955e50 --- /dev/null +++ b/docs/reference/ledger/envelope.md @@ -0,0 +1,164 @@ +# Envelope + +**Local page, not a mirror.** Part of the [ledger reference](README.md), which is +marked as a named exception in [the Reference index](../README.md). + +Everything here is kind-independent: it holds for a session ledger and a crew +ledger alike. What differs between the two kinds is which `type` domains and which +`src` values are allowed, which [session-types.md](session-types.md) and +[crew-types.md](crew-types.md) cover. + +## File layout + +``` +/ledgers/crews//ledger.jsonl +/ledgers/sessions//ledger.jsonl +``` + +`ledgers` is one shared root for both kinds, which is what lets a single fence +entry cover every unit kind at once. Under it, `crews/` holds crew units and +`sessions/` holds session units. + +`` is not the raw unit id. It is a readable-prefix-plus-digest fold of +it (`session_ledger._store_name`), because a unit id may legitimately carry a colon +— a channel session key looks like `slack:1712793600.123` — and a raw id must never +decide a directory name. The raw id is recorded in the header, and the path is +re-checked for containment symlink-safely when it is built. + +Each unit directory holds: + +| Path | What it is | +|---|---| +| `ledger.jsonl` | The segment whose first entry is `seq` 1. The only segment a writer creates. | +| `ledger..jsonl` | A later segment. The filename's middle is its first entry's `seq`, all digits and greater than 1. | +| `.lock` | The append lock. Held only for the duration of one append, and it is what makes `seq` assignment serial. | +| `.lease` | The write-ownership lease. Held for as long as a handle can still append. See [reading-and-writing.md](reading-and-writing.md#ownership). | + +Line 1 of a segment is a header object. Every later line is an entry. **Every +segment repeats the header**, not just the first one, so a unit whose oldest +segment is gone is still a readable ledger. A reader validates each segment's +header against the unit it opened and checks that the filename's declared first +`seq` matches the segment's first readable entry. + +Reading crosses segments oldest first. A `seq` gap *between* two surviving segments +is damage ([`segment_gap`](errors.md#segment_gap)); a first segment that starts +above 1 is a pruned front and is read without complaint. + +## Header fields + +`type` on the header line is the kind, and `createdAt` is epoch milliseconds. +`version` is the header schema version, `1`. + +Common to both kinds: + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `type` | string | required | The unit kind this file belongs to. | `crew`, `session` | +| `version` | int | required | Header schema version. A file declaring a version this build does not read is refused with [`unsupported_version`](errors.md#unsupported_version). | `1` | +| `id` | string | required | The raw unit id. Must match the id the reader opened. | | +| `createdAt` | int | required | Epoch milliseconds the unit's ledger was created. | | + +A **crew** header carries nothing beyond those four: a crew's identity is its id, +and every fact about it is an entry rather than a header field. + +A **session** header adds the facts fixed for the session's whole life: + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `owner` | string | required | The session's owner. | | +| `agent` | string | required | The agent the session runs. | | +| `task` | string \| null | optional | The task the session was opened for. | | +| `pack` | string \| null | optional | The pack the session belongs to. | | +| `slot` | string \| null | optional | The slot key serving the session. | | +| `thread` | object \| null | optional | Back-pointer to the crew row that dispatched this session: `{crew, seq}`. | | +| `cwd` | string \| null | optional | Working directory. | | +| `remote` | object \| null | optional | Remote placement facts. | | + +The header's `thread` is the one place `thread` is an object rather than an int. +Its two wire fields are `crew` (the crew unit's id, a string) and `seq` (the +dispatching entry's `seq`, a positive int). A malformed `thread` is refused when a +header is built ([`bad_header_field`](errors.md#bad_header_field)) but folded to +absent when one is read, so a damaged back-pointer costs the pointer and not the +file. + +Building a header refuses an unknown key. Reading one ignores unknown keys, which +is what lets an older build read a file a newer writer extended. + +### Header example + +```json +{"type":"session","version":1,"id":"s-7f3a","owner":"default","task":null,"pack":null,"agent":"kirocrew","slot":"dashboard:3","thread":{"crew":"qa","seq":41},"cwd":"/home/u/proj","remote":null,"createdAt":1789000000000} +``` + +## Entry fields + +Eight fields, four always written and four written only when set. Field order on +the wire is fixed: the four required fields, then `thread`, `ref`, `ignorable` when +present, then `data` last. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `type` | string | required | `domain/action`, partitioned on the first slash. The action may not itself contain a slash, so a namespace is exactly one level deep. A guest form `app:/` exists for crew ledgers. | | +| `seq` | int ≥ 1 | required | Writer-assigned under the unit's lock, contiguous from 1 after the header. | | +| `time` | int | required | Writer-assigned epoch milliseconds. | | +| `src` | string | required | Which emitter wrote the line. | `gateway`, `acp`, `dashboard`, `patrol`, or `session:` / `crew:` / `app:` | +| `thread` | int ≥ 1 | optional | The `seq` of an earlier entry in this same file, used as a grouping key. Refused if it names no earlier readable entry ([`bad_thread`](errors.md#bad_thread)). Session emitters never set it. | | +| `ref` | object | optional | A citation into another or the same ledger. | | +| `ignorable` | `true` | optional | The writer's promise that a reader may skip this line when it does not know the type. Only a literal `true` is written; the key is absent otherwise. | `true` | +| `data` | object | required | The type-specific payload. Must be a JSON object holding JSON-serializable values ([`bad_data`](errors.md#bad_data)). | | + +`src` names either a whole subsystem carrying no instance id (`gateway`, `acp`, +`dashboard`, `patrol`) or one instance (`session:`, `crew:`, +`app:`). Session entries are written with `gateway` or `acp` and nothing +else; the values a crew entry may carry are on +[crew-types.md](crew-types.md#allowed-src). + +`ignorable` is what lets an older reader keep folding a newer writer's file. It is +a claim about *dependency*, not importance: setting it says nothing later in the +file needs this entry to have been interpreted. A sampled body slice qualifies; a +closer never does. + +The serialized line ceiling is 64 KiB (`MAX_ENTRY_BYTES`). A line over it is +refused with [`entry_too_large`](errors.md#entry_too_large) before any byte is +written. + +### Entry example + +```json +{"type":"turn/started","seq":12,"time":1789000000200,"src":"gateway","data":{"turn":3,"actor":"user","depth":0}} +``` + +## `ref` + +A `ref` is a citation, not a copy: the bytes stay in the ledger they were written +to, and a reader resolves the pointer when it wants them. + +```json +{"unit":"session","id":"s-7f3a","from":12,"to":40} +``` + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `unit` | string | required | The kind of ledger being cited. | `crew`, `session` | +| `id` | string | required | The cited unit's raw id. | | +| `from` | int ≥ 1 | required | First cited `seq`. | | +| `to` | int ≥ `from` | optional | Last cited `seq`. Absent cites the single line at `from`. | | + +The span `to - from + 1` may be at most 500 lines (`MAX_REF_SPAN`). Every shape +failure — an unknown unit, an unusable id, a non-positive or inverted bound, an +over-wide span — is [`bad_ref`](errors.md#bad_ref), raised when the ref is +constructed rather than when it is resolved. + +Resolving a ref answers with a status rather than an exception, because a cited +ledger may legitimately be gone or pruned. The four statuses are on +[reading-and-writing.md](reading-and-writing.md#resolving-a-ref). + +## `thread` + +`thread` groups entries inside one file: it holds the `seq` of an earlier entry in +the same ledger, and a reader pages a group by that anchor. It is validated +against the file at append time, so it can never name a line that does not exist +or one that comes later. + +Session entries never carry it. It is the crew ledger's shape, where a report +threads onto the dispatch it answers. diff --git a/docs/reference/ledger/errors.md b/docs/reference/ledger/errors.md new file mode 100644 index 000000000000..ecb6581900ce --- /dev/null +++ b/docs/reference/ledger/errors.md @@ -0,0 +1,262 @@ +# Errors + +**Local page, not a mirror.** Part of the [ledger reference](README.md), which is +marked as a named exception in [the Reference index](../README.md). + +Every refused ledger operation raises `LedgerError` carrying a stable `code`, and +`field` naming the offending input when one field is to blame. A caller branches on +`code`; the human-readable message is free to change without breaking it. + +**Codes are additive-only.** A shipped code string is never renamed or repurposed — +it is API surface, not a log string. A new refusal gets a new code. + +A `LedgerError` means the operation was **declined before any byte was written**. For +the one case where a write may have left residue, see +[`IndeterminateAppend`](#indeterminateappend). + +## Summary + +| Code | Path | Trigger | +|---|---|---| +| [`bad_kind`](#bad_kind) | both | Requested kind is neither `crew` nor `session`. | +| [`invalid_id`](#invalid_id) | both | Unit id is empty, holds a separator or NUL, or escapes its root. | +| [`bad_root`](#bad_root) | both | A kind's root under `ledgers` is a link or resolves outside the data home. | +| [`already_exists`](#already_exists) | create | `create` was asked for a ledger whose file is already there. | +| [`no_ledger`](#no_ledger) | open, append | No content-bearing segment exists. | +| [`already_owned`](#already_owned) | write | Another process owns writes to this unit. | +| [`bad_header`](#bad_header) | read | Line 1 is missing, unparseable, or names a different unit. | +| [`bad_header_field`](#bad_header_field) | create | A header field is missing, wrongly typed, or unknown. | +| [`unsupported_version`](#unsupported_version) | read | The header names a format version this build does not read. | +| [`bad_type`](#bad_type) | write | `type` is not spelled `domain/action`. | +| [`bad_src`](#bad_src) | write | `src` is neither a fixed emitter nor a well-formed namespaced one. | +| [`bad_data`](#bad_data) | write | `data` is not a JSON object, or holds something unserializable. | +| [`event_type_not_owned`](#event_type_not_owned) | write | This kind does not own that `type` domain. | +| [`namespace_violation`](#namespace_violation) | write | A guest wrote outside its own namespace. | +| [`bad_thread`](#bad_thread) | write | `thread` names no existing earlier entry in this file. | +| [`bad_ref`](#bad_ref) | write | `ref` is malformed or spans too many lines. | +| [`entry_too_large`](#entry_too_large) | write | The serialized line is over the size ceiling. | +| [`unknown_entry_type`](#unknown_entry_type) | read | A declaring reader met a type it does not know that is not `ignorable`. | +| [`bad_segment`](#bad_segment) | read | A segment's header or declared first `seq` does not match. | +| [`segment_gap`](#segment_gap) | read | `seq` is not contiguous across a segment boundary. | + +## Identity and location + +### `bad_kind` + +The requested kind is neither `crew` nor `session`. + +**Caller action** — A programming error. Fix the call; there is no runtime recovery. + +### `invalid_id` + +The unit id is empty, carries a path separator or a NUL, or resolves outside its own +root. + +A unit id names a directory, so a separator in it would let the ledger escape its +root. The raw id is refused loudly rather than folded to something safe: a fold would +silently merge two units into one file, and two units sharing a ledger is worse than +a refused write. + +**Caller action** — Refuse the operation upstream. Do not sanitize and retry. + +### `bad_root` + +A kind's root directory under `ledgers` is a link, or resolves outside the data +home's own ledger tree. The message names the directory. + +Containment resolves its base first, so a linked kind root would make the link's +target the containment root and every path under it would pass — private history +readable and forgeable at a location the data home's protections do not cover. + +**Caller action** — Operational, not programmatic. Surface it for an operator to +inspect what the directory points at. Do not create or replace it automatically. + +## Lifecycle + +### `already_exists` + +`create` was asked for a ledger whose file is already there. Field: `id`. + +**Caller action** — Call `open` instead. The check runs twice, the second time under +the lock, so this code also means a racing create won. + +### `no_ledger` + +`open` was asked for a ledger with no content-bearing segment. Field: `id`. + +**Caller action** — Call `create`, or treat the unit as having no history. When it +comes from resolving a `ref`, it surfaces as the `gone` status rather than as this +error. + +### `already_owned` + +Another process owns writes to this unit's ledger, so this one appended nothing. + +Distinct from every shape code: the entry was well formed and the file is healthy, +this process is simply not the writer. + +**Caller action** — **Report the loss; do not retry.** Ownership is held for the life +of the owning process, so every later entry for that unit would queue behind the same +wait. In the session emitter this is one of the paths that ends in a +[`write/dropped`](session-types.md#writedropped) marker. + +## Header + +### `bad_header` + +Line 1 is missing, unparseable, or describes a different unit than the one asked +for. A ledger without a readable header is not a ledger. Also raised for a non-integer +`createdAt` or `version`. + +**Caller action** — Treat the file as damaged. Quarantine it rather than appending +to it. + +### `bad_header_field` + +A header field is missing, of the wrong type, or unknown. Building a header refuses +an unknown key; reading one ignores unknown keys, so this is a write-path code. + +**Caller action** — Fix the call. A malformed session `thread` raises this when a +header is built, but is folded to absent when one is read — a damaged back-pointer +costs the pointer, not the file. + +### `unsupported_version` + +The header names a format version this build does not read. + +Kept distinct from `bad_header` on purpose: the file is not damaged, this process is +old. A newer version may legitimately fail every structural check, so the caller has +to be told to upgrade rather than told the log is corrupt. + +**Caller action** — Report that an upgrade is needed. Do not truncate, repair or +append. + +## Entry shape + +### `bad_type` + +`type` is not spelled `domain/action`. Field: `type`. + +The split is on the first slash, and the action may not contain one, so a namespace +is exactly one level deep and no action can smuggle a second domain behind it. + +**Caller action** — Fix the call. + +### `bad_src` + +`src` is neither one of the fixed emitters nor a well-formed namespaced one. Field: +`src`. + +**Caller action** — Fix the call. The values each kind accepts are on +[envelope.md](envelope.md#entry-fields) and +[crew-types.md](crew-types.md#allowed-src). + +### `bad_data` + +`data` is not a JSON object, or holds something not JSON-serializable. Also raised on +field `cite` when an `append_many` citation callback returns something other than a +dict. + +**Caller action** — Fix the call. This is the code a per-type payload validator +reports through, so it may also mean a payload that is a valid object but does not +match its type's contract. + +### `event_type_not_owned` + +This ledger kind does not own that `type` domain. The message names the domains the +kind does own. + +**Caller action** — Write to the right kind of ledger, or use the right type. See +[crew-types.md](crew-types.md#the-layering-rule). + +### `namespace_violation` + +A guest emitter wrote outside its own namespace, or wrote a guest type into a session +ledger. + +**Caller action** — A guest's own name is its permission. Write `app:/…` types +as `app:`, into a crew ledger. + +### `bad_thread` + +`thread` does not name an existing, earlier `seq` in this same ledger — it is past +the newest entry, or names nothing readable. Field: `thread`. + +**Caller action** — Anchor on a `seq` this ledger returned. `thread` is never a +pointer into another unit; that is `ref`. + +### `bad_ref` + +`ref` is malformed: a bad unit, a bad id, a non-positive bound, an inverted bound, or +a span wider than 500 lines (`MAX_REF_SPAN`). Fields: `unit`, `ref.id`, `ref.from`, +`ref.to`. + +Raised when the ref is **constructed**, not when it is resolved, so an unresolvable +ref and an unrepresentable one are different problems. + +**Caller action** — Cite a narrower span. A citation is a pointer, not a bulk export: +an unbounded span would let one entry make its reader materialize a whole history. + +### `entry_too_large` + +The serialized entry line is over the 64 KiB ceiling (`MAX_ENTRY_BYTES`). + +**Caller action** — Split the payload. A body is split into +[`message/chunk`](session-types.md#messagechunk) entries by the emitter; the line is +refused, never clipped, so a reader never meets a half-recorded fact. + +## Read path + +### `unknown_entry_type` + +A reader that declared the types it understands met one it does not, and the entry +did not mark itself `ignorable`. Field: `type`. Raised on the **read** path only. + +An entry a reader cannot interpret may change the meaning of every entry after it, so +reconstruction stops rather than silently skipping it. + +**Caller action** — Either teach the reader the type, or pass no `known` set if the +reader does not interpret entries at all. Do not catch this and continue folding: the +state you build past it is not trustworthy. + +### `bad_segment` + +A segment's header does not belong to the ledger being read, its schema version +disagrees, or its filename's declared first `seq` disagrees with its readable physical +first entry. The message names the offending segment. + +**Caller action** — Quarantine the named segment. When it comes from resolving a +`ref`, it surfaces as the `corrupt` status instead of being raised. + +### `segment_gap` + +`seq` is not contiguous across a segment boundary: the segment after the gap does not +start where the one before it ended. + +Retention removes whole segments off the **front**, which leaves no gap between the +ones that remain — so a gap here is damage or a partial copy, not a pruned log. + +**Caller action** — Treat the chain as damaged. A missing *front* is not this error; +it reads normally and resolves as `pruned`. + +## Not a `LedgerError` + +### `IndeterminateAppend` + +An append failed **and** could not be rolled back. + +The ordinary failure is definite: bytes reach the file before the fsync runs, so an +append that fails anywhere truncates the file back to the size it had and re-raises, +leaving nothing behind for a retry to reason about. This is the case where that +cleanup also failed — likely, since whatever broke the write is often still broken — +so the file may hold bytes no entry claims. + +It carries `written` and `offset` rather than a `code`, and it is deliberately not a +`LedgerError`: a `LedgerError` is a refusal, meaning nothing was written and retrying +is pointless. Here the file has been touched and the outcome is unknown, which is the +opposite. + +**Caller action** — Do not retry the entry blind. The residue is an unterminated or +unparseable tail, which is the shape the next `open` truncates, so the recovery +already exists; the distinct type is there so a caller can tell "nothing happened" +from "something may have". diff --git a/docs/reference/ledger/reading-and-writing.md b/docs/reference/ledger/reading-and-writing.md new file mode 100644 index 000000000000..85f4f8c8359c --- /dev/null +++ b/docs/reference/ledger/reading-and-writing.md @@ -0,0 +1,270 @@ +# Reading and writing + +**Local page, not a mirror.** Part of the [ledger reference](README.md), which is +marked as a named exception in [the Reference index](../README.md). + +Every name here is from `kiro_crew.ledger`. Each refusal names a code from +[errors.md](errors.md). + +## Opening and creating + +`Ledger` is the handle. Both entry points are classmethods. + +The subagent-aware repair on this page -- the `child_gone` predicate on `Ledger.open` +and `repair_interrupted_turn`, and the `approval/decided` and `subagent/failed` +closers -- lands with #11185. On a build without it, `open` and +`repair_interrupted_turn` take no `child_gone` argument and repair writes the +`tool/completed` and `turn/completed` closers only. + +| Call | Signature | Refuses with | +|---|---|---| +| `Ledger.exists` | `(kind, unit_id) -> bool` | `bad_kind`, `invalid_id`, `bad_root` | +| `Ledger.create` | `(kind, unit_id, **header_fields) -> Ledger` | `bad_kind`, `invalid_id`, `bad_root`, `already_exists`, `bad_header_field`, `bad_data`, `entry_too_large` | +| `Ledger.open` | `(kind, unit_id, *, repair=False, child_gone=None) -> Ledger` | `bad_kind`, `invalid_id`, `bad_root`, `no_ledger`, `bad_header`, `unsupported_version`, `already_owned` | + +All three resolve the unit's path before doing their own work, so all three can refuse +`bad_kind`, `invalid_id` and `bad_root` first; the codes after those are what each call +adds. `create` reaches `bad_data` and `entry_too_large` because it serializes a header +line, which every later append is measured against the same way. + +`exists` asks whether the unit has any segment carrying content; a zero-byte file +counts as absent. + +`create` writes the header atomically — temp file, fsync, rename — and refuses a +unit whose file is already there. The check runs twice, the second time under the +lock, so two racing creates cannot both win. The returned handle has `last_seq` 0. + +`open` reads the header from the oldest segment and positions on the newest one +carrying content. It **always** truncates a torn tail, whether or not repair is +asked for. With `repair=True` it additionally takes write ownership *before* +running closers, so it can raise `already_owned`; if anything then fails, the lease +is released rather than leaked. `child_gone` is a predicate the subagent closer +consults — see [crash repair](#crash-repair). + +`repair_interrupted_turn(*, child_gone=None) -> int` runs the same repair on a +handle already held and returns how many closers it wrote. + +Read-only helpers at module level: `ledger_root(kind)`, `ledger_dir(kind, unit_id)`, +`ledger_path(kind, unit_id)`, `segment_paths(kind, unit_id)`, +`segment_first_seqs(kind, unit_id)`, and `now_ms()`. Handle properties: `kind`, +`id`, `path`, `header`, `last_seq`. + +## Reading + +### `iter_from` + +`iter_from(seq=1, *, known=None) -> Iterator[Entry]` yields every entry from `seq` +onward, oldest first, across segments. + +`known` is the whole point of the call. It declares the types this reader +understands, and it changes what happens when the reader meets one it does not: + +- `known=None` — yield everything. No type is refused. This is the right choice for + a tool that copies, counts or exports lines without interpreting them. +- `known={…}` — an entry whose type is not in the set is **skipped if and only if + it is marked `ignorable`**. Otherwise the iterator raises + [`unknown_entry_type`](errors.md#unknown_entry_type) and reconstruction stops. + +Stopping is deliberate. A required entry a reader cannot interpret may change the +meaning of every entry after it, so a reader that folds state must not be handed a +partial history that looks complete. `ignorable` is the writer's promise that +nothing later depends on this line having been read, which is what lets an older +reader keep folding a file a newer writer extended. + +Crossing segments can also refuse: [`bad_segment`](errors.md#bad_segment) for a +segment whose header or declared first `seq` does not match, and +[`segment_gap`](errors.md#segment_gap) for a gap between two surviving segments. A +first segment starting above 1 is a pruned front and is read without complaint. + +### `get`, `page`, `thread_page` + +`get(seq) -> Entry | None` returns the entry at one `seq`, or `None`. It stops once +the stream passes `seq`, so it does not read the whole file. + +`page(before=None, limit=DEFAULT_PAGE_LIMIT) -> Page` returns up to `limit` entries +**older** than `before`, exclusive, newest first — the shape a UI scrolling +backwards wants. `limit` is clamped to `[1, MAX_PAGE_LIMIT]`; the default is 50 +(`DEFAULT_PAGE_LIMIT`) and the cap is 500 (`MAX_PAGE_LIMIT`). There is no `known` gate +on this path: paging is for display, not +for folding state. + +`thread_page(anchor, before=None, limit=DEFAULT_PAGE_LIMIT) -> Page` returns one +thread — the entries whose `thread` is `anchor`, plus the anchor itself — newest +first with the anchor last. + +### Resolving a `ref` + +`resolve(ref) -> Resolution` follows a citation. A same-unit ref resolves against +the open handle; anything else opens the cited unit. The result is a `Resolution` +carrying a `status` and a tuple of `entries`, with `.ok` true only for `ok`. + +It answers with a status rather than raising, because a cited ledger being gone is +a normal outcome of a long-lived pointer, not a bug in the reader: + +| Status | Meaning | +|---|---| +| `ok` | Every cited `seq` was read back: the covered seqs are distinct and their count matches the span. | +| `gone` | The cited ledger does not exist at all. | +| `pruned` | The span reaches below the oldest surviving segment's first `seq`, and what survives is intact. | +| `corrupt` | The ledger exists but will not open, or the read was short, or a seq repeated, or the span reaches past the newest entry. A damaged segment chain is reported here rather than raised. | + +Two things `resolve` does **not** do. It makes no authorization decision and takes +no access callback, so there is no `forbidden` status: a caller that needs a +permission model applies it around this call. And it does not enforce the span cap — +that is checked when the `ref` is constructed, raising +[`bad_ref`](errors.md#bad_ref) for a span wider than 500 lines. + +## Writing + +### Type ownership + +A kind owns a set of `type` domains, matched by prefix so a new action under an +owned domain needs no registration. Writing a type the kind does not own is +[`event_type_not_owned`](errors.md#event_type_not_owned). The session kind's +domains are on [session-types.md](session-types.md); the crew kind's are on +[crew-types.md](crew-types.md). + +### Guest namespacing + +A guest writer is one whose `src` carries its own name — `crew:` or +`app:`. That name is its permission: an `app:` writer may write only +`app:/…` types, and only into a crew ledger. Writing outside its own +namespace is [`namespace_violation`](errors.md#namespace_violation). An `src` that +is neither a known fixed emitter nor a well-formed namespaced one is +[`bad_src`](errors.md#bad_src). + +### Ownership + +One writer owns a unit's ledger at a time, held as an advisory lock on the unit's +`.lease` file. + +It is taken **non-blocking**: a writer either owns the file or is told it does not, +and never queues. Within one process the lease is refcounted by the lease file's +path, so several handles on the same unit share it and are not in contention with +each other; the last release closes the descriptor. After locking, the held inode is +compared with the file now at that path, retried a small fixed number of times, so a +lease file replaced underneath a writer is reported as contention rather than +silently trusted. + +There is **no expiry**. Ownership lasts as long as the handle can still append — it +is bound to the handle's lifetime by a finalizer — which means a wedged but live +writer keeps ownership until its process exits. That is the intended trade: a +timeout would let a second writer start appending to a file the first one may still +be writing. + +When another **process** owns the ledger, an append raises +[`already_owned`](errors.md#already_owned) and writes nothing. This is a hard +refusal, not a retry hint: ownership is held for the life of the owning process, so +every later entry for that unit would queue behind the same wait. A caller reports +the loss instead. + +### `append` + +`append(type, data, *, src, thread=None, ref=None, ignorable=False) -> Entry`. + +Validation happens **before any byte is written**, in this order: `data` must be a +JSON object of serializable values ([`bad_data`](errors.md#bad_data)); the type must +be well formed and owned, and the `src` acceptable +([`bad_type`](errors.md#bad_type), [`bad_src`](errors.md#bad_src), +[`event_type_not_owned`](errors.md#event_type_not_owned), +[`namespace_violation`](errors.md#namespace_violation)); a `thread` must name an +existing, earlier, readable entry in this same file +([`bad_thread`](errors.md#bad_thread)); the serialized line must fit the ceiling +([`entry_too_large`](errors.md#entry_too_large)). + +Then ownership is claimed and the tail is re-scanned under the unit's `.lock`. +`seq` is assigned as `last_seq + 1` where `last_seq` is read back from the file's +own tail window on **every** write, never trusted from the cached property. `time` +is assigned by the writer. So a refusal always means nothing happened, and a return +always means the line is on disk and fsynced. + +### `append_many` + +`append_many(items, *, src, cite=None) -> list[Entry]` writes a group in one write +and one fsync. An empty list returns an empty list. Every item's `data` and the +ownership are validated up front, and each serialized line must fit the ceiling +individually. + +`cite` is what makes an oversize body atomic. It is called **inside the lock** with +the seqs just allocated, and whatever dict it returns is appended last — so the +citing entry can name the seqs of the chunks it cites, and a reader either sees the +whole group or none of it. A `cite` that returns something other than a dict is +[`bad_data`](errors.md#bad_data) on field `cite`. + +This path does not accept `thread`. + +### When an append cannot be rolled back + +An ordinary failure is definite: bytes reach the file before the fsync, so a failed +append truncates the file back to the size it had and re-raises, leaving nothing for +a retry to reason about. + +`IndeterminateAppend` is the case where that cleanup **also** failed, which is +likely, since whatever broke the write is often still broken. The file may hold +bytes no entry claims. It is deliberately not a `LedgerError`: a `LedgerError` means +the entry was declined before any byte was written, and this is the opposite. The +residue is an unterminated or unparseable tail, which is exactly the shape the next +`open` truncates, so the recovery already exists — the distinct type is so a caller +can tell "nothing happened" from "something may have". + +### The fail-soft emitter + +The storage layer refuses. It does not retry, and it does not decide what a lost +write means. + +That policy lives in the session emitter above it, which is where the write-behind, +the loss accounting and the [`write/dropped`](session-types.md#writedropped) marker +belong. The emitter's contract is that a session is never slowed or failed by its +own ledger: a write it cannot land is dropped and accounted for, and the count and +byte total surface as one `write/dropped` entry at the head of the next batch. +Further loss merges into the same pending marker rather than adding entries. Because +a lost job never took a `seq`, the loss leaves `seq` contiguous — which is why +`write/dropped` is the only thing that can tell a reader the record is incomplete. + +## Crash repair + +Repair is deterministic: the same file always repairs to the same bytes. + +**Torn tail — every `open` and every `append`.** Trailing unterminated bytes are +classified. Bytes that do not parse are dropped. Bytes that parse whole are kept and +the next append re-supplies the missing newline. An interior line that is +newline-terminated but unparseable is left on disk and skipped on read, because +truncating at it would discard everything after it. + +**Interrupted-turn closers — session ledgers only, and only with `repair=True`.** +One closer per still-open opener, written in this order with a fixed value per type: + +| Open opener | Closer written | Fixed value | +|---|---|---| +| `approval/requested` | `approval/decided` | `decision: "unknown"` (#11185) | +| `tool/called` | `tool/completed` | `status: "unknown"` | +| `subagent/spawned` | `subagent/failed` | `outcome: "unknown"` (#11185) | +| `turn/started` | `turn/completed` | `stop_reason: "interrupted"` | + +Every closer is written with `src` `gateway` and reuses the **last real entry's +`time`** rather than the current clock, so repair does not invent a timestamp for +something that happened before the crash. Seq continues from the tail. + +Two guards keep repair from writing over uncertainty. A trailing orphan +`message/chunk` run whose citing entry never landed is truncated *before* any closer +is written. And if the fold that decides what is open skipped any record at all — +blank, malformed, or unreadable — repair writes **nothing** and reports zero +closers, rather than closing openers based on a history it could not fully read. + +`subagent/spawned` is the one opener repair will leave open. It is closed only for a +child the `child_gone` predicate reports finished; with no predicate supplied the +spawn stays open, because a running child must not be recorded as failed. + +## Segments and retention + +The read side is built for many segments: `segment_paths` lists them ascending by +first `seq`, each is validated against the unit it belongs to, and `seq` contiguity +is checked across every boundary. A pruned front — an oldest segment starting above +1 — is a supported state that `iter_from` reads and that `resolve` reports as +`pruned`. + +The write side creates one segment. No code in `kiro_crew.ledger` rolls a new +segment or prunes an old one, so `ledger.jsonl` is the only segment a writer +produces. Retention removes whole segments off the **front**, which is why a front +gap is legal and a mid-chain gap is +[`segment_gap`](errors.md#segment_gap). diff --git a/docs/reference/ledger/session-types.md b/docs/reference/ledger/session-types.md new file mode 100644 index 000000000000..669f1247ccde --- /dev/null +++ b/docs/reference/ledger/session-types.md @@ -0,0 +1,912 @@ +# Session ledger entry types + +**Local page, not a mirror.** Part of the [ledger reference](README.md), which is +marked as a named exception in [the Reference index](../README.md). + +Twenty-six types. Read [envelope.md](envelope.md) first for the fields every entry +carries; this page covers only each type's `data`. + +Session entries are written with `src` `gateway` or `acp` and nothing else. They +never carry `thread`, and no session emitter sets `ref`. A repair closer is the one +writer that can put `gateway` on a type whose ordinary emitter uses `acp`, and it +reuses the last real entry's `time` rather than the clock. + +Every subsection below follows one template: summary, kind and `src`, when written, +pairing, fields, invariants, example, reader hint, since. + +## Summary + +The **Emitter** column says which build writes the type. `live` means an emitter +writes it today. `#11185` means the type and its shape are settled and its emitter +lands with that pull request: on a build without it the entry is never written, so a +reader needs no handling for it yet, and each such subsection says the same thing in +its **Since** line. A type this kind owns with no emitter anywhere is under +[Removed types](#removed-types) instead of here. + +| Type | One line | Emitter | `src` | Pairing | +|---|---|---|---|---| +| [`session/opened`](#sessionopened) | The ledger was created, or a claim re-attached to it. | live | `gateway` | — | +| [`session/closed`](#sessionclosed) | The gateway stopped serving this session. | live | `gateway` | — | +| [`turn/started`](#turnstarted) | A turn was authorized and is about to run. | live | `gateway` | opener of `turn/completed` | +| [`turn/refused`](#turnrefused) | A gate refused to run a dispatched turn. | live | `gateway` | terminal on its own | +| [`turn/completed`](#turncompleted) | A turn ended; its outcome and cost. | live | `acp`, `gateway` | closer, written last | +| [`write/dropped`](#writedropped) | Writer losses, accounted for once. | live | `gateway` | — | +| [`message/received`](#messagereceived) | The body of a message accepted into the session. | live | `gateway` | cites `message/chunk` | +| [`message/sent`](#messagesent) | A finished assistant message. | live | `acp` | cites `message/chunk` | +| [`message/chunk`](#messagechunk) | One slice of an oversize body. | live (overflow only) | inherited | cited by its body entry | +| [`message/queued`](#messagequeued) | A message arrived while a turn was running. | live | `gateway` | — | +| [`request/configured`](#requestconfigured) | The request configuration, when it changed. | live | `gateway` | — | +| [`context/composed`](#contextcomposed) | What was put in front of the model, block by block. | live | `gateway` | — | +| [`step/started`](#stepstarted) | Opens one model call inside a turn. | live | `gateway` | opener of `step/completed` | +| [`step/completed`](#stepcompleted) | Closes one model call. | live | `gateway` | closer | +| [`tool/called`](#toolcalled) | A tool call, arguments digested. | live | `acp` | opener of `tool/completed` | +| [`tool/completed`](#toolcompleted) | A tool call's terminal frame. | live | `acp`, `gateway` | closer, by `call_id` | +| [`approval/requested`](#approvalrequested) | A tool call is waiting on a human. | #11185 | `gateway` | opener of `approval/decided` | +| [`approval/decided`](#approvaldecided) | How an approval resolved. | #11185 | `gateway` | closer, by `approval_id` | +| [`model/selected`](#modelselected) | A model swap, and why. | live | `gateway` | — | +| [`compaction/applied`](#compactionapplied) | A compaction, as context-usage percentages. | live | `gateway` | — | +| [`plan/updated`](#planupdated) | The session's task list, as just restated. | #11185 | `acp` | — | +| [`background/completed`](#backgroundcompleted) | A model call made on the session's behalf. | #11185 | `gateway` | — | +| [`subagent/spawned`](#subagentspawned) | A child this session dispatched. | #11185 | `gateway` | opener | +| [`subagent/steered`](#subagentsteered) | A correction sent into a running child. | #11185 | `gateway` | — | +| [`subagent/completed`](#subagentcompleted) | A child finished its work. | #11185 | `gateway` | closer, by `agent_id` | +| [`subagent/failed`](#subagentfailed) | A child did not finish its work. | #11185 | `gateway` | closer, by `agent_id` | + +## Session and turn + +### `session/opened` + +The ledger was created, or this claim re-attached to a conversation already on +disk. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — Once per session, on create or on re-attach. A warm reuse of a +handle the process already holds is silent. Not turn-scoped, so it carries no +`turn`. + +**Pairing** — None. It is not an opener: `session/closed` is a teardown marker for +the gateway's own serving, not a closer for this entry. On the resume path this +entry's write is the point the interrupted-turn repair runs. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `agent` | string | required | Agent name, defaulted to `kirocrew` when the caller names none. | | +| `slot` | string | required | Slot key. May be empty. | | +| `model` | string | required | Configured model. Empty when the backend serves its own default. | | +| `cwd` | string | required | Working directory. May be empty. | | +| `owner` | string | required | Owner, defaulted to `default`. | | +| `resumed` | bool | required | `true` when this claim re-attached to an existing ledger. | | + +**Invariants** — At most one per create and one per re-attach. The session's +*starting* model rides here rather than in a `model/selected` entry. + +```json +{"type":"session/opened","seq":1,"time":1789000000000,"src":"gateway","data":{"agent":"kirocrew","slot":"dashboard:3","model":"","cwd":"/home/u/proj","owner":"default","resumed":false}} +``` + +**Reader hint** — `resumed: true` means entries below this line belong to earlier +runs of the same conversation, so a reader building "this run" starts here rather +than at `seq` 1. + +**Since** — #10091. + +### `session/closed` + +The gateway stopped serving this session, for a stated reason. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — At teardown. Carries no `turn`. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `reason` | string | required | The gateway's own end reason, verbatim. | | + +**Invariants** — **Not the end of the file.** Entries from turns already in flight +may land after it. + +```json +{"type":"session/closed","seq":210,"time":1789000090000,"src":"gateway","data":{"reason":"reset"}} +``` + +**Reader hint** — Do not stop folding here, and do not treat a later entry as +corruption. Read to the end of the last segment. + +**Since** — #10091. + +### `turn/started` + +A turn was authorized and is about to run. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — Once the turn is authorized, after the permit, shutdown and +stop-before-dispatch gates. A `turn/started` therefore always means the turn ran. + +**Pairing** — Opener. Closed by [`turn/completed`](#turncompleted). An open +`turn/started` at the tail is what repair closes. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Message-boundary ordinal identifying the turn. | | +| `actor` | string | required | Who caused the turn. An unrecognized value is folded to `other`. | `user`, `app`, `crew`, `cron`, `autonudge`, `subagent`, `gateway`, `other` | +| `depth` | int | required | Prompt depth. | | +| `message_seq` | int | optional | `seq` of the causing message entry. Omitted when 0 or unknown. | | +| `attempt` | int | optional | Which try at this ordinal. Omitted at 1; present and greater than 1 on a rerun of the same ordinal. | | + +**Invariants** — A turn ordinal may appear more than once when a turn is rerun; +`attempt` is what separates the tries. + +```json +{"type":"turn/started","seq":12,"time":1789000000200,"src":"gateway","data":{"turn":3,"actor":"user","depth":0}} +``` + +**Reader hint** — Key a turn on `(turn, attempt)` rather than `turn` alone, taking +a missing `attempt` as 1. + +**Since** — #10091. + +### `turn/refused` + +A turn was dispatched but a gate refused to run it. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — At the refusing gate, instead of a `turn/started`. + +**Pairing** — Terminal for that ordinal on its own. It is deliberately not an open +opener, so a refusal is never mistaken for a turn that died mid-flight and never +attracts a repair closer. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `actor` | string | required | Who caused the turn. Same folding as `turn/started`. | `user`, `app`, `crew`, `cron`, `autonudge`, `subagent`, `gateway`, `other` | +| `reason` | string | required | Which gate refused. The writer records the caller's value without constraining it. | `not_authorized`, `gateway_closing`, `stopped_before_dispatch` | +| `depth` | int | required | Prompt depth. | | + +**Invariants** — No `turn/completed` follows it for that ordinal. + +```json +{"type":"turn/refused","seq":13,"time":1789000000210,"src":"gateway","data":{"turn":4,"actor":"user","reason":"stopped_before_dispatch","depth":0}} +``` + +**Reader hint** — Count refusals separately from turns. Folding them together +makes a session look busier than it was. + +**Since** — #10091. + +### `turn/completed` + +A turn ended; records its outcome and its cost. + +**Kind and `src`** — `session`; `src` is `acp` on the measured close, `gateway` on +the failed close and on a repair closer. + +**When written** — From the turn's `finally`, after all output has flushed — +not from the terminal stream event. Three writers reach it: the measured close, the +in-process failed close, and crash repair. + +**Pairing** — **Closer, and written last.** Its position marks the turn boundary, +so anything after it belongs to an already-closed turn. Closes +[`turn/started`](#turnstarted). Repair writes it with `stop_reason` `interrupted` +for a turn that was still open. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `depth` | int | on a measured or failed close | Prompt depth. Absent on a crash-repair closer. | | +| `stop_reason` | string | required | How the turn ended. `failed` is the in-process failed close; `interrupted` is written **only by crash repair**; otherwise the provider's terminal reason. | `end_turn`, …, `failed`, `interrupted` (repair only) | +| `duration_ms` | int | on a measured or failed close | Measured turn duration. Absent on a crash-repair closer, which has none to report. | | +| `model` | string | on a measured or failed close | Model the turn served on. Absent on a crash-repair closer. | | +| `provider` | string | on a measured or failed close | Provider. Absent on a crash-repair closer. | | +| `credits` | float | optional | Present on the measured close, absent on a synthesized one. | | +| `tokens` | object | optional | `{input, output, cache_read, cache_write}`, all ints. Present with `credits`, absent on a synthesized close. | | +| `error` | string | optional | Exception class name — never its message — on the failed close. | | + +**Invariants** — `credits` and `tokens` travel together. An in-process close +carrying neither is synthesized, and its `duration_ms` is still real. A crash-repair +closer is narrower than either: it carries `turn` and `stop_reason` and nothing else, +so a fold must read every other field with a default rather than by subscript. Cost is +measured per turn and appears only here, never on `message/sent`. + +```json +{"type":"turn/completed","seq":40,"time":1789000001500,"src":"acp","data":{"turn":3,"depth":0,"stop_reason":"end_turn","duration_ms":1300,"credits":0.0021,"model":"claude","provider":"anthropic","tokens":{"input":812,"output":143,"cache_read":0,"cache_write":0}}} +``` + +**Reader hint** — Sum cost over these entries alone. Absent `credits` means +unmeasured, not free, so a total should carry a count of synthesized closes beside +it. + +**Since** — #10091. + +### `write/dropped` + +One durable account of writer losses, before later entries resume. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — At the head of the session's next batch after a loss. Further +loss merges into the same pending marker rather than adding entries. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `dropped_count` | int | required | How many appends were lost. | | +| `dropped_bytes` | int | required | Size hint for the lost jobs. | | + +**Invariants** — It records a hole without making one: a lost job never took a +`seq`, so `seq` stays contiguous across the loss. It carries **no reason code**, +because the writer cannot separate a malformed entry from an ownership refusal at +the point it gives up. + +```json +{"type":"write/dropped","seq":55,"time":1789000002000,"src":"gateway","data":{"dropped_count":3,"dropped_bytes":2048}} +``` + +**Reader hint** — This is the one entry that says the record is incomplete. A +reader reporting on a session should surface it rather than fold it away, and +should not try to infer *which* facts are missing. + +**Since** — #10091. + +## Message, request and step + +### `message/received` + +The body of a message the gateway accepted into this session. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — Before the dispatch gates, so a refused turn still shows what +was said. + +**Pairing** — None, but on an oversize body it cites its +[`message/chunk`](#messagechunk) entries and is written in the same batch as them. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `role` | string | required | Message role. | | +| `source` | string | required | Surface the message arrived on. May be empty. | | +| `text` | string | conditional | The redacted body, when it fits one line. Replaced by `chunks` when it does not. | | +| `attachments` | [string] | optional | Attachment ids. Present only when there is at least one. These are ids, not `ref`s. | | +| `attachments_omitted` | int | optional | How many ids were dropped because the list would not fit. | | +| `chunks` | [int] | optional | Chunk `seq`s, present instead of `text` on an overflow body. | | +| `chars` | int | optional | Character count of the full body, present with `chunks`. | | + +**Invariants** — Exactly one of `text` or `chunks` is present. The body is redacted +at the emitter — exfiltration URLs first, then credentials — before it is measured +or split. + +```json +{"type":"message/received","seq":11,"time":1789000000180,"src":"gateway","data":{"turn":3,"role":"user","text":"fix the build","source":"dashboard"}} +``` + +**Reader hint** — Handle both body forms. Reconstruct an overflow body by reading +the `chunks` seqs in order and concatenating their `delta` values. + +**Since** — #10091. + +### `message/sent` + +A finished assistant message — one model call's worth of text. + +**Kind and `src`** — `session`; `src` is `acp`. + +**When written** — When a reply is complete. An empty body writes nothing. + +**Pairing** — None, but cites its [`message/chunk`](#messagechunk) entries on an +oversize body. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `step` | int | optional | Model call ordinal. Omitted when 0 or unknown. | | +| `text` | string | conditional | The redacted body, when it fits one line. | | +| `interrupted` | bool | optional | Written only as `true`, when a steer cut this reply. | `true` | +| `chunks` | [int] | optional | Chunk `seq`s, present instead of `text` on overflow. | | +| `chars` | int | optional | Character count of the full body, present with `chunks`. | | + +**Invariants** — Exactly one of `text` or `chunks`. `interrupted` is never written +`false`, so absence means "not interrupted". This entry carries no usage: cost +rides on [`turn/completed`](#turncompleted). + +```json +{"type":"message/sent","seq":39,"time":1789000001400,"src":"acp","data":{"turn":3,"step":2,"text":"Done."}} +``` + +**Reader hint** — Several of these per turn is normal — one per model call. Do not +treat the first as the turn's answer. + +**Since** — #10091. + +### `message/chunk` + +One slice of an oversize body. + +**Kind and `src`** — `session`; `src` is inherited from the body entry that cites +it — `acp` under a `message/sent`, `gateway` under a `message/received`. + +**When written** — Only by the overflow split, never on its own. + +**Pairing** — Cited by the body entry that names its `seq` in `chunks`. All the +chunks and the citing entry are written in one batch, so a reader either sees the +whole group or none of it. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `step` | int | optional | Model call ordinal, on assistant bodies. | | +| `delta` | string | required | One redacted slice of the body. | | + +**Invariants** — Always `ignorable: true`. A **trailing** chunk run whose citing +entry never landed is truncated by repair, so a chunk that survives always has a +citing entry. + +```json +{"type":"message/chunk","seq":41,"time":1789000001410,"src":"acp","ignorable":true,"data":{"turn":3,"step":2,"delta":"the first slice of the body"}} +``` + +**Reader hint** — Never read these directly. Start from a body entry's `chunks` +list; a reader that does not understand the type may skip them, which is what +`ignorable` promises. + +**Since** — #10091. + +### `message/queued` + +A message arrived while a turn was already running. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — On arrival, when a turn is in flight. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `source` | string | required | Surface it arrived on. | | +| `bytes` | int | required | Size of the queued message. | | +| `queued_seq` | string | required | The queue entry's own id. A string, not a ledger `seq`. | | + +**Invariants** — Carries **no `turn`**: it belongs to no turn yet. The body is not +recorded here; it lands in [`message/received`](#messagereceived) when the queue +drains. + +```json +{"type":"message/queued","seq":30,"time":1789000000900,"src":"gateway","data":{"source":"slack","bytes":214,"queued_seq":"q-8"}} +``` + +**Reader hint** — `queued_seq` names a queue slot and must not be resolved as a +ledger `seq` or fed to a `ref`. + +**Since** — #10091. + +### `request/configured` + +The request configuration, recorded when it changed. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — Change-only per session. An unchanged configuration is silent, +and the fingerprint is remembered only after the line lands. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `model` | string | required | Model. | | +| `provider` | string | required | Provider. | | +| `context_window` | int | required | Context window size. | | +| `system` | string | optional | sha256 digest of the system prompt, when one is supplied. | | +| `system_bytes` | int | optional | Byte length of the system prompt, present with `system`. | | + +**Invariants** — The system prompt is digested, never recorded. No resolved tool +list is written: with tool search on, the gateway does not receive one. + +```json +{"type":"request/configured","seq":10,"time":1789000000150,"src":"gateway","data":{"turn":3,"model":"claude","provider":"anthropic","context_window":200000}} +``` + +**Reader hint** — The absence of this entry on a turn means the configuration +matches the last one recorded, not that it is unknown. Carry the value forward. + +**Since** — #10091. + +### `context/composed` + +What the gateway put in front of the model, block by block. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — Per model call that composes a prompt. Nothing is written when +there are no blocks to tally. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `sources` | [object] | required | Per-block `{kind, chars, tokens}`, ordered by descending `chars`. | | +| `chars` | int | required | Total characters. | | +| `tokens` | int | required | Estimated tokens. | | +| `tokens_estimated` | bool | required | Always `true`. | `true` | +| `step` | int | optional | Model call ordinal. Omitted when 0. | | + +**Invariants** — `tokens` is an estimate derived from `chars`, which is why +`tokens_estimated` is written on every entry rather than only when it is true. +Blocks with no classification are folded into a single `other` source, so `sources` +does not enumerate every injected block by name. + +```json +{"type":"context/composed","seq":9,"time":1789000000140,"src":"gateway","data":{"turn":3,"sources":[{"kind":"system","chars":4000,"tokens":1000},{"kind":"other","chars":1200,"tokens":300}],"chars":5200,"tokens":1300,"tokens_estimated":true}} +``` + +**Reader hint** — Do not report these token numbers as billed usage. The billed +figures are on [`turn/completed`](#turncompleted). + +**Since** — #10091. + +### `step/started` + +Opens one model call inside a turn. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — Right after `turn/started` once the turn is authorized, and +again at each transition from a tool group back to text. + +**Pairing** — Opener. Closed by [`step/completed`](#stepcompleted). + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `step` | int | required | Model call ordinal, from 1. | | + +**Invariants** — Steps are numbered within a turn, so `(turn, step)` identifies a +model call. + +```json +{"type":"step/started","seq":14,"time":1789000000220,"src":"gateway","data":{"turn":3,"step":1}} +``` + +**Reader hint** — Step count per turn is the cheapest measure of how much +tool-calling a turn did. + +**Since** — #10091. + +### `step/completed` + +Closes one model call and records how long it took. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — At the next transition, and beside the turn's completion event. +Nothing is written when the step ordinal is 0. + +**Pairing** — Closer for [`step/started`](#stepstarted). Repair writes no step +closer, so a step may be left open by a crash. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `step` | int | required | Model call ordinal. | | +| `ms` | int | required | Duration, never negative. | | + +**Invariants** — `ms` measures the model call, not the turn. + +```json +{"type":"step/completed","seq":38,"time":1789000001390,"src":"gateway","data":{"turn":3,"step":1,"ms":900}} +``` + +**Reader hint** — An open `step/started` with a `turn/completed` after it means the +turn ended mid-step. That is expected, not damage. + +**Since** — #10091. + +## Tool and approval + +### `tool/called` + +A tool call, identified by id. Arguments are digested, never recorded. + +**Kind and `src`** — `session`; `src` is `acp`. + +**When written** — On the call frame. + +**Pairing** — Opener. Closed by [`tool/completed`](#toolcompleted) matched on +`call_id`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `call_id` | string | required | Tool call id. Empty when the frame carried none. | | +| `name` | string | required | Trusted tool name. Empty when the backend supplied none. | | +| `server` | string | required | MCP server name. Empty when there is none. | | +| `kind` | string | required | Tool kind. | | +| `call_index` | int | optional | Position among the turn's tool calls. Omitted when 0. | | +| `step` | int | optional | Model call that issued it. Omitted when 0. | | +| `args_hash` | string | optional | sha256 of the serialized arguments, when there are any. | | +| `args_bytes` | int | optional | Byte length of the serialized arguments, present with `args_hash`. | | + +**Invariants** — Argument *content* never reaches the ledger. `args_hash` lets two +calls be compared for equality without recording what was passed. + +```json +{"type":"tool/called","seq":16,"time":1789000000300,"src":"acp","data":{"turn":3,"call_id":"c-01","name":"read","server":"","kind":"fs","call_index":1,"step":1,"args_hash":"9f2b7c41","args_bytes":42}} +``` + +**Reader hint** — `call_id` may be empty, so it is not a safe dictionary key on its +own. Fall back to `(turn, call_index)`. + +**Since** — #10091. + +### `tool/completed` + +A tool call's terminal frame. Results are digested, never recorded. + +**Kind and `src`** — `session`; `src` is `acp` on both the ordinary close and the +turn-end sweep, and `gateway` only on a repair closer. + +**When written** — Three writers: the ordinary terminal frame; the sweep that +closes still-open calls at a tool-group boundary or at turn end; crash repair. + +**Pairing** — Closer for [`tool/called`](#toolcalled), matched on `call_id`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `call_id` | string | required | The same id as the call. | | +| `name` | string | required | Filled from the remembered call frame. | | +| `server` | string | required | Filled from the remembered call frame. | | +| `status` | string | required | Outcome, as an **open set**: the terminal frame passes the backend's own word through unmapped, so a value outside the list below is possible and must not be treated as invalid. `refused` is decided by this process and wins over the frame's word. The sweep writes `completed` at a tool-group boundary and `unknown` at turn end; repair writes `unknown` for a call it cannot match. | `completed`, `failed`, `cancelled`, `canceled`, `refused`, `unknown` | +| `call_index` | int | optional | Present when known. | | +| `step` | int | optional | Present when known. | | +| `elapsed_ms` | int | optional | Present when the call frame was still in memory. | | +| `is_error` | bool | optional | Tri-state: omitted when the caller made no assertion either way. | | +| `result_hash` | string | optional | sha256 of the redacted result, when there is one. | | +| `result_bytes` | int | optional | Byte length. `0` on an output-less close, absent when there was nothing to digest. | | + +**Invariants** — `status: "unknown"` means the writer could not observe the +outcome, not that the tool failed. Both `cancelled` and `canceled` occur, because the +word is the backend's and is not normalized on the way in. A sweep close carries +`result_bytes: 0` and no `result_hash`. + +```json +{"type":"tool/completed","seq":18,"time":1789000000400,"src":"acp","data":{"turn":3,"call_id":"c-01","name":"read","server":"","status":"completed","call_index":1,"step":1,"elapsed_ms":90,"result_hash":"1a3c9e02","result_bytes":512}} +``` + +**Reader hint** — Do not count `unknown` as a failure. Distinguish it from +`is_error: true`, which is a real reported error. + +**Since** — #10091. + +### `approval/requested` + +A tool call is waiting on a human. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — One statement before the `try` whose `finally` records the +decision, so a request can never be lost while its decision is written. + +**Pairing** — Opener. Closed by [`approval/decided`](#approvaldecided) matched on +`approval_id`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `approval_id` | string | required | Approval request id. | | +| `tool` | string | optional | Tool name. Absent when the frame named none. | | +| `reason` | string | optional | The redacted, clipped title shown to the human. Absent when there is none. | | + +**Invariants** — `reason` is what a person saw, not the tool's arguments. + +```json +{"type":"approval/requested","seq":24,"time":1789000000600,"src":"gateway","data":{"turn":3,"approval_id":"a-1","tool":"shell","reason":"remove the build directory"}} +``` + +**Reader hint** — An unmatched request means the process died while a human was +still deciding. + +**Since** — type #10091; emitter #11185. + +### `approval/decided` + +How an approval resolved. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — From the `finally` that every exit path converges on: a human +answer, a timeout, a no-budget decline, a delivery failure, a cancelled wait. + +**Pairing** — Closer for [`approval/requested`](#approvalrequested). Repair closes +an unmatched request with `decision` `unknown`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `approval_id` | string | required | The same id as the request. | | +| `decision` | string | required | The decision. `unknown` is written **only by crash repair**. | `unknown` (repair only) | +| `by` | string | optional | Written only for a host-made decision. Omitted for a person's own answer. | `host` | +| `cause` | string | optional | The host's reason code for an automatic decline. | | + +**Invariants** — Absence of `by` is the signal that a human answered. A reader must +not read it as an unattributed decision. + +```json +{"type":"approval/decided","seq":25,"time":1789000000650,"src":"gateway","data":{"turn":3,"approval_id":"a-1","decision":"approved"}} +``` + +**Reader hint** — To count what a person actually approved, filter to entries with +no `by`. + +**Since** — type #10091; emitter #11185. + +## Model, compaction and plan + +### `model/selected` + +The model a session will serve, and why it was chosen. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — On a swap, such as a fallback. The session's starting model is +not written here; it rides on [`session/opened`](#sessionopened). + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `model` | string | required | Model id. | | +| `source` | string | required | Why this model was chosen. | | +| `turn` | int | optional | The turn the pick was made for. Omitted when 0 or outside a turn. | | + +**Invariants** — An entry here means the served model differs from the one +`session/opened` recorded. + +```json +{"type":"model/selected","seq":26,"time":1789000000700,"src":"gateway","data":{"model":"claude-fallback","source":"fallback","turn":3}} +``` + +**Reader hint** — To know which model served turn N, take the latest +`model/selected` at or before it, falling back to `session/opened`. + +**Since** — #10091. + +### `compaction/applied` + +A compaction, recorded as context-usage percentages. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — When the compaction verdict settles, which may be later than the +turn it measures. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `pct_before` | float | required | Context usage percentage before, rounded to 4 decimal places. | | +| `pct_after` | float | required | Context usage percentage after. | | +| `freed_pct` | float | required | `pct_before` minus `pct_after`. Negative when a deferred reading takes in a later turn's growth. | | + +**Invariants** — Carries **no `turn`**, because its verdict can settle turns after +the compaction it describes. No raw token counts: the boundary measures only +percentages. + +```json +{"type":"compaction/applied","seq":50,"time":1789000001800,"src":"gateway","data":{"pct_before":82.0,"pct_after":41.0,"freed_pct":41.0}} +``` + +**Reader hint** — A negative `freed_pct` is a real reading, not a bug. Do not clamp +it to zero. + +**Since** — #10091. + +### `plan/updated` + +The session's own task list, as the agent just restated it. + +**Kind and `src`** — `session`; `src` is `acp`. + +**When written** — Each time the agent restates its list. A `null` list writes +nothing; an empty list is a cleared plan and is written. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `turn` | int | required | Turn ordinal. | | +| `items` | [object] | required | Each `{id, text, state}`. | `state`: `done`, `open` | +| `total` | int | optional | The real count, written only when the list was clipped by count or by bytes. | | + +**Invariants** — A whole list, not a delta. Always `ignorable: true`. `total` +greater than `len(items)` means what is written is a prefix. + +```json +{"type":"plan/updated","seq":28,"time":1789000000800,"src":"acp","ignorable":true,"data":{"turn":3,"items":[{"id":"1","text":"read code","state":"done"},{"id":"2","text":"write fix","state":"open"}]}} +``` + +**Reader hint** — Diff consecutive entries to see progress. Treat an empty `items` +as "plan cleared", not as "no data". + +**Since** — type #10091; emitter #11185. + +## Background work and children + +### `background/completed` + +A model call the gateway made on this session's behalf. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — When such a call finishes. It runs after a turn ends, on a +separate session, so it carries no `turn`. + +**Pairing** — None. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `kind` | string | required | Which background call this was. The writer records the caller's value without constraining it; the calling sites emit the values listed here. | `title`, `summary`, `memory_consolidation` | +| `model` | string | optional | Served model. | | +| `provider` | string | optional | Provider. | | +| `credits` | float | optional | Written only when the call was billed, so a zero cost is absent. | | +| `tokens` | object | optional | Only the non-zero billed dimensions of `{input, output, cache_read, cache_write}`. | | +| `ms` | int | optional | Wall clock. Omitted at 0. | | + +**Invariants** — Carries **no `turn`**. `tokens` is sparse by construction: a +missing dimension means zero, not unknown. + +```json +{"type":"background/completed","seq":60,"time":1789000002100,"src":"gateway","data":{"kind":"title","model":"claude-haiku","provider":"anthropic","credits":0.0001,"tokens":{"input":40,"output":8},"ms":300}} +``` + +**Reader hint** — Add this cost to a session's total separately from +`turn/completed`; it is real spend that belongs to no turn. Treat `kind` as an open +set and keep an "other" bucket. + +**Since** — type #10091; emitter #11185. + +### `subagent/spawned` + +A child this session dispatched. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — When the run actually starts, not when it is queued. + +**Pairing** — Opener. Closed by [`subagent/completed`](#subagentcompleted) or +[`subagent/failed`](#subagentfailed), matched on `agent_id`. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `agent_id` | string | required | The child's id. | | +| `turn` | int | optional | The asking turn. Absent when no turn asked — a slash command, a cron, a hook. | | +| `agent` | string | optional | Agent name. | | +| `model` | string | optional | The child's model. | | +| `scope` | object | optional | Context-scope flags `{memory, lessons, project}`, all bools. | | + +**Invariants** — Carries **no `ref`**. No subagent path opens a child ledger, so a +citation would name a file that does not exist. + +```json +{"type":"subagent/spawned","seq":32,"time":1789000000850,"src":"gateway","data":{"turn":3,"agent_id":"sub-1","agent":"kirocrew","model":"claude","scope":{"memory":false,"lessons":true,"project":true}}} +``` + +**Reader hint** — A missing `turn` is normal and does not mean the entry is +damaged. Group children by `agent_id`, not by turn. + +**Since** — type #10091; emitter #11185. + +### `subagent/steered` + +A correction sent into a running child. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — When the steer is sent. + +**Pairing** — None. It is written into the parent's ledger, because the child has +none of its own. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `agent_id` | string | required | The child's id. | | +| `mode` | string | optional | How the steer was delivered. | `interrupt`, `follow_up` | + +**Invariants** — The steer text is not recorded here. + +```json +{"type":"subagent/steered","seq":33,"time":1789000000860,"src":"gateway","data":{"agent_id":"sub-1","mode":"follow_up"}} +``` + +**Reader hint** — Several of these may sit between one spawn and its close. + +**Since** — type #10091; emitter #11185. + +### `subagent/completed` + +A child finished its work. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — On the success outcome only. + +**Pairing** — Closer for [`subagent/spawned`](#subagentspawned). + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `agent_id` | string | required | The child's id. | | +| `ms` | int | optional | Elapsed time. Omitted at 0. | | + +**Invariants** — Carries no `tokens` and no `credits`: the subagent runtime +measures neither, so a child's cost is not recoverable from the parent's ledger. + +```json +{"type":"subagent/completed","seq":70,"time":1789000002500,"src":"gateway","data":{"agent_id":"sub-1","ms":40000}} +``` + +**Reader hint** — Do not attribute child cost from this entry. There is none to +attribute. + +**Since** — type #10091; emitter #11185. + +### `subagent/failed` + +A child did not finish its work. + +**Kind and `src`** — `session`; `src` is `gateway`. + +**When written** — On either non-success outcome. + +**Pairing** — Also a closer for [`subagent/spawned`](#subagentspawned). Repair +closes an unmatched spawn with `outcome` `unknown`, and only for a child a +liveness predicate reports finished — with no predicate the spawn is left open +rather than closed on a guess. + +| Field | Type | Required | Meaning | Enum | +|---|---|---|---|---| +| `agent_id` | string | required | The child's id. | | +| `outcome` | string | optional | Which non-success outcome. Defaults to `failed`, so it is absent only when a caller passes an empty value. `unknown` is written **only by crash repair**. | `failed`, `stopped`, `unknown` (repair only) | +| `reason` | string | optional | Redacted, clipped failure reason. | | +| `ms` | int | optional | Elapsed time. Omitted at 0. | | + +**Invariants** — This one type covers both non-success outcomes; `outcome` is what +separates them. + +```json +{"type":"subagent/failed","seq":71,"time":1789000002510,"src":"gateway","data":{"agent_id":"sub-2","reason":"provider error","outcome":"failed","ms":12000}} +``` + +**Reader hint** — A spawn with no close at all means the parent died while the +child was running and no liveness predicate was available. + +**Since** — type #10091; emitter #11185. + +## Removed types + +These nine are owned by the `session` kind in the format and have no emitter in any +open change, so they are removed rather than kept as unwritten declarations. That is +the line between this table and an `#11185` row in the summary: a type here has no +writer to wait for, while an `#11185` row has one on the way. `message/steered` sits +here even though its emitter function exists, because nothing calls it. A reader needs +no handling for anything in this table. + +| Type | Why it is removed | +|---|---| +| `session/seeded` | Specified for a legacy-transcript import path that no code performs. | +| `message/steered` | No single site observes both the interrupted text's flush and the steer echo, so any one writer would record a `seq` that contradicts causality. What the cut site can prove is already recorded by `message/sent.interrupted` and the requeue's `message/queued`. | +| `tool/searched` | A tool search reaches the gateway as an ordinary tool-call frame, so it is already recorded as `tool/called` and `tool/completed`. A second entry would put one fact at two `seq`s, and its query and hit count would break the rule that arguments are digested, never recorded. | +| `tool/loaded` | Its token-cost field has no source anywhere in the repository. | +| `skill/searched` | A skill search is an MCP tool call, already recorded. | +| `skill/loaded` | Reading a skill file is already a tool call, and its token field has no source: the loader counts characters. | +| `summary/written` | Its coverage field names ledger `seq`s, and the summary path holds nothing that can name one, so the field would have to be fabricated. | +| `remote/placed` | Its provider and id fields have no source at the placement site. | +| `remote/lost` | Its reason field has no source at the relay site. | diff --git a/test/test_docs_ledger_reference.py b/test/test_docs_ledger_reference.py new file mode 100644 index 000000000000..9b05af79ac74 --- /dev/null +++ b/test/test_docs_ledger_reference.py @@ -0,0 +1,276 @@ +"""The ledger reference documents types the code actually owns, with valid examples. + +An API reference drifts silently: a type is renamed in ``schema.TYPE_OWNERSHIP`` and +the page that documents it keeps its old spelling, or an example is hand-edited into +a shape the envelope would refuse. Both failures look fine to a reader, which is +exactly why they need a gate rather than a review. + +Two properties are asserted here. + +Every type the page documents is a type this build's session kind owns. Ownership is +per DOMAIN (the part before the slash), matched by prefix so a new action needs no +registry change, so the domain is what can be checked -- and it is the half that +actually moves when the schema is reorganised. + +Every JSON example parses, survives the envelope reader, and would be accepted as a +session entry: owned type, acceptable ``src``, serializable ``data``, and a +serialized line inside the size ceiling. The examples are the part of a reference a +reader copies, so an example the writer would refuse is worse than no example. + +The page's own summary table is checked against its subsections in the same pass. +The table is the index a reader scans; a type in one and not the other means the page +disagrees with itself. + +Scope is the LIVE types only -- the subsection headings. The "Removed types" +appendix names types with no emitter, deliberately kept as a plain table with no +subsection and no example, so trimming them from the schema does not red this test. +""" + +from __future__ import annotations + +import functools +import json +import re +from pathlib import Path + +import pytest + +from kiro_crew.ledger.schema import ( + KIND_SESSION, + MAX_ENTRY_BYTES, + MAX_REF_SPAN, + TYPE_OWNERSHIP, + Entry, + check_ownership, + require_data, + require_entry_line, + serialize, + split_type, +) +from kiro_crew.ledger.store import DEFAULT_PAGE_LIMIT, MAX_PAGE_LIMIT + +_DOCS = Path(__file__).parent.parent / "docs" / "reference" / "ledger" +SESSION_TYPES_DOC = _DOCS / "session-types.md" + +#: A live type's subsection heading: ``### `domain/action` ``. +_SUBSECTION_RE = re.compile(r"^### `([a-z]+/[a-z_]+)`\s*$", re.MULTILINE) + +#: A summary-table row's first cell, which links to the subsection anchor. +_SUMMARY_ROW_RE = re.compile(r"^\| \[`([a-z]+/[a-z_]+)`\]\(#[a-z]+\) \|", re.MULTILINE) + +#: A fenced ``json`` block. Every one on this page is a single entry line. +_JSON_BLOCK_RE = re.compile(r"^```json\n(.*?)\n```$", re.MULTILINE | re.DOTALL) + +#: The count the page's own prose claims. Pinned so adding a subsection without +#: updating the page's opening line fails here rather than misleading a reader. +EXPECTED_LIVE_TYPES = 26 + +#: The only two emitters that write a session entry. Kept as a literal rather than +#: read from ``FIXED_SOURCES``, which holds the crew-side values too. +SESSION_SOURCES = frozenset({"gateway", "acp"}) + + +def _doc_text() -> str: + return SESSION_TYPES_DOC.read_text(encoding="utf-8") + + +def _documented_types() -> list[str]: + return _SUBSECTION_RE.findall(_doc_text()) + + +def _json_examples() -> list[str]: + return _JSON_BLOCK_RE.findall(_doc_text()) + + +def test_reference_page_exists() -> None: + """A missing page would make every parametrized case below vacuous.""" + assert SESSION_TYPES_DOC.is_file(), f"{SESSION_TYPES_DOC} is missing" + + +def test_documented_type_count() -> None: + types = _documented_types() + assert len(types) == EXPECTED_LIVE_TYPES, f"documented types: {sorted(types)}" + assert len(set(types)) == len(types), "a type has two subsections" + + +@pytest.mark.parametrize("entry_type", _documented_types()) +def test_documented_type_domain_is_owned_by_the_session_kind(entry_type: str) -> None: + domain, _action = split_type(entry_type) + assert domain in TYPE_OWNERSHIP[KIND_SESSION], ( + f"{entry_type} is documented as a session type, but the session kind does " + f"not own the {domain!r} domain" + ) + + +def test_summary_table_matches_the_subsections() -> None: + assert set(_SUMMARY_ROW_RE.findall(_doc_text())) == set(_documented_types()) + + +def test_every_documented_type_has_an_example() -> None: + """A type documented without an example is a contract nobody can copy.""" + example_types = {json.loads(block)["type"] for block in _json_examples()} + assert set(_documented_types()) <= example_types + + +@pytest.mark.parametrize("block", _json_examples()) +def test_example_is_one_line_of_valid_json(block: str) -> None: + assert "\n" not in block, "an entry example must be one line" + assert isinstance(json.loads(block), dict), "an entry example must be an object" + + +@pytest.mark.parametrize("block", _json_examples()) +def test_example_passes_the_envelope_reader(block: str) -> None: + entry = Entry.from_dict(json.loads(block)) + assert entry is not None, "the envelope reader would discard this example" + + +@pytest.mark.parametrize("block", _json_examples()) +def test_example_would_be_accepted_as_a_session_entry(block: str) -> None: + """The writer's own gates, in the order ``append`` applies them.""" + entry = Entry.from_dict(json.loads(block)) + assert entry is not None + require_data(entry.data) + check_ownership(KIND_SESSION, entry.type, entry.src) + require_entry_line(serialize(entry.to_dict())) + + +@pytest.mark.parametrize("block", _json_examples()) +def test_example_type_is_documented_on_this_page(block: str) -> None: + """An example may not illustrate a type the page does not document.""" + assert json.loads(block)["type"] in set(_documented_types()) + + +@pytest.mark.parametrize("block", _json_examples()) +def test_session_examples_carry_no_crew_only_envelope_fields(block: str) -> None: + """Session entries never thread, and no session emitter cites a ref.""" + raw = json.loads(block) + assert "thread" not in raw, "thread is the crew ledger's shape" + assert "ref" not in raw, "no session emitter sets ref" + + +@pytest.mark.parametrize("block", _json_examples()) +def test_session_examples_name_only_a_session_emitter(block: str) -> None: + """``src`` on a session example is one of the two emitters that write one. + + ``check_ownership`` accepts any fixed source for any kind, so ``dashboard`` or + ``patrol`` on a session entry passes the writer's gates while naming an emitter + that writes to crew ledgers. That makes it precisely the mistake a reference + page can carry without anything catching it, so the page is pinned here. + """ + assert json.loads(block)["src"] in SESSION_SOURCES + + +#: A summary row's type together with its ``Emitter`` cell (the third column). +_EMITTER_CELL_RE = re.compile( + r"^\| \[`([a-z]+/[a-z_]+)`\]\(#[a-z]+\) \| [^|]*\| ([^|]*)\|", re.MULTILINE +) + + +def _subsection(entry_type: str) -> str: + """The body of one type's subsection, up to the next one.""" + text = _doc_text() + start = text.index(f"### `{entry_type}`") + nxt = text.find("\n### ", start + len(entry_type)) + return text[start : len(text) if nxt < 0 else nxt] + + +def test_summary_emitter_cell_agrees_with_the_subsection_since_line() -> None: + """A row claiming an unlanded emitter and its ``Since`` line say the same thing. + + The two facts sit ~850 lines apart -- a table cell at the top, a ``Since`` line in + the subsection -- so they drift without anything noticing, and the drift has a + direction that misleads: a row reading ``live`` for a type whose emitter has not + landed tells a consumer to handle entries nothing writes. Pinned in BOTH + directions, because either half alone can be the stale one. + """ + rows = _EMITTER_CELL_RE.findall(_doc_text()) + assert len(rows) == EXPECTED_LIVE_TYPES, f"summary rows parsed: {len(rows)}" + for entry_type, cell in rows: + pending = cell.strip().startswith("#") + since_is_pending = "emitter #" in _subsection(entry_type) + assert pending == since_is_pending, ( + f"{entry_type}: summary Emitter cell {cell.strip()!r} and its Since line " + f"disagree about whether the emitter has landed" + ) + + +#: Every page in the reference, for the checks that span more than session-types. +_PAGES = sorted(_DOCS.glob("*.md")) + +#: How each constant is written in prose. A change to the constant changes the +#: string the pages must carry, which is the whole point of asserting it here. +_CONSTANT_RENDERINGS = { + "MAX_ENTRY_BYTES": f"{MAX_ENTRY_BYTES // 1024} KiB", + "MAX_REF_SPAN": str(MAX_REF_SPAN), + "MAX_PAGE_LIMIT": str(MAX_PAGE_LIMIT), + "DEFAULT_PAGE_LIMIT": str(DEFAULT_PAGE_LIMIT), +} + + +def _pending_types() -> list[str]: + """Types whose summary row says their emitter has not landed yet.""" + return [t for t, cell in _EMITTER_CELL_RE.findall(_doc_text()) if cell.strip().startswith("#")] + + +@functools.lru_cache(maxsize=1) +def _emit_functions_called_in_the_tree() -> frozenset[str]: + """Every ``on_*`` emitter function called under ``src/kiro_crew``. + + A call site is what makes a type actually written. The emitter module is skipped + because it DEFINES these functions -- an emitter API with no caller writes nothing, + which is exactly the state a pending mark claims. + + Read ONCE for the whole module rather than once per pending mark. The tree is + ~1570 files and ~50 MiB, so scanning it per mark multiplies that by the number of + marks, which is invisible on a warm Linux filesystem and is shard time on a cold + Windows one. + """ + root = Path(__file__).parent.parent / "src" / "kiro_crew" + called: set[str] = set() + for path in root.rglob("*.py"): + if path.name == "session_ledger_emit.py": + continue + try: + text = path.read_text(encoding="utf-8", errors="replace") + except OSError: # pragma: no cover - unreadable file is not this test's subject + continue + # Cheap guard first: almost no file mentions the emitter at all. + if "session_ledger_emit" not in text: + continue + called.update(re.findall(r"\b(on_[a-z_]+)\s*\(", text)) + return frozenset(called) + + +def test_a_pending_emitter_mark_is_not_contradicted_by_the_code() -> None: + """A type marked as awaiting its emitter must have no caller in THIS build. + + The agreement test above compares the page against itself, so both halves can be + stale together -- which is what happens the moment the pending change merges and + nobody flips the marks. This is the half that reads the code, and it is + deliberately self-clearing: when the emitter gains a caller, this fails and names + the type, so the page cannot keep claiming the entry is never written. + """ + called = _emit_functions_called_in_the_tree() + stale = sorted(t for t in _pending_types() if "on_" + t.replace("/", "_") in called) + assert not stale, ( + "these types are marked as awaiting their emitter but the code already calls " + f"it, so each one's summary Emitter cell and Since line must be flipped: {stale}" + ) + + +@pytest.mark.parametrize("page", _PAGES, ids=lambda p: p.name) +def test_a_documented_constant_carries_its_current_value(page: Path) -> None: + """A paragraph naming a constant states that constant's value. + + The pages spell these numbers out, because "64 KiB" is what a reader needs and + ``MAX_ENTRY_BYTES`` is not. Spelling them out is also how they rot, so each claim + is checked against the constant it names. A mention in a signature default + (``limit=DEFAULT_PAGE_LIMIT``) asserts no value and is not a claim. + """ + for block in page.read_text(encoding="utf-8").split("\n\n"): + for name, rendering in _CONSTANT_RENDERINGS.items(): + if name not in block.replace(f"={name}", ""): + continue + assert re.search(rf"\b{re.escape(rendering)}", block), ( + f"{page.name}: a paragraph names {name} without stating its value " f"{rendering!r}" + )