Repository navigation
fix(crew-log): declare the subagent and background types the emitter already writes - #12550
Conversation
|
Intent: Make a session's crew log foldable again after it dispatches a sub-agent or the gateway makes a background model call for it, by declaring the five entry types the emitter already writes non-ignorable, and pin the writer-to-registry direction so the next wired emitter cannot silently reintroduce the same fold-killer. Not a goal: changing any writer, marking existing writes ignorable, adding new entry types, touching the crew-log reader or the fold implementation, or declaring |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing to check. [DESIGN-REVIEWED] e6fe6d7 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS The AST ratchet only sees literal type names — its own docstring says a computed type name escapes both directions; verify no writer does that today. What this change shipsInventory (8 items) — 8 justifiedIntent: FIX — a session that dispatched a sub-agent gets its recoverable work record back (issue #12535); writers verified present on base at
The reversed pin (item 6) carries the evidence the contract demands: its comment claimed the "emitters are not wired on this base," yet all six writers exist in [FIRST-PRINCIPLES-REVIEWED] e6fe6d7 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
19f9351 to
473a461
Compare
|
Rebased: main moved in New head This rebase was curative rather than routine. #12479 landed
Re-run on the rebased tree, named files with Sequencing note for the two remaining siblings that also carry this count assertion (#12314, #12303): whichever lands after this one will hit the same one-line conflict on |
473a461 to
9a1d976
Compare
|
Rebased again: main moved in New head No conflict, and the diff is byte-identical to the previous head: the same three files, 546 insertions and 18 deletions. Unlike the #12479 rebase above, this drift touches neither the crew-log package, nor Re-run on this base, named files with |
9a1d976 to
1430897
Compare
|
Rebased onto current main and corrected a stale claim this PR had introduced into its own diff. New head The correction. The module docstring of Post-rebase mutation verification. The derived-set ratchet was previously mutation-verified before the rebase only, which says nothing about whether it still bites after a merge moved the registry. Re-verified on this tree: retargeting the Reverted, and the file re-verified: 29 declarations, Deletion accounting for the registry, since a rebase is the step that can silently revert someone else's lines: the diff against the new base is 238 insertions and 3 deletions in Re-run on this base, named files with |
1430897 to
bae04d9
Compare
|
Rebased: main moved in New head
Verified against the new conftest, named files with Nothing in these eight commits touches |
A reader that folds state passes known=KNOWN_TYPES to iter_from, which refuses an entry whose type it does not know and which is not marked ignorable. The emitter writes subagent/spawned, subagent/steered, subagent/completed, subagent/failed and background/completed non-ignorable, and entry_types.py declared none of them. So any session that dispatched a child, or that the gateway made one background model call for, held a crew log that could never be folded again: session_ledger.read_state caught the refusal and answered with the empty record for the rest of that session's life. plan/updated is the sixth. Its write stays ignorable=True and is left alone, so it was never part of the refusal above -- but an undeclared ignorable type is SKIPPED by a folding reader, and a skip leaves a gap in the seqs the class fold receives, which marks that history damaged and makes the reader refuse on it. Declaring it removes the skip, so there is no gap and no damage. Declaring it also lets the new pin be a set equality rather than a one-way containment: a declared type with no writer is now as loud as a written type with no declaration. The six types are declared with the field specs their emit call sites actually produce, cross-checked against docs/reference/crew-log/session-types.md, which has documented all six as live since their writers landed. Only the registry was out of sync. background/completed's tokens members are optional, unlike turn/completed's, because its writer drops every zero dimension. subagent/failed has a second writer, the crash-repair closer in store._closer_entries, which supplies only agent_id and outcome, so every other field of that type stays optional. plan/updated's item state is the one closed enum added here: the writer computes it as done-or-open from a single boolean, so no caller can produce a third value. Declaring rather than flipping the writes to ignorable: the timeline projection consumes subagent/spawned, subagent/completed and subagent/failed, so skipping them would trade a broken fold for missing timeline rows. Three pins. A derived-set test parses BOTH writers' syntax trees -- the emitter and the crash-repair closer -- for every type they append, and asserts that set equals the declared set, so a new wired emitter and an orphaned declaration each fail loudly and name the offending type. The old test could not catch the first direction because it walked the declared set and so could never see a type missing from it, and its hand-written companion list went further and asserted these types stay undeclared. A regression test drives the real session_ledger fold past a subagent/spawned entry end to end. The real-turn emitter test now exercises all six types and asserts dropped_writes() == 0, which is what proves the new declarations do not narrow what production already writes -- declaring a type turns validation on for it, and a too-narrow spec drops writes silently rather than raising. The declared-type count moves 22 -> 28 with the roles it names, and the markdown dump's ignorable-marker count is now derived from the registry instead of hardcoded. No migration: declaring a type is reader-side, so an already-broken log recovers on its next fold attempt.
bae04d9 to
e6fe6d7
Compare
GPT 5.6 Review —
|
|
Rebased to clear an inherited red: main moved in New head The red was not this PR's. Three lanes failed on the previous head: That test touches nothing in this diff, which is the crew-log type registry and two crew-log test files. Attribution measured rather than assumed:
After the rebase the fix is an ancestor, and the previously failing test passes here on its own: Also re-run on this base, named files with Note for the two remaining siblings that carry this count assertion (#12314, #12303): the declared count is still 29 here, and no commit in this window touched |
GPT 5.6 Review —
|
|
The seven failures on head At 18:25:43 UTC the Everything else in that batch is the same exhaustion seen from a different angle. Each review lane failed at its shared The workflow's own remedy is to re-run the job, and that works:
Nothing was pushed for this and the tree is untouched. The diff at this head remains the same three files, 238/3, 264/15 and 46/0, with 304 passing locally under |
buluoray
left a comment
There was a problem hiding this comment.
Verified at head e6fe6d7b. Approving. This one is worth more than its three-file diff suggests, and the PR description undersells it.
I checked the producer side rather than the enum, and the gap was real in one direction only. Six types are appended by live writers in crew_log/emit.py and were not declared: plan/updated, subagent/spawned, subagent/steered, subagent/completed, subagent/failed, background/completed -- each with a production call site (chat_runner.py:13230, admission/gate.py:1286, continuation.py:71, terminal.py:87 and :89, llm_helpers.py:1616). Nothing is declared-but-unproduced, and the PR's own test asserts that emptiness in both directions.
What made this severe rather than cosmetic is what an undeclared type does, and I confirmed it in source. Through a folding reader, store.py:2194 raises CODE_UNKNOWN_ENTRY_TYPE for an undeclared non-ignorable type and reconstruction stops there. session_ledger.py:685 folds with known=projection.KNOWN_TYPES, and :791 catches that and logs ledger: folding this slot's crew logs failed. So any session that spawned a subagent or made a background LLM call had an unreadable ledger fold, permanently, with the log itself intact -- and the only symptom was a warning. record() raises, because it folds to compute its return. The ignorable one, plan/updated, is skipped instead, which leaves a seq discontinuity that sets damaged=True on the class fold -- so a session whose agent touched its task list had a damaged class record.
That is the ledger whose whole purpose is to survive compaction, silently empty for exactly the sessions that delegate. Worth stating plainly in the PR body.
The behaviour change is contained: the ledger and class folds do not newly branch on the six types, they only stop refusing them. The one real consequence is correctly guarded -- declaring a type turns validation on for it, and a validation refusal is a dropped write rather than a raise, so test_every_entry_a_real_turn_produces_validates drives the real emitters and asserts dropped_writes() == 0, proving the specs are not narrower than production. The timeline projection already listed three of the subagent types and only now becomes reachable.
The test suite is not membership-only, which was my main worry with a "declare the types" PR. Undeclaring subagent/spawned reddens the vocabulary test at test_crew_log_types.py:439 and test_session_ledger_projection.py:500, where the second is behavioural: it writes a real subagent/spawned entry after a ledger entry and asserts read_state(SLOT)["goal"] reconstructs correctly with no fold-failure warning. That is the consumer doing the right thing, not a name sitting in a list. 304 passed at head.
GPT's state here needed care: three comments exist for this SHA, two "review incomplete" (18:26, 18:35) and one real "no blocking findings" at 18:40:46, which is the latest and therefore authoritative. Opus completed clean at 18:42. Worth noting for anyone reading the thread that the incomplete pair is superseded rather than outstanding.
…kirodotdev#12550) A reader that folds state passes known=KNOWN_TYPES to iter_from, which refuses an entry whose type it does not know and which is not marked ignorable. The emitter writes subagent/spawned, subagent/steered, subagent/completed, subagent/failed and background/completed non-ignorable, and entry_types.py declared none of them. So any session that dispatched a child, or that the gateway made one background model call for, held a crew log that could never be folded again: session_ledger.read_state caught the refusal and answered with the empty record for the rest of that session's life. plan/updated is the sixth. Its write stays ignorable=True and is left alone, so it was never part of the refusal above -- but an undeclared ignorable type is SKIPPED by a folding reader, and a skip leaves a gap in the seqs the class fold receives, which marks that history damaged and makes the reader refuse on it. Declaring it removes the skip, so there is no gap and no damage. Declaring it also lets the new pin be a set equality rather than a one-way containment: a declared type with no writer is now as loud as a written type with no declaration. The six types are declared with the field specs their emit call sites actually produce, cross-checked against docs/reference/crew-log/session-types.md, which has documented all six as live since their writers landed. Only the registry was out of sync. background/completed's tokens members are optional, unlike turn/completed's, because its writer drops every zero dimension. subagent/failed has a second writer, the crash-repair closer in store._closer_entries, which supplies only agent_id and outcome, so every other field of that type stays optional. plan/updated's item state is the one closed enum added here: the writer computes it as done-or-open from a single boolean, so no caller can produce a third value. Declaring rather than flipping the writes to ignorable: the timeline projection consumes subagent/spawned, subagent/completed and subagent/failed, so skipping them would trade a broken fold for missing timeline rows. Three pins. A derived-set test parses BOTH writers' syntax trees -- the emitter and the crash-repair closer -- for every type they append, and asserts that set equals the declared set, so a new wired emitter and an orphaned declaration each fail loudly and name the offending type. The old test could not catch the first direction because it walked the declared set and so could never see a type missing from it, and its hand-written companion list went further and asserted these types stay undeclared. A regression test drives the real session_ledger fold past a subagent/spawned entry end to end. The real-turn emitter test now exercises all six types and asserts dropped_writes() == 0, which is what proves the new declarations do not narrow what production already writes -- declaring a type turns validation on for it, and a too-narrow spec drops writes silently rather than raising. The declared-type count moves 22 -> 28 with the roles it names, and the markdown dump's ignorable-marker count is now derived from the registry instead of hardcoded. No migration: declaring a type is reader-side, so an already-broken log recovers on its next fold attempt.
…kirodotdev#12550) A reader that folds state passes known=KNOWN_TYPES to iter_from, which refuses an entry whose type it does not know and which is not marked ignorable. The emitter writes subagent/spawned, subagent/steered, subagent/completed, subagent/failed and background/completed non-ignorable, and entry_types.py declared none of them. So any session that dispatched a child, or that the gateway made one background model call for, held a crew log that could never be folded again: session_ledger.read_state caught the refusal and answered with the empty record for the rest of that session's life. plan/updated is the sixth. Its write stays ignorable=True and is left alone, so it was never part of the refusal above -- but an undeclared ignorable type is SKIPPED by a folding reader, and a skip leaves a gap in the seqs the class fold receives, which marks that history damaged and makes the reader refuse on it. Declaring it removes the skip, so there is no gap and no damage. Declaring it also lets the new pin be a set equality rather than a one-way containment: a declared type with no writer is now as loud as a written type with no declaration. The six types are declared with the field specs their emit call sites actually produce, cross-checked against docs/reference/crew-log/session-types.md, which has documented all six as live since their writers landed. Only the registry was out of sync. background/completed's tokens members are optional, unlike turn/completed's, because its writer drops every zero dimension. subagent/failed has a second writer, the crash-repair closer in store._closer_entries, which supplies only agent_id and outcome, so every other field of that type stays optional. plan/updated's item state is the one closed enum added here: the writer computes it as done-or-open from a single boolean, so no caller can produce a third value. Declaring rather than flipping the writes to ignorable: the timeline projection consumes subagent/spawned, subagent/completed and subagent/failed, so skipping them would trade a broken fold for missing timeline rows. Three pins. A derived-set test parses BOTH writers' syntax trees -- the emitter and the crash-repair closer -- for every type they append, and asserts that set equals the declared set, so a new wired emitter and an orphaned declaration each fail loudly and name the offending type. The old test could not catch the first direction because it walked the declared set and so could never see a type missing from it, and its hand-written companion list went further and asserted these types stay undeclared. A regression test drives the real session_ledger fold past a subagent/spawned entry end to end. The real-turn emitter test now exercises all six types and asserts dropped_writes() == 0, which is what proves the new declarations do not narrow what production already writes -- declaring a type turns validation on for it, and a too-narrow spec drops writes silently rather than raising. The declared-type count moves 22 -> 28 with the roles it names, and the markdown dump's ignorable-marker count is now derived from the registry instead of hardcoded. No migration: declaring a type is reader-side, so an already-broken log recovers on its next fold attempt.
Problem / Motivation
A session that dispatches a sub-agent gets a work record it can never read again.
The session ledger is a fold of the session's crew log. One
subagent/spawnedentry stops that fold, and because a crew log is append-only the entry never goes
away. So every later read answers with the empty record: the goal, phase, next
step and event tail the session wrote before it dispatched a child are gone for
the rest of that session's life, while the log itself sits intact on disk. One
background model call the gateway makes on the session's behalf does the same
thing.
The reader says so out loud, then the ledger swallows it:
Why it matters
The session ledger exists so a long-running session can recover what it was doing
after context compaction. Dispatching a sub-agent is ordinary work for exactly
those sessions, so the record breaks in the case it was built for.
Nothing tells the caller.
read_stateis best-effort by contract and answers theempty record, so a refused fold and a slot that genuinely recorded nothing look
identical from the outside.
What changed (motivation, approach, change)
A reader that folds state hands
known=KNOWN_TYPEStoiter_from, which refusesan entry it does not know unless the writer marked it skippable.
KNOWN_TYPESisthe declared vocabulary, and the emitter and the store append six types
entry_types.pynever declared. They break in two different ways.Five are written without the skippable mark:
subagent/spawned,subagent/steered,subagent/completed,subagent/failedandbackground/completed. One such entry ends every later fold of that log,permanently.
The sixth,
plan/updated, is written skippable, and that is not safe either. Itis skipped, a skip is a discontinuity in the sequence the fold receives, and the
class fold reads a gap as damage, after which
recorded_classrefuses on it.Declaring it removes the skip, so there is no gap and no damage. Its write stays
ignorable=Trueexactly as it is: the declaration is reader-side only, and thespec's own
ignorableflag is documentation, since the store reads the flag oneach entry rather than on the type.
So all six are declared, with the field specs their call sites actually produce.
docs/reference/crew-log/session-types.mdhas documented every one of them aslive since its writer landed, with the same fields; only the registry was out of
sync, so this brings it back in line rather than inventing a shape.
background/completed'stokensmembers are optional, unliketurn/completed'srequired four, because its writer drops every zero dimension and requiring them
would refuse a call billed on one.
subagent/failedhas a second writer, thecrash-repair closer, which knows only the child's id and that nothing will report
for it, so every field of that type except
agent_idstays optional. Declaring atype turns validation on for it, and a refused write is dropped rather than
raised, so a spec narrower than its writer would lose entries silently.
The alternative was to mark the five non-ignorable writes skippable instead. It
was rejected twice over. The timeline projection reads
subagent/spawned,subagent/completedandsubagent/failed, so skipping them trades a broken foldfor missing timeline rows. And skippable is not a fix in the first place, as
plan/updatedshows: it is already skippable and still damages the class record.The old guard could not catch any of this. It walks the declared set and reports
which members have a writer, so a type that is written and never declared is
invisible to it by construction. Its hand-written companion list went further and
asserted these six stay undeclared. Both are corrected, the declared-type count
moves 23 to 29 with the roles it names, and a new test measures the set equality
from the writers' side.
Backwards compatibility
Compatible: declaring a type is reader-side, so nothing that writes today changes
and no stored log is rewritten. A log already holding one of these entries folds
on its next attempt, so broken slots recover with no migration. The field specs
were read off the existing call sites and every optional field stays optional, and
a test drives the real emitters and asserts
dropped_writes() == 0, so no writethat lands today starts being refused.
Tests
test_the_declared_vocabulary_is_exactly_what_the_writers_appendparses theemitter's and the store's syntax trees for every type they append, keyed on the
append primitive rather than on any type-shaped literal, and asserts the
appended set and the declared set are equal in both directions. Appended and
undeclared is what breaks readers; declared and never appended states a shape no
site produces. It carries three controls: both sides non-empty, four
hand-verified types that must come back on both sides, and the skippable flag
read in both of its values. Mutation-verified in each direction: removing a
declaration fails naming
['subagent/steered'], then['background/completed'],and adding a declaration with no writer fails naming
['remote/lost']. The setreally is derived from the writers rather than from a list kept by hand.
test_the_sampled_types_are_the_ignorable_onespins the skippable marker to thetwo types that sample a stream, which is what keeps a future "just mark it
ignorable" from passing the ratchet above.
test_a_subagent_entry_does_not_stop_the_folddrives the real session ledgerread past a
subagent/spawnedentry end to end, ordered so the child entryfollows the ledger entry and the fold has to walk past it. Mutation-verified: it
fails on the goal assertion, and the refusal it reproduces is the same message
the field reports.
every shape
plan/updatedproduces (with and without a total, and clipped atthe item bound), and asserts both that each type was appended and that
dropped_writes()is still zero.which validates each example, reduces it to its required fields, drops each
required field, rejects an undeclared key and rejects a wrong JSON type per
field.
Manual verification
N/A: unit coverage is sufficient. The defect is a refusal inside the fold, and the
regression test drives the real
read_statepath the field symptom comes from.Related Issues
Fixes #12535
Pattern harvest
Rule candidate: review-prompt
Pattern: a registry guard that iterates the declared set can only ever report a
subset of it, so it cannot detect a written-but-undeclared member. A set equality
between a writer-derived set and a registry has to iterate the writer side, and it
has to assert both directions, since each direction fails in a different way.
Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)