Repository navigation
fix(session-ledger): answer a refused crew log fold with a code, not a bare 500 - #16666
Conversation
Design Review (Opus 5.5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS The field failures came and went (OK → 500 → OK in one session), but the fix types them as permanent damage and never finds why the fold re-reads entries. Watch
[DESIGN-REVIEWED] c9c3523 Description read[DESCRIPTION-READ] 29efa434a2d50cf5eb30a539d89b1440e14a3fab35cac9e53d80773dc7bb2ebf That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Check that the field 500s really were What this change shipsInventory (3 items) — 3 justifiedIntent: an agent whose session-ledger write is refused learns why, and whether retrying can help, instead of getting a bare 500. This is a FIX (#16664).
[FIRST-PRINCIPLES-REVIEWED] c9c3523 Description read[DESCRIPTION-READ] 5419f07a4b1534186265fe2526be7ad6f75f2138fc909ded1345e22ff8b2b99e That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
…a bare 500 The state record is a projection of the session's crew log, so every session_ledger_record write folds that log before appending to it: record_update -> _fold_checkpoint -> projection.fold_slot_warm -> advance(). advance() refuses seq damage by raising CrewLogError, which subclasses neither ValueError nor OSError, so none of the route's four typed branches named it. An unnamed exception out of an aiohttp handler is answered as a bare "500 Internal Server Error / Server got itself in trouble" - a body with no code, so the caller cannot tell what failed or whether a retry could ever clear it. Local session transcripts show 61 such calls across 20 sessions in one day, interleaved with successes in the same session and uncorrelated with payload size. The route now names CrewLogError and answers 409 crew_log_unreadable, carrying the crew log's own code in the message. The import is inside the handler, not at module scope: this module is on the gateway's boot path and the crew log's storage package is optional behind its own flag, so AUTOSDE's no-new-work-on-gateway-boot-path rule gates the import itself, and the sibling crew log routes import it per function for that reason. That is the answer the sibling work-ledger route already gives for the same refusal on the same store, so this adds no new vocabulary. 409 rather than 503: a refused fold is not a busy filesystem, so "try again" would be false. The read route is unaffected and unchanged - read_state folds inside except Exception and answers the empty record, so it cannot leak this. Test drives the real store and the real route rather than a patched raise: it records one entry, appends a byte-identical copy of the newest record so the last seq appears twice, drops the warm fold so the damage is met by a fold that reads the file, and asserts a coded 409. It fails on the base commit with CrewLogError escaping the handler. Fixes #16664
4c35d3a to
c9c3523
Compare
Manual verification: pod outputPod built from this branch at Part A: the pod's own running gateway, over its real socket
Both refusals are typed and carry a code; neither is a 500. The pod's own process cannot be driven all the way into the new branch from Part B: the real route over a real socket, pod interpreter and pod storeRun against the pod's data home with the branch's own interpreter. The request Step 1 is a control and it earned its place. A second run that reused the same Part C: the boot-path gate
This module is on the gateway's boot path and the crew log's storage package is Remaining red
|
iamwhatever
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: fix (2 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: session_ledger route catches CrewLogError (subclass of neither ValueError nor OSError) and answers 409 crew_log_unreadable instead of a codeless aiohttp 500, matching the sibling work-ledger route; 1 handler + 1 test file.
Problem / Motivation
Goal: an agent whose session-ledger write is refused is told what refused it
and whether a retry can help, instead of losing the update to an opaque 500.
session_ledger_recordanswers a bare, codeless HTTP 500:Measured in local session transcripts: 61 calls across 20 sessions, all inside
one day, 01:43:41Z to 19:14:15Z. The failures are explained by neither the
caller nor a dead gateway:
and 4 failures; another alternates OK -> 500 -> OK -> 500 inside 22 minutes.
succeeding 402-3079.
typed
event: exceeds max length 2000refusal. The caller trimmed to 1976and the next call 500'd, so the failure is downstream of validation.
Server got itself in troubleis aiohttp's default body for an exception thatleft the handler unhandled.
Why it matters
A session ledger update is what survives context compaction, so losing one
costs an agent its own record of where the work stands.
The answer carries no
code, so the caller cannot tell what failed or whetherretrying could ever help. The transcripts show the consequence: most affected
sessions hold exactly two of these failures - call, retry once, give up. The
update is gone, and the next cycle resumes from a record that never received
it.
The route's stated contract is typed answers. A crew-log refusal is the failure
class a crew-log-backed record is most likely to meet, and it was the one class
that escaped untyped.
Not a goal
surfaces as a server fault rather than being silently relabelled as a
storage refusal.
IndeterminateAppend, the crew log's other bareExceptionsubclass. The ledger hands its append to the writer job rather than running
it in the caller's frame, so it is not reachable from this route on the
evidence here, and a branch for an unreproduced path would be speculation.
advance()refuses it.
What changed (motivation -> approach -> change)
Symptom to root cause to change:
folds that log before appending to it:
record_update->session_ledger._fold_checkpoint->crew_log.projection.fold_slot_warm->advance().advance()refuses a log whose seq does not advance - the damage anappend-only writer cannot itself produce - by raising
CrewLogError(
src/kiro_crew/crew_log/errors.py:91).CrewLogErrorsubclassesExceptiondirectly, notValueErrorand notOSError, so none of the route's four branches named it:LedgerUnavailable409,LedgerEntryTooLarge413,ValueError400,OSError503.itself, which is the bare 500 above.
The change:
api_session_ledger_recordnamesCrewLogErrorand answers 409crew_log_unreadable, carrying the crew log's own code in the message.The approach was chosen over the alternatives because it is already the
repository's answer for this refusal. The sibling ledger route over the same
store does exactly this:
So no new status, code or vocabulary is introduced. 409 rather than 503 because
a refused fold is not a busy filesystem, so "try again" would be false; rather
than 400 because the request is well formed and nothing the caller sends
changes the answer.
Backwards compatibility
Compatible: the new branch is reached only where the route previously raised
through to aiohttp's 500 handler, so no request that succeeds today changes its
answer, and the four existing typed branches are untouched and ordered as
before.
Removes nothing: the diff adds one
exceptclause and one import. It deletesno key, record, validator or public function, hides no path, and prunes no
data. The only behaviour it replaces is an unhandled crash, which no caller can
hold a contract on - a 500 with an empty body carries no field or code for a
reader to depend on.
Tests
One test added,
test_route_record_answers_a_typed_refusal_when_the_crew_log_refusesin
test/test_session_ledger.py, driving the real store and the real routerather than a patched raise:
test cannot pass by the write never working at all;
twice, which is the case
advance()refuses withbad_data;rather than answering from its cached cell;
code == "crew_log_unreadable", andbad_datain the message.It locks in that a crew-log fold refusal reaches the caller as a code, and that
the code is the crew log's own.
RED on the base commit, before the source change:
That exception escaping the handler is the reported bare 500.
GREEN with the fix, with the two files that own this path run whole:
Gates with the worktree's own
.venv/bin/tools:blackclean,mypySuccess: no issues found in 1 source file, comment-history gate passed afterits own selftest (
21 flagged probes, 15 clean probes).Manual verification
Verified on a pod built from this worktree: the real route, over HTTP, against
a real crew log damaged the same way the test damages it. Output in the PR
comments.
The read route needs none and gets none:
session_ledger.read_statefoldsinside
except Exceptionand answers the empty record, soGET /api/session-ledgercannot reach this path.Related Issues
Fixes #16664
Pattern harvest
Rule candidate: review-prompt
Pattern: a handler enumerates exception types by the module it calls directly,
missing the refusal type of the store that module reads through - so the one
failure class most native to the backing store is the one that escapes untyped.
The generalizable check is cheap to state: for each
exceptladder in adashboard handler, list the exception types every transitive callee raises, and
flag any that subclass
Exceptiondirectly rather thanValueError/OSError.Here the sibling route over the same store already had the branch, so the
divergence between two handlers on one store is itself the smell. Same class as
#7790, fixed earlier for a different component.
Checklist