Repository navigation
feat(decisions): strip shows what was sent and how long Jev took - #12245
Conversation
The shipped decision strip opened onto four counts, and three of them told the reader nothing. `Selection rounds` and `Skills cut from the offer` printed 0 on every turn -- the menu is asked in ONE question, and the history budget ships at 0 -- and `Context sent (characters)` counted only the history, so the row read 0 while a message excerpt had in fact left the machine unmentioned. So the card now names the egress as the two halves it actually has, and adds the one measurement it was already carrying and not printing: * `Sent to Jev - message 96 chars / history 1,800 chars`. `message_chars` is a new field on the outcome record, measured by `message_chars()` over the same `message_excerpt()` that `build_state_rows` sends, so the number on the receipt cannot overstate what left or miss the 2,000-char cap. * `Jev latency 197 ms` on the expanded card, and `(0.81, 197 ms)` in the collapsed parenthetical beside the score. No new field: `latency_ms` is one of the six core fields `log.build_row` stamps, and the record published to the strip IS that row, so the reader keys on the log's own spelling rather than a duplicate the point would have to reconcile. * the two always-zero rows are gone, and the record stops emitting `truncated`. Clipping is a fact about the transcript READ, which the call row still carries; a strip row for it had no label that said so. The record contract moves on both sides in one change: the spec's fenced fixture in `docs/system-specs/modules/decisions.md` SS 8 is what `decisionStripContract.test.ts` parses, so the gateway half and the reader half cannot drift. The spec's no-message-text bound now says explicitly that a length is not the text, and the contract test proves `"message"` itself is still absent. Tests extend the existing suites rather than adding files: the excerpt/count identity through `build_state_rows`, the outcome row's fields and the three absent ones, the reader's floors, both single-measurement cases of the collapsed parenthetical with the legend each one gets, and the expanded rows. The seven new i18n keys are translated across all eleven catalogs; three keys go.
ScreenshotsCaptured with Expanded — the card this PR is aboutThe three rows that printed light: dark: Collapsed — the score now carries the latency beside it
diverged, light: agreed, light: diverged, dark: agreed, dark: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify old records with nonzero Suggestions
[DESIGN-REVIEWED] 45e9a7c |
UX Review (Fable 5) — ✅ PASSUX-level review of UX-Verdict: PASS Verify legacy records (stamped before [UX-REVIEWED] 45e9a7c |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsFINDING -- docs/system-specs/modules/decisions.md:102 -- False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Verify the one residual: an owner who raised What this change shipsInventory (10 items) — 10 justifiedIntent: make the decision strip's receipt truthful — print the real egress and latency instead of three rows that structurally printed 0 — a FIX (defect provenance is checkable on base:
Checks run: [FIRST-PRINCIPLES-REVIEWED] 45e9a7c |
|
[operator: iamwhatever] The four failureseach asserting Cause: two PRs collided mid-air
On Windows the early Reproduction, with nothing from this diffDetached worktree at plain Same four tests, same four messages as CI. The platform flag is the whole difference. Why it is not fixed hereThis diff touches |
|
[operator: iamwhatever] The Windows red above now has a fix in flight: #12260, against |
|
[operator: iamwhatever] UX Review — CONCERNS: upheld and fixed in
Correct, and it is the defect this PR exists to remove, reintroduced one field over. The row it replaced was merely uninformative; this one would have been wrong, which is worse. The fix, and the one place it does NOT apply
The middle case matters more than it looks: the shipped history budget is VerificationRevert-verified in both directions, since this is two guards and not one: The spec's § 8 count rule now states the exception, so the fenced fixture and the reader cannot drift on it — Green locally on the new head: 58 strip tests, Still outstanding, and still not this diff
|
74babc2 to
6c8474c
Compare
|
[operator: iamwhatever] Screenshots re-captured for expanded, light: expanded, dark: collapsed diverged, light: collapsed agreed, light: collapsed diverged, dark: collapsed agreed, dark: |
|
[operator: iamwhatever] UX Review — CONCERNS on Watch — "Jev" is an unidentifiable recipient
The product already says exactly that, on the gate the reader must pass before any record can exist.
That is the identity and the egress, on the consent card, beside the endpoint address it is consent to. A And the row is not where "Jev" first appears on this card. Six strings on the same strip use the bare noun, five of them predating this PR:
A legend on Worth naming the method limit too: the blind reader could not place "Jev" because they were shown one expanded card and not the consent flow that precedes it. That is a fact about the review, not about the product. Suggestion — "chars" should be "characters"
Spelling it out on this row alone would make it the only surface in the product measuring the same quantity by a different name. Consistency is the stronger property here, and the row is expanded-only so nothing is being saved by brevity — it is being kept the same word. (The removed label did say "characters", but it also said "Context sent", which is the phrasing the owner called confusing. Its wording is not a baseline to preserve.) Watch from the previous round, for the recordThe earlier CONCERNS — "pre-upgrade records will read message 0 chars" — was upheld and fixed in this same round, with its own disposition. The lane's own suggested remedy is what shipped. State
|
Three review findings, all upheld, all in the reviewers' favour. ## The egress row claimed zero for records that never stated one The UX lane caught the new row reintroducing the defect this change exists to remove. `asCount` folds "absent" into `0`, and the row was drawn unconditionally -- so every record stamped before `message_chars` existed rendered "Sent to Jev - message 0 chars", asserting that nothing left the machine on a turn that did send an excerpt. The row it replaced was merely uninformative; this one would have been wrong. `message_chars` is now the one count whose absence is NOT zero. Zero characters is not a credible measurement of it -- a turn that reached the selector had text -- so the reader returns `null` for a value that is not a count, and the strip draws no egress row at all, exactly as it draws no latency row without a `latency_ms`. `history_chars` keeps the floor-to-zero rule in full, deliberately: 0 there is a real observation, and it is how "no history was reachable" is told apart from a field that was never sent. That asymmetry is the whole point, so it is pinned from both sides -- an absent message length hides the row, and a history of 0 must not. The shipped history budget is 0, which makes "history 0 chars" the COMMON record rather than an edge case, so gating the row on the wrong half would have hidden it on every turn. ## The row's two halves were joined by a hardcoded separator `fmtList` now owns that join, as it already did for the collapsed parenthetical above: what goes between two list items is a locale's decision, and a hardcoded separator sat directly beneath a comment saying exactly that. Pinned by asserting the JOINED string -- two separate substring assertions pass under any separator, so they could not have caught it. ## The message-leak ratchet was narrowed so the new field would fit `"message" not in json.dumps(row)` became `'"message"' not in ...`, and the frontend's forbidden list dropped `message` for a quoted form. That is a ratchet loosened so a diff fits: a `message_text` field carrying the conversation itself would have passed both. The field is a LENGTH, so it is removed by exact key before the check instead, and the patterns go back to what they were. Admitting a known-safe field costs the assertion no width that way; narrowing the pattern cost it all of it. Both sides now carry the teeth as an assertion of their own, so masking by key cannot drift back into masking by pattern. ## Verified by mutation, not by reading render gate relaxed 1 test fails reader's asCount restored 5 tests fail hardcoded separator reinstated 1 test fails a message_text leak injected backend exit=1, frontend exit=1
6c8474c to
45e9a7c
Compare
|
[operator: iamwhatever] GPT F1 — upheld and fixed in
Correct on every point, including the remedy. I narrowed a leak guard so my own field would fit it, which is exactly what that rule exists to catch. The fix is the one the finding names. What changedThe field is a length, so it is removed by exact key before the check, and both patterns go back to what they were: assert isinstance(row["message_chars"], int), "a length, and only a length"
without_length = {key: value for key, value in row.items() if key != "message_chars"}
assert "message" not in json.dumps(without_length), "no conversation text in a row"const { message_chars: _length, ...withoutLength } = fixture
const serialized = JSON.stringify(withoutLength).toLowerCase()
for (const forbidden of ['api_key', 'secret', 'prompt', 'message', 'description', 'content']) { … }Admitting a known-safe field by name costs the assertion no width. Narrowing the pattern cost it all of it — that is the asymmetry I missed. Both sides also now assert the teeth, so masking by key cannot drift back into masking by pattern: assert "message" in json.dumps({**without_length, "message_text": "review this code"})Verified by injecting the leak, not by reading the diffThe claim is that a Also in this pushTwo UX findings, both answered in the disposition above and the one after it: the absent- Green locally on The three CI reds on the previous head, none of them this diff
|
|
[operator: iamwhatever] The inherited Windows red is gone, and no rebase is needed. |
bolichen97
left a comment
There was a problem hiding this comment.
Reviewed against the pr-triage-pipeline gates plus a full-diff read.
feat(decisions), 22 files. Reworks the decision strip that shows how the skill selector chose skills against the word-match baseline. It drops two rows that always printed 0 (selection rounds/batches, skills cut/truncated) and the mislabeled single 'Context sent (characters)' row, replacing them with a real egress row (message-excerpt chars plus history chars) and a Jev latency row. Backend adds message_excerpt()/message_chars() and stamps message_chars on the outcome record, dropping truncated; latency_ms is read from the existing core log row rather than newly stamped. Frontend reads message_chars as nullable via asCountOrNull so absent is not 0, gates the egress row on a stated message length, and joins list parts via fmtList. Plus 7 new i18n keys and 3 removed across 13 catalogs.
Why I am approving despite feat: being outside the automated safe-category set: no correctness defect was found. The nullable handling is right (absent, negative, NaN or string all resolve to null and suppress the row, while an explicit 0 is kept), message_excerpt and message_chars share one cap so the receipt cannot overstate egress, None text is handled, and the leak ratchet is asserted by exact-key removal plus a positive message_text-leak assertion on both backend and frontend.
One intended behavioral consequence, not a defect: a legacy record stamped before message_chars existed now shows neither message nor history in the expanded card, because the standalone history row was removed and the egress row is gated on a stated message length. Practically moot while the shipped history budget is 0.
All six security-checklist answers NO, no denylist match, CodeQL PR-scoped open-alert query empty, Semgrep success, readiness label applied after the head commit.
Spec ride-along: docs/system-specs/modules/decisions.md changed as a minority of the diff (1 file / 10 lines against 21 files / 563 lines) and was not reviewed as a design decision.
…odotdev#12245) * feat(decisions): strip shows what was sent and how long Jev took The shipped decision strip opened onto four counts, and three of them told the reader nothing. `Selection rounds` and `Skills cut from the offer` printed 0 on every turn -- the menu is asked in ONE question, and the history budget ships at 0 -- and `Context sent (characters)` counted only the history, so the row read 0 while a message excerpt had in fact left the machine unmentioned. So the card now names the egress as the two halves it actually has, and adds the one measurement it was already carrying and not printing: * `Sent to Jev - message 96 chars / history 1,800 chars`. `message_chars` is a new field on the outcome record, measured by `message_chars()` over the same `message_excerpt()` that `build_state_rows` sends, so the number on the receipt cannot overstate what left or miss the 2,000-char cap. * `Jev latency 197 ms` on the expanded card, and `(0.81, 197 ms)` in the collapsed parenthetical beside the score. No new field: `latency_ms` is one of the six core fields `log.build_row` stamps, and the record published to the strip IS that row, so the reader keys on the log's own spelling rather than a duplicate the point would have to reconcile. * the two always-zero rows are gone, and the record stops emitting `truncated`. Clipping is a fact about the transcript READ, which the call row still carries; a strip row for it had no label that said so. The record contract moves on both sides in one change: the spec's fenced fixture in `docs/system-specs/modules/decisions.md` SS 8 is what `decisionStripContract.test.ts` parses, so the gateway half and the reader half cannot drift. The spec's no-message-text bound now says explicitly that a length is not the text, and the contract test proves `"message"` itself is still absent. Tests extend the existing suites rather than adding files: the excerpt/count identity through `build_state_rows`, the outcome row's fields and the three absent ones, the reader's floors, both single-measurement cases of the collapsed parenthetical with the legend each one gets, and the expanded rows. The seven new i18n keys are translated across all eleven catalogs; three keys go. * fix(decisions): never claim zero egress, and keep the leak ratchet broad Three review findings, all upheld, all in the reviewers' favour. ## The egress row claimed zero for records that never stated one The UX lane caught the new row reintroducing the defect this change exists to remove. `asCount` folds "absent" into `0`, and the row was drawn unconditionally -- so every record stamped before `message_chars` existed rendered "Sent to Jev - message 0 chars", asserting that nothing left the machine on a turn that did send an excerpt. The row it replaced was merely uninformative; this one would have been wrong. `message_chars` is now the one count whose absence is NOT zero. Zero characters is not a credible measurement of it -- a turn that reached the selector had text -- so the reader returns `null` for a value that is not a count, and the strip draws no egress row at all, exactly as it draws no latency row without a `latency_ms`. `history_chars` keeps the floor-to-zero rule in full, deliberately: 0 there is a real observation, and it is how "no history was reachable" is told apart from a field that was never sent. That asymmetry is the whole point, so it is pinned from both sides -- an absent message length hides the row, and a history of 0 must not. The shipped history budget is 0, which makes "history 0 chars" the COMMON record rather than an edge case, so gating the row on the wrong half would have hidden it on every turn. ## The row's two halves were joined by a hardcoded separator `fmtList` now owns that join, as it already did for the collapsed parenthetical above: what goes between two list items is a locale's decision, and a hardcoded separator sat directly beneath a comment saying exactly that. Pinned by asserting the JOINED string -- two separate substring assertions pass under any separator, so they could not have caught it. ## The message-leak ratchet was narrowed so the new field would fit `"message" not in json.dumps(row)` became `'"message"' not in ...`, and the frontend's forbidden list dropped `message` for a quoted form. That is a ratchet loosened so a diff fits: a `message_text` field carrying the conversation itself would have passed both. The field is a LENGTH, so it is removed by exact key before the check instead, and the patterns go back to what they were. Admitting a known-safe field costs the assertion no width that way; narrowing the pattern cost it all of it. Both sides now carry the teeth as an assertion of their own, so masking by key cannot drift back into masking by pattern. ## Verified by mutation, not by reading render gate relaxed 1 test fails reader's asCount restored 5 tests fail hardcoded separator reinstated 1 test fails a message_text leak injected backend exit=1, frontend exit=1 --------- Co-authored-by: Joe Guo <zejiangg@amazon.com>












The ask
The shipped decision strip opens onto four counts. Three of them tell the reader nothing:
Skills offered 44Selection rounds 0Context sent (characters) 00Skills cut from the offer 00budgetSo a card whose whole job is "here is what this decision cost and what left this machine" answered
0 · 0 · 0, while a 2,000-char message excerpt had in fact left, and the latency was already on the row and simply not printed.What this changes
One row for the egress, with both halves.
Sent to Jev — message 96 chars, history 1,840 chars, the two halves joined byfmtListbecause what goes between two list items is a locale's decision.message_charsis one new int on the outcome record, measured by a newmessage_chars()over the samemessage_excerpt()thatbuild_state_rowssends — one function, so the number on the receipt cannot overstate what left or miss the 2,000-char cap.The latency, which needed no new field.
Jev latency 197 mson the expanded card and(0.81, 197 ms)in the collapsed parenthetical beside the score.latency_msis one of the six core fieldslog.build_rowstamps, and the record published to the strip is that row — so the reader keys on the log's own spelling instead of a duplicate the point would have to keep in step. A record without one prints no row rather than0 ms, which would read as an answer that arrived instantly.The two always-zero rows are gone, and
build_outcomestops emittingtruncated. Clipping is a fact about the transcript READ, which the call row still carries; on the strip it had no label that said so.The contract moved on both sides in one change
docs/system-specs/modules/decisions.md§ 8 holds the fenced record fixture thatwebsite/src/test/decisionStripContract.test.tsparses — so the gateway half and the reader half cannot drift apart silently, which matters here because the reader's fail-safe is to draw nothing and that is byte-identical to a release that stamps no record at all.One bound in that section needed care: it says the record carries no message text, and the contract test enforced it by asserting the serialized fixture contains no
message. A length is not the text, so the test now forbids"message"with its quotes and assertsmessage_charsis present and numeric — the narrower key cannot smuggle the wider one back in.Tests
Extended in place, no new files.
sel.message_chars/sel.message_excerptacross empty, short, at the cap and over it (parametrised on the LENGTH — a multi-kB parameter value gets baked into the pytest node id), plus the identity throughbuild_state_rowsso a second cap cannot drift from the one that ships.message_charsis what left,latency_msis an int, andbatches/rounds/truncatedare each asserted ABSENT.message 96 chars,history 1,800 chars,Jev latency, and the two removed labels asserted gone.Gates run locally
test_decisions_*(9 files, 563 tests)DecisionStrip.test.tsx+decisionStripContract.test.ts(54)src/i18n/vitest (42 files, 692)i18n:check(19 checks,I18N_BASE_REFset)typecheck,eslint,lint:theme-colors,lint:phantom-classes,lint:i18ndocs_lint.py,check_feature_map.py,check_comment_history.py,check_brand_name.py,check_focus_cue.py,check_harness_parity.py,check_subprocess_encoding.pyblack --checkon the three Python filesSeven i18n keys added and translated across all eleven catalogs plus the regenerated pseudolocale; three keys removed.
check_black_formatting.pycould not run on this host (it pins Black 26.3.1, the host has 26.5.1) — plainblack --checkon the changed files reports them unchanged.Screenshots
Captured with
website/scripts/capture-decision-strip.mjsagainst the real built SPA (website/dist) behind the shared transcript harness — every/api/**call answered from fixtures, soreadDecisionStrip, the consent gate, the transcript virtualizer and the strip itself render exactly as in production. Light and dark, English pinned.Expanded — the card this PR is about
The three rows that printed
0forever are gone. In their place:Sent to Jev — message 96 chars · history 1,840 charsandJev latency 197 ms.light:
dark:
Collapsed — the score now carries the latency beside it
(0.81, 197 ms)on the diverged line,(0.94, 142 ms)on the agreed one. One parenthetical, because both numbers describe the same answer and the line truncates.diverged, light:
agreed, light:
diverged, dark:
agreed, dark: