Skip to content

feat(decisions): shadow-score auto-compaction with Jev and show what it would keep - #12496

Merged
iamwhatever merged 1 commit into
mainfrom
feat/decisions-compaction-shadow
Sep 21, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
feat/decisions-compaction-shadow

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

What this does

At every automatic compaction, Jev is asked — per tool call — whether the call and its result were worth keeping, and the answer is shown as one line on the compaction notice:

✂ Jev would keep 23 of 61 tool calls · 41% of the characters · 👍 👎

Nothing is kept. There is no apply path. The compaction runs byte-identically on every backend whatever Jev answers, and this PR's only product effect is one extra line on a notice plus rows in the decision log.

Why shadow rather than live

Nothing in Kiro Crew can replace a harness's message list — LLMProvider has no set-messages call and /compact summarizes inside the harness process. The one place a keep-list could land is the transcript Crew re-injects after a recycle (context.build_session_replay), and that replay already admits user / assistant / inject rows only, so a live keep-list could only displace text inside the same 80 000-character budget. Whether that trade is worth making is a question about data nobody has. This produces the data: one row per compaction carrying what the transcript held, what today's replay keeps, and what Jev's answer would have kept.

Manual /compact is excluded by construction

The dashboard dispatches that command as an ordinary turn through provider.stream_command("/compact") and never calls the coordinator. test_decisions_compaction_hook.py::TestManualCompactIsExcluded pins it by asserting no caller of _compact_session exists outside the session layer, so the exclusion cannot be lost by a later edit without a red test.

It cannot cost the compaction

The hook is at the top of CompactionCoordinator._compact_session — the one funnel both automatic entry points reach (check_context_usage → _trigger_compaction, and compact_if_needed) — above the Claude arm, _recycle_unmanaged and _compact_in_place, so every backend is covered.

The task is created and never awaited. test_the_outcome_is_identical_with_and_without_the_hook runs one scripted compaction twice and holds the outcome and arm equal; test_a_scoring_that_never_finishes_does_not_delay_the_compaction runs it against a scoring that hangs for an hour and asserts the compaction completes with the scoring still outstanding.

Which compactions are scored

"Every backend" would be false, so the docs name the population exactly. The hook sits inside _compact_session, which means it covers the compactions Crew itself dispatches — and a harness that compacts on its own initiative is declined by _compaction_gate_decision before that funnel is ever reached.

Compaction shape Scored?
Threshold auto-compaction Crew dispatches (/compact on kiro-cli, codex, opencode) yes
The awaited between-turn check yes
A recycle — the /compact failed, or the backend has none (deepseek) yes
Claude-Code-managed sessions (cc_managed) no — Claude compacts in-prompt; Crew never dispatches it
A harness that summarizes unasked (KAS, ACP_BACKENDS_HARNESS_MANAGED_COMPACTION) no — it compacts and reports it; no Crew-side dispatch exists
Manual /compact no — dispatched as an ordinary turn
Any other gate decline (below threshold, unconfirmed, in progress, cooldown) no — _compact_session is never scheduled

The two harness-managed rows are not gaps to close from here: measuring either would mean a different hook on a different event, and the live design (#12516) targets the recycle path, which is exactly what this hook already covers.

What leaves the machine

Part of the transcript Share of characters (measured, 206 real sessions) What is sent
Conversation text 6.4% verbatim, cut only at the aggressive fitting rungs
Tool inputs 33.4% redacted then clipped at the fitting rung
Tool results 53.2% never — replaced by ok, N chars (omitted)

Redaction runs over each field in full and the clip is taken after it, because a cut placed first halves a secret into a fragment neither redactor matches. A redactor that fails yields the empty string rather than the input, and the gate's own scan still refuses the whole request for a spelling the pass missed. Registered as a sink in security_posture.py.

Fitting, and the ~8% that never fits

Five rungs from the P3 spike — inputs at 1000, then 200, then 60 characters, then the conversation text to head+tail halves, then to a quarter with each call on one line. The first rung under ~25K estimated tokens is sent, so an ordinary session pays the mildest one. The rungs admit 25% / 67% / 80% / 88% / 92% of measured sessions; the rest write a state-too-large row and send nothing, so the population that cannot be measured is visible rather than silently absent.

One logical tool call reaches the transcript as up to three role="tool" rows sharing one tool_call_id — the dispatch row, plus the approval path's own approved/rejected row — so the rows are folded by that id before anything is counted. A per-row count inflated total_calls, sent the same input twice in the state, asked two questions about one call, and counted its characters twice in the denominator. Each field of the fold takes the first non-empty value, pinned is the OR across the rows, and a row carrying no id gets a key of its own because two rows that name no call cannot be shown to be the same call. The fold finishes before the cap, so the cap counts calls rather than rows.

Both character arms are upper bounds, and say so

chars_today_eligible counts the conversation rows today's recycle replay admits by role. build_session_replay then applies per-role row quotas and an 80 000-character budget scaled by the model window, so a long session is replayed with less than this arm reports; chars_jev_eligible inherits the bound because it seeds from that arm. So the names carry _eligible and the card says up to 41% of the characters.

The true figure is deliberately not re-derived here. The quota walk takes a ConversationLog rather than a row list, and the budget is applied inline while formatting — so the only thing available to measure is a rendered string whose length carries Role: prefixes and separators, which would put formatting overhead inside a content measurement and is large enough to break the <= chars_all invariant. Re-implementing the selection in this module would be a second copy of it, free to disagree with the one that runs. Naming the bound is the honest option; the alternative was a number that quietly overstates.

A third consent scope

compaction on the keystone, beside tool_args. Absent reads as not consented, an omitted PUT field preserves, disabling clears.

tool_args does not grant it, and that is the whole reason for a third leaf: that scope was reviewed as the arguments of the one call about to run; this sends the conversation and every tool input the session has accumulated, in a request one to two orders of magnitude larger. An owner who granted either of the other two is inert here.

gate._tool_args_scoped becomes _scope_consented and drives both scopes off one table (_POINT_SCOPES), so a third scope is not a third predicate to forget. No new endpoint — same PUT /api/decisions/consent, and the thumbs post to the existing POST /api/decisions/feedback.

The card grows a third switch under the tool-argument one, visible only while the main switch is on: Also send the conversation and tool-call inputs so Jev can score compaction. Its description says the compaction is unchanged, that tool output is never sent, and that it starts off even for an owner who already granted the other two.

Partial answers publish nothing

Questions are batched 25 per request with the same state each, 4 concurrent, the whole run under the shared 10s ceiling. A batch is all-or-nothing, and if any batch refuses the outcome row is written with partial-answers and no card line is published — 23 of 61 over a denominator including calls nobody was asked about is a wrong number rather than an incomplete one. The frontend reader independently refuses a record whose three tallies do not sum to total_calls.

A walk that hits its 1 000-call cap is a different case and is not withheld: it keeps the newest 1 000 calls in chronological order, counts the ones it dropped as calls_truncated, and the line states that count — Jev would keep 23 of 61 tool calls (+140 not scored) — because a bare “23 of 61” is wrong when 61 is the cap rather than the session’s call count, and drawing nothing hides a measurement that is accurate about everything it covers. The newest end is chosen rather than fallen into: a forward walk that simply stops admitting keeps the oldest calls, which are the least likely to still matter and are not what the pinned tail is chosen for. The dropped calls’ characters stay out of both arms, so the two remain one universe.

Screenshots

Captured by website/scripts/capture-decision-compaction-keep.mjs against the real built SPA with only the network stubbed — no gateway, no agent, no Jev call. Fourteen frames, one card per frame, each its own harness instance and its own fixture.

The previous set was a picture of a folded turn

Worth stating plainly, because the fix is a lesson about what a DOM assertion proves. TurnBlock wraps a turn's INTERIM rows in a CollapsibleSection rendered at height: 0; opacity: 0 until it is expanded, and the fixture put a message after the compaction row — which made the compaction row interim. The card contributed no pixels at all. So two scenes whose DOM provably differed (23 of 61 vs 78 of 148, 85% vs 93%, the overflow phrase present vs absent) wrote byte-identical files, however the frame was clipped and whichever capture path took it. Every "capture strategy" agreed because none of them was photographing the card.

Every assertion in the old harness passed throughout: they all read textContent, and a folded card's text is intact.

Two changes:

  1. The compaction row is the turn's last row, so no fold wraps it — which is also what a reader sees the instant a compaction lands.
  2. Each shot asks the browser whether its subject is on screen: no ancestor at zero height or zero opacity, and document.elementFromPoint at the card's own centre must land inside the card. The harness also fails if any two frames come out byte-identical, so this cannot recur silently.

Frames went from ~5.8 KB of blank to 19–57 KB. All twelve transcript frames are distinct, and each was matched back to its uploaded attachment by content hash rather than by upload order.

Light Dark
Threshold success notice — 23 of 61 kept, no overflow qualifier
Recycle notice — a session Jev would keep almost none of (7 of 61)
Failure — renders through the shared error surface, 31 of 61 kept
Overflow — calls_truncated: 140, so the line says (+140 not scored) before the share
Summary card collapsed — the one shape with a disclosure, the line outside the fold
The same card expanded — the digest open and scrolling internally, the line still in place
The third consent switch, and the point row it unlocks

Per frame the harness still asserts the theme it is named for, that exactly one compaction card is in the scene, that the captured element contains the keep line, that the notice's own words are inside the captured element (a page-scoped check passed on a frame that had cut the sentence off), and that the line says what makes it that frame — including what it must not say, since a positive check alone cannot tell a frame from the one it is contrasted with.

The withheld pass serves the SAME fixtures with the scope revoked and asserts the switch reads off and the compaction.keep row is absent — "it appears when granted" says nothing about what an ungranted install sees.

Tested

  • test/test_decisions_compaction_keep.py (81) — the transcript walk, the pinning split, the ladder and its monotonicity, redaction-before-clip, no result byte at any stage, batching, the all-or-nothing batch read, the tallies and both character arms, the record store's three bounds, the gate's refusal without the scope, and the transcript boundary (a row appended after the fire is dropped; no boundary reads everything; a missing or unparseable stamp is KEPT; the boundary itself is inclusive; the row cap applies after the filter).
  • test/test_decisions_compaction_hook.py (15) — both automatic entry points, a recycle-only backend, the hang test, a broken point, outcome equality, the absent manual caller, the stamp onto the notice row, and that the boundary the coordinator supplies precedes anything the compaction appends — both boundary tests go red when before= is dropped from the create_task call.
  • test/test_decisions_compaction_scope.py (23) — the keystone reader, writer and route, including that tool_args does not grant it and that the three scopes move independently.
  • website/src/test/CompactionKeepLine.test.tsx (25) — the reader's refusals, the line on every card shape, and the thumbs label.
  • Plus the third switch in decisionsCard.test.tsx and the spec's fenced record held against its reader in decisionStripContract.test.ts.

The 12 locale catalogs and the 11 locale style suites were run with I18N_BASE_REF set, so the [changed-values] rules actually saw the renamed label rather than skipping.

Verified locally: the decisions + compaction + posture + settingref pytest suites (1149 tests), mypy src/kiro_crew/ (1665 files), isort, flake8, black on the new files, npm run typecheck, npm run lint, npm run gen:settings, npm run i18n:check 19/19 PASS, and the diff-scoped gates (brand, comment-history, focus-cue, feature-map, changelog, harness-parity, memory-store, sync-io, subprocess-encoding, agent-sdk-boundary, loop-bound-locks, acp-frame-host-data, testpaths-coverage, docs-lint).

scripts/check_black_formatting.py cannot run on this host — it pins Black 26.3.1 and only 26.5.1 is installed. Every file in the diff was checked against the installed Black instead, and the baseline-exempt files were left unformatted.

Docs

A section in both decisions docs stating exactly what leaves the machine (src/kiro_crew/docs/decisions.md for the owner, docs/system-specs/modules/decisions.md § 11 for the contract), and a feature-map row.

The first push's three reds

All three are fixed in e346d26a98, and two of them were real:

  • Backend Tests (Windows) (5) — the record store's TTL test set the TTL to 0 and relied on time.monotonic() advancing between the publish and the take. Windows' monotonic has ~15.6 ms resolution, so both readings were the same value and the entry was correctly inside a <= 0.0 TTL. The clock is pinned instead, and a companion case asserts an entry inside its TTL is still handed over, so the sweep cannot become "expire everything" and pass.
  • Frontend Tests (2) — ko.md §2.1: a 조사 after an interpolation must be written in both forms, because the particle depends on the final consonant of a value that does not exist until render. {{productName}}가 → {{productName}}이(가).
  • Screenshot Evidence — the capture harness above, including the overflow frame.

Follow-ups

  • decisions: live compaction.keep on the recycle path (consumes the shadow measurements from #12496) #12516 — decisions: live compaction.keep on the recycle path (consumes the shadow measurements from #12496). The decision this measurement exists to inform, so the dataset has a committed consumer rather than being collected against a maybe: option B from the spike (a keep-set threaded into _replay_rows on the recycle path only, inside the unchanged 80 000-character budget), the five things the shadow rows have to show first, the paths it deliberately excludes, and the fact that the consent copy stops being true once anything is applied and needs re-reviewing. Opened in answer to First Principles item 1.
  • feat(decisions): let Jev narrow what the memory_recall tool returns #12492 — owns the FeaturePreviewsSection consent-PUT contract for all scopes: a scope-only PUT carrying neither enabled nor endpoint. That is the server-side half, and it predates this PR — the same shape exists on the tool_args switch that shipped before it. The client-side half for this PR's own switch is fixed here: its pending write freezes the master switch, and a refused write renders the shared ErrorNotice. The equivalent guards for tool_args are deliberately not added, because feat(decisions): let Jev narrow what the memory_recall tool returns #12492 removes the need for them rather than duplicating them.

Not in this PR

The live keep-list. _replay_rows is untouched, so the replay is byte-identical to today — that change is #12516, and this PR's data is what decides it.

@iamwhatever
iamwhatever requested review from a team and hoang-phan98 September 21, 2026 09:10
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 120219bfeba3cebf7f2317ca6a8465271ab09a36 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the App bundle ceiling bump: main alone is already 1,712 B over budget, so this PR absorbs another PR's drift into its own gate change.

Suggestions

  • Split main's 1,712 B of pre-existing App-chunk drift into its own chore re-baseline PR — as written, check-bundle-size.mjs's comment says every open PR currently fails that gate, and burying the re-baseline inside a feature PR hides that fact from the maintainers who should fix it once, centrally.

[DESIGN-REVIEWED] 120219b

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — 🟡 CONCERNS

UX-level review of 120219bfeba3cebf7f2317ca6a8465271ab09a36 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

The keep line's numbers confuse a cold reader: the "(+140 not scored)" total is ambiguous, and "would keep" beside a failure notice reads as contradiction.

Watch

  • "(+140 not scored)" (shot-07): the reader guessed the meaning but said "I genuinely don't know … whether 148 includes them" — every capped session shows this, persistent mild confusion that also undermines the thumbs rating ("makes me even less able to rate fairly"). Smallest fix: make the relation explicit, e.g. ({{count}} more not scored) in pages.chat.compactionKeep.not_scored.
  • "Jev would keep 7 of 61" under the restart-failure notice (shot-03): reader — "I can't tell what those 7 refer to if nothing was compacted", and (blind-read "Could not tell") the chat line "sounds like it describes what happened" while settings call it a measurement. Hits every opted-in user on every failed compaction; friction not task failure, since the reader did reach the right reading. Worth a human judging whether the failure-card variant needs a hypothetical cue beyond "would".

Suggestions

  • pages.chat.compactionKeep.not_scored: "(+{{count}} not scored)" → "({{count}} more not scored)" — removes the include/exclude ambiguity the blind reader named.

[UX-REVIEWED] 120219b

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 120219bfeba3cebf7f2317ca6a8465271ab09a36 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Confirm #12516 is a real, live design effort: this feature's whole reason to exist is feeding that decision, and its cost (up to ~40 judge requests per compaction) is paid now.

What this change ships

Inventory (10 items) — 10 justified

Intent: Let a consenting owner see, at each automatic compaction, which tool calls an oracle would have kept — producing the data the live keep-list design (#12516) needs — without changing any compaction. ADDITION.

  1. Third opt-in consent switch (send conversation + tool inputs for compaction scoring), off even when the other two are on — justified
  2. Each Crew-dispatched auto-compaction is shadow-scored by Jev in the background; the compaction itself is unchanged on every backend — justified
  3. Compaction notice gains one "Jev would keep X of Y · up to Z%" line with thumbs, riding the existing decisions_strip field and feedback endpoint — justified
  4. Decision log gains per-compaction rows, including state-too-large / partial-answers / run-timeout outcomes — justified
  5. Consented installs now send conversation text and redacted tool inputs (results never) to the judge; registered as a security_posture sink — justified
  6. Consent PUT accepts a scope-only body (enabled omitted); the existing tool-args switch now writes that way, closing the stale-view re-grant race at its cause — justified
  7. App bundle ceiling 3533→3538 KB, absorbing main's own measured 1,712 B overrun that fails every open PR's gate — justified
  8. New settings-registry/command-palette entry plus strings in 14 locales (i18n gates) — justified
  9. Owning spec, user docs and feature-map updated in the same commit; screenshot capture script matching the repo's 576-sibling pattern — justified
  10. A transcript that never fits (~8%) or answers partially shows no line; the log row says why, so the unmeasured population stays visible — justified

[FIRST-PRINCIPLES-REVIEWED] 120219b

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings (all downgraded on adjudication)

GPT 5.6 flagged blocking issues on 120219bfeba3cebf7f2317ca6a8465271ab09a36; Opus 4.8 adjudication downgraded every one of them to advisory. Adjudication downgraded all 1 blocking finding(s) to advisory: the remedy each one requires is disproportionate to the harm it prevents. Read them as advice, not as merge conditions.

This comment is updated in place on each push.

Review details

BLOCKING -- src/kiro_crew/decisions/points/compaction_keep.py:459 -- The row cap reports a truncated tail as the whole transcript
return usable[-MAX_ROWS:]
A transcript exceeding 4,000 small rows -> read_rows drops older tool calls -> build_transcript cannot count them, so the log and UI underreport calls_truncated.
Anchor: a-bound-bounds-every-field-it-retains
Fix: Refuse scoring on row overflow, or propagate the omitted count into the outcome and UI qualifier.
[BLOCK-MERGE-DOWNGRADED] 120219b
[GPT-REVIEWED] 120219b

Adjudication (Opus 4.8) — is blocking on each finding proportionate?

F1 — compaction_keep.py:459 return usable[-MAX_ROWS:].

Conditions: a session transcript must exceed MAX_ROWS=4000 rows (compaction_keep.py:197, :459), so read_rows drops the oldest rows before build_transcript ever sees them; calls_truncated (:704, truncated = max(0, len(folded) - MAX_CALLS)) then counts only the calls dropped by the MAX_CALLS fold, not those lost to the row cap, so the "+K not scored" qualifier underreports.

Recovery/effect (:566-577, :718-724): this is a shadow-scoring/observability decision point — the count is an informational qualifier on a card and a log line, not an input to the actual compaction. The score remains self-consistent within its declared universe (the newest rows, which the docstring at :438 states is deliberately the subject). No system state, data, or governance is affected; the metric is visible and re-derived on the next compaction.

Harm rung: LOW — an informational undercount on an experimental "what it would keep" surface, only when a transcript exceeds 4000 rows, self-consistent and non-persistent. The real fix threads a new omitted-row count across read_rows → build_transcript → Transcript → UI, adding a field and cross-module contract that exists solely for this rare diagnostic-accuracy edge; the permanent maintenance load clearly exceeds a display qualifier being conservative.

[ADJUDICATION] 120219bfeba3cebf7f2317ca6a8465271ab09a36 total=1 uphold=0 downgrade=1
DOWNGRADE F1 src/kiro_crew/decisions/points/compaction_keep.py:459 reason=disproportionate-remedy
[GPT-ADJUDICATED] 120219bfeba3cebf7f2317ca6a8465271ab09a36

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 120219bfeba3cebf7f2317ca6a8465271ab09a36: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 120219bfeba3cebf7f2317ca6a8465271ab09a36 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 120219b

Verdict parsed from the review's SHA-scoped output markers for commit 120219bfeba3cebf7f2317ca6a8465271ab09a36.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 120219bfeba3cebf7f2317ca6a8465271ab09a36: <one-sentence reason>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

Screenshot evidence, captured by website/scripts/capture-decision-compaction-keep.mjs against the real built SPA with only the network stubbed. Every frame's own assertions passed (26 checks).

01-compacted-light

01-compacted-dark

02-recycled-light

02-recycled-dark

03-failed-light

03-failed-dark

04-consent-light

04-consent-dark

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — 🔴 changes requested (blocking)

GPT 5.6 found at least one blocking issue that must be resolved before merging e346d26a982624dddc6e1ab6ff06a55ee2e945a0. Adjudication produced no [GPT-ADJUDICATED] marker for e346d26: the adjudication stage FAILED TO RUN (its raw output is in the Adjudication details block -- typically an API or setup error), so no proportionality review took place and GPT's blocking verdict stands unreviewed.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/decisions/points/compaction_keep.py:307 -- companion credential patterns bypass redaction

cleaned, _warnings = redact_credentials(raw)
Companion-defined credential token -> transcript scoring -> baseline-only scrub -> credential sent to Jev
Anchor: residual/security
Fix: redact through platform.context.redact_via_context before clipping.

BLOCKING -- website/src/pages/settings/FeaturePreviewsSection.tsx:233 -- scope update can undo consent revocation

api.saveDecisionsConsent(true, view.configuredEndpoint, undefined, value),
Scope toggle followed by master-off -> concurrent PUTs -> late scope PUT restores enabled: true
Anchor: residual/security
Fix: disable the master switch while any scope mutation is pending.

BLOCKING -- src/kiro_crew/decisions/points/compaction_keep.py:449 -- injected rows corrupt character percentages

if role in kept_roles:
chars_today += len(content)
Injected replay row -> numerator includes it while denominator skips it -> displayed share exceeds 100%
Anchor: residual/crash-data-loss-corruption
Fix: include replay-kept injected content in chars_all.

[BLOCK-MERGE] e346d26
[GPT-REVIEWED] e346d26
False positive or not applicable? A repository writer can comment:
/ai-review override gpt e346d26a982624dddc6e1ab6ff06a55ee2e945a0: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@iamwhatever
iamwhatever force-pushed the feat/decisions-compaction-shadow branch from e346d26 to f1c668c Compare September 21, 2026 10:13
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Dispositions for the round-1 and round-2 findings, all on f1c668c61b (rebased onto 1fa7f5ba50, which also clears Fable's stale-base note).

Fixed in this push

GPT F3 / compaction_keep.py:449 — inject rows put the character share above 100% (UPHOLD-FENCED; Opus found the same thing independently at :909). Real, and it was mine. chars_today counted every row the replay keeps — which includes inject — while chars_all counted only STATE_ROLES, so an ordinary cron notification or subagent-completion event put bytes in the numerator that the denominator skipped. chars_all now counts every row either arm counts. chars_today <= chars_jev <= chars_all is stated as the invariant it is and asserted under every one of the three answers with a 5 000-character inject row present (TestOneCharacterUniverse). The reader holds it from its own side too: a share above 1 prints no percentage rather than being clamped, because a clamp would print a plausible number over a broken measurement.

GPT round-2 #1 / compaction_keep.py:307 — companion credential patterns bypassed redaction. Real. The state is a whole transcript and redacted scrubbed it with security.redact_credentials, which knows only the patterns this repository ships. It now routes through platform.context.redact_via_context before the stage's clip. A PlatformCompositionError — which the shim deliberately re-raises so a host that could not compose its companion cannot silently downgrade — is caught here and the field is dropped, the same direction every other arm of that function fails in. The shipped exfiltration-URL pass runs on top. The security_posture sink entry names the route and the drop; the test stubs the shim by behaviour, so a later edit reaching past it fails.

GPT F2 / compaction_keep.py:702 — a straggling scoring attached to the next compaction (machine-FLAGGED as an edge case). Fixed rather than overridden: the fix is a token, not a wider TTL, and it is cheaper than the override would have been to justify. Each compaction mints an attempt token synchronously before the scoring task exists; a run whose token is no longer the session's newest publishes nothing, and minting retires whatever the previous attempt left pending — the rule outcomes.discard applies at the start of a turn. The rows are still written: the observation happened.

GPT F1 / compaction_keep.py:459 — capped calls reported as the complete transcript (adjudicated DOWNGRADE, disproportionate-remedy). The adjudicator's reasoning holds — overflow counting plus admission-time bounding is too much machinery for a shadow diagnostic on a tail that mostly writes ERROR_TOO_LARGE anyway — but the cheap half of it is worth having and is in: the walk sets calls_capped, and both the point and the reader decline to draw a line for a capped record, because "N of M" over an M that is the cap rather than the session's call count is a number a reader cannot tell from a right one. The overflow rows' characters are left out of both arms, so the two stay on one universe. No permanent state and no new branch on the compaction path.

Design Review / UX Review BLOCK — no rendered evidence. Closed on the previous push: website/scripts/capture-decision-compaction-keep.mjs writes eight frames over the real built SPA with only the network stubbed, and they are in the description. The line is photographed on all three compaction shapes, because the scoring starts before the compaction picks an arm and a line placed inside one branch would be absent from the other two. Every frame also asserts the words that make it that state, and the harness's withheld pass serves the same fixtures with the scope revoked and asserts the switch reads off and the point row is gone.

Not fixed here, deliberately

GPT round-2 #2 / FeaturePreviewsSection.tsx:233 — a scope PUT can undo a consent revocation. Real, and not this PR's to fix: #12492 owns it in that shared component, and this branch inherits the fix on its next rebase. Confirmed real while investigating — a scope PUT carries enabled: true, so a scope flip followed by a master-off is two concurrent writes to one keystone and the scope one landing second restores consent the owner just revoked; the route's lock makes each write atomic and says nothing about ordering. The finding predates this PR: the same shape exists on the tool_args switch that shipped before it, which is why the fix belongs in the shared component rather than being written twice.

First Principles CONCERNS — the sizing evidence points at nothing anyone can open. Fair, and worth stating plainly rather than arguing: the STAGES ladder, the 25 K ceiling and the 206-session shares come from a read-only spike run against origin/main at 1e40f7f0f5, whose report and three measurement scripts live outside this repository. The numbers in the spec are therefore claims this PR cannot let a reader verify, and only the shares are load-bearing — the ladder's correctness does not depend on them, because fit_state measures each rung against the ceiling at runtime and TestFitting derives its own ceiling from the rungs rather than from the spike. Whether the live keep-list question gets a recorded owner is a call for the maintainer, not something this PR can assert.

Verified locally on f1c668c61b

598 pytest across the decisions, compaction, posture and settingref suites (and the hook suite passes in both -p no:randomly and shuffled order, after an autouse fixture removed an ordering dependency the attempt counters introduced); mypy src/kiro_crew/ over 1 665 files; isort, flake8, black on the touched files; npm run typecheck, npm run lint, 188 frontend tests over the seven decision/compaction suites; npm run i18n:check 19/19 PASS; and 14 diff-scoped gates including comment_history, which caught three "no longer" phrasings in the new docstrings and one "used to" in a test comment — all rephrased in present tense.

scripts/check_black_formatting.py still cannot run on this host: it pins Black 26.3.1 and only 26.5.1 is installed. Every file in the diff was checked against the installed Black instead, and baseline-exempt files were left unformatted.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 21, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Screenshot evidence for f1c668c61b+ (round 3), captured by website/scripts/capture-decision-compaction-keep.mjs against the real built SPA with only the network stubbed. 12 frames, 38 in-harness checks, all passing. The summary pair is the collapsed/expanded state of the one card shape that has a disclosure; its record carries calls_truncated: 140, which is the conditional "(+140 not scored)" copy.

05-summary-collapsed-light

06-summary-expanded-light

05-summary-collapsed-dark

06-summary-expanded-dark

01-compacted-light

01-compacted-dark

02-recycled-light

02-recycled-dark

03-failed-light

03-failed-dark

04-consent-light

04-consent-dark

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for b670dfd39f47f1e32bb69630dadca7098fb1f3b9; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b670dfd39f47f1e32bb69630dadca7098fb1f3b9: <one-sentence reason>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Round 3 on b670dfd39f. The remaining items from the review, and the one question that needed an answer rather than a change.

compaction.keep keeps its own pending store — three concrete cases, not a TTL preference

The suggestion was to carry the record in decisions/outcomes.py with the attempt id and delete compaction_keep._pending, since the two are the same TTL + cap + destructive-read shape. They are the same shape and different lifecycles, and each of these is a live defect rather than a rarity:

1. One slot per session, and skills.select already owns it. outcomes holds exactly one entry per session key and consume POPS. skills.select publishes into that slot from prompt assembly, and chat_runner._decisions_strip_meta claims it when the reply is finalized. A compaction is not synchronised with that: state.py::_on_compacted runs on the session manager's background task. So a turn that published a skill strip and has not yet been finalized, with a compaction firing in between, ends with the compaction notice wearing the skill decision's baseline/jev lists and the reply wearing nothing. Both surfaces read meta.decisions_strip, so nothing downstream would notice — the strip would simply appear on the wrong row. Adding the attempt id does not fix this one: the collision is between two different POINTS, not two attempts of one.

2. outcomes.discard actively destroys the record. chat_runner calls it at the start of every turn, deliberately, so a turn that published and died leaves nothing for the next reply to wear. That is correct for a receipt whose consumer is a reply — and wrong for one whose consumer is a compaction notice. A user sending a message between the scoring's publish and the notice's append would silently delete the measurement. The record would be lost, not mis-attributed, and it would look exactly like a scoring that missed its deadline.

3. The 600 s TTL is ten times the window the record can be correct in. RECORD_TTL_SECONDS is 60 because the consumer is the callback at the end of this compaction, seconds away. At 600 s an entry from eight minutes ago is still claimable. The attempt token closes the ordinary case of that, but the token is deliberately subtractive — it can only refuse a publish, never authorise one — so on a key whose counter was evicted by the cap the TTL is the only thing left, and 60 s is the correct bound there.

The store is ~40 lines and the point's whole surface for it is begin_attempt / publish_record / take_record / reset_records, which is the duplication actually at stake. Merging the two would mean giving outcomes a second entry class with its own TTL and its own discard policy — more machinery than the file it deletes.

Also in this push

GPT #2, capped calls — changed shape from the previous round. I had made the overflow a boolean that suppressed the card line, which answers the finding by removing the measurement. It is now a count the surface states: calls_truncated on the record, and the line reads Jev would keep 23 of 61 tool calls (+140 not scored) · 41% of the characters, only when the count is non-zero. A capped record is now published, because a qualified true sentence beats both alternatives — a bare "23 of 61" is wrong when 61 is the cap rather than the session's call count, and drawing nothing hides a measurement that is accurate about everything it covers. The qualifier sits inside the count's clause rather than trailing the line, because the character share is measured over what the arms did cover and a trailing caveat would read as covering both numbers. The truncated calls' characters stay out of both arms, so the two remain one universe; the count is what says that universe is smaller than the session. Read through asCount, so a record stamped before the field existed reads 0 and adds nothing.

First Principles item 1 — no committed consumer. #12516 opened: the live keep-list on the recycle path, with option B from the spike (a keep-set threaded into _replay_rows inside the unchanged 80 000-character budget), the five things these shadow rows have to show before it is worth building, the paths it deliberately excludes, the re-read signal the shadow cannot give, and the fact that the consent copy — which says the compaction is unchanged whatever Jev answers — stops being true the moment anything is applied and needs re-reviewing rather than inheriting. Linked under Follow-ups in the description. No code.

Design / UX / Screenshot Evidence. Twelve frames now, including the pair that was missing: the summary card collapsed and expanded. That is the one card shape with a disclosure and the keep line sits outside the fold, so both states are claims rather than one. That record also carries calls_truncated: 140, so the conditional copy is photographed rather than described. 38 in-harness checks, all passing; every attachment verified 200.

The two CI reds in the review — the ko 조사 and the Windows clock — were already fixed in c62f2ac6ab: {{productName}}이(가), and the TTL test pins time.monotonic instead of relying on it to advance.

Verified on b670dfd39f

Touched files only, as asked: 76 pytest across test_decisions_compaction_keep.py and test_decisions_compaction_hook.py (shuffled and in file order), 198 vitest across the keep line, the card, the spec contract and the four i18n suites the new string touches, flake8 and mypy on the touched Python, eslint on the touched TypeScript, and gen-pseudolocale.

Nothing here looks like a further real defect

Everything still open on the lanes is either answered above or a claim about evidence that lives outside this repository. If a later round surfaces something I think is real, I will say so rather than dispositioning it.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for b670dfd39f47f1e32bb69630dadca7098fb1f3b9; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b670dfd39f47f1e32bb69630dadca7098fb1f3b9: <one-sentence reason>

@iamwhatever
iamwhatever force-pushed the feat/decisions-compaction-shadow branch from b670dfd to cdc0379 Compare September 21, 2026 10:47
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 21, 2026
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 21, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

All three on 2f36042123, still one commit. Item 1 was a defect I introduced two pushes ago; item 3 does not hoist, and I said why at each site rather than writing a reason that is not true.

(1) The admission clip ran before redaction — real, and mine

content[:MAX_FIELD_CHARS] cut a credential spanning the bound into a fragment neither redactor matches, and the fragment is what would travel: a stage's own clip in build_state only ever sees what admission kept, so redacting later cannot recover it. Exactly the #12338 rule, and this module's own redacted docstring states it — I broke it while adding the bound.

The admission bound now goes through redacted, which is already redact-then-clip. Three call sites (message text, tool input, tool name), and the fold's or became a guard so a call that already has a name does not pay for a redaction pass whose result is thrown away.

Two tests:

  • test_a_secret_straddling_the_admission_bound_is_redacted — bound falls six characters into an AKIA… key, asserted on both a message and a tool input, and asserting the fragment is absent as well as the whole key.
  • test_the_admission_bound_still_holds_after_redaction — redaction can lengthen text (a placeholder is longer than most secrets), so the clip has to run after it rather than instead of it.

Revert-verified: restoring the slice fails the straddling case.

Lengths are still taken from the RAW value, before both the redaction and the clip, so chars_all, both _eligible arms, input_chars, result_chars and result_placeholder are unchanged.

(2) scopeMut.isPending added

Taken, and the comment that argued against it is gone. It is one hazard, not one per scope: a scope PUT carries enabled: true, so flipping any scope and then revoking consent is two concurrent writes to one keystone whichever scope moved. Naming only compactionMut left the condition covering part of its own subject.

This is the main switch's own disabled expression rather than a guard bolted onto the tool-argument row, so it does not collide with #12492 — which still owns the server side for every scope at once.

freezes the main switch while the TOOL-ARGUMENT scope write is in flight too is the companion test; revert-verified (dropping the token fails it). The tool-argument switch's error surface is still not asserted here, for the reason it was not before.

(3) The local imports do not hoist — reason stated at each site

Hoisting would break something a test pins. All five sit inside a try/except whose job is to degrade rather than raise, and this module is imported from the compaction path: an import that fails has to fail there, where it costs one field or one measurement, rather than at module scope where it would break importing the point at all. TestItCannotCostTheCompaction::test_a_point_that_raises_on_import_leaves_the_compaction_alone is the test for that.

I did not write "circular import", because it would be false — all four targets import cleanly on their own:

kiro_crew.history                ok  0.46s
kiro_crew.context                ok  0.03s
kiro_crew.platform.context       ok  0.00s
kiro_crew.security               ok  0.00s

So each site now names the real reason and what the degraded answer is: redacted drops one field, read_rows yields no rows (a compaction that goes unmeasured), _is_after returns False and KEEPS the row. replay_roles already carried its own reason.

If the convention wants module scope regardless, that is a change to the "a broken point cannot cost a compaction" contract rather than an import move, and I would rather you decide it than assume it.

Verified locally

128 pytest across the three compaction suites; 42 vitest on the settings card; both fixes revert-verified; flake8, isort, black, mypy, eslint; brand, comment-history, feature-map and docs gates. § 11 now says the admission bound is itself redact-then-clip.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@iamwhatever
iamwhatever force-pushed the feat/decisions-compaction-shadow branch from 2f36042 to 15d2a06 Compare September 21, 2026 15:34
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Rebased onto b6f22ccba0 as 15d2a06e2a, one commit, MERGEABLE on three spaced reads. Both reds fixed — and the registry one was hiding something worse than staleness.

The conflict — en.context.json, resolved as a three-way key merge

One conflict, both sides appending to entries: main added four pages.membersPage.webview_* keys, this branch adds nine. Resolved by computing the sets rather than taking the union, because a union is wrong in one direction — a key main DELETED is still present on this side, so a union puts it back.

entries: base 456  main 460  mine 465
I added (9)        main added (4)
main deleted (0)   I deleted (0)
both edited the same key (0)
resolved 469

Asserted rather than assumed: the resolved key set equals main's plus my nine, and nothing main deleted is present. Order follows main with my nine appended, so the file diffs as 9 added lines against main and nothing is reformatted.

(1) Frontend Tests (3) — the anti-stale guard was the symptom

npm run gen:settings did not just refresh a stale file. It deleted developer.decisions-jev from the registry, which is the main Decisions switch disappearing from settings search and the command palette.

Cause is mine, from last push. settingsExtract.extractJsxProps scans from the opening tag to its closing angle bracket and treats ' as a string opener even inside a // comment. My rewritten disabled comment introduced the word "row's" — one unmatched apostrophe — so the scan never left string state, hit its 4000-char ceiling, and the element was skipped. Isolated rather than guessed:

variant entry
as committed DROPPED
disabled back on one line DROPPED
comment removed from the props KEPT

So the formatting was innocent and the prose was the cause. The reasoning now lives above the element, where the scanner never reads it — which also means no future edit to that paragraph can drop the entry again. The condition itself is unchanged.

With that, the generated files are byte-identical to what the rebased commit already carried, and the guard passes.

(2) Frontend Tests (1) — the skip was one I created, so it is gone rather than accounted

settingsCoverage.test.ts reported FeaturePreviewsSection.tsx: skipped 1 (accounted 0). Not one of the three switches: every label in the file is a literal i18nT('…').

tagStartRe is a regex over the raw source, and my new comment spelled the primitive with its leading angle bracket. The extractor matched that prose as an element, found no label, and counted a dynamic-label skip. Reworded to name the primitive without the bracket, and the comment says why so the next editor does not put it back.

That is why there is no new EXPECTED_DYNAMIC_SKIPS entry: the right fix was to stop manufacturing the skip, not to record it. Repo-wide skips are back to 18 = 16 + 1 + 1, all pre-existing and all accounted.

Worth a separate issue, not this PR

extractJsxProps reading quotes inside // and /* */ comment spans is a live trap for every settings panel: one apostrophe in a comment placed among an element's props silently removes that setting from search, and the only signal is a SKIP (ceiling) warning in generator output nobody reads. I have not touched the shared script — changing its scanning could move other files' entries — but say the word and I will open it.

Verified locally after the rebase

gen:settings (212 entries, 18 skips, generated files unchanged); settingsCoverage.test.ts + settingsRegistry.test.ts 17 pass; 219 vitest across the 11 locale style suites (with I18N_BASE_REF set so [changed-values] ran), the settings card, the keep line, the spec contract and both settings-extract suites; i18n:check exit 0, all sub-gates ok; 128 pytest across the three compaction suites; eslint. git diff origin/main...HEAD --stat lists 44 files, all this branch's.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running labels Sep 21, 2026
@iamwhatever
iamwhatever force-pushed the feat/decisions-compaction-shadow branch from 15d2a06 to 484ec18 Compare September 21, 2026 16:24
@github-actions github-actions Bot added readiness: checking Automated validation is still running merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention labels Sep 21, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Head ae507494a5, one commit, 45 files, MERGEABLE on three spaced reads. The blocking item is settled as a class at the route, and the rebase turned out to carry a real integration with #12267.

The class, not the line

span=src/kiro_crew/decisions/consent.py — KEEP_ENABLED, and save_enabled resolving it
span=src/kiro_crew/dashboard/handlers/decisions.py — the scope-only PUT and its 400s
span=website/src/api/client.ts — enabled?: boolean
span=website/src/pages/settings/FeaturePreviewsSection.tsx:224,233 — both scope mutations
span=test/test_decisions_compaction_scope.py — five route tests
span=website/src/test/decisionsConsentScopeBody.test.ts — four body tests

The finding named two lines; the fixable thing is the SHAPE of a scope write. A body carrying enabled can only carry what its sender last read, so no client-side guard can make it safe — the staleness is in the value. So the switch left the body.

enabled omitted on a body naming at least one scope is now a scope-only write: the route hands consent.KEEP_ENABLED down, and save_enabled resolves the switch and the endpoint from the keystone inside the same _SAVE_LOCK the other three sentinels resolve in, writing both back unchanged. Such a write can neither grant nor revoke consent, and cannot rebind the endpoint.

body result
{compaction: true} on a revoked record enabled false, scope false
{compaction: true} on an enabled record scope true, switch + endpoint + other scope unchanged
{endpoint: <other>, compaction: true} recorded endpoint unchanged
{}, {endpoint}, {history_budget_chars} 400
{enabled: "true"} / 1 / null 400

The last row is the one that matters for the omission not being a widening: accepting an absent enabled did not change what counts as consent. The fleet ceiling still gates only the ENABLING direction — a scope-only write cannot turn the seam on, so it acquires no authority a withdrawn ceiling would have to refuse.

Both revert-verified: making the writer treat a scope write as enabling fails 2 route tests; putting the switch back in the client's body fails 2 body tests.

The four existing card assertions moved from true to undefined. The new vitest reads the JSON actually sent rather than the arguments passed, because "the card passes undefined" and "no enabled key leaves the machine" are different claims and the route's safety rests on the second.

The rebase carried #12267

Main moved 11 commits mid-round and model.route landed. Six conflicts, and two were real integration rather than text:

span=website/src/pages/chat/decisionRecord.ts — main's list/dispatch structure kept whole; my readCompactionKeepRecord stays a separate reader and out of the DecisionRecord union, because the compaction record rides the compaction notice row and is read directly by the card, not dispatched out of an assistant row's strip
span=website/src/test/decisionStripContract.test.ts — both fixtures, 31 tests (main's 25 + my 6)
span=src/kiro_crew/decisions/gate.py — DECISION_POINT_NAMES carries all five
span=docs/feature-map/README.md — main's rewritten developer row is the base, my fourth point and THIRD scope re-applied into it
span=docs/system-specs/modules/decisions.md — my section renumbered § 11 → § 12, since #12267 took § 11; the intro now says five integrations, three deciding

Verified nothing of #12267 was lost by counting its own symbols on both sides rather than by reading the diff:

model.route           main=84  mine=84
model_route           main=49  mine=49
DECISIONS_MODEL_POINT main=6   mine=6
readModelRecord       main=29  mine=29
MAX_RECORDS           main=7   mine=7
readDecisionRecords   main=9   mine=9

One process note, since it bit me: my first splice of the contract test cut main's describe mid-loop, because the conflict markers fell inside a for block. Rebuilt by taking each side's describe whole instead of splicing at markers.

Dispositions

docs/system-specs/modules/decisions.md carries two ## 8. headings — "Settings and validation" and "The decision strip's record and feedback". Pre-existing on origin/main, not from this branch, and not touched here.

settingsExtract.extractJsxProps reading quotes inside // comments stays untouched, as instructed.

Verified locally

366 pytest across the six decisions suites; 1292 pytest on the -k decision selection; 31,653 vitest across 1,952 files (all of src/test, src/pages/settings, src/components/commandPalette, src/i18n/style); tsc --noEmit; gen:settings (212 entries, 18 skips); i18n:check exit 0; eslint, flake8, black, isort, mypy; brand, comment-history, feature-map and docs gates. The three pytest import errors are imageio_ffmpeg missing on this host in test_transcribe.py and test_web_verify_downscale.py — neither file is in the diff.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Round 12 — the three reds on ae507494a5, classified with measurements.

Bundle Size Gate — mine, fixed. Reproduced locally byte-for-byte (same chunk hash App-BfUyRwDE.js as CI): 3,618,907 B against the 3533 KB (3,617,792 B) ceiling, 1,115 B over.

Attribution was measured on a pristine origin/main worktree rather than assumed:

build App chunk vs 3533 KB ceiling
main @ e4b00db9df alone 3,616,146 B 1,646 B under (0.045% headroom)
this branch merged onto it 3,618,907 B 1,115 B over
this surface's delta +2,761 B

So main fits and this surface is what tips it. The ceiling moves by exactly that 2,761 B delta, rounded up to the table's whole-KB unit: 3533 → 3536 KB (+3,072 B). The number is in the commit body.

Two things worth recording rather than leaving implied. The entry's comment says "measured 3364.4 KB … ~5% headroom", and main now measures 3531.4 KB — the documented margin has been consumed by ~167 KB of main-side growth without the entry being re-measured, so its real headroom was 0.045%, not 5%. I did not restore the 5% the entry's own convention prescribes: that is a re-measure of main's drift, not this surface's cost, and widening a shared ceiling beyond my own delta inside a frozen PR is not mine to do. The consequence is that at 1,646 B of remaining headroom the next app-core addition trips this entry again — the same recurrence the all and t entries above it document twice.

Backend Tests (Windows) (3) and (6) — both main-side. Test names pulled from the job logs:

  • shard 3 — test/test_mcp_preflight.py::TestOneServerCannotEndThePass::test_deep_annotations_do_not_discard_the_other_verdicts, RecursionError
  • shard 6 — test/test_embedding_rebuild_generation.py::test_store_opened_after_final_snapshot_requires_current_vectors[member-late], assert False

The decisive evidence is the shard map. scripts/ci_file_shards.py assigns whole files by sha256(relative_path) % 8 + 1, so a shard's membership depends only on path, never on how many tests a branch adds. Those two files hash to shards 3 and 6 — the two that failed — and this branch adds or edits no test file on either:

this branch's test file shard
test_decisions_compaction_hook.py (new) 8
test_decisions_compaction_keep.py (new) 5
test_decisions_compaction_scope.py (new) 2
test_decisions_consent.py (edited) 4
test_decisions_gate.py (edited) 2
test_decisions_tool_args_scope.py (edited) 2

Enumerated both shards on both trees: shard 3 collects the same 332 files and shard 6 the same 362 files on this branch as on origin/main — zero added, zero dropped, identical sets. Neither failing test imports any module this diff touches (test_embedding_rebuild_generation.py imports dashboard.handlers.memory, not handlers.decisions), and both files pass in full here on both trees (113 tests).

Both failure mechanisms are environmental and readable in the traceback, not assertions about product behaviour:

  • The preflight test builds a deliberately 2000-deep dict and asserts pytest.raises(RecursionError). The failure is one level further out — CPython's own traceback.py:355 _get_code_position exhausting the stack while formatting that expected exception. It depends on stack remaining, which Windows has less of.
  • The embedding test's failing line is assert await asyncio.to_thread(entered.wait, 5) — a 5-second wall-clock deadline on a threading.Event under an xdist worker.

I cannot run Windows here, so I am not claiming either is a proven flake; the claim is narrower and sufficient: the two shards collect identical file sets to main's, so nothing test-side in this branch reaches them. Both are yours to rerun.

Unchanged and confirmed in the diff: the consent-restore fix. A scope write omits enabled entirely (saveDecisionsConsent(undefined, …) at both scope toggles), the PUT resolves it to consent.KEEP_ENABLED and reads the switch and endpoint from the keystone inside the writer's lock, and a body naming neither enabled nor a scope is still a 400. test_decisions_consent.py and decisionsConsentScopeBody.test.ts both still hold it.

Verified locally on the rebased tree: 366 pytest across the six touched backend files, 131 vitest across the six touched suites, tsc --noEmit clean, eslint clean on the changed script, and the comment-history / brand-name / focus-cue gates green with base refs set.

…it would keep

Adds a SHADOW decision point `compaction.keep`: when Crew compacts a session
it asks Jev which tool calls it would keep, records the answer, and shows it
on the compaction card. Nothing about the compaction changes -- the point is
measurement only, behind its own consent scope, off by default.

The App chunk's bundle budget moves 3533 -> 3538 KB. Measured, not estimated,
by building pristine main in its own worktree and then this branch on top:
main @ 20261b7 alone builds the chunk at 3,619,504 B against the 3533 KB
(3,617,792 B) ceiling, so main exceeds it by 1,712 B with no PR code in the
tree; this surface adds 2,761 B of first-party code, for a merge result of
3,622,265 B. The new ceiling covers both parts and leaves 647 B of headroom.
It is deliberately NOT restored to the ~5% margin that entry's comment
prescribes: main's drift consumed that margin and re-measuring it is a
separate change, not this surface's cost.
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Rebased onto main 20261b7633. No code changes; one number moved, and it moved for a reason worth recording.

The App bundle ceiling had to move again, and most of it is not this PR

I re-derived the delta on the new main rather than carrying the previous absolute number forward. Two analyze builds: pristine main in its own detached worktree, then this branch on top of it.

build App chunk vs 3533 KB ceiling (3,617,792 B)
main 20261b7633 alone 3,619,504 B 1,712 B over
this branch on top 3,622,265 B 4,473 B over
this branch's own delta +2,761 B

The first row is the finding: check-bundle-size.mjs run against pristine main's own report fails — FAIL assets/App-rOzhdheH.js: exceeds its 3.45 MB budget by 1.7 KB. There is no PR code in that tree. The Bundle Size Gate is red on main today, so it is red on the merge ref of every open PR.

That is why raising the entry by exactly my measured delta does not clear it: 3533 KB + 2,761 B rounds to 3536 KB, which my branch still exceeds by 1,401 B, because main has already spent 1,712 B of the budget. The ceiling therefore moves to 3538 KB (3,622,912 B), which covers both parts and leaves 647 B of headroom. The entry comment and the commit body both name the split — 1,712 B main's drift, 2,761 B this surface's — rather than presenting the whole move as this PR's cost.

I deliberately did not restore the ~5% margin that entry's older lines prescribe. Doing that is a re-measure of main's growth, which is a different change from this one. The consequence is that 647 B of headroom will not survive the next app-core addition, which is the same recurrence the all and t entries in that table already document twice.

Only App was over. The other measured deltas — all +15,731 B, t +1,115 B, useVisualViewport +885 B, vendor +222 B, client +106 B — all sit inside their own budgets; the gate now reports 697 chunks within budget at exit 0.

Still in the diff

The consent-restore fix is intact and unchanged by the rebase: a scope toggle sends no enabled key at all, the PUT resolves an omitted enabled to KEEP_ENABLED and reads the switch and endpoint from the stored keystone inside the writer's lock, and a body naming neither enabled nor a scope is still a 400. Nine tests hold it.

Verified locally on the rebased tree

133 pytest across the three compaction test files, 204 vitest across 16 files (the keep line, the strip contract, the consent scope body, the decisions settings card, settings coverage, and all 11 locale style suites), i18n:check exit 0, eslint exit 0, five diff-scoped doc gates exit 0, bundle gate exit 0. git diff origin/main...HEAD --stat lists 46 files, all this branch's, in one commit.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ⚠️ review incomplete

GPT 5.6 did not produce a complete verdict for 120219bfeba3cebf7f2317ca6a8465271ab09a36; inspect the workflow logs and re-run it.

This comment is updated in place on each push.

See the GPT 5.6 Review job logs; this commit has no completed GPT verdict.

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 120219bfeba3cebf7f2317ca6a8465271ab09a36: <one-sentence reason>

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at head 120219bf. Not approving, and neither reason is a defect in this code. Both are decisions above a reviewer's pay grade, so I am leaving them with the maintainer rather than settling them by approving.

What leaves the machine

Answering this first because it is the whole review. The body POSTed to https://api.typesafe.ai/v1/systemone (impl_jev.py:189-197, allow_redirects=False) carries a state built by compaction_keep.build_state from ConversationLog().read_messages(session_key) -- the session's own JSONL. Concretely:

  • messages: user and assistant text, verbatim at the first three rungs (text_share 1.0, 1.0, 1.0), head+tail halved and quartered at the last two. Not the current turn -- every turn of this slot's conversation up to the pre-compaction boundary, newest MAX_ROWS = 4000.
  • calls: each tool call's tool name and its input, redacted then clipped to stage.input_chars (1000 → 200 → 60), up to MAX_CALLS = 1000.
  • Tool results are never sent at any stage -- result_placeholder(chars) yields ok, N chars (omitted). That is the widest omission in the seam and it keeps the largest half of a transcript local.

Admission clip is MAX_FIELD_CHARS = 4096 per retained string, redact-then-clip (correct order -- a slice placed inside a secret leaves a fragment neither pattern matches). STATE_TOKEN_CEILING = 25,000 tokens picks the first fitting rung; a transcript that never fits sends nothing.

The gate chain is sound and I verified each link: sealed keystone (decisions_consent.json, a _CREW_SECRET_LEAVES / _CREW_READONLY_LEAVES leaf, unreadable ⇒ off), permits() requiring enabled is True plus a verbatim endpoint match, scrub_reason over the same json.dumps rendering the wire uses with no interposed rewrite, and a third scope consented_compaction on a strict data.get("compaction") is True -- "true" and 1 both read False, and an unreadable scope refuses. A tool_args-only install is inert here.

I mutation-verified the scope guard on scratch copies rather than taking it on trust:

permits(endpoint) with compaction=False : True    # switch+endpoint fine; refusal is purely the scope
is_enabled compaction=False (BASELINE)  : False   # refused
is_enabled compaction=True              : True    # permitted
is_enabled compaction=False (MUTATED)   : True    # guard neutralized -> flips to permitted
is_enabled compaction=False (RESTORED)  : False

Load-bearing, not decorative.

And it is shadow-only -- no third-party score can discard local data. I looked for a write path and there is none: score_compaction → publish_record → take_record, consumed only by dashboard/state.py:_on_compacted to attach meta["decisions_strip"] to a display notice row. _shadow_score_compaction is an un-awaited create_task at the top of _compact_session, so the compaction proceeds on its own coroutine before any arm is chosen. A missing record just yields no card line. That was my largest worry going in and it is cleanly answered.

Why I am not approving

One: this is a bigger egress category than the memory-content point, and that one is still undecided. I deferred #12492 to the maintainer because it was the first point to send text the user had stored in a previous session rather than typed in the current turn. This sends strictly more: the whole conversation, both sides, verbatim at the first three rungs.

And calls[].input is not only command lines. I traced the plumbing: the transcript row's input is _redact_tool_field(event.tool_input) (chat_runner.py:3114), i.e. the tool's own argument payload, and for an edit tool that payload is the content -- derive_edit_diff (acp/_dispatch.py:505-560) renders a create's content/fileText/text as a whole-file addition and a strReplace's oldStr/newStr as a replace hunk. So up to 1000 characters of a file the agent wrote can travel, per call, for up to 1000 calls, under the 25,000-token ceiling.

Redaction covers credentials and exfiltration URLs -- routed through redact_via_context so a companion's own patterns apply, and dropping the field outright on a composition failure, which is the right direction. It does not cover source code or private prose, and it is not meant to. The docstring is explicit and, I think, correct that a credential in a tool argument is ordinary (an aws command, a curl header) and that refusing on it would mean the point never fires on the transcripts most worth measuring. I am not disputing any of that. I am saying the category -- conversation text plus authored file content leaving the machine -- is a product call, and it is the kind that gets decided by default when a reviewer approves a clean implementation.

Two: GPT 5.6 has no verdict on this head, and the green rollup hides it. The check-run reports SUCCESS while the comment says: "GPT 5.6 did not produce a complete verdict for 120219bf…; inspect the workflow logs and re-run it." The same thing happened on b670dfd3, so it is not a one-off. Opus 4.8 (the other blocking-capable lane) did complete with no findings, and Design and First Principles both PASS, so this is not #12229's situation where every blocking lane was dark. But on the largest egress surface in the repo I would rather have the second one than infer it from a green tick, especially since the tick is what a triage pass reads.

UX Review's CONCERNS I verified as accurate and non-blocking: "(+140 not scored)" maps to a real calls_truncated, and the "Jev would keep 7 of 61" line reading as descriptive under a restart-failure notice is display-copy nuance. Neither touches egress or correctness.

Tests: test_decisions_compaction_keep.py + scope logic 110 passed; scope route tests 28 passed with the head handler; test_decisions_gate.py + test_decisions_consent.py 194 passed. Four cases in test_decisions_compaction_hook.py reference head dashboard/state.py I did not overlay -- overlay artifacts, and I verified that function from the diff instead.

Re-run GPT on this head and settle the category, and I will approve on the mechanism -- it is the most carefully gated of the three Jev points.

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving at head 120219bf. The egress category is now a decided question -- the maintainer has ruled that conversation content may go to Jev -- so the mechanism is what remains, and I verified it before and re-state the load-bearing parts here.

My earlier comment on this PR withheld approval for two reasons. The first is settled by that ruling. The second is not settled and I am recording it rather than letting it pass silently, because a green check-run conceals it.

What the ruling covers, stated plainly so the record is not vague. The state POSTed to https://api.typesafe.ai/v1/systemone carries user and assistant text verbatim at the first three fitting rungs, across every turn of the slot's conversation up to the pre-compaction boundary (newest MAX_ROWS = 4000), plus each tool call's input redacted and clipped to 1000 / 200 / 60 characters by rung (MAX_CALLS = 1000). Tool results never leave -- result_placeholder replaces each with ok, N chars (omitted), which keeps the largest half of a transcript local. The page's own wording for this is accurate and worth keeping as the canonical description: "a whole session transcript, the conversation text plus the INPUT of every tool call in it, with each tool RESULT replaced by its own character count so no output byte leaves."

One consequence deserves to be explicit in the record, since it is the part a consenting owner is least likely to picture: a tool call's input is the tool's own argument payload (chat_runner.py:3114 stores _redact_tool_field(event.tool_input)), and for an edit tool derive_edit_diff (acp/_dispatch.py:505-560) renders a create's content as a whole-file addition and a strReplace's oldStr/newStr as a replace hunk. So up to 1000 characters of a file the agent authored can travel, per call. Redaction covers credentials and exfiltration URLs -- through redact_via_context, so a companion's own patterns apply, and dropping the field outright on a composition failure -- and does not cover source code, which is the correct scope for a redactor and the reason the consent scope is where this decision belongs.

The scope design is the right shape and I checked each link. Sealed keystone (decisions_consent.json, a _CREW_SECRET_LEAVES / _CREW_READONLY_LEAVES leaf, unreadable reads as off), permits() requiring enabled is True plus a verbatim endpoint match, scrub_reason over the same json.dumps rendering the wire uses with no interposed rewrite, and a third independent scope on a strict data.get("compaction") is True -- "true" and 1 both read False, an unreadable scope refuses, and an already-consented tool_args install is inert here rather than retroactively signed up. That last property is the one that makes a third switch worth its cost, and the stated reason is the right one: the narrower yes was reviewed as the arguments of the one call about to run, not as everything the session has run.

I mutation-verified the scope guard rather than taking it on trust -- neutralizing _scope_consented flips the point from refused to permitted with the scope unconsented, and restoring it refuses again.

And it cannot lose local data. I went looking for a write path for the score and there is none: score_compaction → publish_record → take_record, consumed only by dashboard/state.py:_on_compacted to attach meta["decisions_strip"] to a display notice row. _shadow_score_compaction is an un-awaited create_task at the top of _compact_session, so the compaction proceeds on its own coroutine before any arm is chosen; a missing record just yields no card line. A third party scoring what compaction discards, with no ability to change what it discards, is the version of this that is safe to ship.


The open item I am approving over, recorded so it is not lost: GPT 5.6 has no verdict on this head. Its comment says so -- "did not produce a complete verdict for 120219bf…; inspect the workflow logs and re-run it" -- while the GPT 5.6 Review check-run reports SUCCESS, so the green rollup does not show it.

Two things I established that bear on what to do about it. It is not diff size: #12425 at 153 files and 4,263 additions got a real verdict. And the remedy the bot itself advertises has already been tried -- the GPT run for this head is at attempt=2 with every step reporting success, and the verdict comment has not moved since it was first posted. So a further re-run is not obviously the fix, and I did not establish the root cause.

Opus 4.8, the other blocking-capable lane, completed with no findings, and Design and First Principles both PASS. That is why this is a recorded gap rather than a hold. Closing it is a repository-writer action -- either a working re-run or /ai-review override gpt 120219bf…: <reason> -- and it is the maintainer's to take.

UX Review's CONCERNS I verified as accurate and non-blocking: "(+140 not scored)" maps to a real calls_truncated, and the "Jev would keep 7 of 61" line reading as descriptive under a restart-failure notice is display-copy nuance. Neither touches egress or correctness.

Tests: compaction-keep plus scope logic 110 passed, scope route tests 28 passed against the head handler, gate plus consent 194 passed. Four hook cases reference head dashboard/state.py I did not overlay; I verified that function from the diff instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants