Repository navigation
fix: put an unpinned Jev tier back on the session's own model - #12872
Conversation
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm the old spec's "a routed turn is NOT reverted" sentence was a rationalized defect, not a product decision — this PR inverts a documented deliberate behavior. Suggestions
[DESIGN-REVIEWED] 111ce37 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The description says Not justified as shipped
What this change shipsInventory (8 items) — 6 justifiedIntent: FIX — a routed turn moved the live session onto a pinned tier's model and nothing put it back, so a partially pinned map captured the session and the receipt's "Model without routing" named the previous tier.
[FIRST-PRINCIPLES-REVIEWED] 111ce37 |
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsI've traced the single candidate. The claim is that the Falsifying reachability (requirement (a) — a concrete input that occurs in practice):
(a) comes out as "could not construct" — the candidate itself rates reachability "low" and I cannot re-derive a concrete input at 80+. It dies under falsification. No further grounded defect surfaced while tracing. No findings. [OPUS-REVIEWED] 58e0902 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
148e9d2 to
ccbc5de
Compare
|
Fixed in F1 — cross-alias pick invalidation. Upheld and taken. The latch now reads BOTH pick signals, the same pair Two things came with it, both named in the fix line:
Both halves are revert-verified: with the epoch comparison removed, and with the live re-read removed, |
GPT 5.6 Review —
|
ccbc5de to
75e2f43
Compare
|
Both blocking findings upheld and fixed in GPT F1 / Opus BLOCKING — the baseline now lives on the session, not the slot. The alias cache is re-read from the live session on every routed turn, not on a signal. A routed turn through a sibling alias moves the session's model and bumps no counter at all, so there is nothing to gate the read on. This is the half of F1 that made B skip restoration: B's cache and the baseline were stale-equal, so Opus's absent-epoch case is closed by comparing the stored epoch for EQUALITY rather than with an Opus FINDING — the in-lock guard re-reads the shared epoch beside the flag and the live model, since a cross-alias pick landing inside the A pooled runtime claimed for a new session is covered by the session id in the record, so a reused host cannot hand the next session the previous one's baseline. The per-slot fields and their teardown hook are gone — Revert-verified, five mutations, each breaking a different test: latch moved back onto the slot (4 tests fail), live re-read removed, in-lock epoch re-read removed, session id dropped from the record, epoch dropped from the record. Restored tree: 38 pass. The sibling-alias test runs alias B's turn on a WARM session ( |
75e2f43 to
1092f51
Compare
|
F1 upheld and fixed in the amended commit. The asymmetry was mine: before this PR an unpinned tier attempted no switch, so a row silent about One formula for both switch kinds. The raising arm carries the verdict too. A restore that raises records A row that attempted NOTHING — no baseline latched, the session already on it, a baseline the backend never named — still carries neither field, which is the honest absence. No frontend change: Two new tests: a restore the provider accepts and ignores (the ladder-exhaustion shape), and one it refuses outright. Both assert Revert-verified, seven mutations. Six break a test: the latch moved onto the slot, the live re-read removed, the in-lock epoch re-read removed, the session id dropped from the record, the epoch dropped from the record, and the raising arm left silent. The seventh — measuring |
GPT 5.6 Review —
|
1092f51 to
1887fea
Compare
a7a6519 to
58e0902
Compare
|
F1 upheld and fixed. Same class as the earlier rounds, and this closes the enumeration: fallback activity was read from ONE slot's sticky copy while the fact belongs to the session. Both reads now take the shared provider marker first. A marker that names no primary latches nothing. Returning Four movers, all read from the session now: an explicit pick (the shared Three new tests, each driven through the real hook with the sticky fields left OFF the acting slot — which is what a sibling alias has: the latch takes the primary, the unpinned turn attempts no restore, and an unnameable primary writes no record. One of them needed its assertion fixed rather than the code: Ten mutations revert-verified, each breaking at least one test: fallback activity read per slot, an unnameable primary latching the substitute, the substitute latched as the baseline, a restore firing during the ladder, the latch moved onto the slot, the live re-read removed, the in-lock epoch re-read removed, the session id dropped from the record, the epoch dropped from the record, and the raising arm left silent. Restored tree: 45 pass in the two routing suites. Rebased onto |
58e0902 to
ece5024
Compare
|
F1 upheld and fixed: Why not "read the configured pin instead of latching runtime state". It cannot fire: a routed slot has no pin by construction. Comment surgery, no behaviour in it. Eleven mutations revert-verified, each breaking at least one test, the two new ones included: the refusal record unread (2 fail), the throttle marker read per slot only (3), an unnameable primary latching the substitute, the substitute latched as the baseline (5), a restore firing during a substitution (3), the latch moved onto the slot (6), the live re-read removed, the in-lock epoch re-read removed, the session id dropped, the epoch dropped, the raising arm left silent. Restored: 47 pass. One note on the new tests, which is the third time this shape has bitten me here: the stub epoch host answers EVERY attribute, so |
A routed turn switches the live session onto the answered tier's model and nothing puts it back, so on a PARTIALLY pinned tier map -- which is what every install is on the way to, since all three tiers ship empty -- the first answered pinned tier captures the session for the rest of its life. A one-line rename answered `simple` moves the chat to the cheap pin; the redesign answered `complex` is unpinned, applies nothing, and leaves it there. The owner's configured model is never reached again, and turn two's own receipt names the cheap model as its "model without routing". An unpinned tier now switches BACK to the model the session ran on before routing first moved it, through the same seam and under the same two locks a pin uses. The receipt still says the tier applied nothing, because it did: `model_chosen` stays UNPINNED and `applied`/`model_used` are written for a pin only. A restore that cannot be expressed attempts nothing and keeps the model, which is what every refusal already does: no baseline latched, a session already on it, a baseline the backend never named, or no switch seam at all. That pre-route model is also what `baseline_model` reports on every turn. Read off the live session it is the PREVIOUS turn's tier from turn two on, which is how the dearest turn of a conversation comes to claim the cheapest model as its own default. It is latched on the SESSION, not the slot: one attribute on the same host the explicit-pick epoch lives on. Several slot aliases can drive one wire session while each alias's `served_model` is only that alias's cache of the session's answer, so a per-slot latch gives a second alias the first one's routed model as its own baseline and leaves no alias able to put the session back. For the same reason the alias's cache is re-read from the live session on every routed turn -- a routed turn through a sibling alias moves the session's model and bumps no counter to gate that read on. The record carries the session id and the pick epoch it was taken under, and one equality invalidates it on all three grounds: a moved epoch (an explicit pick by hand, through this slot or any other alias, including the session's first), a different session id (a pooled runtime claimed for a new session), or no record at all. A model the owner just chose is a newer instruction than anything latched before it, and a restore must never put the session back on a model they have left. The in-lock guard re-reads that epoch beside the live model, because a cross-alias pick landing inside the `decide` round trip leaves this slot's own cache untouched. Nothing is persisted, the same rule `jev_route` follows.
ece5024 to
111ce37
Compare
GPT 5.6 Review —
|
1 similar comment
GPT 5.6 Review —
|
…tdev#12872) A routed turn switches the live session onto the answered tier's model and nothing puts it back, so on a PARTIALLY pinned tier map -- which is what every install is on the way to, since all three tiers ship empty -- the first answered pinned tier captures the session for the rest of its life. A one-line rename answered `simple` moves the chat to the cheap pin; the redesign answered `complex` is unpinned, applies nothing, and leaves it there. The owner's configured model is never reached again, and turn two's own receipt names the cheap model as its "model without routing". An unpinned tier now switches BACK to the model the session ran on before routing first moved it, through the same seam and under the same two locks a pin uses. The receipt still says the tier applied nothing, because it did: `model_chosen` stays UNPINNED and `applied`/`model_used` are written for a pin only. A restore that cannot be expressed attempts nothing and keeps the model, which is what every refusal already does: no baseline latched, a session already on it, a baseline the backend never named, or no switch seam at all. That pre-route model is also what `baseline_model` reports on every turn. Read off the live session it is the PREVIOUS turn's tier from turn two on, which is how the dearest turn of a conversation comes to claim the cheapest model as its own default. It is latched on the SESSION, not the slot: one attribute on the same host the explicit-pick epoch lives on. Several slot aliases can drive one wire session while each alias's `served_model` is only that alias's cache of the session's answer, so a per-slot latch gives a second alias the first one's routed model as its own baseline and leaves no alias able to put the session back. For the same reason the alias's cache is re-read from the live session on every routed turn -- a routed turn through a sibling alias moves the session's model and bumps no counter to gate that read on. The record carries the session id and the pick epoch it was taken under, and one equality invalidates it on all three grounds: a moved epoch (an explicit pick by hand, through this slot or any other alias, including the session's first), a different session id (a pooled runtime claimed for a new session), or no record at all. A model the owner just chose is a newer instruction than anything latched before it, and a restore must never put the session back on a model they have left. The in-lock guard re-reads that epoch beside the live model, because a cross-alias pick landing inside the `decide` round trip leaves this slot's own cache untouched. Nothing is persisted, the same rule `jev_route` follows. Co-authored-by: Joe Guo <zejiangg@amazon.com>
What
A Jev-routed turn switched the live session onto the answered tier's model and nothing put it back. An UNPINNED tier applies nothing, so on a partially pinned tier map the first answered pinned tier captured the session for the rest of its life:
simple→ pinned to the cheap model → session moves there.complex→ unpinned → applies nothing → still on the cheap model.Every install is on that path: all three tiers ship as
""(no model id may be hardcoded as a default), so an owner pins one tier first. The owner's configured model was then never reached again for the life of that session, and turn two's own receipt named the cheap model as itsbaseline_model— the strip captions that field "Model without routing".Two changes, both in
_route_model_for_turn:resolve_substitute_set_modelseam and under the same two locks a pin uses. The receipt still says the tier applied nothing, because it did:model_chosenstaysUNPINNED, andapplied/model_usedare written for a pin only — a pair of fields saying whether a switch took would report a switch the row itself says was never asked for.baseline_modelis that pre-route model on every turn, latched on the slot at the first routed turn (_jev_route_baseline). Read off the live session it was the previous turn's tier from turn two on, which is how the dearest turn of a conversation came to claim the cheapest model as its own default.The bug lives entirely in the live session plus the
served_modelcache —slot.modelstaysautothroughout and is untouched here.Why these edges
""baseline (the backend never named the session's model, so no id aset_modelcould ask for), or no switch seam at all. A failed restore records the ordinary unpinned outcome row rather than anerrorcategory, because no tier model was dropped.baseline_model. Those were one value before and differ from the second routed turn on; comparing the baseline would drop every later turn as a phantom re-pick.jev_routeis not: it describes one live session, and a respawned session latches afresh on its first routed turn._default_session_modelis deliberately not the source of the restore target: a routed slot holdsslot.model == "auto", and that helper returns""for any slot whosemodelis set, so it names nothing for exactly the slots that route.Tested
test/test_decisions_model_route_ratchet.py— new, 4 tests. It landed as a green repro of the leak (turn two reportsbaseline_model=cheapand stays on the cheap model); the assertions are inverted here, plus a third turn routing from the base model again, a session whose served model the backend never named, and a hand pick becoming the new baseline. Its client double is what makes any of this observable:served_modelFOLLOWSset_model, where the apply suite's fixed-attribute double cannot show a second turn at all.test/test_decisions_model_route_apply.py— 31 passed unchanged, includingtest_the_shipped_unpinned_map_switches_nothing_but_still_reports(a never-routed slot has no baseline to return to, so noset_modelis attempted) and the manual-pick-during-the-await pair.test_decisions_model_route.py,test_decisions_strip_rides_message.py,test_dashboard_state_ws.py: 193 passed together.mypy src/kiro_crewclean,flake8clean,black --checkclean on the touched files,scripts/docs-lint.shpassed.scripts/local-gate.py: no failure in any chat, slot or decisions suite. The reds it reports are host/main-side and untouched by this diff (test_acp_runtime.pyunder parallel load,test_cron_timezone_display.pyhost-TZ,test_autonudge_reconciler.pycollection errors).Specs updated in the same commit: section 11 of
docs/system-specs/modules/decisions.md, and the one user-facing sentence insrc/kiro_crew/docs/decisions.mdthat said an empty level "just runs on the model the chat was already using".Pattern harvest
Rule candidate: review-prompt
Pattern: a per-turn apply whose apply is CONDITIONAL and whose restore does not exist. The condition holds on one turn and not the next, so the applied value is one-way: whatever the last satisfied condition wrote stays written for the life of the session. Here the condition is "this tier is pinned" and the value is the live session's model, so one cheap turn captured every later one. The reviewer question is the one this PR could not answer from the code: when this branch does not fire, who puts the value back? A conditional apply needs a restore arm or an explicit statement that the value is meant to persist — "the point runs again next turn" is not that statement, because the next run can decline to act.
Secondary, and the reason the receipt lied as well as the session: a function that READS a field as its input and WRITES the same field as its output is self-aliasing across invocations.
slot.served_modelwas turn N's output and turn N+1'sbaseline_modelinput, so "what would this turn have run on without routing" answered with the previous routing's result. The fix is the general one: latch the pre-state ONCE, before the first write, and read the latch — never the live field — for any "what would have happened" claim. Worth a reviewer prompt on any before/after, baseline, delta or savings figure computed from a field the same code path mutates. Sibling seams in this module already carry the latch (_fallback_primary_model,_fallback_slot_model) plus a staleness signal for a value the owner has since changed by hand (_model_pick_gen), which is what this fix reuses rather than inventing.Not a semgrep shape: both are data-flow questions spanning two invocations of one function, with the state living on an object between them. A matcher would either miss the aliasing or flag every read-then-write of a cached field.