Skip to content

feat(decisions): let Jev route a chat turn to a model tier - #12267

Merged
iamwhatever merged 1 commit into
mainfrom
feat/decisions-model-route
Sep 21, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
feat/decisions-model-route

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

A chat session runs every turn on one model. "Rename this variable" and "redesign the scheduler" are the same session, so they get the same model and the same price. A user who wants the cheap one for small asks has to change the model by hand, remember to change it back, and pay for the times they forget.

Why it matters

The two directions both cost something. Staying on a strong model makes every small ask expensive. Staying on a cheap one makes the hard asks worse. Nobody wants to drive the picker by hand between messages, so most people pick one model and live with the wrong answer half the time.

What changed (motivation → approach → change)

You can now let Jev pick the model for each message. Open the model picker under the chat box and choose Auto (Jev). From then on Jev judges each message as simple, medium or complex, and that message runs on the model you mapped that level to. Small asks can go cheap; hard asks can go strong. A message a scheduled job, a sub-agent or an app sends is never routed.

Part What it does
Auto (Jev) in the model picker The only way to turn this on. Shown only when the Decisions switch is on and your org allows it
decisions.model_route in config.json Which model answers each level. Every level ships unpinned — you write the ids your own picker shows
A line under the reply The level Jev picked, the model that answered, the model that would have answered otherwise, and thumbs
Everything else Unchanged. Pick any model by hand and the routing stops

This rides the Jev seam that already ships rather than building a second one: the same gate, consent file, sampling, log, receipt and thumbs route. Three things were widened so a second question fits. The point list gained model.route. The receipt registry is keyed per question, so a reply decided twice carries both instead of the second erasing the first. The receipt is a list, and which row is drawn comes from each record's own point.

Auto (Jev) is not a model. Picking it sends auto:jev, which the gateway turns into plain auto at one place and remembers as a flag, so everything downstream sees a normal auto.

When Jev is slow, unavailable, or answers outside the three levels, the message runs on the model the session was already using — what happens today for everyone. The log says which, so "Jev answered and nothing happened" is visible rather than silent.

Not yet on the receipt: a level pinned to a model this account cannot run keeps the session's model and writes only a log row — the failure rendering has no producer. Tracked in #12289.

Cost: routing to a stronger model spends more. So only the OWNER can turn it on: a non-owner is refused, a fork inherits it only for the owner, and the entry needs a live chat. A pick made by hand always wins, even mid-answer.

No model id ships as a default. Every level of model_route is "" — inherit — until an owner pins one, and "auto" means the same. model-selection.md requires it: a hardcoded id fails on the first prompt for anyone not entitled to it. Values go through normalize_agent_model, the normalizer agent.role_models uses.

The choice is never read back from disk. It is an owner pick that spends money and transcript metadata is agent-editable, so persisting it would let a prompt-injected agent write "jev_route": true, restart, and be granted routing nobody selected. The flag is in-memory only, so a restart leaves the chat on its usual model and the owner re-picks. The entry is withheld for a remote-crew session too, whose turns run on the peer and never reach the hook.

An unpinned level is still answered. Not a refusal: it is asked, logged and shown as simple → (unpinned), and the turn keeps its model — so a new install can read where its messages land before pinning anything. An unpinned level and a pin the account cannot run both keep the model, and is_unpinned tells them apart so only the second records an error.

Where each piece lives:

Area What is there
src/kiro_crew/decisions the new points/model_route.py; gate.py lists the question and reads the map; outcomes.py keys receipts per question; the prior-turn read moves into points/__init__.py so both questions share one budget, and points/skills_select.py reads it from there
src/kiro_crew/dashboard chat_handlers.py resolves the picker id and commits the flag on each success path only; chat_runner.py asks the question and applies the model; state.py holds the flag, slot_projection.py reports it, chat_fork.py inherits it, and chat_persistence.py / channel_slots.py deliberately neither save nor reload it
src/kiro_crew/config sections.py adds decisions.model_route with all three levels present and "", each normalized through normalize_agent_model; config-baseline.json regenerated
src/kiro_crew/context.py reads the two shared prior-turn constants from the package that owns them
website/src/pages, website/src/components, website/src/lib, website/src/types lib/jevRoute.ts decides when the entry is offered; ChatPage.tsx and components/ChatPane.tsx draw it; components/ModelDropdownList.tsx shows its label; chat/decisionRecord.ts, chat/DecisionStrip.tsx and chat/AssistantMessage.tsx read and draw the receipts; settings/decisionsPreview.ts and types/index.ts name the question and the flag
website/src/i18n the row's strings in all 12 catalogs
website/scripts capture-decision-model-route.mjs, the screenshot capture behind the shots below: it drives the real built dashboard with the network stubbed and asserts what it photographs. No shipped surface of its own
docs, src/kiro_crew/docs docs/system-specs/modules/decisions.md § 9 is the contract; docs/feature-map/README.md and src/kiro_crew/docs/decisions.md say how to turn it on

Backwards compatibility

Compatible: no turn changes model until an owner picks Auto (Jev) — which needs the Decisions consent that is itself off by default — AND pins a level in decisions.model_route, where every level ships unpinned. decisions.model_route is a new optional key whose absence pins nothing. The receipt on a reply is now a list; the dashboard reader accepts the single object earlier replies carry, so old transcripts render unchanged. No field became required, no validator got stricter, and no key was renamed or removed.

The receipt reports what happened, not what was asked for. A switch can be accepted and still not take — some backends judge the model id itself and keep their own default — so the reply says not applied — ran on <model>, and the recorded row says the same. That row is never rewritten and an owner pins from it.

One limit worth stating plainly: picking a model by hand stops the routing, and it does not move the session back to the model it was on before routing started. A routed session stores auto, so choosing plain Auto to switch routing off takes the same-value path and changes no model. The revert — captured when routing is armed, restored when it is switched off or when a tier is unpinned — is tracked in #12289.

Tests

  • test/test_decisions_model_route_apply.py drives the real turn runner against a recording client and asserts the model id the provider is handed. Two mutation checks sit beside it — one changes the level-to-model map, one changes Jev's answer — so a hook that switched to a fixed model, or switched on nothing, fails instead of passing by luck. Plus every refusal (not picked, no consent, unsampled, an id the account cannot run, a failing switch, no switch seam) asserted as "the provider was never asked to switch", and the five turn kinds that are never routed.
  • test/test_decisions_model_route.py covers the question and its bounds, reading the answer, the map, resolving a model against the advertised list, prior turns under the shared budget, and the error rows. It pins that every level ships unpinned, that "auto" and a non-string both collapse to inherit, that the shipped map still answers and applies nothing, and that an unpinned level is told apart from a pin the account cannot run. Two tests guard the comparison: two ids one character apart are never folded together, and the registry's dotted/dashed pair stays separate. Tests use placeholder ids (model-a); the registry pair is the one exception, because a placeholder has no registry entry to test with.
  • website/src/test/DecisionStripModel.test.tsx covers the reader and the row: a record without a level draws nothing, an EMPTY model is accepted and rendered as (unpinned), a record is dispatched on its own point and never read as the other kind, and a reply decided twice draws two rows.
  • website/src/test/jevRoute.test.tsx covers the picker's two gates as fail-closed, the row's position, that it shows a label and not the wire id, and that a routed session highlights that row rather than auto.
  • website/src/test/decisionStripContract.test.ts now also parses the model record out of the spec, so the two halves cannot drift apart silently.
  • Existing suites updated for the list-shaped receipt and the two-point registry.

Manual verification

website/scripts/capture-decision-model-route.mjs runs the real built dashboard with the network stubbed and asserts as well as photographs: the row is the picker's first entry, a routed session reads as selected on it, the row is absent when the org answer is false, and a past routed turn still shows its receipt then. All assertions pass. A real Jev call needs an API key and consent to send test data, so the routing decision itself is covered by the unit and runner tests above.

Screenshots / video

The shipped state — the level was answered, that level is unpinned, the turn kept its session's model:

An unpinned tier reported on the reply, light

A reply decided by both questions, with the level pinned:

Both receipts under one reply, light

The model picker with Auto (Jev) beside auto, selected:

Model picker showing Auto (Jev) selected, dark

Opened, the row shows the level, both models and what the question carried:

Expanded model row, dark

A switch that was accepted and did not take — the row names the model the turn ran on, not the one it asked for:

Routing row reading not applied, ran on claude-opus-4.8, light

Opened, "Model used" agrees with that line rather than crediting the model that was never used:

Expanded routing row whose Model used names claude-opus-4.8, light

The row's failure rendering, drawn from a hand-written record rather than a live one — nothing publishes this state today: record_error writes the log row and deliberately does not publish, so this is what the copy will look like once it does. The missing producer is tracked in #12289; the frame is here because the copy is reviewable now:

Expanded routing row showing a model-not-advertised failure, light

Every variant

An unpinned tier reported on the reply, dark

Both receipts under one reply, dark

Model picker showing Auto (Jev) selected, light

Expanded model row, light

Routing row reading not applied, ran on claude-opus-4.8, dark

Expanded routing row whose Model used names claude-opus-4.8, dark

Expanded routing row showing a model-not-advertised failure, dark

And the withheld case — the org answer is false, so the entry is not offered at all:

Model picker with the routed entry withheld

Related Issues

no linked issue: this is the second decision point on the Jev seam, dispatched alongside #12245 (strip polish) and #12253 (tool risk); none of the three has a tracking issue.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and pepmach September 20, 2026 09:49
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Intent: Let a chat session ask Jev how hard each message is and answer that message on the model mapped to that difficulty, so a small ask can run cheap and a hard one can run strong without the user driving the model picker between turns.
Not a goal: Routing anything but a normal dashboard chat turn (cron, sub-agents, crew-relayed turns, app injections and autonudge wakes stay on their own model tier); overriding a model the user picked by hand; turning anything on by default; a second decision seam, a new endpoint, or a new thumbs side.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 20, 2026
@github-actions

github-actions Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 0893809c56125c5d86afd1760bd4843b33606cc6 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Routing has no anchor model: after the first switch, every refusal, outage, or disarm keeps the last routed tier's model, not the owner's pick.

Watch

The failure story claims a refusal "runs on the model the session was already using — what happens today for everyone," but that holds only for the first routed turn: _route_model_for_turn never reverts, so once a tier switches the session, a Jev timeout or outage pins subsequent turns to whatever tier ran last (a cheap model answering complex asks, or a dear one billing simple ones), and choosing plain Auto to disarm "takes the same-value path and changes no model." The missing revert and the missing disarm-restore are one root cause — no pre-routing model is captured when routing is armed — deferred to #12289 while the arm ships now.
Clears when: #12289 lands (capture the model on arm, restore on disarm and use it as the refusal anchor), or a human explicitly accepts shipping the arm before the revert.

[DESIGN-REVIEWED] 0893809

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 20, 2026
@github-actions

github-actions Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 0893809c56125c5d86afd1760bd4843b33606cc6 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Pin-vs-advertised matching is reimplemented in the new point beside the resolver the repo already names for exactly that job, and one response field ships unread.

Not justified as shipped

  • Item 8 — duplicate of src/kiro_crew/acp/client.py: _same_model/resolve_model (decisions/points/model_route.py) is a second definition of "does this pin match an advertised id" beside resolve_pin_spelling/resolve_usable_model — the resolver AGENTS.md names for substitute choices, already run at the wire chokepoint (acp/session_handle.py:1985). Counted 3 backend comparators after this PR (acp/client.py:3544, dashboard/handlers/agents.py:1864 — whose docstring says "same model?" has "ONE definition" — and model_route.py:277). The config's own help says pins are written "exactly as the chat model picker spells it", so the extra registry/dot-dash fold matches nothing the documented contract needs, and a pre-check that accepts what the chokepoint later collapses to "" records applied: false where model-not-advertised is the category the pre-check exists to produce.
    Subtraction: delete _same_model and judge membership through resolve_pin_spelling, the shared fold every other pin site uses.
    Clears when: _same_model is deleted, or a concrete picker spelling only the registry comparator matches is named.
  • Item 6 — zero consumers: the "jev_route" key added to api_chat_slot_model's three response bodies is read nowhere — grep of website/src for jev_route shows both writers set the flag "from the REQUEST, not from the response".
    Subtraction: drop the key from the response bodies; the slot projection already carries the flag.
    Clears when: a counted reader of the response field exists, or the key is deleted.

What this change ships

Inventory (10 items) — 8 justified

Intent: let a chat owner hand each message's model choice to Jev by difficulty tier — an ADDITION.

  1. Auto (Jev) entry in the chat model picker, owner-only, consent- and capability-gated, withheld for remote sessions — justified
  2. A routed turn asks Jev a tier and runs on the model the owner mapped to it (model.route point) — justified
  3. New config key decisions.model_route, all three tiers shipped unpinned, not PATCH-writable — justified
  4. A receipt line with thumbs under a routed reply; unpinned tiers shown as (unpinned) — justified
  5. Reply receipts became a list (two decisions per turn survive); old bare-object rows still render — justified
  6. Slot projection and the model-pick response report jev_route; the response copy has no reader — zero consumers (response field only)
  7. Any manual pick — single, bulk, or fork-inherited by non-owner — clears routing and broadcasts — justified
  8. Tier pins matched against advertised ids via a new comparator before the switch — duplicate of acp/client.py:resolve_pin_spelling
  9. Shared prior-turn budget/walk moved to decisions/points/__init__.py, one implementation for both points — justified
  10. New day-file rows and three error categories so a discarded answer is visible; docs and a dev screenshot harness — justified

[FIRST-PRINCIPLES-REVIEWED] 0893809

@github-actions

github-actions Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 0893809c56125c5d86afd1760bd4843b33606cc6 — this comment is updated in place on each push.

Review details

No findings that block; one advisory finding.

FINDING — src/kiro_crew/decisions/points/model_route.py:367 — routed_model's returned dict and build_outcome (line 384) omit history_chars/truncated (they are put only on extra for the core.decide call row, model_route.py:330-331), so the published outcome carries neither field; readModelRecord reads root.history_chars/root.truncated through asCount(undefined) → 0 (decisionRecord.ts:371-372) and the expanded strip renders "History characters: 0" / "Prior turns clipped: 0" (DecisionStrip.tsx:169-176) even when an owner raised decisions.history_budget_chars above 0 and the turn actually sent transcript — a false egress figure, whereas skills.select emits history_chars on its own outcome (skills_select.py:619) → Fix: carry history_chars (and truncated, if the strip shows it) on the dict routed_model returns and include them in build_outcome, mirroring skills_select.build_outcome; or drop the historyChars/truncated fields from DecisionModelRecord/the model panel.

[OPUS-REVIEWED] 0893809

Verdict parsed from the review's SHA-scoped output markers for commit 0893809c56125c5d86afd1760bd4843b33606cc6.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0893809c56125c5d86afd1760bd4843b33606cc6: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — 🟡 CONCERNS

UX-level review of 0893809c56125c5d86afd1760bd4843b33606cc6 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: CONCERNS

"Auto (Jev)" sits beside "auto" with nothing telling them apart, and the shipped default receipt "(unpinned)" means nothing to a first-time reader.

Watch

  • Picker pair indistinguishable (shot-03/10): auto's subtitle is just "Default" while Auto (Jev) gets a sentence; the reader "cannot tell which one I'd want, and picking the wrong one seems easy" — a wrong pick silently switches routing off. Every picker open, every routed user. Fix: give the plain auto row a contrasting description ("One model for every turn").
  • "(unpinned)" (shot-01/08) is the state every un-configured install sees first, and the reader said "I do not know what this is"; the surface never says what pinning is or where (decisions.model_route is reachable only out-of-band). Fix: replace model_unpinned with plain words ("no model set for this tier") in the collapsed line or the expanded panel.
  • The collapsed line truncates its payload (shot-05/06/12/13): "not applied — ran on…" chops the model that actually ran because confidence, latency and default: … are shrink-0 while the message span truncates — and expanding never un-chops it; the reader had to infer the answer from "Model used". Fix: let the trailing model_baseline_named segment truncate before the message, or drop "ran on {{model}}" redundancy since the expanded "Model used" row carries it.
  • Failure state reads as a contradiction (shot-07/14): red "Decision failed model-not-advertised" under "Model used claude-fable-5.1" left the reader unable to "tell whether the routing worked", and model-not-advertised is a raw category slug (lens-2: what happened + what next, no codes). Fix: human-readable error text, and make the header line reflect the failure the way the "not applied" state does.

Suggestions

  • One term for one fact: the collapsed line says default: {{model}} while the expanded row calls the same value "Model without routing", and "Default" is also the plain auto row's subtitle — pick one term (e.g. "without routing: {{model}}") for model_baseline_named.
  • "Context sent (characters)" showing 0 read as "odd" to the blind reader; consider "Conversation sent to Jev" with a formatted char count, matching the existing sent_history phrasing.

[UX-REVIEWED] 0893809

@github-actions

github-actions Bot commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 0893809c56125c5d86afd1760bd4843b33606cc6 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/decisions/points/model_route.py:244 -- "any(_same_model(...))" rejects stored namespaced IDs when the provider advertises the bare ID, leaving valid tier pins unapplied -> Fix: resolve and return the advertised spelling via resolve_pin_spelling.
[GPT-REVIEWED] 0893809

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0893809c56125c5d86afd1760bd4843b33606cc6: <one-sentence reason>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Re-captured after the owner changed the complex tier default to claude-fable-5.1. These are the current pixels; the description now points at these URLs.

Captured by website/scripts/capture-decision-model-route.mjs, which asserts as well as photographs — the entry is the picker's first row, a routed session reads as selected on it rather than on auto, the row is absent when decisions_enabled is false, and a past routed turn still shows its receipt in that state. All assertions pass.

Both receipts under one reply, light and dark:

Both receipts under one reply, light

Both receipts under one reply, dark

A simple turn routed to haiku, light:

A simple turn routed to haiku, light

Expanded, dark and light:

Expanded model row, dark

Expanded model row, light

The picker with Auto (Jev) selected, dark and light:

Model picker showing Auto (Jev) selected, dark

Model picker showing Auto (Jev) selected, light

And withheld — the org answer is false, so the entry is not offered:

Model picker with the routed entry withheld

@iamwhatever
iamwhatever force-pushed the feat/decisions-model-route branch from 54a0dcb to cd77ce8 Compare September 20, 2026 10:03
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 20, 2026
@iamwhatever
iamwhatever force-pushed the feat/decisions-model-route branch from cd77ce8 to 69bce07 Compare September 20, 2026 10:21
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] fixed — Automated Rule Check / model-selection, at 69bce079b0a72a384179ff9b0e0d9628bc8c7e22.

The finding. The model-selection — no hardcoded model id as a default/fallback rule matched DECISION_MODEL_ROUTE_DEFAULT in config/sections.py, all three lines.

Legitimate. Yes, and the reasoning is the feature's own. docs/system-specs/common/model-selection.md opens with "never hardcode a model id as a default or a fallback ... a hardcoded id fails at runtime — and silently, until the first prompt — for anyone not entitled to it", and names exactly one sanctioned concrete fallback (_BACKGROUND_CC_MODEL, because that backend cannot resolve auto). A shipped tier map naming three concrete ids is the rule's central case, not an edge of it.

Fixed, not exempted. decisions.model_route now ships empty, and coerce_model_route fills nothing in — an absent tier stays absent. The behaviour that follows was already there per tier: an unmapped tier resolves to "", which keeps the session's current model. So an unconfigured install behaves exactly as main does.

One thing the fix adds, because the empty default made it reachable: routed_model now reads the map before it sends. An owner who picked Auto (Jev) and has named no tier cannot have any turn moved, so asking would spend a paid provider call — and a message excerpt — on an answer that provably could not be applied. Empty map, no question.

I did not widen the gate. Note for the record that the spec's own ## The gate section says the check excludes "model_registry*, the config schema, and tests", while the implementation excludes src/kiro_crew/model_registry.py only — sections.py is the config schema. That mismatch is real but it is not mine to resolve inside a feature PR, and the headline rule covers a shipped default either way, so the fix stands on the rule rather than on the exclusion list.

Tests. test_no_model_id_ships_as_a_default pins that nothing ships and that from_raw({}) stays empty; test_an_unconfigured_map_asks_nothing_and_routes_nothing pins the no-send path; test_a_map_naming_only_another_tier_still_asks pins that one mapped tier is enough to make the question worth asking, so the short-circuit is not over-eager. test_an_unreadable_section_routes_nothing makes the fail-closed direction explicit. Every routing test now supplies its own map, which is legitimate in test/ — outside the tree the rule reads.

Docs. Spec § 2 and § 9 and src/kiro_crew/docs/decisions.md now say the map is empty by default, why no id is named for the user, and that an empty map sends nothing. The config example is labelled an example rather than a default.

Owner note. This reverses the default values I was asked for one round ago (simple claude-haiku-4.5, medium claude-opus-4.8, complex claude-fable-5.1) while keeping the ids as the documented example to copy. I took the gate as authoritative to unblock the PR rather than sit red. If you want those three shipping as real defaults it needs a ruling and a separate change — either the gate's exclusion list matching its own spec, or the ids entering model_registry.json — and claude-fable-5.1 is marked "[Internal] DEVELOPMENT USE CASES ONLY" in the catalogue, which is its own argument against shipping it to every install.

PR Readiness was the cascade of this one red and needs no separate action.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] Re-captured for the ruled shape: every tier ships unpinned, so the row that most readers see first is simple → (unpinned) — answered and reported, nothing applied. The description now points at these URLs.

website/scripts/capture-decision-model-route.mjs asserts as well as photographs: an unpinned tier says so instead of leaving a gap where a model id belongs, the Auto (Jev) entry is the picker's first row, a routed session reads as selected on it rather than on auto, the entry is absent when decisions_enabled is false, and a past routed turn still shows its receipt in that state. All assertions pass.

The shipped state — the tier was answered, that tier is unpinned, the turn kept its session's model:

An unpinned tier reported on the reply, light

An unpinned tier reported on the reply, dark

A reply decided by both questions, with the model tier pinned:

Both receipts under one reply, light

Both receipts under one reply, dark

Expanded, dark and light:

Expanded model row, dark

Expanded model row, light

The picker with Auto (Jev) selected, dark and light:

Model picker showing Auto (Jev) selected, dark

Model picker showing Auto (Jev) selected, light

And withheld — the org answer is false, so the entry is not offered:

Model picker with the routed entry withheld

@iamwhatever
iamwhatever force-pushed the feat/decisions-model-route branch from 69bce07 to d489286 Compare September 20, 2026 10:40
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever] fixed — Automated Rule Check / model-selection, at d4892868b82c0c07ecfceb1d2e5d6aaaf6f14f4b. Maintainer ruling applied; this is the last code round on this finding.

The shape. decisions.model_route.{simple,medium,complex} all default to "" — inherit, i.e. the turn keeps the model its session is already on. Each value goes through normalize_agent_model, the same normalizer coerce_role_models uses, so "auto" and a non-string both collapse to "" and "inherit" has one spelling. No model id is named anywhere in code, config-baseline.json defaults, or tests.

An unpinned tier is answered, not skipped. It records tier and p in the log and on the strip (complex → (unpinned)) and applies nothing. That is deliberate: the answer is what tells an owner which tier their turns land in, which is how they decide what to pin, so a new install gets the reading before it gets the routing. model_route.is_unpinned(tier, mapping) is what separates it from a pin the account cannot run — both keep the model, only the second is a finding, and conflating them would hide the second. The runner returns before set_model on the unpinned path, so no switch is ever attempted for it.

Ids removed. config/sections.py, decisions/points/model_route.py, config-baseline.json and both test suites now carry no claude-* literal. Tests use placeholders (model-a, model-b, model-c.1). Two exceptions, both deliberate and both named in place:

  • test_two_registry_entries_that_differ_only_in_punctuation_stay_distinct keeps claude-opus-4.8 / claude-opus-4-8. Those two are the registry PAIR whose distinctness is the claim — a placeholder has no registry entry, so it would exercise the string fold and leave the registry half of the comparison untested. Neither is a default or a pin.
  • src/kiro_crew/docs/decisions.md keeps the ONE worked example of an owner pinning claude-haiku-4.5 / claude-opus-4.8 / claude-fable-5.1 in config.json, per the ruling.

Tests. test_every_tier_ships_unpinned_and_no_model_id_is_named pins the default; test_inherit_has_one_spelling pins the normalize_agent_model collapse over six inputs; test_the_shipped_map_answers_and_applies_nothing pins that the tier is still asked, recorded with its p, and reported with an empty model_chosen and no error row; test_an_unpinned_tier_is_told_apart_from_a_pin_that_cannot_run pins the two apart; test_the_shipped_unpinned_map_switches_nothing_but_still_reports drives the real runner hook and asserts set_model is never called while the receipt still lands. On the frontend, the reader now accepts an empty model and the row renders (unpinned) with data-pinned="false".

Evidence re-captured for the shipped state (the unpinned row is the one most readers see first) and re-attached above; the description points at those URLs.

Local: 381 python tests in the decisions/config/slot scope, 88 website tests, flake8/black/isort, tsc/eslint, i18n:check 19/19 PASS, and 11 repo gates at exit 0. The model-selection rule, run exactly as CI runs it over this diff, reports 0 hits.

@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Sep 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR placed the routing hook after prompt assembly and gave the point no context-window awareness.

The ordering is exactly as stated and I verified each line: full_message is assembled from 10032 through 10156, the routing hook runs at 10244, and the stream opens at 10625. Between the switch and the stream there is one read of read_context_tokens(client) and it feeds the crew log, not a guard. Nothing in model_route.py knows about windows or tokens; its only size bound is on what gets sent to the oracle.
So a long session whose next message is simple can be moved onto a cheaper tier with a smaller window, carrying a prompt that was sized for the larger one. That is the feature working as designed, which is what makes it a real defect rather than an edge case.
Two bounds worth recording, neither of which makes it go away. It needs a deliberate pin — every tier ships unpinned, so no install hits this until an owner maps a tier to a smaller-window model. And the same wall exists today without this feature: picking a 200K model by hand in a session grown past 200K fails the same way, because the slot-model route has no window check either. What routing changes is that the move happens unasked.
Not fixed here: the maintainer froze this branch, and the one exception — the record-integrity finding — is spent. The fix is also not small in either shape the lane offers: moving the hook ahead of model-dependent assembly reorders a 600-line stretch of the turn, and refusing-or-compacting on a downward switch puts window arithmetic inside the decision point. Either wants its own round with its own tests.
Tracked in #12289 as the highest-severity item on that list, with both shapes written out. I am not posting an override: a crash on an ordinary path is not something I will wave through, and the lane's fence is right to hold it.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR added "jev_route": jev_route to all three success returns of api_chat_slot_model.

Agreed, and this lane now cites my own code comment as the evidence: the frontend says outright that the flag is written from the REQUEST rather than the response, and the 11 hits across website/src are that write plus reads of the slot projection. No branch reads the response field.
It is the last item this lane holds against the change. The duplicate-predicate item and the undeclared capture script are both gone from its output, which is the useful signal here.
Not deleted on this branch: the freeze stands and its one exception is spent. Dropping a key from a shipped response shape is a change with its own review surface, however small the diff.
Tracked in #12289 as a one-line subtraction — drop it from the three returns, leave the projection as the only reporter.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: I captured the failure frame from a hand-written record last push and described it in the body as shipped behaviour.

  • Phantom failure state: the error row has no producer — my claim was wrong, description corrected (span=626aedca384f)

You caught me overstating my own evidence, and you are right on every part of it. record_error's docstring says "Never PUBLISHED" in my own words, routed_model returns None on model-not-advertised, and the frames I attached last push came from a record the capture harness wrote by hand. The description then said the row "says why, instead of leaving it in a log file", which the shipped code does not do.
Fixed where the error was, which is the description, not the code: it now says plainly that a level pinned to a model this account cannot run keeps the session's model and writes only a log row, with no line on the reply, and that the failure rendering has no producer yet. The screenshot caption says the same — hand-written record, nothing publishes this today, here so the copy can be reviewed now.
I kept the frame rather than deleting it, because the previous round blocked on not being able to see this state at all. An honestly labelled frame answers both rounds; a deleted one re-opens the first.
The missing producer itself is the item I logged in #12289 two heads ago, and it is still the right fix: publish an error record for the cases that keep the model, so the refusal becomes a receipt. Not done here — the branch is frozen and its one exception is spent — and the copy rewrite you suggest for model-not-advertised is tracked beside it, so the sentence and the producer land together.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 20, 2026
@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: the not-applied sentence was added to the collapsed row, which truncates, and the expanded panel repeats the model but not the sentence.

Agreed. The collapsed line is a single truncating row carrying the tier, the arrow, the model, my new sentence, the score, the latency and the baseline — so the newest and longest of those is the one that gets clipped, and a reader is left with a sentence that stops mid-air.
Partly answered already, and I want to be precise about how much: the expanded panel's "Model used" now names the model the turn ran on rather than the one it asked for, which was your previous round's Watch item. So the fact resolves when the row is opened; the SENTENCE does not, and a reader who does not open it has no way to know there is more.
Your fix is the right one — repeat the full notice as its own row in the expanded panel, where nothing truncates. I would pair it with the collapsed line being the short form on purpose rather than by accident.
Not applied here: the branch is frozen and the record-integrity exception is spent. This is a readability defect in a sentence that is already true, which is why I am not treating it as the same class as the panel that was saying something false.
Tracked in #12289 with the fix as you wrote it.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR added the Auto (Jev) row next to auto.

Agreed, and the lane has sharpened why it matters rather than just repeating it: the two adjacent rows differ in that one of them starts paying for a per-turn oracle call, and a reader who cannot tell them apart cannot consent to that.
Your suggestion to give plain auto a contrasting subtitle is better than only rewriting mine, and I had not considered it: the confusion is symmetric, so fixing one side leaves the reader comparing a described row against an undescribed one.
Not fixed here: the branch is frozen and its one exception is spent, and this is copy across 12 catalogues plus the pseudo-locale.
Tracked in #12289, now with both halves — the routed row's own sentence, and a subtitle on plain auto so the pair reads as a contrast.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: the ruled tier defaults made this the first state; this PR wrote the label.

Agreed for the fourth round, and the new detail is the one that stings: the reader checked the EXPANDED views too and still found no explanation. So my defence — that the row says enough — is not true even one click in.
Every tier ships unpinned, so this is the first thing a new install shows, and the feature reads as broken until someone finds the docs. That is a worse first impression than an error would be.
Not fixed here: the branch is frozen and the exception is spent.
Tracked in #12289 with both parts: plain words for what happened, and one expanded panel line naming decisions.model_route in config.json as where pinning happens.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]

Agreed on both counts, and the second is the part I had not stated in earlier rounds: the fragment does not merely collide, it also names the session's current model while the word "default" points a reader at an account-level setting.
The lane suggests "otherwise: {model}", and an earlier round suggested "without routing: {model}" to match the expanded label. I am keeping the second in the issue as the one to ship, because matching the expanded label removes a second reading rather than replacing one word with another.
Not fixed here: the branch is frozen and the exception is spent.
Tracked in #12289 with that wording settled.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR renders the log's error code into the notice.

Agreed, and your sentence is better than mine: "this account can't run {model}" names the thing the owner can act on, where my code names the seam that noticed.
One dependency, stated because it changes the order: nothing publishes this record yet, so the copy has no live surface. Shipping the sentence before the producer would add a string no one can reach, which is how the current frame ended up hand-written.
Not fixed here: the branch is frozen and the exception is spent.
Tracked in #12289 with the producer, deliberately in that order.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR added both rows to the expanded panel.

Agreed. With the history budget at its shipped 0 the question carries the message and nothing else, so both rows are honest and both mislead: a reader sees zero and concludes the oracle was sent nothing at all, which is not what it means.
Hiding them at zero is the right shape rather than rewording them, because at zero there is genuinely nothing to report — and when an owner raises the budget the rows reappear carrying real numbers.
One ordering note: the Opus lane found these numbers read 0 even when history WAS sent, because the two fields never reached the outcome row. That fix lands first, or hiding-at-zero would hide a real measurement.
Not fixed here: the branch is frozen and the exception is spent. Tracked in #12289 with that order recorded.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR placed the routing hook after prompt assembly and gave the point no window awareness.

The maintainer has ruled this a disposition rather than a fix, and I am recording it — with one correction to the premise, because a record that states something untrue is worth nothing to the next reader.
The premise was that the shipped tiers all share one window. src/kiro_crew/model_registry.json does not support that: the entries split evenly between 1,000,000 and 200,000, and the split runs THROUGH a family — opus-4.8-1m carries 1,000,000 while opus-4.8 carries 200,000. So the pair this finding describes is not hypothetical: a session served by the 1M variant, routed onto the 200K one, is two ids that differ by a suffix.
What does hold, and what the disposition rests on: every tier ships "", so nothing routes until an owner writes model ids into decisions.model_route themselves, and which id they write is their decision. Crew does not choose a smaller-window model on their behalf. The same wall is also already reachable without this feature — picking opus-4.8 by hand in a session grown past 200,000 tokens fails identically, because the slot-model route has no window check either. This feature automates a move the product already permits; it does not invent the hazard.
So the honest shape of the accepted risk is narrower than "the windows all match" and wider than "it cannot happen": an owner who pins across the 1M/200K boundary can have a long turn refused, and the failure surfaces as a provider error on that turn rather than as data loss.
Recorded in #12289 as a watch item with that trigger named and both fix shapes kept — refuse the downward switch and record a receipt, or route before model-dependent assembly. No code on this branch; the freeze stands.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

[operator: iamwhatever]
self-added: yes
mechanism: this PR added the consent query in both picker hosts and made the entry conditional on its answer.

  • Consent-read failures are silently hidden in the picker — accepted as tracked, no code (span=fd89d98b48d2)

The condition is accurate: the query carries retry: false, and the entry's gate tests consent?.enabled === true, so a 500 and a 404 produce the same outcome — the row is not offered, with nothing said. My comment there only explains the 404 case.
The maintainer's ruling is that the picker is the wrong surface to report it on, and checking the other surface, that holds: FeaturePreviewsSection.tsx already separates the two cases — backendMissing is the 404, and a non-404 read failure renders an ErrorNotice. So the switch that owns this consent reports its own read failure; the picker entry is a convenience built on that answer, and its absence leaves the manual model picker exactly as it was. Nothing is lost and nothing is destroyed: fail-closed here means one fewer row.
The residual, stated so the tracking issue is honest: an owner who never opens Settings sees the entry missing and is told nothing, so they cannot distinguish "my org turned this off" from "the read failed". That is a real gap in explanation, not in function.
Added to #12289 with the shape the lane proposes, kept to what it is worth: report a non-404 consent read failure where the owner can act on it, and keep 404 meaning unsupported. No code on this branch.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt b86057e: CI fully green; the substantive findings (owner-only arming, in-flight manual pick wins, no auto:jev on a new slot, owner-only routing, applied=false record integrity) are fixed; the two remaining items (context window of an owner-pinned model, Auto entry hidden on a failed consent read while the manual picker stays) are owner-only edge cases tracked in #12289.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for b86057ec5cf5a63a31ed208aa004733dccb79aba.

CI fully green; the substantive findings (owner-only arming, in-flight manual pick wins, no auto:jev on a new slot, owner-only routing, applied=false record integrity) are fixed; the two remaining items (context window of an owner-pinned model, Auto entry hidden on a failed consent read while the manual picker stays) are owner-only edge cases tracked in #12289.

This decision applies only to this commit. A new push requires a new judgment.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt 8d5a159: rebase of the already-overridden head b86057e onto main; CI fully green; substantive findings fixed (owner-only arming and routing, in-flight manual pick wins, no auto:jev on a new slot, applied=false record integrity); remaining owner-only edge cases tracked in #12289.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for 8d5a1597dc3b4a72b3025cd4aa3eb3f578fabacf.

rebase of the already-overridden head b86057e onto main; CI fully green; substantive findings fixed (owner-only arming and routing, in-flight manual pick wins, no auto:jev on a new slot, applied=false record integrity); remaining owner-only edge cases tracked in #12289.

This decision applies only to this commit. A new push requires a new judgment.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

/ai-review override gpt e0478f4: rebase onto main after #12253; CI fully green; the one new finding (manual pick via a session alias during the Jev await) is the narrower alias-path variant of the already-fixed in-flight-pick race, owner-only, recorded in #12289; all earlier substantive findings remain fixed.

@github-actions

Copy link
Copy Markdown
Contributor

Human judgment recorded

@iamwhatever marked the gpt AI finding as false positive, not applicable, or explicitly accepted for e0478f46354bd5c48692593bbc0c4f7eb90e52ca.

rebase onto main after #12253; CI fully green; the one new finding (manual pick via a session alias during the Jev await) is the narrower alias-path variant of the already-fixed in-flight-pick race, owner-only, recorded in #12289; all earlier substantive findings remain fixed.

This decision applies only to this commit. A new push requires a new judgment.

chenmingwei23
chenmingwei23 previously approved these changes Sep 21, 2026
bolichen97
bolichen97 previously approved these changes Sep 21, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Same consent shape as #12338 and it holds for the same reasons: off by default, the owner must pick Auto (Jev) in the model picker per session, and a manual model pick clears the flag so a concrete pin is never overridden by a tier. Cron deliveries, sub-agent turns, crew-relayed turns, app injections and autonudge wakes are excluded at the call site, each already resolving its model through its own tier — and the stated reason is the right one: nobody is watching the price of those answers at the moment they run.

Two design calls worth naming:

  • Every tier ships as "" rather than a model id. model-selection.md forbids hardcoding one, and the concrete reason is good: an id an account is not entitled to fails on the first prompt rather than when it was set. So the operator copies the example and fills in what their own picker offers.
  • An unpinned tier is answered, recorded and shown — complex → (unpinned) — rather than treated as a refusal. That is what makes the feature observable before anyone commits to it: an owner can watch which tier their turns land in for a while, then pin only the levels worth moving. Collapsing "auto" and a non-string to "" so "inherit" has one spelling is the right normalization.

Refusal leaves the session on the model it was already using, including when the mapped id is one the account cannot run, and the log distinguishes the two so nothing is dropped silently.

The gpt override is accepted: it carries forward an already-overridden head rebased onto main, the substantive findings (owner-only arming and routing, in-flight manual pick wins, no auto:jev on a new slot, applied=false record integrity) are fixed, and the one new item is the narrower alias-path variant of the already-fixed in-flight-pick race, owner-only and recorded in #12289.

@bolichen97

Copy link
Copy Markdown
Contributor

Approved above, and it was next in the merge queue — but #12338 landed one minute earlier and took the same seam, so this went dirty before it could merge. Flagging so the approval is not read as ready against the current base.

git merge-tree origin/main gives 14 conflicts across 9 files. Every one is keep-both in shape, because the two PRs register sibling points into one registry:

file conflicts
website/src/pages/chat/decisionRecord.ts 4
docs/system-specs/modules/decisions.md 2
website/src/test/decisionStripContract.test.ts 2
src/kiro_crew/decisions/gate.py 1
src/kiro_crew/decisions/points/__init__.py 1
src/kiro_crew/docs/decisions.md 1
test/test_decisions_gate.py 1
website/src/components/ChatPane.tsx 1
website/src/pages/settings/decisionsPreview.ts 1

The two that decide the rest:

  • gate.py — DECISION_POINT_NAMES is ("skills.select", "tool.risk", "message.steer") on main and ("skills.select", "tool.risk", "model.route") here. It wants all four. The two comment rationales are also both true and neither survives picking a side: nothing is keyed by point name, and decisions.model_route is keyed by tier rather than by point.
  • points/__init__.py — both sides rewrote the module docstring to say "two adapters". It is three now. Worth keeping this branch's paragraph on why the prior-conversation read lives in the package rather than in an adapter — with a third point registered, "a second copy of that walk would be a second place for the budget to be spent differently" is a stronger argument than it was when you wrote it, not a weaker one.

Everything else (config-baseline.json, chat_runner.py, chat_handlers.py, state.py, context.py, all 13 locale catalogs, the feature map) auto-merges.

Two notes before the push:

Say the word and I will do the rebase and run the decisions suites; I would rather you re-issue the override yourself.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of d2cb11e1bb628ba19c25e30311b4359c77418bd4 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d2cb11e

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d2cb11e1bb628ba19c25e30311b4359c77418bd4: <one-sentence reason>

A chat slot runs every turn on one model: the id the owner pinned, or the
backend's default. That is a single setting for a conversation whose turns are
not one difficulty -- "fix this typo" and "design the migration" are the same
slot and the same price.

Adds a second decision point on the shipped Jev seam. `model.route` asks the
oracle to put ONE turn in one of three tiers (simple / medium / complex) and
runs that turn on the model `decisions.model_route` maps the tier to. A
refusal -- no consent, unsampled, an answer outside the three tiers, an id this
account cannot run, a timeout, a provider failure -- leaves the session on the
model it was already on, which is what every unconsented install does.

Never behind the owner's back. The chat model picker carries an `Auto (Jev)`
entry, drawn only while the fleet's `decisions_enabled` AND the owner's keystone
consent are both an exact `true`, and selecting it records `slot.jev_route`. Any
concrete pick clears the flag: picking a model by hand is the owner answering
the very question this point asks. Only a normal dashboard chat turn is routed --
cron deliveries, sub-agent turns, crew-relayed turns, app injections and
autonudge wakes each already resolve their model through their own tier.

Reuses the shipped seam rather than adding a second one: the same gate, the same
consent keystone, the same bucket, the same JSONL log, the same strip, the same
feedback route and side. Three things were widened to carry a second point:
`DECISION_POINT_NAMES`, the outcome registry (keyed per point, so two decisions
on one turn both reach the reply), and the strip record (a list, dispatched on
each record's own `point`; a bare object still reads as one record).

The sentinel `auto:jev` is resolved at the top of the slot-model handler and
nowhere else, so every reader below it -- the guard, `session/set_model`, the
reset, the session allocation, the composer chip -- takes the plain `auto` the
session actually runs on. It is deliberately not stored in `slot.model`: that
field is a provider model id, and a value no provider advertises would need a
filter at each of those readers.

Arming answers to the OWNER, not merely to an authenticated caller: the
slot-model route's cross-app guard admits an allow-listed identity whose `app`
claim is empty, so the `auto:jev` branch alone is gated on
`is_owner_dashboard_request`, and a fork inherits the flag only for the owner.
A plain pick on the same route is untouched.

Provenance, not the actor, decides whose turn is routed. A dispatch that names
no actor falls back to `user` by design, so the rewind, regenerate and
OpenAI-compatible paths arrive as `user` while carrying an app's provenance --
which would have billed the owner for a routed turn nobody typed. The guard asks
for `_directive_user_origin`, the flag derived from the auth middleware's app
claim, and keeps the actor check beside it for the wakes that do declare
themselves.

Two ordering rules make the apply path honest. The answer's premise is re-read
inside the two locks -- the flag, and the model the baseline named -- so a pick
made by hand during the oracle's network round trip wins and the answer is
dropped rather than applied over it. And the picker entry requires a live slot:
without one a pick is held as the pending model and forwarded into slot
creation, which would store the sentinel as a new slot's model with routing off.

A returning `set_model` is not proof of a switch. A backend that judges the model
VALUE can exhaust its candidate ladder and stay on its own default without
raising, so the model the session reports is read after the switch and recorded:
`applied` is false when the turn neither reached the chosen id nor moved off its
baseline at all, `model_used` names what it ran on, and the reply's routing line
says `not applied -- ran on <model>`. The row is append-only and is what an owner
reads to decide what to pin, so it states what happened rather than what was
asked for.

Cost note: routing to a dearer model spends more. That is why the point is
reachable only through the picker entry -- the owner opted in by selecting it,
and clearing it is one pick.

Tests: the apply path is driven through the real runner hook against a recording
client, with two mutation checks beside it (break the map, change the answer) so
a hook that ignored either fails rather than passing by coincidence. Plus the
tier question and its bounds, every refusal, the error rows, the reader and the
strip row, and the picker's two fail-closed gates.

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified at head 0893809c. Approving.

The question that decides this PR is what leaves the machine, and it is verbatim user text: model_route.build_state() sends {"message": (text or "")[:2000]} plus optional history to https://api.typesafe.ai/v1/systemone via gate.py:610. I traced the chain in front of it rather than assuming it:

  • Consent lives in the sealed decisions_consent.json, not the agent-writable config.json, and unreadable means off. Then the governance ceiling, the point allowlist, and the sample bucket, then scrub_reason over the rendered state, which refuses the whole request on any credential or exfil hit and on a scan failure.
  • History is restricted structurally rather than by convention: build_history walks only HISTORY_ROLES = frozenset({"user", "assistant"}), so tool output and code are never projected. The default budget is 0 and the ceiling is min(config, consented_ceiling).
  • The call site fires only on slot.jev_route, _directive_user_origin, _crew_log_actor == "user", and not synthetic/slash/recovery — and it sends _jev_route_text captured before any runner prepend, so no cancelled-turn preamble or drained app context rides along.
  • Arming "Auto (Jev)" is owner-only: a non-owner pick gets _owner_denial_response plus a SEL row.

I gave the -103 in skills_select.py its own pass, since a large deletion in a decision surface is where a guard quietly disappears. It is a pure extract-to-shared-module: build_history, _prior_turns, _history_budget, MAX_HISTORY_MESSAGES and HISTORY_ROLES moved into points/__init__.py, and the load-bearing part is byte-identical in its new home — HISTORY_ROLES itself and the if role not in HISTORY_ROLES: continue skip. Both points now share the one guard. Nothing was removed.

Both earlier blocking findings are genuinely fixed here rather than force-pushed away. jev_route is memory-only (state.py __slots__, init False) and all three restore paths explicitly refuse to read it back from the agent-editable transcript, so a prompt-injected agent cannot self-arm paid routing; fork inheritance is owner-gated. And _route_model_for_turn re-reads both premise halves inside the locks, so a manual pick during the round trip drops the routed answer. The two dismissed human reviews were dismissed by the rebase — one was an approval, the other is empty.

No hardcoded model id: all tiers default to "", "auto" normalises to "", and the only frontend model literal is an illustrative example inside a comment. The frontend cannot invent a tier — jevRoute.ts says in its own docstring that neither read is an enforcement point, and decisionStripContract.test.ts positively forbids api_key, secret, prompt, message and content from the strip payload.

One nuance worth stating rather than blocking on: adding model.route to DECISION_POINT_NAMES means a consent given when only skills.select shipped also authorises this point to send the 2000-char excerpt. The payload category is the same one that consent covered, and this point additionally needs the per-slot owner arm that did not exist before, so no turn is sent without a fresh explicit owner action. Defensible — but the base keystone is reused rather than re-prompted, and a reviewer should know that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants