Repository navigation
feat: one governance seam and neutral naming for decision models - #36267
bolichen97 wants to merge 1 commit into
Conversation
Decisions were named after one provider, and governance knew only two special cases (the hosted host and SageMaker). Every decision model now has its own entry in a per-model allow list, and the UI names the feature, not the provider. - capabilities.decisions gains a `providers` inner list (SCOPE_CATALOG data, no evaluator change). Ids are the picker's own: jev, each local preset, sagemaker, custom. The configured model's entry is asked with the row in one policy-generation bracket at every check that reads the row for it: the send path, the consent and provider writes, the consent carry, the SageMaker test and the runtime resume. A refused model is neither selectable (403) nor used, whoever wrote config.json; its recorded consent stays in place and inert. No policy, or a policy without the list, allows every model, so saved selections keep working. - GET /api/decisions/provider reports provider_permitted per model. The picker draws a refused model greyed with "Not allowed by your administrator." instead of hiding it, and the card draws a plain "turned off by your administrator" notice when the row is off. - "Decision model" is the feature in every label, hint, error and catalog; Jev is named only inside the hosted provider's own entry, which now holds its TypeSafe API key field. Stored ids (jev, auto:jev, catalog key names) are unchanged. Catalog and render tests pin it. - Specs (decisions, governance) and the user docs describe the list, the notice and the new labels; before and after screenshots are under temp-screenshots/decision-model-governance.
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. One note: the hosted provider's API key field moves into that provider's picker entry, and the PR doesn't name anyone who couldn't find it where it was. Notes
What this change shipsInventory (10 items) — 9 justifiedIntent: an administrator can switch decision models off or allow only some of them, enforced by the gateway, and every user sees the feature by a neutral name and can read why a model is unavailable. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 23655f1 Description read[DESCRIPTION-READ] e3174aeffae1424c0757faab6fd8cf73b65a3d16377d2e598c2264dfc4cdcdf0 That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — 🔴 changes requested (blocking)Reviewed Blocked: a settings-search hit on the hosted API key opens the key field and then closes it again, so search can't be used to set the key. BLOCKING — website/src/pages/settings/DecisionsProviderPicker.tsx:443 [BLOCK-MERGE] 23655f1 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — 🔴 changes requested (blocking)GPT 6.1 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- website/src/pages/settings/DecisionsProviderPicker.tsx:443 -- Preserve the search-opened key field after highlight cleanup
[BLOCK-MERGE] 23655f1 Adjudication (Opus 5.5) — is blocking on each finding proportionate?Harm is MEDIUM at most: a UI navigation regression with no security, credential or data harm. When the highlight param is removed ( This finding was fenced, so FLAG is the only annotation available. The FLAG bar is "conditions so extreme a human would accept the residual risk", and that bar is not met: one settings-search click with a non-hosted provider configured triggers it on an ordinary path. The real fix is cheap: hold one piece of local state. False positive or not applicable? A repository writer can comment: |
Code change approver recommendation: A person should decideWhy: the decider raised 1 possible problem and estimates a 50% chance that this change introduces a defect; a person should check it. The per-model allow list is enforced consistently at every gateway enforcement point, fails closed if the policy changes mid-read, and leaves both the off switch and the standalone default (no policy: every model allowed) as they were. It should still not merge yet: a new test assigns Things to check (possible problems the decider raised; at this confidence many turn out to be fine):
An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
Superseded: this diff was over the review lanes' 1 MB prefetch limit, so the work is split. #36271 carries the governance seam (the |
Pull request was closed
Problem / Motivation
Goal: An administrator can turn decision models off entirely or allow only specific ones, enforced by the gateway, and the dashboard calls the feature "decision model" instead of naming one provider.
Today the feature is named after one provider ("Decisions (Jev)", "Jev API key", "Share of sessions Jev answers for", "Jev chose: ...") even when a local model or a SageMaker endpoint answers, and governance knows only two special cases: the hosted host and the SageMaker host. An administrator cannot allow Laya and refuse everything else, and a policy that turns the row off makes the card disappear with no word of why. The hosted provider's API key sits in a "Shared settings" block although only that provider uses it.
Why it matters
A managed fleet (the internal edition first) must be able to ship with one provider not allowed, and every user must be able to read why a model is unavailable. Naming the feature after one provider reads wrong for every user whose decisions are answered by another model.
Not a goal
jevstays the preset id and lane value,auto:jevthe slot model, catalog key names (auto_jev,rate_jev) unchanged.pointslist, asmodelsandendpointsalready are.CONTRACT_VERSION, the evaluator, or the standalone default (no policy: every model allowed).What changed (motivation → approach → change)
Approach. Generalize the existing
capabilities.decisionsceiling instead of adding a parallel mechanism: one more inner list on the same row, asked wherever the row is already asked for the configured provider.Policy keys and data shape.
capabilities.decisions.enabledstays the full off switch.capabilities.decisions.scopes.providers(identifiermatcher, aSCOPE_CATALOGdata row). One entry per decision model, by the picker's own id:jev(hosted),plumb-4b,strands-decider-2b,laya,sagemaker(anysagemaker://address),custom(any other hand-written address). "No model" is never asked. No list means every model is allowed.Enforcement.
capability.provider_id(endpoint, model)names the configured model;capability.is_decisions_denied(provider=...)asksproviders:<id>together with the row(s) inside one policy-generation bracket (fails closed if the policy moved between the reads). It is asked at every chokepoint that reads the row for the configured provider:gate._consented_for, sodecide,is_enabledandwarmrefuse a refused model whichever writer named it inconfig.json;PUT /api/decisions/provider(switch_refused):403 decisions_capability_denied, nothing written;PUT /api/decisions/consent: an enabling or scope-granting write for a refused model is403; a disabling write is never refused;local_runtime.resume_configured(a refused preset is not started).The keystone is never touched by a refusal: the recorded consent stays in place and inert, and counts again if a later policy allows that model.
Settings read.
GET /api/decisions/provideraddsprovider_permitted: {id: bool}, each the AND of the model's row, theendpointslist for a model that leaves the machine, and its own entry (dashboard/handlers/decisions_offers.py, asked without a SEL row per id since it is an offer; every request that acts is audited).hosted_permitted/sagemaker_permitted/configured_refusedfold the list in, so a configured hosted default the list refuses reads as "No model" with no config rewrite.UI.
role="status"). Previously the card vanished.DecisionsHostedKeyField.tsx), opened when the entry is chosen or a settings search targets the key. The shared block keeps only the shared settings (share of sessions, history budget). Old settings-search ids are rewritten to the new ones (useSettingHighlightlegacy table).session_set_modeldescription, three config-schema descriptions);Auto (decision model)is refused for a non-owner like the old label spellings.Specs and docs, same commit.
docs/system-specs/modules/decisions.md(naming rule, section 7 and new 7.2),governance.md(theproviderslist), the feature map and spec label quotes, and the user docs undersrc/kiro_crew/docs/(new "When your administrator limits decision models" section). No entry indocs/decisions/covers these labels or this placement, so no superseding entry was needed.What the internal edition sets (requirement 4). In the companion's bundled
security_policy.json(the edition's posture file, precedence tier "bundled"), add under the existing top level:security_policy.jsoncapabilities.decisions.scopes.providers{"mode": "deny", "deny": ["jev"]}{"version": 1, "capabilities": {"decisions": {"scopes": {"providers": {"mode": "deny", "deny": ["jev"]}}}}}If the bundled policy already names
capabilities.decisions, merge thescopes.providerskey into that object (do not add"enabled"unless the edition means to turn the feature off; for the full off switch it iscapabilities.decisions.enabled = false). A governance profile bound to a surface can carry the samescopes.providersobject to narrow that surface only. This exact fragment is pinned bytest_decisions_provider_allowlist.py::TestTheEditionSnippet.Inventory of the naming sweep (greps for
Jev,jev,JEV,System One,TypeSafe,typesafe,PRESET_JEVacrosswebsite/src, every catalog, the generated UI and settings indexes, backend strings that reach the UI, CLI and MCP text, andsrc/kiro_crew/docs/):decisions_provider_jev("Jev, hosted by TypeSafe", the only catalog value naming Jev);decisions_provider_jev_detail(names TypeSafe);decisions_api_key_label("TypeSafe API key", drawn inside the entry);decisions_hosted_refused(interpolates the provider's name and is drawn inside the entry);decisions_custom_loopback_no_key(names the TypeSafe key that is not sent to a loopback address); "System One" in the SageMaker entry's detail and its connection-test protocol error (the wire format that endpoint must speak).jevpreset id and lane value,auto:jev,PRESET_JEV,jevRoute.ts,useJevAutoSend, catalog key names,TYPESAFE_API_KEY,jev-latest, log lines, comments).Backwards compatibility
Compatible: no policy, or a policy without the
providerslist, allows every model, so every saved provider selection, consent and config keeps working; no config key, API field or ceiling key is renamed or removed (provider_permittedis added; old settings-search ids are rewritten to the new ones).The diff tightens what a policy can refuse and changes what the card draws under a ceiling, so its readers:
Reader: src/kiro_crew/decisions/gate.py:_consented_for -- chat -- test_decisions_provider_allowlist.py::TestTheGateHonoursTheList::test_no_policy_sends_to_every_saved_selection
Reader: src/kiro_crew/decisions/gate.py:_consented_for -- cron -- the autonudge judge reads the same consent; test_decisions_judge_llm.py passes unchanged
Reader: src/kiro_crew/decisions/gate.py:_consented_for -- subagent -- browser.step and spawned-task routing funnel through the same read; test_browser_reflex.py passes unchanged
Reader: src/kiro_crew/dashboard/handlers/decisions.py:_carry_consent -- chat -- test_decisions_local_provider.py::TestProviderRoutePut::test_two_switches_never_interleave_write_and_carry
Reader: src/kiro_crew/dashboard/handlers/decisions_sagemaker.py:switch_refused -- chat -- test_decisions_sagemaker_provider.py::test_a_fleet_denial_of_the_hosted_side_refuses_it
Reader: src/kiro_crew/dashboard/handlers/decisions.py:_provider_payload -- chat -- with no list the side answers are unchanged; test_decisions_sagemaker_provider.py::TestRefusedOptionIsNotOffered
Reader: src/kiro_crew/decisions/local_runtime.py:resume_configured -- release -- runs at gateway start; test_decisions_provider_allowlist.py::TestTheRuntimeResume, test_decisions_local_runtime.py
Reader: src/kiro_crew/dashboard/session_control_parts/model_switch.py:_JEV_ROUTE_SPELLINGS -- subagent -- old spellings kept, the new label added; test_session_control_set_model.py::test_auto_is_owner_only
Reader: website/src/pages/settings/DecisionsCard.tsx:DecisionsCard -- chat -- a denied row draws the notice instead of nothing; decisionsCard.test.tsx "says the administrator turned decision models off, and offers no control"
Reader: website/src/pages/settings/DecisionsProviderPicker.tsx:DecisionsProviderPicker -- chat -- refused options shown greyed instead of hidden; decisionsProviderPicker.test.tsx
Reader: website/src/hooks/useSettingHighlight.ts:resolveLegacyHighlightId -- chat -- old ids (developer.decisions-jev, developer.jev-api-key, three scope switches) resolve to the new ones; settingHighlightAliases.test.ts
Reader: src/kiro_crew/guide_catalog.py:_REGISTRY_PATH -- chat -- regenerated registry; test_guide_ui_find.py, test_find_ui_heldout.py
Tests
test/test_decisions_provider_allowlist.py(new, real policy and evaluator): the catalog row;provider_idper address shape; at the gate, no policy and a list-less policy send, a refused model (deny, allow-other, empty allow) sends nothing and leaves the keystone byte-identical, consent counts again once allowed, a hand-written custom address answers to its own entry, a local preset to its own entry, the off switch refuses all, a policy swap between row and entry denies, "none" is never asked; the provider route (per-model GET, 403 and no write for refused jev, preset, SageMaker, "none" never refused); the consent route (403 for a refused model, disabling allowed, GET reportspermits: falsewith consent shown as recorded); runtime resume; the edition snippet.gate._consented_for,switch_refused,_seam_withdrawn,resume_configured, forcingpermitted_providerstrue, removing the generation bracket, and mis-classifying SageMaker each fail the file.website/src/i18n/decisionModelNaming.test.ts(new): every catalog names Jev only under the hosted provider's own keys, with a self-check sample.website/src/pages/settings/decisionModelNaming.test.tsx(new): the card in each state (all allowed with the hosted entry and key open, an allow list of one, every model refused, SageMaker active, a local model active, turned off by the administrator) and the transcript rows (steer line, skill strip) under each active provider: no text,aria-labelortitleoutside the hosted entry names Jev. Mutation-tested: reintroducing "Jev" into a catalog value, dropping the entry marker, dropping the notice, or ignoringprovider_permittedeach fail.decisionsProviderPicker.test.tsx: the key field lives inside the hosted entry only, opens on a search hit, and is not drawn for a refused provider; refused options are greyed, not hidden. Existing card, scope, strip and picker tests updated to the new labels.--platform linux(clean), comment-history, brand-name, file line cap, harness parity, flake-pattern ratchet,npm run i18n:checkwith the merge base (all rows pass),tsc -p tsconfig.app.json, eslint on touched files, jscpd.Manual verification
Ran the real built SPA through the screenshot harness (
website/scripts/capture-decision-model-governance.mjs, sharedserve-distandstub-dashboard-apihelpers) against this branch and against a build oforigin/main(--before), with every/api/**call answered from fixtures; the after run asserts each state's text and that nothing outside the hosted entry names Jev. The steer trace line was captured with the existingcapture-decision-message-steer.mjson both builds. No gateway or live provider was used.Screenshots / video
Committed under
temp-screenshots/decision-model-governance/(light theme):temp-screenshots/decision-model-governance/before/all-allowed.pngtemp-screenshots/decision-model-governance/after/all-allowed.pngtemp-screenshots/decision-model-governance/before/allow-list-one.pngtemp-screenshots/decision-model-governance/after/allow-list-one.pngtemp-screenshots/decision-model-governance/before/admin-disabled.png(no card)temp-screenshots/decision-model-governance/after/admin-disabled.pngtemp-screenshots/decision-model-governance/before/sagemaker-active.pngtemp-screenshots/decision-model-governance/after/sagemaker-active.pngtemp-screenshots/decision-model-governance/before/local-active.pngtemp-screenshots/decision-model-governance/after/local-active.pngtemp-screenshots/decision-model-governance/before/trace/01-queued-light.pngtemp-screenshots/decision-model-governance/after/trace/01-queued-light.pngtemp-screenshots/decision-model-governance/before/trace/02-steered-light.pngtemp-screenshots/decision-model-governance/after/trace/02-steered-light.pngtemp-screenshots/decision-model-governance/before/trace/03-picker-light.pngtemp-screenshots/decision-model-governance/after/trace/03-picker-light.pngAfter, turned off by the administrator:
After, transcript trace row (before it read "Jev chose: run after this turn"):
After: allow list of one model, and all models allowed with the hosted entry and its key open
Related Issues
Maintainer request (Bolin Chen, bolichen97), 2026-11-12: one governance seam over all decision models; the internal edition ships with the hosted provider not allowed.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.