Repository navigation
feat: let an administrator allow or refuse each decision model - #36271
Conversation
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change adds one inner list to the existing Notes
[DESIGN-REVIEWED] fda7148 Description read[DESCRIPTION-READ] bff31f18d5ce371011a06936f31917ec833f5be19157f76e1f253603ab70f3ae That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Two UI changes ride along with no named user who was failing: the hosted API key field moved, and the picker's button was relabelled. Notes
What this change shipsInventory (10 items) — 8 justifiedIntent: let an administrator refuse decision models, all of them or one at a time, and have the gateway enforce it and the dashboard explain it. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] fda7148 Description read[DESCRIPTION-READ] 9f7a9f249abdc6552ecea3c03daae3b255ec7278f6fbe7974dfd554d00cd10a7 That sha256 names the PR description this verdict read. Recompute it with |
UX Review (Opus 5.5) — ✅ PASS (a BLOCK with no Trigger and no Outcome does not block)UX-level review of UX-Verdict: PASS cannot evaluate: missing the picker's renamed save button ("Use {{name}}" / "Use no model", which replaces "Use this model") in the state where a pick is chosen but not yet saved. None of the 14 screenshots shows it. Evidence gaps
Notes
[UX-REVIEWED] fda7148 Description read[DESCRIPTION-READ] 634ba07f27b05278c5cc22ac064e6f563ca54a0ac5ee40d42b02b6505ce311b6 That sha256 names everything this verdict read: the PR description, plus the 17 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
3e0efa2 to
9de2c8c
Compare
|
Answering the three First Principles notes. Each point is an explicit instruction from the maintainer, Bolin Chen (bolichen97), decided on 2026-11-12; none is a choice this PR made on its own.
The remaining feature-level "Jev" wording, including the notice's title "Decisions (Jev)" that the UX review pointed out, is the wording pass. It ships in the follow-up PR opened on main once this one merges, so this PR changes only the governance wording it introduces. |
Code change approver recommendation: A person should decideWhy: the decider thinks the change is probably fine but estimates a 25% chance that this change introduces a defect, above the 12% limit for approving on its own. I recommend approving. The PR adds a per-model An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
b4360f1 to
06f766d
Compare
06f766d to
1501dc0
Compare
1501dc0 to
302cca6
Compare
302cca6 to
89d65bb
Compare
|
Round 7. Maintainer decision (Bolin Chen, 2026-11-12): "用户自己用custom节点不管我们事情, 我们只要api.typesafe.ai disable就行了, 不用REGEX匹配" (a user's own custom endpoint is not ours to govern; the edition only needs to disable the hosted api.typesafe.ai provider; no regex or host matching). What that changed on this head:
|
89d65bb to
ca05283
Compare
|
Round 8, self-review sweep with the lanes' lens before this push. What it found and what changed: (a) References to a control this PR renamed or removed:
(b) Local state and what can unmount it:
(c) Cached values shown while their refetch fails:
(d) Machine-wide claims:
(e) Loading, error and empty states of the new UI: each has copy. The administrator notice, the listed-models banner, the refusal lines, the key field's vault-read error, the stale provider read, and each preset note. Also in this round:
|
ca05283 to
300ef2a
Compare
|
Round 9 self-review sweep, with the lanes' lens. Docs lint. This PR adds or renames no doc; it changes four existing ones ( Custom-address refusal from before round 7. Fixed in the feature map, the spec, the API client comment, the harness frames and the PR body. Nothing else in specs, user docs, catalogs, Every status or label derived from a denial, and the refusal it actually obeys:
Every state of the card (one combination per row):
|
Governance over decision models knew two special cases, the hosted host and the SageMaker host. Every decision model now has its own entry in an allow list, enforced in the gateway, and the dashboard says plainly what the administrator does not allow. - capabilities.decisions gains a `providers` inner list (SCOPE_CATALOG data, no evaluator change). Ids are the picker's own: jev, each local preset, sagemaker, custom. The configured model's entry is asked with the row in one policy-generation bracket at every check that reads the row for it: the send path, the consent and provider writes, the consent carry, the SageMaker test and the runtime resume. A refused model is neither selectable (403) nor used, whoever wrote config.json; its recorded consent stays in place and inert. No policy, or a policy without the list, allows every model, so saved selections keep working. - GET /api/decisions/provider reports provider_permitted per model. The picker draws a refused model greyed with "Not allowed by your administrator." instead of hiding it, and the card draws a plain "turned off by your administrator" notice when the row is off. - The hosted provider's entry holds its own API key field; the shared block keeps only the settings every model shares. - Specs (decisions, governance) and the user doc describe the list and the notice.
300ef2a to
fda7148
Compare
Problem / Motivation
Goal: An administrator can turn decision models off entirely or allow only specific ones, the gateway enforces it at every write and every use, and the dashboard says plainly which models the administrator does not allow.
Governance over decision models knows only two special cases today: the hosted host and the SageMaker host. An administrator cannot allow Laya and refuse everything else, nor ship an edition with the hosted provider refused while local models stay available. A policy that turns the row off makes the card vanish with no word of why, and a refused hosted option disappears from the picker instead of saying it is not allowed. The hosted provider's API key sits in a "Shared settings" block although only that provider uses it.
Why it matters
A managed fleet (the internal edition first) must be able to ship with one provider not allowed and have that hold in the gateway, not only in the UI, and every user must be able to read why a model is unavailable.
Not a goal
pointslist, asmodelsandendpointsalready are.CONTRACT_VERSION, the evaluator, or the standalone default (no policy: every model allowed).What changed (motivation → approach → change)
Approach. Generalize the existing
capabilities.decisionsceiling instead of adding a parallel mechanism: one more inner list on the same row, asked wherever the row is already asked for the configured provider.Policy keys and data shape.
capabilities.decisions.enabledstays the full off switch.capabilities.decisions.scopes.providers(identifiermatcher, aSCOPE_CATALOGdata row). One entry per decision model, by the picker's own id:jev(hosted),plumb-4b,strands-decider-2b,laya,sagemaker(anysagemaker://address). These are stable policy ids, pinned in a test. "No model" is never asked. An endpoint the owner writes by hand is the owner's own and is not governed by the list (maintainer decision, 2026-11-12); the full off switch (capabilities.decisions.enabled: false) and the existingendpointshost list still apply to it as on main. No list means every model is allowed.Enforcement.
capability.provider_id(endpoint, model)names the configured model;capability.is_decisions_denied(provider=...)asksproviders:<id>together with the row(s) inside one policy-generation bracket (fails closed if the policy moved between the reads). It is asked at every chokepoint that reads the row for the configured provider:gate._consented_for, sodecide,is_enabledandwarmrefuse a refused model whichever writer named it inconfig.json;PUT /api/decisions/provider(switch_refused):403 decisions_capability_denied, nothing written;PUT /api/decisions/consent: an enabling or scope-granting write for a refused model is403; a disabling write is never refused;local_runtime.resume_configured(a refused preset is not started).A refusal never touches the keystone: the recorded consent stays in place and inert, and counts again if a later policy allows that model.
Exact checks, no host parsing.
jevis the configured address equal to the hosted default endpoint, a preset is the address the provider route builds for it,sagemakeris thesagemaker://scheme. Nothing parses or pattern-matches a host, and there is no regex in the new code.Live policy changes. A governance-generation change on the slots frame now invalidates the provider and consent reads as well as the dashboard config (
hooks/websocket/slotList.ts), so an open card follows a policy that allows or refuses a model.Settings read.
GET /api/decisions/provideraddsprovider_permitted: {id: bool}, each the AND of the model's row, theendpointslist for a model that leaves the machine, and its own entry (dashboard/handlers/decisions_offers.py, asked without a SEL row per id since it is an offer; every request that acts is audited).hosted_permitted/sagemaker_permitted/configured_refusedfold the list in, so a configured hosted default the list refuses reads as "No model" with no config rewrite.UI.
role="status"), instead of vanishing. It speaks about this dashboard only and claims nothing about what is sent or runs elsewhere, since a policy may still permit Slack or another channel under a standing consent. Controls the card has drawn stay mounted, hidden, while the policy withholds them, so a typed key (or a SageMaker endpoint, or a budget) is there again when the policy lifts; a failed refetch of the policy shows the read-failure notice, not the last denial.GET /api/decisions/consentreports that refusal aswithdrawn, so a consent recorded for the refused address shows the refusal alone, never the "you agreed to a different address, turn this off and on again" warning, whose enabling write would be refused. When every listed model is refused the card drops its "pick one below" line, and the picker says "Your administrator allows none of the listed decision models.", which never claims nothing is sent (a hand-written endpoint may still be). The picker's switch button names what it writes ("Use Laya", "Use no model"). Over a refused model still saved in the settings (the hosted default, or a SageMaker endpoint), no option is drawn selected and nothing is "In use"; the saved model is tagged "Saved, not allowed", and its entry says "{{name}} is still saved in your settings, but your administrator does not allow it. Clear it to stop using decision models, or pick another model below." with a Clear from settings button that writes "No model". Picking "No model" instead offers the usual Use no model.DecisionsHostedKeyField.tsx), drawn while that entry is chosen; the shared block keeps only the shared settings. A settings-search hit on the key opens nothing: it lands on the hosted provider's entry and highlights it (the registry entry insettingsManual.tsanchors the id on the entry, so the highlight waits for it behind the lazy picker), and the reader opens the entry, and so the key, by choosing it. The flash is a dashed neutral outline (data-setting-flash="neutral"), not the accent ring a selected option wears. Once drawn, the field stays mounted (hidden) while another model is picked, the policy starts refusing the hosted provider, or a re-read fails, so a typed, unsaved key is not lost. The field reports a failed vault read itself ("Could not load secrets: ...") and offers no Save until the read succeeds.decisions_admin_disabled,decisions_all_models_blocked,decisions_configured_blocked,decisions_provider_blocked,decisions_hosted_refused,decisions_configured_blocked_consent,decisions_provider_clear_saved,decisions_provider_saved_refused,decisions_provider_downloaded) ship in all 12 catalogs, verified withi18n-translate.mjs verify.Specs and docs, same commit.
docs/system-specs/modules/decisions.md(section 7, new 7.2, 7.1),governance.md(theproviderslist), and the user docsrc/kiro_crew/docs/decisions.md(new "When your administrator limits decision models" section). No entry indocs/decisions/covers these placements.What the internal edition sets. In the edition companion's bundled
security_policy.json:capabilities.decisions.scopes.providers, value:{"mode": "deny", "deny": ["jev"]}(the hosted provider the picker offers, and a config naming the exact hosted default endpoint)capabilities.decisions.scopes.endpoints, value:{"mode": "deny", "deny": ["api.typesafe.ai"]}(the hosted host in any spelling: only the exact hosted address counts asjev, so a hand-written near copy is refused by theendpointshost list, which the send path asks for every address){"version": 1, "capabilities": {"decisions": {"scopes": {"providers": {"mode": "deny", "deny": ["jev"]}, "endpoints": {"mode": "deny", "deny": ["api.typesafe.ai"]}}}}}If the bundled policy already names
capabilities.decisions, mergescopes.providersandscopes.endpointsinto that object; the full off switch iscapabilities.decisions.enabled = false. A governance profile bound to a surface can carry the samescopes.providersobject to narrow that surface only.test_decisions_provider_allowlist.py::TestTheEditionSnippetpins this fragment.Backwards compatibility
Compatible: no policy, or a policy without the
providerslist, allows every model, so every saved provider selection, consent and config keeps working; no config key, API field, ceiling key or settings-search id is renamed or removed (provider_permittedand the consent read'swithdrawnare added; the settings-search entry for the hosted key keeps its id and moves its anchor, below).The diff tightens what a policy can refuse and changes what the card draws under a ceiling, so its readers:
Reader: src/kiro_crew/decisions/gate.py:_consented_for -- chat -- test_decisions_provider_allowlist.py::TestTheGateHonoursTheList::test_no_policy_sends_to_every_saved_selection
Reader: src/kiro_crew/decisions/gate.py:_consented_for -- cron -- the autonudge judge reads the same consent; test_decisions_judge_llm.py passes unchanged
Reader: src/kiro_crew/decisions/gate.py:_consented_for -- subagent -- browser.step and spawned-task routing funnel through the same read; test_browser_reflex.py passes unchanged
Reader: src/kiro_crew/dashboard/handlers/decisions.py:_carry_consent -- chat -- test_decisions_local_provider.py::TestProviderRoutePut::test_two_switches_never_interleave_write_and_carry
Reader: src/kiro_crew/dashboard/handlers/decisions_sagemaker.py:switch_refused -- chat -- test_decisions_sagemaker_provider.py::test_a_fleet_denial_of_the_hosted_side_refuses_it
Reader: src/kiro_crew/dashboard/handlers/decisions.py:_provider_payload -- chat -- with no list the side answers are unchanged; test_decisions_sagemaker_provider.py::TestRefusedOptionIsNotOffered
Reader: src/kiro_crew/decisions/local_runtime.py:resume_configured -- release -- runs at gateway start; test_decisions_provider_allowlist.py::TestTheRuntimeResume, test_decisions_local_runtime.py
Reader: website/src/pages/settings/DecisionsCard.tsx:DecisionsCard -- chat -- a denied row draws the notice instead of nothing; decisionsCard.test.tsx "says the administrator turned decision models off, and offers no control"
Reader: website/src/pages/settings/DecisionsProviderPicker.tsx:DecisionsProviderPicker -- chat -- refused options shown greyed instead of hidden; decisionsProviderPicker.test.tsx
Reader: website/src/pages/settings/DecisionsHostedKeyField.tsx:DecisionsHostedKeyField -- chat -- the hosted API key field moves from the card's shared block into the hosted provider's entry, drawn while that entry is chosen; decisionsProviderPicker.test.tsx "draws the key field inside the hosted entry while that entry is chosen, and nowhere else"
Reader: website/src/components/commandPalette/settingsManual.ts:developer.jev-api-key -- chat -- the settings-search id is unchanged, its anchor moves from the key field to the hosted provider's entry, which the hit highlights without opening; decisionsHostedKeyHighlight.test.tsx
Tests
test/test_decisions_provider_allowlist.py(new, real policy and evaluator): the catalog row;provider_idper address shape; at the gate, no policy and a list-less policy send, a refused model (deny, allow-other, empty allow) sends nothing and leaves the keystone byte-identical, consent counts again once allowed, a local preset answers to its own entry, the off switch refuses all, a policy swap between row and entry denies, "none" is never asked; the provider route (per-model GET, 403 and no write for a refused hosted provider, preset or SageMaker, "none" never refused); the consent route (403 for a refused model, disabling allowed, GET reportspermits: falsewith consent shown as recorded); runtime resume; the edition snippet.gate._consented_for,switch_refused,_seam_withdrawnorresume_configured, forcingpermitted_providerstrue, removing the generation bracket, and mis-classifying SageMaker each fail that file.decisionsCard.test.tsx(the administrator notice and no control; the credential tests now open the hosted entry; a refused SageMaker address),decisionsProviderPicker.test.tsx(refused options greyed, not hidden; the key field lives inside the hosted entry only, is not drawn for a refused provider),previewSurfaces.test.tsx(the card's button census).decisionsProviderPicker.test.tsx"keeps a typed, unsaved key while another model is looked at".decisionsCard.test.tsxkeeps "Sent to" with the refusal after it;decisionsProviderPicker.test.tsx"selects nothing, marks the saved model as saved and refused, and offers to clear it" and "keeps a typed key when the policy starts refusing the hosted provider".decisionsCard.test.tsx"keeps a recorded consent revocable under the refusal" shows the refusal and no moved-address warning,decisionsPreview.test.tsreadswithdrawn, andtest_decisions_provider_allowlist.py/test_decisions_consent.pypin the field (true under a refusal, false for a moved address);decisionsHostedKeyHighlight.test.tsx(a search hit highlights the hosted entry and opens and chooses nothing) and the picker's one-anchor test. The search auto-open, its latch and its tests are gone.useWebSocket.frameTrace.test.tsrecords the two extra invalidations on a governance generation change (the Frontend Tests failure on the previous head).TestAHandWrittenEndpointIsTheOwners(no list id for a hand-written endpoint, a near copy of the hosted address and a SageMaker runtime URL included; no list, allow-none, allow-jev or deny-every-listed, stops it on the send path, the consent PUT or the provider read; the off switch still does); the exact hosted default is the hosted entry; the stable ids pinned literally; the two 403 sentences;decisionsCard.test.tsx(a hand-written endpoint is never shown refused, even with the local row withdrawn; the egress stays disclosed under a refusal with a recorded consent; the listed-models banner text). The host classifier,impl_sagemaker.is_runtime_hostand their tests are deleted. The key-field waits indecisionsProviderPicker.test.tsxcarry a named ceiling (HOSTED_KEY_FIELD_DRAWS).decisionsCard.test.tsx"keeps a typed key through a live withdrawal and gives it back when the policy lifts", "shows a failed refresh of the policy, not the last denial as if it were current", and the notice text (about this dashboard only);decisionsProviderPicker.test.tsxnames the drawn button in the ready note, gives no step for a model already configured, and gives the administrator's line instead of steps for a refused model, ready or being prepared;TestTheEditionSnippetrefuses the hosted host in three spellings and leaves an address of the owner's alone.capture-decisions-local-runtime.mjsdrives the renamed button.test_decisions_provider_allowlist.py"the judge llm lane stays on under a refused model" (GET and PUT; the off switch still turns it off);decisionsProviderPicker.test.tsx"selects nothing, marks the saved model as saved and refused, and offers to clear it" and "clears a refused hosted default from the settings, from inside its entry";decisionsCard.test.tsxthe refusal comes first, before the egress sentence and "Sent to", the consent wording, and no share line.docs/feature-map/README.mdgains the "Decision model allow list" and "Hosted provider API key" rows.jevdeny on the send path, the consent PUT, the provider read andswitch_refused(TestNearCopiesOfTheHostedAddress); the consent read reports no point on for a refused model;useWebSocketGitlabHosts.test.ts"re-reads the decision-model answers on a governance-ceiling generation change";decisionsCard.test.tsx"a configured model the administrator does not allow" (three cases); the named switch button indecisionsProviderPicker.test.tsx. The credential waits indecisionsCard.test.tsxcarry a named ceiling (HOSTED_KEY_DRAWS) over the lazy chain.--platform linux, comment-history, brand-name, file line cap, harness parity, flake-pattern ratchet, the code-review rule checks,npm run i18n:checkwith the merge base,tsc -p tsconfig.app.json, eslint on touched files, jscpd.Manual verification
Ran the real built SPA through the screenshot harness (
website/scripts/capture-decision-model-governance.mjs, sharedserve-distandstub-dashboard-apihelpers) against this branch and against a build oforigin/main(--before), with every/api/**call answered from fixtures; the after run asserts each state's text. No gateway or live provider was used.Screenshots / video
Each pair shows main first, then this branch. Light theme.
Turned off by the administrator, before (no card at all, the page ends at the group above) and after:
Allow list with one model (Laya), before (the policy cannot express it; the hosted provider and SageMaker were hidden when refused) and after:
All models allowed with the hosted entry chosen (its API key inside the entry), SageMaker active, a local model active
The three refusal states this PR adds (after only; main has none of them):
Every listed model refused by the allow list: every option greyed except "No model", with the sentence about the listed models above them:
A configured local model the list refuses, with a consent recorded for it: the refusal first, saying the switch is the recorded agreement, the egress still said, no share line, "Sent to" naming the address, no "agreed to a different address" warning, and the switch still revocable:
A refused hosted default still saved: no option selected, the hosted entry tagged "Saved, not allowed", and inside it the sentence saying it is still saved and what clearing does, with Clear from settings:
A settings-search hit on the hosted provider's key while a local model is in use: the hosted entry is highlighted (a dashed neutral flash, unlike the selected option's ring), nothing opens, the model in use stays selected, and no switch is queued:
Related Issues
Supersedes #36267 (split for the review lanes' diff limit). Maintainer request (Bolin Chen, bolichen97), 2026-11-12.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.