Repository navigation
Conversation
4c288b0 to
6bb0bd8
Compare
UX Review (Opus 5.5, fork) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing blocks. The nine screenshots show every changed surface, but this fork lane has no blind first-time reader, so whether users understand it is still unknown; pushing the branch to this repository runs one. Notes
[UX-REVIEWED] 93244ab Description read[DESCRIPTION-READ] c01f6031dd2d35237a0ab5e7594d7b96c2c87646792d2515736ae887e55066e2 That sha256 names everything this verdict read: the PR description, plus the 12 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the gate fails closed (exact version pin, mode read-back, unasked-call tripwire, Notes
[DESIGN-REVIEWED] 93244ab Description read[DESCRIPTION-READ] a123b87ba5a704df6c7ee39eddab86bd7244b783b83815468068c57cf0514df7 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS
Not justified as shipped
What this change shipsInventory (9 items) — 7 justifiedIntent: a Qoder user can run Kiro Crew on qodercli, with every tool call still going through Crew's gate. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 910771c Description read[DESCRIPTION-READ] 4bb019e3ff3cf0f651403f64bfb83168150ec4bc663515f4484d588aec0b5fbd That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — 🔴 changes requested (blocking)Reviewed Blocks: the background Qoder catalog refresh is cancelled after the same 10s limit as the request, so the probe never gets its own launch bound and a slow cold start can never refresh the picker. BLOCKING — src/kiro_crew/dashboard/handlers/agents_models.py:105 Trigger: GET /api/models with a Qoder catalog older than 600s calls |
GPT 6.1 Review (fork) — 🔴 changes requested (blocking)Reviewed 1 of 2 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands. BLOCKING -- src/kiro_crew/acp/harness/qoder.py:109 -- Qoder MCP approvals bypass the MCP governance ceiling BLOCKING -- src/kiro_crew/model_registry.py:554 -- Labeled catalog admission suppresses overflow reporting [BLOCK-MERGE] 93244ab Adjudication (Opus 5.5) — is blocking on each finding proportionate?F2 — the harm reading would be LOW: a too-large catalog is trimmed with no warning, and the next session's refresh writes over it. But the finding's anchor, F1 (fenced) — UNBOUNDED, a governance-ceiling bypass. It only needs an operator who denies an MCP tool by policy and runs a Qoder session. That is ordinary operation, and the new |
6bb0bd8 to
9e02d81
Compare
9e02d81 to
287504f
Compare
910771c to
f3216ac
Compare
f3216ac to
ef61cea
Compare
ef61cea to
93244ab
Compare
|
Dispositions for fixed — GPT 6.1 blocking, The Qoder witnesses no longer run inside Kiro's initialization or entitlement-probe paths. Kiro's initialize and entitlement paths are back to their original shape; the Qoder-specific work happens in fixed — Design CONCERNS, probe leaves a Qoder session behind every 10 minutes.
fixed — First Principles CONCERNS,
fixed (copy) / needs screenshots — UX BLOCK. The three stop notices and the version refusal no longer use internal wording. The unasked-call notice now reads "Qoder ran a tool without asking you first, so Kiro Crew stopped this chat. Start a new chat."; the mode-change notice reads "Qoder stopped asking before it runs a tool, so Kiro Crew stopped this chat. Start a new chat."; the version refusal names the exact install command ( |
93244ab to
d730ed1
Compare
d730ed1 to
247e86e
Compare
|
Fixes for the three blocking findings + UX/Design notes on F1 (GPT, security — MCP governance bypass): Tests: F2 (GPT — overflow reporting): Test: F3 (Opus — probe timeout): Test: UX notes:
Full test suite (10 files): 870 passed, exit 0. Lint: isort + black clean. |
Qoder CLI serves ACP natively (`qodercli --acp`). This makes it a selectable
harness whose every tool call must reach Kiro Crew's gate, or the session is
refused or stopped.
Routing: a new `Routing.PINNED_ASK_RULE`. qodercli exposes no resolved-config
read-back, so the guarantee is argv-pinned and runtime-witnessed instead of
verified:
- spawn argv pins `--settings {"permissions":{"ask":["*"],"defaultMode":"default"}}
--permission-mode default --strict-mcp-config` (never --allowed-tools,
--dangerously-skip-permissions or --config-dir);
- exact measured-version gate (1.1.67 only; anything else is refused);
- default-mode read-back on open/restore and a mid-session mode tripwire
(current_mode_update and the mode config option);
- unasked-tool-call tripwire keyed by toolCallId: an executing frame needs a
prior ALLOW answer for that call, armed by the first live prompt so replayed
history cannot trip it;
- allow_always is never selected and a refused one is recorded as a denial;
- QODER_* variables that can relocate config/home or set permissions are
stripped from the child env; the PID reclaim requires the `--acp` argv token.
Models: Thinking effort and the model write go through session config options.
The picker lists the models the harness itself advertises (ids are the wire
values, Qoder's display names are the labels) and refuses ids it did not
advertise. The catalog is re-read on every session, on an unknown-id refusal
(one refresh, then refuse) and, for the picker, through a TTL refresh that runs
a handshake-only probe (no prompt, no model call) and fails closed: a failed
probe keeps the last snapshot and never widens it.
Also: `~/.qoder/.auth` is declared as the (inferred, never read) credential
leaf; qoder joins the independent-setup list so onboarding offers it.
Capability sets that were not measured (compaction, steering, eviction,
session sharing, ...) stay out.
Evidence is committed as scrubbed live frame corpora (1.1.67 handshake,
pinned ask, load replay with and without a tool call, the 17-model catalog, an
unpinned bypass control, session-array MCP ask) and documented per version in
harness-onboarding.md, including that the docs' "PreToolUse allow skips the
permission pipeline" is not what 1.1.17 or 1.1.67 do for an `ask` rule.
Supersedes kirodotdev#15038. Refs kirodotdev#13803.
247e86e to
8d7dd92
Compare
Problem / Motivation
Goal: Qoder CLI can be chosen as the agent harness in Kiro Crew, with every tool call it makes still reaching Kiro Crew's approval gate (or the session being refused or stopped).
Qoder CLI serves ACP natively (
qodercli --acp) and #13803 asks for it as a backend. #15038 landed it as a known-but-not-selectable id because two things that decide the approval gate were unmeasured: whether an operator's own Qoder permission rules or hooks can run a tool without raisingsession/request_permission, and where the sign-in lives. Those measurements are now done on a real binary, so this PR makes it selectable and replaces #15038 (Supersedes #15038; Refs #13803).Why it matters
Without it, a user who already works in Qoder cannot use it from Kiro Crew, and the onboarding page tells them it is "not ready on this host". Making it selectable without a measured gate would be worse than leaving it dormant: a harness whose tool calls bypass Kiro Crew's deny rules and audit log.
Not a goal
/compact, session eviction or sharing: measured as not available over ACP or not measured, so they stay out of their capability sets.~/.qoder/settings.jsonallow rules/hooks) and whereallow_alwayswould persist: not measured (needs writes to the operator's real config).What changed (motivation → approach → change)
Gate. qodercli cannot read back its resolved permission rules, so nothing is claimed as "verified"; the measured behaviour is in
harness-onboarding.md. NewRouting.PINNED_ASK_RULEpins--settings {"permissions":{"ask":["*"],"defaultMode":"default"},"disableAllHooks":true},--permission-mode defaultand--strict-mcp-config(measured to beat a project file that allows everything, an allow-returning hook and--allowed-tools, and to gate MCP tools), and adds: an exact version gate (1.1.67; the refusal names the install command), a default-mode read-back plus a mid-session mode tripwire, an unasked-tool-call tripwire keyed bytoolCallId(armed by the first live prompt, bounded bookkeeping, fails closed),allow_alwaysnever selected,QODER_*env scrubbed, PID reclaim by the--acpargv token. Hooks: with the old argv a project.qoder/settings.jsonPreToolUse command hook fired outside Kiro Crew's gate and audit; withdisableAllHooksinline it does not (probe on the real 1.1.67: marker written in 4 of 4 control runs, 0 of 6 runs with the key; the permission request was still raised for Bash, Read, Write and an MCP tool; the inline value also beat a project-tierdisableAllHooks:falseinsettings.jsonandsettings.local.json). Cost: the operator's own user-level and plugin hooks do not run in Crew-run Qoder sessions (the Kiro-derived spec and the Claude Side Chat already accept that). Not measured: user-level hooks, non-command hook types, events other than PreToolUse, a mid-session settings rewrite, managed tiers, other releases (docstring and onboarding doc list them). A mode move that arrives after the session response and before the first prompt is no longer checked in the init drain (that keeps the Kiro drain unchanged, H13); it is caught by the unasked-call tripwire when a tool call starts.Models and effort. The picker lists the models the harness advertises, with display names (the id stays the wire value; unlisted ids are refused). The catalog is re-read per session and by a 10-minute handshake-only probe that fails closed; it is separate from
_CatalogCachebecause Qoder's catalog must survive a restart (observed-at stamp). Effort levels are ordered least to most effort for Qoder only (ACP_BACKENDS_EFFORT_LADDER_ORDERED);change_effortno longer reports success for a write that was not made. Labels apply to exclusive catalogs only.Sign-in and the rest. No ACP
authenticateis sent (measured: a browser round trip per session); a signed-out-32000maps to "runqodercli login", Qoder only.~/.qoder/.authis the credential leaf. The gate lives in a new owner module,acp/client_qoder.py; shared functions carry only guarded hooks. Review lane: the change editstool_gate.py, a sensitive path, so it needs the maintainer take-over.Where the Qoder branches live. The runtime witnesses no longer sit inside shared paths.
ProcessAdaptergained three seams with no-op defaults,on_initialized,on_session_openedandcatalog_probe, andQoderLaunchimplements them.client_start.pyandclient_model_selection.pynow call the seams unconditionally, so neither file tests for Qoder or forACP_BACKENDS_ADVERTISED_MODEL_EXCLUSIVE;git grepfor either name in those two files returns nothing. The newCatalogProbePolicyanswers two separate questions: whether a probe may run at all, and which verb ends the session that answered it. Qoder answers withsession/close, taken from the recorded 1.1.67 frames, so the 10-minute catalog probe no longer leaves a session behind on every poll.provider_models.jsonkeeps its id-list shape for every provider; only the catalog whose host authors labels (Qoder) is stored as stamped rows, so adding labels for one backend no longer changes the persisted shape for the rest.Backwards compatibility
Compatible: the new routing member, sets and tables only add a harness; no existing harness's accepted values change. No existing harness changes behaviour. The one place a shared path was touched:
change_effortstill treats an adapter's "unknown config option" answer as before for every harness (skipped, no session reset); only Qoder, whose effort option can disappear after a model switch, reports failure and rolls the effort push back.persist_advertised_modelsnow writes each provider's rows throughadvertised_model_rows, so an ids-only catalog (kiro/KAS, old sidecars) survives a restart instead of being written as empty rows. The model-label change is display-only and limited to exclusive catalogs. Downgrade: the advertised-model sidecarprovider_models.jsonis now written as{"version": 1, "providers": ...}with rows and an observed-at stamp;model_registry._load_advertised_modelsreads both the old ids-only shape and the new one, but an older build reading the new file finds no lists and drops its cached picker catalogs until the next session repopulates them (nothing is lost permanently). The observed-at stamp is persisted only for exclusive catalogs (Qoder), so another backend's session start does not rewrite the file unless its catalog changed.Nothing that
mainserves today is removed. The one take-away shape the PR Hygiene check detects is the sandbox mask the Qoder adapter passes at spawn (extra_hidden_dirs, the same refuse-then-mask preflight its sibling harnesses run). qodercli was a known id but never spawnable onmain, so no existing session reads a path that mask hides; the readers it could affect are the three entries that spawn harness sessions:Reader: src/kiro_crew/acp/harness/qoder.py:QoderLaunch.resolve_spawn -- chat -- qodercli was not spawnable on main, so no chat session read a path this mask now hides; test_acp_qoder_backend.py::test_credential_declaration_is_the_inferred_whole_auth_directory pins that the harness's own credential directory is carved out of the mask
Reader: src/kiro_crew/acp/harness/qoder.py:QoderLaunch.resolve_spawn -- subagent -- a subagent only reaches qodercli through the same selectable harness and the same spawn plan, with the same carve-out and no earlier Qoder subagent session to break; test_acp_qoder_backend.py::test_spawn_argv_pins_ask_rule_and_default_mode_and_has_no_escape_hatch
Reader: src/kiro_crew/acp/harness/qoder.py:QoderLaunch.resolve_spawn -- cron -- a cron child that selects Qoder takes the same spawn plan; there was no working Qoder cron on main to break; test_acp_qoder_backend.py::test_credential_declaration_is_the_inferred_whole_auth_directory
The catalog sidecar still accepts the old id-list shape and the new labeled shape in the same file, keyed per provider, so an older build reading the file sees the ids it expects and a downgrade keeps working.
Tests
test_acp_qoder_backend.py,test_acp_qoder_models.py,test_acp_qoder_effort_option_refresh.py: argv, version gate (1.1.17, 1.1.66, 1.1.68, 2.0.0, absent refused), mode read-back and tripwire, unasked-call tripwire lifecycle (reused ids, answer required, replay exemption, deny then run),allow_alwaysnever chosen, env scrub, catalog refresh add/remove/fail-closed/TTL/one-refresh-then-refuse, probe never prompts and kills the child,autohandling, effort option refresh and no false success.Replay tests over scrubbed live corpora in
test/fixtures/acp_frames/qoder/(1.1.67 handshake, pinned ask, load replay with and without a tool call, the 17-model catalog, an unpinned bypass control, session-array MCP ask).Guards updated or satisfied: capability sets, host-contract parity, harness parity, PID lifecycle exact partition, backend cards, black baseline, SDK boundary (the probe is reached through
agent_sdk.drivers.acp), subagent scratch seam registry, log-sanitization sites, configuration doc backend table.Sign-in: no
authenticateis ever sent (normal start and catalog probe); the-32000signed-out error maps to theqodercli loginremedy only for Qoder with the exact code and message; the same code with other text, and non-Qoder clients, stay plain errors.Tripwire bookkeeping: 20 approved-and-completed calls retain nothing; an approved call's mapping survives eviction pressure and a late reject still trips; overflow with every slot protected stops the harness (killed, audited, no leaked entries); every terminal status then a reused id trips; a duplicate terminal is treated as unasked (fail closed).
Replay snapshots:
v167-load-replay-liveandv167-load-replay-toolcall-livewere re-recorded once for themessage_idkeymainnow emits on thinking/text chunks (7 added lines, nothing else changed).Persistence: an ids-only catalog survives persist and reload; a changed kiro catalog replaces the cached rows instead of persisting stale ones; the aws_control packaging fence and the shared validator agree on the
.qoder/.authcredential leaf.Effort ordering unit tests (Qoder order, unknown level keeps advertised order, every other harness list unchanged) and the effort-option refresh after a model switch.
Frontend (vitest): label mapping, ChatPanel default-model/selectable-models/pin-notice labels, reply-footer label, the composer pin row showing the model name rather than its id, unknown context window handling (including cache clear).
Local runs on the final commit: the broad guard/ACP/permission/model run passed (the only red were two
test_acp_bindings.pytests that were failing onmainuntil test(providers): list approval_withdrawable and approve_tool's new signature #20902, which this branch now includes, and a dashboard boot-chain test that failed only in my worktree because it holds a built, git-ignoredwebsite/distregistering 8 extra app-window routes; it passes on a cleanorigin/maincheckout). A related-tests gate run showed onecode_review_sagetest failing under load that passes alone.Seam routing:
on_initializedruns beforesession/loadandsession/newand a refusal there opens no session;on_session_openedsees the response that opened the session and itsresumedflag; every otherProcessAdapterkeeps the no-op default (asserted by the harness-parity suite).Catalog probe: the probe's verb order is
initialize→session/new→session/close, and a repeated probe does not grow the host's on-disk session count.Sidecar shape:
test_a_labeled_catalog_is_the_only_entry_whose_shape_changespins that an ids-only provider stays an id list while the labeled catalog carries its stamp.Copy: the three stop notices and the version refusal assert the plain-language wording, including the install command in the refusal.
Manual verification
Built from source (
make build) and driven through a local gateway with the measured qodercli: the Agent Harness card lists Qoder; a mutating Bash call raises Kiro Crew's[a] allow once / [d] denyprompt and runs only after allow; deny creates nothing; a workspace whose own.qoder/settings.jsonallows everything and setsbypassPermissionsstill gets prompted for every call; the spawned process shows the pinned argv;GET /api/modelsreturns the advertised catalog with names. A real session-less catalog probe returned the 17 models in about 9 s and left no process or temp directory behind; pointed at 1.1.17 it returned nothing (refused) in under 2 s.Not verified: operator USER-level Qoder settings and hooks, where
allow_alwayspersists, and other qodercli releases. A Windows-only CI failure,test_pathres_loop_starvation.py::test_the_inline_anchors_are_the_bounded_anchors(an extra\\?\-prefixed spelling of a pytest temp directory in the inline set), appeared on one CI run of this branch and not on the previous one with no path-resolution change in between; nothing in this PR touches that code. A later Windows run (shard 7) failedtest_app_mcp_tool_policy_scope, which passes locally on macOS;mainhas since changed that test andmcp_shared.pyfor an app on another Windows drive (#20802) and this branch is rebased onto it, but I could not re-run Windows to confirm that it was the cause. The effort slider appears only after a session exists on a reasoning-capable model (Qoder offers no effort option while the model isauto).Screenshots / video
Built from this branch and run against the measured
qodercli1.1.67 through a local gateway (the two failure states use a gateway with no sign-in and one pointed at 1.1.17). The surfaces touched: the Agent Harness card, the Settings → Chat → Model lists, the composer model picker and chip, the reply footer, and the two refusal messages a user can hit when starting a Qoder chat.Agent Harness card: the Qoder card with its supported-feature list; the card says Kiro Crew makes it always ask before each tool, and the sign-in line appears once
Settings → Chat → Model: the Default Model select opened, listing Qoder's display names (the id stays the wire value)
Chat reply: a real Qoder turn; the reply footer and the composer chip show the model's display name, and the composer has no context meter because Qoder reports no window size
Composer model picker: the picker lists the models Qoder advertises with their names and credit multipliers; the pin row names the model, not its id
Not signed in: a qodercli with no login answers
Authentication required; Kiro Crew says what to do and opens no browser tabUnsupported release: qodercli 1.1.17 is refused with the detected version and the one command that fixes it; the chat shows only that remedy (the gate explanation goes to the log), and Resume works once the right release is installed
Settings → Chat → Model, Selectable Models: the list opened, showing Qoder's display names with their credit multipliers
Settings pin notice: with an explicit default model and an agent pinned to Ultimate, the notice names the pinned model by its display name. The pin itself is simulated: the screenshot was taken against the live gateway with the agent roster response intercepted, because writing a real agent pin would edit the operator's shared agent home
Composer effort block: the model picker on a reasoning-capable Qoder model (Ultimate) with the effort slider and the "Use default effort" switch off (Medium selected)
Not shown: the stop that follows a mode change or an unasked tool call (a real qodercli cannot be made to do that without defeating the gate; the unit tests cover it).
Related Issues
Supersedes #15038. Refs #13803.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.