Skip to content

fix(doctor): stop a sandboxed shell reporting the decoder as missing - #31461

Merged
bolichen97 merged 1 commit into
mainfrom
fix/27030-doctor-decoder-vantage
Nov 6, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/27030-doctor-decoder-vantage

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: kirocrew doctor run from an agent's shell on macOS no longer reports an authentic audio decoder as missing and no longer tells the user to reinstall Kiro Crew.

Run from a crewmate's own shell on macOS, kirocrew doctor printed Ignoring ffmpeg-macos-aarch64 in a pinned decoder location: its bytes do not match the digest pinned for that filename. and ffmpeg: ❌ not found with Fix: reinstall Kiro Crew, and counted ffmpeg as an issue. The decoder was authentic (codesign --verify passes). The real failure was a PermissionError staging the decoder's snapshot under the data home's run/ root, which every agent sandbox denies by design.

Why it matters

Doctor is the command a user, or their agent, runs first when something looks wrong. From an agent's shell on a macOS desktop release it named a supply-chain style failure that did not exist and sent the user to reinstall the app for nothing, and the run exited non-zero over it.

Not a goal

  • Changing which decoders authenticate: the pinned digest and the Developer ID signature anchors stay exactly as they are.
  • Making the run/ snapshot root reachable from an agent sandbox, or any other sandbox, seal or mask change.
  • The browser node/npm row from the same report: it came from the retired playwright-cli stack, which feat: add the live browser harness to chat #26950 removed (first released in v0.12.0). origin/main has no such row and no browser_cli/, so its "fall back to os.geteuid()" point has no code left to change.
  • Linux: there the snapshot is a memfd the sandbox allows, so a miss from a confined shell is real and keeps today's verdict.

What changed (motivation → approach → change)

On macOS every decoder except a fixed system one (Homebrew) runs from a private snapshot staged under <data home>/run/. An agent's sandbox denies that root, so from an agent's shell the staging call fails before a single byte is hashed. _authenticated_ffmpeg folded that OSError into the same None as a digest or signature refusal, and _open_authenticated_in logged every None as a digest mismatch; doctor then reported the decoder as absent.

Where From an agent's shell on macOS, before After
transcribe_ffmpeg warning its bytes do not match the digest pinned for that filename. it could not be read or staged for execution (Operation not permitted).
doctor ffmpeg: row ❌ not found, Fix: reinstall Kiro Crew, counted as an issue ⏭ not checkable from this shell, run kirocrew doctor from your own terminal, no issue

An OSError, which only reading or staging the decoder raises, now propagates out of _authenticated_ffmpeg instead of returning None, and _open_authenticated_in logs it as a staging failure, naming only the module-constant filename and the system's text for the errno, never a path. A size, digest or signature refusal keeps the digest message, and a decoder that raised is skipped as before, so the set of decoders that authenticate is unchanged. The execution path (_open_ffmpeg_for_execution) resolves through the same _open_authenticated_in, so there too a decoder that cannot be staged is skipped, now under the staging message. In doctor's Speech-to-Text section (doctor_checks/features.py), a decoder not found from a confined shell (confinement._doctor_vantage_confined(), the signal the task-store skip already uses) on macOS prints the skip row instead of the failure, because from there a miss cannot be told apart from the sandbox's denial. A found decoder, an unconfined run and Linux are unchanged. docs/system-specs/modules/cli.md item 9 states the rule.

Backwards compatibility

Compatible: the same decoders authenticate and execute; only one warning's text and one doctor row from a confined macOS shell change.

Removes nothing: doctor still counts a missing decoder from an unconfined terminal and on Linux, and nothing reads the warning text (git grep "pinned decoder location" finds only the logger).

Tests

  • test/test_transcribe_ffmpeg.py::TestBundledFfmpeg::test_a_snapshot_that_cannot_be_staged_is_not_reported_as_a_digest_mismatch: a snapshot that raises PermissionError(EPERM, ..., "<path>/run") leaves no decoder, logs the staging message and not the digest one, and the path never reaches the log. Skipped on Windows, which stages nothing.
  • test_same_size_replacement_is_rejected_by_digest additionally asserts a real digest mismatch keeps the digest message.
  • test/test_cli_stt_memory_doctor.py::TestDoctorStt::test_doctor_in_a_confined_macos_shell_does_not_report_the_decoder_missing: a desktop release whose decoder is not found, seen from a confined macOS shell, prints not checkable from this shell, no reinstall line, and ffmpeg is not among the issues.
  • test_doctor_in_a_confined_linux_shell_still_reports_the_decoder_missing: the same from a confined Linux shell keeps ❌ not found and the ffmpeg issue.
  • The autouse host-fact fixtures TestDoctorStt._hermetic_doctor and TestDoctor._hermetic_sandbox (test/test_cli.py) now pin _doctor_vantage_confined to unconfined, next to the sandbox backend they already pin. Without that, a macOS run of these files from an agent's own shell takes the new skip row, and eight existing tests that assert the ffmpeg: row (the four test_doctor_stt_marker_arms_match_platform cases among them) fail on that row. Simulated with KIROCREW_SANDBOX_ACTIVE=1, they failed before the pin and pass with it; the two new doctor tests set their own value.

Proof: PROVEN test/test_cli_stt_memory_doctor.py::TestDoctorStt::test_doctor_in_a_confined_macos_shell_does_not_report_the_decoder_missing, test/test_transcribe_ffmpeg.py::TestBundledFfmpeg::test_a_snapshot_that_cannot_be_staged_is_not_reported_as_a_digest_mismatch (head 58284c8)

Determinism: the four tests above and the four test_doctor_stt_marker_arms_match_platform cases passed 20 of 20 runs at -n0 and 10 of 10 at -n 2 --dist loadgroup --max-worker-restart=2 beside test_cli_doctor_refactor_family_reads.py and test_cli_doctor_parts.py; every test in the three changed test files ran green in a shuffled order (seed 24417, 115 passed, 1 skipped). A mutation run broke each new guard in turn (dropping the macOS gate, folding the OSError back into None, logging the error's own text, giving every refusal the staging message) and a named test failed every time.

Run locally on macOS, ten existing TestDoctorStt tests and TestDoctorEmbeddings::test_doctor_blames_the_override_dir_not_the_bundled_tree fail on unmodified main the same way: the macOS tls trust: doctor row reports the system trust store as not enabled under pytest and adds an issue. They pass with SSL_CERT_FILE set. The runs above and prove.py also used a scratch HOME, because the runner's own kiro-cli settings add an unrelated doctor issue that fails the same exit-code assertions on unmodified main. The new doctor tests do not assert the exit code for these reasons.

Manual verification

On macOS 26.7.2 (Apple silicon), against the installed KiroCrew Nightly 0.13.0-nightly.20261105t060755 desktop release and this branch. The agent-shell vantage is the command run inside the Seatbelt profile the gateway confines agent sessions with (sandbox.sandboxed_spawn_argv, mode auto), with a scratch HOME so nothing touched a real data home; the bundled decoder is the release's own kirocrew_ffmpeg/bin/ffmpeg-macos-aarch64, which codesign --verify --strict accepts.

Before (unmodified release, kirocrew doctor, agent-shell vantage). Excerpt in output order: ... marks rows left out, and the stderr WARNING line, which printed glued onto an earlier stdout row, is shown on its own line without its timestamp:

WARNING kiro_crew.transcribe: Ignoring ffmpeg-macos-aarch64 in a pinned decoder location: its bytes do not match the digest pinned for that filename.
...
Speech-to-Text
  provider:    ✅ local
  engine:      ✅ local recogniser loadable (whisper.cpp, in-process)
  model:       ⏹ base not downloaded yet (147 MB, fetched on first use)
  ffmpeg:      ❌ not found
               Fix: reinstall Kiro Crew (the bundled audio decoder is missing)
...
❌ Fix these issues: Kiro Crew's KAS engine is not installed yet (the first chat downloads it), ffmpeg

The same release from an unconfined terminal prints ffmpeg: ✅ available. Neither run has a browser node row. From the agent-shell vantage, sandbox_unavailable.agent_confinement_evidence() returns the kernel reports this process is Seatbelt-confined (with the launcher marker dropped, as cli.main() does), and None unconfined. _open_authenticated_in(<release decoder dir>, allow_signature_anchor=True) on main there logs the digest warning and returns []; the snapshot call itself raises PermissionError errno 1.

After (this branch):

  1. Agent-shell vantage, _open_authenticated_in(<release decoder dir>, allow_signature_anchor=True): Ignoring ffmpeg-macos-aarch64 in a pinned decoder location: it could not be read or staged for execution (Operation not permitted). and [].
  2. Unconfined, scratch data home, same call: 1 (the signed decoder still authenticates through its Developer ID signature). The scratch home was deleted afterwards.
  3. Agent-shell vantage, kirocrew doctor from this branch's source install: ffmpeg: ✅ available, because this Mac has a Homebrew ffmpeg, a fixed system decoder that is not staged.
  4. Agent-shell vantage, the release's own interpreter running this branch's code (so the bundled decoder is the one resolved). Excerpt in output order, with the same conventions as above:
Speech-to-Text
  provider:    ✅ local
  engine:      ✅ local recogniser loadable (whisper.cpp, in-process)
  model:       ⏹ base not downloaded yet (147 MB, fetched on first use)
WARNING kiro_crew.transcribe: Ignoring ffmpeg-macos-aarch64 in a pinned decoder location: it could not be read or staged for execution (Operation not permitted).
  ffmpeg:      ⏭  not checkable from this shell
               Run `kirocrew doctor` from your own terminal to check the decoder.
...
❌ Fix these issues: Kiro Crew's KAS engine is not installed yet (the first chat downloads it)

The same run unconfined prints ffmpeg: ✅ available.

Screenshots / video

N/A: no frontend change. The doctor text is quoted under Manual verification.

Related Issues

Closes #27030

Sibling sweep

Swept: git grep -n "_find_ffmpeg|_open_ffmpeg_for_execution|ffmpeg_source()|_new_executable_snapshot" -- src/
Sibling: src/kiro_crew/transcribe_ffmpeg.py:660 -- changed here
Sibling: src/kiro_crew/doctor_checks/features.py:242 -- changed here
Sibling: src/kiro_crew/dashboard/handlers/core_stt_settings.py -- not affected -- runs in the gateway process, which an agent sandbox never confines
Sibling: src/kiro_crew/dashboard/handlers/core_stt_status.py -- not affected -- runs in the gateway process, which an agent sandbox never confines
Sibling: src/kiro_crew/stt/decoder.py:640 -- not affected -- the decoder fetch runs in the gateway process, never from an agent shell

Pattern harvest

Rule candidate: review-prompt
Pattern: a broad except OSError that returns the same sentinel as a verification refusal, so the caller's message blames the bytes for an environment failure

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license (see the Licensing section of CONTRIBUTING.md).

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 6, 2026 14:31
@bolichen97 bolichen97 added the ci:macos Run the on-demand macOS suite against this PR head label Nov 6, 2026
@bolichen97
bolichen97 requested a review from Zedmor November 6, 2026 14:31
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 58284c869c5a7f55603bb05c234bbba6831cc323 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 58284c869c5a7f55603bb05c234bbba6831cc323 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Both changes fix the code that misreported the failure, and each has one existing caller.

What this change ships

Inventory (3 items) — 3 justified

Intent: when the user's agent runs kirocrew doctor on macOS, a decoder the sandbox blocked is no longer reported as tampered with or missing, and doctor no longer says to reinstall. This is a FIX.

  1. When reading or staging the decoder fails, the warning now says "could not be read or staged for execution" instead of reporting a digest mismatch. Only one function calls _authenticated_ffmpeg. — justified
  2. On macOS, doctor run from a confined shell prints "⏭ not checkable from this shell" for a decoder it cannot find, and does not count it as an issue. It reuses the existing confinement._doctor_vantage_confined check, which the task-store row already uses. — justified
  3. Item 9 of the cli.md spec states the new rule. The spec has to change in the same commit as the code. — justified

[FIRST-PRINCIPLES-REVIEWED] 58284c8

Description read

[DESCRIPTION-READ] 65252aa4a21802c9b2f25c53fa450cb875a3257edba3fd4446cb9eace451bb83

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 58284c869c5a7f55603bb05c234bbba6831cc323 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks. The fix goes to the cause: a staging OSError is no longer reported as a digest refusal, the set of decoders that authenticate is unchanged, and the doctor skip only applies to confined macOS shells.

Notes

  • A decoder that is really missing, seen from an agent's shell on macOS, now also prints ⏭ not checkable instead of an issue. The row tells the user to re-run from their own terminal, and the change is easy to revert, so it is not a blocker.
  • _doctor_vantage_confined() is a broad signal. The skip could be limited to cases where _open_authenticated_in actually hit a staging OSError, so a missing payload still shows ❌ from a confined shell. That would need the resolver to pass that outcome back to doctor, so it fits better as a follow-up.

[DESIGN-REVIEWED] 58284c8

Description read

[DESCRIPTION-READ] 462b790d84cb0614c98b8d304f1aaee3b8a8d2f139352eb618269c3340ec18c5

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 58284c869c5a7f55603bb05c234bbba6831cc323 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 58284c8

Verdict parsed from the review's SHA-scoped output markers for commit 58284c869c5a7f55603bb05c234bbba6831cc323.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 58284c869c5a7f55603bb05c234bbba6831cc323 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 58284c869c5a7f55603bb05c234bbba6831cc323 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 58284c8

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 58284c869c5a7f55603bb05c234bbba6831cc323 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 6, 2026 14:43
@bolichen97

Copy link
Copy Markdown
Contributor Author

[operator: bolichen97] The red Backend Tests (3.12, 2), Backend Tests (Windows) (2) and macOS Tests / Backend Tests (macOS) (2) rows, and the Coverage Gate that fails closed behind them, are main's red, not this change's. Each of the three shards has exactly one failure, test/test_security_gate_liveness.py::test_the_package_stays_within_its_line_budget ("package grew to 34733 lines" against a budget of 34699). origin/main at b3c17ad measures the same 34733 lines on its own: the package went over budget when #30600 merged, and this PR changes nothing under src/kiro_crew/security/.

#31469 raises that budget to main's measured size. This PR waits for it: no rerun and no override for this red. Once #31469 lands, this branch rebases onto it (which also absorbs the docs/system-specs/modules/cli.md change main picked up since this head ran) and CI re-runs.

The earlier failing Security Scope Review row on this head is the concurrency-cancelled run 42234694413; its replacement, run 42234694454, passed on the same head.

On macOS every audio decoder except a fixed system one runs from a
snapshot staged under the data home's run/ root, which every agent
sandbox denies. Run from an agent's shell, kirocrew doctor therefore
failed to stage an authentic decoder, logged it as a digest mismatch,
reported "ffmpeg: not found" and told the user to reinstall.

An OSError, which only reading or staging the decoder raises, now
propagates out of _authenticated_ffmpeg, and _open_authenticated_in
logs it as a decoder that could not be read or staged, naming the
errno's system text and never a path. A size, digest or signature
refusal keeps the digest message, and the set of decoders that
authenticate is unchanged.

From a confined shell on macOS, doctor prints "not checkable from this
shell" for a decoder it did not find, adds no issue, and points at the
user's own terminal. Linux stages into a memfd, so a miss there keeps
its verdict.

Closes #27030
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 6, 2026
@bolichen97
bolichen97 force-pushed the fix/27030-doctor-decoder-vantage branch from 69a9836 to 58284c8 Compare November 6, 2026 16:47
@bolichen97

Copy link
Copy Markdown
Contributor Author

[operator: bolichen97] #31469 merged, so this branch is rebased onto main at 89a6107, which carries it. That fix clears the one failure behind the red Backend Tests (3.12, 2), Backend Tests (Windows) (2) and macOS Tests / Backend Tests (macOS) (2) rows (test/test_security_gate_liveness.py::test_the_package_stays_within_its_line_budget), and the Coverage Gate that failed closed behind them. The change itself is the same patch as before (same git patch-id). The rebase also takes in main's unrelated docs/system-specs/modules/cli.md row for cron preview. The old head's remaining queued run was cancelled before the push. prove.py reports PROVEN on the new head 58284c8, and the PR body's Proof: line names it. CI is re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 6, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by Crew Captain for the on-call, who accepted the open items on this PR. Review lanes clean on this head.

@bolichen97
bolichen97 merged commit d6fc73c into main Nov 6, 2026
105 checks passed
@bolichen97
bolichen97 deleted the fix/27030-doctor-decoder-vantage branch November 6, 2026 17:45
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 6, 2026
buluoray added a commit that referenced this pull request Nov 8, 2026
…31461)

On macOS every audio decoder except a fixed system one runs from a
snapshot staged under the data home's run/ root, which every agent
sandbox denies. Run from an agent's shell, kirocrew doctor therefore
failed to stage an authentic decoder, logged it as a digest mismatch,
reported "ffmpeg: not found" and told the user to reinstall.

An OSError, which only reading or staging the decoder raises, now
propagates out of _authenticated_ffmpeg, and _open_authenticated_in
logs it as a decoder that could not be read or staged, naming the
errno's system text and never a path. A size, digest or signature
refusal keeps the digest message, and the set of decoders that
authenticate is unchanged.

From a confined shell on macOS, doctor prints "not checkable from this
shell" for a decoder it did not find, adds no issue, and points at the
user's own terminal. Linux stages into a memfd, so a miss there keeps
its verdict.

Closes #27030

(cherry picked from commit d6fc73c)

Co-author on main: Bolin Chen
buluoray added a commit that referenced this pull request Nov 8, 2026
…31461)

On macOS every audio decoder except a fixed system one runs from a
snapshot staged under the data home's run/ root, which every agent
sandbox denies. Run from an agent's shell, kirocrew doctor therefore
failed to stage an authentic decoder, logged it as a digest mismatch,
reported "ffmpeg: not found" and told the user to reinstall.

An OSError, which only reading or staging the decoder raises, now
propagates out of _authenticated_ffmpeg, and _open_authenticated_in
logs it as a decoder that could not be read or staged, naming the
errno's system text and never a path. A size, digest or signature
refusal keeps the digest message, and the set of decoders that
authenticate is unchanged.

From a confined shell on macOS, doctor prints "not checkable from this
shell" for a decoder it did not find, adds no issue, and points at the
user's own terminal. Linux stages into a memfd, so a miss there keeps
its verdict.

Closes #27030

(cherry picked from commit d6fc73c)

Co-author on main: Bolin Chen
buluoray added a commit that referenced this pull request Nov 8, 2026
…31461)

On macOS every audio decoder except a fixed system one runs from a
snapshot staged under the data home's run/ root, which every agent
sandbox denies. Run from an agent's shell, kirocrew doctor therefore
failed to stage an authentic decoder, logged it as a digest mismatch,
reported "ffmpeg: not found" and told the user to reinstall.

An OSError, which only reading or staging the decoder raises, now
propagates out of _authenticated_ffmpeg, and _open_authenticated_in
logs it as a decoder that could not be read or staged, naming the
errno's system text and never a path. A size, digest or signature
refusal keeps the digest message, and the set of decoders that
authenticate is unchanged.

From a confined shell on macOS, doctor prints "not checkable from this
shell" for a decoder it did not find, adds no issue, and points at the
user's own terminal. Linux stages into a memfd, so a miss there keeps
its verdict.

Closes #27030

(cherry picked from commit d6fc73c)

Co-author on main: Bolin Chen
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:macos Run the on-demand macOS suite against this PR head

Projects

None yet

Development

Successfully merging this pull request may close these issues.

doctor run from an agent shell reports false ffmpeg and Node failures (sandbox vantage)

2 participants