Skip to content

fix(spawn): isolate the bundled interpreter from user site - #11846

Merged
bolichen97 merged 1 commit into
mainfrom
fix/bundled-interpreter-user-site-isolation
Sep 19, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/bundled-interpreter-user-site-isolation

Conversation

@pepmach

@pepmach pepmach commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Kiro Crew-owned children launched directly with the bundled Python interpreter could load packages from the user's site directory. A package under ~/.local could therefore override a bundled dependency and break gateway startup or another internal service after an update.

Why it matters

The desktop bundle is expected to run against its shipped dependency set. Ambient user packages make startup and repair behavior depend on machine-local state that Kiro Crew does not control.

What changed (motivation → approach → change)

Added platform_compat.isolated_python_argv, which inserts CPython's -s flag for the bundled interpreter and parents whose user site is already unavailable. It preserves existing -s, stronger -I, attached -m…/-c… targets, interpreter option values, executable selection, PYTHONPATH, and current-directory import semantics. A non-bundled parent that allows user site keeps that policy because Kiro Crew itself may be installed there. Only callers whose launcher supplies the child's import path, or whose target is a script or entry path that does not import kiro_crew from the user site, set force_isolation=True. The path-based dependency shim is one such launcher-owned path. A PYTHONPATH inherited from the operator or declared by an app manifest is not, so neither case forces isolation. A module-style -m kiro_crew... child does not force isolation; the bundled interpreter still receives -s automatically. Existing sandbox launchers keep -I -S; user scripts and project tools remain outside this contract.

Routed the inventoried internal service, installer, re-exec, app backend, MCP gateway/stub, overlay, pod fallback, test harness, app-registry build, resident Piper worker, and dependency-repair boundaries through that helper. Generated MCP stubs use the isolated prefix while legacy overlays remain readable. The MCP overlay fingerprint includes the helper's effective isolation prefix, so a changed bundle or user-site policy regenerates cached argv instead of serving the previous -s decision. The platform compatibility system spec records the cross-cutting rule.

Ownership map:

  • src/kiro_crew/platform_compat.py owns argv construction and re-exec; src/kiro_crew/agent.py and src/kiro_crew/mcp_discovery.py keep managed MCP fallback and probe classification aligned.
  • src/kiro_crew/apps/ covers app backends, app MCP commands, registry builds, Dev Fleet, and app installers; src/kiro_crew/mcp_gateway/ covers daemon and generated-stub launch paths.
  • src/kiro_crew/cli.py, src/kiro_crew/cli_server.py, src/kiro_crew/computer_use/, and src/kiro_crew/dashboard/ cover CLI fallbacks, overlay launch, and runtime installers.
  • src/kiro_crew/piper_runtime.py, src/kiro_crew/pod/, src/kiro_crew/slack/, and src/kiro_crew/testing/ cover the resident Piper worker, pod fallback, dependency repair, and harness children.

Observed argv evidence from the isolated verification pod:

Before:
python3.12 -m kiro_crew.mcp_gateway.gatewayd
python3.12 -m kiro_crew.apps.builtins.dev_fleet.server
python3.12 -m kiro_crew.mcp_gateway.stub

After:
python3.12 -s -m kiro_crew gateway
python3.12 -s -m kiro_crew.mcp_gateway.gatewayd
python3.12 -s -m kiro_crew.apps.builtins.dev_fleet.server
python3.12 -s -m kiro_crew.mcp_gateway.stub

Tests

  • Added a spawn inventory test covering the shared helper and all 23 owned bundled-interpreter boundaries, including the resident Piper worker.
  • Red-first coverage includes test_effective_user_site_policy_change_invalidates, test_worker_argv_follows_shared_user_site_policy, the Piper inventory case, test_ambient_pythonpath_keeps_user_site_for_uvicorn, and test_manifest_pythonpath_keeps_user_site_for_gateway_module; the old implementation failed these cases. test_preflight_runs_after_seed_and_preserves_its_files locates the embedded source after -c, so an optional -s prefix cannot disable its replace guard.
  • test_a_python_backend_with_deps_launches_through_the_shim and test_the_deps_dir_is_prepended_to_a_stdio_server_pythonpath pin the launcher-owned dependency transports. test_dash_x_falls_back_to_the_pythonpath_transport pins the unshimmable launcher-injected PYTHONPATH case. test_bundled_interpreter_disables_user_site_for_python_backends pins automatic -s insertion for both ASGI and plain Python backends.
  • Required seven-file backend gate: 1,252 passed and 26 skipped.
  • scripts/local-gate.py was not run because its stale-base selector is not authoritative for this work order. The explicit backend gate and spawn/security audits stand in for it.
  • Black, subprocess encoding, isort, flake8, mypy, docs lint, diff checks, and both disclosure scans passed.

Manual verification

Used the disposable kirocrew-wt-11587 pod to capture the before/after process argv above and verify the gateway, gateway daemon, Dev Fleet backend, generated MCP stub, and existing -I -S sandbox launchers.

The pod's mcp-core and mcp-cron console-script processes do not display -s; they already run inside a virtual environment where Python disables the user site. The pod evidence is therefore for consistency of direct bundled-interpreter argv, not a suggestion that virtual-environment console scripts need another flag.

Related Issues

Fixes #11587

Pattern harvest

Rule candidate: lint
Pattern: Kiro Crew-owned bundled-interpreter children must use the shared user-site isolation argv builder.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@pepmach
pepmach requested review from a team and dwu96 September 18, 2026 17:30
@pepmach

pepmach commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Intent: Make every Kiro Crew-owned child launched by the bundled interpreter use one user-site isolation policy, so ambient ~/.local packages cannot shadow the bundle's pinned dependencies.
Not a goal: Change user-authored scripts, project tools, virtual-environment console scripts, or non-bundled installations that rely on user site.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 18, 2026
@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify on a pip install --user host that the force_isolation=True sites (Slack repair script, deps shim) still import kiro_crew under -s.

Suggestions

  • isolated_python_argv decides the bundle branch from sys.executable even when executable= names a different interpreter; checking the resolved child path for BUNDLED_BACKEND_DIST_DIRNAME would make a non-bundled parent launching the bundled interpreter isolate too, instead of relying on each such call site to pass -s explicitly (as agent.py's .cmd unwrap does today).

[DESIGN-REVIEWED] 6476ae4

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

One owned bundled-interpreter boundary is left raw: worktree_ops.py:2014 spawns the same dep_sync snapshot the Slack repair path now isolates.

Not justified as shipped

  • Item 2 — unfixed sibling: the spec added here claims "Every Kiro Crew-owned Python service, helper, and runtime dependency installer that can run under the bundled interpreter routes argv through isolated_python_argv", but grepping [sys.executable in src/kiro_crew leaves 1 in-contract sibling: apps/builtins/dev_fleet/worktree_ops.py:2014 launches [sys.executable, str(dep_sync_snapshot), repo, target_py] raw — the same dep_sync script family slack/gateway.py::_auto_apply_update routes with force_isolation=True in this diff. The other remaining raw spawns are already -I/-I -S (sandbox, validation, cron launcher) or user/project workloads the contract excludes.
    Clears when: worktree_ops.py:2014 is routed through the helper and added to _INTERNAL_PYTHON_SPAWN_SITES, or the boundary is named as out-of-contract with the reason the Slack twin is not.

What this change ships

Inventory (10 items) — 9 justified

Intent: stop user-site packages from shadowing the desktop bundle's shipped dependencies in Kiro Crew-owned child processes — a FIX (issue #11587).

  1. Bundled-install internal children (gateway restart, MCP servers/stubs, app backends, installers, overlay, Piper, pod fallback) now start with -s — justified
  2. All owned spawn boundaries routed through one shared argv builder (23 counted consumers) — unfixed sibling: worktree_ops.py:2014 left raw
  3. Launcher-injected import-path spawns (deps shim, host CLI pin, dep repair, harness) always isolated, even on user-site hosts — justified
  4. Manifest stdio entries naming an interpreter path now receive the -s prefix (executable preserved; in-place comment updated) — justified
  5. MCP overlay cache regenerates when the user-site policy changes (new fingerprint key, red-first test) — justified
  6. Managed-MCP first-party check now also refuses the -s -m kiro_crew fallback form — justified
  7. Piper worker's hand-rolled conditional -s deleted in favor of the helper — justified
  8. New AST inventory test makes any new internal spawn site a reviewed change — justified
  9. platform-compat spec row and section recording the rule — justified
  10. Non-bundled user-site parents keep their policy unchanged (no default flipped for pip/--user installs) — justified

[FIRST-PRINCIPLES-REVIEWED] 6476ae4

@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 6476ae4

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 18, 2026
@github-actions

github-actions Bot commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 6476ae4

Verdict parsed from the review's SHA-scoped output markers for commit 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 6476ae4fb23fdd38cad3ef883a8f3cfb1e732a7e: <one-sentence reason>

@pepmach
pepmach force-pushed the fix/bundled-interpreter-user-site-isolation branch from 23fc35d to dfffd74 Compare September 18, 2026 19:17
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 18, 2026
@pepmach

pepmach commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

CI note: Backend Tests (3.12, 2) and Backend Tests (Windows) (2) fail on test/test_ci_fleet_routing_expression_parity.py::test_every_ci_yml_resolver_consumer_reads_the_resolver_not_a_literal -- the resolver-consumer allowlist has been red on main since #11699 landed (the allowlist did not move with the new backend-test consumer); the heal is #11876. This PR does not touch that test; the pair clears with the heal. macOS Tests / Backend Tests (macOS) (1) fails on test/test_cloud_login_paths.py::TestPrivateLoginDir::test_guard_reads_mode_back_through_bsd_stat_with_bash, tracked in #11884 (since #11825); also unrelated to this diff.

@pepmach
pepmach force-pushed the fix/bundled-interpreter-user-site-isolation branch 2 times, most recently from 3a76838 to 7f37bdf Compare September 18, 2026 20:58
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 18, 2026
@pepmach
pepmach force-pushed the fix/bundled-interpreter-user-site-isolation branch from 7f37bdf to bbc92e6 Compare September 18, 2026 22:25
@pepmach

pepmach commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: MCP overlay fingerprints include the effective interpreter isolation prefix

  • span=74a9af46f464 fixed

Fixed in bbc92e64ef08aef0f8f31ee35bc551b2f44a8018: _rewrite_inputs_fingerprint records isolated_python_argv()[1:].
A bundle or user-site policy flip changes the fingerprint and forces an overlay rewrite before cached argv can be reused.
Red-first test_effective_user_site_policy_change_invalidates failed on the prior head and passes on this commit.

@pepmach

pepmach commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the test harness locates the embedded source relative to the Python -c marker

  • span=18ff8e3aa4b0 fixed

Fixed in bbc92e64ef08aef0f8f31ee35bc551b2f44a8018: the guard reads argv[argv.index("-c") + 1].
Optional interpreter prefixes can no longer make the replace check inspect the constant -c flag.

@pepmach

pepmach commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: changed comments describe the isolation prefix as policy-dependent

  • span=53782b78323f fixed

Fixed in bbc92e64ef08aef0f8f31ee35bc551b2f44a8018: the changed comments use optional or conditional -s wording.
The documentation now matches the non-bundled user-site-enabled path, which remains flagless.

@pepmach

pepmach commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: PiperRuntime uses the shared user-site policy instead of a duplicate conditional

  • span=fcb382c85440 fixed

Fixed in bbc92e64ef08aef0f8f31ee35bc551b2f44a8018: PiperRuntime._start routes its worker argv through isolated_python_argv and is part of the explicit inventory.
The system spec no longer claims coverage while leaving the resident Piper worker on a hand-written policy.

@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 18, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Sep 18, 2026
@pepmach
pepmach force-pushed the fix/bundled-interpreter-user-site-isolation branch from bbc92e6 to d13bac1 Compare September 19, 2026 00:01
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026
@pepmach
pepmach force-pushed the fix/bundled-interpreter-user-site-isolation branch from d13bac1 to 7cb49ee Compare September 19, 2026 00:44
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 19, 2026
@pepmach
pepmach force-pushed the fix/bundled-interpreter-user-site-isolation branch from 7cb49ee to 6476ae4 Compare September 19, 2026 01:15
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 19, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after verification: PR Readiness success on this head, no active change requests or unresolved threads, mergeable against current main.

@bolichen97
bolichen97 enabled auto-merge (squash) September 19, 2026 05:02
@bolichen97
bolichen97 merged commit 5ff0dce into main Sep 19, 2026
86 of 88 checks passed
@bolichen97
bolichen97 deleted the fix/bundled-interpreter-user-site-isolation branch September 19, 2026 05:02
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 19, 2026
amadsalmon pushed a commit to amadsalmon/KiroCrew that referenced this pull request Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bundled interpreter: -s is applied inconsistently across spawn sites, so user-site packages shadow the bundle

2 participants