Skip to content

docs: forbid unrequested sandbox widening in guidance and review rules - #12200

Merged
bolichen97 merged 1 commit into
mainfrom
fix/sandbox-delegated-overlap-wording
Sep 20, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/sandbox-delegated-overlap-wording

Conversation

@bolichen97

@bolichen97 bolichen97 commented Sep 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Two incidents this week share one missing rule: nothing in the repository says
that widening the sandbox needs asking for.

#11556 added a module-level assert over the full sealed-leaf union while
writing wording for two of five leaves. import kiro_crew.sandbox then raised
on main, so every backend test in the repository failed at collection —
including test_every_strict_leaf_of_either_shape_is_covered_by_the_delegated_guard,
written to pin that exact invariant, which could not report the gap because it
could not be collected either. An import-time self-check silences its own
witness. (Reported as #12182; the code is fixed on main by 2b7d378116. This
PR records the rule, and carried the same code fix until that landed.)

#12103 spent eleven review rounds on one span, each round naming one
more environment variable that "also reaches it" — PYTHONPATH, PYTHONHOME,
PYTHONUSERBASE, zip/egg loaders, .pth hooks, per-user site. Each round was a
real new mechanism, and that is the point: enumerating them does not terminate.

Why it matters

What the sandbox seals, masks or no-follow pins is an operator-facing
contract
about which of their own files their agent can still reach. A
widening nobody asked for does not surface as a failed review. It surfaces days
later as work that stopped, reported as a pattern instead of a cause, with the
legitimate operation gone and no record that anyone chose to lose it.

security-scope.md already owns one direction of this — "which legitimate
operations does this change newly refuse" — and explicitly hands the stricter
direction to the other lanes. Those lanes had no upper bound written down.

What changed (motivation → approach → change)

The scope is the operator's to widen: not an author's to add on the way past
something else, and not a reviewer's to request. Three placements, one per
reader, because each reads a different file:

  • AUTOSDE.yaml — a new blocking rule
    a-sandbox-widening-is-not-incidental-scope, scoped to sandbox.py,
    security.py, hooks.py. It flags an unrequested seal/mask/deny, a fence
    tightened past the threat the PR names, hardening added only to answer a
    reviewer's "spelling Y reaches it too", a widening justified solely as defense
    in depth with no reachable input named, and an invariant whose own inputs the
    diff leaves incomplete. It explicitly does NOT flag a widening the PR's intent
    names with its input and consequence written down, a completion of an existing
    invariant's data, or any narrowing.
  • AGENTS.md — the same rule as a security invariant, placed beside the
    existing "a regex spelling-chase is a review smell" entry it generalizes. The
    review lanes read AGENTS.md too, so this is also where the rule reaches the
    reviewer.
  • .github/review-prompts/gpt-repo-context.md — a bound on the lane. It was
    already told to "prefer a finding on the SUBJECT or the SANDBOX", with nothing
    saying it may not ask for a wider sandbox. Now it must cite a seal that fails
    on a path the PR already changes, or report nothing.

Backwards compatibility

Breaking for review, deliberately, and only for review: a new blocking
AutoSDE rule means a PR that widens the sandbox without saying why can now be
flagged where it previously passed. That is the intent, and it is what blocking
is for. The rule states its own allowed cases so a legitimate widening still
passes by naming its input and consequence.

No runtime effect: three text files, no Python touched, src/ unchanged.

Tests

N/A — no executable change. The files are consumed by the Claude and Codex PR
reviewers and by agents reading AGENTS.md; there is no unit-testable behaviour.

Verified instead: AUTOSDE.yaml parses (20 rules, no duplicate id, the new
rule present with blocking: true and its three file patterns); 1,172 tests pass
across the six suites that read AUTOSDE.yaml, the review workflows and the
prepare-pr profiles (test_ci_surface_tests.py, test_ai_review_workflows.py,
test_fix_loop_discipline.py, test_agent.py, test_prepare_pr_profiles.py,
test_skill_home_paths.py); docs-lint, black and subprocess-encoding gates green.

Manual verification

Read each of the three files in place to confirm the rule lands in the section a
reader actually reaches: the AutoSDE rule after a-ratchet-may-only-tighten
(whose "may only tighten" direction it bounds), the AGENTS.md entry inside
"Security invariants (do NOT weaken)", and the lane bound as the bullet
immediately after the one that sends findings toward the sandbox.

Related Issues

Context: #12182 (the import failure this generalizes), #12103 (the eleven-round
span), #11556 (the incomplete invariant).

@bolichen97
bolichen97 requested review from a team and chenmingwei23 September 20, 2026 05:20
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 20, 2026
What the OS sandbox seals, masks or no-follow pins is an operator-facing
contract about which of their own files their agent can still reach. A
widening nobody asked for does not surface as a failed review; it surfaces
days later as work that stopped, reported as a pattern instead of a cause,
with no record that anyone chose to lose the operation. So the scope is the
operator's to widen: not an author's to add on the way past something else,
and not a reviewer's to request.

Two incidents this generalizes, both on record:

#11556 added a module-level assert over the full sealed-leaf union while
writing wording for two of five leaves, so `import kiro_crew.sandbox`
raised on main and every backend test failed at COLLECTION -- including
`test_every_strict_leaf_of_either_shape_is_covered_by_the_delegated_guard`,
written to pin that exact invariant, which could not report the gap because
it could not be collected. An import-time self-check silences its own
witness. Fixed on main by 2b7d378; this records the rule.

#12103 spent eleven review rounds on one span, each round naming one more
environment variable that "also reaches it". A review comment is not the
operator's authorization, and that direction converges on nothing.

Three placements, one per reader:

AUTOSDE.yaml gets a blocking rule so the Claude and Codex reviewers flag an
unrequested seal, a fence tightened past the named threat, hardening added
only to answer a "spelling Y reaches it too" comment, a widening justified
solely as defense in depth with no reachable input, and an invariant whose
own inputs the diff leaves incomplete.

AGENTS.md carries the same rule as a security invariant, beside the
"regex spelling-chase is a review smell" entry it generalizes.

gpt-repo-context.md bounds the lane: it was already told to prefer a
finding on the SANDBOX, with nothing saying it may not ask for a wider one.
@bolichen97 bolichen97 changed the title fix(sandbox): give every sealed leaf its delegated-overlap wording docs: forbid unrequested sandbox widening in guidance and review rules Sep 20, 2026
@bolichen97
bolichen97 force-pushed the fix/sandbox-delegated-overlap-wording branch from dbbffc0 to 1cabbbd Compare September 20, 2026 05:28
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 1cabbbd18fe6321578e66bd9d14be65eeac598b8 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- AUTOSDE.yaml:755 -- "src/kiro_crew/security.py" no longer exists, so changes under security/ bypass the new rule -> Fix: use "src/kiro_crew/security/**/*.py". (origin: validation)
[GPT-REVIEWED] 1cabbbd

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 1cabbbd18fe6321578e66bd9d14be65eeac598b8: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 1cabbbd18fe6321578e66bd9d14be65eeac598b8 — this comment is updated in place on each push.

Review details

The YAML is well-formed — indentation matches the sibling prefer-subtracting-a-duplicate-mechanism rule exactly (4-space keys, 6-space block-scalar body), the rule is purely additive, and adding a blocking: true rule is explicitly not a violation. The other two changes are prose additions to policy docs. There is no executable code path in this PR, so no concrete input → call path → observable wrong outcome can be grounded.

No findings.

[OPUS-REVIEWED] 1cabbbd

Verdict parsed from the review's SHA-scoped output markers for commit 1cabbbd18fe6321578e66bd9d14be65eeac598b8.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 1cabbbd18fe6321578e66bd9d14be65eeac598b8: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 1cabbbd18fe6321578e66bd9d14be65eeac598b8 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the lane's "or report nothing" bound still lets a reviewer surface a genuinely unfenced new credential leaf via the disposition-pin rule, not a fence request.

[DESIGN-REVIEWED] 1cabbbd

@bolichen97
bolichen97 merged commit b5b9c62 into main Sep 20, 2026
86 of 90 checks passed
@bolichen97
bolichen97 deleted the fix/sandbox-delegated-overlap-wording branch September 20, 2026 05:35
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant