Skip to content

test(sandbox): pin how each protected name is held - #13935

Merged
iamwhatever merged 2 commits into
mainfrom
fix/hold-protected-names-13840
Sep 25, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
fix/hold-protected-names-13840

Conversation

@chenmingwei23

@chenmingwei23 chenmingwei23 commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

What is the problem?

A protected name's mask is placed once, at spawn, and nothing holds it afterwards.

The launcher masks each protected path by binding a stand-in over it. That mount
covers the object it was aimed at. It does not hold the NAME. The name's parent
directory stays live, so a host-side atomic replace of that name puts a fresh,
same-uid object there while the mask hangs off whatever was present at spawn.

The triggering writer is the operator's own tooling, not an attacker. A Dev Fleet
cutover republishes the live-target pointer by atomic replace. A config save
republishes the governance ceilings the same way, denied_commands.json and
computer_use.json among them. Either one landing while a sandboxed session is
alive is routine administration.

Nothing in the tree measured how large that population is, or distinguished the
two ways a name can be held. This change measures both and pins them.

Why this issue matters to the user

The names in question are the ones the gateway reads back as authoritative. A
writable stand-in at one of them is not an information leak; it is control. The
live-target pointer selects the checkout the gateway executes at its next start.
denied_commands.json carries the deny-gate opt-out. computer_use.json is the
computer-use primary enable. Each of those files says so in its own docstring.

Without a recorded population, a new protected name can be added that only a leaf
mask holds, and nothing objects. Without the seal-versus-mask distinction written
down, a future reader can count a read-only seal as a durable hold and report the
window closed when it is open.

How our fix solves it

This PR adds no product change. It is the measurement and the ratchet. The
mechanism fix is tracked separately, and this body records why.

The chain from symptom to root cause:

  1. The symptom is a protected name that becomes writable mid-session.
  2. The immediate cause is that the mask covers an object, not a name.
  3. The root cause is that a hold on a name requires a mount on an ANCESTOR
    directory. A lookup that crosses such a mount terminates inside the stand-in
    the launcher created, so the real directory's entries are never consulted and
    a host-side republish cannot appear in that namespace at all.
  4. So the property is structural, not sampled, and it is decidable from source:
    for each protected name, does a strict ancestor carry a stand-in mask?

The pin computes exactly that, reading the names back out of the generated
launcher script for all three tiers rather than recomputing them from the
constants, so it follows what the child is really handed.

It also pins the distinction that makes the answer meaningful. A read-only seal
binds a path over ITSELF. Same directory entries, same rename window; write
access withheld and nothing else. It holds nothing beneath it. Counting one as a
hold is asserted to be an error, because that single reclassification would turn
a true zero into a false one.

Measured baseline, from source, per tier:

tier protected names durably held held only by their own leaf name
standard 235 0 235
cc 242 0 242
strict 243 0 243

Nothing is durably held today. Per crew data-home root there are 49 hidden
leaves and 24 read-only leaves against 7 declared visible, and 69 of them sit
directly in the root, whose parent is writable in-sandbox. The launcher emits one
absolute spelling per data-home prefix and per relocation, which is why a single
added leaf moves each tier's count by three.

The leaf-only counts are recorded as a ceiling, not a target. Adding a protected
name that only a leaf mask holds now has to be a visible edit to that number.

What this PR deliberately does NOT do: it does not add a second spawn-time check.
Any finite number of instants leaves gaps between them, so a re-check cannot
become a hold no matter how many are added.

Review round two

Three review findings, all taken, all in the test file.

The load-bearing one: the durable-hold assertion presented itself as a set
equality and computed only one direction. It built the set of satisfied records by
ITERATING the recorded names, which makes that set a subset of them by
construction, so subtracting it could only ever surface a recorded hold that went
missing. A hold the launcher genuinely gives that nothing records passed silently,
and because it was recorded nowhere its own later loss could not redden either, so
the ratchet went quiet in exactly the direction it exists for. It is now two named
tests computing the two directions from different sides, and the second direction
is computed from held rather than from the constant. Mutation M4 below is the
proof; the first round never ran that direction.

The partition assertions restated _split's own control flow over a deduplicated
input, so they held for every possible state of sandbox.py, and one of them was
repeated verbatim in a second class. Decorative either way. They are replaced by
two assertions that can fail: one cross-checks the two independent derivations of
the masked set, the constants against the generated script, so a leaf the builder
stops emitting reddens; the other requires every emitted path to be absolute and
normalized, because the child mounts by that exact string and a relative or
dot-dot entry masks somewhere else while leaving the intended path open.

The module is now skipped on Windows. The skip is the legitimate kind, not a
dodge: the launcher mask mechanism does not exist on Windows, where the wrap is
skipped entirely, so the invariant asserted here has no subject on that platform
-- as opposed to a pin whose subject IS Windows, where skipping would assert
nothing about the thing under test. Running it there asserted nothing and crashed
doing it, because the launcher builder reads os.getuid. That crash was the
whole of the Windows shard's red: all 18 failures on Backend Tests (Windows) (8)
were this file raising AttributeError: module 'os' has no attribute 'getuid',
confirmed from the check-run annotations rather than assumed.

What tests we did

test/test_sandbox_protected_name_holds.py, 25 tests, all passing. Static: no
namespace, no mount, no privilege. POSIX only, for the reason given above.

Run scoped to the one file this change owns, with an explicit -n0.

Four mutations, the full matrix in both directions. Each proven to have applied by
comparing sha256 before and after, each with a passing control on both sides, and
both files verified back to pristine bytes afterwards. A mutation that scores
CAUGHT without a hash change is not evidence.

mutation what it fakes caught by verdict
M1 add a masked leaf carrying no enclosing hold a new protected name shipped with only leaf-name holding test_leaf_only_count_has_not_grown RED, 3 failed
M2 count a read-only seal as a durable hold a bind-over-self seal reported as if it held the names beneath it, turning the true zero into a false one test_each_hold_names_a_real_stand_in_ancestor, test_no_hold_is_unrecorded RED, 6 failed
M3 record a hold the launcher does not give a name listed as durably held with no enclosing stand-in mask behind it, a fix claimed rather than made test_no_recorded_hold_went_missing RED, 3 failed
M4 grant an enclosing hold that nothing records the opposite direction, and the one the first round never tested: a real hold that the ratchet is not watching test_no_hold_is_unrecorded RED, 3 failed

M3 and M4 are each caught by exactly one of the two opposite assertions and by
nothing else, which is what makes them a proof of both directions rather than of
one. M4 is built to isolate that: masking the parent of an already-masked leaf
moves one name from the leaf-only class into the held class and adds one new
leaf-only name, so the debt ratchet nets to zero and only the unrecorded-hold
assertion can fire.

Control on pristine bytes: 25 passed, sha256(sandbox.py)=212701df99f147d8,
sha256(pin)=333cf001f4d2a092.

Fast gates run locally: comment-history gate passed on the committed file,
flake8, isort and black clean, body and file pure ASCII.

Limitation, stated rather than papered over: the kernel-level premise that a
host-side rename moves the dentry out from under a leaf mount is inherited from
the issue and is NOT verified here. unshare(CLONE_NEWUSER) returns EPERM inside
an agent session, because nested sandboxing is denied by design, so the namespace
experiment could not be run. The pin does not depend on that premise: that a
lookup crossing a bind mount terminates in the mount's root is ordinary path
resolution, and the assertions rest only on it.

Pattern harvest

Rule candidate: a protection verified once at an instant, then relied upon as if
it held across an interval. The verifying code is correct about the moment it
looks; the defect is in the lifetime it is read as covering. Adding another check
cannot repair it, because any finite set of instants leaves gaps. The repair is
always to hold the thing rather than re-read it, and the hold has to name the
same unit the threat acts on -- here a NAME, not the object, which is why pinning
the object closes a different window and leaves this one open.

Second rule candidate, earned by this PR's own review round: an assertion
described as an equality that computes one containment. The tell is a set built by
iterating the very constant it is about to be compared against, which makes the
comparison a subset test wearing an equality's name. Both directions need their
own mutation, because the missing one is invisible to reading and its absence is
silent rather than loud.

The generalizable test shape: enumerate the protected population from source
rather than listing sites, classify each member by WHICH mechanism holds it, and
pin the classification in both directions. A mutation reverting any single member
to the sampled form then reddens, a mechanism mistaken for a hold reddens
separately, and a hold nobody recorded reddens separately again.

Any other suggestions on the work

Two follow-ups, both out of scope here and neither folded in.

First, the mechanism fix for the proven-reachable cutover path. Holding a name
requires its payload to live inside a masked directory, so the live-target
pointer has to move under one. pointer_path() is a single choke point, but
test_sandbox_dev_fleet_live_target.py asserts the pointer is a direct child of
the crew home in roughly fifteen places, and existing installs need a one-shot
migration on the gateway's own exec path. Two smaller shapes were considered and
rejected: rewriting the pointer in place keeps the name but loses atomicity on
the boot decision, and interposing a directory without moving the payload is not
possible.

Second, the read-only ceilings are the same class with a sharper consequence, and
they are republished by the ordinary config save path. They deserve their own
issue rather than a third layout change inside one review.

Relationship to #13715, checked against its head 896ac78827: independent in both
merge orders. That PR pins the OBJECT each mask covers, by opening the target once
and mounting over /proc/self/fd/N. This pin is about the NAME, and the two are
different units. Concretely, at that head it adds only REQUIRED_MASK_TARGETS to
the launcher payload and renames none of SENSITIVE_DIRS, SENSITIVE_FILES or
READONLY_DIRS, which are what this pin parses; it adds and removes no protected
leaf; and it leaves the payload assembly untouched, so the counts above are
unaffected whichever lands first. The two converge on one point without
conflicting: it adds a comment recording that _build_launcher_script calls
POSIX-only os.getuid, which is the same constraint behind this file's Windows
skip.

Refs #13840

Enumerate the protected names from source for every sandbox tier and pin
which of the two holds each one has: a stand-in mask, which covers every
name beneath it for the namespace lifetime, or its own leaf name, which
covers one object at one instant.

A read-only seal binds a path over itself, so it leaves the directory
entries and the rename window unchanged and holds nothing beneath it.
Counting one as a hold is pinned as an error.

Static: no namespace, no mount, no privilege.

Refs #13840
@chenmingwei23
chenmingwei23 requested a review from a team September 25, 2026 18:32
@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Sep 25, 2026
@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 071155de84db145c60b4065fe83c9c5c58e7fdfd and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- test/test_sandbox_protected_name_holds.py:144 -- any(path.endswith(...)) stays green when one canonical, legacy, or relocated path disappears -> Fix: compare every expected absolute path identity per tier.
[GPT-REVIEWED] 071155d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 071155de84db145c60b4065fe83c9c5c58e7fdfd: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 071155de84db145c60b4065fe83c9c5c58e7fdfd — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 071155d

Verdict parsed from the review's SHA-scoped output markers for commit 071155de84db145c60b4065fe83c9c5c58e7fdfd.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 071155de84db145c60b4065fe83c9c5c58e7fdfd: <one-sentence reason>

The durable-hold assertion built its satisfied-record set by iterating the
recorded names, so that set is a subset of them by construction and the
subtraction could only surface a record whose hold went missing. A hold the
launcher gives that nothing records passed silently, and being recorded
nowhere its own later loss could not redden either. Two named tests now
compute the two directions from different sides.

Replace the partition assertions, which restated the classifier's own
control flow and held for every state of the module under test, with a
cross-check of the constants against the generated script and a requirement
that every emitted path is absolute and normalized.

Skip the module on Windows: the launcher mask mechanism does not exist
there, so the invariant has no subject, and the builder reads os.getuid.

Refs #13840
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 071155de84db145c60b4065fe83c9c5c58e7fdfd — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

Two undeclared riders — the hosted-black wrapper switch and the RFC status flip — duplicate already-merged base commits; rebased, this diff is one test file.

Not justified as shipped

  • Item 2 — undeclared, duplicate of 4926e50b3 ("ci(lint): run hosted black through bounded_black (ci(lint): run hosted black through bounded_black #13698)", the base tip): the description says "Run scoped to the one file this change owns," yet the diff rewrites scripts/check_black_formatting.py, ci.yml, test_black_fleet_budget.py and docs/ci/ci-and-reviews.md — all byte-identical to what ci(lint): run hosted black through bounded_black #13698 already merged (checkout verified identical).
    Subtraction: rebase onto current main; these hunks disappear as already-landed content.
    Clears when: the diff against the current base contains only test/test_sandbox_protected_name_holds.py.
  • Item 3 — undeclared, duplicate of 33bb70691 ("docs(rfc): mark composable-layout in-progress (docs(rfc): mark composable-layout in-progress (PR 1 of 3 in flight) #13673)"): the same draft→in-progress flip and README row, never mentioned by the description, byte-identical to the merged change.
    Subtraction: same rebase; drop the two RFC hunks.
    Clears when: same rebase, riders gone from the diff.

Neither rider is product-shape (a CI gate's launcher path and a docs flip already decided by merged maintainer PRs), and both merge as no-ops — hence CONCERNS, not BLOCK. The declared item itself is sound: the leaf-mask-vs-name window is an OS mount-semantics fact, the counts are computed from source, and no sibling test classifies holds (test_sandbox_governance_mask.py pins disposition coverage, not durability).

What this change ships

Inventory (3 items) — 1 justified

Intent: pin, from source, whether each sandbox-protected name is held for the namespace lifetime or only at spawn, and ratchet the leaf-only population — an ADDITION (measurement and tests, no product change).

  1. New 25-test POSIX-only suite recording that zero protected names are durably held (leaf-only ceiling 235/242/243 per tier) — justified
  2. Hosted CI's black gate now always runs the recycling wrapper, native branch removed (script, workflow, budget test, docs) — undeclared, duplicate of base commit 4926e50b3 (ci(lint): run hosted black through bounded_black #13698)
  3. RFC composable-layout-mechanism flipped draft→in-progress with its README row — undeclared, duplicate of base commit 33bb70691 (docs(rfc): mark composable-layout in-progress (PR 1 of 3 in flight) #13673)

[FIRST-PRINCIPLES-REVIEWED] 071155d

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 071155de84db145c60b4065fe83c9c5c58e7fdfd — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify the leaf-only count ratchet is green on every CI shard — EXPECTED counts spellings, which depend on the conftest-pinned environment, not just the leaf population.

Suggestions

  • Deduplicate emitted spellings to leaf identity before counting in test_leaf_only_count_has_not_grown: the ratchet still reads what the child is handed, but the number stops moving by 3 per leaf (or by ~70 if relocation emission or conftest home pinning ever changes), and its failure message — which currently attributes any growth to a new protected name — stays truthful.

[DESIGN-REVIEWED] 071155d

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 25, 2026
@chenmingwei23

Copy link
Copy Markdown
Contributor Author

Disposition: First Principles CONCERNS on 071155de84 -- items 2 and 3 are not in this branch

Both riders are a stale-base artifact of how the verdict's diff was taken, not content this
PR ships. Measured:

my head     = 071155de84db145c60b4065fe83c9c5c58e7fdfd
merge base  = 07a05a2b1c13b5f3b55a7e428b31178e450a09a4
origin/main = 4926e50b3c98e9b27e1f537a0cf52624e23ad24f

git log --oneline <merge-base>..HEAD
  071155de84 test(sandbox): compute both directions of the hold record
  d16ed50d13 test(sandbox): pin how each protected name is held

git diff --stat <merge-base>...HEAD
  test/test_sandbox_protected_name_holds.py | 288 ++++++++++
  1 file changed, 288 insertions(+)

4926e50b3  in-my-head=no   in-origin/main=yes   ci(lint): run hosted black through bounded_black (#13698)
33bb70691  in-my-head=no   in-origin/main=yes   docs(rfc): mark composable-layout in-progress (#13673)

Neither named commit is an ancestor of this head, and this branch touches none of
scripts/check_black_formatting.py, ci.yml, test_black_fleet_budget.py,
docs/ci/ci-and-reviews.md, or the RFC files. GitHub's own file list for this PR is the same
one file.

The mechanism: origin/main advanced to 4926e50b3 after this branch's base was cut at
07a05a2b1c. A diff taken against the main TIP rather than the merge base renders every
commit that landed on main in between as if it were on this PR's side, inverted. That is
exactly why the hunks read byte-identical to the merged change -- they ARE the merged change,
seen from the wrong base. It also explains the review's own subtraction ("rebase onto current
main; these hunks disappear as already-landed content"): a rebase moves the base forward to
include them, which is a property of the base, not of this diff.

The stated clearing condition, "the diff against the current base contains only
test/test_sandbox_protected_name_holds.py", already holds against the merge base, as the
diffstat above shows.

Not rebasing to clear it. A rebase would re-roll the review lanes, and this head already
carries a recorded GPT 5.6 pass; buying a cosmetic change to an advisory verdict at the cost
of re-rolling a pass on a one-new-file diff with no conflict surface is the wrong trade. The
declared item is untouched by any of this and the review agrees it is sound.

Worth flagging beyond this PR: the same artifact will appear on any PR whose base is behind
main, with the phantom riders being whatever landed in between, so a verdict of this shape is
likely to recur across the fleet rather than being specific here.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) September 25, 2026 23:11
@iamwhatever
iamwhatever merged commit 2a4457f into main Sep 25, 2026
97 of 98 checks passed
@iamwhatever
iamwhatever deleted the fix/hold-protected-names-13840 branch September 25, 2026 23:11
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 25, 2026
bolichen97 pushed a commit that referenced this pull request Sep 26, 2026
…het (#13685 x #13935)

#13935 pinned how each protected name is held and recorded the leaf-only
population per tier (235 / 242 / 243). #13685 merged after it without a
rebase and added credential_redaction.json -- the owner's credential-redaction
switch -- to the three keystone leaf lists file_delivery_consent.json already
sits in, so main has read 238 / 245 / 246 since, and every PR's Backend Tests
shard 8 is red. This is the deliberate, visible edit the ratchet asks for: the
number moves by exactly the three names that shipped, with the reason beside
it. The enclosing-directory hold the ratchet wants remains the shared debt of
every keystone leaf on that floor.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants