Repository navigation
feat(aws-control): curate a crew into a portable bundle - #9213
Conversation
First Principles Review (Fable 5) — 🟡 CONCERNSPremise-level review of All verification is done. Emitting the review. First-Principles-Verdict: CONCERNS A zero-caller builder citing an out-of-repo contract, plus two riders — a repo-wide Windows gate change and vendor-token constants — whose consumers this PR itself disables or contradicts. Not justified as shipped
What this change shipsInventory (10 items) — 6 justifiedIntent: package a local crew into a digest-verified bundle so it can be deployed off its owner's machine — ADDITION.
Watch
Subtractions
[FIRST-PRINCIPLES-REVIEWED] 641cd15 |
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of All the evidence I need is gathered. Emitting the review. Design-Verdict: CONCERNS Sound, staged builder — but a core security-gate change rides along undocumented, and the format's normative contract lives outside this repo. Watch
Suggestions
[DESIGN-REVIEWED] 641cd15 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
38dacc8 to
f0e65a6
Compare
f0e65a6 to
5b71879
Compare
Opus 4.8 Review — ✅ no blocking findingsReviewed Review detailsNo findings. The load-bearing candidate (the digest-order mismatch, candidate 1) depends entirely on The remaining candidates were self-flagged low confidence and do not clear the bar: candidate 2 is a docstring gap with no reachable non-regular leaf established; candidate 3 is operator-self-inflicted OOM on [OPUS-REVIEWED] 641cd15 Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
/ai-review override gpt 5b71879: Provider REFUSED (not a finding); gate says re-runs do not clear this class. Diff is one pure-Python package + 17 tests, no AWS/Docker/network, main() uncalled by design, security guards each pinned by a mutation test, 137 pass, Opus did not block. |
94c7062 to
e2c2515
Compare
e2c2515 to
6fba123
Compare
Rebuttal: concurrent staging claim,
|
|
@chenmingwei23 From the open-PR relationship audit (audited at 0636f23). This PR overlaps #9223 in exactly two files, and the two are complementary rather than competing, so both should land. Where they overlap. Both PRs add What differs. This PR is the bundle producer ( Suggestion. Pick one owner for Separately, #8470 duplicates all 17 files of this PR's packaging slice, 10 of them byte-identical. It should rebase and take this PR's Posted from the 2026-09-08 open-PR relationship audit (read-only, one auditor per PR); reply here if any of this is wrong. |
GPT 5.6 Review —
|
UX Review (Fable 5) — ⏭️ skippedRevision |
Turns a local crew into a portable, reviewed bundle: agent.json, mcp.json, skills/ and a manifest carrying the content digest. Deny-by-default -- nothing ships that a signed curation plan did not select, and every string is scanned before it becomes bytes. Addresses the three blocking items from tech-lead review. **The builder aborted every Windows build.** ``Path.write_text(text, encoding="utf-8")`` leaves ``newline`` at ``None``, which translates "\n" to os.linesep on write. The content pin compares ``_tree_hash`` (SOURCE bytes) against ``_staged_tree_hash`` (SHIPPED bytes), so an ordinary LF-authored skill hashed differently once staged and the build refused with "changed while the bundle was being written". ``bundle_digest`` runs over those same staged bytes, so the digest was platform-dependent too. Every ``write_text`` in the module now pins ``newline=""``, including the two whose bytes are not hashed -- a rule with exceptions is one nobody can apply from the call site. The suite could not catch this because it wrote its fixtures through the same call, so both sides of the comparison moved together; the new tests write source bytes with ``write_bytes``, and an AST tripwire holds the rule itself so the next such call is covered rather than only today's four. **skill_count undercounted nested ids.** A skill id is ``relative_to(skills_root).as_posix()`` and may nest, so ``aws/ec2`` and ``aws/s3`` are two skills under one top-level ``aws`` directory. Counting directories reported 1 for that pair, in the printed summary and in SMC_BUNDLE_JSON alike. It now counts the ids the plan selected, which is the population ``_copy_skill`` was driven from. **The prompt injection is deferred, not documented.** ``_inject_fingerprint_challenge`` prepended a ``[deployment verification]`` block to every deployed prompt, and its only consumer is the deploy gate, which is not in this change. The whole fingerprint path goes with that gate: the challenge, the injection, and the reported value, whose purpose the content digest already serves here. What ships now is the prompt the operator wrote, so a reviewer reading agent.json sees what will run. A source-text test keeps the block from returning uncalled, and ``test_fingerprint.py`` travels with the gate. Verified end to end on a real crew: three skills including two nested, one MCP server carrying a live-looking token. The token appears in no file in the bundle, the prompt is byte-identical to the source, skill_count is 3, the skill bytes are unchanged with no CRLF, and the manifest digest recomputes from the artifact.
The bundle builder inlines an agent's persona when the spec names it as file://<path>, so a curated crew ships one self-contained agent.json. Four shipped pptx_maker specs need this. The read is routed to hooks.safe_read_file_bytes_nolink, which owns the sensitive-path verdict, the fstat on the opened descriptor, the st_nlink refusal and containment against within_root. Redirects in the chain are judged before the path is handed over, because resolve() collapses links. Refuses a NUL in the reference: the target comes from the crew's spec, and a NUL-bearing string reaches a syscall as a bare ValueError. Checked on the string, since Path accepts it and defers the error past every point that could still name the reference. Removes a local opener stack that duplicated hooks and had no production caller, with the tests that pinned it. Scopes the byte ceiling to the prompt read, so an oversized agent spec or plan is not refused by a limit named for prompts. Restores nine POSIX-only markers dropped in a merge.
What this is
The packaging half of deploying a crew somewhere other than its owner's machine:
turn a local crew into a four-entry bundle (
agent.json,mcp.json,manifest.json,skills/) whose manifest carries a digest over every file itdescribes.
Pure Python. No cloud dependency, no Docker, nothing to configure. This is the
first of three pieces; the container runtime and the Fargate backend follow
separately.
There is no in-repo design document for the bundle format yet, and no in-repo
caller of
python -m packaging.build. The RFC on the Fargate backend covers thecompute side and does not mention bundling, so it is not the provenance for this
change and is not cited as such. What the format has to satisfy is written down in
this module's own docstrings and pinned by its tests; a design document and the
driver that consumes the bundle come with the piece that needs them.
Why it is 39 files
34 of them are tests: 31 new files under the packaging tree and three existing
repo-level suites this change touches. 363 test functions, run against a real
temporary filesystem rather than a dry-run stand-in. Most of them cover refusals,
because this module's job includes three recursive deletes and every one of them is
a way to destroy something the operator owns.
--outand<out>.previous, shared by one function<out>.stagingrmtreethenrenameis two operations, and a failure between them lost the old bundle with nothing to restoreis_file()is False for an empty directory, a FIFO, a socket and a link to a directoryNo caller lands here
build.pyexposesmain()and nothing in this diff invokes it. That isdeliberate: the driver that calls it belongs with the Fargate backend, and landing
the builder on its own keeps that review about the cloud path instead of about
bundle semantics. Reviewing this one needs no AWS account and no image build.
Filesystem access layer
build.pywalks, reads, writes, copies, renames and deletes against operator- andauthor-supplied paths on POSIX and Windows. Rather than harden each operation as a
reviewer reaches it, every operation routes through one access layer with four
properties.
No-follow reads on both platforms.
_walk_no_reparse(os.scandir, neverdescends a reparse point) for walks,
_read_text_nofollow/_read_text_openatforreads. On POSIX
O_NOFOLLOWrefuses a reparse final component at the open; whereO_NOFOLLOWis 0, the reader lstat-refuses a reparse point before opening, so ajunction to a UNC share is never followed into an SMB or NTLM probe.
Mutation through a descriptor, in a run-private location. The
<out>.previousrecursive delete goes through
_purge_via_private_aside: rename the tree into adirectory this build alone created, verify ownership on the moved entry, which is the
exact inode the rename captured and cannot now be swapped, and delete only if it
passes. A tree
swapped in before the rename is renamed back untouched. The inode verified is the
inode deleted, so the check-to-rename window is closed rather than narrowed.
Nothing is created or destroyed by name alone. Every file this module creates is
claimed with
O_CREAT | O_EXCL, so the outcome distinguishes "this call created it"from "it was already there" without an
is_file()check that a racer can slipbetween.
write_planreturns which of the two happened, because re-runningplanonan already-planned crew is a normal thing to do while truncating a concurrent
writer's plan is not. Reads of anything this module did not just create go through
hooks.safe_read_file_bytes_nolink, the one authority that owns the sensitive-pathverdict, the descriptor
fstatand the hard-link refusal, across 21 call sites,composed rather than reimplemented.
Explicit inclusion policy, failing closed. A file selected for a bundle that
cannot be read as scannable UTF-8 refuses the build and names the file. A directory
that exists but cannot be enumerated refuses rather than reading as empty. An
existing report or signed plan that cannot be read refuses rather than being treated
as absent, which would delete the report on rollback or overwrite the plan with stale
bytes. Absent, unreadable and unscannable never read as "not selected". The
standalone sensitive-path floor also catches a credential file by name (
.env,*.pem) and not only a credential directory, so--allowon a credential leaf failsclosed when the shared validator is unavailable.
Checks
59 check lanes green at
8cd7599ee, nothing failing and nothing running. Thatincludes all nine test lanes (four Linux shards, four Windows shards and the
namespace sandbox), plus both AI review lanes, the coverage gates and the
cross-platform portability, inclusive-language and first-principles gates.
packages = find:discovers the new package, so nothing insetup.cfgorMANIFEST.inneeds to change: the tree is Python only, and the non-Python payloadthat does need a
MANIFEST.inentry (templates, shell, Dockerfiles) arrives with thepieces that own those files.
Scope note
Two tests that pinned this module's
O_NOFOLLOWopener against the containersidecar's deliberately duplicated copy are held back for the change that brings that
copy. The property is "the two agree", which cannot be asserted while only one
exists, and asserting it here with a skip would be a test that passes for the wrong
reason.
The builder is POSIX-only for now:
_refuse_without_nofollow_primitive()refuses atthree entry points rather than running without the primitive its guarantees rest on.
That hold is tracked in #9496.