Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
"displayName": "Agent Stack",
"source": "./plugins/agent-stack",
"description": "Four skills: agent-orchestrator — tool-calling loops, pipelines with checkpoints, provider routing with fallback, memory architecture, plus the wallet side of reselling LLM access; agent-evals — run/trace/thread evals, LLM judges, and fixtures grown from production; agent-interop — MCP servers and clients, A2A agent cards, the MCP Registry, and gateways; agent-harness — system prompts, tool shaping, workflow-vs-agent, and auditing an agent system.",
"version": "0.27.0",
"version": "0.27.1",
"author": {
"name": "ssheleg",
"url": "https://x.com/sshlg93"
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/validate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -695,6 +695,27 @@ jobs:
echo "OK: every temp tree goes through the residue ledger"
fi

- name: Negative self-test (a shipped script's temp tree must not escape the check)
run: |
set -eu
rm -rf /tmp/residue-ship-copy && cp -R . /tmp/residue-ship-copy
python3 test/plant_guard.py snap /tmp/residue-ship-copy
# AG-06a and B-158 together: a maker outside `test/`, called through an alias. The
# check used to read `test/*.py` only and `tempfile.` as the only receiver, so this
# plant passed it twice over.
python3 - <<'PY'
import pathlib
p = pathlib.Path("/tmp/residue-ship-copy/plugins/agent-stack/skills/agent-harness/scripts/zz_plant.py")
p.write_text("import tempfile as tf\nd = tf." + "mkdtemp" + "()\n", encoding="utf-8")
PY
python3 test/plant_guard.py verify /tmp/residue-ship-copy "Negative self-test (a shipped script's temp tree must not escape the check)"
if out=$(python3 /tmp/residue-ship-copy/test/validate.py 2>&1); then
echo "ERROR: validator accepted an aliased temp-dir call in a shipped script"
exit 1
fi
echo "$out" | grep -F "zz_plant.py: creates a temp tree outside the residue ledger — 'tf.mkdtemp(' at line 2"
echo "OK: a shipped script's temp tree is refused, by file and line"

- name: Negative self-test (a description must stay inside the working limit)
run: |
set -eu
Expand Down
54 changes: 54 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,57 @@
## 0.27.1 — the temp-tree check reads the whole repository, and the old leak can be reaped by content

Closes board rows B-158, AG-06a and AG-06b (sshlg-skills plan R28).

- **Any receiver, and a renamed from-import (B-158).** `check_temp_trees_go_through_the_residue_ledger`
matched `tempfile.` as the only receiver, so `import tempfile as tf; tf.mkdtemp()` and
`from tempfile import mkdtemp as m` passed. The pattern is now make-skill 0.31.0's: a direct
call through one receiver or none, and a one-line renaming from-import, are refused; two
negative controls (alias, renamed import) were watched failing on the old pattern. Only
whole-line `#` comments and `"""` blocks are skipped; chained receivers, function references,
`getattr` and parenthesised multi-line imports are not seen. Line numbers are now the file's
own, not the stripped source's.
- **Recursive, and beyond `test/` (AG-06a).** The check read `test/*.py`; it now reads every
`*.py` the repository ships or runs (`test/` recursively, `plugins/`, `bin/`), skipping only
`.git`, `node_modules`, `__pycache__` and `fixtures`. Its first run found a real bypass,
`test/audit_regressions/instruction_budget.py:21` (`tempfile.TemporaryDirectory(`), now a
`residue.workspace()` (the audit run reports `14 temp tree(s) created, 14 removed`).
Exemptions are exact paths with a reason and a pinned count of makers: `test/residue.py`
(1, the ledger itself) and `agent-harness/scripts/audit_agent.py` (2, a shipped tool's
context-managed self-test that cannot import the suite's ledger). A third maker in an
exempt file is refused, and an exemption for a file that no longer exists fails. A shipped
script is told its own remedy. New CI negative self-test: an aliased maker in a plugin
script, refused by file and line.
- **`python3 test/residue.py --reap-own` (AG-06b).** The trees the old `plant_guard_test.py`
leaked are identifiable by content, so they can be reaped without claiming anybody else's
directory: `test/reap_own.py` claims a direct child of the temp root only when it is named as
`mkdtemp()` names, is not a symlink, holds exactly `copy/` (one of the fixture's five end-state file sets,
byte for byte, regular files only) and optionally the fixture's own manifest, and has not
changed for 24 hours (`--min-age-hours`). Dry run by default; `--apply` re-checks each tree
before removing it. `test/reap_own_test.py` (6 cases, in `npm test`) runs in a sandbox
temp root and plants near-misses by one byte, entry, name, symlink and manifest. Run
against this machine's real `$TMPDIR` on 2026-10-12, dry: `0 tree(s) match` — the 2592
trees counted on 2026-08-20 are already gone (`find … -path '*/copy/sub/b.sh' | wc -l` → 0),
and both suites that leaked them now use `residue.workspace()`.
- **Review round on the first cut (blind reading of PR #45).** Seven fixes, each case written
first and watched failing:
- the reaper refuses a temp root that is empty, `/`, `$HOME` or an ancestor of `$HOME`,
compared after `realpath` (exit 2, `CANNOT RUN`); `--tmpdir /` and `--tmpdir $HOME` were
accepted;
- `--min-age-hours` refuses non-finite values and anything under 1 h, and a non-numeric
value is `CANNOT RUN` instead of a traceback;
- `--apply` renames each tree atomically to a quarantine name in the same root, re-verifies
the renamed tree, and only then removes it; a tree changed in that window goes back to
its own name;
- the fingerprint includes the directory set (`copy/sub`, plus `copy/.git` for the churn
state), so an extra empty directory makes a tree somebody else's;
- the counts are the fixture's: eight cases end in five distinct file sets. The first cut
said seven and carried a sixth set (`.git/index` = `before`) the fixture never ends in;
- an exempt shipped file is held to the `with … TemporaryDirectory(` form line by line: a
bare `mkdtemp()` swapped in for one of `audit_agent.py`'s two context managers kept the
count at 2 and passed;
- stated as a limit, not fixed: a string literal containing `"""` pairs with the next one in
the file and hides every maker between them from the check.

## 0.27.0 — trust by machine, Fabric jobs, and room in every description

Integration wave 4 of the family's waves run (sshlg-skills plan R23–R26). The first cut
Expand Down
2 changes: 1 addition & 1 deletion SKILL-CARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
| Field | Value |
|---|---|
| Pack | `agent-stack` |
| Version | `0.27.0` |
| Version | `0.27.1` |
| Skills | `agent-orchestrator`, `agent-evals`, `agent-interop`, `agent-harness` |
| License | MIT |
| Source | https://github.com/ssheleg/agent-stack |
Expand Down
8 changes: 5 additions & 3 deletions docs/evidence/backlog.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,14 @@ row per line, appended, so two runs closing two rows do not land on the same lin
| B-155 | **`agent-interop/references/gateway.md` names per-caller authorisation and per-hop policy (`:49`, `:108`) but no on-behalf-of model.** Nothing says how authority travels down a chain of agents. Fabric's delegation model (ADR-0144, CO-258) asks the doctrine to describe it: one issuer mints a short delegation JWT carrying an RFC 8693 `act` chain, one audience, capabilities, an effect ceiling and a grant; the callee verifies it offline against published JWKS; calls go direct, with no proxy in the path; `approval/1` and the common refusal codes apply. Doctrine only, no product change. Write it once the plan lands on Fabric main, citing that commit instead of the branch. | peer request fabric-workspace-91, 2026-10-11; plan `passioncode-ai/fabric@32046d14:docs/evidence/plans/2026-10-11-delegation-model/README.md` (branch agent/release-040-wave1) | 3 | 0 | 2 | **1.5** | **closed 2026-10-11** — 0.26.0, `gateway.md` § Delegation; pinned to Fabric main `812876cf` and the contract's `8e49fbd` |
| B-156 | **The per-reply `degraded` / `human_steps` envelope is taught nowhere, because it has no contract home.** Several of the estate's MCP tools (Copylot, Observatory, Research Agent) answer every call with `degraded` and `human_steps` beside the result. The contract defines `degraded` only on the service's well-known document (fabric-agent-contract `docs/specification/service.md` *Well-known document* at `bcada4a`), and `agent-interop/references/mcp.md` teaches that one. The per-reply form is an estate convention; teaching it as a contract would put a second, unsourced home beside the contract's. Teach it once the contract (or Fabric) gives it a rule to cite, and pin that commit. | sshlg-skills plan R24 (`docs/evidence/briefs/2026-10-11-integration-waves.md:117`, branch `waves/2026-10-11-integration`); wave 4 fix round, 2026-10-12 | 3 | 0 | 1 | **3** | **open** — deferred: waits on a contract rule for the per-reply envelope |
| B-157 | **The back-links to the Fabric adapter run one way.** agent-interop now points to `fabric-agent-adapter`'s `creating-fabric-agents`, `adapting-projects-to-fabric` and `building-fabric-services`; of those, only `adapting-projects-to-fabric/SKILL.md:25-26` points back (wire questions). The adapter belongs to the Fabric owner: ask there for a pointer from the other two to agent-interop's tier doctrine. Nothing contradicts today — the adapter carries its own copy of the tier rule (`delegation.md:47-55`, 0.20.0). | wave 4 seam reading, 2026-10-12 | 1 | 0 | 1 | **1** | **open** — an ask to the Fabric adapter's owner |
| B-158 | **The residue-ledger check misses an aliased temp-dir call.** `TEMP_MAKERS` (`test/validate.py`) excludes every dotted receiver but `tempfile.`, so `import tempfile as tf; tf.mkdtemp()` and `from tempfile import mkdtemp as m` pass. make-skill 0.31.0 ported this check and closed the gap there (any receiver, renamed from-imports refused, two negative controls); port that back. No current file here uses an alias. | make-skill W4-B seam reading, 2026-10-12 | 1 | 0 | 1 | **1** | **open** |
| B-158 | **The residue-ledger check misses an aliased temp-dir call.** `TEMP_MAKERS` (`test/validate.py`) excludes every dotted receiver but `tempfile.`, so `import tempfile as tf; tf.mkdtemp()` and `from tempfile import mkdtemp as m` pass. make-skill 0.31.0 ported this check and closed the gap there (any receiver, renamed from-imports refused, two negative controls); port that back. No current file here uses an alias. | make-skill W4-B seam reading, 2026-10-12 | 1 | 0 | 1 | **1** | **closed in v0.27.1 (2026-10-12)** — `TEMP_MAKERS` is make-skill 0.31.0's pattern (any single receiver, one-line renaming from-import refused); the alias and renamed-import negative controls in `test/validate.py` were watched failing on the old pattern (`B-158 self-test: a temp-dir call through a module alias (\`tf.\`) was not found`, `… a renamed from-import of a temp-dir maker was not found`), then passed. Evidence: verification.md, v0.27.1 |
| AG-06a | **A temp tree created outside `test/` is still invisible.** `check_temp_trees_go_through_the_residue_ledger` reads this repository's own suite. A tree made by anything else — a script under `plugins/`, a future helper outside `test/` — is not accounted for, and `residue.report()` counts only what `workspace()` handed out. The mechanism catches a *reviewed* bypass; it does not observe `$TMPDIR`. Observing it would mean a run claiming ownership of directories it cannot prove it made, which is the reaping mistake `agent-sync` refuses by design. | AG-06 port, 2026-08-20; plan R28 of the hub's 2026-10-11 integration waves | 1 | 0 | 1 | **1** | **closed in v0.27.1 (2026-10-12)** for the static half: the check now reads every `*.py` the repository ships or runs (`test/` recursively, `plugins/`, `bin/`) with exact-path, exact-count, reasoned exemptions (`test/residue.py` 1, `agent-harness/scripts/audit_agent.py` 2). Its first run refused a real bypass, `test/audit_regressions/instruction_budget.py` line 21, now `residue.workspace()`. Plants: an aliased maker in a plugin script, a renamed from-import under `bin/`, a third maker in `audit_agent.py` — each refused by file and line. What stays true: the check reads source, it does not observe `$TMPDIR`; that half stays with AG-06b's content fingerprint. Evidence: verification.md, v0.27.1 |
| AG-06b | **The 2592 existing trees are reported and left alone.** `find "$TMPDIR" -maxdepth 4 -type f -path '*/copy/sub/b.sh' \\| wc -l` → **2592** on 2026-08-20, of which **16 were added by this run's own verification** — one run of the leaking fixture before the port, and one of the v0.12.0 archive's copy of it. They are plain `tmpXXXXXXXX` and indistinguishable from every other program's, so they are named rather than swept. The identical fixture still leaks in **`seo-aeo-audit`**, which another agent holds; referred, not edited. | AG-06 port, 2026-08-20; plan R28 of the hub's 2026-10-11 integration waves | 2 | 0 | 1 | **2** | **closed in v0.27.1 (2026-10-12)** — the trees ARE identifiable by content, so they are reapable without claiming another program's directory: `python3 test/residue.py --reap-own` (dry run by default, `--apply` to remove) claims only an `mkdtemp()`-named, non-symlink child of the temp root holding exactly one of the fixture's five end-state file sets and its directory set byte for byte, plus optionally its own manifest, unchanged for 24 h; root, age floor and the removal window hardened after the PR #45 reading (verification.md, v0.27.1). `test/reap_own_test.py` 6/6 in a sandbox temp root; a planted permissive reaper (any tree holding `sub/b.sh`) was refused: `near-misses were claimed as own: ['tmpextrab01', 'tmpextraf01']`. Real `$TMPDIR`, dry, 2026-10-12: `0 tree(s) match`; the `find` count is 0 — the 2592 trees are already gone, and both leaking suites (`agent-stack`, `seo-aeo-audit`) now use `residue.workspace()`. Nothing in the real `$TMPDIR` was deleted |
| B-159 | **`seo-aeo-audit` still has the AG-06a/B-158 hole.** Its temp-tree check reads less than the whole repository and misses aliased calls, and `test/negatives.py:195` makes a bare `tempfile.mkdtemp(prefix="tp-negatives-snap-")` outside any residue ledger (read on its `origin/main`, 2026-10-12). Port agent-stack 0.27.1's whole-repository scan, any-receiver pattern and exact-shape exemptions there, and route that snapshot through the ledger. | agent-stack PR #45 blind reading, 2026-10-12 | 1 | 0 | 1 | **1** | **open** — another repository; referred, not edited |
| B-160 | **make-skill's temp-tree check reads `test/` only.** make-skill 0.31.0 has the any-receiver pattern but walks `test/` alone, so a maker in a shipped script is invisible there exactly as AG-06a described here. Port agent-stack 0.27.1's whole-repository scan with exact-path, exact-count, exact-shape exemptions back to make-skill. | agent-stack PR #45 blind reading, 2026-10-12 | 1 | 0 | 1 | **1** | **open** — another repository; referred, not edited |

## Open, and why

| id | What | Why it is still open |
|---|---|---|
| AG-06a | **A temp tree created outside `test/` is still invisible.** `check_temp_trees_go_through_the_residue_ledger` reads this repository's own suite. A tree made by anything else — a script under `plugins/`, a future helper outside `test/` — is not accounted for, and `residue.report()` counts only what `workspace()` handed out. The mechanism catches a *reviewed* bypass; it does not observe `$TMPDIR`. Observing it would mean a run claiming ownership of directories it cannot prove it made, which is the reaping mistake `agent-sync` refuses by design. | Open: the ledger reads this repository's own suite, so a tree made anywhere else is still unattributable. Plan R28 of the hub's 2026-10-11 integration waves; deferred from wave 4 W4-A, 2026-10-12. |
| AG-06b | **The 2592 existing trees are reported and left alone.** `find "$TMPDIR" -maxdepth 4 -type f -path '*/copy/sub/b.sh' \\| wc -l` → **2592** on 2026-08-20, of which **16 were added by this run's own verification** — one run of the leaking fixture before the port, and one of the v0.12.0 archive's copy of it. They are plain `tmpXXXXXXXX` and indistinguishable from every other program's, so they are named rather than swept. The identical fixture still leaks in **`seo-aeo-audit`**, which another agent holds; referred, not edited. | Open by decision: state this run cannot prove is its own is reported, never deleted. Plan R28 of the hub's 2026-10-11 integration waves; deferred from wave 4 W4-A, 2026-10-12. |
| AG-04a | **This board's own priority is a scalar.** The header computes `P = blast × (1 + age_runs) / effort` while `audit.md` now argues that publishing axes beats multiplying them. The two are not the same claim — the audit is doctrine this pack ships to users about somebody else's system, the board is one repository's own triage — but the tension is real and it is not resolved by saying so. The method's home is the umbrella board (`sshlg-skills/docs/evidence/backlog.md`), which another agent holds; the check added here deliberately scopes to documents under `plugins/`, so it neither polices nor blesses this file. Referred to the umbrella. | Open: the board's formula and `audit.md` disagree, and neither has been chosen. |
Loading
Loading