Skip to content

fix(apps): prove the gateway peer before Code Review Sage secret sends - #23502

Closed
JLargent42 wants to merge 2 commits into
mainfrom
fix/peer-proof-app-senders
Closed

JLargent42 wants to merge 2 commits into
mainfrom
fix/peer-proof-app-senders

Conversation

@JLargent42

@JLargent42 JLargent42 commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: bind the Code Review Sage backend's X-Internal-Secret sends to a proven gateway peer, so the secret is never written to a foreign process that holds the loopback port.

The Code Review Sage app backend sends an X-Internal-Secret to the local gateway over a plain loopback opener. A foreign process that holds the loopback port at send time therefore receives the secret. The exposure is sharpest in the port probe (_probe), whose job is to dial candidate ports that may have something other than this install's gateway listening.

#23283 added a connection-bound gateway-peer proof for the CLI internal-secret senders (kiro_crew.cli_server_peer._owner_verified_secret_urlopen): the secret is written only after the kernel-reported peer on that very connection is proven to be this install's recorded gateway. This change routes the Code Review Sage senders through the same proof.

Part of #26301 (the peer proof for the remaining sandboxed senders). Follow-up to #23270, closed by #23283.

Namespace posture

The Sage app backend is itself launched through the agent sandbox (apps/backend.py → wrap_argv(cmd, mode="standard", …)). On Linux that is unshare(CLONE_NEWUSER) then unshare(CLONE_NEWNS): a new user and mount namespace, with the host pid and network namespaces. On macOS it is sandbox-exec with the Seatbelt profile.

  • Linux, unix socket (the ordinary case): proven. test/test_sage_sender_proves_peer_in_sandbox.py plays the gateway (records the run marker for its own pid and serves the dashboard unix socket), runs the unmonkeypatched proof in a child confined by the real standard namespace sandbox, and asserts a live MATCH. It runs in the Backend Tests (namespace sandbox) job, which asserts userns_available() first.
  • macOS, Seatbelt: proven. The same case under wrap_argv(mode="standard") runs by node id in its own Backend Tests (macOS) step, which fails unless the log reports exactly 1 passed.
  • Linux, TCP-only gateway: proven, by a challenge-response. When the gateway has no dashboard socket (its socket path is too long for AF_UNIX, so the listener degrades to TCP-only), the existing TCP arm cannot reach MATCH from inside the Linux namespace sandbox: the child can list the gateway's /proc/<pid>/fd but cannot read its links, so the peer pid does not resolve. Sage therefore proves a TCP-only gateway with the challenge-response described below, which needs no /proc entry and no new privilege. test_a_tcp_only_gateway_is_proven_inside_the_sandbox runs the Sage sender under the real standard namespace sandbox against the real /api/health handler over TCP and asserts MATCH and that the request arrived; test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox asserts a listener without the proof is refused and receives no secret.
  • macOS, Seatbelt, TCP-only gateway: test_a_tcp_only_gateway_is_proven_inside_seatbelt runs the same TCP-only case under wrap_argv(mode="standard").

TCP-only gateway: the challenge-response

On the connected socket, before any credential byte, the sender sends GET /api/health with a fresh 256-bit nonce in X-Gateway-Peer-Challenge and no credential. The gateway answers a direct-local TCP caller with X-Gateway-Peer-Proof = HMAC-SHA256 keyed with its internal secret over a domain tag, the nonce, its own local endpoint and the caller's endpoint, as its kernel reports them. The sender recomputes the HMAC with the secret it is about to send and its own view of the same connection (peer address, local address), compares in constant time, and writes the secret-bearing request on that same connection only on an exact match.

Why another local process listening on the port cannot spoof it:

  • It does not hold the secret. The proof is an HMAC keyed with the gateway's 128-bit internal secret. A process that holds the secret gains nothing from receiving it.
  • It cannot relay the challenge. A listener that forwards the challenge to the real gateway over its own connection gets a genuine proof, but over that connection's endpoints (the gateway's address, the relay's client port), not the sender's (127.0.0.1:<port>, the sender's client port). For the two to match, the gateway would have to be the far end of the sender's own connection, which is what the proof establishes. The kernel allows only one connection per endpoint pair, and the gateway does not set SO_REUSEPORT, so no other process can share its listening endpoint. test_an_impostor_that_relays_the_challenge_to_the_real_gateway_is_refused asserts the relay obtained a real proof and was still refused.
  • It cannot replay an answer. The nonce is fresh per connection and chosen by the sender. test_a_proof_captured_from_an_earlier_answer_is_not_accepted covers this.
  • It cannot use the gateway as a signing service. The gateway computes the endpoints itself from its kernel, not from the request, and the domain tag keeps the proof from being a valid MAC for anything else keyed with the secret.
  • It cannot get the secret through framing. Any answer that is not a single, bounded, Content-Length-framed HTTP/1.1 200 that keeps the connection open, with exactly one matching proof and nothing after it, within 5 s, is refused with a SEL denied event before the request is written.

The proof's only inputs are the secret Sage already reads (run/gateway-<port>.secret, readable inside the sandbox today) and the socket itself.

What changed

  • review_driver.py adds _verified_urlopen(req, port, timeout, *, operation). When kiro_crew is importable it sends a credential-bearing request through _owner_verified_secret_urlopen if the dashboard socket exists, and through sage_lib/loopback.py's challenge_verified_urlopen if it does not (TCP-only). On the standalone path (python3 sage_lib/review_driver.py), which runs outside an install and has no gateway to prove against, it falls back to the existing proxy/redirect-disabled opener.
  • _api_request sends through the proof with operation="sage_api"; an unproven peer raises _GatewayPeerRefused (not an OSError) and the call returns an error with nothing sent, carrying the proof's own reason.
  • _probe sends through the proof with operation="sage_probe"; a refused peer is a foreign listener, so the secret is not sent and the port reads as "not our gateway" (False).
  • .github/workflows/platform-tests.yml runs the Seatbelt case by node id with -v, and .github/workflows/ci.yml adds the in-sandbox file to the namespace-sandbox job. .github/black-baseline.txt drops a file this change formats.
  • kiro_crew/loopback_peer_challenge.py (new) holds the challenge's message layout (header names, nonce form, endpoint text, HMAC), shared by the gateway and the sender so the two cannot drift. dashboard/handlers/core.py's api_health adds the proof header when a direct-local TCP caller sends a well-formed challenge; its body and every other response are unchanged.
  • docs/system-specs/modules/cli.md documents why the TCP arm does not reach MATCH inside the Linux sandbox and specifies the challenge-response.
  • sage_lib/loopback.py (new) holds the proxy- and redirect-disabled loopback opener guard, moved out of review_driver.py unchanged to keep it under the 2,000-line cap, and the TCP-only sender challenge_verified_urlopen; review_driver.py re-exports loopback_urlopen, the Sage README.md lists the module, and test/test_review_driver_loopback.py runs the guard from its new file.

No rule, config key, or public function is removed. The standalone fallback preserves today's behavior where no install is present.

Why it matters

A local secret written to whatever process currently holds a loopback port is a credential-exposure path. The proof binds the credential to a connection whose peer is confirmed to be this install's own gateway, closing the window where the gateway dies and a foreign listener binds the freed port between resolve and send.

Not a goal

Tests

  • test/test_sage_sender_proves_peer.py: with the dashboard socket present, both _api_request and _probe run through the socket peer proof with the right operation and the X-Internal-Secret header; a refusal sends nothing and keeps the proof's own reason; the probe maps a refusal to "not our gateway".
  • test/test_sage_tcp_peer_challenge.py (real loopback sockets; the genuine gateway is the real api_health handler served by aiohttp; every impostor records the bytes it received):
    • a genuine TCP-only gateway passes and receives the request, on one TCP connection; _api_request and _probe reach it end to end;
    • impostors are refused and receive no secret: no proof, a guessed proof, a relay to the real gateway, a replayed earlier proof, a port taken after the gateway exits;
    • nearby inputs: a port reused after a gateway restart (passes only with the new generation's secret; the stale secret is refused and the new gateway receives nothing), a slow responder (refused at the challenge bound), and eleven malformed answers each carrying a valid proof (not HTTP, HTTP/1.0, 403, Connection: close, chunked, no Content-Length, duplicate proof header, trailing bytes, oversized head, header without a colon, close without answering), with a well-framed control that passes;
    • the unix-socket path is unchanged: with a socket the sender takes the socket proof and never the challenge;
    • the gateway side gives no proof without a challenge, for a malformed or uppercase nonce, to a remote or forwarded caller, over a unix socket, or without a secret; IPv4-mapped endpoints are named the same way on both sides.
  • test/test_sage_sender_proves_peer_in_sandbox.py: the live in-sandbox cases above (Linux unix socket MATCH, Linux TCP-only MATCH with the request delivered, Linux TCP-only impostor refused with nothing received, macOS Seatbelt unix-socket MATCH, macOS Seatbelt TCP-only MATCH).
  • test/test_review_driver_loopback.py and test/test_macos_platform_tests_gate.py updated for the new sender and the pinned macOS step.

The new tests fail on the previous head (ff4431a). With only the shared message module added so the file imports, 32 of the 38 cases in test_sage_tcp_peer_challenge.py and test_sage_sender_proves_peer.py fail; _api_request to a TCP-only gateway returns gateway peer not verified; request not sent: … could not be proven to be this install's gateway. Without it, the file fails at import. In the sandbox, ff4431a's own TCP-only case asserted the refusal that test_a_tcp_only_gateway_is_proven_inside_the_sandbox now asserts against.

Manual verification

None beyond CI: each in-sandbox case runs in a lane that provides its sandbox and cannot skip there.

Related Issues

Part of #26301
Follow-up to #23270 (closed by #23283)

Pattern harvest

Rule candidate: review-prompt
Pattern: a credential-bearing loopback/localhost send that writes the secret before proving the connected peer is this install's own recorded gateway. The whole class was the subject of #23270 (the CLI senders were fixed in #23283; the Code Review Sage senders here; the remaining sandboxed senders tracked in #26301). A reviewer prompt that flags any new X-Internal-Secret-bearing send not routed through the connection-bound peer proof would stop the class at authoring time.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@JLargent42
JLargent42 requested review from a team and KiroCrewCaptain as code owners October 23, 2026 16:54
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision fa84516f7800300ba3f6dc221ceb7c93c3a065d7 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 23, 2026
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of fa84516f7800300ba3f6dc221ceb7c93c3a065d7 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Each item fixes a secret-exposure path that issue #26301 tracks, and tests run inside the real sandbox show the fix. No mechanism already on base does a TCP proof without reading /proc.

What this change ships

Inventory (9 items) — 9 justified

Intent: keep Code Review Sage's internal secret from reaching a foreign process that holds the gateway's loopback port. This is a FIX.

  1. Sage API calls refuse to send the secret unless the dashboard-socket peer is proven to be this install's gateway — justified
  2. With a TCP-only gateway, Sage proves the peer by a challenge-response before sending the secret — justified
  3. The gateway's /api/health adds a proof header when a direct-local caller sends a challenge — justified
  4. The port probe treats an unproven listener as "not our gateway" and sends no secret — justified
  5. Sage sends now emit SEL allowed/denied events (sage_api, sage_probe) — justified
  6. The loopback opener moves unchanged to sage_lib/loopback.py, so review_driver.py stays under the line cap — justified
  7. CI runs the in-sandbox proof in the namespace job and as its own macOS Seatbelt step — justified
  8. The cli.md spec documents the challenge and why the TCP arm fails in the sandbox — justified
  9. The black baseline drops a test file that is now formatted — justified

[FIRST-PRINCIPLES-REVIEWED] fa84516

Description read

[DESCRIPTION-READ] b38ba220cc1f0ac564868e5eaf2a8d393e6ef26cc7a28127c2e59f80f3ca2fff

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of fa84516f7800300ba3f6dc221ceb7c93c3a065d7 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the TCP-only challenge-response covers the one case the existing peer proof cannot reach inside the sandbox, and the message layout lives in one module that both sides import.

Notes

  • prove_tcp_peer refuses when there is no secret ("no internal secret to prove the gateway with"). So on a TCP-only gateway, a _probe or _api_request with an empty _local_secret(port) now finds no gateway, where before any response counted as a match. A request with no credential has nothing to protect, so it could skip the proof and go out on the plain opener.

[DESIGN-REVIEWED] fa84516

Description read

[DESCRIPTION-READ] 22e4110eb3769ef6b342dbd43edd637c3feb8a3fb608eef8f2ca6fd45cc60770

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed fa84516f7800300ba3f6dc221ceb7c93c3a065d7 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] fa84516

Verdict parsed from the review's SHA-scoped output markers for commit fa84516f7800300ba3f6dc221ceb7c93c3a065d7.

False positive or not applicable? A repository writer can comment:
/ai-review override fable fa84516f7800300ba3f6dc221ceb7c93c3a065d7 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of fa84516f7800300ba3f6dc221ceb7c93c3a065d7 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] fa84516

False positive or not applicable? A repository writer can comment:
/ai-review override gpt fa84516f7800300ba3f6dc221ceb7c93c3a065d7 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/peer-proof-app-senders branch from 8bf544e to befd287 Compare October 23, 2026 17:45
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/peer-proof-app-senders branch from befd287 to c65f09d Compare October 23, 2026 20:08
@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: c65f09d
  • What rule changed: apps/builtins/code_review_sage/sage_lib/review_driver.py — the Code Review Sage app backend's two internal-secret senders (_api_request, _probe) now route through the connection-bound gateway-peer proof (_owner_verified_secret_urlopen) instead of a plain loopback opener. This push only updates a pre-existing regression test (test_review_driver_loopback.py) that the behavior change broke; no production rule changed since the prior head.
  • Before -> after (what was allowed/blocked, now): unchanged from the prior head — the X-Internal-Secret is written only after the connected peer is proven to be this install's recorded gateway; an unproven peer raises _GatewayPeerRefused and nothing is sent. The test now stubs the proof to delegate to the same no-proxy loopback opener so its proxy-bypass assertion still holds.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): the peer proof fails closed (refuses the send) when the peer is unproven, so the risk direction is a Sage review that cannot send, not a leaked secret. The namespace-sandbox in-sandbox-resolution gap is tracked in the First Principles discussion and the PR description; this push does not change it.
  • Evidence checked (which sandbox result proves it): the two broken loopback tests now pass; all 7 review_driver/sage peer-proof tests pass; flake8/isort/black (py310) clean on the test file. The 23 test_design_tweak_backend.py 403 failures reproduce with this change stashed — a pre-existing local-sandbox env issue, not from this PR.
  • No regression:
    • Tools still work (which tools/commands were run, result): review_driver + sage peer-proof suites, 7 passed, 0 failed.
    • Backward compatible (old config / old data / old callers still work, how checked): no public function, config key, or rule removed; the standalone import fallback is preserved.
    • Existing tests (which suites passed on this head): the review_driver/sage suites above, all green.
  • How to undo: revert this commit; no data or config migration.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/peer-proof-app-senders branch from c65f09d to 2305236 Compare October 23, 2026 21:07
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 23, 2026
@JLargent42
JLargent42 force-pushed the fix/peer-proof-app-senders branch from 2305236 to 1c5b82b Compare October 24, 2026 00:13
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 24, 2026
@JLargent42

Copy link
Copy Markdown
Contributor Author

Addressed the First Principles premise block on 1c5b82b525 — the Linux in-sandbox half is now proven by a live run.

The BLOCK's concern was an unverified availability premise: if the sealed mount namespace masked the gateway pidfile, the dashboard socket, or /proc/<peer-pid>, the proof would fail closed and every Sage review spawn/publish would break, and no live run confirmed it did not. This push adds test/test_sage_sender_proves_peer_in_sandbox.py, which runs the UNMONKEYPATCHED _owner_verified_secret_urlopen from inside the real standard namespace sandbox and asserts a live MATCH: the parent plays the gateway (records the run marker for its own pid and serves the dashboard unix socket, so SO_PEERCRED reports it and the recorded pid matches), and the sandboxed child reaching MATCH proves the pidfile + socket + /proc are visible through the seal. It is wired into the Backend Tests (namespace sandbox) CI job's file list — the only lane with unprivileged user+mount namespaces — and skips (with a stated reason) only where those are unavailable, asserting userns_available() on the lane so it cannot pass vacuously.

macOS (Seatbelt) is NOT covered: no CI lane builds a sandboxed backend on Seatbelt, so the live MATCH there is still unproven. The PR body's Namespace posture and Manual verification sections say this, and recommend a maintainer confirm Seatbelt before relying on Sage review on macOS. The proof is designed to resolve on macOS via the dashboard socket's LOCAL_PEERCRED (no external tool); only lsof-less macOS on the TCP fallback hard-refuses, with a documented remedy.

@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 1c5b82b
  • What rule changed: no production rule changed since the prior head. This push adds a test (test/test_sage_sender_proves_peer_in_sandbox.py) and wires it into the Backend Tests (namespace sandbox) job's file list in .github/workflows/ci.yml. The underlying change (routing the Code Review Sage senders through the connection-bound gateway-peer proof) is unchanged.
  • Before -> after (what was allowed/blocked, now): unchanged behavior — the new test proves that the already-shipped proof reaches a live MATCH from inside the Linux standard namespace sandbox, rather than changing what is allowed or blocked. No floor, glob, or rule was widened or narrowed.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): the test only asserts; it cannot weaken a control. If the test itself were wrong it would fail (never falsely pass — it asserts userns_available() on the lane and skips elsewhere with a stated reason). The proof it exercises still fails closed on an unproven peer, so the risk direction remains over-refusal, not under-refusal.
  • Evidence checked (which sandbox result proves it): locally the test skips (this host has no unprivileged user namespaces — unshare(CLONE_NEWUSER) EPERM); flake8/isort/black(py310)/mypy clean on the new file; it collects cleanly (1 skipped). The live MATCH is proven by the Backend Tests (namespace sandbox) CI lane, which enables and asserts userns.
  • No regression:
    • Tools still work (which tools/commands were run, result): targeted pytest collects and skips cleanly on this host (1 skipped); the review_driver/sage suites were green on the prior head.
    • Backward compatible (old config / old data / old callers still work, how checked): test-only + a CI file-list addition; no public function, config key, or rule touched.
    • Existing tests (which suites passed on this head): the new test's local collection + the prior head's green suites.
  • How to undo: revert this commit; no data or config migration.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 24, 2026
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 29, 2026
@JLargent42

Copy link
Copy Markdown
Contributor Author

Follow-up to the GPT span ae37fdfa4f97 disposition 6485965658: the failing stage is now named. In namespace-sandbox job 123227072326 on head 08f72b9ef8d28a3fd4ad68f2393fea08f0f5069e (job green, 894 passed), test_a_tcp_only_gateway_is_refused_inside_the_sandbox reported TCP_STAGES {"fd_readlink": "PermissionError: Permission denied", "peer_pid": null, "recorded_pid": 2302, "owner_uid": 1001, ...} with gateway_pid 2302 and own_uid 1001.

So the sandboxed child can list the gateway's /proc/<pid>/fd directory but cannot read its links, which is the kernel's ptrace read check for a process outside the child's user namespace. The TCP arm therefore cannot name the peer pid from inside the sandbox, while the recorded pid and owner uid both resolve. That matches the finding's mechanism; my earlier note that listing succeeded tested the directory, not the links. Reading the links from inside the namespace is not available without the capability, and dropping the pid match would weaken the proof, so the PR keeps the fail-closed refusal with the shorter-data-home remedy. A gateway-side short-path owner-only socket remains the way to make TCP-only homes work for every sandboxed sender.

@JLargent42
JLargent42 force-pushed the fix/peer-proof-app-senders branch from 08f72b9 to cc29fbc Compare October 29, 2026 02:43
@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: cc29fbc
  • What rule changed: no permission rule changed. On the same base (465cc2d), this head makes the TCP-only case explicit. _api_request in review_driver.py keeps the proof's own refusal reason in its error, and adds the remedy text only when _sandboxed_tcp_only holds (Linux, KIROCREW_SANDBOX_ACTIVE=1, no dashboard socket). The Linux in-sandbox TCP-only test now asserts the fail-closed refusal and reports each input of the TCP verdict (fd-link read, resolved peer pid, recorded pid, owner uid) in the warnings summary. docs/system-specs/modules/cli.md documents that the TCP arm cannot name the peer inside the Linux sandbox. The peer proof itself (kiro_crew.cli_server_peer) is unchanged.
  • Before -> after (what was allowed/blocked, now): the Code Review Sage sender still writes its gateway header only after the connected peer is proven to be this install's recorded gateway. Against a TCP-only gateway from inside the Linux sandbox the proof refuses (as on the previous head); the returned error reads gateway peer not verified; request not sent: <the proof's reason> plus the shorter-data-home remedy. Every other refusal (a socket present, outside the sandbox, Windows) carries the proof's reason with no remedy appended.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): Code Review Sage cannot publish against a TCP-only gateway from the Linux sandbox until the data home is short enough for the socket; nothing is sent to an unproven peer. A wrong remedy text would mislead the user but cannot widen what is sent.
  • Evidence checked (which sandbox result proves it): in namespace-sandbox job 123205489287 (run 40798581132, previous head), the TCP-only case was refused inside the sandbox while the unix-socket case reached MATCH in the same run. The new assertions run in the same job on this head, after userns_available() is asserted.
  • No regression:
    • Tools still work (which tools/commands were run, result): pytest -n 2 -q over test/test_sage_sender_proves_peer.py, test/test_sage_sender_proves_peer_in_sandbox.py, test/test_review_driver_loopback.py, test/test_instance_credential_pairing.py, test/test_sage_backend_routes_coverage.py, test/test_mcp_call_site_auth_coverage.py, test/test_ci_file_shards.py, test/test_slack_team_id_env.py, test/test_macos_platform_tests_gate.py, test/test_xdist_worker_restart_contract.py, test/test_ci_failure_annotations.py, test/test_cli_token.py, test/test_routes_parts_sage_composition_contract.py: 433 passed, 3 skipped (the three sandbox cases on this host), 0 failed. The Sage app's own tests/ directory: 1011 passed, 1 skipped, 1 failed; the one failure (test_store_ancestor_chain.py::...test_a_link_planted_inside_the_window_is_followed_outside_every_root) fails identically with this change stashed, so it is not from this diff.
    • Backward compatible (old config / old data / old callers still work, how checked): no function signature or config key changed; the standalone path (no kiro_crew import) gets a stub _existing_dashboard_socket and keeps its proxy- and redirect-disabled opener, covered by test/test_review_driver_loopback.py.
    • Existing tests (which suites passed on this head): the files above, plus black (py310) through scripts/check_black_formatting.py, flake8 and isort on the changed Python files, mypy on review_driver.py, the comment-history gate against origin/main, and scripts/docs-lint.sh.
  • How to undo: revert commit cc29fbc; no data or config migration.

@JLargent42

Copy link
Copy Markdown
Contributor Author

Disposition for the Opus 5.5 block on 08f72b9ef8: fixed on head cc29fbcc1c894add09f95ac9fd147b7d9c764061.

  • span=ba84b291b85a review_driver.py:215 _api_request throws away the proof's own refusal reason — fixed as suggested. _api_request now catches _GatewayPeerRefused as exc and always returns gateway peer not verified; request not sent: {exc}, so the proof's reason (stale identity record, lsof, foreign peer) reaches the operator. The shorter-data-home remedy is appended only when the new _sandboxed_tcp_only(port) holds: sys.platform == "linux", KIROCREW_SANDBOX_ACTIVE == "1" (set by the sandbox launcher), and no dashboard socket. Native Windows (no AF_UNIX, so no socket file) and an unsandboxed process therefore never get it. test/test_sage_sender_proves_peer.py covers both error shapes and each _sandboxed_tcp_only branch (Linux sandbox without a socket; a socket present; outside the sandbox; Windows).

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 29, 2026
@JLargent42
JLargent42 force-pushed the fix/peer-proof-app-senders branch from cc29fbc to ff4431a Compare October 29, 2026 03:19
@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: ff4431a
  • What rule changed: no permission rule changed. On the same base (465cc2d), this head moves the proxy- and redirect-disabled loopback_urlopen import guard out of review_driver.py into the new sibling sage_lib/loopback.py unchanged, to keep review_driver.py under the 2,000-line file cap (now 1,987 lines); review_driver.py re-exports loopback_urlopen, and test/test_review_driver_loopback.py runs the guard from its new file and checks the re-export is the same object. The rest is as on the previous head: this head makes the TCP-only case explicit. _api_request in review_driver.py keeps the proof's own refusal reason in its error, and adds the remedy text only when _sandboxed_tcp_only holds (Linux, KIROCREW_SANDBOX_ACTIVE=1, no dashboard socket). The Linux in-sandbox TCP-only test now asserts the fail-closed refusal and reports each input of the TCP verdict (fd-link read, resolved peer pid, recorded pid, owner uid) in the warnings summary. docs/system-specs/modules/cli.md documents that the TCP arm cannot name the peer inside the Linux sandbox. The peer proof itself (kiro_crew.cli_server_peer) is unchanged.
  • Before -> after (what was allowed/blocked, now): the Code Review Sage sender still writes its gateway header only after the connected peer is proven to be this install's recorded gateway. Against a TCP-only gateway from inside the Linux sandbox the proof refuses (as on the previous head); the returned error reads gateway peer not verified; request not sent: <the proof's reason> plus the shorter-data-home remedy. Every other refusal (a socket present, outside the sandbox, Windows) carries the proof's reason with no remedy appended.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): Code Review Sage cannot publish against a TCP-only gateway from the Linux sandbox until the data home is short enough for the socket; nothing is sent to an unproven peer. A wrong remedy text would mislead the user but cannot widen what is sent.
  • Evidence checked (which sandbox result proves it): in namespace-sandbox job 123205489287 (run 40798581132, previous head), the TCP-only case was refused inside the sandbox while the unix-socket case reached MATCH in the same run. The new assertions run in the same job on this head, after userns_available() is asserted.
  • No regression:
    • Tools still work (which tools/commands were run, result): pytest -n 2 -q over test/test_sage_sender_proves_peer.py, test/test_sage_sender_proves_peer_in_sandbox.py, test/test_review_driver_loopback.py, test/test_instance_credential_pairing.py, test/test_sage_backend_routes_coverage.py, test/test_mcp_call_site_auth_coverage.py, test/test_ci_file_shards.py, test/test_slack_team_id_env.py, test/test_macos_platform_tests_gate.py, test/test_xdist_worker_restart_contract.py, test/test_ci_failure_annotations.py, test/test_cli_token.py, test/test_routes_parts_sage_composition_contract.py, test/test_security_posture.py, test/test_security_posture_facade.py, test/test_source_corpus.py, test/test_redaction_mirror_parity.py: 545 passed, 3 skipped (the three sandbox cases on this host), 0 failed. The Sage app's own tests/ directory: 1011 passed, 1 skipped, 1 failed; the one failure (test_store_ancestor_chain.py::...test_a_link_planted_inside_the_window_is_followed_outside_every_root) fails identically with this change stashed, so it is not from this diff.
    • Backward compatible (old config / old data / old callers still work, how checked): no function signature or config key changed; the standalone path (no kiro_crew import) gets a stub _existing_dashboard_socket and keeps its proxy- and redirect-disabled opener, covered by test/test_review_driver_loopback.py.
    • Existing tests (which suites passed on this head): the files above, plus black (py310) through scripts/check_black_formatting.py, flake8 and isort on the changed Python files, mypy on review_driver.py, the comment-history gate against origin/main, and scripts/docs-lint.sh.
  • How to undo: revert commit ff4431a; no data or config migration.

@JLargent42

Copy link
Copy Markdown
Contributor Author

Disposition for GPT 6.1 span ae37fdfa4f97 (raised on 5d518dd40f), current as of head ff4431a9fc29097736260607e1c0338fe5393ad8. Status: accepted limitation pending operator decision. This supersedes my earlier GPT dispositions for this span (6484652206, 6485573639 and 6485965658); the first was a rebuttal and was wrong.

  • span=ae37fdfa4f97 TCP-only gateways cannot be verified from the Linux Sage sandbox — the finding is confirmed. On head 08f72b9e, test_a_tcp_only_gateway_is_refused_inside_the_sandbox ran in the namespace-sandbox job and reported fd_readlink: "PermissionError: Permission denied" and peer_pid: null, while the recorded gateway pid and owner uid resolved. Inside the Linux sandbox, reading the gateway's /proc/<pid>/fd links is denied, so the TCP arm cannot name the peer, a TCP-only gateway cannot be proven, and Sage refuses before sending anything. Posture: it fails closed with a clear error. _api_request returns gateway peer not verified; request not sent: <the proof's reason>, and _probe treats the port as not this install's gateway. The case is reachable only when the gateway is TCP-only because the dashboard socket path under the data home overflows the AF_UNIX limit (a data home of roughly 85 or more characters). The remedy is named in the error when Sage runs in the Linux sandbox against a gateway with no dashboard socket: use a data home short enough for the socket path to fit, then restart the gateway. The test pins the refusal, and docs/system-specs/modules/cli.md documents the limit. The operator decides between accepting this fail-closed limitation and adding a TCP peer path that works from the sandbox (for example a gateway-side owner-only socket at a short path). No code changed for this record.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes (issue required) at ff4431a. This PR also stays reserved for a human reviewer because it changes which peer receives the gateway's internal secret and edits CI workflows.

I found no defect in the code. Both Sage senders now prove the peer before the first byte (review_driver.py#L204-L213, L739-L762), a refusal sends nothing, and the standalone path keeps the old proxy- and redirect-disabled opener.

What blocks: the deferred work points at #23270, which was closed on 2026-10-27 as fixed by #23283. Two items remain without an open issue:

  • The other sandboxed senders listed under Not a goal still use plain loopback_urlopen on main (cron_trigger.py, mcp_computer.py, mcp_core_parts/gateway.py).
  • The gateway-side change that would let a TCP-only gateway be proven from the Linux sandbox. Until then _TCP_ONLY_REMEDY refuses Sage reviews on hosts whose data home is too long for a unix socket. Both the Design and First Principles lanes asked for this to be tracked.

Please open one issue per item (or reopen #23270 if it still describes them) and link it in the description in place of the closed #23270.

The red macOS shard 5 is test_probes_see_a_driver_declared_in_a_nested_repo, a known flake unrelated to this diff; the PR's own Seatbelt step passed.

— crew-pr-reviewer (r135)

@JLargent42

Copy link
Copy Markdown
Contributor Author

@iamwhatever thanks for the review at ff4431a9fc29097736260607e1c0338fe5393ad8. What changed since (no code change, no push; the head is unchanged):

@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 4edda1e
  • What rule changed: the Code Review Sage sender's TCP-only arm. With no dashboard socket, review_driver._verified_urlopen now proves the peer with a challenge-response (sage_lib/loopback.py challenge_verified_urlopen, message layout in kiro_crew/loopback_peer_challenge.py) instead of the /proc-based TCP verdict. The gateway's GET /api/health (dashboard/handlers/core.py api_health) adds an X-Gateway-Peer-Proof response header when a direct-local TCP caller sends a well-formed X-Gateway-Peer-Challenge nonce. The unix-socket arm is unchanged.
  • Before -> after (what was allowed/blocked, now): before, a TCP-only gateway was refused from inside the Linux namespace sandbox (peer pid unresolvable), so Sage could not send. After, the secret-bearing request is written only when the peer returns HMAC-SHA256(internal secret, domain tag + nonce + the connection's two endpoints as the gateway sees them) matching the sender's own computation for the same connection; everything else is still refused with a SEL denied event and nothing sent. /api/health gains no access: the proof grants nothing, is bound to a caller-chosen fresh nonce and kernel-reported endpoints, and is domain-separated from every other use of the secret.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): if the framing checks were too strict, a genuine TCP-only gateway would be refused, which was the behavior before this change. If the proof were spoofable, a foreign loopback listener could receive the Sage secret; the tests cover a listener with no proof, a guessed proof, a relay to the real gateway, a replayed proof, a port taken after the gateway exits, a slow responder and eleven malformed answers carrying a valid proof. All are refused and the listener receives no secret.
  • Evidence checked (which sandbox result proves it): locally (no unprivileged user namespaces on this host) test/test_sage_tcp_peer_challenge.py and test/test_sage_sender_proves_peer.py pass, 38 tests; 32 of them fail on ff4431a. The live in-sandbox proof is test_a_tcp_only_gateway_is_proven_inside_the_sandbox and test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox in the Backend Tests (namespace sandbox) lane, plus test_a_tcp_only_gateway_is_proven_inside_seatbelt on macOS.
  • No regression:
    • Tools still work (which tools/commands were run, result): the affected files plus test/test_api_health.py, test/test_core_handlers_facade.py, test/test_review_driver_loopback.py, the in-sandbox file and the Sage app's own tests, 1409 passed at -n 2. The one failure (test_store_ancestor_chain.py) fails the same way with this change stashed, so it is local to this host.
    • Backward compatible (old config / old data / old callers still work, how checked): /api/health's body is unchanged and its header is added only on a challenge, so test/test_api_health.py's frozen probe contract passes. The unix-socket arm and the standalone fallback are unchanged. No config key or public function is removed.
    • Existing tests (which suites passed on this head): the suites above; isort, black, flake8, mypy on the changed source files, the file line cap and the comment-history gate are clean.
  • How to undo: revert 4edda1e; no data or config migration. Sage then refuses a TCP-only gateway again.

@JLargent42

Copy link
Copy Markdown
Contributor Author

@iamwhatever on the second deferred item from your review (the TCP-only gateway that the Linux sandbox could not prove): it is now covered in this PR rather than tracked separately, at head 4edda1e4f3c5a42e2114473d87dcf69e61ca2fe4.

  • What changed: with no dashboard socket, Sage proves the gateway with a challenge-response on the connection the secret will travel over. It sends GET /api/health with a fresh nonce and no credential; the gateway answers with an HMAC keyed with its internal secret over the nonce and the connection's endpoints as its kernel reports them; Sage writes the request on that connection only on an exact match. It reads no /proc entry and needs no new privilege. _TCP_ONLY_REMEDY and _sandboxed_tcp_only are removed. The PR description's new "TCP-only gateway: the challenge-response" section explains why a different local listener on the port cannot pass it.
  • Tests:
    • in the real Linux namespace sandbox: test_a_tcp_only_gateway_is_proven_inside_the_sandbox (MATCH, request delivered) and test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox (refused, nothing received); under Seatbelt, test_a_tcp_only_gateway_is_proven_inside_seatbelt;
    • against real loopback sockets: test/test_sage_tcp_peer_challenge.py, covering the genuine gateway, impostors (no proof, a guessed proof, a relay to the real gateway, a replayed proof, a port taken after exit), a restart on the same port, a slow responder, eleven malformed answers each carrying a valid proof, and the unchanged unix-socket arm.
  • Remaining sandboxed senders: still tracked in security: prove the gateway peer before cron trigger, computer-use and in-agent MCP secret sends #26301 (unchanged).

The Sensitive change review template for this head is posted above.

The Code Review Sage backend sender wrote its X-Internal-Secret to the local
gateway over a plain loopback opener, so a foreign process that held the port
could receive it -- most acutely in the port probe, which dials candidate ports
by design. Route both the credential-bearing API call and the probe through the
gateway's connection-bound peer proof (_owner_verified_secret_urlopen), so the
secret is written only after the connected peer is proven to be this install's
recorded gateway; an unproven peer is refused and nothing is sent. The standalone
path (no kiro_crew import) keeps its proxy/redirect-disabled opener.

Run the same live in-sandbox proof under wrap_argv(mode="standard") on
macOS, where the Sage backend runs under sandbox-exec. The case skips only
where /usr/bin/sandbox-exec is absent, and fails if the wrap did not apply
Seatbelt or the kernel does not report the child as sandboxed.

Add a Linux in-sandbox case for a TCP-only gateway (loopback listener, no
dashboard socket), the configuration the listener degrades to when the
socket path exceeds the AF_UNIX limit. The proof's TCP arm does not reach
MATCH from inside the Linux namespace sandbox: the child cannot read the
gateway's /proc/<pid>/fd links, so the peer pid does not resolve. The case
pins that fail-closed refusal and reports each input of the TCP verdict
(fd-link read, resolved peer pid, recorded pid, owner uid) in the warnings
summary. A refused Sage _api_request now keeps the proof's own reason, and
adds the remedy (a data home short enough for the socket path) only when it
runs in the Linux sandbox against a gateway with no dashboard socket. The
cli spec's TCP-arm entry documents the limit.

Move the proxy- and redirect-disabled loopback opener guard out of
review_driver into sage_lib/loopback.py unchanged, so review_driver stays
under the 2,000-line file cap; review_driver re-exports loopback_urlopen,
and the fallback-guard test runs the guard from its new file.

The sharded macOS run is pytest -q, which does not name passing tests,
and a skip exits 0. Run
test_sage_peer_proof_reaches_match_inside_seatbelt by node id with -v
beside the existing peer-identity canary, and fail the step unless it
reports exactly 1 passed. Pin the node id in
test_macos_platform_tests_gate.py's by-name list.
@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: 0262be3
  • What rule changed: the Code Review Sage sender's TCP-only arm. With no dashboard socket, review_driver._verified_urlopen now proves the peer with a challenge-response (sage_lib/loopback.py challenge_verified_urlopen, message layout in kiro_crew/loopback_peer_challenge.py) instead of the /proc-based TCP verdict. The gateway's GET /api/health (dashboard/handlers/core.py api_health) adds an X-Gateway-Peer-Proof response header when a direct-local TCP caller sends a well-formed X-Gateway-Peer-Challenge nonce. The unix-socket arm is unchanged. Since 4edda1e: the branch is rebased onto main c21f5a0, the challenge answer's status code and Content-Length must be plain ASCII digits (bounded length), and any unexpected failure while checking the answer is now an audited refusal (SEL denied event) instead of an unaudited one.
  • Before -> after (what was allowed/blocked, now): before, a TCP-only gateway was refused from inside the Linux namespace sandbox (peer pid unresolvable), so Sage could not send. After, the secret-bearing request is written only when the peer returns HMAC-SHA256(internal secret, domain tag + nonce + the connection's two endpoints as the gateway sees them) matching the sender's own computation for the same connection; everything else is still refused with a SEL denied event and nothing sent. /api/health gains no access: the proof grants nothing, is bound to a caller-chosen fresh nonce and kernel-reported endpoints, and is domain-separated from every other use of the secret.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): if the framing checks were too strict, a genuine TCP-only gateway would be refused, which was the behavior before this change. If the proof could be produced by another process, a foreign loopback listener could receive the Sage secret; the tests cover a listener with no proof, a guessed proof, a relay to the real gateway, a replayed proof, a port taken after the gateway exits, a slow responder, and malformed answers carrying a valid proof (now including non-ASCII digits in the status and Content-Length). All are refused, audited, and the listener receives no secret.
  • Evidence checked (which sandbox result proves it): locally (no unprivileged user namespaces on this host) on 0262be3 test/test_sage_tcp_peer_challenge.py, test/test_sage_sender_proves_peer.py, test/test_sage_sender_proves_peer_in_sandbox.py, test/test_review_driver_loopback.py, test/test_macos_platform_tests_gate.py and test/test_facade_mirror_guards.py give 136 passed, 5 skipped at -n 2. The live in-sandbox proof is test_a_tcp_only_gateway_is_proven_inside_the_sandbox and test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox in the Backend Tests (namespace sandbox) lane, plus test_a_tcp_only_gateway_is_proven_inside_seatbelt on macOS.
  • No regression:
    • Tools still work (which tools/commands were run, result): the files above plus the dashboard handler tests that import handlers.core (399 passed at -n 2).
    • Backward compatible (old config / old data / old callers still work, how checked): /api/health's body is unchanged and its header is added only on a challenge, so the frozen probe contract in test/test_api_health.py holds. The unix-socket arm and the standalone fallback are unchanged. No config key or public function is removed.
    • Existing tests (which suites passed on this head): the suites above; black (py310), isort, flake8 and mypy on the changed files, the file line cap, and the comment-history gate against origin/main are clean. review_driver.py is 1970 lines.
  • How to undo: revert 0262be3 and 695d4a0; no data or config migration. Sage then refuses a TCP-only gateway again.

@JLargent42

Copy link
Copy Markdown
Contributor Author

GPT 6.1 finding on 4edda1e4f3c5a42e2114473d87dcf69e61ca2fe4 (sage_lib/loopback.py:107, non-ASCII digits in the challenge answer refuse the send without the SEL denial): fixed in 0262be33adea2622977786b930c78e536b180efb.

  • _read_challenge_answer now accepts the status code and Content-Length only when they are 1 to 3 and 1 to 6 ASCII digits (_is_ascii_number), so "²" and oversized numbers raise ChallengeRefused before int() runs. This covers both sites the review named.
  • challenge_verified_urlopen._verify now audits every failure while checking the answer, not only ChallengeRefused: an unexpected exception records the SEL denied event with its type name and still refuses the send.
  • In test/test_sage_tcp_peer_challenge.py, the non_ascii_status and non_ascii_content_length cases of test_a_malformed_answer_is_refused_even_with_a_valid_proof send a superscript-digit status and Content-Length with an otherwise valid proof, and test_an_unexpected_error_in_the_challenge_is_an_audited_refusal forces an unanticipated error. Each asserts the refusal and that the listener receives no secret. All three fail against the 4edda1e loopback.py (ValueError / RuntimeError escape) and pass on this head.

The head is also rebased onto main c21f5a0101c78325f23a41dcb6535678a504f84d (authorized by the owner; the old merge-base made the comment-history gate flag a line main had removed).

@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: bbccff6
  • What rule changed: the Code Review Sage sender's TCP-only arm. With no dashboard socket, review_driver._verified_urlopen now proves the peer with a challenge-response (sage_lib/loopback.py challenge_verified_urlopen, message layout in kiro_crew/loopback_peer_challenge.py) instead of the /proc-based TCP verdict. The gateway's GET /api/health (dashboard/handlers/core.py api_health) adds an X-Gateway-Peer-Proof response header when a direct-local TCP caller sends a well-formed X-Gateway-Peer-Challenge nonce. The unix-socket arm is unchanged. Since 4edda1e: the branch is rebased onto main c21f5a0, the challenge answer's status code and Content-Length must be plain ASCII digits (bounded length), and any unexpected failure while checking the answer is now an audited refusal (SEL denied event) instead of an unaudited one.
  • Before -> after (what was allowed/blocked, now): before, a TCP-only gateway was refused from inside the Linux namespace sandbox (peer pid unresolvable), so Sage could not send. After, the secret-bearing request is written only when the peer returns HMAC-SHA256(internal secret, domain tag + nonce + the connection's two endpoints as the gateway sees them) matching the sender's own computation for the same connection; everything else is still refused with a SEL denied event and nothing sent. /api/health gains no access: the proof grants nothing, is bound to a caller-chosen fresh nonce and kernel-reported endpoints, and is domain-separated from every other use of the secret.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): if the framing checks were too strict, a genuine TCP-only gateway would be refused, which was the behavior before this change. If the proof could be produced by another process, a foreign loopback listener could receive the Sage secret; the tests cover a listener with no proof, a guessed proof, a relay to the real gateway, a replayed proof, a port taken after the gateway exits, a slow responder, and malformed answers carrying a valid proof (now including non-ASCII digits in the status and Content-Length). All are refused, audited, and the listener receives no secret.
  • Evidence checked (which sandbox result proves it): locally (no unprivileged user namespaces on this host) on bbccff6 test/test_sage_tcp_peer_challenge.py, test/test_sage_sender_proves_peer.py, test/test_sage_sender_proves_peer_in_sandbox.py, test/test_review_driver_loopback.py, test/test_macos_platform_tests_gate.py and test/test_facade_mirror_guards.py give 136 passed, 5 skipped at -n 2. The live in-sandbox proof is test_a_tcp_only_gateway_is_proven_inside_the_sandbox and test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox in the Backend Tests (namespace sandbox) lane, plus test_a_tcp_only_gateway_is_proven_inside_seatbelt on macOS.
  • No regression:
    • Tools still work (which tools/commands were run, result): the files above plus the dashboard handler tests that import handlers.core (399 passed at -n 2).
    • Backward compatible (old config / old data / old callers still work, how checked): /api/health's body is unchanged and its header is added only on a challenge, so the frozen probe contract in test/test_api_health.py holds. The unix-socket arm and the standalone fallback are unchanged. No config key or public function is removed.
    • Existing tests (which suites passed on this head): the suites above; black (py310), isort, flake8 and mypy on the changed files, the file line cap, and the comment-history gate against origin/main are clean. review_driver.py is 1970 lines. Since 0262be3 only test/test_sage_tcp_peer_challenge.py changed (its autouse fixture patches through _floor_monkeypatch, and the slow-responder test holds its answer on an event instead of a fixed sleep), and semgrep 1.78.0 with the repo rules reports 0 findings on the changed files.
  • How to undo: revert bbccff6 and 695d4a0; no data or config migration. Sage then refuses a TCP-only gateway again.

@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: fab3f9c
  • What rule changed: the Code Review Sage sender's TCP-only arm. With no dashboard socket, review_driver._verified_urlopen now proves the peer with a challenge-response (sage_lib/loopback.py challenge_verified_urlopen, message layout in kiro_crew/loopback_peer_challenge.py) instead of the /proc-based TCP verdict. The gateway's GET /api/health (dashboard/handlers/core.py api_health) adds an X-Gateway-Peer-Proof response header when a direct-local TCP caller with a served Host (check_host, the same gate as the identity fields in _liveness_payload) sends a well-formed X-Gateway-Peer-Challenge nonce. The unix-socket arm is unchanged. Since 4edda1e: the branch is rebased onto main c21f5a0, the challenge answer's status code and Content-Length must be plain ASCII digits (bounded length), and any unexpected failure while checking the answer is now an audited refusal (SEL denied event) instead of an unaudited one. Since bbccff6: the proof is withheld from a loopback request whose Host the gateway does not serve, so a DNS-rebound page learns only {"ok": true} from /api/health, as before this PR.
  • Before -> after (what was allowed/blocked, now): before, a TCP-only gateway was refused from inside the Linux namespace sandbox (peer pid unresolvable), so Sage could not send. After, the secret-bearing request is written only when the peer returns HMAC-SHA256(internal secret, domain tag + nonce + the connection's two endpoints as the gateway sees them) matching the sender's own computation for the same connection; everything else is still refused with a SEL denied event and nothing sent. /api/health gains no access: the proof grants nothing, is bound to a caller-chosen fresh nonce and kernel-reported endpoints, and is domain-separated from every other use of the secret.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): if the framing checks were too strict, a genuine TCP-only gateway would be refused, which was the behavior before this change. If the proof could be produced by another process, a foreign loopback listener could receive the Sage secret; the tests cover a listener with no proof, a guessed proof, a relay to the real gateway, a replayed proof, a port taken after the gateway exits, a slow responder, and malformed answers carrying a valid proof (now including non-ASCII digits in the status and Content-Length). All are refused, audited, and the listener receives no secret.
  • Evidence checked (which sandbox result proves it): locally (no unprivileged user namespaces on this host) on fab3f9c test/test_sage_tcp_peer_challenge.py, test/test_sage_sender_proves_peer.py, test/test_sage_sender_proves_peer_in_sandbox.py, test/test_review_driver_loopback.py, test/test_macos_platform_tests_gate.py plus every test file that exercises api_health / _liveness_payload (test/test_api_health.py, test/test_core_handlers_facade.py, test/test_dashboard_handlers_core_coverage.py, test/test_relay_header_strip.py and 7 more) give 601 passed, 5 skipped at -n 2. The new rebound-host case fails against the bbccff6 core.py and passes here. The live in-sandbox proof is test_a_tcp_only_gateway_is_proven_inside_the_sandbox and test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox in the Backend Tests (namespace sandbox) lane, plus test_a_tcp_only_gateway_is_proven_inside_seatbelt on macOS.
  • No regression:
    • Tools still work (which tools/commands were run, result): the files above; on 0262be3 also test/test_facade_mirror_guards.py and the dashboard handler tests that import handlers.core (136 and 399 passed at -n 2). The real Sage sender sends Host: 127.0.0.1:<port>, which a production gateway's loopback allowlist serves.
    • Backward compatible (old config / old data / old callers still work, how checked): /api/health's body is unchanged and its header is added only on a challenge, so the frozen probe contract in test/test_api_health.py holds. The unix-socket arm and the standalone fallback are unchanged. No config key or public function is removed.
    • Existing tests (which suites passed on this head): the suites above; black (py310), isort, flake8 and mypy on the changed files, the file line cap, and the comment-history gate against origin/main are clean. docs-lint.sh passes. review_driver.py is 1970 lines. semgrep 1.78.0 with the repo rules reports 0 findings on the changed files.
  • How to undo: revert fab3f9c and 695d4a0; no data or config migration. Sage then refuses a TCP-only gateway again.

@JLargent42

Copy link
Copy Markdown
Contributor Author

Opus 5.5 finding on bbccff652ebe41590e36c7e55df5b66ea24992a3 (dashboard/handlers/core.py:368, a DNS-rebound page could read X-Gateway-Peer-Proof because the challenge answer skipped check_host): fixed in fab3f9c0bf6563139c3c9a40f6e1bb24d27ea513, as the review suggested.

  • _gateway_peer_proof now answers only when is_direct_local_request(request) and check_host(request), the same gate _liveness_payload puts on the identity fields. A rebound request carries a Host the gateway does not serve, so it gets no proof and /api/health again tells it only {"ok": true}.
  • The Sage sender already sends Host: 127.0.0.1:<port>, which a real gateway's loopback allowlist serves, so a genuine TCP-only gateway still passes. The test gateway in test/sage_tcp_gateway_helpers.py now carries the loopback origins for its port, as the review asked.
  • New case rebound-host in test_the_gateway_gives_no_proof_outside_a_direct_local_tcp_challenge (test/test_sage_tcp_peer_challenge.py): a loopback request with a valid nonce and Host: rebound.example:5476 gets no proof. It fails against the bbccff6 core.py and passes on this head. The spec text in docs/system-specs/modules/cli.md ("Gateway peer challenge") names the check_host gate.

Also in this head since 0262be33adea2622977786b930c78e536b180efb: the two SAST (semgrep) findings in the new test file are fixed. The autouse fixture patches through _floor_monkeypatch, and the slow-responder test holds its answer on an event instead of a fixed sleep.

…et sends

When the gateway has no dashboard socket (its socket path is too long for
AF_UNIX, so it listens on loopback TCP only), the sandboxed Sage backend could
not prove its peer: the TCP arm resolves the peer pid through the gateway's
/proc/<pid>/fd links, which the Linux namespace sandbox cannot read, so every
review spawn and publish was refused.

Prove a TCP-only gateway with a challenge-response on the connection the
secret will travel over. Before any credential byte, the sender sends
GET /api/health with a fresh 256-bit nonce in X-Gateway-Peer-Challenge. The
gateway answers a direct-local TCP caller with X-Gateway-Peer-Proof:
HMAC-SHA256 keyed with its internal secret over a domain tag, the nonce, and
the connection's two endpoints as its own kernel reports them. The sender
recomputes it with the secret it would send and its own view of the same
connection and writes the request on that connection only on an exact match.
A listener without the secret cannot answer; one that relays the challenge
to the real gateway gets a proof over its own connection's endpoints, which
do not match; the nonce rules out replay. Every other answer (non-200, a
missing or wrong proof, Connection: close, non-Content-Length framing,
oversized, extra bytes, or no answer within 5 s) refuses with a SEL denied
event and nothing is sent. The proof reads no /proc entry and nothing in
run/ beyond the secret the sender already reads.

The message layout lives in kiro_crew.loopback_peer_challenge, shared by
both sides; the sender is sage_lib/loopback.py's challenge_verified_urlopen;
the gateway's answer is in api_health. With the dashboard socket present
the sender keeps the socket peer-credential proof unchanged. The
shorter-data-home remedy is removed, since the case it described now passes.

Tests: a real aiohttp api_health gateway passes over TCP, on one connection;
impostors (no proof, a guessed proof, a relay to the real gateway, a replayed
proof, a port taken after the gateway exits) are refused and receive no
secret; a restarted gateway on the same port passes only with its new
secret; a slow responder and eleven malformed answers carrying a valid proof
are refused. The in-sandbox file runs the Sage sender under the real Linux
namespace sandbox and under Seatbelt against a TCP-only gateway (MATCH, the
request arrives) and against an impostor (refused, nothing received).
@JLargent42

Copy link
Copy Markdown
Contributor Author

Sensitive change review

  • Head commit: fa84516
  • What rule changed: the Code Review Sage sender's TCP-only arm. With no dashboard socket, review_driver._verified_urlopen now proves the peer with a challenge-response (sage_lib/loopback.py challenge_verified_urlopen, message layout in kiro_crew/loopback_peer_challenge.py) instead of the /proc-based TCP verdict. The gateway's GET /api/health (dashboard/handlers/core.py api_health) adds an X-Gateway-Peer-Proof response header when a direct-local TCP caller with a served Host (check_host, the same gate as the identity fields in _liveness_payload) sends a well-formed X-Gateway-Peer-Challenge nonce. The unix-socket arm is unchanged. Since 4edda1e: the branch is rebased onto main c21f5a0, the challenge answer's status code and Content-Length must be plain ASCII digits (bounded length), and any unexpected failure while checking the answer is now an audited refusal (SEL denied event) instead of an unaudited one. Since bbccff6: the proof is withheld from a loopback request whose Host the gateway does not serve, so a DNS-rebound page learns only {"ok": true} from /api/health, as before this PR.
  • Before -> after (what was allowed/blocked, now): before, a TCP-only gateway was refused from inside the Linux namespace sandbox (peer pid unresolvable), so Sage could not send. After, the secret-bearing request is written only when the peer returns HMAC-SHA256(internal secret, domain tag + nonce + the connection's two endpoints as the gateway sees them) matching the sender's own computation for the same connection; everything else is still refused with a SEL denied event and nothing sent. /api/health gains no access: the proof grants nothing, is bound to a caller-chosen fresh nonce and kernel-reported endpoints, and is domain-separated from every other use of the secret.
  • Worst case if wrong (breaks commands / leaks secret / locks user out): if the framing checks were too strict, a genuine TCP-only gateway would be refused, which was the behavior before this change. If the proof could be produced by another process, a foreign loopback listener could receive the Sage secret; the tests cover a listener with no proof, a guessed proof, a relay to the real gateway, a replayed proof, a port taken after the gateway exits, a slow responder, and malformed answers carrying a valid proof (now including non-ASCII digits in the status and Content-Length). All are refused, audited, and the listener receives no secret.
  • Evidence checked (which sandbox result proves it): locally (no unprivileged user namespaces on this host) on fa84516 test/test_sage_tcp_peer_challenge.py, test/test_sage_sender_proves_peer.py, test/test_sage_sender_proves_peer_in_sandbox.py, test/test_review_driver_loopback.py, test/test_macos_platform_tests_gate.py plus every test file that exercises api_health / _liveness_payload (test/test_api_health.py, test/test_core_handlers_facade.py, test/test_dashboard_handlers_core_coverage.py, test/test_relay_header_strip.py and 7 more) give 601 passed, 5 skipped at -n 2. The new rebound-host case fails against the bbccff6 core.py and passes here. The live in-sandbox proof is test_a_tcp_only_gateway_is_proven_inside_the_sandbox and test_an_impostor_on_a_tcp_only_port_is_refused_inside_the_sandbox in the Backend Tests (namespace sandbox) lane, plus test_a_tcp_only_gateway_is_proven_inside_seatbelt on macOS.
  • No regression:
    • Tools still work (which tools/commands were run, result): the files above; on 0262be3 also test/test_facade_mirror_guards.py and the dashboard handler tests that import handlers.core (136 and 399 passed at -n 2). The real Sage sender sends Host: 127.0.0.1:<port>, which a production gateway's loopback allowlist serves.
    • Backward compatible (old config / old data / old callers still work, how checked): /api/health's body is unchanged and its header is added only on a challenge, so the frozen probe contract in test/test_api_health.py holds. The unix-socket arm and the standalone fallback are unchanged. No config key or public function is removed.
    • Existing tests (which suites passed on this head): the suites above; black (py310), isort, flake8 and mypy on the changed files, the file line cap, and the comment-history gate against origin/main are clean. docs-lint.sh passes. Since fab3f9c only test/sage_tcp_gateway_helpers.py changed: its two .result() start/stop waits take the 30 s ceiling that test/test_wait_discipline_ratchet.py (rule D1) requires, and that ratchet passes locally. review_driver.py is 1970 lines. semgrep 1.78.0 with the repo rules reports 0 findings on the changed files.
  • How to undo: revert fa84516 and 695d4a0; no data or config migration. Sage then refuses a TCP-only gateway again.

@JLargent42

Copy link
Copy Markdown
Contributor Author

First Principles BLOCK on 4edda1e4f3c5a42e2114473d87dcf69e61ca2fe4 (the gateway-side TCP peer challenge was shipped while the frozen ## Not a goal still called it "pending owner decision"): resolved by the owner's decision, not by removing the challenge.

  • The owner (largjaco) decided against accepting the TCP-only fail-closed limit and asked for the TCP-only proof to be made in this PR. That is the human Goal amendment the review's Clears when names. The description now lists the TCP-only proof as a goal, and the "pending owner decision" note and the _sandboxed_tcp_only remedy wording are gone.
  • First Principles passed on every later head (bbccff65, fab3f9c0, fa84516f7800300ba3f6dc221ceb7c93c3a065d7). Its remaining note (prove_tcp_peer and ChallengeRefused exported with no outside importer) is non-blocking. I left it alone because the owner has paused further changes to the TCP-only path until a scope review finishes.

Facade guard: test/test_facade_mirror_guards.py passes locally on the rebased branch (as part of the 136 passed, 5 skipped run at -n 2), and the CI backend shards that run it are green on fa84516f7800300ba3f6dc221ceb7c93c3a065d7.

@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at fa84516 for a human reviewer.

Thank you for opening #26301 and for the explanation. My earlier issue request is cleared: #26301 covers the other sandboxed senders, and the TCP-only proof is now built in this PR, so it needs no issue.

I found no defect. The proof is an HMAC keyed with the internal secret over a domain tag, a fresh 256-bit nonce and the connection's kernel-reported endpoints, compared in constant time (loopback_peer_challenge.py). /api/health answers it only for a direct-local caller with a served Host (core.py L353-L394), so a DNS-rebound page still gets only {"ok": true}. Every lane passes and every check is green on this head, including the namespace-sandbox and Seatbelt steps.

Why it is held: this PR changes which peer is trusted to receive the gateway's internal secret, and it adds a secret-keyed response to a network route. Our review policy reserves that for a maintainer. You also noted the owner paused the TCP-only path pending a scope review. A maintainer's sign-off clears this hold.

— crew-pr-reviewer (r138)

@bolichen97

Copy link
Copy Markdown
Contributor

Thanks for this. One follow-up is needed before this can move. The deferred work points at #23270, which is closed, so two follow-ups have no open issue yet. Could you open one issue for each and link both in the description?

  1. Move the other sandboxed senders off plain loopback_urlopen, so they prove the gateway peer the same way the two Code Review Sage senders now do.
  2. Verify a TCP-only gateway from the Linux sandbox. Today the PR refuses Sage reviews on a host whose data home path is too long for a unix socket.

@iamwhatever, once those issues are linked, could you take another look? That is the point your change request asked for.

A maintainer will still need to sign off on the security change itself after that. At fa84516 all 102 checks pass.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer decision (security sign-off): not adopted. The maintainer's rule for security changes is that we do not raise security strictness, so the maintainer will not add this restriction: Code Review Sage senders proving the gateway peer (owner-verified socket or the new loopback challenge-response) before every secret-bearing send.

Reviewed at fa84516. As I read it, almost all of this PR is that new check: the _verified_urlopen path in review_driver.py, the new loopback_peer_challenge.py and the challenge reply in api_health, the peer-proof tests, and the two CI steps that run them. The existing proxy- and redirect-disabled loopback opener is already on main, so moving it to sage_lib/loopback.py on its own does not change behavior.

Please drop the peer-proof parts. If something that is not a tightening remains that you want to keep, push that alone and I will review it; otherwise please close the PR. Thank you for the careful work and the thorough sandbox tests.

@JLargent42

Copy link
Copy Markdown
Contributor Author

Closing per the maintainer security sign-off (not adopted: we do not raise security strictness). Thanks @bolichen97 for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:macos Run the on-demand macOS suite against this PR head merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants