Skip to content

feat(instances): fetch a crew's session back to the hub - #19440

Open
koenvincent wants to merge 2 commits into
kirodotdev:mainfrom
koenvincent:feat/instances-fetch-session
Open

koenvincent wants to merge 2 commits into
kirodotdev:mainfrom
koenvincent:feat/instances-fetch-session

Conversation

@koenvincent

@koenvincent koenvincent commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Let the hub copy one of a connected crew's sessions back to itself, with full context when the crew allows it.

Send session already copies a session hub -> crew over the open tunnel, Layer B included. The opposite direction has no route: to bring a session a crew ran back to the hub, the owner has to open the crew's pane, export the session to a file, download it, and import that file on the hub. docs/system-specs/modules/instances.md §14.6 names a hub-initiated pull as the natural follow-on for exactly this reason: a crew usually cannot reach back to the hub.

Why it matters

Work that starts on a crew and has to continue on the owner's machine (or the other way round, then back) is a round trip today with a file in the middle, and the file step is easy to get wrong: the download lands in the browser's folder, and the import has to happen in the right dashboard. A pull over the tunnel the hub already holds makes the return trip one request, through the same arrival path every other transfer uses.

Not a goal

  • No UI in this PR. The crew-window action that calls the route is tracked in Crew window: "Copy to this machine" action that calls fetch-session #20548 and opens as its own PR once this lands; this PR is the server route and its contract.
  • No change to what the crew's export carries, or to who may ask for Layer B. The crew's own export_include_layer_b opt-in and owner check decide it, unchanged.
  • No new transport and no new token-crossing route.
  • Chained crews (§17) are not special-cased: the pull rides whatever forward proxy_request already uses for that crew.

What changed (motivation → approach → change)

  • Approach: reuse the two pieces that already exist instead of a second transfer implementation. The read is SshTunnelManager.proxy_request (manager-held cookie, one re-mint on 401/403, allow_redirects=False) against the crew's own GET /api/chat/slots/{slot}/export?include_layer_b=true, which is inside the proxy's api/chat grant. The write is the one arrival path, _install_arrived_bundle. The browser-facing proxy route is not used: it forwards JSON and SSE only and would refuse the gzip export.
  • session_transfer_intake.py / session_transfer_install.py (were session_transfer.py before refactor(dashboard): split session_transfer and handlers_instances #21111): _read_bundle_body and _install_arrived_bundle take an optional source, anything with a content stream. It defaults to the request, so the existing import route is unchanged. A new install_peer_export(request, source) runs the arrival with the crew's live response as the body. Format sniff, upload and expansion admissions, memory admission, redaction, arrival folders and copy semantics are the same code.
  • handlers_instances_peer.py (was handlers_instances.py before refactor(dashboard): split session_transfer and handlers_instances #21111; the facade serves it as handlers_instances.api_instances_fetch_session): POST /api/instances/{id}/fetch-session with body {"slot": "<the crew's slot key>"}. Owner-only through _guard. The slot key must match _FETCH_SLOT_KEY_RE and is percent-quoted into the path, so it cannot leave the export segment. A crew 404 that carries the export's own export_slot_not_found becomes 404 transfer_slot_not_found; a bare 404 (a crew without the route) is a refusal, not a missing session. Any other crew refusal becomes 502 transfer_peer_refused with peer_status, which already tells "never exportable" (400) from "try again" (503) and "failed" (500). Nothing of the crew's body is relayed: it is peer text, and at most 4 KiB of a 404 is read only to check for export_slot_not_found. Proxy failures keep their own codes (503 for not connected or no credential, 502 otherwise). Audited as instances_fetch_session.
  • Sender name (declared): a fetched copy is filed and titled under the hub's own registry name for that crew (Imported / from <crew name>, (from <crew name>)), not under any origin in the crew's bytes. install_peer_export passes it as origin_override, and only this route sets it; every other arrival keeps the origin from its own bundle. The crew's export leaves origin empty, and the crew's bytes are untrusted, so the hub's name is the one a reader can rely on.
  • A bad bundle is the crew's fault (declared): the arrival answers 400 for a bundle it cannot read (transfer_body_unreadable, transfer_invalid_gzip, transfer_invalid_json, a validation code). Here that bundle is the crew's reply, not the caller's request, so the route re-answers it as 502 and keeps the arrival's code (_peer_bundle_failure). The caller's own body (the slot key) is still a 400.
  • Layer B: the hub asks, the crew decides. The crew's export includes it only when its operator enabled dashboard.export_include_layer_b and the request is the owner's. A minted token's subject is the crew's owner, so the pull qualifies; test_the_hubs_minted_credential_reaches_the_crews_export_as_its_owner drives that chain (see Tests). Otherwise the arrival is transcript-only and the import's resume_mode says so.
  • Route: registered next to send-session, before the catch-all proxy route.
  • Docs: instances.md §14.4 gets the route row and its contract; §14.6 no longer calls the pull a follow-on.

Backwards compatibility

Compatible: one new route. The import route reads its own request body exactly as before, because source defaults to the request.

Removes nothing: the diff adds a route and an optional parameter, and changes no existing contract.

Tests

In test/test_session_transfer_fetch.py (its own file since #22132 split the old suite by topic):

  • test_fetch_session_installs_the_peers_export_as_a_new_local_slot: the hub asks the crew's export route with include_layer_b=true, and a gzipped export arrives as one new local slot.
  • test_fetch_session_refuses_a_slot_key_that_could_leave_the_export_path: ../status, chat/1, empty, over-long and %2e%2e are refused with 400 and never reach the crew.
  • test_fetch_session_names_the_sender_from_the_hub_not_the_crews_bytes: a crew-supplied origin does not choose the folder or the title.
  • test_fetch_session_maps_a_missing_peer_session_to_404: only with the export's own export_slot_not_found; no slot is created.
  • test_fetch_session_does_not_read_a_crew_without_the_route_as_a_missing_session: a bare 404 is 502 transfer_peer_refused with peer_status: 404 and nothing else.
  • test_fetch_session_relays_the_status_of_an_export_refusal_and_nothing_else: each of the export's refusals arrives as its peer_status only.
  • test_fetch_session_maps_only_the_exports_own_404_to_a_missing_session: a 404 with another code, an over-long body or non-JSON is a refusal, not a missing session.
  • test_fetch_session_blames_the_peer_not_the_caller_for_a_bad_bundle: a corrupt gzip or malformed export from the crew is a 502 keeping the arrival's code.
  • test_fetch_session_relays_only_the_status_of_a_peer_refusal: a credential-shaped string in the crew's refusal body does not appear in the response.
  • test_fetch_session_reports_a_peer_that_is_not_connected: 503 proxy_peer_not_connected.
  • test_fetch_session_is_owner_only: an app token is refused before the crew is asked anything.
  • test_the_hubs_minted_credential_reaches_the_crews_export_as_its_owner: the premise the Layer B claim rests on, with no stub between the steps. The crew's real /api/token/local mint (what kirocrew token calls), the manager's link exchange (GET /api/status?token=, keeping the port-scoped cookie) and the export request with that cookie through the crew's real token_auth_middleware; the export's own gate, is_owner_dashboard_request, answers true. Covered for both owner shapes: a configured owner id, and none (local-app). test_session_export.py::test_an_export_carries_layer_b_on_explicit_opt_in covers the other half (owner + flag + operator setting -> Layer B).

Mutation check: dropping the source pass-through in _read_bundle_body fails the arrival test, and dropping the slot-key check fails the traversal test. Minting for any other subject fails the premise test (both cases); treating any 404 as a missing session, reading a 404 past the cap, and relaying any field beyond peer_status each fail their test.

Run locally: test/test_session_transfer.py (262 passed), test/test_dashboard_route_table.py and test/test_dashboard_server_composition_contract.py (113 passed), test/test_error_code_contract.py, test/test_instances.py and test/test_remote_crew_peer_slots.py (528 passed). Five test_instances.py forwarder-identity tests fail the same way on unmodified main in my environment, because the sandbox cannot read process start-time identity; they are unrelated to this diff. scripts/check_feature_map.py against origin/main passes, and black is clean.

After the rebase onto main d06b611 (head 443ff30): the only conflict was the route block in routes/connections.py, where main added crew-turn at the same spot; both routes are kept. Re-run: test_session_transfer.py, the route-table and composition-contract tests (120 passed), feature-map gate clean.

After the rebase onto main 75726a8 (head efc7f4a): the only conflict was the feature-map row, where main renamed pages/ChatSidebar.tsx to pages/chat-sidebar/SessionRow.tsx; main's row is kept with this PR's three additions. Re-run: test_session_transfer.py, test_error_code_contract.py, the route-table and composition-contract tests (389 passed), test_remote_crew_peer_slots.py (45 passed), feature-map gate, black, flake8 and mypy clean.

Head d625163 (same base): peer_code and its allowlist are removed (First Principles item 3, no reader); a refusal carries peer_status only. Re-run: test_session_transfer.py, the test_instances* files and the three error-code contract tests (890 passed; the 5 failures are forwarder-process tests that fail identically on efc7f4a in this sandbox and are untouched), black, isort, flake8 and mypy clean.

After the rebase onto main 5d1a459 (head 43fb15c, now 1bb5ee6): #21111 split session_transfer.py and handlers_instances.py into owners. Nothing in the behaviour changed. The changes moved with the code: source on _stalling_chunks/_stream_request_to_file/_read_bundle_body is now in session_transfer_intake.py, install_peer_export and the _install_arrived_bundle keywords are in session_transfer_install.py, and the route with _FETCH_SLOT_KEY_RE, _peer_slot_is_missing and _peer_bundle_failure is in handlers_instances_peer.py ("the routes that read a peer", per the facade's owner list). All of them are in the facades' TYPE_CHECKING lists. Re-run: test_session_transfer.py, test_handlers_instances* and the facade tests (3444 passed; the only 6 failures are in test_pod_runtime_refactor_facade.py, which my sandbox refuses a scratch directory for, and which this PR doesn't touch), plus mypy, black, flake8, docs-lint and the feature-map gate. At 1bb5ee6, _peer_slot_is_missing reads the crew's 404 through the existing read_capped_response instead of its own chunk loop (First Principles blocker on 43fb15c). Removing the cap check turns 2 fetch tests red.

After the rebase onto main 453cc4b (head 3eb7987): #22132 split test/test_session_transfer.py by topic, so this PR's fetch-session section, its helpers and the credential-premise tests moved unchanged into a new test/test_session_transfer_fetch.py; it imports the shared helpers from session_transfer_helpers and keeps its own copy of the six-line _filed_folder. No source change. Re-run: the test_session_transfer* and test_handlers_instances* files and the test_instances* files (898 passed); 5 failures in test_instances_orphan_reclaim.py and test_instances_forwarder_hints.py, which this PR does not touch, need a readable process start time that this sandbox does not give. mypy, flake8, docs lint and the feature-map check are clean.

Rebased onto main f0d0cd2 (head 15e419d) because main changed the feature-map row this PR extends; the row keeps main's text plus our three insertions, and nothing else moved. Before that: rebased onto main f62ab4f (head 3e6a734) only to pick up #22433: the run on 3eb7987 failed in test_chat_follow_default_slot_kinds.py, which main had broken by moving _import_thread_to_slot and fixed in #22433. No conflicts and no change to this PR's patch; the fetch tests, test_session_transfer.py, that slot-kinds file and the test_handlers_instances* files (85) pass, as do mypy, flake8, docs lint and the feature-map gate.

Manual verification

Not yet run against a live crew. The credential premise is now covered in-process (see Tests); what only a live run adds is the real tunnel and the crew's real export bytes. Planned, and posted on this PR when run: connect a crew, call the route for one of its sessions with the crew's export_include_layer_b on, and check that the arrival's resume_mode reports a full resume and that the session continues with its context. Whether that run gates the merge is the open maintainer decision.

Related Issues

Follow-on named in docs/system-specs/modules/instances.md §14.6.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

Live run: the two-gateway run is tracked in #30231, so it can be deferred without holding this PR.

@koenvincent
koenvincent requested review from a team and KiroCrewCaptain as code owners October 12, 2026 13:42
@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 12, 2026
@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 8e56a0099b03d46df97b6b478aa73bedfa8593c7 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 8e56a0099b03d46df97b6b478aa73bedfa8593c7 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns: the pull reuses the existing proxy_request read and the single arrival path. Nothing is taken away and no UI is added.

Notes

[DESIGN-REVIEWED] 8e56a00

Description read

[DESCRIPTION-READ] c92f96873d6f5bb8b4eb654c27560ecc61b01c691eef10405172e0a82a68ac72

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5, fork) — ✅ PASS

Premise-level review of 8e56a0099b03d46df97b6b478aa73bedfa8593c7 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

The pull route ships with no caller in this repo yet. That is a reversible, declared deferral, not a blocker.

Notes

What this change ships

Inventory (7 items) — 6 justified

Intent: let the owner copy a crew's session back to the hub in one request instead of a manual export/download/import round trip. ADDITION.

  1. New owner-only POST /api/instances/{id}/fetch-session pulls a crew session over the tunnel — zero consumers (UI deferred to Crew window: "Copy to this machine" action that calls fetch-session #20548)
  2. A fetched copy is filed and titled under the hub's own registry name for the crew — justified
  3. A corrupt or malformed bundle from the crew answers 502 with the arrival's code instead of 400 — justified
  4. Only the export's own export_slot_not_found 404 maps to transfer_slot_not_found; any other crew refusal relays its status only — justified
  5. The arrival path accepts an optional body source, defaulting to the request, so import is unchanged — justified
  6. Layer B is requested and the crew's existing opt-in decides it — justified
  7. Feature-map row and instances.md §14.4/§14.6 describe the pull — justified

[FIRST-PRINCIPLES-REVIEWED] 8e56a00

Description read

[DESCRIPTION-READ] c6bb5cc310c975006d39496b27a44971de744372fba6f5e143df342a6bf1b318

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 8e56a0099b03d46df97b6b478aa73bedfa8593c7 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 8e56a00

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 8e56a0099b03d46df97b6b478aa73bedfa8593c7 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

Notes:

  • handlers_instances_peer.py:154: _FETCH_SLOT_KEY_RE.match with a trailing $ lets through a key that ends in \n. quote turns it into %0A, so it stays inside the export segment. The only effect is a 404/502 where a 400 was meant; fullmatch would fix it.

[OPUS-REVIEWED] 8e56a00

@koenvincent

Copy link
Copy Markdown
Contributor Author

Addressing the review lanes on a6d60f9c8:

  • GPT 6.1 (blocking, feature map): fixed in 3d1b2ca8e. The crew-window row in docs/feature-map/README.md now names the copy and lists POST /api/instances/{id}/fetch-session and install_peer_export. scripts/check_feature_map.py against origin/main passes and test/test_check_feature_map.py is green.
  • First Principles (zero consumers): the consumer exists and is kept out of this PR on purpose, so the server contract can be reviewed on its own. It is a Copy to this machine action in the crew window, on koenvincent:feat/crew-window-copy-here, stacked on this branch: CrewChatWindow.tsx, the client call, six strings in every catalog, three vitest cases, and the user-guide paragraph in remote-crew.md. I will open it as soon as this one lands, with screenshots. If you would rather review both as one PR, say so and I will fold it in here.
  • Design review (live check): agreed. The tests stand in for proxy_request. Before asking for a merge I will pull a session from a crew connected over SSM with the crew's export_include_layer_b on, and post the arrival's resume_mode and message count here.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 12, 2026
@koenvincent

Copy link
Copy Markdown
Contributor Author

CI on 3d1b2ca8e: the only failing test is test/test_provider_spawn_off_loop_18172.py::TestBoundsAndCleanupArePreserved::test_cancellation_during_the_spawn_still_reaps, on both Ubuntu and macOS. That is the known failure tracked in #19451 and #19310, with a fix in #19358. It fails the same way on an unmodified origin/main (c4c51aac5) locally, and this diff touches no spawn code. Coverage Gate fails closed because that shard is red. I will re-run once #19358 lands.

@koenvincent

Copy link
Copy Markdown
Contributor Author

Update on the live check I promised: I could not run this branch as a hub on my machine yet. A pod for the worktree provisions now, after a venv workaround (the bundled Python has no ensurepip), but bootstrapping the pod's launchd agent fails with Bootstrap failed: 5: Input/output error, so no pre-release hub is reachable here.

Unless a reviewer can run it sooner, I will do the check on the first insider build that carries this route: pull a session from a crew connected over SSM, with the crew's export_include_layer_b on, and post the arrival's resume_mode and message count here.

@github-actions github-actions Bot added the merge conflict Branch has merge conflicts with its base — author must resolve before merge label Oct 12, 2026
@koenvincent
koenvincent force-pushed the feat/instances-fetch-session branch from 3d1b2ca to 98d21e0 Compare October 12, 2026 17:43
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 12, 2026
@koenvincent
koenvincent force-pushed the feat/instances-fetch-session branch from 98d21e0 to 754c15f Compare October 12, 2026 18:44
@koenvincent

Copy link
Copy Markdown
Contributor Author

@koenvincent

Copy link
Copy Markdown
Contributor Author
  • span=8660f916dc01 — accepted: peer_code and its allowlist are removed in d625163

    Right: nothing read peer_code, and peer_status already carries the distinction it was added for. The export answers "never exportable" with 400 (export_slot_not_persistent, export_bundle_empty), "try again" with 503 (export_snapshot_unstable) and "failed" with 500 (export_failed), so a caller needs only the status.

    Removed: _PEER_EXPORT_CODES, _peer_refusal_code and both peer_code branches. A refusal is now always {error, code: transfer_peer_refused, peer_status}. What stays is item 4's single check, _peer_slot_is_missing. It reads a 404 under the same 4 KiB cap and compares its code with export_slot_not_found. I kept the cap, because without it that one comparison would buffer an unbounded peer body.

    Tests: the refusal test now asserts the exact three-field body for all four export refusals. The allowlist test became test_fetch_session_maps_only_the_exports_own_404_to_a_missing_session: a 404 with another code, with export_slot_not_found past the cap, or with non-JSON is a 502 refusal. Each of these mutations fails a test: treating any 404 as missing, dropping the cap, and adding a field beyond peer_status. §14.4 of instances.md and the description say the same.

@koenvincent

Copy link
Copy Markdown
Contributor Author

Thanks for r101. Head is now d625163, on the same base (main 75726a8) and still 2 commits, both signed by GitHub.

3. First Principles on efc7f4a. One disposition record per item, above:

4. Design: does aiohttp gunzip the export on the way in? No. I checked it rather than reasoning about it. I served a staged gzip through the crew's real _StagedExport and read it back with a default aiohttp.ClientSession, which is what proxy_request opens:

  • the request did carry Accept-Encoding: gzip, deflate;
  • the response carried no Content-Encoding, only Content-Type: application/gzip from session_export.py#L590;
  • the body arrived starting 1f8b, byte-identical to the staged file.

aiohttp's client inflates only on a Content-Encoding header. The crew's gateway never sets one: there is no enable_compression anywhere in src/kiro_crew/dashboard, and _StagedExport is a plain StreamResponse that writes the file's bytes. So the arrival sniffs the real gzip, as the stubbed tests assume.

Button. It stays out of this PR unless a maintainer says otherwise. It is tracked in #20548.

Still open, for maintainers:

  • The two rulings: API-first or fold in the button; merge on unit coverage or wait for the live resume_mode run. If the live check is deferred, I will track it in an issue.
  • A human review.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Oct 15, 2026
@koenvincent

Copy link
Copy Markdown
Contributor Author

Rebased onto main 9a77e12; new head 1848038.

The red backend shards on d625163 (Linux 3.12 (1), Windows (1), macOS (1)) were one test of this PR: test_the_hubs_minted_credential_reaches_the_crews_export_as_its_owner. It stubbed member_memory_auth.local_owner_bootstrap_allowed, but #20368 moved api_token_local to local_owner_bootstrap_provenance, so on the merge ref the real gate ran and refused (member_owner_token_refused). The test now stubs local_owner_bootstrap_provenance with an empty LocalOwnerProvenance(); no production code changed. On the new base the old stub reproduces the 2 failures and the new one passes (13 fetch-session tests green, black and flake8 clean).

Windows (2) failed on an unrelated test: an xdist worker crashed on a timeout in test_spawn_delete_queued.py::test_an_internal_parent_deletes_its_own_queued_spawn. This PR touches no spawn code; the push re-runs it.

@koenvincent

Copy link
Copy Markdown
Contributor Author

Rebased onto main 83e97cd (new head c873dca) after it went CONFLICTING again: only the crew-window row in the feature map, where main had moved the sidebar file paths. I kept main's row and re-added the three fetch-session additions; the feature-map gate passes and 269 session-transfer tests are green.

The one red on 1848038, Backend Tests (Windows) (7), was test_managed_agent_shadow.py reaching the runner's real data home at teardown, unrelated to this PR; filed as #20627.

@koenvincent

Copy link
Copy Markdown
Contributor Author

CI on c873dca: the only failing test is test_agent_scratch.py::TestGitTreesAreQuarantined::test_a_failed_move_keeps_the_tree_in_place_and_copies_nothing[18] (FileNotFoundError on git's transient .git/objects/maintenance.lock), the flake tracked in #20325; 22581 passed in that shard. Coverage Gate and PR Readiness follow from it. This PR doesn't touch that area. Could a maintainer re-run Backend Tests (3.12, 8)?

@koenvincent

Copy link
Copy Markdown
Contributor Author

Live check against a real crew (as asked in the HOLD), with this branch's hub code at c873dca:

  • Crew side, real gateway over the real tunnel: the owner token was minted with the crew's own kirocrew token, exchanged with one GET /api/status?token= that keeps the session cookie (the hub manager's link exchange), then GET /api/chat/slots/<key>/export?include_layer_b=true on a 2-message persistent slot. The reply was 200, application/gzip, with no Content-Encoding, 42,964 bytes, and the bundle carries layer_b (with envelope). Without the flag the reply was 328 bytes, with layer_b_skipped and no layer_b.
  • Hub side: those exact bytes, fed as the peer's reply into api_instances_fetch_session (only the proxy stubbed, so the arrival path is real), gave 200, resume_mode: session_load. The copy is filed under from <crew> with (from <crew>) in the title and no "transcript only" marker, and both messages arrived. The peer was asked exactly once: GET api/chat/slots/<key>/export with include_layer_b=true.
  • Not covered: the route served by a running hub process with its own tunnel. That run needs a full build, so it follows with the first insider build that carries this.

@koenvincent

Copy link
Copy Markdown
Contributor Author

@iamwhatever friendly ping for a re-review when you have a moment. Since your HOLD: the live check against a real crew is posted above, the branch is rebased on main (head d50168afa, all checks green apart from PR Readiness), and the one open choice is yours: API-only here with the button in #20548 as a follow-up, or the button in this PR.

@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 1bb5ee6.

The code still looks right after the #21111 split: the owner _guard runs before any peer call, the slot key is checked and percent-quoted, a crew refusal relays only peer_status, a bad crew bundle becomes a 502, and the sender name comes from the hub's registry (

async def api_instances_fetch_session(request: web.Request) -> web.Response:
"""POST /api/instances/{id}/fetch-session — copy a peer's session HERE.
Body: ``{ "slot": "<the peer's slot key>" }``.
The pull counterpart of :func:`api_instances_send_session`. The hub asks the
peer's own export route for the session over the already-open tunnel, with
the manager-held credential, and installs the reply through the one local
arrival path (``install_peer_export``). Nothing on the peer changes: its
export is a read, and the arrival is a NEW local slot (copy semantics).
Layer B is asked for (``include_layer_b=true``) and the PEER decides: its
export carries the context window only when its own operator enabled
``dashboard.export_include_layer_b`` and the request is the owner's, which
the manager-held credential is (a minted token's subject is the peer's
owner). Otherwise the copy is transcript-only and the answer says so through
``resume_mode``, as an import from a file does.
Owner-only, like the proxy: the request runs on the peer with the owner's
credential, so a non-owner dashboard subject must not be able to pull a
peer transcript through it.
"""
# ``_guard`` is the owner-only gate (see its comment); nothing weaker passes.
denied = _guard(request, "fetch_session")
if denied is not None:
return denied
state: DashboardState = request.app["state"]
instance_id = request.match_info["id"]
inst = await asyncio.to_thread(_registry(state).get, instance_id)
if inst is None:
_audit("fetch_session", "denied", request_id=instance_id, error="not found")
return web.json_response({"error": "not found", "code": "instance_not_found"}, status=404)
mgr = getattr(state, "instances_manager", None)
if mgr is None:
_audit("fetch_session", "denied", request_id=instance_id, error="manager unavailable")
return web.json_response(
{"error": "instances manager not running", "code": "instances_manager_down"},
status=503,
)
try:
body = await request.json()
except Exception:
return web.json_response(
{"error": "invalid JSON body", "code": "transfer_invalid_json"}, status=400
)
slot_key = body.get("slot") if isinstance(body, dict) else None
if not isinstance(slot_key, str) or not _FETCH_SLOT_KEY_RE.match(slot_key):
return web.json_response(
{"error": "slot must be a crew session key", "code": "transfer_missing_slot"},
status=400,
)
path = f"api/chat/slots/{quote(slot_key, safe='')}/export"
try:
async with mgr.proxy_request(
instance_id, "GET", path, params={"include_layer_b": "true"}
) as upstream:
if upstream.status != 200:
if upstream.status == 404 and await _peer_slot_is_missing(upstream):
_audit(
"fetch_session", "denied", request_id=instance_id, error="peer slot missing"
)
return web.json_response(
{
"error": "session not found on the crew",
"code": "transfer_slot_not_found",
},
status=404,
)
# The peer's own refusal (a non-persistent session, a mid-turn
# snapshot it could not take), or a 404 without the export's own
# code: a crew with no such route, which is not a missing
# session. The body is peer text, so only the status is relayed.
# The export's statuses already separate "never exportable"
# (400) from "try again" (503) and "failed" (500).
_audit(
"fetch_session",
"failure",
request_id=instance_id,
error=f"peer export HTTP {upstream.status}",
)
return web.json_response(
{
"error": f"the crew refused the export (HTTP {upstream.status})",
"code": "transfer_peer_refused",
"peer_status": upstream.status,
},
status=502,
)
# The sender is named from OUR registry entry, never from the crew's
# bytes: its export leaves ``origin`` empty.
response = await install_peer_export(
request, upstream, origin=str(getattr(inst, "name", "") or instance_id)
)
if response.status == 400:
# Every 400 the arrival answers is about the BUNDLE, and here the
# bundle is the peer's reply, not the owner's request (whose own
# body was validated above). A tunnel that dropped mid-stream
# (``transfer_body_unreadable``), a corrupt gzip or a malformed
# export is a peer or proxy failure, so it is a 502 that keeps the
# arrival's code, never a 400 that blames the caller.
response = _peer_bundle_failure(response)
except ProxyRequestError as e:
_audit("fetch_session", "failure", request_id=instance_id, error=e.code)
if e.http_status == 503:
return web.json_response({"error": e.message, "code": e.code}, status=503)
return web.json_response({"error": e.message, "code": e.code}, status=502)
_audit(
"fetch_session",
"success" if 200 <= response.status < 300 else "failure",
request_id=instance_id,
)
return response
). Thanks for the live crew check. Checks are green.

What blocks approval:

  1. Human review is required by policy: this is a new credentialed hub-to-crew read route that asks for Layer B (
    instance_id, "GET", path, params={"include_layer_b": "true"}
    ).
  2. A maintainer still has to choose API-first with Crew window: "Copy to this machine" action that calls fetch-session #20548 as the follow-up, or fold in the button. The First Principles zero-caller item on this head clears on that ruling or on a target=first-principles disposition for 1bb5ee6.
  3. Your live check did not cover a running hub serving the route over its own tunnel, and you said that follows with the first insider build. If a maintainer accepts that, please open an issue for it and link it in the description.

No PR code was run locally.

— crew-pr-reviewer (r112)

@koenvincent

Copy link
Copy Markdown
Contributor Author

GPT 6.1, fetch rejects valid peer session keys (handlers_instances_peer.py:60): that's right, and it's fixed on a4c8ca76c2. _FETCH_SLOT_KEY_RE now accepts every key _normalize_slot_key keeps ([\w.-], leading _ or . included, up to 249 characters) plus the : of a channel key, and still refuses /, %, an empty key and a bare . or ..; the key is still percent-quoted. A new test fetches _notes, .hidden, a 201- and a 249-character key and a slack: key through to the peer path, and pins the 249 to _SLOT_KEY_MAX_CHARS. The branch is also rebased on 85a081707c.

@koenvincent

Copy link
Copy Markdown
Contributor Author

Rebased onto main 7be0ae4029 and fixed the two real reds on a4c8ca76c2. New head d66d27fa94, still 2 commits.

  • Opus 5.5 blocker (503 without the restart hint): api_instances_fetch_session now returns _manager_unavailable_response() when the instances manager is not running, like send-session and every other /api/instances/{id}/* route. The hand-written instances_manager_down body is gone. New test test_fetch_session_answers_a_stopped_manager_like_every_instances_route pins the shared code and message and that the peer is never asked; it fails against the old code.
  • Backend shard 2 (test_every_mirrored_facade_manifest_names_what_it_forwards): the handlers_instances facade forwards _FETCH_SLOT_KEY_MAX_CHARS from handlers_instances_peer, and its manifest now names it. The guard fails without the entry and passes with it.
  • Integration (test_a_second_boot_touches_only_known_module_globals): not from this PR; it was Flaky: test_a_second_boot_touches_only_known_module_globals: governance._CONFIRM_PATTERNS_MEMO (from #22511) is not reset between boots #22676, which main has fixed, and the rebase picks that up.

Checked locally: test_session_transfer_fetch.py + test_session_transfer_landing.py (47), test_facade_mirror_guards.py (70), flake8, mypy on the module, and the line-cap, comment-history and feature-map gates.

@iamwhatever

Copy link
Copy Markdown
Contributor

HOLD at 1691911. A human review is required, and one maintainer decision is still open.

The code still looks sound. The owner _guard runs before any crew call. The slot key is checked and percent-quoted, and a crew refusal relays only peer_status. A bad crew bundle becomes a 502, and a stopped manager now answers like the other instances routes (handlers_instances_peer.py#L114-L224). The fetched body uses the existing streaming intake, so it adds no new size path. Every lane passes with no findings, and the UI follow-up is tracked in #20548. CI is green: 100 checks pass or are skipped, plus PR Readiness.

What blocks approval:

  1. Policy reserves this PR for a human reviewer: it adds a network route that reads a crew's export with the hub's owner credential and asks for Layer B.
  2. Whether merging needs the live run first (a hub serving the route over its own tunnel against an export_include_layer_b crew) is still an open maintainer decision, as the description says. If a maintainer defers it, please open an issue for that run and link it in the description.

Reviewed head: 1691911

— crew-pr-reviewer (r121)

POST /api/instances/{id}/fetch-session is the pull counterpart of
send-session. The hub reads the peer's own export route over the open
tunnel (proxy_request: manager-held cookie, one re-mint, no redirects)
and installs the reply through the one arrival path, so the format
sniff, admissions, redaction, arrival folders and copy semantics are the
same code a pushed or file-imported session meets.

Layer B is requested and the peer decides it: its export carries the
context window only when its operator enabled export_include_layer_b and
the request is the owner's, which a minted token's subject is.

The peer slot key is validated and percent-quoted into the path; a peer
refusal relays its status only, never its body. Owner-only via _guard.
@koenvincent

Copy link
Copy Markdown
Contributor Author

On the open maintainer decision from r121 (does merging need the live run of a hub serving fetch-session over its own tunnel first):

I tried to run it today on an isolated Linux host with this branch at 3593981a7e, hub and peer as two separate gateways. It could not start there: kirocrew pod needs a systemd user bus, and the only single-host transport is ssh to localhost, which that host's sandbox refuses, as it refuses minting the owner token. The 10-15 check above did cover the crew side over a real tunnel.

So I would ask a maintainer to defer the full hub-side run. If you agree, I will open an issue for it and link it in the description. That leaves only the human review the policy reserves for this route. @chenmingwei23 could you take it?

@koenvincent

Copy link
Copy Markdown
Contributor Author

Opened #30231 for the two-gateway run and linked it in the description, so a maintainer can defer it there. The rest of this PR waits only on human review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor) merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants