diff --git a/docs/feature-map/README.md b/docs/feature-map/README.md index f6c646be840b..b0296b80fd32 100644 --- a/docs/feature-map/README.md +++ b/docs/feature-map/README.md @@ -82,7 +82,7 @@ this area is reached from inside it unless stated otherwise. | Feature | What it is | Reach it | Page | Handler | Endpoints | |---|---|---|---|---|---| | Shared Crew backend | Fair-admission capacity and accepted work, collaboration state and capability reporting | API only: `/api/shared-crew/*` | None (backend only) | `shared_crew/routes_admission.py`, `shared_crew/routes_collab.py`, `shared_crew/routes_capabilities.py` | `GET /api/shared-crew/capacity`, `GET /api/shared-crew/work`, `POST /api/shared-crew/work/{work_id}/cancel`, `GET /api/shared-crew/presence`, `GET /api/shared-crew/capabilities` | -| Redaction cards | A removed credential renders as a lock tag and a suspicious URL as a Blocked link chip; each opens a card explaining the removal, naming the credential's source (Open in Terminal pre-fills without running) or showing the link's full address (Copy URL, Open once, Allow for this host). Resting the mouse on a tag or chip, or focusing it from the keyboard, shows a read-only preview stating the rule in plain words; every action stays on the card. No key is taken, a focus-opened preview stays under a passing mouse, the preview closes when its marker scrolls out of view, and a tap opens the card directly. The first reply with removed values in a session carries a one-time coach. Allowed hosts are listed and revoked in Settings → Security → Redaction | Inside an assistant reply that had a value or link removed | `components/RedactionCards.tsx`, `pages/settings/SecurityPanel.tsx` (the Redaction section: `SecurityPanelRedaction.tsx`) | `handlers/redaction.py` | `GET,POST,DELETE /api/redaction/allowed-hosts` | +| Redaction cards | A removed credential renders as a lock tag and a suspicious URL as a Blocked link chip; each opens a card explaining the removal, naming the credential's source (Open in Terminal pre-fills without running) or showing the link's full address (Copy URL, Open once, Allow for this host). Resting the mouse on a tag or chip, or focusing it from the keyboard, shows a read-only preview stating the rule in plain words; every action stays on the card. No key is taken, a focus-opened preview stays under a passing mouse, the preview closes when its marker scrolls out of view, and a tap opens the card directly. The first reply with removed values in a session carries a one-time coach. The credential card offers the owner Show / Copy / Hide for the removed value while the gateway holds it in memory (owner browser only; every reveal audited; gone after a restart or 24 h). Allowed hosts are listed and revoked in Settings → Security → Redaction | Inside an assistant reply that had a value or link removed | `components/RedactionCards.tsx` (shared controls: `components/redactionPrimitives.tsx`; owner reveal: `components/RedactedValueReveal.tsx`), `pages/settings/SecurityPanel.tsx` (the Redaction section: `SecurityPanelRedaction.tsx`) | `handlers/redaction.py`, `handlers/redacted_reveal.py` (store: `security/reveal_store.py`) | `GET,POST,DELETE /api/redaction/allowed-hosts`, `POST /api/security/redacted-value` | | Remote media gate | A remote image, video or audio in an assistant reply loads only when the reader clicks it, so rendering never fetches an agent-chosen URL; media pointing at the gateway's own routes is refused | Inside an assistant reply that embeds remote media | `components/markdown/remoteMedia.tsx`, `components/markdown/ImgWithFallback.tsx` (composed by `components/MarkdownRenderer.tsx`) | — | — | | Loaded skill card | A skill selected through the `$` picker appears as a collapsed transcript card with a book icon. A one-skill card expands to its detail line and capped body. A card with several skills expands to one detail line and one row per skill; each saved body's native row button opens only that skill's keyboard-scrollable capped body, so every other row header stays reachable. A structured skill with an empty body stays non-interactive and states that the skill has no instructions. Each saved body has a bottom fade that clears at the scroll end, and compact headings keep skill instructions below assistant answers in the transcript hierarchy. Name-only rows from older gateways remain non-interactive and state that instructions were not recorded for older turns | Session transcript → composer `$` picker → select and send a skill | `pages/chat/SkillLoadCard.tsx`, `pages/chat/MarkdownDisclosureCard.tsx` (shared disclosure shell with `pages/chat/CompactionCard.tsx`), `pages/chat/transcriptRenderers.tsx`, `app-sdk/messageRenderers.tsx` | `chat_turn/prompt_assembly.py` (`_expand_dollar_skills` snapshots the loaded names and bodies) | `POST /api/chat`, `GET /api/ws` (`chat_message` row with `meta.kind=skill_load`) | | Crew dispatch notice | A one-per-person-turn product notice row the gateway writes into a Crew Mode conductor's own chat when it dispatches a crew member with `session_create`, so a dispatch that is otherwise only a folded tool line stays visible where it happened. The row names the right-panel **Board** (view `workers`) and carries an **Open Board** button with the Board's bolt glyph that reveals the panel (the single host reveal `handleOpenCrewTab`). It is written, not model text, so it always appears independent of what the conductor said: only on a successful dispatch, only from a Crew Mode root (a worker's own sub-dispatch writes nothing), and only on a turn the person sent in the dashboard (an automated conductor turn writes nothing); N dispatches in one person-turn collapse to one notice. The row's English `content` is a fallback for channel mirrors and SSE readers; the dashboard keys its localized copy and the button on `meta.kind=crew_dispatch_notice` | Inside a Crew Mode conductor's transcript, after it dispatches a crew member on a person-sent turn | `pages/chat/CrewDispatchNoticeCard.tsx`, `pages/chat/page/transcriptRendering.tsx`, `pages/chat/transcriptRenderers.tsx`, `pages/chat/useChatPageResourcesController.tsx` (`handleOpenCrewTab`, the host reveal threaded down), `components/ChatPane.tsx`, `components/ChatPaneTranscript.tsx`, `components/SessionGridView.tsx`, `pages/ChatPage.tsx` | `crew_dispatch_notice.py` (`maybe_write_dispatch_notice`, the writer and its one-per-turn gate), `handlers/session_control.py` (`_write_crew_dispatch_notice`, called after a committed `session_create`) | `POST /api/session-control/create`, `GET /api/ws` (`chat_message` row with `meta.kind=crew_dispatch_notice`) | diff --git a/docs/system-specs/modules/security.md b/docs/system-specs/modules/security.md index d25cb7e4d1e6..6e3561ea785e 100644 --- a/docs/system-specs/modules/security.md +++ b/docs/system-specs/modules/security.md @@ -757,6 +757,7 @@ Two mechanisms make "the split changed nothing for a caller" a tested claim rath keeping the credential: keyed per-process HMAC fingerprints of redacted values and bounded per-turn evidence naming the tool call that produced one (see the credential-records bullet under XPIA Hardening). +- `reveal_store.py` — removed credential values kept in process memory only (bounded, 24-hour TTL) behind a random `reveal_id`, so the owner can reveal one through `POST /api/security/redacted-value`; never written, logged or pickled. - `hosts_file.py` and `host_addresses.py` — pure helpers for the ssh self-target floor in `argv_floor_hosts.py`: the hosts-file line parser and its digest, and the per-platform readers of this machine's own interface addresses. `argv_floor_hosts` owns every read, the @@ -1898,6 +1899,7 @@ exactly `outbound` so a second spelling cannot accrete. - Applied on **every** output path — each boundary where agent output reaches a human or an external service. The authoritative list is the `redaction_paths` control in `security_posture_controls.py` (see "Security Posture Detail Registry" below), which is what Settings → Security renders; do NOT restate the count as a literal here - **Deny-surface tool titles** (`dashboard/chat_turn/tool_approval.py`, `dashboard/chat_turn/permission_prompt.py`, `dashboard/chat_turn/deny_chokepoints.py`): `event.title` prefers the model's own `description` field (`_select_tool_title`), so it is agent-controlled display text. Every permission-deny surface — the 🚫 blocked transcript row (broadcast AND persisted to the ConversationLog) and the SEL audit `tool_name` — renders it only through `_redact_display_text()` (both redactors, idempotent, byte-identical for clean titles). The two deny shapes are rendered in exactly one place each: `_reject_invalid_tool()` (name validation failed) and `_reject_hook_error()` (PreToolUse fire raised), beside `_reject_hook_blocked()` (hook exit-2 block), so a permission path added later cannot publish the raw title by omission. `test_dashboard_approval.py::TestDenyRowTitleRedaction` pins each path behaviorally plus a structural zero-raw-interpolation guard - **Owner's credential-redaction switch** (`security/redaction_switch.py`, keystone `credential_redaction.json`, default ON, OWNER-VIEW scoped, PER-REQUEST): `redact_credentials()` stays unconditional on every surface EXCEPT inside an explicit `redaction_switch.owner_view()` scope, and only there does it consult the owner's switch -- with `enabled: false` it returns its input unchanged with no warnings. The scope is opened for exactly ONE surface, the file viewer, and only after the CALLER has verified, for that request, that the requester is the dashboard owner (`owner_view_for_request(request)`, a fail-closed wrapper over `is_owner_dashboard_request`; a Slack allow-listed non-owner running `!dashboard` authenticates with `app == ""` and `sub != owner_id` and gets the unconditional pass): the two handlers that feed the viewer, served by `handlers/files.py` and defined in `dashboard/file_api/editor.py`, `api_file_read` (the buffer) and `api_file_diff` (the `original` the buffer is compared against), both taking one verdict per request from `_owner_view_bypasses_credential_pass` -- one side raw and the other masked would render an unchanged credential line as a hunk -- and both going through `platform.context.redact_owner_view_via_context` so a loaded companion's exfil carve-outs and extra patterns still apply. That is the whole surface, and it is the one the motivating workflow needs -- ask the agent to write the value to a file, open the file in the Files view (the built-in Terminal already streams live output unredacted; the switch is for the rendered document view). `api_file_watch` is deliberately NOT a seam: the same SSE stream serves file-backed artifact live reload, and neither consumer renders its frame -- both re-read through `api_file_read`. Pinned as an allowlist by `test_only_the_named_owner_seams_open_the_scope` and per call site by `test_file_viewer_opens_the_scope_only_for_the_owner`: adding a seam is a review decision. What deliberately does NOT open the scope, and why: **the chat, in any form.** `chat_turn.segment_flush._flush_segment` redacts the assistant text BEFORE `slot.append`, so the transcript holds the redacted bytes and no display-time scope could restore them; a chat seam would advertise a raw transcript it cannot deliver (a raw-plus-redacted dual representation through finalization and persistence is the change that would make it real, and it is out of this PR's scope). The live SSE/WS wire streams (`_wsred` / `_thinkred`) and the broadcast display boundary fan ONE chunk to every connected client, owner or not, besides (pinned by `test_the_chat_surface_never_opens_the_scope`); Slack, Webex, Discord, Telegram and the shared messaging renderer redact at their own sinks and never import the scope (`test_channel_renderers_never_open_the_scope`); every ADMISSION predicate that decides whether a file may reach a THIRD PARTY (`_gate_upload_file` and a non-owner's outbox download, both of which read `redact(text) != text`) is untouched (`test_file_admission_gates_use_the_unconditional_pass`); the owner's own delivery surfaces (`file_send`, the outbox card, the owner's outbox and file downloads) do not scan content, by the owner's decision, so the switch has nothing to govern there; the persisted-history LOAD pass in `chat_persistence` stays unconditional because those bytes also feed the ACP prompt; the diagnostics bundle, the SEL and every log never open the scope. For third-party surfaces this keeps the `backend-security-controls` invariant ("scan before posting to Slack, the dashboard, or any external surface"): a scope a caller must OPEN, after an owner check, cannot be inherited by a sink that forgot to opt out. The switch governs the credential pass ONLY: `redact_exfiltration_urls` runs inside the scope too (the control against an injected agent carrying secrets out in a URL the dashboard renders and the browser fetches, with its own exemption seam in `CredentialPolicy.exempt_exact_hosts`), and every request-BLOCKING gate in `exfil.py` reads the pattern table through `_contains_fixed_credential` rather than the redaction pass. The scope is a `ContextVar`, not a global, so an owner-view render on one task cannot leak the bypass into a channel post the same gateway is making on another (`test_scope_does_not_leak_across_tasks`). The verdict is supplied by the seam and held in the `ContextVar` for the scope's lifetime (`owner_view()`, entered only when the seam has established both the owner requester and the disabled switch -- there is no `bypass` flag), so every redaction call inside one owner-view render sees the same verdict even if the owner flips the switch mid-render -- the flip is seen by the next request (`test_the_verdict_is_the_one_the_scope_was_entered_with`). Why it exists: the scrubber is shape-based and deliberately over-inclusive, so it also swallows values the owner legitimately needs to read back -- the `?token=` value of a one-time approval-workflow link (the pass-4 regression report that motivated it), a blob the owner generated on purpose -- and piping such a value to a file did not help because `api_file_read` runs the same pass. Stated limit: the switch reaches the CREDENTIAL pass only, so a URL the exfiltration-URL pass independently classifies as carrying data out (a long high-entropy query to a non-exempt host) is still replaced whole in the file viewer; whether a given approval link becomes readable depends on that classification and on a loaded companion's `exempt_exact_hosts`, not on this switch. Why the record is a keystone and not `config.json`: switching the scrubber off is an authorization, and the party it constrains is the agent, so a `config.json` switch would be flippable by a prompt-injected agent shell; the leaf sits on `security._CREW_SECRET_LEAVES` (agent file tools refuse it), `sandbox._CREW_READONLY_LEAVES` (mounted read-only for the agent's shell) and `_CREW_PRECREATE_READONLY_FILE_LEAVES` (pre-created as `{}`, which reads as ON, so the absent-file default cannot be created from inside the namespace sandbox). Fail direction: every read that cannot positively establish `enabled: false` (absent, unreadable, malformed, non-UTF-8, non-boolean) answers ON. **Read cost, and the event loop:** there is no cache. The seam is a request handler that already does its file I/O off the loop, so it reads the keystone with `asyncio.to_thread(read_state)` once per owner request (the keystone is a fixed trusted path, not caller-supplied, so the default executor is the right one) and passes the verdict in; a verdict is never older than the request it serves and the loop never touches the filesystem for it. The ONLY writer is the owner-gated dashboard pair `GET`/`PUT /api/security/credential-redaction` (`dashboard/handlers/credential_redaction.py`; Settings -> Security -> Credential redaction in file views), which refuses a non-owner and an app token with 403 and SEL-audits every change as `credential_redaction.enabled` / `.disabled` every owner read as `.read` / `allowed` (non-critical: a read never fails on a briefly unavailable log), every switch change with the REAL subject as the caller (with no `owner_id` configured more than one local identity passes the owner check, and this row must say which one flipped it), and every refusal as `.denied` with the refused SUBJECT as the caller (never a constant, so repeated probing by one allow-listed non-owner is distinguishable); the audit record is written BEFORE the switch as a `critical=True` (synchronous, fail-loud) SEL write -- the default `sel.log` only enqueues and its writer swallows a failed append, so a queued record proves nothing -- and both run as ONE closure on the worker thread under the store's own re-entrant lock (`redaction_switch.transaction()`), so two interleaving PUTs cannot leave the latest SEL record contradicting the persisted switch; a successful PUT then pushes `credential_redaction_changed` to OWNER sockets only (`broadcast_ws_owners`, which bypasses the app-scope gate by construction, so no app token sees it) and every owner dashboard document drops its cached file bodies (`['file-read']`, `['file-diff']`, RESET rather than removed so a mounted observer such as the Library's `SessionDocPreview` sees `data: undefined` at once and refetches under the pass now in force, instead of keeping its last raw result rendered with nothing to re-render it) and its open clean tab bodies, not just the document that flipped the switch -- and, on a WebSocket reconnect (the push has no replay), re-reads the switch and purges if its position moved while the socket was down, or if the document had never read it and it is now ON and has been flipped before (`changed_at` set -- a file opened raw while OFF may be on screen; the shipped never-flipped default cannot have produced one), so a transient drop with nothing to catch never closes diff tabs or empties file bodies; a document the owner gate refuses asks once and then stops, so a non-owner's reconnects do not write audited refusals for a subject that took no action; and the card that made the change purges only on a write the server ACCEPTED, never on a refused one, because `asyncio.to_thread` cannot cancel a running thread and a handler cancelled mid-write (client gone, gateway stopping) must not leave the authorization changed without its trace; a DISABLE whose record cannot be written is refused with 503 and the switch stays ON, while an ENABLE (the fail-safe direction) proceeds regardless (`test_write_and_audit_are_one_unit_on_the_worker_thread`, `test_a_disable_whose_audit_cannot_be_written_does_not_happen`). Known limit, stated: the owner gate is an identity check, so an agent driving the owner's authenticated browser through computer use could flip it; what that buys is bounded to the owner's own dashboard file viewer, and the flip leaves an audit record. Pinned by `test/test_credential_redaction_switch.py` and `website/src/pages/settings/SecurityPanel.credentialRedaction.test.tsx` +- **Owner reveal of a removed credential** (`security/reveal_store.py`, `dashboard/handlers/redacted_reveal.py`, RFC *Owner reveal*): redacted values may be revealed only to the owner through the browser; no agent-reachable path may ever return them; originals never touch disk. When an agent reply is saved, `credential_records(..., reveal_scope=slot.key)` keeps each removed value in process memory (bounded entries and size, 24-hour TTL, no repr, not picklable) and the record carries only a random `reveal_id`. `POST /api/security/redacted-value` is the one reader: it refuses `internal_auth` (every MCP tool, agent, crewmate, sub-agent, cron), the `X-Internal-Secret` header, any app claim, a non-dashboard-user credential and every non-owner, before reading the store; a handle opens only in the slot it was kept for. Each reveal writes a SYNCHRONOUS SEL row (`redacted_value.reveal`: caller, handle, never the value) before the value is returned, and no row means no value (503). A restart or the TTL answers 410 `reveal_unavailable`. Pinned by `test/test_redacted_value_reveal.py` and `website/src/test/RedactionCards.reveal.test.tsx`. - **Code-text scope** (`security/redaction_switch.py` `code_text()`, opened only by `platform.context.redact_code_via_context`): inside it pass 3 of `redact_credentials()` spares exactly two shapes and judges every other run as it does outside. Pass 3 reads a base64-alphabet run as a secret key by entropy and shape, which is a guess about prose; in source code it rewrote what the author wrote on purpose and the program then ran on the redaction marker with no error. The two shapes are told from a key by what they are, never by a name near them (`_is_code_literal`): an ordered alphabet, where every 40-character window has at least 30 adjacent pairs that step up by one character code (`abc...`, `012...`; a random key has about one such pair in 64), and a payload of 64 characters or more directly behind a `data:;base64,` header. A key in any other shape, labelled or not, in call syntax or across a line break, glued to an alphabet or standing alone, is masked exactly as the full pass masks it. Every other pass (key ids, `aws_secret_access_key` and `Authorization: Bearer` values, PEM blocks, JWTs, prefixed tokens, decoded base64 credentials, `token=` values) and the exfiltration-URL pass still run, through the same context-aware shim, so a loaded companion's patterns apply to the text too. The one seam is the member dashboard page scrub (`member_dashboard._template_text_safe`): it takes the full pass over the whole page as its baseline and puts the shim's result in place of the baseline's for the text of each `