Repository navigation
feat: show the prompt each turn sent in the context tab - #36795
Conversation
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design-level concerns. It is an owner-only, in-memory, read-only addition that follows the recorded 2026-11-06 decision, it captures the text after the envelope swap, and it removes nothing. Notes
[DESIGN-REVIEWED] 8f6d764 Description read[DESCRIPTION-READ] 6e3d224c322baade3678860eb2600affca8d541fe73d193fcbc0b2688d001c4f That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing to check. What this change shipsInventory (10 items) — 10 justifiedIntent: when a turn goes wrong, let the dashboard owner read the exact text that turn sent the agent, instead of only its per-block character counts. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 8f6d764 Description read[DESCRIPTION-READ] 889832c065e06e9cb018eef7578483af6c1119d04060d8c42527e50d026e95f2 That sha256 names the PR description this verdict read. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
UX Review (Opus 5.5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing blocks. The screenshots show every state of the new disclosure, and the blind reader correctly identified "Prompt as sent" as the main control and said they would use it. Notes
[UX-REVIEWED] 8f6d764 Description read[DESCRIPTION-READ] 11854654e0afaedc65c83f08c4773f843a4a99e2326f16c70daa2fd2d58dae6b That sha256 names everything this verdict read: the PR description, plus the 14 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
[operator: bolichen97] Rebased onto current main to resolve the merge conflict in docs/decisions/README.md, docs/feature-map/README.md, src/kiro_crew/dashboard/chat_runner.py and website/src/test/ApiClient.refactor.surface.test.ts. Both sides are kept in each file. The same push also carries three fixes reviewed since the last push: the new route is listed with the other redaction sinks in Settings -> Security, the open prompt is keyed on the turn row's version so a turn sent again shows its new text, and the masking test uses a checksum-valid synthetic token, which main's credential redactor now requires. |
ad8b29c to
e7664eb
Compare
Sensitive Change ReviewSensitive files changed:
|
|
self-added: no
|
|
Rebased: main moved in docs/feature-map/README.md and website/src/test/ApiClient.refactor.surface.test.ts since this head's CI ran (8826a22 -> 256d5af), and both conflicted with this branch; CI re-running. The branch now sits on current main. Main replaced the hand-kept API key list in that test with a property check, so the file takes main's version and leaves this diff; in the feature map main added a row right above the Context breakdown row this change edits, and both rows are kept. The same push carries the two fixes for the last head's reds: the prompt-as-sent handler reads the usage fold through the [operator: bolichen97] |
|
self-added: no
|
e7664eb to
d5f8d1b
Compare
|
self-added: no
|
|
[operator: bolichen97] The reds on d5f8d1b are main's, not this change's, and each already has a fix open; this PR waits for them and then re-syncs.
This diff touches neither |
The Context tab says how many characters each block of a turn had, but not what the text was, so a rule the model ignored or a block that doubled could only be chased through the opt-in wire recorder's file. Add a "Prompt as sent" disclosure under the selected turn. The gateway keeps the text the transport received (after the receipt-envelope swap) for each session's newest turns, in process memory only and bounded per session and in total; a restart forgets it and nothing is written to disk. GET /api/telemetry/prompt-as-sent joins the tab's turn ordinal to the composer's turn number through the usage fold and returns the text with credentials and exfiltration URLs masked, and is listed with the other redaction sinks in Settings -> Security. A prompt over the size cap keeps a marker instead of its text, so the read answers 404 too_long and the tab says it was too long to keep rather than that it was dropped. The read is owner-only through require_owner_dashboard_request, as the Crew log is; an app caller gets a 404 and a SEL line. Restricted (incognito, temporary) sessions are never kept. No new switch, per the maintainer decision recorded under docs/decisions/.
d5f8d1b to
8f6d764
Compare
|
Rebased: main moved in test/conftest.py and test/conftest_resets.py (#39479, the per-test reset behind the module-cache ratchet red on Backend Tests (3.12, 1) and Backend Tests (Windows) (4)) since d5f8d1b was tested (256d5af -> 3688176); CI re-running. The patch is unchanged. The Backend Tests (3.12, 7) red in test_source_link_github_case is also main's, and its fix (#39475) is still open. [operator: bolichen97] |
Sensitive change review
[operator: bolichen97] |
KiroCrewCaptain
left a comment
There was a problem hiding this comment.
Reviewed head 8f6d764 by tracing the capture path from the dashboard runner into the provider's delivery hook. The hook fires once, only for the prompt object the runner registered, after the receipt swap, so slash commands and internal streams are never recorded. The read route refuses app callers with an indistinguishable 404 and an audit line, refuses anyone who is not the dashboard owner, skips restricted sessions, and masks secrets on the way out. The in-memory store keeps its character total exact through rewinds, oversize prompts and eviction. The turn-number join will not show one conversation's text under another's turn. The new and touched backend tests pass locally on this head, and CI is green, so nothing here blocks.
Problem / Motivation
Goal: The side panel's Context tab shows, under the selected turn, the exact text that turn sent the agent, with secrets masked, readable only by the dashboard owner and held in memory only.
The Context tab tells you how many characters each block of a turn had, but you can't see the text itself. When a prompt goes wrong (a rule the model ignored, a block that doubled, a marker in the wrong place), the dashboard has no way to show what the agent actually read.
Why it matters
When the agent ignores a rule or a block turns up twice, the Context tab's per-block character counts say something is off but not what. Today the only way to see the text itself is to switch on the opt-in wire recorder and read its file on disk, which most people debugging a turn never do.
Not a goal
What changed (motivation → approach → change)
Expanding Prompt as sent under the selected turn in the Context tab shows the text that turn sent the agent, with secrets masked. Only the dashboard owner can read it, and the gateway forgets it on restart. When there is no text to show, the tab says why in plain words. This follows the maintainer decision of 2026-11-06, recorded under
docs/decisions/: no switch, owner-only, a 404 for apps, masked on read, nothing on disk.The text kept is the one the transport received, not the one the composer built, because
EssentialDeliverycan swap a receipt envelope out of a member turn just before the send.chat_runner.pynames a prompt's turn just before the stream, andEssentialDelivery.report_next_send(essential_delivery.py) reports the text it handssend, once, after the envelope swap. Before any stream opens, every turn with a number (a prompt, a slash command, an agent-started turn) drops that turn and every later one, so a retry, a regenerate or a turn after a rewind never shows an earlier turn's text. Only a prompt keeps text. Nothing is kept for an incognito or temporary chat (by its key or the slot's own memory mode), or while the crew log is off.prompt_trace.py: the newest 8 prompts per conversation (one crew-log unit, the chat's ACP session), at most 32 conversations and 8M characters in all; the conversation written longest ago goes first, with one log line naming it and the cap, never its text. A prompt over 1M characters keeps a marker instead of its text and logs one line naming the turn. Process memory only.GET /api/telemetry/prompt-as-sent?slot=&turn=(dashboard/handlers/prompt_as_sent.py) maps the tab's turn to the composer's turn number withcontext_turn_numberincrew_log/projection_usage_measured.py(exported fromcrew_log/projection.py), for the chat's newest conversation only. An app caller gets the 404 an unknown route gives, plus a SEL line; anyone else who is not the owner gets403 owner_onlyfromrequire_owner_dashboard_request, the Crew log tab's check. Then400 slot_required/400 turn_invalid,404 too_long,404 not_kept. The text is masked withredact_exfiltration_urls, thenredact_credentials, and the route is listed with the other redaction sinks in Settings → Security (security_posture_sinks_gateway.py).PromptAsSent.tsx, under the blocks list inContextBreakdownPanel.tsx, is closed by default, fetches when opened and again when the turn is sent again (the query is keyed on the turn row's version), never polls, names the prompt while it loads, says in a plain line why there is no text (not_kept,too_long), shows theowner_onlyrefusal throughErrorNoticewith its sentence and the agent hand-off, and puts the text in a labelled, keyboard-scrollable region. Six strings in all 13 locales (website/src/i18n) and one API client method (api/client/telemetry.ts).This is the dashboard half of the issue; the durable half, outbound frames in the opt-in wire recorder, landed in #31099.
Backwards compatibility
Compatible: one new read-only route, one new disclosure and six new catalog keys; no existing route, field, setting or stored record changes.
Removes nothing: the diff only adds an in-memory store, a route, a one-shot delivery hook, a projection reader (
context_turn_number), a UI row and one redaction-sink row in Settings → Security.Tests
test/test_prompt_as_sent.py, over a real crew log:ghp_token masked, and the tab's ordinal joins the composer's turn number;403 owner_only; an app caller gets404 not_foundand a SEL denial;404 too_long, counts no characters, and logs one line without its text;MAX_SESSIONS, without its text;test/test_essential_delivery.py::test_prompt_as_sent_is_the_text_the_wire_received: the kept text is exactly what the transport received (receipt header swapped out); another stream in between drops a pending report.website/src/test/PromptAsSent.test.tsx: nothing is fetched while closed; opening shows the text; an open prompt re-reads when its turn is sent again and never shows an earlier attempt while it runs;not_keptandtoo_longeach show their sentence;owner_onlyshows its sentence in anErrorNoticealert with the agent hand-off; the loading line names the prompt.test/test_security_posture.py's redactor call-site scan finds the route registered as a redaction sink.Determinism: every added or changed backend test passed 20 times with
-n0, 10 times with-n 4 --dist loadgroup --max-worker-restart=2beside the rest oftest/test_essential_delivery.py, and once with its 11 collected ids in reversed order (no random-order plugin is installed);PromptAsSent.test.tsxpassed 10 times.Manual verification
On Linux (Ubuntu 24.04, x86_64), against an isolated gateway running this patch on
mainat 910f01f (the rebases since then changed only context lines next to this diff), withspawn_feature_gateway("minimal", approval="yolo"), the packaged fake ACP backend, a throwawayKIROCREW_HOME, and every request to127.0.0.1as the dashboard owner. Every gateway was stopped by its own pid.On
mainthe Context tab has no such row, andGET /api/telemetry/prompt-as-sentanswers 404.With this change, one chat: turn 1 was "Please note" with a fake AWS key id (
AKIA…12KL) and a fake GitHub token (ghp_and 36 characters), then "The purple walrus audits lighthouse ledgers on Tuesdays."; turn 2 was 1,099,999 characters of ordinary words; turn 3 was "Marker alpha: the copper heron counts lanterns at midnight."The token comes back masked and the bare AWS key id as typed (abbreviated above): on this base the shared
redact_credentialsclasses a bare access key id as an identifier, not a secret (#29423), and this route runs that chain unchanged, afterredact_exfiltration_urls.GET /api/security/posturelists the row "The prompt a turn sent" fordashboard/handlers/prompt_as_sent.py.After
GatewayHandle.restart(), turns 1 to 3 answer404 {"code": "not_kept"}, and a fresh turn 4 answers 200 with its own text.On disk,
grep -rlF "purple walrus"over the isolated home finds only the chat transcript and the crew log (the user's own message), and no file holds the composer markerCURRENT USER REQUESTthat every assembled prompt carries, so no file holds the assembled prompt.Through the dashboard and Playwright Chromium, with the SPA built from this commit: with Prompt as sent open on turn 3 ("Marker alpha: the copper heron counts lanterns at midnight."), the turn was sent again with Edit & Resend as "Marker beta: the silver otter paints harbor maps at noon."; the disclosure stayed open (
aria-expandedstayedtrue), and the same panel then ended in the beta line.The app caller's
404with its SEL denial and the non-owner403rest on the unit tests, because the harness mints only the owner's identity.The harness's default backend answers slash commands locally before any stream, so the forwarded slash-command case is covered by the unit test through the real runner, not driven here.
Screenshots / video
Every picture below comes from a build of this patch (on
mainat 910f01f, as above), served by isolated gateways; the last one is thatmain's own build. The turn with the fake secrets, open in light and dark (dark set through the app's own theme setting). Its user line readsPlease note AKIA…12KL and [REDACTED: credential]. The purple walrus audits lighthouse ledgers on Tuesdays.: the token is masked and the bare key id is shown as typed, as the Manual verification above explains:The same turn sent again with Edit & Resend while the disclosure stays open: before, then after it settles:
The 1,099,999-character turn, and a turn from before a gateway restart:
Owner-only and loading (routed responses), closed by default, and main's Context tab
The harness mints only the owner's identity, so these two states come from a Playwright route on
/api/telemetry/prompt-as-sent: a403 owner_onlyanswer, shown as an error notice in light and dark, and a request held unanswered.Related Issues
Closes #16613
The issue's other half, outbound frames in the opt-in wire recorder, landed in #31099.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.