Skip to content

fix(terminal): drop terminal questions from the reconnect replay - #36185

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
crew/kiro-crew-contribution/terminal-replay-query-replies
Nov 13, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
crew/kiro-crew-contribution/terminal-replay-query-replies

Conversation

@dwu96

@dwu96 dwu96 commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Opening or reattaching the dashboard terminal no longer leaves junk like 11;rgb:ffff/ffff/ffff;1R typed on the shell prompt.

When the user opens the terminal panel, the zsh prompt already holds text they never typed: 11;rgb:ffff/ffff/ffff;1R11;rgb:ffff/ffff/ffff4;1R;1R;1. Pressing Enter runs it, and zsh prints command not found: 11, no such file or directory: rgb:ffff/ffff/ffff, and so on. The text is the terminal's own answers to old questions: the background colour (OSC 11) and the cursor position (DSR 6).

Why it matters

Anyone whose shell, prompt theme or tools ask the terminal a question (colour probe, cursor position, device attributes) gets junk on the prompt after a reload, a popout or a reconnect. It reads like the dashboard typed into their shell, and one Enter runs it as a command.

Not a goal

  • Changing how live output or live questions are answered once the socket owns the PTY.
  • Changing the frontend onData wiring, the takeover protocol, or any auth or origin check on the terminal socket.
  • 8-bit C1 forms of the questions (0x9b / 0x9d); real output uses the 7-bit ESC forms.

What changed (motivation → approach → change)

Symptom → cause → fix:

Step What happens
Reattach the gateway replays the session's 50 KB scrollback to the new socket (_replace_terminal_ws)
Replay the frontend reset()s xterm.js and writes the old bytes into it
Cause xterm.js answers every old question in those bytes through term.onData, which sends to the PTY
Symptom the asker is long gone, so the shell's line editor gets the answers as typed text

terminal_transport.py gains _strip_terminal_queries (re-exported through the terminal.py facade), a byte scanner that walks each ESC, reads the CSI, OSC or DCS by the terminal grammar, and removes only the questions xterm.js answers (DSR/CPR, DA1/DA2, DECRQM, DECRQSS, OSC 4/10/11/12 colour queries) from the replay snapshot. Look-alikes such as CSI > 4 n (a key-modifier reset) and colour SETs, and any sequence cut off at the end of the buffer, pass through unchanged. A question draws nothing, so the replayed screen is the same. Catch-up bytes produced during the takeover itself are left as they are: they are live output and may still want an answer.

The fix sits on the server so every client that reattaches (page reload, popout, mobile, a second window) gets clean history, with no frontend change.

Backwards compatibility

Compatible: replay drops only terminal questions, which never change the screen; live output, catch-up bytes and every other replayed byte are sent unchanged.

Removes nothing: no route, key, frame type or public function is removed or renamed; the socket protocol is unchanged.

Tests

  • test_replay_drops_terminal_questions_but_keeps_the_screen (test/test_terminal_handler_pty_writes.py): a scrollback holding a prompt, ten question kinds and an OSC 7 cwd report replays as the prompt and the cwd report only, and the new socket still takes ownership.
  • test_replay_strip_keeps_everything_that_is_not_a_question (same file): look-alikes, colour SETs, a DECRQSS ended by BEL, and cut-off sequences pass through byte for byte.

Proof: PROVEN test/test_terminal_handler_pty_writes.py::TestWriteSerialization::test_replay_drops_terminal_questions_but_keeps_the_screen (head 34311e2)

Local: scripts/local-gate.py (backend related 110 passed, frontend related 14672 passed), make build, flake8, isort, black, mypy, tsc -p tsconfig.app.json, semgrep (semgrep/, diff-scoped), test/test_flake_pattern_ratchet.py. The new test is a pure in-memory mock with no timing or ordering dependency.

Manual verification

N/A — unit coverage sufficient: the defect is which bytes the replay sends, and the test asserts those bytes exactly. Repro steps are in #36172.

Screenshots / video

N/A — backend-only diff; no frontend file changed.

Related Issues

Closes #36172

Sibling sweep

Swept: git grep -nE "scrollback" -- 'src/**/*.py'
Sibling: src/kiro_crew/dashboard/handlers/terminal_transport.py -- changed here
Sibling: src/kiro_crew/sprites_session/terminal.py -- not affected -- runs as the PTY child and keeps no buffer; its output rides the same transport replay
Sibling: website/src/utils/terminalRegistry.ts -- not affected -- writes only server frames into xterm; the frontend keeps no replay buffer of its own

Pattern harvest

Not generalizable: the terminal has one replay path (_replace_terminal_ws), and this PR fixes it there.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@dwu96
dwu96 requested review from a team and KiroCrewCaptain as code owners November 12, 2026 23:18
@dwu96
dwu96 requested a review from mbriones98 November 12, 2026 23:18
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 12, 2026
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 34311e223b42c26cf52f94099e3ac44feaf930eb touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 34311e223b42c26cf52f94099e3ac44feaf930eb — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: removing questions from the replay snapshot on the server fixes the stated cause for every client that reattaches, and a revert undoes it cleanly.

Notes

  • A program still waiting when the socket reattaches (say, a probe sent while no client was connected) loses its answer, because its question is now stripped from the replay. The asker has to time out rather than get a reply. This is acceptable, but _strip_terminal_queries's docstring should say so.
  • The other design would mute term.onData while xterm writes the replay. That covers every query form, including the 8-bit C1 ones, without a grammar table. The PR's "Not a goal" rules out a frontend change, so this is a possible follow-up only if more query kinds turn up.

[DESIGN-REVIEWED] 34311e2

Description read

[DESCRIPTION-READ] 6f9f16e9f06996f8d9b1b849f545b12a574933a17e7d053f09706c432a06ee97

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 34311e223b42c26cf52f94099e3ac44feaf930eb — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

This is a reported defect (#36172), fixed on the one replay path, with tests that pin the exact bytes the replay sends. Nothing blocks.

What this change ships

Inventory (3 items) — 3 justified

Intent: after a reload, popout or reconnect, the dashboard terminal's shell prompt no longer holds junk such as 11;rgb:…;1R that one Enter would run. FIX.

  1. A reattached terminal's replayed scrollback no longer contains terminal questions (cursor position, device attributes, DECRQM, DECRQSS, OSC 4/10/11/12 colour queries), so xterm.js stops typing answers into the shell — justified
  2. Catch-up bytes produced during the takeover and every non-question replayed byte are sent unchanged, as before — justified
  3. Five new private transport names are listed in the terminal.py facade's type-checking import block — justified

[FIRST-PRINCIPLES-REVIEWED] 34311e2

Description read

[DESCRIPTION-READ] 6763598580e15bdc893a855641253430208f1e7911b6bd7404227f5eee86586e

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 34311e223b42c26cf52f94099e3ac44feaf930eb — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 34311e2

Verdict parsed from the review's SHA-scoped output markers for commit 34311e223b42c26cf52f94099e3ac44feaf930eb.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 34311e223b42c26cf52f94099e3ac44feaf930eb false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings (all downgraded on adjudication)

GPT 6.1 flagged blocking issues on 34311e223b42c26cf52f94099e3ac44feaf930eb; Opus 5.5 adjudication downgraded every one of them. Downgraded: the remedy each one needs is disproportionate to the harm it prevents. Adjudication downgraded all 1 blocking finding(s): the remedy each one requires is disproportionate to the harm it prevents.

This comment is updated in place on each push.

Review details

BLOCKING -- src/kiro_crew/dashboard/handlers/terminal_transport.py:350 -- Preserve unanswered queries emitted while disconnected

retained_output = _strip_terminal_queries(retained)
Trigger: While the browser is disconnected, a running shell command executes printf '\033[6n'; read -rs -d R reply. The surviving PTY reader buffers the unanswered query through _record_and_forward_terminal_output; reconnecting calls api_terminal_ws → _replace_terminal_ws and strips it.
Outcome: The reattached xterm never receives the query and cannot reply, leaving the running command blocked instead of recovering on reconnect. Catch-up does not resend it because its bytes were already counted in the snapshot.
Anchor: defect
Fix: Track output buffered without a connected client separately from delivered history, and preserve its unanswered queries during reattachment.

[BLOCK-MERGE-DOWNGRADED] 34311e2
[GPT-REVIEWED] 34311e2

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 — LOW harm. I confirmed every condition the failure needs. _record_and_forward_terminal_output puts every PTY read into sess.scrollback whether or not anyone is attached (terminal_transport.py:458-461); a disconnected session (sess.ws is None) only skips the forward at :465. The reconnect snapshot is taken at :314-315 and stripped at :350. Catch-up only sends bytes newer than sent_output_bytes (:331-345), so a query that is already in the snapshot is never sent again. Two more things must also hold: the query is issued while no browser is attached, and the program waits for the reply with no timeout (vim, neovim, fish and tmux all give up after a timeout). There is no automatic recovery: the stuck command is visible on the reattached screen, and Ctrl-C or re-running it clears it. Nothing is corrupted or exposed.

The real fix needs a new "delivered up to" byte offset on the session. It has to be written on the hot output path and at disconnect, and kept consistent with the ring's truncation at :463-464. That adds a new ordering contract between the forwarder, the disconnect path and _replace_terminal_ws, only for a rare no-timeout query asked while nobody is connected. That cost is more than this low, visible, recoverable harm.

[ADJUDICATION] 34311e223b42c26cf52f94099e3ac44feaf930eb total=1 uphold=0 downgrade=1
DOWNGRADE F1 src/kiro_crew/dashboard/handlers/terminal_transport.py:350 reason=disproportionate-remedy
[GPT-ADJUDICATED] 34311e223b42c26cf52f94099e3ac44feaf930eb

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 34311e223b42c26cf52f94099e3ac44feaf930eb false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates an 8% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

The server now strips the questions xterm.js answers (DSR/CPR, DA1/DA2, DECRQM, DECRQSS, OSC 4/10-12 colour queries) from the scrollback snapshot replayed on reattach. Catch-up and live output are untouched. Catch-up accounting uses output_bytes/sent_output_bytes, not the replay length, so dropping bytes shifts no offsets. scrollback has one writer and is read only in _replace_terminal_ws, the single replay path. Each regex branch matches only query sequences, and colour-set and draw sequences are kept. The new test pins the replayed bytes exactly and would fail on base. No auth, protocol or frontend change; the trust surface only shrinks.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 1f8ba446c9 · 67 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@dwu96
dwu96 force-pushed the crew/kiro-crew-contribution/terminal-replay-query-replies branch from 1f8ba44 to d7103e0 Compare November 13, 2026 00:40
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the full diff. The fix is right: removing terminal questions from the replayed scrollback stops a fresh xterm.js from answering them into the shell (#36172). It leaves catch-up bytes alone, and the test pins that the screen, the colour change and the OSC 7 cwd report survive. One part needs a change.

Please replace the regex with a byte scanner. This PR adds _TERMINAL_QUERY_RE = re.compile(...) and _TERMINAL_QUERY_RE.sub(b"", data) in terminal_transport.py. The project does not take new regex-based handling in product code. A pattern over escape sequences is also easy to get subtly wrong in a security-relevant stream.

A small structured pass fits better: walk data and, on ESC (0x1b), look at the next byte.

  • [: read the CSI. Take an optional private marker (? / >), then parameter bytes (digits and ;), then optional intermediate bytes (0x20-0x2f, e.g. $), then the final byte (0x40-0x7e). Drop the whole sequence only for the question forms you listed: final n, final c, or intermediate $ with final p. Otherwise copy it through.
  • ]: read the OSC up to BEL or ESC \. Split the body on ; and drop it only when the first field is 4 (with every value ?) or 10/11/12 with ? values.
  • P: drop a DCS whose body starts with $q, up to ESC \.
  • Anything else, and any sequence cut off at the end of the buffer, is copied through unchanged, as the regex does today.

That keeps the same behavior, reads as the terminal grammar it implements, and makes the "which forms count" list explicit. The existing test can stay as is. Everything else in the PR is fine.

@KiroCrewCaptain
KiroCrewCaptain enabled auto-merge (squash) November 13, 2026 02:07

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: fix (3 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: the terminal reconnect replay drops old terminal queries so a fresh terminal does not type their answers into the shell; live catch-up bytes are unchanged. Test added.

@dwu96
dwu96 force-pushed the crew/kiro-crew-contribution/terminal-replay-query-replies branch from d7103e0 to 6935534 Compare November 13, 2026 02:18
@dwu96

dwu96 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

@bolichen97 thanks for the review.

Replace the regex with a byte scanner -- fixed in 6935534.

fixed in 6935534: _strip_terminal_queries is now a byte scanner; _TERMINAL_QUERY_RE and the re import are gone.
On each ESC it reads [ as a CSI (marker, digit/; params, intermediates 0x20-0x2f, final 0x40-0x7e), ] as an OSC up to BEL or ESC \, and P + $q as a DCS up to ESC \.
The forms it drops are listed in _CSI_QUERIES and _is_colour_query. CSI > Ps n is kept: it resets key modifiers and is not a question.
Anything else, and any sequence cut off at the end of the buffer, is copied through unchanged.
Same behaviour as before: 300,000 random escape-heavy inputs gave identical output under the old regex and the scanner.
The existing test is unchanged; test_replay_strip_keeps_everything_that_is_not_a_question pins the look-alikes and cut-off cases.

The branch is rebased onto current main because the push guard required it for this push. No conflict.

@dwu96
dwu96 requested a review from bolichen97 November 13, 2026 02:19
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: passed Eligible automated validation passed for the current revision labels Nov 13, 2026
@dwu96

dwu96 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Backend Tests (3.12, 8) is red on test/test_reply_format_once.py::TestEveryMessageWhereTheBlockCannotBeTakenOut::test_the_table_names_every_selectable_harness. That is main's red (#36279, fix in #36315), not this diff. Waiting for the fix to land; no rerun or override for it.

A reattach replays the session's scrollback into a freshly reset xterm.js.
xterm.js answers every old OSC 11 / DSR / DA question in it through onData,
and the answers reached the shell as typed junk on the prompt
(11;rgb:ffff/ffff/ffff;1R). Strip those questions from the replay snapshot.

Closes #36172
@dwu96

dwu96 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in test/test_reply_format_once.py (#36290 fixed main's red #36279, which failed Backend Tests shard 8 on Linux, macOS and Windows and the Coverage Gate here) (9b32941 -> b7a4e1e); CI re-running. Shard 7 was not main's red: attempt 1 hit 120 s timeouts in unrelated tests, attempt 2 lost its self-hosted runner. No change to this PR's three files.

@dwu96
dwu96 force-pushed the crew/kiro-crew-contribution/terminal-replay-query-replies branch from 6935534 to 34311e2 Compare November 13, 2026 04:10
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@dwu96

dwu96 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor Author
  • Preserve unanswered queries emitted while disconnected span=6121480447ab

Rebutted (disproportionate, already downgraded by adjudication): the case needs a question asked while no browser is attached AND a reader that waits with no timeout; vim, neovim, fish and tmux all time out.
Before this PR the same reply arrived late into a shell that had already moved on, which is the junk in #36172; a blocked read is visible on the reattached screen and Ctrl-C clears it. Nothing is lost or exposed.
Keeping those questions needs a new delivered-up-to offset written on the hot output path and at disconnect, plus a new ordering contract with _replace_terminal_ws; that is new scope beyond the Goal.
Catch-up bytes produced during the takeover are left unstripped, so a question asked while a window is reattaching is still answered.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Thanks for the change. I re-reviewed the whole diff on this head. All checks are green, including Publish readiness signal after its rate-limited run was rerun.

  • The regex from our earlier change request is gone: no product file in this diff imports re or compiles a pattern. _strip_terminal_queries walks each ESC with plain byte checks. _csi_query_end reads the marker, parameters, intermediates and final byte, and looks up the result in _CSI_QUERIES. _string_end finds the BEL or ST that closes an OSC/DCS string, and _is_colour_query splits the OSC body on ;.
  • It drops only the questions xterm.js answers: DSR/CPR, DA1/DA2, DECRQM, the OSC 4/10/11/12 colour queries and DECRQSS. Look-alikes (CSI > 4 n, colour sets) and sequences cut off at the ring's end pass through byte for byte, as the second test pins.
  • The strip runs only on the retained scrollback replayed into a fresh terminal. Catch-up bytes are live output and stay as they are, so a program still running gets its answer. Nothing is refused.

Auto-merge is already on, so I left it as it is.

@KiroCrewCaptain
KiroCrewCaptain merged commit 48dee00 into main Nov 13, 2026
161 of 176 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the crew/kiro-crew-contribution/terminal-replay-query-replies branch November 13, 2026 07:52
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Terminal: opening the panel leaves 11;rgb:ffff/ffff/ffff;1R junk on the zsh prompt

3 participants