Skip to content

feat(app-sdk): make the chatembed poll read incrementally - #20917

Merged
iamwhatever merged 2 commits into
mainfrom
feat/chatembed-incremental-poll
Oct 16, 2026
Merged

iamwhatever merged 2 commits into
mainfrom
feat/chatembed-incremental-poll

Conversation

@buluoray

@buluoray buluoray commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: An app's embedded chat stays current by fetching only the rows that changed, instead of re-downloading its whole 200-row page every second.

ChatEmbed polls GET /api/chat/slots/{slot}?limit=200 every 1 s while a turn runs and every 5 s idle. Each poll sends back the full page, so a running turn that adds one streamed sentence still moves 200 rendered rows over the wire, and the embed parses and diffs all of them. The RFC's P5-f row and #19994's phase P-d record this as the open item.

Why it matters

Every app that mounts ChatEmbed (spec-builder, ops-mission-control, third-party apps) pays this per second per open embed. On a long session the page is the 200 largest-recent rows, so the cost tracks the size of history rather than the size of what changed.

Not a goal

  • No WebSocket path for the embed. It keeps one scoped poll.
  • No change to the poll cadence, the stale-poll notice (fix: flag a chat embed whose poll keeps failing as stale #19899), the load-earlier bar, optimistic rows or the queue strip.
  • No change to the dashboard's own chat reads, which do not send the cursor.
  • No server CPU saving: the server still renders about one page to check the held rows (see below).

What changed (motivation → approach → change)

Choice: since-cursor over the WebSocket stream. The WebSocket route would need a store-free copy of the dashboard's chat frame reducer (chunks, tool rows, in-place chat_message_update, approvals, queue frames), it only delivers for slots the app owns unless the host grants a slots:* scope, and the default subscribeFn is a no-op. The cursor keeps the embed's single read through the app's scoped api, so allowedApiPaths, the stall/back-off logic and the load-earlier widening all stay as they are.

Why a plain index cursor is not enough. Rows are edited in place far from the tail (_mark_permission_resolved, card patches in chat_cards.py, MCP sign-in rows, steer meta). A cursor of "rows after N" would miss those edits. So the cursor carries a digest of the rows the reader already holds, and the server checks it.

Backend (src/kiro_crew/dashboard/slot_detail_delta.py, wired into api_chat_slot_detail):

  • ?limit=L&cursor=1 returns the same page plus cursor: {from, since, digest}: the first row's index, the last row's index, and a digest of the rows in [from, since). The digest is built from each row's render-memo identity (wire_render.prepare_message_rows: a keyed digest of the stored row plus the redaction context and running flag), never from display text. Each row is serialized once and both its fragment and its identity come from that snapshot. Rows with no identity (a streaming chunk, an OAuth row, any row in an owner-view scope) issue no cursor over them. The last row (the folded streaming row) is always outside the span.
  • ?limit=L&from=F&since=S&digest=D composes the page reaching back to F, and compares the digest of [F, S) again. Equal: only rows from S on, plus delta: {from, since} and next_before = new window start. Not equal (in-place edit, rewind, rotation, changed redaction context, forged digest, span wider than a page): the full page, no delta.
  • A page whose rendered rows do not line up one-to-one with their indices, or whose span holds a row that renders as nothing, issues no cursor, so the next read is a plain full one.
  • wire_render.prepare_message_rows returns per-row rendered text and identities; prepare_messages_json joins the text, same bytes as before.
  • The read reaches back to F so the span can be checked, but only rows from the older of F and the new window start are rendered (render_from): about one page plus the rows new since the last poll.
  • The span ceiling is SLOT_DETAIL_MAX_LIMIT, the handler's own page ceiling.
  • Malformed triples and cursor + before are 400 invalid_query_params. Cursor reads always take the paged route.

Frontend (website/src/app-sdk/embedIncrementalRead.ts, ChatEmbed.tsx): the query function reads the held page from the query cache, echoes its cursor, and splices a delta onto the held rows (dropping rows before the new window start). A delta that does not answer the cursor sent, or does not fit, triggers one full re-read. A response with no delta replaces the page, as today.

RFC (docs/request-for-change/rfc-chat-core-extraction.md): §4.1's standalone-publish gate said to wait on #8651. #8651 closed as delivered on 2026-10-11 (ChatEmbed mounts without the store; the RFC links the closing comment), so the gate is marked met and the 2026-09-05 host-only decision is marked superseded for ChatEmbed; the note that ChatInput itself is still store-bound stays. This correction was asked for in the same dispatch as the poll change (tracking #19994). The #8651 row is updated, and the P5-f row is split so the embed slice reads "in review" and the ChatPage/ChatPane slices keep their own status.

Spec: docs/system-specs/modules/history.md documents the cursor contract next to the paginated slot-detail section.

Backwards compatibility

Compatible: the cursor is opt-in. A read without cursor=1 or from/since/digest is served exactly as before (same fields, same order). An embed talking to a gateway without this change never gets a cursor and keeps full reads. The request still goes to /api/chat/slots/{slot}, so the /api/chat grant still covers it.

Removes nothing: only new optional query parameters and response fields are added.

Tests

  • test/test_slot_detail_delta.py (backend, 24): row identities follow the stored row, OAuth/chunk rows and owner-view rows carry none, a row without identity inside the span issues no cursor, identities are length-framed in the digest; an in-place edit during render cannot pair the old fragment with the new identity (memo and non-memo paths); assistant, markdown, link and tool rows hash the same across polls so a running turn still gets deltas; a cursor read renders only the rows it hashes or serves (12, not the 19 composed); a cursor=1 read serves the same rows as a plain read; an echoed cursor gets only the re-sent last row plus new rows and the slid window start; a row edited in place inside the span returns the full page; a forged digest returns the full page; cursor without limit stays bounded; malformed cursors and cursor + before are 400; plan_page edge cases (misaligned rows, a dropped row inside vs at the end of the span, empty session, cursor behind the composed page, position-bound digest).
  • website/src/test/embedIncrementalRead.test.ts (6): path building, cursor validation (rejects a digest that would inject query text), splice arithmetic, refusal of deltas it cannot place.
  • website/src/test/ChatEmbed.boundedPoll.test.tsx (3 new): the embed echoes the held cursor and shows held-kept + delta rows; a full page replaces; a mismatched delta triggers a full re-read.
  • Mutation checks run locally: dropping the digest comparison reddens test_a_row_edited_in_place_falls_back_to_the_full_page and test_a_forged_digest_is_a_full_page; never applying a delta reddens test_echoed_cursor_gets_only_the_new_rows; rendering from the composed page start reddens test_a_cursor_read_renders_about_one_page ([10, 19] != [10, 12]); ignoring the held cursor in ChatEmbed reddens "echoes the held cursor and splices…".
  • Updated URL expectations in ChatEmbed.boundedPoll.test.tsx and ChatEmbed.test.tsx (first read now ends &cursor=1); chatCoreTranscript.contract.test.ts now pins the bounded path inside embedIncrementalRead.ts.

Manual verification

Not run against a live gateway. Covered by the handler tests above, which drive the real aiohttp route. Local gate: black/isort/flake8 clean on touched files, comment-history and brand gates pass, tsc -p tsconfig.app.json clean, 322 ChatEmbed/app-sdk/app vitest cases pass. local-gate.py backend step reported 97 failures, none in slot-detail or chat tests; the same files (test_file_explorer_app.py, test_onboarding_import_refactor_contract.py, test_perf_boot_path.py, test_design_tweak_backend.py, test_cli.py) fail the same way on a clean origin/main worktree on this host. mypy reports 3 errors in wecom/client.py, untouched here.

Screenshots / video

Why no screenshot: the embed renders the same rows as before; only how the poll fetches them changed.

Related Issues

Refs #19994 (phase P-d). Overlap with open PRs, checked with git merge-tree against this head: #19898, #19887, #19903 and #20178 touch ChatEmbed.tsx (and #19898 slot_detail.py, #20178 the RFC) and all merge cleanly; #19890 and #19879 share no files. #19879's contract page still describes the poll correctly (same path and cadence).

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license; see the Licensing section of CONTRIBUTING.md.

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners October 15, 2026 18:05
@buluoray
buluoray requested a review from mbriones98 October 15, 2026 18:05
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 15, 2026
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Run one live turn in an embed and check that streamed text and an in-place approval update appear without a stale row. The PR says this was never tried against a gateway.

[UX-REVIEWED] 4f100d8

Description read

[DESCRIPTION-READ] fdab2528afd3fe7017e1e67c36a9b39df04c5a01b8890676af71f77545e5aed4

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Have a maintainer confirm that the #8651 standalone-publish gate is met. That RFC ruling comes along in this poll PR and is not part of the poll change.

[DESIGN-REVIEWED] 4f100d8

Description read

[DESCRIPTION-READ] e28cd7d86577a14b8d78125c0e07ab23fa47a68f498d45b1bb8ec3e0eddbce16

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — 🟡 CONCERNS

Premise-level review of 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

The poll change earns its place. A second, unrelated decision rides along with it: the RFC's ratified standalone-publish gate is marked met inside a polling PR.

Not justified as shipped

What this change ships

Inventory (7 items) — 6 justified

Intent: an app's embedded chat stays current while downloading only the rows that changed. This is an ADDITION.

  1. An embed's poll during a running turn now carries only new and changed tail rows, not the full 200-row page — justified
  2. Slot detail accepts cursor=1 / from+since+digest and returns cursor and delta fields (1 consumer: ChatEmbed) — justified
  3. Any in-place edit or forged digest inside the held span gets the full page back — justified
  4. cursor combined with before, or a malformed cursor, is now a 400 — justified
  5. The renderer exposes per-row fragments and identities; full-read output is the same bytes — justified
  6. The RFC §4.1 publish gate is marked met and the 2026-09-05 decision superseded — rides along, unrelated to polling
  7. The P5-f row is split, and the history spec documents the cursor contract — justified

[FIRST-PRINCIPLES-REVIEWED] 4f100d8

Description read

[DESCRIPTION-READ] 18d509d80d399498a275cd8ff09ef4fd1a2592afff064497dfce114aed96aa12

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 4f100d8

Verdict parsed from the review's SHA-scoped output markers for commit 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 4f100d8

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 4f100d8a3df8f0b67c2388c56baf5c11e97e9dca: <one-sentence reason>

@buluoray

Copy link
Copy Markdown
Contributor Author

Dispositions for the review lanes on 69811c3, addressed in 5db6c73:

  • Opus 5.5 (render cost): correct. A cursor read rendered every row it composed to reach the span, about two pages. Fixed: only rows from the older of the span start and the new window start are rendered (slot_detail_delta.render_from). test_a_cursor_read_renders_about_one_page pins it and goes red ([10, 19] != [10, 12]) when the slice is removed. history.md now says what is rendered. The misaligned fallback keeps serving the composed rows unchanged: trimming it would make next_before name a row the response does not start with, and it only happens when a page that was aligned on the last poll is not aligned now.
  • First Principles item 6 (duplicate constant): fixed. DELTA_MAX_SPAN is now SLOT_DETAIL_MAX_LIMIT, imported, not redefined.
  • First Principles item 5 (RFC §4.1 rides along): kept, by request. The dispatch for this item (tracking chat-core: migrate ChatEmbed onto shared transport, composer and transcript #19994, phase P-d) asked for the §4.1 and status-row correction in this PR. Your two factual points are fixed: the 2026-09-05 host-only decision line is now marked superseded for ChatEmbed, and the RFC links the chat-core P3: store-free seam for ChatInput so ChatEmbed no longer requires the host store #8651 closing comment instead of resting on this description. A maintainer can still drop the hunk.
  • Design (tool/assistant rows hash stably): checked. test_assistant_and_tool_rows_hash_the_same_on_every_poll polls a page holding markdown, a link and a tool row and gets a delta.
  • UX (live gateway run): not done on this host; still stated as unverified in the PR body.
  • Frontend Tests (2): self-caused. chatCoreTranscript.contract.test.ts pinned the old inline URL in ChatEmbed.tsx; it now pins the bounded path in embedIncrementalRead.ts.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 15, 2026
ChatEmbed re-read its whole bounded page (200 rows) every second while
a turn ran and every five seconds idle. Slot detail now issues a cursor
on request (cursor=1): the page's first and last row indices and a hash
of the rendered rows before the last one. Echoing it back answers only
the rows from the cursor on, marked delta, or the full page when any
earlier row renders differently (an in-place edit, a rewind, a
rotation, a forged digest). The embed splices a delta onto the page it
holds and reads the page whole when a delta does not fit.

The cursor was chosen over the WebSocket stream: the embed keeps its
one scoped read, the stale-poll notice and the load-earlier bar, and
needs neither a store-free copy of the chat frame reducer nor an event
grant from its host app. A gateway without the cursor keeps serving
plain pages.

The RFC's standalone-publish gate is marked met: #8651 closed as
delivered, since ChatEmbed already mounts without the store.

Refs #19994
@buluoray
buluoray force-pushed the feat/chatembed-incremental-poll branch from 5db6c73 to e1cb822 Compare October 15, 2026 19:38
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 15, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 15, 2026
@buluoray
buluoray force-pushed the feat/chatembed-incremental-poll branch from e1cb822 to 6a85a76 Compare October 15, 2026 20:12
@buluoray

Copy link
Copy Markdown
Contributor Author

Rebased onto main e492fda as asked (range-diff showed both commits unchanged). After that, CodeQL raised a high alert on e1cb822: py/weak-sensitive-data-hashing at slot_detail_delta.py:120, because the span digest hashed rendered display text. That text can carry what redaction was built to find. Fixed in code on 6a85a76, not dismissed: the digest is now built from each row's render-memo identity (a keyed digest of the stored row plus the redaction context, the same key wire_render already uses), and never from display text. Rows with no identity (streaming chunk, OAuth row, owner-view scope) issue no cursor over them. A mutation that makes the identity constant reddens test_a_row_edited_in_place_falls_back_to_the_full_page and test_identity_tracks_the_stored_row_not_the_object. Still 2 commits.

On e1cb822 the Opus and GPT lanes failed to publish (API rate limit exceeded for installation, 19:47Z). That is infrastructure, and this push re-runs them. One more thing: test_the_lazy_imports_stay_inside_the_functions_that_need_them (84 != 83) still fails locally on clean main e492fda, so the backend shard 6 red may come back. It is not from this diff.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 15, 2026
A cursor read composed the page back to the held span's start so it
could check it, then rendered every composed row: about two pages per
poll where one plus the new rows is enough. Only rows from the older
of the span start and the new window start are rendered now.

The span digest hashed rendered display text, which can carry what
redaction was built to find. It is now built from each row's
render-memo identity: a keyed digest of the stored row plus the
redaction context, the same thing the memo trusts to mean "renders
the same". Rows with no such identity (a streaming chunk, an OAuth
row, anything in an owner-view scope) issue no cursor over them. Each row
is serialized once and both its render and its identity come from that
snapshot, so an edit the event loop makes mid-render cannot certify a
stale row.

The span ceiling reuses SLOT_DETAIL_MAX_LIMIT instead of a second
constant, the transcript contract test follows the read path into
embedIncrementalRead, and the RFC marks its 2026-09-05 host-only
decision as superseded and links the #8651 closing comment.

Refs #19994
@buluoray
buluoray force-pushed the feat/chatembed-incremental-poll branch from 6a85a76 to 4f100d8 Compare October 15, 2026 20:41
@buluoray

Copy link
Copy Markdown
Contributor Author

Dispositions for 6a85a76, addressed on 4f100d8:

  • GPT 6.1 BLOCKING (security-class) and Opus FINDING, same race: both correct. The worker thread rendered the live row and then serialized it a second time for its identity, so an in-place edit in between (steer consumed, card patched) could certify a stale fragment. Fixed in code: each row is serialized once (wire_render._snapshot), and both the render (from the decoded copy) and the identity come from that one string, on the memo and non-memo paths. test_an_edit_mid_render_cannot_pair_a_fragment_with_another_rows_identity[True|False] edits the live row inside the render. Both cases go red when the identity is re-serialized after the render.
  • First Principles item 7 (prepare_message_fragments is a pass-through): fixed. The wrapper is gone and prepare_messages_json calls prepare_message_rows. The old _memo_row, now without callers, is gone too.
  • First Principles item 8 (RFC §4.1 rides along): kept, as answered on the earlier head. The dispatch for this item (chat-core: migrate ChatEmbed onto shared transport, composer and transcript #19994, P-d) asked for it, and it cites the chat-core P3: store-free seam for ChatInput so ChatEmbed no longer requires the host store #8651 closing comment. A maintainer can drop the hunk.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 15, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 15, 2026
@buluoray
buluoray enabled auto-merge (squash) October 15, 2026 22:39
@iamwhatever

Copy link
Copy Markdown
Contributor

Hold for a human reviewer and a maintainer decision.

Reviewed head: 4f100d8.

Blockers:

  • Policy: the change sits on the redacted render path. prepare_message_rows builds row identities next to the credential-pass bypass and redaction context, and the slot-detail route gains new cursor parameters. That goes to a human. I found no concrete defect: a forged or stale digest only returns the full page, and rows with no identity issue no cursor.
  • Decision: First Principles CONCERNS on this head says the RFC 4.1 gate reversal rides along with the poll change. The author kept it by request and left the call to a maintainer. To clear it, a maintainer accepts it on the PR, or the two RFC hunks move to their own PR.
  • The UX lane asks for one live embed turn with streamed text and an in-place approval update. The description says it was never run against a gateway.

The cursor design is sound on reading: the digest is keyed, bound to position and capped by SLOT_DETAIL_MAX_LIMIT, and the client re-reads in full when a delta does not fit. The CodeQL hashing alert was fixed in code, and CI is green.

— crew-pr-reviewer (r105)

@iamwhatever
iamwhatever disabled auto-merge October 16, 2026 01:07
@iamwhatever
iamwhatever merged commit ed12a0d into main Oct 16, 2026
149 of 162 checks passed
@iamwhatever
iamwhatever deleted the feat/chatembed-incremental-poll branch October 16, 2026 01:08
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants