Skip to content

perf(dashboard): return new chat before full slot broadcast - #13499

Merged
bolichen97 merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/chat-create-latency
Sep 26, 2026
Merged

bolichen97 merged 1 commit into
kirodotdev:mainfrom
Pearcekieser:perf/chat-create-latency

Conversation

@Pearcekieser

@Pearcekieser Pearcekieser commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Creating an empty dashboard chat waits for the gateway to serialize and broadcast every open slot before returning the new slot. The requesting tab already inserts the POST response into local state, so this O(N) work delays first paint without helping that tab.

Why it matters

The delay grows with the number of open sessions. At 55 starting sessions, the original two-stage profile measured a 34.8 ms median handler time and 741 ms of handler CPU across 20 creates. The single-pass projection change from #12711 reduced the median to 17.6 ms. Moving the ordinary create broadcast after the response reduced it to 9.7 ms and 101 ms of handler CPU across 20 creates.

What changed (motivation → approach → change)

Path Behavior
src/kiro_crew/dashboard/, ordinary local New Chat Returns the complete new slot, then sends one full slot-list snapshot 10 ms later.
Metadata-bearing, app-owned, mode-specific, ephemeral, or remote create Keeps the full snapshot synchronous.
Any create error Keeps the full snapshot synchronous so existing failure propagation remains visible to the caller.

DashboardState.suspend_slots_push() yields an operation-only callback with a state-owned 10 ms delay. Existing callers ignore it and remain synchronous. A successful handoff cancels an armed trailing timer before the delay starts. The callback repeats that cancellation before publishing and starts the normal coalescing window. It transfers publication debt to any active suspension, while an overlap forces synchronous publication at depth zero. Failures log the slot name and retry once.

api_chat_slot_create() defers only a new local persistent-memory create whose body uses the known create keys and carries no folder, title, artifact, or mode metadata. New request keys fail closed to synchronous publication. The owning dashboard system specification records this ordering exception and the synchronous compatibility paths.

Backwards compatibility

Compatible: request and response shapes are unchanged, and all metadata-bearing, remote, app-owned, mode-specific, ephemeral, and error paths keep synchronous publication.

Tests

test/test_chat_slot_create_folder.py proves that the ordinary response precedes exactly one complete slot-list snapshot. The existing folder, pinned-title, ownership, and durable-write tests continue to pin synchronous ordering where it is required.

Focused validation covers 28 create-ordering tests and 91 broadcast-state tests. The green-age overlap suite adds 147 member-context tests.

Manual verification

The isolated aiohttp handler profile used 55 starting sessions, one owner WebSocket, and 20 measured creates. Median latency fell from 34.8 ms at baseline to 17.6 ms after the single-pass projection change and 9.7 ms after the deferred ordinary-create broadcast.

Related Issues

Fixes #13498

Pattern harvest

Rule candidate: review-prompt
Pattern: A create endpoint should not synchronously serialize an O(N) collection after its response already contains the created entity, unless that serialization is part of the endpoint's durability or failure contract.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (not applicable; no user-facing contract changed)
  • No secrets, credentials, or internal references in the diff

@Pearcekieser
Pearcekieser requested review from a team and iamwhatever September 24, 2026 19:41
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Intent: Return an ordinary local New Chat before serializing the full slot list, while preserving synchronous publication for paths whose durability or failure contract depends on it.
Not a goal: Change slot payloads, persistence semantics, remote creation, or metadata-bearing create ordering.

@github-actions github-actions Bot added fork Pull request from a fork (external contributor) readiness: checking Automated validation is still running labels Sep 24, 2026
@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5, fork) — 🟡 CONCERNS

Premise-level review of dba3087878ca02085e21af4054a9ba7595361f17 via the fork AI-review pipeline — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

_deferred_slots_flush rebuilds the existing trailing flush over the same coalescer state, plus a retry with no named failure it saves, for one caller.

Not justified as shipped

  • Item 4 — duplicate of src/kiro_crew/dashboard/state.py::_trailing_slots_flush: the state already has a callback that fires later, cancels/stamps _slots_broadcast_timer/_slots_broadcast_last, and calls _do_slots_broadcast (grep _trailing_slots_flush: 3 sites in state.py); the new method is a second spelling of it that additionally checks _slots_push_suspend and retries once — so the two trailing callbacks now diverge on suspension semantics in the same commit, and the retry's only exercised failure (the test's TypeError("not serializable")) fails identically 0.2s later. The docstring's stated reason for the bypass — keeping the exception out of "asyncio's generic callback handler" — does not hold: _deferred_slots_flush is itself a call_later callback.
    Subtraction: replace _deferred_slots_flush with arming the existing _trailing_slots_flush at _DEFERRED_SLOTS_FLUSH_DELAY_S (via _schedule_trailing_flush), deleting the retry and retries_remaining.
    Clears when: the deletion is made, or a named transient failure is shown that the retry recovers and the trailing path provably mishandles.

What this change ships

Inventory (8 items) — 7 justified

Intent: make New Chat's response return before the O(N) full slot-list broadcast (measured 17.6→9.7 ms median at 55 sessions, issue #13498) — a FIX of a measured cost.

  1. New Chat responds sooner; other windows see the new chat ~10 ms after the response instead of before it — justified
  2. Metadata, app-owned, mode, ephemeral, remote, and unknown-key creates keep the synchronous broadcast (fail-closed key allowlist) — justified
  3. A create-broadcast failure no longer fails the HTTP request; it is logged instead — justified
  4. New deferred-flush callback path with once-retry beside the existing trailing flush — duplicate of state.py::_trailing_slots_flush, retry has no named transient failure
  5. Overlapping suspensions force synchronous publication (_slots_push_overlapped) — justified
  6. suspend_slots_push yields an opt-in defer callback; 4 of 5 existing call sites ignore it unchanged — justified
  7. Dashboard spec section rewritten for the ordering exception, same commit — justified
  8. One-broadcast guard test rewritten to pin response-first ordering; the "must not drop the announcement" pin is still asserted — justified

[FIRST-PRINCIPLES-REVIEWED] dba3087

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5, fork) — ✅ PASS

Design-level review of dba3087878ca02085e21af4054a9ba7595361f17 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify in a two-window session that rapid plain creates show no flicker in the non-creating window during the 10 ms handoff.

[DESIGN-REVIEWED] dba3087

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review (fork) — ✅ no blocking findings

Reviewed dba3087878ca02085e21af4054a9ba7595361f17 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] dba3087

@github-actions

github-actions Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review (fork) — ✅ no blocking findings

Reviewed dba3087878ca02085e21af4054a9ba7595361f17 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] dba3087

@Pearcekieser
Pearcekieser force-pushed the perf/chat-create-latency branch 2 times, most recently from 4042fc9 to f139558 Compare September 24, 2026 22:45
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: state-owned deferred slot flush delay

fixed at f139558: removed the caller-selected delay; the callback now accepts only an operation label and state owns the fixed 10 ms interval.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: documented plain-create publication exception

fixed at f139558: updated the owning dashboard system specification with the deferred plain-create path and every synchronous compatibility path.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: fail-closed create-body key allowlist

fixed at f139558: deferral now requires the complete known create-key set; an unknown key stays synchronous, with a regression test.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

rebutted: this is conditional on future evidence that the 10 ms timer loses the response race. The measured handler profile and current ordering test show the intended result; adding a post-write hook without a reproduced failure would exceed this latency fix.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: pre-handoff trailing-timer cancellation

fixed at f139558: a successful handoff cancels and clears the armed trailing timer before the fixed delay; the callback repeats the guard for timers armed during the delay.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

fixed at f139558: the handler comment now distinguishes request-visible pre-handoff errors from logged, once-retried delayed publication failures.

@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in import-overlap files since this head went green (c08b4e2 -> 958e43f); overlap-scoped tests passed and CI is re-running.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 24, 2026
@Pearcekieser
Pearcekieser force-pushed the perf/chat-create-latency branch from f139558 to 6993504 Compare September 24, 2026 23:52
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in related docs and import-overlap files since this head went green (958e43f -> a992d5c); 532 overlap-scoped tests passed and CI is re-running.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 25, 2026
@Pearcekieser

Copy link
Copy Markdown
Contributor Author

chore (non-blocking): retrigger incomplete Opus review

Opus 5 Review on 6993504ab1c0 ended as review incomplete and posted no finding, review comment, or stable span. Backend Lint ended with exit 143 after the runner received a shutdown signal; all Linux and Windows test shards passed. git merge-tree --write-tree origin/main HEAD is clean, and no workflow on this branch changed within the last 10 minutes. This is the authorized same-tree retrigger; the source tree is unchanged.

Defer the coalesced full-sidebar snapshot by 10 ms only after a successful ordinary local chat creation. Keep metadata-bearing, app-owned, mode-specific, remote, and error paths synchronous.
@Pearcekieser
Pearcekieser force-pushed the perf/chat-create-latency branch from 6993504 to dba3087 Compare September 25, 2026 21:18
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 25, 2026
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 25, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 25, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed as part of a full sweep of open PRs targeting main (2026-09-26 06:40Z).

Verified on the current head: readiness: passed label present; the sole required status check (PR Readiness, per ruleset 20088190) is green; no pending, failing or cancelled required check; mergeable; no live CHANGES_REQUESTED; and no unresolved blocking change request in any review thread (every remaining review-lane comment on this head is PASS or advisory-only, and any earlier finding is either answered by the author or pinned to a superseded head).

Approving on that basis and enabling GitHub native auto-merge. No admin override, no bypass of branch protection — this still waits for every required check and for code-owner review.

@bolichen97
bolichen97 merged commit 0c0249a into kirodotdev:main Sep 26, 2026
97 of 99 checks passed
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fork Pull request from a fork (external contributor)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reduce ordinary New Chat create latency

2 participants