feat(chat): per-reply timing and usage readout, and whole-session usage in the composer - #458
Open
molicherry wants to merge 4 commits into
Open
molicherry wants to merge 4 commits into
molicherry wants to merge 4 commits into
Conversation
molicherry
force-pushed
the
feat/per-reply-timing-footer
branch
2 times, most recently
from
September 17, 2026 09:14
2f00b82 to
af9b798
Compare
…eply The runtime already anchors the provider request (requestStartedAt) and the start of the stream (streamStartedAt). This adds the two anchors a first-token latency needs: requestIssuedAt, stamped inside the stream function the moment the provider request goes out, and firstTokenAt, stamped on the first non-empty text or thinking delta. The wait is emitted as an optional responseFirstTokenMs on the assistant message end, next to responseDurationMs, and mirrored into host-core's UiMessage plus the transcript metadata so a reloaded session keeps it. It is additive optional metadata: no protocol or storage-schema change.
A completed reply now carries one muted line under it: the turn's usage total, its elapsed time, and the local completion time. The usage segment and the elapsed segment are separate triggers, and each opens its own card: the usage card lists provider and model, the cache-hit rate, uncached input, cache read, cache write when the turn wrote any, output and the generation rate, and the elapsed card carries the elapsed duration and the first-token latency. A prompt's own send time sits under the sent message instead of under the answer. The turn owns those numbers rather than the fragment that happens to carry text: the timing values read every assistant row of the turn, so a first reply that streamed only reasoning still supplies the first-token latency, and the usage total counts cache write — the same four fields the composer's session summary adds up. A partly estimated turn divides the runtime's summarized count, and a stopped reply with no usage bucket keeps its estimated rate. The clock is the locale's, whatever hour cycle it uses. Everything is read from that turn's own persisted messages, so the readout survives a reload. The line opens on click alone, its text is selectable, and each card has its own copy control. The component spec, the IPC message mirror, the E2E scenario catalog, the locales and the presentation tests are updated with it.
The composer's usage panel only ever mirrored the newest turn. It now carries a Session row above the provider row: the session's completed-turn count, total, input, output, cache read and cache hit rate. The transcript is a paged window, so the renderer cannot reconstruct a session total — host-core aggregates the session's completed turns (new additive session.getUsage RPC, pi-desktop/session/getUsage IPC, api.getSessionUsage) and the composer refreshes it when the active session changes and once a run goes idle. The row stays absent for a session with no completed turn, and the aggregate is bound to the session it was read for, so switching to a session that is already running never shows the previous session's numbers. Session totals count completed turns only and define the total as input + output + cache read + cache write, matching the existing token-history aggregation and the reply's own usage card. No protocol version, storage schema or migration change; per-turn usage stays under each reply. Upstream took D431 and D432 while this branch was open, so the branch decisions are numbered D433 (the timing readout), D434 (per-reply usage and speed) and D435 (these session totals).
Main claimed D443-D446 while this pull request was open, so the three decisions this branch adds move past them: D447 for the per-reply timing readout, D448 for the per-reply usage and speed readout, and D449 for the composer's session totals. Only this branch's references move; upstream's own D443-D445 citations stay as they are.
molicherry
force-pushed
the
feat/per-reply-timing-footer
branch
from
September 18, 2026 06:35
af9b798 to
209d3f2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
wait to first token is measured as
responseFirstTokenMs, persisted next toresponseDurationMsasadditive optional message metadata (host-core mirror + reload-safe transcript metadata).
and the local completion time. The usage and the elapsed segments are separate click-only triggers, each
opening its own card (usage: provider/model, cache-hit rate, uncached input, cache read, cache write when
the turn wrote any, output, generation rate; elapsed: duration and first-token latency). The timing values
read every streamed row of the turn, so a reply that streamed only reasoning still supplies its
first-token latency; the total counts cache write, the same four fields the composer's session row adds
up; a partly estimated turn divides the runtime's summarized count, and a reply with no usage bucket keeps
its estimated rate. Text is selectable and each card copies.
Sessionrow above the provider row (completed turns, total,input, output, cache read, cache hit rate) behind a new additive read-only
session.getUsageRPC(
pi-desktop/session/getUsage). The transcript is a paged window, so the renderer cannot sum it; theaggregate is bound to the session it was read for, so switching to a running session never shows the
previous session's numbers.
Decisions and specs
D447 (the per-reply timing readout), D448 (per-reply usage and speed) and D449 (composer session totals)
in
docs/spec/08-meta/decisions-log.md; the component spec, the IPC message mirror and the E2E catalogcarry E2E-CHAT-per-reply-timing-readout, E2E-CHAT-per-reply-usage-readout and
E2E-CHAT-composer-session-usage-totals. No protocol or storage-schema change; the metadata and both RPCs
are additive. Main claimed D443-D446 while this branch was open, hence the numbering (D431 and D432 went to
other decisions earlier in the same window).
Validation
Re-run after rebasing onto
5f0f38d7(the currentmain):cargo test -p host-core --lockedmcp_servers,user_skills) that exist verbatim onmain; this branch touches neither pathpnpm build:js, desktoptsc --noEmit,pnpm lintJS build / typecheck / lint / architecture / test: success)docs:checkpnpm test:e2e:boot/:transcript/:transcript-disclosurepnpm test:e2e:layoutdarwinpreview-header geometry cases (theirwin32/linuxcounterparts pass) and a seeded project-group fixture chain; none of the branch's files are in that surfacetest:e2e:layoutalso needs an empty~/.pisessions directory on the running machine: with ≥14 real pisessions the capture fixture bails at
capture-rig.ts:1064and the suite times out before its first check.Reproduced identically on pristine
main.