[duytbp]

Change

This is the history-first step requested in #2610, cut down to the smallest move after the review. Only the bulk reads of server/claude-command-history.ts change. Everything else is main's code.

• Reverted to main exactly: claude-subagents.ts, claude-task-scan.ts, file-tasks.ts and their tests. claude-task-state.ts, claude-task-history.ts and their tests are gone, along with the proof/revision/pending overlay and the task-scan retry. Also reverted: ci.yml, remote-bundles.yml and scripts/remote-bundle-smoke.ts. The remote bundle copies server/ whole, so it carries the new file without them.
• Kept as on main: the synchronous API (historyVersion, sameHistory, commandHistory, evidence, vouchedEnds, waitCommandHistory and the rest). vouchedEnds still rechecks known ends against a grown transcript synchronously, so an ordinary append keeps an ended command ended and Claude pane reads working while only its background subagents run (title spinner after end_turn) #2069 holds. fix(claude): recheck a same-size history rewrite within one timestamp tick #2620's parts are main's code too, merged in unchanged: evidence's 3 s rule and its start over on a rewritten line, since, the shared recheck per tick, and vouchLines' coalesced, budgeted re-reads. They stay synchronous on readSync, on the request path.
• Changed: the drain's scan and check read through transcriptReader (server/transcript-reader.ts) instead of calling readSync themselves. On network filesystems and on macOS and Windows, the reader reads through the thread pool. On local Linux filesystems it still reads synchronously inside the slice (below). The drain still runs one slice at a time, with a setImmediate between slices, except when a slice is stuck (below).
• Local filesystems (the round after the review of c5103d0): a call through the thread pool costs more CPU than a read the page cache answers: a wake of the pool thread, a copy on another core and a wake of the event loop. So on Linux, a transcript whose directory is on ext2/3/4, btrfs, xfs, tmpfs, zfs, f2fs or bcachefs (statfs type, LOCAL) is opened, statted, read and closed on the event loop, inside the same lease, slices and turns.
  • FUSE (SSHFS, virtiofs), NFS, SMB, 9p (WSL's Windows drives), overlayfs and any type not listed keep the thread pool. So do macOS and Windows.
  • The statfs itself is async, since it can block on a hung NFS mount. It is cached for 60 s per directory, and a lease that has to look it up opens the transcript at the same time, so a network filesystem pays no extra round trips before its open.
  • Any call on the event loop that takes over 2 ms is taken as answered by the disk, not the page cache. The rest of that lease and that transcript's leases for the next second then go through the thread pool, with the read ahead. A 1 MiB read from the page cache takes about 0.2 ms here. The open and the close on the event loop are timed the same way, and a call that throws counts as slow too.
  • The lookup also takes the directory's device (stat beside the statfs), and the lease's first stat compares the transcript's own device with it before any read. A transcript on another device, such as a file bind-mounted from NFS or FUSE into a local directory, goes through the thread pool, and its later leases do too for 60 s.
  • The reader tells the drain whether the event loop ran while the lease's last call waited (waited()). A slice reading on the event loop counts from the start of its step and stops at its 4 ms, as main's does; reads through the thread pool keep the rules above.
  • No read ahead starts while reads are on the event loop. Every byte taken in is still read between two equal stats, from fstatSync on the same descriptor. Cancellation and the deadline are checked before and after every call.
• Slices:
  • A slice ends after 4 ms of wall time from its start, as on main, so a large session on slow storage hands over after each read.
  • Parsing a chunk that arrives after the slice is up through the thread pool stops 4 ms after its bytes arrive, not 4 ms after the slice started. On main the read is inside the slice. On storage where one 1 MiB read takes longer than that (SSHFS), the parse stops after a single record and the next slice reads the same chunk again from there. A chunk read on the event loop gets no such extension, since the loop had no turn while it was read. A slice on the event loop counts from the start of its step, as main's does, so its open and first stats come out of its 4 ms.
  • When nobody else is waiting, the session keeps its transcript open into the next slice. Opening a transcript and checking it is the path it was let in for costs several round trips on remote storage. It lets go at the end of the slice in which nobody watches or waits for the history any more, another session that can have a slice is queued, another reader waits for the reader, the read starts over, or a watcher has a new end to hear. It also lets go once half the read's 10 s deadline has passed, counted from when the read was asked for. A request for the same session does not end it. In production both readers keep it: the subagent tracker watches the history while it needs it (watchCommandHistory), and the task scan waits for it (waitCommandHistory).
  • A turn inside a held lease waits for two setImmediates, not one. An immediate set while an I/O callback runs comes before the timers already due. With one, a slice that needs no read (the rest of a window, below) could follow the last slice before them: 44–51 event-loop stalls over 8 ms per dense read in profiling, against 0–1 with two.
• Read-ahead and reuse (kept, re omo M1): they carry the thread-pool path that network filesystems take, which the fast path doesn't touch. Over SSHFS, without them the 276 MB sparse transcript took 9.55 s and the 95 MB dense one 9.2 s; with them, 5.1–5.6 s and 2.7–2.9 s.
  • A lease can start one read ahead into a second buffer, beside the operation it awaits. After a window is read, the next window, from that window's last whole line, is read while the stat after it is on its way and the window is scanned. A window() that asks for those bytes gets them with no second read. One that asks for other bytes lets the read ahead settle first. A lease has at most two native operations in flight, so the reader has at most four.
  • The check reads the next span ahead the same way, while it hashes the current one.
  • No read ahead starts when nobody listens, since the read then ends with that slice and a read ahead left over would hold up the lease's close. None starts when the window already reached the end, since there is nothing past it to read yet.
  • A slice that ends inside a window leaves that window's whole lines for the next slice. The next slice takes them in with no read and no stat, as long as no stat came since the one after they were read. On the dense transcript this cuts the reads from about three times the file to once.
  • Every byte taken in is still bracketed by two equal stats: a read ahead starts after a stat equal to checked, and reused bytes come from a read that did. A rewrite is still caught by the next stat or check.
• A stuck transcript holds up only its own history (the review's HIGH):
  • A slice is taken as stuck when it has neither ended nor taken a turn for 50 ms (STALL_MS) while another session that can have a slice waits. The drain leaves it to end by itself and starts the next session's slice beside it.
  • Turns are counted per session. A held lease that takes its turns is reading, not stuck, and lets go at the end of its next turn, so healthy storage still runs one slice at a time.
  • At most the reader's capacity (2) of slices run at once, one per transcript. A held lease lets go only for a session that can have a slice, not for one whose transcript a stuck slice still reads.
  • A healthy lease's open and its checks on remote storage, several round trips, can outlast 50 ms too. The next slice then starts beside it.
• Cancel on the last release (the review's first MEDIUM): resetCommandHistory() aborts the reads of every slice begun before it. The slice gives up at once, and the drain ends. What it cannot do: the native operation already started keeps its reader lease until it settles, since a syscall cannot be interrupted, and a new read of that transcript waits for that lease. commandHistorySettled() resolves once no slice runs and none waits to start. commandHistorySize().draining counts the slices left running too.
• Guards the async reads need:
  • A stat is compared with the transcript as it was known when the stat was asked for. A stat older than one a status poll took in between is no news. A transcript cut short in place, even back to the size a poll saw, is read anew.
  • A chunk's offset, ends and checked range are published together, after the stat that follows the read. A request in between sees the read as it was.
  • A read that gives up on that stat (deadline or busy) keeps the chunks it already read and checked.
  • A session asked for again while its slice waits is queued once.
  • If the path names another file by the time the reader admits it, the read starts anew, as main's stat would.
  • Since fix(claude): recheck a same-size history rewrite within one timestamp tick #2620, a request that finds an end's line rewritten starts the history over (evidence) while a slice may be waiting for the disk. The slice drops the window or check it was reading then and reads again from the start. Without this, it took in a window read from the old position as the transcript's first bytes, until the next request or check corrected it.
• Busy reads: a TranscriptBusyError (reader queue full, or a read past its 10 s deadline) backs off without using the failure budget (CodeRabbit thread on claude-command-history.ts:98). Other errors count as on main.
• Reader: one shared in-process reader, for this history now and the image path later. It uses plain stats, the same form lstatSync gives, so its versions compare with historyVersion. window() reads into one of two 1 MiB buffers per lease, taken from a pool of the buffers of leases that ended. A buffer goes back to the pool only once every native read into it has settled, a read ahead included. The deadline timer does not keep the process alive.

Known limits

Not fixed here; each is written in the code's comments:

• A local disk that stops answering inside a call on the event loop blocks the loop, as main's readSync does. The fallback notices a slow call only once it returns. Network filesystems can't hit this, since they never take the fast path.
• A network file bind-mounted into a local directory is opened and statted once on the event loop before its device gives it away, once per 60 s for that path. The open is timed, so a slow one also sends it to the thread pool.
• macOS and Windows have no fast path, so their local reads keep the thread pool's CPU cost. On macOS, statfs gives no fixed type for APFS; comparing the transcript directory's type with that of / should work, but I haven't tried it on a Mac, so it's left for a follow-up.
• A native operation that never settles keeps its lease. Two of them, on two paths, keep both of the reader's leases, and every later read waits out its 10 s deadline. A read ahead counts as one.
• A cancelled lease keeps its path until its operations settle. A new read of that transcript waits behind it.
• Transcript identities use number stats (dev:ino, as lstatSync gives them). On a filesystem whose numbers pass 2^53, two files may share an identity.
• A transcript whose reads keep running into the 10 s limit is tried again every second while it is watched, with no longer backoff. A busy reader is no failure of the transcript.

Tests

Every assertion from main's claude-command-history.test.ts is unchanged: against main, that file changes only its imports. At the review's request (the second MEDIUM), the assertions on lease and turn counts in tests this PR added earlier now assert outcomes:

Test	Asserted	Asserts now
keeps a transcript open across turns while no other session waits	1 lease	each open costs 100 ms of a controlled clock, and the read took less than its reads plus two opens
lets go of a transcript it keeps open at the end of the turn another session is asked for in	small answered at read 3; 2 leases	small answered with its end while the large one still had reads to go
lets go of a transcript it keeps open well before the read's deadline	2 leases	the clock moves 5 s at read 2 and 5 s more at read 4; the read still comes back complete in one wait (a lease held past half its deadline would run out of time and give up)
gives each turn of a transcript it keeps open a slice of its own	fewer setImmediate calls than reads / 2	each turn's held time on a controlled clock: at most 5 ms, and every turn but the last at least 2 ms
reads on in the same lease when the session it reads is asked for again	1 lease at each read	at its last read, the clock shows its reads plus exactly one open
reads one slice for a request nobody watches or waits for	polled up to 1000 event-loop turns; 1 read, 1 lease	waits on commandHistorySettled(); read on some way but not through, nothing left queued, draining or timed
reads on in one lease for a tracker's watch after it hears the first end	2 leases	the watcher hears the first end before the read is through
lets go of a transcript it keeps open at the end of the turn it started over in	heard before read 8	heard while the restarted read is not through
counts the time it keeps a transcript open from when its read was asked for	2 leases	the clock moves 6 s before the lease opens and 5 s more at read 3; the read still comes back complete

Merging main with #2620 changed two of this PR's own tests (none of #2620's, none of main's), because of #2620's rules:

• "lets go of a transcript it keeps open at the end of the turn it started over in" rewrote the first record. Under fix(claude): recheck a same-size history rewrite within one timestamp tick #2620 the next request's evidence finds that rewrite first and starts over before any slice, so the turn-break went untested. It now rewrites the record after the end's line, so only a check finds it, and its ends assertion is ["a"], not ["b"]. Dropping || restarted from the turn-break still fails it.
• "keeps the chunks it read and checked when a read gives up on its stat after a chunk" now sets its file's mtime a minute back. Read within 3 s of the write, it got fix(claude): recheck a same-size history rewrite within one timestamp tick #2620's shared recheck, 5 more windows. No assertion changed.

Every other assertion of those tests, and of every other test, is unchanged.

This round also moves the tests off real time (Terra LOW):

• The reader's event-loop tests run on a clock only they move (stillClock()), so a preempted read can't cross the 2 ms bar. 48 parallel runs of transcript-reader.test.ts beside 64 CPU spinners: 42 of 48 passed on 93f74fb, 48 of 48 now (also 48 of 48 with 16 spinners).
• The test helpers that stand in for slow or held storage (slowReads, holdWindows) now say their reads waited off the event loop, and the one that answers at once says they didn't, so the tests behave the same whatever filesystem the host's /tmp is on.
• "reads another transcript while one transcript's read is stuck on its disk" and "starts no more slices beside stuck ones than the reader has room for" used a real 150 ms wait. They now move a fake clock past the 50 ms stall instead.
• The two reader tests that spun on Date.now() past a deadline now set the system time inside the callback, or while the close is pending.
• "reads on by whole chunks when every read outlasts a slice" did not catch its fix: with the parse stop counted from the slice start it still passed, at c5103d0 too, because reusing a window's rest kept its read count in bounds. It now runs on a clock that moves only with the reads, and also asserts that the ends known each time the event loop runs grow by more than 100 at a time (a chunk holds about 450), not a record at a time. With the fix undone it fails.
• "reads no other session beside a transcript it keeps open that takes its turns, however long it reads" no longer intercepts every 50 ms timer. The small session now asks between two turns of the large one, on a fake clock. With the old timing it passed with its fix undone, at c5103d0 too; it fails now.

Each test below fails with its fix undone; I undid each fix and ran the files:

Test	Fails when
backs off a read the reader is too busy for…	a busy read counts as a failure
keeps the chunks it read and checked when a read gives up on its stat…	offset and ends are published before the stat
queues a session asked for while its slice reads only once	the requeue guard is removed
keeps what it read when a slice's stat is older than one a poll took…	an older stat discards the read
starts over on a transcript cut short in place while a slice reads it	a stat matching the poll's version is accepted though the transcript shrank
reads on by whole chunks when every read outlasts a slice	the parse stop counts from the slice start
lets a small session have its turn after each slow read of a large one	the slice counts only CPU time, or the queue is not checked
never keeps the process alive for a read's deadline alone	the deadline timer is not unref'd
keeps a transcript open across turns while no other session waits	each slice ends the lease
lets go of a transcript it keeps open at the end of the turn another session is asked for in	the queue is not checked
lets go of a transcript it keeps open at the end of the turn another reader waits for it in	the reader's waiting count is not checked
lets go of a transcript it keeps open at the end of a turn that told a watcher of an end	a new end for a watcher is not checked, or a waiter counts as a watcher
lets go of a transcript it keeps open well before the read's deadline	the half-deadline is not checked
reads on in the same lease when the session it reads is asked for again	any queued session ends the lease, or each slice ends it
gives each turn of a transcript it keeps open a slice of its own	the slice is not renewed after a turn, or there is no turn
reads one slice for a request nobody watches or waits for	a read with no listener is kept open
reads on in one lease for a tracker's watch after it hears the first end	a new end for a watcher does not end the lease
lets go of a transcript it keeps open at the end of the turn it started over in	a start over does not end the lease
counts the time it keeps a transcript open from when its read was asked for	the half-deadline counts from when the lease opened, or is not checked
takes in the rest of a window a slice ended in without reading it again	the rest of a window is read again
reads another transcript while one transcript's read is stuck on its disk	the drain waits for a stuck slice
reads no other session beside a transcript it keeps open that takes its turns, however long it reads	a turn does not count as progress
starts no more slices beside stuck ones than the reader has room for	both capacity checks are removed, the drain's and next()'s (either alone holds the bound)
gives up a read stuck in flight once the last server lets go	a reset does not abort the slice's read
reads ahead while the lease awaits a stat, and hands the bytes to the window that asks for them	the buffers are not swapped, so the next read ahead overwrites the window just returned
lets a read ahead settle before a window that asks for other bytes	that window does not wait for the read ahead
closes a lease and gives its buffers to another only once a read ahead nobody asked for settled	the lease does not track its read ahead
ends a cancelled lease's read ahead without an unhandled rejection	the lease does not track its read ahead
drops a window read across a start over that a request made while the window waited for the disk	the slice takes in a window read across a start over
reader: reads a transcript on a local filesystem on the event loop, and one on any other through the thread pool (btrfs, ext4, tmpfs; NFS, FUSE, 9p)	every filesystem counts as local, or there is no fast path
reader: reads through the thread pool for a second once a read on the event loop was slow, then tries the event loop again	a slow call doesn't switch the lease, or the transcript's second isn't checked
reader: gives up a read on the event loop between windows once cancelled	the cancel checks around a sync call are removed
reader: opens a transcript while its filesystem is looked up, and reads it on the event loop once that says local	the open waits for the lookup, or the lease stays on the thread pool after it
reads another transcript while one on a network filesystem is stuck in its native read	every filesystem counts as local, so the stuck read blocks the event loop; or the drain waits for a stuck slice
holds the event loop no longer than a slice and the call that ended it when it reads on the event loop	a window read on the event loop after the slice's end gets a slice of its own, or the first slice counts from when the work starts instead of from the step's start, or the reader says every call waited
reader: reads through the thread pool for a second once an open on the event loop was slow or threw	a slow open or one that throws isn't counted, or the lease whose open was slow reads on the event loop
reader: reads through the thread pool for a second once a close on the event loop was slow or threw	a slow close or one that throws isn't counted
reader: reads a transcript on another device than its local directory through the thread pool	there is no device comparison, or a mismatch doesn't mark the path
reader: reads through the thread pool for a second once a read on the event loop threw	a call that throws isn't counted
reader: tells the work whether the event loop ran while the lease waited	a queued lease counts as not having waited, a call on the event loop doesn't clear it, or a call through the thread pool doesn't set it

Three tests have no fix to undo:

• "lets the event loop run while a slice waits for the disk" checks that the drain yields while a read is held. Main's sync read cannot be held.
• "starts over on a transcript rewritten and grown while it reads a window ahead" guards the read-ahead against a rewrite. Main reads no window ahead, and passes it too.
• Removing the read ahead's own .catch does not fail "ends a cancelled lease's read ahead…": the lease's operation bookkeeping also handles that rejection.

Four lines have no unit test:

• The 2 ms bar's value. The tests inject slow calls far past it; where it sits between a page-cache read (about 0.2 ms per MiB) and a cold NVMe read (1.9 ms typical) is a measurement, below.
• A chunk that arrives within a slice is parsed until that slice ends, not for 4 ms from its arrival. It bounds event-loop delay, and none of the tests' clocks shows a difference. Without it, event-loop p95 on SSHFS rose to 14 ms in the profiling run.
• The second immediate of a turn. It orders the turn after the timers already due, which no test clock shows. Profiling measured it (see Slices).
• A check pass whose check a start over cleared returns at once. Without it, the pass ends on the emptied spans and marks offset 0 checked, which the next scan's start overwrites before any end can be told, so no test can see it. It is there so check and scan treat a start over alike.

One of #2620's tests sees less here: "shares one recheck within a version's tick among the requests that wait…" counts fs.readSync, which on this branch is only vouchLines; the drain reads through the reader. Its assertion holds. A probe wrapping the reader's window() around the same five waits found no drain reads, as on main.

The inherited same-millisecond matching issue is out of scope.

Possible follow-up, not in this PR: waitCommandHistory and commandHistory still call lstatSync through historyVersion, as on main. On SSHFS that holds the event loop about 100 ms per call. #2620's vouchLines is the same kind of read: synchronous, up to 4 MiB per call, run whenever the history is not vouched, which since #2620 includes any transcript checked within 3 s of its mtime. Moving either off the request path is a change to main's synchronous API.

Validation

• Toolchain: Bun 1.4.2, Node 22.23.4, herdr 0.9.3. The head is 7edf4df: c5103d0 (which merged main 05c9133) plus the test rework, the local-filesystem fast path, the changelog, opening during the filesystem lookup, the slice clock on the event loop, two commits strengthening one test, the reader tests' clock, and timing the open and close with the device check. The full benchmark below ran on b9b2acd; a local run on 7edf4df is after it.

• bun run check fast on 7edf4df: 10755 passed, 0 failed, 41 skipped in 663 files; workflow syntax, generated types and themes, typecheck and build all ok. It includes fix(claude): reconcile background commands after cold history reads #2608's tests (claude-subagents, claude-task-scan) and file-view and file-pool, which use file-tasks.ts.

• server/transcript-reader.test.ts and server/claude-command-history.test.ts: 82 passed, 0 failed. The history tests, 8 copies at once, 3 rounds: 24 of 24 passed.

• GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=core.excludesFile GIT_CONFIG_VALUE_0=/dev/null bun run check run bun test server/claude-session.contract.test.ts server/api.contract.test.ts: 207 passed, 0 failed, on an isolated herdr 0.9.3.

• git diff upstream/main -- server/claude-subagents.ts server/claude-task-scan.ts server/file-tasks.ts server/claude-subagents.test.ts server/claude-task-scan.test.ts .github scripts is empty against 05c9133.

• The review's cases ran as synthetic scripts against main and c5103d0 with the same output on both (listed in the earlier round). The review's repro, a held read of A while B is asked for, is the test "reads another transcript while one transcript's read is stuck on its disk", and its network-filesystem twin covers the stuck native read.

• Measured with the same synthetic benchmark as before (gist below): b9b2acd against main 05c9133, interleaved, median [min–max]. The host also runs a VM using about 8.5 cores, so load was 9–13 and every sample is flagged; treat local differences under ~20% as noise, and CPU as the steadier figure.

• Local disk, 5 runs each. Ratios are this branch over main per interleaved pair, so <1 is faster:

  	main	this branch	ratio
  276 MB sparse, page cache warm	65 ms [57–70]	74 ms [66–79]	1.15 [0.96–1.34]
  276 MB sparse, cold	158 ms [154–189]	164 ms [146–203]	1.02 [0.85–1.28]
  95 MB dense, warm	1027 ms [961–1166]	949 ms [915–1124]	0.94 [0.89–0.96]
  95 MB dense, cold	854 ms [759–994]	819 ms [735–854]	0.93 [0.86–1.03]
  CPU (user + sys), sparse warm	64 ms	90 ms	1.25 [1.06–1.40]
  CPU, sparse cold / dense warm	157 / 1172 ms	190 / 1104 ms	1.22 / 0.96
  longest event-loop gap, sparse warm / dense warm	4.7 / 7.3 ms	4.7 / 7.3 ms
  other requests during the sparse read, p95	4.1 ms	4.1 ms
  1 MB session, warm / cold	5 / 5 ms	8 / 7 ms	1.60 / 1.63

  Sparse warm CPU is the figure the last review blocked on: 78 ms on main against 165 on c5103d0 in your run. Three full runs across this round's commits on this host put it at 1.16, 1.10 and 1.25 times main's (fd22a07, b994f29, b9b2acd), against 1.48 for c5103d0 in the same harness here. The branch's own sparse warm CPU held at 77–93 ms across those runs; main's moved between 64 and 88, which is most of the spread in the ratio. What remains is mostly JIT time for the extra code paths: in a profile of the second read in one process, main-thread work matched main's within 2 ms, and the pool threads did none. Sparse warm read time overlaps main's in every run (1.02–1.15).
  Cold sparse costs ~20% more CPU than main: on a cold read the first slow call (over 2 ms) moves the rest of the lease to the thread pool, which is what keeps a local disk that stalls from blocking the event loop for more than one call.
  The longest event-loop gap now matches main's. On the fast path, slices take only what fits in their 4 ms, as main's do. Loop-gap max for the 1 MB session reads 4.6–4.8 ms against main's ~1 ms, but that is how the harness samples, not a longer block. The sampler records a gap only when its 2 ms tick fires before the read resolves, so a read that ends on sync work never has its last gap counted. Main's small read usually ends inside its first gap. One main run that got a second tick recorded 5.4 ms. A probe that also counts the last gap found the longest block at 4.3–5.1 ms on main and 3.3–5.4 ms here: parsing the 1 MiB window inside one slice.

• Local disk on 7edf4df against main 05c9133, 5 runs each: sparse warm 61 → 67 ms (ratio 1.15 [1.04–1.62]), CPU ratio 1.22 [1.13–1.68]; dense warm 0.95, dense cold 0.86; longest event-loop gap, sparse warm 4.6 / 4.7 ms and dense warm 6.6 / 6.4 ms. Sparse cold CPU came out 1.39 [1.07–1.76] in this run (1.22 on b9b2acd); a cache miss now also stats the directory beside its statfs, and every bench read is a fresh process, so each pays it once. I haven't separated that from load; your spot-check on a quiet host will show it better.

• Over SSHFS: sshfs 3.7.6 with cache=no,direct_io,no_readahead, 14 ms round trip, 3 runs each. These are unchanged from c5103d0; network filesystems never take the fast path. An interleaved SSHFS run of c5103d0 against b9b2acd's parent gave ratios of 1.00 and 1.02 for the small-session rows.

  	main	this branch
  276 MB sparse, total	61.6 s [61.5–61.7]	5.18 s [5.14–5.20]
  other requests during it, p95 / max	155 / 275–345 ms	1.0–1.8 / 101–103 ms
  longest event-loop stall	177–247 ms	101–102 ms
  95 MB dense, total	not done after 300 s, 3 of 3 (1.0 MB read)	2.58 s [2.54–2.71]
  other requests during it, p95 / max	153 / 274–276 ms	33–42 / 100–102 ms
  1 MB session alone	7.75 s	0.49 s
  1 MB session while the 276 MB one reads	15.3 s	0.47 s
  1 MB session while the 95 MB one reads	15.4 s	0.48 s
  CPU, sparse (user + sys)	0.36 s	0.31 s

  Main sets the slice's 4 ms clock before its blocking read, and one 1 MiB read over SSHFS takes ~110 ms, so each slice parses about one record and the next slice reads the same chunk again. This branch's single ~100 ms stall per read fits one round trip of the synchronous lstatSync in historyVersion (see the follow-up).

• System calls on the transcript, page cache warm, under strace:

  	main	this branch
  dense: reads / MiB read / opens	668 / 422.7 / 341	91 / 90.6 / 2
  sparse: reads / fstats / opens	286 / 573 / 35	282 / 283 / 2

  Main with fix(claude): recheck a same-size history rewrite within one timestamp tick #2620 reads the dense transcript about 4.6 times over; this branch reads it once.

• A small session's extra ~3 ms is once per process: starting Bun's thread pool for the first statfs and open of a directory, and the first run of the reader's code paths. After that, on a local filesystem, a read makes no thread-pool round trip, and compiled it is within noise of main's (2.3 against 2.1 ms in the profile).

• Not run locally: the integration and browser suites, SSH, native Windows and macOS. Upstream CI covers them.

• The benchmark is a standalone harness anyone can run: https://gist.github.com/duytbp/470e90421331b31ccd953de200d6aae0 (synthetic transcripts only; bench.sh <rev> for local, MODES=sshfs adds the SSHFS lab, which needs Linux and sudo).

• Optional follow-up for the owner: adding server/transcript-reader.test.ts and server/claude-command-history.test.ts to the Native Windows and remote Windows test lists. That is a workflow change, so it is left out until approved.

All fixtures are synthetic files in temporary directories. No live herdr session or real transcript was used.

[duytbp]

Independent agent review covered the frozen implementation separately from its author; it was not a CodeRabbit review. All concrete findings were fixed before this push:

• Resource ownership: cancellation/deadlines retain physical capacity and descriptors until native settlement/close; a queued hot path cannot consume the spare lane needed by a healthy path. Caller-local deadlines do not poison shared storage backoff.
• Task authority: reparse local state after the history wait, check the actual current version, and vouch partial/changed-version terminal lines before applying them. An unverified timeout candidate cannot close a shell, including after structured cloning. The former single-snapshot implementation fails the new freshness regression.
• Tracker lifecycle: stopped/replaced trackers abort owned reads; process-start discovery reclassifies old launches; historical completion reuse requires provenance and the exact verified parent version. Changed versions use local pending/running state until proof, rather than retaining unchecked terminals after a rewrite.
• Continuation and precision: a superseded current proof triggers one fresh poll without a stable-error retry loop; releasing/reacquiring a requeued history watch cannot wedge the scan; exact-millisecond clocks avoid bigint conversion double-rounding. The queue regression fails when the old queue flag is restored.

The final semantic review independently passed all 225 focused tests and the external process-start, parent replacement/truncation and rewrite-plus-growth reproductions. The final resource/platform delta review passed 58 focused tests. Root validation results are in the PR description. Windows execution remains an upstream CI gate; native shared-pool and stuck-I/O limitations are explicit there.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

• Source mode: Auto
• Repository: devswha/ovis
• Branch: main
• File: .coderabbit.yaml
• Commit: 95b3c10
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 8436d391-a49f-4cba-9d51-990eff6abd0e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

Summary

Summary by CodeRabbit

• Bug Fixes
  • Improved background command and task history accuracy when transcripts change, reducing the chance of applying stale completion results.
  • History checks now handle delayed reads more reliably and avoid letting one cancelled request disrupt other readers.
  • Task status is better maintained when sessions or transcript files are replaced, truncated, or updated during a scan.

The change adds a bounded asynchronous transcript reader and uses it for command-history scanning and verification. Claude task snapshots are folded with version-checked history evidence, and task tracking and scan paths are updated to use the asynchronous flow.

Changes: Claude transcript history

Layer / File(s)	Summary
Bounded transcript reader server/transcript-reader.ts, server/transcript-version.ts, server/transcript-reader.test.ts	Adds a shared reader for bounded positioned reads and metadata lookup. It supports cancellation, deadlines, queue limits, file-version checks, and reset and lifecycle queries.
Asynchronous command-history verification server/claude-command-history.ts, server/claude-command-history.test.ts, scripts/remote-bundle-smoke.ts, changelog.d/claude-command-history-async-reader.md	Moves history scanning and line verification to asynchronous reader operations. Adds bounded retries, abort and deadline handling, and unverified candidates when a wait expires. The bundle smoke check exercises positional reads and command-history results.
Task folding and history proofs server/claude-task-state.ts, server/claude-task-state.test.ts, server/claude-task-history.ts, server/claude-task-history.test.ts	Adds task snapshot and folded-state types. Matching command records and verified history determine task outcomes; task-history checks reject evidence that does not match the current parent transcript.
Task tracking and scan integration server/claude-subagents.ts, server/claude-subagents.test.ts, server/claude-task-scan.ts, server/claude-task-scan.test.ts, server/file-tasks.ts, .github/workflows/ci.yml, .github/workflows/remote-bundles.yml	Updates subagent tracking, task scans, and file tasks to use snapshots and asynchronous history reads. Tests cover pending reads and transcript or session changes. Windows test jobs add the reader and history tests.

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Bug fix

Possibly related PRs

• devswha/ovis#2608: Introduced Claude command-history evidence used by subagent tracking and task scans, which this change updates to asynchronous reads.
• devswha/ovis#2511: Moved Claude task-list scans to the file-task worker; this change updates the same scan and file-task data flow.

Suggested reviewers: devswha

Merge Risk: Moderate · previous review 4dd31

Under sustained reader saturation or slow storage, background-command history can give up permanently, so finished commands may keep showing as running. Stop counting busy or timeout rejections toward the failure limit before merging.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 46.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Title check		The title clearly identifies the primary change: preventing slow storage from blocking Claude command-history reads.
Description check		The pull request description includes the required Change and Validation sections. It explains the problem and resulting behavior, links the related issue, lists validation commands with results, iden…

Full details: Description check

Explanation

The pull request description includes the required Change and Validation sections. It explains the problem and resulting behavior, links the related issue, lists validation commands with results, identifies skipped suites, and provides relevant performance and platform coverage details.

Full details: Docstring Coverage

Explanation

Docstring coverage is 46.43% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 28 functions across 15 files. (3 skipped: 3 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

CodeRabbit Verify

• Run CodeRabbit Verify to test the changes in this pull request.

Recent review info

Run configuration

• Source mode: Auto
• Repository: devswha/ovis
• Branch: main
• File: .coderabbit.yaml
• Commit: 5456638
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 8e9d4176-ea0a-479b-b10f-5bef9c25ef69

Commits

Reviewing files that changed from the base of the PR and between 5456638 and 4dd31af.

Files selected for processing: 18

• .github/workflows/ci.yml
• .github/workflows/remote-bundles.yml
• changelog.d/claude-command-history-async-reader.md
• scripts/remote-bundle-smoke.ts
• server/claude-command-history.test.ts
• server/claude-command-history.ts
• server/claude-subagents.test.ts
• server/claude-subagents.ts
• server/claude-task-history.test.ts
• server/claude-task-history.ts
• server/claude-task-scan.test.ts
• server/claude-task-scan.ts
• server/claude-task-state.test.ts
• server/claude-task-state.ts
• server/file-tasks.ts
• server/transcript-reader.test.ts
• server/transcript-reader.ts
• server/transcript-version.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

[devswha]

Thanks @duytbp for splitting this out as planned. Review (omo + Codex Terra xhigh) found that this step does change behaviour, so it can't merge yet. Since you're actively iterating on this, we'll leave the fixes to you on this branch.

HIGH (reproduced against main): an ended command flips back to running after an ordinary write.

• Setup: an old subagent command whose completion notice sits before the 16 MiB cold window. Wait until the tracker shows the pane at 0 running, then append one ordinary line to the parent transcript and poll.
• origin/main: the count stays 0 on every poll.
• This branch: the count goes back to 1 and the pane's change callback fires 1, 0, 1, 0, ... as writes continue.
• Claude pane reads working while only its background subagents run (title spinner after end_turn) #2069 breaks with it: when the turn's rest is written, main reports restsOn=false, count=0; this branch reports restsOn=true, count=1, so the pane is briefly working because of a command already proven ended.
• Cause: the cached proof and pending outcomes are dropped on any size/mtime change (server/claude-subagents.ts:1589-1607, server/claude-task-state.ts:79-86). Main re-checks the known ends against the grown file instead.
• The branch's own test "checks changed-generation shell status..." in claude-subagents.test.ts asserts countOf("p1") is 1 ("changed bytes have no historical authority until vouched"); on main it is 0. That test encodes the regression.
• Inferred, not reproduced: on the slow storage this targets, a session that appends faster than one proof completes would keep finished commands "running" for as long as it is active.

MEDIUM

• The task list can call a finished command running: with two appends during a scan, claude-task-scan.ts:39-49 answers without history, and claude-task-scan.test.ts:295-309 (mutations=2) expects command B to be "running". On main it is completed.
• Existing assertions were weakened or moved: "drops every read and timer once the last server lets go" lost waiting: 1, and "counts an ended command once its history is read despite a skipped record" now asserts a separate historicalState() path instead of the tracker that serves panes.
• eventual() in the tests polls on setImmediate for up to 2 s. Please wait on the actual change event (with a bounded timeout) instead.
• Scope: for a no-behaviour-change step this adds a lot of new tracker state (revision/proof/read, a pending overlay, per-path backoff, a snapshot/fold split, a task-scan retry protocol); the two regressions above come from it. The smaller the move, the easier it is to prove nothing changed.
• Inherited from main, noted for completeness (Terra): an end at the launch's millisecond with a matching Bash call id still closes the launch even when its row comes before the launch.

Cost (file already in the page cache, so no slow-disk benefit shows): total read time on a 276 MB sparse history was 61-64 ms on main vs 132-178 ms here, and about 1.2 s vs 1.3-1.4 s on a 95 MB dense one; the longest event-loop stall was about equal.

Acceptance for this step

1. No behaviour regression against main (the cases above included), with Claude pane reads working while only its background subagents run (title spinner after end_turn) #2069 intact.
2. fix(claude): reconcile background commands after cold history reads #2608's tests unchanged in substance: same assertions on the same paths that serve panes and the task list.
3. No slower total read without a measured win to show for it (for example on the sshfs setup from fix(conversation): isolate Claude image disk reads #2610).
4. The ci.yml / remote-bundles.yml additions are fine as test-only, but any workflow change still needs the owner's go before merge.

Thanks again; happy to re-review as soon as you push.

[coderabbitai]

• Fix comments: Fix CodeRabbit comments on this PR

[duytbp]

Thanks for the concrete reproductions. Agreed: the pending/running transition is a behavior regression, including its effect on #2069. I will fix publication/reconciliation so ordinary appends preserve the proven result, retain valid completion evidence through task-list appends, restore the original tracker/serving-path assertions, and replace the new polling helpers with bounded waits on actual events. The inherited same-millisecond issue stays outside this step.

Both Windows failures are the same new fixture's attempt to replace an open destination. The actual bundled async-reader/history smoke passed. A reviewed test-only correction preserves native positional/buffer and replacement checks, with deterministic mid-lease identity-change and rejected-admission coverage; it adds no skip or retry. I will include it with the semantic fixes in one consolidated push.

I will also run the matching synthetic SSHFS comparison and report total time and other-request latency alongside local measurements. The existing harness now has a separately frozen, minimal adapter that awaits history evidence on both sources; prior measurements remain unchanged. No image implementation changes have started, and this PR remains pending the behavior and measured-benefit gates you listed.

[duytbp]

@devswha Sorry about the previous push. It wasn't the small step we agreed on. It added tracker state that changed behaviour, the regressions you found came from that, and it cost you a review to find them. I've cut it back to what the step was meant to be: only the bulk reads of server/claude-command-history.ts move to async reads through a shared reader. Everything else is main's code.

Your points

• Ended command flips back to running (HIGH), and Claude pane reads working while only its background subagents run (title spinner after end_turn) #2069: claude-subagents.ts, claude-task-scan.ts, file-tasks.ts and their tests are byte-identical to main again, and claude-task-state.ts / claude-task-history.ts are gone. vouchedEnds is main's, so a grown transcript rechecks known ends as before. I ran your setup against main and this branch, and the output matched: the count stays 0 after an ordinary append and over 20 more appends, and when the rest is written, restsOn=false, count=0.
• Task list with two appends (MEDIUM): main's scan and test, unchanged. B is completed.
• Weakened or moved assertions: restored. Every existing assertion in claude-command-history.test.ts is unchanged, waiting: 1 included. The test diff adds tests and two names to the bun:test import, nothing else.
• eventual() polling: gone. The new tests wait on the change event, or step a fake clock.
• Scope: no new tracker state. The new state is the reader (server/transcript-reader.ts), which you asked to be shared, plus what the async reads need to stay correct. That's listed under "Guards" in the description.
• Same-millisecond issue: left as on main.
• Workflows: no workflow or script changes. The Windows test-list addition is an optional follow-up, only if you want it.

Cost. A cold history read on a synthetic transcript, median [min–max]:

	main 5456638	this branch
SSHFS (14 ms RTT, cache=no), 276 MB sparse	61.7 s	9.6 s
SSHFS, 95 MB dense	not done after 300 s	9.2 s
SSHFS, other requests during the read, p95	~155 ms	1–3.5 ms
SSHFS, 1 MB session while the 276 MB one reads	15.4 s	0.48 s
Local, page cache, 276 MB sparse	65 ms [65–80]	103 ms [98–108]
Local, page cache, 95 MB dense	1.05 s [1.00–1.10]	1.20 s [1.18–1.53]
Local, longest event-loop stall (sparse)	4.7 ms	2.4 ms

The two dense rows and the SSHFS p95 come from the full run on 83a5101, one commit before the last review fixes. The other rows are a recheck of the final head.

On a local disk this branch is slower, 1.15–1.66x main per interleaved pair (1.51x on sparse at the final commit). The cost is Bun's async read and the fstat after each chunk. The description names the follow-up that closes most of it: reading the next chunk while that stat is in flight. I left it out to keep this step small. If you'd rather this step be no slower locally first, I'll add it here.

Run it yourself. The benchmark is at https://gist.github.com/duytbp/470e90421331b31ccd953de200d6aae0. It uses synthetic transcripts only and needs no herdr or Claude account. bench.sh <rev> in any checkout gives the local numbers in about 5 minutes against 5456638. MODES=sshfs adds the SSHFS lab, which needs Linux and sudo, and takes about 30 minutes.

The description has the full tables and the checks that were run. Thanks for the careful review. I'll keep the image step (#2610) a draft on top of this until this one is settled.

[devswha]

Thanks @duytbp, this is the step we agreed on, and thanks for the honest note and the benchmark gist. Both reviewers (omo and Codex Terra xhigh) re-ran the earlier repros against main and this head with identical results: ended stays ended over appends, #2069's rest holds, the task list matches main, and every existing assertion is unchanged. omo approves the move as behaviour-preserving. Terra found one remaining gap in the isolation this step is meant to give, plus a few smaller items:

HIGH (reproduced): one stalled transcript read still holds up every other transcript's history. claude-command-history.ts:265 has a single drain that awaits each step before starting the next, so the reader's second slot can't serve transcript B while A's window is stalled. With A held, B's waitCommandHistory returned complete: false, through: 0 after 150 ms, and that holds until A's 10 s deadline. Main is worse (its sync read blocks the whole server), so it's not a regression, but "a slow read can't wedge other transcripts" is what this step is for. Please let different paths drain independently (up to the reader's limit).

Repro (bun test, R2616_ROOT=<checkout>)

import { it, expect, spyOn } from 'bun:test';
import { mkdtempSync, writeFileSync, rmSync, statSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
const root = process.env.R2616_ROOT!;
const { holdCommandHistory, waitCommandHistory, commandHistorySize } = await import(`${root}/server/claude-command-history.ts`);
const { transcriptReader } = await import(`${root}/server/transcript-reader.ts`);
const gate = () => { let open!: () => void; const promise = new Promise<void>(r => open=r); return {promise, open}; };
it('held window delays unrelated history and survives last-server release', async () => {
  const dir=mkdtempSync(join(tmpdir(),'sched-')); const a=join(dir,'a'), b=join(dir,'b');
  const row=JSON.stringify({type:'queue-operation',operation:'enqueue',timestamp:'2026-10-01T12:00:00Z',content:'<task-notification><task-id>done</task-id><status>completed</status><summary>Background command finished</summary></task-notification>'})+'\n';
  writeFileSync(a,row); writeFileSync(b,row);
  const entered=gate(), held=gate(); const real=transcriptReader.read.bind(transcriptReader);
  const spy=spyOn(transcriptReader,'read').mockImplementation((path:any,work:any,opts:any)=>real(path, path===a ? (file:any)=>work({...file,window:async (...args:any[])=>{entered.open(); await held.promise; return file.window(...args);}}) : work,opts));
  const release=holdCommandHistory();
  try {
    const first=waitCommandHistory(a,statSync(a).size,60000);
    await entered.promise;
    const other=await waitCommandHistory(b,statSync(b).size,150);
    console.log('healthy while a held:', {complete:other?.complete,through:other?.through, history:commandHistorySize(),reader:transcriptReader.size()});
    expect(other?.complete).toBe(false); expect(other?.through).toBe(0);
    release(); await new Promise<void>(r=>setImmediate(r));
    console.log('after last release:',{history:commandHistorySize(),reader:transcriptReader.size()});
    expect(commandHistorySize().draining).toBe(true); expect(transcriptReader.size().active).toBe(1);
    const next=await waitCommandHistory(b,statSync(b).size,150);
    console.log('new history after release:',{complete:next?.complete,through:next?.through});
    expect(next?.complete).toBe(false);
    held.open(); await first; await transcriptReader.settled();
  } finally { held.open(); release(); spy.mockRestore(); rmSync(dir,{recursive:true,force:true}); }
}, 3000);

MEDIUM

• Releasing the last holder doesn't cancel a read already in flight: in the repro above, after release() the history still reports draining: true and the reader active: 1, and a new history request stays incomplete until the held window returns. The existing shutdown test only releases before the read starts; please cancel the active request on release, with a test that holds it in flight.
• Tests: claude-command-history.test.ts:509 polls up to 1000 event-loop turns; please wait on the actual change instead. Several new tests also assert exact lease and turn counts (:419, 434, 446, 501, 512, 526, 552, 573), which ties them to the current scheduling; asserting the outcome instead would keep them valid after the read-ahead.

Known limits, fine to document rather than fix here: two native reads that never settle keep both reader slots (the deadline only ends the callers' waits); dev:ino uses number stats like main, so it inherits main's precision limit; a transcript that keeps hitting the 10 s limit is retried every second without backoff while watched.

Cost, and the read-ahead. Our own local page-cache pairs agree with yours: 276 MB sparse 82-92 ms on main vs 91-151 ms here, 95 MB dense 1.12-1.29 s vs 1.32-1.43 s, with a shorter or equal longest stall. Yes please: add the read-ahead (reading the next chunk while the fstat is in flight) in this PR, then re-run the same table (local sparse and dense total, longest stall, SSHFS, other requests' p95). The bar is: local no slower than main within noise, the network-mount win holds, and #2608's cases don't regress. If it gets there, we'll merge this step.

FYI, not from this PR: on native Windows, invalidates completion authority on replacement, truncation and same-size rewrites fails intermittently on main too (a same-size rewrite within one timestamp tick looks unchanged); no CI job runs that file on Windows. We'll track it separately.

[duytbp]

Thanks. Both reviews are clear, and I'm working on all of it in this PR:

• different paths drain independently, up to the reader's limit, with your repro as a test;
• the last release cancels a read in flight, tested with the read held;
• the tests wait on the change instead of polling turns, and assert the outcome instead of lease and turn counts;
• the read-ahead.

Then I'll re-run the same table: local sparse and dense total, longest stall, SSHFS, other requests' p95, and #2608's cases. The known limits will go in the description.

[devswha]

Heads-up @duytbp: #2620 just merged into main (05c9133). It changes server/claude-command-history.ts and its test: on Windows a same-size rewrite inside one timestamp tick often keeps id, size and mtime, so a check now only vouches for a version without re-reading when it began at least 3 s after that version's mtime, waiters share one recheck per tick, and end lines are re-read in coalesced runs. Your branch now conflicts in both files, so please merge main in (no rebase needed) before the next round and the re-measure, so the table is taken against the new main.

[duytbp]

Pushed ba3d01e with every item from the review. The description has the details and the new tables.

• HIGH: a slice that has neither ended nor taken a turn for 50 ms, while another session waits, is left to finish on its own, and the next session's slice starts beside it. At most the reader's capacity (2) run at once, one per transcript. Your repro is now the test "reads another transcript while one transcript's read is stuck on its disk": B comes back complete with its end while A's window is held.
• MEDIUM, cancel: the last release aborts the slice's read in flight. The test "gives up a read stuck in flight once the last server lets go" holds A's window, then releases. The request gives up, the drain ends, and a new request for B comes back complete while A's operation is still held. What it can't do: the native operation already started keeps its reader lease until it returns, so a new read of A itself waits for it. That limit is listed under known limits.
• MEDIUM, tests: the 1000-turn poll now waits on commandHistorySettled(). The lease- and turn-count assertions now check outcomes; the description has a before/after table for each.
• Known limits: listed in the description and in the code's comments.
• Read-ahead: the next window is read while the stat after the current one is on its way. The next slice also takes in the rest of a window a slice ended in, with no new read. Main reads the dense transcript nearly four times over; this branch reads it once.

Against the bar, on ba3d01e (5 interleaved local pairs; ratios are this branch over main):

• Local: dense 1.06 warm and 1.03 cold, sparse 1.04 cold. Sparse warm is the noisy row: 1.27 [1.00–1.73] in this run, and 0.80 [0.72–1.12] in a run just before the last fix, which doesn't change the path a lone read takes. Our host also runs a VM that keeps load around 13, so I read those as within noise. Its CPU is a real cost, though: about 1.3–1.75× main on the sparse transcript, from the thread pool's round trips.
• Network mount: sparse 62.2 s → 5.15 s, dense more than 300 s (did not finish) → 2.9 s, a 1 MB session behind either 15.3 s → 0.47 s, and other requests' p95 during sparse 156 → 1.5 ms.
• fix(claude): reconcile background commands after cold history reads #2608's cases: same output as main on ba3d01e. check fast passes (10730 tests), the history tests pass 24 of 24 with 8 copies at once, and the contract tests pass 207 of 207.

One row stays slower, and I'd like to ask you to accept it rather than tune it here: a small session read on its own. It costs about 1.9 ms once per process, to start Bun's async file thread pool and run the reader code for the first time. After that it costs 0.3–0.5 ms per read, for six thread-pool round trips. In the table that's 4 → 5 ms warm and 5 → 9 ms cold. On a network mount the same read goes from 7.75 s to 0.48 s.

[devswha]

Thanks @duytbp, that covers every item; we'll review ba3d01e's approach as soon as it sits on current main. The branch still conflicts with main in server/claude-command-history.ts and its test (#2620, merged at 05c9133, see my earlier note): please merge main in (no rebase) and re-run the table on the merged head, since #2620 changes when a check vouches for a version and how end lines are re-read, both on the paths you timed. The small-session row is noted; the decision on it goes with the re-measured table.

[duytbp]

Thanks. Sorry I missed your heads-up about #2620. I'm merging main in now, without a rebase. I'll keep #2620's behaviour and its test assertions unchanged, then re-run the same table on the merged head and post it here.

[duytbp]

Merged main in (05c9133, a merge commit, no rebase) and pushed c5103d0, so you can review now. The table is re-running on the merged head against 05c9133. I'll post it as soon as it's done.

• Conflicts: only claude-command-history.ts and its test. I kept both sides. fix(claude): recheck a same-size history rewrite within one timestamp tick #2620's code and its seven tests are unchanged; against main, the test file still drops only its three import lines.
• One interaction, fixed in c5103d0: fix(claude): recheck a same-size history rewrite within one timestamp tick #2620's evidence can now start the history over from a request while a slice waits on its disk. The slice then took in a window read from the old position as the transcript's first bytes. It corrected itself on the next request or check, but in between a request could get [] instead of an end. The slice now drops a window or check read across a start over and reads again from 0. New test: "drops a window read across a start over that a request made while the window waited for the disk". It fails without the guard.
• Two of this PR's own tests changed for fix(claude): recheck a same-size history rewrite within one timestamp tick #2620's rules (none of fix(claude): recheck a same-size history rewrite within one timestamp tick #2620's or main's tests did). The description has the details. One now rewrites a later record, so only a check sees the rewrite, and its ends assertion is ["a"]. The other backdates its file's mtime past the 3 s tick.
• Synchronous reads: vouchLines stays synchronous on the request path, as fix(claude): recheck a same-size history rewrite within one timestamp tick #2620 wrote it. It's listed with the lstatSync follow-up.
• Checks on c5103d0: check fast passes (10744 tests). The history tests pass 24 of 24 with 8 copies at once. The contract tests pass 207 of 207. The review's synthetic cases give the same output as main 05c9133.

[duytbp]

The re-measured table is in the description: c5103d0 against main 05c9133, same harness, interleaved runs. Against the bar:

• Local: dense 0.87 warm and 0.98 cold, sparse 0.91 cold, and sparse warm 1.21 [0.65–1.46]. That last one is the noisy row: its medians are 85 vs 89 ms. The longest stall is shorter or equal. CPU on the sparse transcript is 1.3–1.5× main's. That's the cost of the async reads, and it holds across runs.

• Network mount:

  • sparse: 61.7 s → 5.57 s;
  • dense: more than 300 s (did not finish) → 2.66 s;
  • a 1 MB session behind either: 15.3 s → 0.47 s;
  • other requests' p95 during sparse: 155 → 1.0 ms.

  Main with fix(claude): recheck a same-size history rewrite within one timestamp tick #2620 reads the dense transcript about 4.6 times over; this branch reads it once.

• fix(claude): reconcile background commands after cold history reads #2608's cases: same output as main; their tests are in check fast, which passes on c5103d0.

On the small-session row, measured on its own: 4 → 8 ms warm and 5 → 8 ms cold. That's 1–2 ms more than the 1.9 ms once-per-process plus 0.3–0.5 ms per read that I profiled before the merge, and I haven't accounted for the rest yet. With a large transcript reading at the same time, it's even with main (5 vs 5 ms). I'd still ask you to accept it for this step. If you'd rather I profile the gap first, I will.

[devswha]

Thanks @duytbp, the merge with #2620 looks right and the start-over fix is a good catch. Both reviews of c5103d0 are in (omo approves with a watch, Codex Terra finds no HIGH): the stuck-read and cancel fixes work, the old repros match main, and #2620's and #2608's tests are unchanged and pass (omo also ran the history tests 6x on native Windows, all green). The small-session row (+3 to +4 ms, the thread pool's first start) is accepted for this step.

One thing blocks the merge, and it's the round we'd like now: the page-cached sparse read is slower and costs about twice the CPU. Our own ABBA run on a quiet host (load 1.4, 0 flagged samples, both orders agree), same harness:

case	main read ms	PR read ms	main CPU ms (user+sys)	PR CPU ms	main loop gap max	PR loop gap max
sparse 276 MB warm (A)	71 [67-71]	84 [82-85]	77	164	4.7	2.8
sparse warm (B)	70 [68-72]	84 [81-87]	78	167	4.7	2.8
sparse cold (A)	345 [344-349]	312 [306-318]	124	211	5.3	2.8
sparse cold (B)	341 [338-343]	304 [302-310]	123	207	5.4	2.8
dense 95 MB warm (A)	971 [950-995]	950 [935-971]	1226	1236	5.4	6.7
dense warm (B)	966 [957-971]	936 [909-954]	1226	1211	5.5	6.1
dense cold (A)	1106 [1088-1126]	954 [949-977]	1265	1258	5.3	6.4
dense cold (B)	1109 [1096-1127]	947 [929-976]	1267	1253	5.5	6.1
small warm (A/B)	4 [4-4]	7 [7-8]	6	9	0.9	3.6
small cold (A/B)	6 [5-7]	9 [8-9]	6-7	10	4.6	2.6-2.8
Other requests' ping p95 during the read: sparse 4.0-4.6 -> 0.3-0.7 ms, dense ~4.0 -> ~3.9 ms, small 2.3-3.9 -> 0.5-0.7 ms.

Sparse warm doesn't overlap main (70 [67-72] vs 84 [81-87] ms), and CPU on the sparse transcript is ~85 ms more per read (warm 78 -> 165 ms). Terra isolated the cost: a scratch copy of the scanner with its file calls made synchronous used 87-106 ms CPU, close to main, so it's the per-window async dispatch, worker wakeups and completion handoffs, not the scan itself. A cached file shouldn't pay thread-pool round trips for every window: e.g. take a fast path when the bytes are already read (a window that is already in hand, or several positional reads and checks batched per async handoff), keeping the event loop responsive, cancellation and fairness as they are now. Bar: the same ABBA table after, with sparse warm overlapping main and the extra CPU on big cached reads mostly gone; dense, sparse cold, the network-mount numbers and #2608's cases staying as they are.

While you're there (smaller, from the reviews):

• Complexity (omo M1): claude-command-history.ts:340-520 stacks a 50 ms stall race, leases held across turns, read-ahead with a buffer pool, reuse of a window's remaining bytes, and turn/restart counters; the last commit fixed an interaction among them. If the fast path makes some of these unnecessary (read-ahead and the byte reuse are the first candidates), please drop them.
• Tests pinned to internals (omo M2): :754 intercepts every 50 ms timer, :646-647 caps reads per turn, :554 and :616 count leases; please assert outcomes. Two tests still rely on a real 150 ms window and the reader tests spin on Date.now() (Terra LOW): a controlled clock would make them deterministic.
• Known limits, fine to keep documented: two native reads that never return still hold both reader slots (the changelog line slightly overstates the isolation: please say "one stuck transcript"); up to 4 MiB of spare buffers stay alive; a busy transcript is retried every second while watched.

Thanks for sticking with this; the network-mount and dense wins are real.

[duytbp]

Pushed 93f74fb (fast-forward from c5103d0, no rebase) for the CPU round, and the description is updated. The change is a fast path: on a local Linux filesystem (ext4, btrfs, xfs, tmpfs, zfs, f2fs, bcachefs, by statfs type), the lease opens, stats and reads on the event loop, inside the same slices and turns. Network filesystems, macOS and Windows keep the thread pool. A call on the event loop that takes over 2 ms, i.e. one the disk answered rather than the page cache, sends the rest of that lease and that transcript's next second through the pool.

Against your bar, b9b2acd against main 05c9133, same harness, interleaved (my host is loaded, so all samples are flagged):

	main	this branch
sparse warm, read	65 ms [57–70]	74 ms [66–79], ratio 1.15 [0.96–1.34]
sparse warm, CPU	64 ms	90 ms, ratio 1.25 [1.06–1.40]
sparse warm, longest loop gap	4.7 ms	4.7 ms
sparse cold, read / CPU	158 / 157 ms	164 / 190 ms
dense warm, read / CPU	1027 / 1172 ms	949 / 1104 ms
dense warm, longest loop gap	7.3 ms	7.3 ms
SSHFS sparse / dense	61.6 s / not done in 300 s	5.18 s / 2.58 s

• Sparse warm CPU, the row that blocked: in your run c5103d0 took 165 ms against main's 78 (2.1×). Here, three full runs across this round's commits give 1.10, 1.16 and 1.25× main's, against 1.48× for c5103d0 in the same harness. The branch held at 77–93 ms in those runs, while main moved between 64 and 88. The rest is mostly first-run compilation: compiled, the main-thread work matches main's within 2 ms, and the pool threads do none. Read time overlaps main's in every run. Your quiet-host ABBA would settle the last few percent better than mine.
• Dense, SSHFS and fix(claude): reconcile background commands after cold history reads #2608's cases are as before. An interleaved SSHFS run against c5103d0 gave ratios of 1.00 and 1.02.
• Sparse cold is now even with main in read time (it was about 10% faster), and costs ~20% more CPU than main. Once a read takes over 2 ms, the rest of the lease goes through the thread pool, so a stalling local disk blocks the loop for one call, after which the thread pool takes over.
• Small session: +3 ms, as accepted. Its loop-gap max reads 4.7 ms against main's ~1, but that is the harness's sampler: it records a gap only when a tick fires before the read resolves, so main's small read rarely gets its parse counted. A probe that counts the last gap too found 3.3–5.4 ms here and 4.3–5.1 ms on main.

The smaller items:

• Complexity (M1): the read-ahead and the reuse of a window's rest stay. They carry the thread-pool path that SSHFS takes, which the fast path doesn't touch. Without them, SSHFS sparse took 9.55 s and dense 9.2 s.
• Tests pinned to internals (M2): the lease and turn counts now assert outcomes on a controlled clock. The 150 ms waits and the Date.now() spins are gone, and the 50 ms timer interception is replaced by a fake clock. Two tests turned out not to catch their own fixes, at c5103d0 too: "however long it reads" and "reads on by whole chunks". Both are fixed, and each fails with its fix undone. The description lists every changed assertion.
• Changelog: it now says "one stuck transcript".

Checks on 93f74fb: check fast 10751 passed, 0 failed; the history tests 24 of 24 with 8 copies at once; contract 207 passed.

Known limits added to the description: a local disk that stops answering inside a call on the event loop blocks the loop, as main's readSync does. And macOS and Windows keep the thread pool's cost. For macOS, comparing the directory's statfs type with /'s should work, but I haven't tried it on a Mac, so it's a possible follow-up.

[devswha]

Thanks @duytbp, this round closes the gap: our own interleaved run (main 3e51902 vs 93f74fb, both orders, 7 reps each) has sparse warm at 68 [67-70] / 73 [70-77] ms on main vs 71 [70-75] / 76 [72-80] on the PR (overlapping), +8 ms CPU per warm read (was +85), dense cold about a third faster, and other requests' p95 during a sparse cold read down from ~9 ms to 0.3 ms. Both checks we set for the event-loop path hold: the worst single event-loop block reading 64-200 MB transcripts was 8.17 ms (main 8.40 ms), and a real sshfs mount, overlayfs, squashfs, mocked NFS/CIFS/9p/ceph/ecryptfs and a failing statfs all stayed on the pool.

One last small round, then we merge:

Required

1. Controlled clock in the four Linux fast-path reader tests (transcript-reader.test.ts around :373, :407, :426, :444). They time real reads against the 2 ms threshold, so a preempted read trips the fallback: 8 of 48 runs failed under heavy load (48 parallel runs beside 64 CPU spinners); the previous round's tests passed 48/48 there. :385 already shows the pattern.
2. Time the synchronous open and close too (:196, :71), and count a call that throws. An injected 80 ms openSync delayed a timer by 80 ms and the next lease opened synchronously again instead of cooling down.
3. Take the filesystem type of the opened file, not its parent directory (near() at :371, cached 60 s). A file bind-mounted from NFS/FUSE into an ext4 directory would be read synchronously (simulated: the parent reported ext4, the file NFS, only the parent was asked). fstatfs on the fd isn't in Bun/Node, so e.g. compare fstat(fd).dev with the dev of the directory you statfs'd and use the pool when they differ.

Optional, only if the timing table stays met: send the first read of a range through the pool and use the event loop only for bytes this process has already read once. Today a synchronous read can't be interrupted, so one stuck local read blocks the bridge for its full length (an injected 100 ms read delayed a timer by 101 ms). Main makes the same synchronous calls, so this is pre-existing and doesn't block on its own.

Not needed: Linux remote bundles taking the fast path is fine (a remote bundle is just the server on that PC).

After your push we'll run a delta review and a spot-check of the same timing table.

[duytbp]

Thanks for the quick and thorough check. Working on the three required items now: a controlled clock in the four fast-path reader tests, with the same 48-parallel stress run beside spinners; timing the synchronous open and close, with a throw counting as slow; and comparing the opened file's dev with its directory's, using the pool when they differ.

I'll leave out the optional pool-first read. Every first read of a range in a process, page-cached or not, would then pay the thread-pool cost again. That is most of what a history read does after a restart, and what the CPU row measures. As you say, the uninterruptible synchronous read is main's too.

[duytbp]

Pushed 7edf4df (fast-forward from 93f74fb) with the three required items, and the description is updated:

1. Controlled clock: the reader's event-loop tests now run on a clock only they move (stillClock()), :385 and the third part of :444 included, since those mixed real time with an offset. Under 48 parallel runs of transcript-reader.test.ts beside 64 CPU spinners: 93f74fb passed 42 of 48 (the same tests you saw fail), and 7edf4df passes 48 of 48, with 16 spinners too.
2. Open and close timed: an open or close on the event loop over 2 ms, or one that throws, marks the transcript for the thread pool for a second. A slow open also sends the rest of its own lease there. A read or stat that throws now counts too. New tests: one each for a slow or throwing open, a slow or throwing close, and a throwing read.
3. The file's own device: the lookup stats the directory beside its statfs and caches its dev. The lease's first stat, which comes before any read, compares the transcript's fstat().dev with it. On a mismatch the rest of the lease goes through the pool, and the path stays there for 60 s. A known directory still costs no extra call, and a cache miss still opens while the lookup runs. New test: a directory that reports another device than its file. The limit is in the description: a bind-mounted network file is opened and statted once on the loop before its device gives it away, once per 60 s.

Each new test fails with its fix undone. Checks on 7edf4df: check fast 10755 passed, 0 failed; reader and history tests 82 passed; history tests 24 of 24 with 8 copies at once; contract 207 passed.

A local run on 7edf4df against 05c9133, 5 runs each, on my loaded host:

• sparse warm: 67 against 61 ms, CPU ratio 1.22;
• dense warm: 0.95, and cold 0.86;
• the longest event-loop gap matches main's on both: 4.7 against 4.6 ms, and 6.4 against 6.6 ms.

Sparse cold CPU came out 1.39× main's here, against 1.22× on b9b2acd, with a wide range (1.07–1.76). A cache miss now stats the directory too, and every bench read is a fresh process, so I can't separate that from load. Your spot-check will show it more clearly.

The optional pool-first read is left out, as I said above.

[devswha]

Thanks @duytbp for this, and for the measurements, the SSHFS lab and the fast turnarounds across five rounds. Reading old command history no longer blocks the bridge on slow storage: transcripts are read in bounded slices through a shared reader, on the thread pool for network filesystems, macOS and Windows, and on the event loop for page-cached files on local Linux filesystems, falling back to the pool for a file whose own device differs from its directory's or after any slow or failing call.

Merge-pass timing (interleaved, main 3e51902 vs 7edf4df, 7 runs per side in both orders, quiet host):

row	main (A / B order)	PR (A / B order)	verdict
sparse warm, read ms	68 [66-70] / 68 [64-82]	71 [70-72] / 75 [71-75]	same (ranges overlap; +3-7 ms)
sparse warm, CPU ms per read (user+sys)	75 / 82	86 / 88	+6 to +11 ms per read
sparse cold, read ms	336 [334-339] / 338 [336-342]	305 [299-323] / 303 [300-308]	better (-10%)
sparse cold, CPU ms	119 / 125	204 / 213	+85 ms, once per cold transcript
dense warm, read ms	967 / 976	955 / 956	same
dense cold, read ms	1102 / 1109	952 / 946	better (-14%)
small warm / cold, read ms	4 / 5-7	7 / 9	+2-4 ms (accepted row)
other requests' ping p95 during sparse cold	4.4 / 4.5 ms	0.3 / 0.3 ms	better
worst loop gap, any row	5.6 ms	6.7 ms	same size as main

The author's SSHFS lab showed sparse/dense history reads at 61.6 s / not done in 300 s on main vs 5.2 s / 2.6 s here.

Checks set for the event-loop path, both measured: the worst single event-loop block reading 64-200 MB transcripts matched main's (8.17 vs 8.40 ms in round 4); a real sshfs FUSE mount, overlayfs, squashfs, a single file bind-mounted from another device, mocked NFS/CIFS/9p/ceph/ecryptfs and a failing statfs all use the thread pool. The reader tests run on a controlled clock and passed 264 of 264 heavy-load runs (48 parallel beside up to 160 CPU spinners).

Reviews: omo approves; Codex Terra xhigh found no HIGH. Left as known limits, not blocking:

• pre-existing: a synchronous call can't be interrupted, so one stuck local read blocks the loop for its length (main makes the same synchronous calls);
• MEDIUM (unusual setup only, a network file bind-mounted into a local directory): a slow first fstat or a slow close can shorten that file's 60 s pool fallback to 1 s, and the fallback table is cleared whole at 256 entries; after the 60 s, such a file gets one open and stat on the loop again (at most once a minute);
• LOW: no test pins the full 60 s fallback; a non-I/O error also sends a file to the pool for 1 s; one test helper uses as any.

CodeRabbit's check shows it skipped this head; the merge pass and both reviewers reviewed it.