Repository navigation
feat: host search for the palette — files, specs, plans, chat transcripts and previews - #805
Open
danyaberezun wants to merge 28 commits into
Open
danyaberezun wants to merge 28 commits into
danyaberezun wants to merge 28 commits into
Conversation
danyaberezun
requested review from
Olga Lavrichenko (OLavrik),
Rustam Sadykov (SBOne-Kenobi) and
Rinat S (rsolmano)
as code owners
October 22, 2026 05:00
![jetbrains-air[bot]](https://avatars.githubusercontent.com/in/3314082?s=60&v=4){kind=small}
5 tasks done
Olga Lavrichenko (OLavrik)
requested changes
Oct 22, 2026
Olga Lavrichenko (OLavrik)
left a comment
Olga Lavrichenko (OLavrik)
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for this, the host-side search looks solid overall. Requesting changes for four issues, noted inline:
- The current project is searched twice when the active workspace is a sibling worktree.
- A failed file-list reload replaces the good cached list with an empty one.
- A
supersededreply can leave the palette pending forever. - Host results are cleared on every keystroke, so rows and selection flicker.
danyaberezun
force-pushed
the
global-search-host
branch
from
October 26, 2026 00:15
a4b27b2 to
2eaa6fe
Compare
danyaberezun
force-pushed
the
global-search-host
branch
2 times, most recently
from
October 26, 2026 02:09
68434d2 to
3bfc32a
Compare
Olga Lavrichenko (OLavrik)
approved these changes
Oct 26, 2026
danyaberezun
force-pushed
the
global-search-host
branch
from
October 26, 2026 19:50
3bfc32a to
4364cd9
Compare
This was referenced Oct 26, 2026
danyaberezun
force-pushed
the
global-search-host
branch
2 times, most recently
from
October 28, 2026 06:18
879d90c to
8b50ccd
Compare
The host searches files and specs in one root — the active worktree, or the context project's folder through its Default record on Project Home — instead of every open project's folder; search.preview keeps only the transcript window. File and spec rows render through the shared resource renderers (code, images, sandboxed HTML, markdown) injected by the shell, so resolveContainedFile, the text-head reader and the plan preview go.
… the host still holds Address the review: previous host rows stay on screen while a new request is pending; a latest reply the host marked superseded ends empty instead of spinning forever, and seq is seeded from the clock so a reloaded page never reissues a number the host has seen; late host arrivals sort after every row already on screen, across tiers, so the selected row never drops out of a capped group; a scoped query asks for the host's maximum page and the host's total counts into the more row and the group meta. On the host, a reloading file list or a spec snapshot being rebuilt reports indexing so the client polls for the fresh corpus, a failed reload keeps the stale list, and transcript hits order by time alone.
…erms and folding The host providers took the query as plain strings; they now take the matcher's MatchTerms, so a quoted term stays literal in field matches and a quoted phrase is one term for spec bodies and transcripts. The file list's prefilter twin is folded with the matcher's own foldMatchCase, now exported from contracts, so it never drops a path the matcher's case and diacritic folding would match.
…d merge conflicts A change that lands while a list loads now leaves the loaded list stale, so the next ask loads again; before, the load stamped itself fresh and the change stayed invisible for the TTL. A failure that may pass (a timeout, descriptor or memory pressure) now rejects the load, so the cache keeps its stale list, and a first load that fails answers failed, instead of a silent fallback walk or an empty list replacing a good one. A path git lists once per conflict stage is listed once, untracked node_modules no longer spends the cap, a quoted term skips the subsequence pass, and every returned file or spec hit carries its size so the client can decline to preview a large one.
Transcripts are scanned newest chat and newest message first with the budget checked before every message, so a large history answers truncated in time and only older messages go unsearched; chat titles are a provider of their own, matched over every chat. A spec snapshot is built off the request and only when read, never eagerly on every worktree change, at most eight roots are kept and a removed workspace's root is dropped. A plan is parsed again only when its file changed. A throwing provider or history read is logged and answers failed, including the chat providers.
Rows the previous query answered are matched again against the current text while the new reply is pending, so Enter can no longer open a row the typed text does not match. A poll or re-ask for the same input keeps every row its page left out and asks only the kinds still indexing, and rows that land while a selection is held join an arrival cohort sorted after every row already shown, so neither a better page nor a second reply moves the selection; the first arrow after a query change no longer flags rows already on screen as late. A group's "N more" no longer counts chats the browser already lists, a failed, still-indexing or cut-short search says so, polling runs for about 30 seconds instead of six, the placeholder names files and specs, the file and spec scopes need a context project, and a Project Home search reruns once the project's Default row is loaded.
…space that exists A file or spec preview is read only once the selection rests on its row, and not at all over 2 MiB, so sweeping across rows no longer reads every file and a passing selection never pulls a large log. A transcript preview carries its hit's anchor, so a transcript compacted since the search still focuses the right message, and previews are cached per connection. A pick whose workspace its project's list no longer holds, or whose list failed, now tells the user instead of activating an id no row backs; host hits open through openHostHit, now covered for every kind.
…replies The arrival tracking read the rows on screen when React processed its update, after they already held the incoming reply, so a reply's newcomers escaped their cohort; it now captures them first, and an end-to-end test holds a poll until the user moved the selection. A row an earlier query answered is matched again on the fields the host matches it on, and a message or body hit stays in the content tier, so the fresh reply no longer moves it. The selected host row stays listed and visible while the query stands, a plan read that failed is no longer remembered as empty, and a group's overflow counts a local row as one of the host's hits only when the host's own rule matches it.
…can be searched search.query now watches its context workspace, so a root searched from Project Home or before any panel asked for it still hears its changes instead of waiting out the list's TTL. The placeholder names files and specs only when those scopes are offered. The selection test answers the palette's searches itself, so it no longer depends on the host's file list being fresh.
…e hits literal The pinned row now lives only while the same search stands, so a changed root or a host that stops searching drops it instead of keeping a hit from a project that was closed meanwhile. A message or body row an earlier query answered is kept only while every term is a substring of its snippet, the content providers' rule, so an acronym no longer keeps it. A local workspace counts among the host's hits when its name, branch or PR number matches, as the host matches workspaces.
Two terms matching the same text (refresh ref, or a repeated term) gave overlapping ranges, and the highlight drew the text twice; they are merged, as the host does.
…in code comments The comments restated what the search, shell, spec and server-search SPECs already record, against the repository's near-zero comment rule; no code changes.
…e-ask plans the history has not reached A host chat or workspace hit was dropped whenever its id was listed locally, even when the local row did not match (a Default workspace whose branch its row does not show, a chat still labelled by its first prompt), so the query found nothing; it is now dropped only when the local row matches too. A plan whose session the history index has not reached yet was skipped while the group answered ready, so it never came back; the group now answers indexing and the client asks again.
…nned host row With host twins of non-matching local rows now kept, the pin checked for any item with its id and found the non-matching local twin, which ranking then dropped, losing the selection; the pin now counts only rows that rank, the matching local rows and the host rows.
…iles the worktree deleted A transcript hit was identified by its message index, which compaction shifts, so a selected row's id could come to name another message; it is now identified by the message's timestamp and role, with the index kept only as the jump target. A tracked file deleted in the worktree and not staged stayed a hit that opened nothing; a path the stat finds gone is now neither a hit nor counted, and the next matches fill the page.
…bound the deleted-file check by the budget Two user messages queued in the same millisecond got the same transcript id, so selecting the second opened the first; a same-timestamp twin now carries how many earlier messages share its timestamp and role. The check for deleted files ran until the page was full however many paths were gone; it now stops at the provider budget and answers what it verified as truncated.
… the page a scan found The same-millisecond ordinal could still move when compaction split a group of queued messages, so a transcript hit is now identified by the pi session entry its message came from, carried through the history index as entryId (the same projection buildSessionContext returns). The deleted-file check checked the budget before every stat, so a scan that had already spent it answered no rows at all; it now always checks the page's present matches and checks the budget only after a path it finds gone.
… while shown is never read whole The 2 MiB preview limit used the hit's size, but the renderer re-reads a file on every change, so a log that grew past the limit while selected was read whole. fs.readFile now takes an optional maxBytes (v96) and refuses a larger file with FILE_TOO_LARGE before reading it; the palette's preview passes the cap on every read, a refresh included, and shows the too-large message.
…ched again only on a new version
Whether the renderer refreshes first (and the host refuses the read) or a search poll lands first (and the new size keeps the pane from reading at all) depends on timing; the test now keeps the file changing until one of the two shows, and asserts that no read asked after the growth succeeded.
… after it Rewriting the file truncated it, so a read the host served mid-rewrite could see it below the cap; the file now grows once and later nudges only append. A pane that went too-large could also end the test with a read still in flight; the test now waits for every read asked after the growth before asserting.
A checked-out submodule or a tracked symlink to a directory was a file hit that neither previewed nor opened; a path whose stat is not a regular file is now neither a hit nor counted, like a deleted one.
…le hits A tracked symlink that resolves out of the worktree or into .git was a ready file hit whose preview failed with "Path escapes the worktree" and whose Enter opened nothing. The host now injects fs's readableFileSize, the file reader's own containment rule without a workspace lookup, as the search service's required stat, so such a path is neither a hit nor counted, while a symlink to a file inside the worktree stays a hit with its target's size.
…parse the same terms The host searched only the first MAX_SEARCH_QUERY_LENGTH characters while the palette parsed the whole query, so a long pasted query could show host rows that miss a term typed after the cut. The input now holds at most that many characters and parseSearchQuery cuts the text after the prefix to the same length, so the palette's terms and the host's come from the same characters.
…s and the top hit A late reply's rows sorted after the rows on screen only inside their own group, so a newcomer to a group shown above the selected row still pushed that row down, and a newcomer could unseat the top hit shown above everything. Newcomers to a group that already shows rows now continue below every row on screen in a section of their own, each later cohort below the earlier ones, and only rows on screen compete for the top hit.
…as no newcomer Arrivals compared a reply's hits with the rows on screen only, so the hits a group held behind its cap were stamped late by the next reply for the same search, an indexing poll included. Their group's "N more" row then moved below the selection. noteArrivals now also remembers the previous fresh reply's hits (a stale reply lists none), and only a hit that is in neither is a newcomer.
danyaberezun
force-pushed
the
global-search-host
branch
from
October 28, 2026 14:12
5424015 to
442a2e3
Compare
Collaborator
Author
|
@jetbrains-air please review — rebased onto main |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
PR1 (#804) ships the palette with the corpora the browser already holds: commands, settings, projects, workspaces and the chats it knows. The things that take the longest to find by hand still have no search at all: a file by name across a project, a spec by its body, a plan item, the chat where something was said and the exact message. Those corpora live only on the host — 100k paths and hundreds of megabytes of transcripts cannot ship to a remote browser — so the host has to search them and the browser has to merge what comes back without the two rankings drifting.
Stacked on #804; second PR of the global-search stack.
Redesign (Oct 26). With the stack cut down to a finder (see #804), this PR searches one root instead of every open project's folder, previews files and specs through the shared resource renderers instead of a host text head, and drops the file/spec/plan preview targets from the wire (commit
refactor(search): search one root and render file previews client-side; protocol stays at the stack's search version, nothing shipped).Rebased on
main(Oct 28, twice).maintook protocol v92–v95 (#937, #869, #808, #975 repository cloning), soSEARCH_PROTOCOL_VERSIONis now v96, besidemain'sPROJECT_CLONE_PROTOCOL_VERSION(v95). The server module graph keepsmain'senvironmentedge beside the stack'ssearchedge. #804's final matcher takes quoted terms as literals and folds case and Latin accents; the host providers now match the same way (commitfix(search): match host providers with the palette matcher's quoted terms and folding). A host hit opens through #804's guarded palette navigator, so a later pick or navigation wins over a hit still loading its project.Review (Oct 28). A design review and eighteen rounds of code review (GPT-6 Astra, Claude Fable 5.1), plus the review bot, changed:
fs.readFileamaxBytesthe host enforces. A transcript preview follows its message across compaction.fs.readableFileSize, injected by the host), so a tracked symlink out of the worktree or into.gitis no longer a hit that previews to an error. The palette input stops at the host'sMAX_SEARCH_QUERY_LENGTHand both sides parse the same cut text, so a long paste never shows host rows for terms the host never saw.Approach
One matcher, two rings. The host ranks with the same
matchFieldsthe browser uses (the namedcontractsruntime exception from PR1), and every hit carries its tier, score and highlight ranges. The browser ranks host rows with local rows by those numbers instead of re-matching a title, so a file whose folder matched still ranks where the matcher put it. Body and transcript matches get a newcontenttier that always sorts below any field match.One root. Files and specs come from the active worktree, or on Project Home from the context project's folder through its Default record (the shell loads the project's workspaces when the palette opens there, so the record's id exists for reading and opening a hit). No root — no context project, a closed project, a Default record not persisted yet — answers empty file and spec groups. Other open projects and sibling worktrees are never searched (Cursor walked back global multi-root file search; one root is faster and predictable). Chats come only from persisted worktrees of open projects (pi-CLI sessions drop out, as in
history.search); plan files whose session has no transcript are skipped.Bounds at the source.
git ls-files -co --exclude-standard -zstreamed throughgitAsyncStreaminto a capped NUL parser (100k names / 16 MB →truncated), a bounded walk for non-git roots, per-root single-flight caches with a twin folded by the matcher's ownfoldMatchCase, invalidated byworkspace.fsChangedand a TTL. File matching is two-pass: a substring prefilter over the folded twin, then every other path that passes a cheap subsequence test goes through the matcher (acronym, camel hump, subsequence). 100k paths answer in 25–55 ms (measured). Spec snapshots build off the request on a read (60 s TTL, invalidated byfsChanged, at most eight roots). Transcripts are scanned newest first with the budget checked per message. Plans are re-parsed only when their file changed. The history index keeps its cold-build budget, and the transcript preview reads at most seven messages of 600 characters.Honest provider states. Each provider gets a time budget and returns
truncatedwhen it runs out; a cold file list answersindexingand starts loading; a throwing provider answersfailed. The browser polls only the kinds stillindexing(ten times every 300 ms, then every second, about 30 s in all), nevertruncatedorfailed, and says in a status line when what it shows is incomplete.Superseding, not cancellation. The wire has no cancel, so the host keeps the latest
seqper client: an olderseqis answeredsupersededat once, and a query overtaken while it awaited the index returnssupersededinstead of stale groups. The browser's controller is pure and unit-tested: 100 ms debounce, one request in flight plus the latest queued input, replies fenced by input,seq, welcome generation andsuperseded.Rows above the selection never reorder. Host rows that arrive after the user moved the selection join an arrival cohort and sort after every row already shown. A reply for the same query keeps the rows its page left out. The selected host row stays listed and visible while the search stands. Rows the previous query answered are matched again on the host's own fields until the new reply lands.
Narrow reads where the data lives.
HistoryIndex.sessions(),spec.specSnapshot/invalidateSpecSnapshot,todos.listStoredPlans(Git-free — no artifact settling, no review decoration). Thesearchmodule is registry-free and reads no file content;host/search.tscomposes it withlistProjects/listAllWorkspaceRecords, thehistoryScope.tsshape.Previews. The
⌘Ipane has three renderings:search.preview, the only target left on the wire). It carries the hit's anchor, so it survives compaction.ResourcePreviewfrom@thinkrail/resource-renderers, the same renderer as a preview tab, reading throughfs.readFile. It shows highlighted code, images, sandboxed HTML and aSPEC.mdas markdown. It reads only once the selection rests for 150 ms, and never a file over 2 MiB.searchnever imports the renderer package: the shell injects it as thePaletteHost.renderFilePreviewrender prop. A plan row has no preview.Rejected: an all-client search (ships the corpora to remote clients), a second scorer on the host (tiers would drift), FTS5 (needs the history module's no-disk-writes boundary changed — deferred), and live-session titles from the agent registry (a session file appears with its first message and the index revalidates every 2 s; PR4 adds titles to state records).
Changes
packages/contracts:SEARCH_PROTOCOL_VERSION = 95(PROTOCOL_VERSIONbump);foldMatchCaseexported beside the matcher;search.query/search.preview;Search*DTOs and limits:SearchHit.size;SearchPreviewTargetis thechattarget only, withanchorText;SearchPreviewResultis a transcript window orunavailable;MATCH_TIER.content;fs.readFile's optionalmaxByteswithFILE_TOO_LARGE, andMAX_SEARCH_PREVIEW_BYTES.packages/server/src/search(new barrel + SPEC):createSearchService(query, preview,invalidateRoot,forgetClient) andresolveSearchRoot;FileListCache(with overtaken loads and transient-failure keeping),listRootFiles,parseNulList;packages/server:host/search.tscomposition and the two handlers;search.querywatches its context workspace;invalidateRoot; client reap →forgetClient;evictSpecRootonworkspace.remove; a plan memo;fs/files.ts:readableFileSize(root, path), the file reader's containment rule (resolveUnderRoot, factored out ofresolveInWorktree) without a workspace lookup, whichhost/search.tsinjects as the search service's file check;search,host,history,spec,todos,watch,fs, the server module table and graph, andcontracts.apps/web/src/search:createHostSearchController+useHostSearch: stale marking, same-input retention, polls for indexing kinds only,hostSearchStatus;hostItems: host hits → rows; path highlight split between name and folder; dedupe against local chats and workspaces;rematchHitfor stale rows;hostTotals;arrivals.ts: arrival cohorts and the pinned selection;PaletteItem.match/late/previewTarget/file/hostFields;PaletteHost: context,openHit, names,renderFilePreview;/files and#specs scopes, behindsupportsHostSearchand a context project;apps/web/src/shell:SearchPaletteHostsupplies theResourcePreviewrender prop.openHostHit.tsopens a host hit: a file as a tab, a spec in the Specs tool, a transcript at its message (requestChatLocation), a plan with its chat and the Plan tool.paletteSearchContextnames the Default workspace on Project Home once its row is loaded.apps/web/src/transport:supportsHostSearch.search-palette.spec.ts):search.*request.Not in this PR: start work from the typed text (PR3), chats without a project (PR4), continue in workspace (PR5).
Screenshots
Same fixture (one seeded chat about a token refresh in
sample-project), 1280×800, preview pane open (⌘I)./sample.html: the HTML fixture rendered in the sandboxed preview#SPECGRAPHPROBE: a body match with its snippet, the spec rendered as markdownTesting
All on the final head (
38226e21f) unless noted.bun run lint,check:deps,check:boundaries,check:seams,check:spec-surface,check:i18n,check:compiler→ greenbunx turbo run typecheck --filter='!@thinkrail/desktop'→ 24/24 greencd packages/server && bun test→ 2035 pass (38226e21f; the later commits touch onlyapps/web);apps/webbun test→ 2046 passbun run test→ 24/24 (5424015e8, before the second rebase)bun run e2e(full no-agent suite, 8 shards) → all passed on the final head, and on every review head before it (d726abb2f,628055f21,f812f6c8a,b7bf07ca8,8a92b2986,a10694995,8ee103698,546d342f0,5424015e8,38226e21f)bun run e2e e2e/search-palette.spec.ts --repeat-each=3→ 45 passed; the grown-file cap test ×10 → 10 passed (546d342f0). The selection test answers the palette's searches itself and holds the poll until after ArrowDown; it passes 6/6 and fails 3/3 with the arrival-tracking bug reintroduced.openHostHitfor every kind;readableFileSizeagainstreadFile(a contained, an escaping, a.git, a dangling and a looping symlink; a directory; a gone path), and the query cut on both sides;Related issues
Follows up #613 (the deferred type-to-filter palette) and the file-search gap in #644 (file names here; file contents stay a separate Find in Files).
Checklist
bun run lint,bun run typecheck,bun run testbun run e2e, orbun run e2e:fullwhen touching agent behavior)SPEC.md/ top-level specs updated to reflect any boundary, contract, or behavior change