Skip to content

fix(auto_ensure,inspect): serve pre-index edits when the head_sha sidecar is lost; read the L1 overlay in inspect - #863

Merged
coseto6125 merged 2 commits into
mainfrom
fix/head-sha-sidecar-fallback
Oct 23, 2026
Merged

coseto6125 merged 2 commits into
mainfrom
fix/head-sha-sidecar-fallback

Conversation

@coseto6125

@coseto6125 coseto6125 commented Oct 23, 2026 •

Copy link
Copy Markdown
Owner

Why

Outcome. A file created or edited before ecp admin index is now served by the L1 overlay even when the graph.bin.head_sha sidecar is missing or empty: ecp find untracked_fn answers "found":true instead of "found":false. This fixes the macOS (#862) and Windows (#861 run on 121cc51) failures of an_untracked_file_stays_out_of_the_commit_keyed_graph. Door: two-way. No stored format changes; a revert restores the old reader and the detached write. Blast radius: git_fingerprint_shortcut → ensure_index (every query's freshness check; upstream ensure_fresh_with_rebuild, check_in in doctor, maybe_reindex_notice in the post-tool-use hook, fetch_freshness in summary; 77 transitive). write_head_sha_sidecar_with_sha has 3 callers (build publish, fingerprint attach, overlay refresh).

admin index wrote the HEAD-SHA sidecar on a std::thread::spawn that nothing joined. When the process exited first, the sidecar was missing or 0 bytes. git_fingerprint_shortcut then returned None, and ensure_index fell back to the mtime walk, which cannot see files older than graph.bin. The result was Ready, so no overlay fragment was written, and every later query kept answering found:false. Reproduced 100% on 0.16.2 and on main by deleting or truncating the sidecar after admin index. The race predates #861.

Second fix in this PR (FU-2026-06-10-a8af472d45f7): ecp inspect ignored the L1 overlay that find, impact and cypher use. It read the legacy load_overlay + merge_archived layer and always answered with the base node. On a dirty Rust file, a function shifted 6 lines reported its committed span (6-8 instead of 12-14), a new function answered Symbol 'shortcut' not found, and a deleted one stayed inspectable. Door: two-way (no stored format changes; overlay fragments are read through the same accepted_fragments path as before). Blast radius: inspect::run/search_nodes/build_inspect_block (the ecp inspect command and the MCP inspect tool); MergedNode::file_path (shared by impact and cypher, now files.get instead of an index panic on a malformed graph).

What changed

  • fix(auto_ensure): git_fingerprint_shortcut also accepts a graph whose commit-dir name (commit_dir_sha) is HEAD. The dir is published by the same atomic rename as graph.bin, and graph_behind_head already treats the name as the authority. With neither a sidecar nor a parseable dir, it still returns None before any git call, so a legacy layout pays nothing new.
  • fix(auto_ensure): write_head_sha_sidecar_with_sha writes synchronously, as write_builder_fingerprint_sidecar already does for the same reason. The compatible-version sidecar stays detached: its loss only costs a header validation.
  • docs: the comments that called the head-SHA sidecar a pure perf hint now say what its loss costs.
  • test: wait_for_sidecar is deleted; the write it waited for is synchronous now.
  • fix(inspect): search and every block section run over MergedGraph built from engine.overlay_view(): a surviving dirty symbol reports its worktree span and overlay edges (clean callers included), an overlay-only symbol gets a full block, a deleted one is not found. A same-uid redefinition of a non-overload symbol is listed once, as the index does. Sections that need base-only data degrade to honest empty values for an overlay-only symbol: blind_spots empty, decorators from the base twin, contained members miss members the edit added (overlay builds no containment edges).
  • chore: deleted the legacy overlay layer with no production caller left: Overlay, merge_archived, MergeIter (ecp-core), load_overlay, load_overlay_hits, OverlayHit, for_each_fragment (ecp-cli), overlay_merge_zero_copy.rs and the two tests that exercised only load_overlay_hits.
  • test: tests/inspect_overlay_view.rs, 14 languages. Oracle: inspect on the dirty tree equals inspect after git commit + admin index (file, span, Calls names both ways) for caller and fresh; doomed is not found. Plus namesakes in clean and dirty files, owner filter on a dirty method, toon output, clean tree with a session.

How to verify

Red before, green after (source files swapped to HEAD, tests kept):

Test Before After
auto_ensure::test_ensure_index_head_sha_sidecar_lost_graph_at_head_reports_prior_edits_stale (missing and empty sidecar, untracked + tracked edits older than graph.bin) got Ready ok
auto_ensure::test_ensure_index_head_sha_sidecar_other_sha_graph_at_head_reports_untracked_stale FAILED ok
l1_overlay_integration::test_find_head_sha_sidecar_lost_after_index_untracked_file_found {"found":false,...,"total_candidates":0} ok
inspect_overlay_view (5 tests, 14 languages) 4 FAILED: Symbol 'fresh' not found, (2, 4) base span ok
inspect::tests::test_search_nodes_same_uid_redefinition_lists_first_only left: [7, 8] ok
auto_ensure::test_ensure_index_head_sha_sidecar_lost_graph_behind_head_reports_stale (guard: a dir naming an older commit must not take the shortcut) ok ok; red under the mutation dir_sha.is_none()
  • cargo test -j 4 -p egent-code-plexus --tests: 2185 passed, 0 failed; cargo test -j 4 -p ecp-core: 587 passed (6 merge_archived tests deleted with it); cargo test -j 4 -p ecp-mcp: 51 passed.
  • cargo clippy -j 4 -p egent-code-plexus --tests and -p ecp-core --tests: 0 warnings.
  • Review, sidecar fix: opus deep-review (Correctness, Quality, Efficiency) and codex. Every finding at 50+ is fixed: the stale perf-hint doc, two untested states, the synchronous write. The l1 test now derives sidecar paths from graph.bin and asserts that a graph was published.
  • Review, inspect migration: opus deep-review (Correctness, Efficiency) and codex. Clean-tree output is identical to HEAD field by field; the no-overlay path adds no allocation and drops HEAD's second base_constructors call and its all-node uid map. Fixed: same-uid duplicate listing (65), dead legacy reader and its stale docs (55). Filed: FU-2026-10-24-0d2a9372c6a7 (find and cypher keep the same-uid duplicate; pre-existing).
  • Not changed, pre-existing: a file edited after the build's clean check and reverted after publish can leave HEAD-named content that differs from HEAD. This also affects the matching-sidecar path, and it is not new here.

…lost

admin index wrote graph.bin.head_sha on a detached thread nothing joined.
A fast exit left it missing or empty, git_fingerprint_shortcut returned
None, and the mtime walk reported Ready because it cannot see files older
than graph.bin. No overlay was written, so an untracked file left out of
the commit-keyed graph stayed unfindable (CI flake in
an_untracked_file_stays_out_of_the_commit_keyed_graph on macOS/Windows).

- The shortcut also accepts a graph whose commit-dir name is HEAD; with
  neither a sidecar nor a parseable dir it still returns before any git call.
- write_head_sha_sidecar_with_sha writes synchronously, like
  write_builder_fingerprint_sidecar.
- Tests: sidecar missing/empty at HEAD, other-SHA sidecar at HEAD, graph
  behind HEAD with sidecar lost, and the l1 integration case.
@coseto6125
coseto6125 enabled auto-merge (squash) October 23, 2026 16:44
@coseto6125 coseto6125 added the merge-queue Opt-in to Mergify merge queue label Oct 23, 2026
@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: cd24d05e-9145-45a4-9311-a68c86c2d37b
📥 Commits

Reviewing files that changed from the base of the PR and between 9ff296a and ff16697.

📒 Files selected for processing: 4
  • crates/ecp-cli/src/auto_ensure.rs
  • crates/ecp-cli/src/build/orchestrator.rs
  • crates/ecp-cli/tests/auto_ensure.rs
  • crates/ecp-cli/tests/l1_overlay_integration.rs

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The HEAD-SHA sidecar is now written synchronously. The freshness shortcut checks the commit-directory SHA when the sidecar is missing or empty. Tests cover stale detection and session-overlay access under these conditions.

Changes: HEAD-SHA freshness

Layer / File(s) Summary
Sidecar publication and freshness checks
crates/ecp-cli/src/auto_ensure.rs, crates/ecp-cli/src/build/orchestrator.rs, crates/ecp-cli/tests/auto_ensure.rs
The sidecar write is synchronous. The freshness shortcut compares available sidecar and commit-directory SHAs with HEAD. Tests cover stale detection when these SHA sources are missing, empty, or mismatched.
Overlay access with absent sidecar
crates/ecp-cli/tests/l1_overlay_integration.rs
Integration tests verify that an untracked file remains outside the commit-keyed graph and is discoverable through the session overlay after the sidecar is removed or emptied.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related PRs

Merge Risk: ⚪ Minimal · up to ff166

The change makes the HEAD-SHA sidecar write synchronous and lets the freshness check use the commit-directory name when the sidecar is missing, so the L1 overlay is no longer lost to an unjoined background write. No actionable merge-blocking risk was found.

Pre-merge checks: Passed 5
Check name Status Explanation
Docstring Coverage Passed Docstring coverage is 93.75% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 16 functions across 4 files.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Title check Passed The title clearly identifies both primary fixes: serving pre-index edits when the HEAD-SHA sidecar is lost and reading the L1 overlay in inspect.
Description check Passed The description is detailed and covers the change rationale, implementation, scope, tests, and verification results. It uses equivalent headings such as "Why," "What changed," and "How to verify" inst…
Full details: Description check

Explanation

The description is detailed and covers the change rationale, implementation, scope, tests, and verification results. It uses equivalent headings such as "Why," "What changed," and "How to verify" instead of the template headings "Summary" and "Test plan," but the required information is present.

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor
ecp impact cache (0 symbols) — internal, used by ecp dev pr-analyze

[]

@github-actions github-actions Bot added the ecp:risk-low ecp signal label Oct 23, 2026
@coseto6125
coseto6125 marked this pull request as draft October 23, 2026 17:04
auto-merge was automatically disabled October 23, 2026 17:04

Pull request was converted to draft

…mpact

inspect still read the legacy load_overlay + merge_archived layer and
always answered with the base node: a symbol in a dirty file showed its
committed span, a new symbol was "not found", and a deleted one stayed
inspectable. Search and every block section now run over MergedGraph
built from engine.overlay_view(). A same-uid redefinition of a
non-overload symbol is listed once, as the index keeps it.

- Delete the legacy layer with no production caller left: Overlay,
  merge_archived, MergeIter, load_overlay, load_overlay_hits, OverlayHit,
  for_each_fragment and the tests that exercised only them.
- MergedNode::file_path uses files.get instead of indexing.
- tests/inspect_overlay_view.rs: 14 languages, dirty-tree inspect must
  equal inspect after commit + reindex.

Resolves FU-2026-06-10-a8af472d45f7.
@coseto6125 coseto6125 changed the title fix(auto_ensure): serve pre-index edits when the head_sha sidecar is lost fix(auto_ensure,inspect): serve pre-index edits when the head_sha sidecar is lost; read the L1 overlay in inspect Oct 23, 2026
@coseto6125
coseto6125 marked this pull request as ready for review October 23, 2026 17:36
@coseto6125
coseto6125 enabled auto-merge (squash) October 23, 2026 17:36
@coseto6125
coseto6125 merged commit 1e10e94 into main Oct 23, 2026
22 checks passed
@coseto6125
coseto6125 deleted the fix/head-sha-sidecar-fallback branch October 23, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecp:risk-low ecp signal merge-queue Opt-in to Mergify merge queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant