Skip to content

call_edges: replace edges per file version - #117

Open
vladwing wants to merge 1 commit into
facebookexperimental:mainfrom
vladwing:call-edges-per-version
Open

vladwing wants to merge 1 commit into
facebookexperimental:mainfrom
vladwing:call-edges-per-version

Conversation

@vladwing

Copy link
Copy Markdown
Contributor

Callers lookups miss callers at some commits a --git run indexed, and the call_edges row count differs between identical --git runs.

sync_for_files() deletes every call_edges row of each path in a batch, at every version, before appending the batch's edges. A path has one version per commit that changed it, and parallel inserters write those versions in different batches, within one run or across runs. Each batch therefore deletes the edges of the path's other versions, which callers lookups need at the revisions that hold those versions. Which versions survive depends on batch order, which is why the row count varies.

sync_for_versions() replaces sync_for_files() and deletes only the versions the batch writes, with one IN list over the key concat(caller_file_hash, ':', caller_file). The key is exact because a git blob id holds no :.

A process-global lock now serializes call_edges writes, as content upserts already are. Without it, concurrent deletes on one table conflict and retry, and two syncs of one version could both delete before either appends, leaving its rows twice. The backfill runs only on an empty table and appends without deleting.

tests/call_edge_versions.rs fails on main and passes here. It indexes c0..c1 and c1..c2 as two process_git_range() calls and checks the callers at each commit. The same-version concurrency unit test fails on every run with the lock removed.

Callers lookups miss callers at some commits a `--git` run indexed, and
the `call_edges` row count differs between identical `--git` runs.

`sync_for_files()` deletes every `call_edges` row of each path in a
batch, at every version, before appending the batch's edges. A path has
one version per commit that changed it, and parallel inserters write
those versions in different batches, within one run or across runs. Each
batch therefore deletes the edges of the path's other versions, which
callers lookups need at the revisions that hold those versions. Which
versions survive depends on batch order, which is why the row count
varies.

`sync_for_versions()` replaces `sync_for_files()` and deletes only the
versions the batch writes, with one `IN` list over the key
`concat(caller_file_hash, ':', caller_file)`. The key is exact because a
git blob id holds no `:`.

A process-global lock now serializes `call_edges` writes, as content
upserts already are. Without it, concurrent deletes on one table
conflict and retry, and two syncs of one version could both delete
before either appends, leaving its rows twice. The backfill runs only on
an empty table and appends without deleting.

`tests/call_edge_versions.rs` fails on main and passes here. It indexes
`c0..c1` and `c1..c2` as two `process_git_range()` calls and checks the
callers at each commit. The same-version concurrency unit test fails on
every run with the lock removed.
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 11, 2026
@vladwing vladwing closed this Nov 11, 2026
@vladwing vladwing reopened this Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant