Skip to content

Bring the Rust binary to Python 0.15.1 parity - #2

Merged
IronAdamant merged 2 commits into
mainfrom
cursor/rust-python-parity-dfe5
Nov 3, 2026
Merged

IronAdamant merged 2 commits into
mainfrom
cursor/rust-python-parity-dfe5

Conversation

@IronAdamant

@IronAdamant IronAdamant commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

Closes the gaps from the 2026-11-03 parity review of Python v0.15.1 (70d433b) against Rust 0.16.1 (4122fa8). The crate stays std-only. A Python import chisel library is out of scope for this binary.

A follow-up on the same branch makes the large-tree commands faster than Python and matches Python's test-unit set.

Benchmarks

Five runs, median ms (min–max), on Linux 6.12.94+, 4 vCPUs, Intel Xeon, 15.6 GiB RAM, Rust 1.97.0, Python 3.12.3. Small repo: two Python files. Large repo: the django package plus 80 of its test modules (about 1000 code files). History is split into commits of at most 100 paths, because a shallow-clone boundary commit lists every file and the pairing dominates co-change time. Full RSS table: scripts/benchmark_results.md. Reproduce with scripts/benchmark_parity.py.

Command Python small Rust small Python large Rust large
startup (--help) 64.6 (63.4–73.4) 0.3 (0.3–0.5) 71.6 (64.7–76.1) 0.4 (0.2–0.5)
cold analyse 88.5 (86.7–93.9) 8.0 (7.7–10.8) 13692.5 (13542.5–14014.5) 8034.2 (7627.3–8398.0)
warm analyse 82.8 (80.7–89.8) 2.3 (2.2–2.5) 8646.5 (8604.2–8754.3) 63.0 (61.3–77.9)
warm update 71.0 (67.9–74.6) 2.6 (2.2–2.6) 146.4 (139.1–163.6) 68.8 (65.3–71.2)
impact 63.6 (63.0–66.2) 0.6 (0.6–0.7) 111.9 (106.5–113.3) 45.4 (44.6–48.2)
suggest-tests 64.3 (63.7–68.1) 0.6 (0.6–0.7) 125.9 (121.5–131.0) 46.0 (44.6–51.9)
risk-map 67.8 (65.9–75.4) 0.7 (0.6–0.9) 253.6 (233.9–261.0) 57.5 (55.8–64.0)
test-gaps 64.9 (63.2–69.7) 0.6 (0.5–0.6) 163.4 (161.9–180.5) 73.1 (65.0–76.5)
triage 65.8 (64.0–67.1) 0.7 (0.6–0.8) 254.4 (239.9–262.7) 94.9 (93.3–97.5)
churn 64.2 (62.9–66.6) 0.6 (0.6–0.7) 65.2 (64.4–69.0) 48.8 (46.0–51.4)
coupling 64.3 (63.3–65.5) 0.6 (0.6–0.7) 64.1 (63.3–66.1) 47.7 (41.8–53.4)
stats 64.8 (63.8–66.3) 0.6 (0.5–0.6) 67.4 (66.5–68.6) 46.0 (45.0–51.0)

Rust is faster on every command in both trees. On the large tree, cold analyse is 8.0 s against Python's 13.7 s, warm analyse is 63 ms against 8.6 s, and test-gaps is 73 ms against 163 ms. The small sample has 1 test unit on both sides. The large snapshots both have 829 test units, and the (file, name) pairs match.

Why the large tree was slow

  • Test edges walked every code unit for every import of every test unit. Resolution is now a path-suffix index, built once, with dependencies cached per test file.
  • test-gaps and triage scanned every edge for every code unit. Covered ids are a set.
  • Each query parsed the snapshot and cloned every row into indexes, then parsed it again. An unchanged file is not re-read. Indexes store offsets. Unescaped JSON strings share the document buffer.
  • Warm analyse repeated blame, git log -L, and edge building. Matching mtime, size, and HEAD returns the stored summary.
  • Warm update hashed every file. Matching mtime and size skips the read.

The 822 vs 829 test units

Python's extra seven were real tests. Rust missed eight methods in tests/migrations/test_multidb.py because a triple-quoted SQL string containing parentheses was not tracked across lines, so later def lines were skipped. Rust also emitted one placeholder unit for tests/bash_completion/management/commands/test_command.py, a Django management command whose filename looks like a test and which defines no test functions. Python emits no unit for that file. Blank lines no longer drop the enclosing class, so methods are Class.method, matching Python. After that, the large-tree test-unit names are identical (829).

Parity checklist

CLI

  • Global flags --project-dir --storage-dir --json --limit: kept. Rust still accepts them before the subcommand.
  • analyze: kept, including the Rust-only auto-queue above 300 files.
  • update output shape: fixed. code_units_found is the units parsed this run (0 on a no-op). orphaned_results_cleaned and active_file_locks are present.
  • impact --functions: fixed. Names limit the direct edges.
  • suggest-tests file: fixed for Java, C#, and Swift method names. Ruby file-level units are kept (see below).
  • suggest-tests --directory: fixed. Returns a {file: [tests]} map.
  • suggest-tests --fallback: fixed. The flag is accepted. Python’s engine also stem-ranks when the flag is off and nothing matched, and Rust does the same. With working_tree and a file missing from the snapshot, --fallback skips that stem list.
  • suggest-tests stale_db when the file is not in the snapshot: fixed.
  • churn --unit: fixed. One row, with insertions and last_changed from git log -L diff lines.
  • churn without --unit: fixed. The file-level row only.
  • ownership, history, who-reviews, diff-impact: kept. Already matched.
  • coupling --min-count: fixed.
  • coupling --working-tree: fixed. Untracked files get on-disk import partners and the working-tree note.
  • risk-map scores: fixed the causes called out (self-edges, require() as a code unit, missing heuristic edges, test files ranked as code). Remaining numeric gaps come from edge sets that are still not identical to Python’s.
  • risk-map directory: fixed.
  • --no-exclude-tests on risk-map, test-gaps, and triage: fixed.
  • --coverage-mode: kept.
  • --proximity / --no-proximity: fixed. Proximity stays on by default. --proximity is now accepted.
  • _meta.cycles: fixed. Tarjan SCC on directed imports, top three.
  • Lock info on risk-map: fixed. locked_by, _meta.active_locks, _meta.lock_claims when a lock is held.
  • Human risk-map diagnostics: fixed.
  • stale-tests shape: fixed. A list when edges exist. The no_edges object is unchanged.
  • test-gaps basic (calc.py:mod covered by a Java heuristic): fixed via weight-0.2 heuristic edges when a test file has no other edges. Stem match is case-insensitive, as in SQLite LIKE.
  • test-gaps file / --directory: fixed.
  • record-result --duration: fixed. duration_ms is stored and read by the duration-variance map.
  • stats shape: fixed. Adds commit_files, blame_cache, branch_co_changes, churn_stats, file_hashes, and shadow_graph. co_change_query_min is an integer. passed / failed stay (Rust-only).
  • triage directory, --top-n, --no-exclude-tests: fixed.
  • run pytest collection-error junk id: fixed. A status has to end the line. Underscore banners are dropped.
  • run pytest id :: to :: kept-as-Rust-improvement. Recorded ids match test-unit ids. Python stored the raw :: form, which missed those units.
  • run Jest: fixed. Adds --json and --outputFile, stores a project-relative id.
  • run npx / go / cargo flags: fixed. npx is treated as Jest (as in Python). go test gets -json. cargo test gets --message-format=json. Go and cargo output is still not parsed, matching Python.
  • run live output: fixed. Stdout and stderr are streamed as they arrive.
  • start-job positional kind: fixed.
  • job-status / cancel-job, HTTP routes, lock acquire/release/refresh/check: kept.
  • HTTP eval / new Function warning: fixed on suggest_tests for JS/TS files.
  • MCP stdio: fixed. Newline-delimited JSON is the reply for newline clients. Content-Length still works and is answered in that framing. serverInfo.version is the crate version.
  • MCP tool schemas: kept.
  • list-locks --agent-id: fixed.
  • optimize_storage: kept-as-Rust-improvement. There is no SQLite WAL to vacuum. The command still drops expired locks and rewrites the snapshot.
  • Human-readable output: fixed per command, including risk-map diagnostics.
  • Per-subcommand --help: fixed.

Storage and config

  • SQLite chisel.db as the live store: not applicable. The snapshot stays .chisel/store.
  • Migration of recorded results: fixed. chisel migrate [chisel.db] copies test_results (including duration and timestamp). It reads the main database and a sibling -wal when one is present. Other commands do not open chisel.db.
  • CHISEL_PROJECT_DIR, CHISEL_STORAGE_DIR, shards, CHISEL_INCLUDE_IGNORED: kept.
  • CHISEL_BOOTSTRAP / register_dep_extractor: fixed as an executable, which is the constraint-compatible replacement. CHISEL_DEP_EXTRACTOR, or dep_extractor in .chisel/extensions.toml (same allow-flag as the unit extractor), reads {"path","language","content"} and returns dependencies. If CHISEL_BOOTSTRAP is a path to an executable it is used the same way. A dotted Python import is not loaded. A warning is recorded instead.
  • register_extractor: kept as the existing CHISEL_EXTRACTOR executable.
  • Python library API (import chisel): not applicable. This tree is a binary.

Extraction

  • Language map, pytest, Go, Jest blocks, Rust #[test]: kept.
  • Playwright vs Jest label: fixed.
  • Java/Kotlin @Test methods of any name: fixed.
  • C# [Fact] / [Theory] / [Test] / [TestMethod]: fixed.
  • Swift @Test: fixed.
  • C/C++ gtest, Catch2, Boost: fixed. TEST, TEST_F, TEST_P, TEST_CASE, SCENARIO, BOOST_AUTO_TEST_CASE become test units, and those files are test files in risk-map.
  • Ruby RSpec file-level unit: kept-as-Rust-improvement. Python found no RSpec unit and ranked the spec as code. Rust records the file as a test.
  • Test edges, self-links, and require() as a code unit: fixed. Edges no longer point at the test file itself. const x = require(...) is not a function. Tests with no other edges get heuristic edges of weight 0.2.
  • Static test-import rows and import / co-change / blame: kept. JS require still contributes static import rows when the path resolves.
  • Rust-only features (watch, shards, flakiness trend, auto-queue, include_results, extractor gate): kept.
Open in Web Open in Cursor 

cursoragent and others added 2 commits November 3, 2026 12:30
Stdio MCP now answers newline-delimited JSON and still accepts Content-Length. CLI flags and directory scopes reach the engine, test detection covers the annotation and macro frameworks, and chisel migrate copies recorded results out of a legacy chisel.db. Benchmarks compare this build with Python 0.15.1.

Co-authored-by: Aron Amos <IronAdamant@users.noreply.github.com>
Edge building and test-gaps no longer scan every unit for every import or edge. Snapshot loads skip an unchanged file and share one buffer for unescaped strings. Warm analyse and update skip unchanged files. Python method detection now sees defs after triple-quoted strings, so the large tree matches Python's 829 test units.

Co-authored-by: Aron Amos <IronAdamant@users.noreply.github.com>
@IronAdamant
IronAdamant merged commit a84e18c into main Nov 3, 2026
5 checks passed
@IronAdamant IronAdamant mentioned this pull request Nov 3, 2026
3 tasks done
@IronAdamant
IronAdamant deleted the cursor/rust-python-parity-dfe5 branch November 4, 2026 06:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants