Skip to content

refactor: port planner change detection and strategy, progress reporters and compiler glue to the native extension - #1105

Merged
kvlonge merged 40 commits into
mainfrom
feat/native-group-6
Oct 14, 2026
Merged

kvlonge merged 40 commits into
mainfrom
feat/native-group-6

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Why

Rust-hosted SQLBuild port (CHI-585), sixth combined PR: Phase 4 (CHI-593) planner slices with a new planner differential, Phase 3 (CHI-592) progress reporters, and Phase 1 (CHI-589) glue, with user-code and public-API modules marked as staying Python.

Changes

  • Planner: change detection, strategy/action selection, audit scheduling, microbatch bounds and limits, cursor type checks, lineage resolution.
  • make planner-differential: builds projects through 18 change steps and compares plan text/JSON, full plan output and state rows.
  • Progress: build callbacks, audit/nested reporters, build footer, status spinner (no longer uses rich), with an exit-time spinner join fix.
  • Phase 1: render/assembly, spec config resolution, clone pipeline, selection phases, scopes and compile glue, sql_values/references/manifest/lint helpers, Python-node run lifecycle, rule selection.
  • ~45 modules marked stays (user code or public API), recorded on CHI-589.
  • Movable Python: 62,087 → 51,236 lines.

Intentional differences: TTY spinner bytes; traceback frames; internal Python APIs removed or renamed.

Verification

  • Planner differential: 11 projects × 18 steps identical to a pre-change baseline.
  • Generated comparisons vs the old Python: planner strategy/bounds/audits (~132k), progress callbacks (~34k scripted runs), render/declaration and state literals: 0 mismatches.
  • 10,855 focused tests incl. ~27 e2e files; compiler differential with full coverage; byte-identical compile output on a 1k-model and a large private project; make check-ci.
  • Independent review: no findings.

Callers use the native resolve_cursor_intrinsics, validated_model_cursor_intrinsics
and cursor_intrinsics_analysis_sql bindings; the Rust glue calls the shared
helpers directly and owns the reserved executor cursor markers.
Template data, effective vars, early model templates, template read recording
and run ids now run in the Rust glue; Python callers use the native
expand_template_data, expand_effective_vars, record_template_reads and
resolve_run_id bindings.
Declaration scope resolvers, per-path declaration contexts, constant and
enum member rendering, project var substitution and authored SQL expansion
with spans now run in the Rust glue. The Python wrappers and the unused
raw interpolation, declaration scan and declaration expansion bindings go.
The native assemble_native_project_resources binding reads the compile
inputs, renders opaque vars, looks up template environment names and
materialises NativeProjectResources itself. The Python request and resource
helpers, their constants and row types, and the reviewed handler entry for
the removed opaque var renderer go.
Target name and config, effective adapter name, collection rendering,
scenario config and dialect overrides, loader destination parts and dynamic
family matching are native bindings. The spec/contracts main and _helpers
wrappers and the NamePattern class go; Rust callers use sqlbuild._native.
Origin and destination compilation, source deferral targets and the
ClonePipelineResult are assembled by the native run_clone_pipeline_with_options
binding, which the clone command calls directly.
The scope-lookup handoff to NativeDeclarationContexts moves into the Rust
declaration resolver, removing the last model_loop module. Loader namespace
and warehouse templates call the Rust template expansion directly.
Unit-test selector splitting, configured-rule and analysis model selection,
and the static command compile/scope/relation phases are native bindings;
Rust callers use sqlbuild._native.
@kvlonge
kvlonge enabled auto-merge (squash) October 14, 2026 06:21
@kvlonge
kvlonge merged commit 4a3a55a into main Oct 14, 2026
76 of 78 checks passed
@kvlonge
kvlonge deleted the feat/native-group-6 branch October 14, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant