Skip to content

refactor: host planner warehouse inspection and CLI helpers in Rust - #1110

Merged
kvlonge merged 11 commits into
mainfrom
feat/native-group-11
Oct 15, 2026
Merged

kvlonge merged 11 commits into
mainfrom
feat/native-group-11

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Why

Continue Rust-hosted planning and CLI execution while preserving adapter overrides, persisted formats, diagnostics and exit codes.

Changes

  • Native planner warehouse inspection (snapshots, concurrent cursor reads, source inspection and deferral, retention and table-type planning) and planning phases; the migration fingerprint cache is a native class with a byte-identical cache file.
  • Native dbt interop glue: execution argument parsing, config, vars, profile connections, compile reference resolver and init progress.
  • Native CLI namespace, launch-record state, compile entry, build cost finalization, check planning, offline refactor compile and the defer-clone prephase.
  • Adapter calls remain Python callbacks in the same order; modules only imported from Rust gain a leading underscore.
  • Delete replaced Python; movable Python decreases from 27,784 to 21,360 lines.

Verification

  • 11,427 unit/integration tests passed; make check-ci passed with all optional dependencies available.
  • Compiler differential: 371 projects, unchanged goldens. Planner differential: 11 projects matched the main baseline.
  • Cold and edited compile outputs and exit codes match the reference on synthetic and private compile-only projects.
  • Worker captures: 24 warehouse scenarios with identical adapter call sequences, 35 CLI and dbt command captures.
  • Independent review: two low-severity parity findings fixed and verified.

Add pure planner decisions for warehouse inspection: physical cursor read
grouping and statements, retention directions and change phases, table
type copy names, missing source table messages, unknown freshness
warnings, full rebuild causes, managed source call scanning, deferral
target templates, and the migration fingerprint cache key and entry
format, byte-compatible with the stored cache rows.
Move warehouse snapshot gathering, cursor bound reads, source columns,
semantic source completion, source table checks, managed source
deferral, planner source freshness, the execution planning phases,
retention and table type planning, and the migration fingerprint cache
into native code. Adapter, catalog and connection calls stay Python
callbacks in the same order and count; the fingerprint cache keeps its
SQLite file and row format. Delete the replaced Python and move tests of
removed internals to integration tests over the native entry points.
…ue natively

Parse sqb dbt execution flags with the native argparse engine over an exported
parser table, resolve dbt config overrides, vars, plan options, manifest path,
interop adapter, dbt-profile connections and the compile-time dbt reference
resolver in Rust, and render dbt init progress with a native reporter. The
replaced Python modules are deleted.
…ry natively

CliNamespace becomes a native class whose defaults come from a Rust table, the
offered launch record lives in native context variables read from the leading
launch record option, and run_compile builds the reuse request and replays or
runs the compile in Rust. The replaced Python modules are deleted.
…r-clone natively

Move build cost finalization, Python check run-state recording, dependency
relation targets, relevant-check selection and read-side preflight, the offline
refactor compile, and the build defer-clone prephase into the native build,
check and refactor owners. Executor, adapter and planner traversal callbacks stay
in Python; the clone executor entry modules only Rust imports gain a leading
underscore. The replaced Python modules and their dead models are deleted.
@kvlonge
kvlonge enabled auto-merge (squash) October 15, 2026 04:03
@kvlonge
kvlonge merged commit 8ebe322 into main Oct 15, 2026
40 checks passed
@kvlonge
kvlonge deleted the feat/native-group-11 branch October 15, 2026 06:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant