Skip to content

refactor: run sqb dbt, dbt interop, discovery leftovers, selectors and macro expansion natively - #1104

Merged
kvlonge merged 24 commits into
mainfrom
feat/native-group-5
Oct 14, 2026
Merged

kvlonge merged 24 commits into
mainfrom
feat/native-group-5

Conversation

@kvlonge

@kvlonge kvlonge commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Why

Rust-hosted SQLBuild port (CHI-585), fifth combined PR: the last CLI command family (dbt) and dbt interop internals (CHI-592), Phase 1 (CHI-589) discovery, selectors and macro rendering, and the first Phase 4 (CHI-593) planner slice.

Changes

  • sqb dbt command glue; dbt event-stream parsing; combined graph and selection; dbt argv, dbt ls parsing, routing, config, plan assembly and execution decisions.
  • Discovery leftovers; Python node selectors and graph indexes; macro expansion native, with project macro loading, the macro bridge and the store environment fingerprint kept as user code.
  • Planner display helpers (labels, cursor bounds, effective microbatch batch size, aligned bounds).
  • dbt defer-to e2e test now parses the dbt project first (it also failed on main).
  • Movable Python: 69,877 → 62,087 lines.

Intentional differences: when several selection violations exist the first in sorted order is reported; dbt JSON lines with NaN/Infinity are ignored; traceback frames; internal Python APIs removed or renamed.

Verification

  • Generated comparisons vs the old Python: dbt graph/selection/argv/routing/plans/execution decisions (~23k), event streams (3,000 + real dbt logs), Python node selectors (~100k): 0 mismatches.
  • Recorded sqb dbt CLI runs (33) and build/plan runs (52): identical apart from spinner and traceback frames.
  • 10,719 focused tests incl. ~20 e2e files (dbt locally); compiler differential with full coverage; byte-identical compile output on a 1k-model and a large private project; make check-ci.
  • Independent review: no findings.

Port Python-node, unified SQL/Python run and check-exclude selector
resolution plus the Python node graph edge, tag and path indexes to
sqlbuild-analysis. The node inventory that reads loaded user functions
moves to compiler/python_nodes/_helpers/user_inventory.py (user-code).
…xpansion

Project macro module loading and the context mappings user macros read move
to render/user_macros.py, and the macro bridge and its render-stage runner to
macro_bridge/classes/macro_bridge.py, both kept as user-code. Macro expansion
now returns MacroExpansionResult natively, the expansion class table is built
natively, and find_macro_call_names replaces the raw native call-name scan.
@kvlonge
kvlonge enabled auto-merge (squash) October 14, 2026 02:47
@kvlonge
kvlonge merged commit ef0e9b6 into main Oct 14, 2026
40 checks passed
@kvlonge
kvlonge deleted the feat/native-group-5 branch October 14, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant