Repository navigation
refactor: read planner inputs natively and build the plan output once - #1119
Merged
Merged
Conversation
…y in planner readers Fixture metadata, fixture reads, expected fixture types, schema findings and migration column readers use the typed analysis accessors instead of building Python projections: inferred rows, star flags, lineage facts with transform and confidence values, and proven dynamic-pivot inputs. Lane B's accessors are merged in; lineage facts and proven inputs are added.
…gh typed accessors Native compiled models keep their name in the compiled record instead of deriving it from the model file's path on every read. Planner, migration, scenario, SQL-test and janitor readers take a model's name, key, destination and query SQL through the typed model reads, which answer native views directly and fall back to attributes for hand-built records.
…lan context once Per-model planning validates cursor inputs and capped watermark inputs, resolves the materialization, describes and checks full rebuilds and builds cursor input relations through direct native calls instead of looking each one up on the native module. The model plan context's fields are read once per plan instead of once or more per model.
…ect native calls Reference resolution passes the SQL between the source, ref, dbt ref, UDF and table-function resolvers as Rust strings and reads the plan context's relations once per resolution. Cursor bounds, cursor input roles, cursor intrinsics, the unresolved-marker check, schema actions, model warnings and the cursor type check are called directly instead of through the native module, and function SQL resolution no longer builds a plan context record to read it back.
The execution plan calls scope resolution, cursor override typing, change detection, warehouse state, identities, buildability, model actions, source freshness, pruning, reconciliation, entry building and plan output assembly directly, and warehouse state calls model, column and old-name migration planning and the relations context directly, instead of looking each phase up on the native module. The SQL-test planning exit test now fails the plan with a missing source table instead of replacing a phase.
…ld the plan output once The fixture planning context is a native class holding relation columns, authoritative names and expected types as Rust maps, with each model's lineage reads and type projections derived once per context instead of once per SQL test. Fixture completion, selection, empty-fixture rendering, typed-null completion and expected-output checks read it directly, and the completion's expected types are the chain's propagated types over the context's instead of a ChainMap. The Python FixtureColumnMetadata and FixtureRelationMetadata records and the per-step fixture pyfunctions are gone; SQL-test planning and scenario fixtures call the context, chain resolution, test planning and completion directly. A native counter reports context builds and chain resolutions for tests. The plan output is built once: base fields, storage policies, warnings and direct-mode metadata are gathered as fields and the PlanOutput is constructed at the end instead of being rebuilt by each phase.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The planner still read compiled models, fixture metadata and plan context through Python attributes and module lookups, and rebuilt the plan output several times per plan.
Changes
Verification