Repository navigation
fix(cll): support the sqlglot 30.19 set operation scope name - #1559
Conversation
sqlglot 30.19 renamed Scope.union_scopes to Scope.set_operation_scopes. With that version, column-level lineage and breaking change classification returned unknown for queries with set operations such as UNION. Both now read the scope list by either name. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Wei-Chun, Chang <wcchang@infuseai.io>
Codecov Report❌ Patch coverage is
... and 3 files with indirect coverage changes 🚀 New features to boost your workflow:
|
iamcxa
left a comment
There was a problem hiding this comment.
@wcchang1115 Approving — the compatibility shim is correct across the entire declared sqlglot window, verified empirically rather than by reading.
What this PR changes
- Adds
set_operation_scopes(scope)inrecce/util/cll.pythat reads the sub-scope list under either the pre-30.19 name (Scope.union_scopes) or the post-30.19 name (Scope.set_operation_scopes). - Routes both call sites through it:
_cll_set_scope(CLL) and_diff_union_scope(breaking-change classification).
Verification
| Check | Result |
|---|---|
| Rename boundary scan (27.29.0 / 28.0.0 / 29.0.0 / 30.0.0 / 30.18.0 / 30.19.0) | Rename lands exactly at 30.19.0. No version in >=27.29.0,<31.0 exposes both names, so the hasattr precedence can never pick a stale alias. 30.19.0 is the newest release, so <31.0 resolves there today. |
tests/util/test_cll.py + test_breaking.py @ sqlglot 30.19.0, main |
9 failed, 101 passed — AttributeError at change_classifier.py:348. Reproduces the reported break exactly. |
| Same tests @ 30.19.0, this branch | 110 passed |
| Same tests @ 30.18.0 (last pre-rename) | 110 passed |
tests/util/ @ 27.29.0 (floor of the declared range) |
285 passed — old path intact |
Full tests/ @ 30.19.0, this branch |
1707 passed, 5 failed. The 5 are test_server.py SPA-route tests failing identically on main in the same env (no recce/data/ frontend build present) — pre-existing, unrelated. |
black --check / isort --check / flake8 on both changed files |
clean |
| Rollout completeness | git grep union_scopes on this head returns only the helper's own fallback — no missed call site. |
| Import cycle | cll.py does not import change_classifier, so the new change_classifier → cll edge is safe. |
exp.SetOperation scope keys are unchanged in 30.19 (union / intersect / except), so the scope_type dispatch in cll() and the expression.key == "union" asserts in _diff_union_scope need no matching update.
Non-blocking notes
1. The new-name branch has no coverage in the locked environment. uv.lock pins sqlglot==30.11.0, so uv sync (the Test Python Versions jobs, and local make test) always takes the union_scopes fallback — that is the single line Codecov reports as missing on this patch. The new branch is exercised, but only by the Test DBT Versions tox jobs, which build fresh envs and resolve to 30.19.0. That is adequate for now; a follow-up bump of the lockfile to 30.19.0 would make the default dev env match what a fresh pip install recce actually gets, and stop a future regression from passing locally.
2. change_classifier.py:352 — see inline.
Neither note blocks merge.
| old_sub_scopes = set_operation_scopes(old_scope) | ||
| new_sub_scopes = set_operation_scopes(new_scope) | ||
| assert len(old_sub_scopes) == len(new_sub_scopes) | ||
| assert new_sub_scopes is not None |
There was a problem hiding this comment.
NIT (pre-existing, preserved by the rewrite): this is not None guard is unreachable as a guard — line 351 already calls len() on new_sub_scopes, so a None would raise TypeError there first and this assert can never be the thing that fires.
Since these three lines were rewritten anyway, moving it above the len() comparison would make it do what it reads like it does:
old_sub_scopes = set_operation_scopes(old_scope)
new_sub_scopes = set_operation_scopes(new_scope)
assert new_sub_scopes is not None
assert len(old_sub_scopes) == len(new_sub_scopes)
assert len(new_sub_scopes) > 0No behavioural impact on either sqlglot version — purely a readability fix, entirely your call.
PR checklist
What type of PR is this?
fix
What this PR does / why we need it:
sqlglot 30.19.0 renamed
Scope.union_scopestoScope.set_operation_scopes. Recce allowssqlglot>=27.29.0,<31.0, so a new install gets 30.19.0. With that version, column-level lineage and breaking change classification returnunknownfor models with set operations such asUNION. CI jobs that build a new tox environment also fail 9 tests intests/util/test_cll.pyandtests/util/test_breaking.py.This PR reads the scope list by either name, so Recce works with sqlglot before and after the rename.
Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
🤖 Generated with Claude Code