Skip to content

fix(cll): support the sqlglot 30.19 set operation scope name - #1559

Merged
kentwelcome merged 1 commit into
mainfrom
fix/sqlglot-set-operation-scopes
Sep 23, 2026
Merged

kentwelcome merged 1 commit into
mainfrom
fix/sqlglot-set-operation-scopes

Conversation

@wcchang1115

@wcchang1115 wcchang1115 commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

PR checklist

  • Ensure you have added or ran the appropriate tests for your PR.
  • DCO signed

What type of PR is this?

fix

What this PR does / why we need it:

sqlglot 30.19.0 renamed Scope.union_scopes to Scope.set_operation_scopes. Recce allows sqlglot>=27.29.0,<31.0, so a new install gets 30.19.0. With that version, column-level lineage and breaking change classification return unknown for models with set operations such as UNION. CI jobs that build a new tox environment also fail 9 tests in tests/util/test_cll.py and tests/util/test_breaking.py.

This PR reads the scope list by either name, so Recce works with sqlglot before and after the rename.

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

🤖 Generated with Claude Code

sqlglot 30.19 renamed Scope.union_scopes to Scope.set_operation_scopes.
With that version, column-level lineage and breaking change
classification returned unknown for queries with set operations such as
UNION. Both now read the scope list by either name.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Wei-Chun, Chang <wcchang@infuseai.io>
@wcchang1115 wcchang1115 self-assigned this Sep 23, 2026
@codecov

codecov Bot commented Sep 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
recce/util/cll.py 80.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
recce/util/change_classifier.py 91.37% <100.00%> (+0.09%) ⬆️
recce/util/cll.py 94.41% <80.00%> (-0.20%) ⬇️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@wcchang1115
wcchang1115 marked this pull request as ready for review September 23, 2026 07:13

@iamcxa iamcxa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wcchang1115 Approving — the compatibility shim is correct across the entire declared sqlglot window, verified empirically rather than by reading.

What this PR changes

  • Adds set_operation_scopes(scope) in recce/util/cll.py that reads the sub-scope list under either the pre-30.19 name (Scope.union_scopes) or the post-30.19 name (Scope.set_operation_scopes).
  • Routes both call sites through it: _cll_set_scope (CLL) and _diff_union_scope (breaking-change classification).

Verification

Check Result
Rename boundary scan (27.29.0 / 28.0.0 / 29.0.0 / 30.0.0 / 30.18.0 / 30.19.0) Rename lands exactly at 30.19.0. No version in >=27.29.0,<31.0 exposes both names, so the hasattr precedence can never pick a stale alias. 30.19.0 is the newest release, so <31.0 resolves there today.
tests/util/test_cll.py + test_breaking.py @ sqlglot 30.19.0, main 9 failed, 101 passed — AttributeError at change_classifier.py:348. Reproduces the reported break exactly.
Same tests @ 30.19.0, this branch 110 passed
Same tests @ 30.18.0 (last pre-rename) 110 passed
tests/util/ @ 27.29.0 (floor of the declared range) 285 passed — old path intact
Full tests/ @ 30.19.0, this branch 1707 passed, 5 failed. The 5 are test_server.py SPA-route tests failing identically on main in the same env (no recce/data/ frontend build present) — pre-existing, unrelated.
black --check / isort --check / flake8 on both changed files clean
Rollout completeness git grep union_scopes on this head returns only the helper's own fallback — no missed call site.
Import cycle cll.py does not import change_classifier, so the new change_classifier → cll edge is safe.

exp.SetOperation scope keys are unchanged in 30.19 (union / intersect / except), so the scope_type dispatch in cll() and the expression.key == "union" asserts in _diff_union_scope need no matching update.

Non-blocking notes

1. The new-name branch has no coverage in the locked environment. uv.lock pins sqlglot==30.11.0, so uv sync (the Test Python Versions jobs, and local make test) always takes the union_scopes fallback — that is the single line Codecov reports as missing on this patch. The new branch is exercised, but only by the Test DBT Versions tox jobs, which build fresh envs and resolve to 30.19.0. That is adequate for now; a follow-up bump of the lockfile to 30.19.0 would make the default dev env match what a fresh pip install recce actually gets, and stop a future regression from passing locally.

2. change_classifier.py:352 — see inline.

Neither note blocks merge.

old_sub_scopes = set_operation_scopes(old_scope)
new_sub_scopes = set_operation_scopes(new_scope)
assert len(old_sub_scopes) == len(new_sub_scopes)
assert new_sub_scopes is not None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT (pre-existing, preserved by the rewrite): this is not None guard is unreachable as a guard — line 351 already calls len() on new_sub_scopes, so a None would raise TypeError there first and this assert can never be the thing that fires.

Since these three lines were rewritten anyway, moving it above the len() comparison would make it do what it reads like it does:

old_sub_scopes = set_operation_scopes(old_scope)
new_sub_scopes = set_operation_scopes(new_scope)
assert new_sub_scopes is not None
assert len(old_sub_scopes) == len(new_sub_scopes)
assert len(new_sub_scopes) > 0

No behavioural impact on either sqlglot version — purely a readability fix, entirely your call.

@kentwelcome kentwelcome left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kentwelcome
kentwelcome merged commit b1f8218 into main Sep 23, 2026
24 checks passed
@kentwelcome
kentwelcome deleted the fix/sqlglot-set-operation-scopes branch September 23, 2026 07:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants