Skip to content

sql/opt: SplitDisjunctionOfJoinTerms has no fan-out cap, causing planning-time OOM #173718

Description

@yuzefovich

The exploration rules SplitDisjunctionOfJoinTerms and SplitDisjunctionOfAntiJoinTerms (pkg/sql/opt/xform/rules/join.opt) are the join-side analog of the select-side SplitDisjunction rule, but they are missing the fan-out cap that SplitDisjunction got in #172644.

Each split rewrites a join whose ON clause contains an OR into UnionAll(join1, join2), duplicating both scan tables (via DuplicateScanPrivate, which copies the full table's column metadata and never frees it) while preserving all other OR conjuncts. Both children then re-fire the rule. So for an ON clause that is an N-way AND of 2-way ORs, the memo and the retained table metadata grow ~2^N. This is pure optimizer exploration — independent of table size, invisible to --max-sql-memory — so planning alone can exhaust the node's heap.

Unlike SplitDisjunction, this is not bounded by optimizer_max_disjunction_split_count. It also can't be disabled via optimizer_use_improved_split_disjunction_for_joins, since equijoin disjuncts are "interesting" regardless of that setting.

Reproduction (empty tables, EXPLAIN only — no rows are read):

CREATE TABLE t1 (a INT PRIMARY KEY, b INT, c INT, d INT, e INT, f INT, g INT, h INT, i INT, j INT, k INT, l INT, m INT, n INT, o INT, p INT, q INT, r INT, s INT);
CREATE TABLE t2 (a INT PRIMARY KEY, b INT, c INT, d INT, e INT, f INT, g INT, h INT, i INT, j INT, k INT, l INT, m INT, n INT, o INT, p INT, q INT, r INT, s INT);

EXPLAIN SELECT count(*) FROM t1 JOIN t2 ON
  (t1.b = t2.b OR t1.c = t2.c) AND
  (t1.d = t2.d OR t1.e = t2.e) AND
  (t1.f = t2.f OR t1.g = t2.g) AND
  (t1.h = t2.h OR t1.i = t2.i) AND
  (t1.j = t2.j OR t1.k = t2.k) AND
  (t1.l = t2.l OR t1.m = t2.m) AND
  (t1.n = t2.n OR t1.o = t2.o) AND
  (t1.p = t2.p OR t1.q = t2.q) AND
  (t1.r = t2.r OR t1.s = t2.s) AND
  (t1.b = t2.b OR t1.c = t2.c) AND
  (t1.d = t2.d OR t1.e = t2.e) AND
  (t1.f = t2.f OR t1.g = t2.g);

On a node limited to 2GB (cgroup MemoryMax=2G), this OOM-kills the process during planning (Result: oom-kill, signal=KILL). Growth is clearly exponential in the number of OR-conjuncts — measured metadata tables / interned expressions: N=4 -> 34 / 736; N=8 -> 8,194 / 599,146; N=12 -> OOM. Fewer conjuncts (e.g. N=9) already push planning to ~2GB and tens of seconds.

Jira issue: CRDB-67025

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-sql-optimizerSQL logical planning and optimizations.C-bugCode not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.O-agentFiled by an AI agent; usually the result of a human/agent investigation sessionT-sql-queriesSQL Queries Teambranch-masterFailures and bugs on the master branch.

    Type

    No type

    Projects

    Status
    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions