Skip to content

Saved query-backed joins are lost, causing incorrect filtered totals #507

Closed

Description

@mattfaltyn

Problem

A saved query-backed model accepts and persists a declared join, but querying that model drops the relationship during planning. A joined business-segment filter can then succeed with an incorrect total instead of filtering the related rows.

With the existing five-order fixture, customers.tier = 'gold' returns 150 instead of 120, including the silver customer's 30. Declaring the relationship only at query-extension level over an otherwise identical model returns the correct 120. This is reproducible with SQLite and DuckDB.

Reproduction

From the repository root after the documented development setup (poetry install -E all --with dev), run:

SLAYER_USAGE_REPORTING=off poetry run python - <<'PY'
import asyncio
from tests._dev1902_fixtures import _seed_sqlite, renamed_graph
from tests._engine_helpers import seeded_exec_engine
from tests.test_dev1902_join_key_spelling import _customer_totals

async def main():
    async with seeded_exec_engine(
        dialect='sqlite', seed=_seed_sqlite, models=renamed_graph()
    ) as (engine, _):
        model = await engine.save_model(_customer_totals())
        stored = await engine.storage.get_model('customer_totals', data_source='test')
        assert stored is not None and len(stored.joins) == 1
        query = {
            'source_model': 'customer_totals',
            'measures': [{'formula': 'sum(amt)', 'name': 'total'}],
            'filters': ["customers.tier = 'gold'"],
        }
        actual = await engine.execute(query=query)
        print('stored join:', actual.data)
        print(actual.sql)
        assert actual.data == [{'customer_totals.total': 120.0}]

asyncio.run(main())
PY

The fixture has three customers: gold customers 1 and 3 have order totals 30 and 90; silver customer 2 has 30. To reproduce on DuckDB, import _seed_duckdb, select dialect='duckdb' and use that seed.

Expected and actual results

Expected: the saved relationship correlates the filter to each customer, returning 120 (the same result as an equivalent extension-only relationship).

Actual: the stored-join query successfully returns 150 with a semi_join_pushed advisory. Generated SQL checks only whether any gold customer exists:

WHERE EXISTS (
  SELECT 1 FROM customers AS customers
  WHERE customers.tier = 'gold'
)

The predicate does not correlate customer_totals.customer_id with customers.customer_pk. Grouping the joined customer-credit aggregate by customer also returns the global credit total 600 for every customer instead of 100, 200 and 300.

Context and likely boundary

A focused correction should preserve the saved relation's own joins through root and joined/sibling consumption, while keeping ordinary query-stage schemas flat and avoiding inheritance of the backing source's joins. Existing cycle, private-stage-name and extension behavior should remain intact.

No persisted format change or new dependency appears necessary. I intend to follow with a focused fix and SQLite/DuckDB regression tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions