Skip to content

Combine models and stages that have no join path: aggregates across a product hop #496

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ZmeiGorynych merged 14 commits into from
Oct 15, 2026
Merged

Combine models and stages that have no join path: aggregates across a product hop #496

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
2467334
Plan the dev-2082 unrelated-datasets OpenSpec change
ZmeiGorynych Oct 15, 2026
deef3b7
Add failing tests for queries over unrelated datasets
ZmeiGorynych Oct 15, 2026
1f3d851
Merge branch 'origin/main' into egor/dev-2082-combine-models-and-stag…
ZmeiGorynych Oct 15, 2026
9ae4d23
Rebase the tool-surface spec delta onto the requirements main's one-q…
ZmeiGorynych Oct 15, 2026
2fbf05d
Combine datasets no join path connects: product hop, aggregates and o…
ZmeiGorynych Oct 15, 2026
e9f88c0
Existing tests follow unrelated-dataset semantics; drop stale issue r…
ZmeiGorynych Oct 15, 2026
eb3aa44
Division by zero is NULL on every backend
ZmeiGorynych Oct 15, 2026
673b8b9
Stored-stage scenario names its outer measure apart from the stage co…
ZmeiGorynych Oct 15, 2026
9d9511d
Merge branch 'origin/main' into egor/dev-2082-combine-models-and-stag…
ZmeiGorynych Oct 15, 2026
39ea542
Unrelated-dataset tests use AccessFilteredStorage
ZmeiGorynych Oct 15, 2026
e220cbb
Unrelated-dataset warnings and definition discovery cover nested and …
ZmeiGorynych Oct 15, 2026
ff9401d
Stage bases' definitions are scanned, and an unspliceable unrelated q…
ZmeiGorynych Oct 15, 2026
f797f40
Archive OpenSpec change dev-2082-combine-models-and-stages-that-have-…
ZmeiGorynych Oct 15, 2026
b472181
Unrelated-aggregate guidance names the time_spine per-bucket exception
ZmeiGorynych Oct 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
338 changes: 0 additions & 338 deletions .basedpyright/baseline.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28470,344 +28470,6 @@
}
}
],
"./tests/test_recommend_root_model_surfaces.py": [
{
"code": "reportIndexIssue",
"range": {
"startColumn": 15,
"endColumn": 21,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 15,
"endColumn": 21,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 15,
"endColumn": 21,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 15,
"endColumn": 21,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 15,
"endColumn": 21,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 25,
"endColumn": 29,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 39,
"endColumn": 43,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 39,
"endColumn": 43,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 37,
"endColumn": 41,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 19,
"endColumn": 25,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 19,
"endColumn": 25,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 19,
"endColumn": 25,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 19,
"endColumn": 25,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 19,
"endColumn": 25,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 29,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 29,
"endColumn": 35,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 39,
"endColumn": 43,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportIndexIssue",
"range": {
"startColumn": 27,
"endColumn": 33,
"lineCount": 1
}
},
{
"code": "reportAttributeAccessIssue",
"range": {
"startColumn": 37,
"endColumn": 41,
"lineCount": 1
}
}
],
"./tests/test_reference_semantics.py": [
{
"code": "reportArgumentType",
Expand Down
6 changes: 6 additions & 0 deletions architecture/semantics.arc42.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,12 @@ is the coercion from coarser to finer.
together; everything that crosses a non-determining path is defined in terms
of it. [review] Association is derivable from forward join declarations
alone (bidirectional traversal).
- **3.1 Unrelated datasets.** A dataset no join path reaches from a query's
population meets it as the product `P × D`, every row with every row,
reached only by naming `D` as a query path's first hop, never by route
search. Only aggregates and filters (Axiom 14) read across `×`; any other
row-level read is a type error, so `×` never multiplies the population.
[enforced: test:tests/test_unrelated_datasets.py]
[enforced: test:tests/test_dev1853_mirror_parity.py]
[enforced: test:tests/test_dev1935_boolean_lowering.py]
4. **The home-dataset axiom**: an aggregation runs over the rows of its home
Expand Down
4 changes: 2 additions & 2 deletions docs/concepts/formulas.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,10 +264,10 @@ When a measure is renamed via `name`, query filters and ORDER BY entries in the
| `+` | `"sum(revenue) + sum(bonus)"` | `SUM(revenue) + SUM(bonus)` |
| `-` | `"sum(revenue) - sum(cost)"` | `SUM(revenue) - SUM(cost)` |
| `*` | `"avg(price) * sum(quantity)"` | `AVG(price) * SUM(quantity)` |
| `/` | `"sum(revenue) / count(*)"` | `SUM(revenue) / COUNT(*)` |
| `/` | `"sum(revenue) / count(*)"` | `SUM(revenue) / NULLIF(COUNT(*), 0)` |
| `**` | `"sum(value) ** 2"` | `SUM(value) ** 2` |

Parentheses work as expected: `"(sum(revenue) - sum(cost)) / count(*)"`.
Parentheses work as expected: `"(sum(revenue) - sum(cost)) / count(*)"`. Division by zero is NULL on every backend.

All measure names referenced in the formula must exist in the model (except `*` which is always available). For measures from joined models, use a dotted path inside the aggregation: `"avg(customers.score)"` or multi-hop: `"sum(customers.regions.population)"`. Joins are auto-resolved by walking the join graph. See [Cross-Model Measures](queries.md#cross-model-measures).

Expand Down
6 changes: 4 additions & 2 deletions docs/concepts/queries.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ Query results are returned as a `SlayerResponse`:
| `row_count` | int | Number of rows |
| `sql` | string | The generated SQL (useful for debugging) |
| `attributes` | ResponseAttributes | Field metadata split by type: `attributes.dimensions` and `attributes.measures`, each a dict of column alias → FieldMetadata (label, format) |
| `warnings` | list[SlayerWarning] | Advisories, discriminated by `kind`: input normalizations (`"normalization"`), a [cross-model measure broadcast](#cross-model-measures) (`"broadcast"` — `measure`, `location`, and per-dimension `dimensions[].reason`), a distinct-entity attribution over an unattributable dimension (`"associated"` — `measure`, `location`, `dimensions`; cells overlap and are not additive), a semi-join-pushed filter (`"semi_join_pushed"` — `measure` (`null` for a population-level push), `location`, `filter_text`), a query run without its statement timeout (`"statement_timeout_skipped"` — `datasource`, `timeout_seconds`, `reason`: `readonly_user` or `timeout_rejected`), and [`whole_periods_only`](time.md#whole_periods_only) over non-nesting granularities (`"whole_periods_non_nesting"` — `column`, `granularities`, `location`) |
| `warnings` | list[SlayerWarning] | Advisories, discriminated by `kind`: input normalizations (`"normalization"`), a [cross-model measure broadcast](#cross-model-measures) (`"broadcast"` — `measure`, `location`, and per-dimension `dimensions[].reason`), a distinct-entity attribution over an unattributable dimension (`"associated"` — `measure`, `location`, `dimensions`; cells overlap and are not additive), an aggregate over a model no join path reaches (`"unrelated_dataset"` — `dataset`, `population`, `measures`, `location`), a semi-join-pushed filter (`"semi_join_pushed"` — `measure` (`null` for a population-level push), `location`, `filter_text`), a query run without its statement timeout (`"statement_timeout_skipped"` — `datasource`, `timeout_seconds`, `reason`: `readonly_user` or `timeout_rejected`), and [`whole_periods_only`](time.md#whole_periods_only) over non-nesting granularities (`"whole_periods_non_nesting"` — `column`, `granularities`, `location`) |

`columns` — and the key order of each row in `data` — follows the order you
declared fields in the query: dimensions, then time dimensions, then measures,
Expand Down Expand Up @@ -581,7 +581,7 @@ rec.root_model # "orders" (the only model that determines both to-one)
# {"customers.name": "customers.name", "products.category": "products.category"}
```

A root is valid when it **determines** every requested column along provably to-one join paths; saved measures and aggregation-suffixed items are attachments that only need to be reachable and never steer the choice (each `ItemPath` flags `attachment`). Among valid roots, the one with the fewest total join hops wins — the same rule as population inference, so the two surfaces cannot disagree. Root-owned items come back as a bare leaf (`status`); joined items as a dotted path (`customers.regions.name`), each hop spelled by its edge name when it has one (`customers.hr.name` for an edge named `hr`); aggregation suffixes are preserved (`sum(revenue)`).
A root is valid when it **determines** every requested column along provably to-one join paths; saved measures and aggregation-suffixed items are attachments that only need to be reachable and never steer the choice (each `ItemPath` flags `attachment`). Among valid roots, the one with the fewest total join hops wins — the same rule as population inference, so the two surfaces cannot disagree. Root-owned items come back as a bare leaf (`status`); joined items as a dotted path (`customers.regions.name`), each hop spelled by its edge name when it has one (`customers.hr.name` for an edge named `hr`); aggregation suffixes are preserved (`sum(revenue)`). With at least one column item, an attachment on a model no join path reaches never blocks a root: it comes back spelled from the root (`count(events.id)`) and flagged `unrelated`.

When no single model reaches everything, `root_model` is `None`, `reachable` is `False`, and `coverage` lists the best partial roots (each with its reachable / unreachable items) so you can split the request into a multi-stage [`source_queries`](models.md#query-backed-models) query.

Expand Down Expand Up @@ -779,6 +779,8 @@ Example: `{"source_model": "orders", "dimensions": ["status"], "measures": [{"fo

Every input of an aggregate — its source, arguments (`weight=`), definition defaults, and its column-level `filter=` — is traced recursively through derived-column definitions, and an input whose expansion crosses a fanning (not provably to-one) hop fails closed with a typed error instead of silently multiplying rows.

A model (or query stage) that no join path reaches is read only through aggregates: rooted at `customers`, `{"formula": "count(events.id) / count(*)"}` computes `count(events.id)` over every event, repeats it on every row (per bucket of a `time_spine` time dimension) with a `kind: "unrelated_dataset"` warning, and a filter on it restricts its rows, while grouping, ordering or filtering by its columns alongside other models' is a typed error.

A filter **on** the cross-model value itself (`"avg(customers.score) > 4"`)
restricts the result rows, uniformly with local aggregate filters — groups that
fail the predicate are dropped, not returned with `NULL`.
Expand Down
Loading
Loading