docs(sql): 5.2 SQL engine performance guidance#588
Conversation
Document the new Resource-API SQL engine landing in 5.2 (harper #1285): which queries run index-served/streaming vs. which fall back to the legacy full-scan path, and the auto-fallback contract. Replace the blanket "SQL not recommended" warning with accurate, nuanced guidance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the SQL documentation for Harper 5.2, introducing the new SQL engine built on indexed storage and the Resource API. It replaces the previous warning against SQL use in production with an informative block and adds a comprehensive 'Query Performance' section detailing index-served queries, full-scan fallbacks, and practical optimization guidance. There are no review comments, so no further feedback is provided.
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-588 This preview will update automatically when you push new commits. |
…full scan Table.search executes a top-level OR as a union of per-condition index searches (resources/search.ts), and the new engine pushes an OR group down when every branch is index-driving (whereToConditions conditionUsesIndex). So `a = 1 OR b = 2` with both attributes indexed is index-served; only an OR with an un-indexed / non-index-comparator branch falls back to a scan. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Follow-up commit (14d4523) corrects the |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-588 This preview will update automatically when you push new commits. |
Adds a 'Behavior changes from the legacy engine' subsection covering the standard-SQL corrections the new engine applies to queries it plans (empty-set aggregates -> NULL, string MIN/MAX, NULL!=NULL join keys, explicit-null expression results), and scopes the fallback-contract 'results don't change' promise to genuine fallbacks. Adds TOTAL to the reserved-word list and notes that reserved words used as AS aliases must be quoted (e.g. AS `total`). Sourced from the sql-engine-v2 QA differential wave (findings D-217, F-143). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-588 This preview will update automatically when you push new commits. |
…equirement; OFFSET behavior change QA verification against the engine (sql-engine-v2 @ 8a46e0302) found three inaccuracies: - The fallback log line is 'SQL engine v2 fallback:' (router.ts), not 'sql-engine v2 fallback:' — the documented grep string matched nothing. - ORDER BY / GROUP BY on an indexed attribute run on the legacy path unless an index-driving WHERE condition is present (validateScannable requires a scan driver) — state the requirement instead of promising them unconditionally. - Behavior changes: OFFSET past the end of a result now correctly returns no rows (legacy could still return rows). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-588 This preview will update automatically when you push new commits. |
…ual filters, over-indexing, observability, callout
- ORDER BY bullet: correct the WHERE-driver requirement to note the
primary-key exception (validateScannable/sortDrivesIndex only checks
isPrimaryKey; a no-WHERE ORDER BY on the PK is index-served, only a
secondary indexed attribute falls back) — cb1kenobi
- Un-indexed-filter bullet: clarify that ANDed with an indexed predicate
it's a cheap residual filter and the query stays index-served; only a
standalone or OR-joined un-indexed filter forces a scan — kylebernhardy
- Practical guidance: note the storage/write-cost tradeoff of indexing
every attribute — kylebernhardy
- Intro info box: point to the `SQL engine v2 fallback: ...` log line as
the observability signal for query planning — kylebernhardy
- Query Performance intro: add a callout emphasizing that fallback means
reading every row in the table, not just "slower" — kylebernhardy
Verified against harper sqlEngine/optimizer/{validateScannable,whereToConditions}.ts
and unitTests/sqlEngine/pipeline.test.js (D-219 PK exception tests).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Addressed all 5 open threads in 82e074c:
🤖 Signed, Claude Sonnet 5 |
🚀 Preview DeploymentYour preview deployment is ready! 🔗 Preview URL: https://preview.harper-documentation.harperfabric.com/pr-588 This preview will update automatically when you push new commits. |
3 participants
Adds guidance to the SQL operations-api reference for the new Resource-API SQL engine landing in 5.2 (harper #1285 — New SQL engine on the Resource API), per Kris's request to document which SQL is well supported / runs efficiently and what to avoid.
Changes to
reference/operations-api/sql.md## Query Performancesection:INon indexed attributes, range/BETWEEN, prefixLIKE 'x%', index-orderORDER BYwithLIMIT/OFFSETpushdown,GROUP BYon an indexed attribute,INNER/LEFT JOINon an indexed inner key, projection pushdown.OR, suffix/substringLIKE,!=/NOTnegations, un-drivenORDER BY,SEARCH_JSON; plus a note thatRIGHT/FULL JOIN, no-WHEREUPDATE/DELETE, andCOUNT(DISTINCT)run on the legacy engine.autodefault, results never change (only the performance profile), and thesql-engine v2 fallback:log signal.LIKE, constrain before sort/group/join/SEARCH_JSON, use REST for hot paths.Accuracy notes
LIKE/BETWEEN, sort-from-index, validate-scannable) and the phase-5 cutover parity work in #1285.UNION, sub-SELECT, andINSERT … SELECTare unsupported by both engines, so they are called out as "not supported" rather than "slow fallback."RIGHT/FULL JOINfallback claim against source:normalizer.tsthrowsEngineUnsupportedErrorfor both, and the router catches exactly that underautoto fall back to legacy (which supports them).Note: #1285 is still open and targeted at 5.2; the "As of Harper 5.2" framing lines up with that release. If the merge/version slips, the version reference should move with it.
🤖 Generated with Claude Code