diff --git a/cli/compile.mdx b/cli/compile.mdx index 483164e..ddee64f 100644 --- a/cli/compile.mdx +++ b/cli/compile.mdx @@ -34,7 +34,8 @@ sqb --project-dir compile [flags] 3. **SQL validation** - validates SQL syntax (when SQL analysis is enabled) 4. **Column lineage** - analyzes column-level dependencies across models (fast mode by default) 5. **Contract validation** - checks declared column contracts against inferred query output -6. **Artifact write** - writes compiled SQL to `target/compiled/` +6. **Rules** - evaluates selected native built-ins, then selected custom Python Rules +7. **Artifact write** - writes compiled SQL to `target/compiled/` when Rules pass ## Focused compilation @@ -65,9 +66,9 @@ When SQL analysis is enabled (default), compile performs static analysis on your - **Column lineage**: Traces which source columns flow into each output column, including transform classification. See [Column Lineage](/concepts/column-lineage) for details -`sqb compile` checks SQL correctness, contracts, and lineage. [`sqb kata`](/cli/kata) compiles the -project and then applies its separately configured architecture policy. Kata is not run -automatically by `compile`. +`sqb compile` is authoritative for mandatory correctness, contracts, lineage, and configured Rules. +Rules findings block artifact completion. Use [`sqb rules run`](/cli/rules) to focus on one exact +code or family without changing project configuration. ### Contract diagnostics diff --git a/cli/fix.mdx b/cli/fix.mdx deleted file mode 100644 index d6a9ecb..0000000 --- a/cli/fix.mdx +++ /dev/null @@ -1,30 +0,0 @@ ---- -title: "fix" -description: "Apply proven-safe native SQL lint fixes." ---- - -# sqb fix - -Applies only lint fixes SQLBuild can prove equivalent. Findings that require author intent remain -unchanged and reported. - -```bash -sqb fix [flags] -``` - -| Flag | Description | -|------|-------------| -| `--check` | Report whether fixes are needed without writing | -| `--diff` | Print the proposed diff without writing | -| `--json` | Print structured results | -| `--select`, `-s` | Fix selected models | -| `--select-file` | Read selectors from a file | -| `--exclude` | Exclude models from the selected scope | - -```bash -sqb fix --diff -sqb fix --check -sqb fix --select path:models/marts -``` - -Use [`sqb format`](/cli/format) for canonical presentation changes. diff --git a/cli/format.mdx b/cli/format.mdx index f468eb5..aca9e0f 100644 --- a/cli/format.mdx +++ b/cli/format.mdx @@ -26,5 +26,5 @@ sqb format --check sqb format --diff --select tag:marts ``` -Formatting owns presentation. Use [`sqb lint`](/cli/lint) for diagnostics and [`sqb fix`](/cli/fix) -for safe semantic cleanups. +Formatting owns presentation and source rewriting. Compiler-integrated diagnostics belong to +configured [Rules](/concepts/rules), which never mutate source. diff --git a/cli/kata.mdx b/cli/kata.mdx deleted file mode 100644 index 81bdbc3..0000000 --- a/cli/kata.mdx +++ /dev/null @@ -1,177 +0,0 @@ ---- -title: "kata" -description: "Run configured SQL architecture checks, inspect rules, and generate policy guidance." ---- - -# sqb kata - -Compiles the project and applies its configured [Kata architecture policy](/concepts/kata) -to compiled models. Built-in checks run offline and never connect to the warehouse or rewrite SQL. -Kata reports coded, error-only faults with source locations and remediations. Repository-defined -custom rules run in a bounded Python subprocess. - -## Usage - -```bash -sqb --project-dir kata [flags] -sqb kata rule -sqb kata skills [--check] -``` - -## Evaluation flags - -| Flag | Description | -|------|-------------| -| `--json` | Emit structured JSON instead of text | -| `--select`, `-s` | Evaluate selected models using normal SQLBuild selector syntax | -| `--exclude` | Exclude models from a non-empty `--select` scope | - -Rule policy comes from `[kata].select` in `sqlbuild_project.toml`; CLI `--select` and `--exclude` -scope models within that policy. Model selectors support names, `tag:`, `path:`, graph `+`, and -path-between syntax. - - -`--exclude` is applied only when `--select` is also provided. To evaluate all models except one, -start with an explicit broad selector such as `--select path:models`. - - -## Text output - -A clean policy prints its model and cache counts: - -```text -Kata passed: 42 models evaluated, 0 faults (40 cache hits, 2 misses) -``` - -Faults include a source location, rule code, message, and remediation. Model-level checks use -line 1, column 1: - -```text -models/mart/orders.sql:1:1 [SQBKS001] model SQL must keep transformation logic in top-level CTEs - Remediation: Move transformation logic into named top-level CTEs before the terminal SELECT. -Found 1 kata faults -``` - -Project-phase SQL test policy also reports the authored test or scenario path. A finding can name -the compiler-resolved target and exact destination: - -```text -tests/unit/test_stg_orders.sql:1:1 [SQBKT003] unit test block 1 resolves to resources mirrored by tests/unit/staging/ - Remediation: Move this test file beneath tests/unit/staging/. -Found 1 kata faults -``` - -`SQBKT` rules run once per project, including projects with direct-resource tests but no models. -Path-scoped exceptions and ignores match the reported test or scenario path. - -## JSON output - -```bash -sqb kata --json -``` - -```json -{ - "cache_hits": 0, - "cache_misses": 1, - "evaluated_models": 1, - "fault_count": 1, - "faults": [ - { - "code": "SQBKS001", - "column": 1, - "line": 1, - "message": "model SQL must keep transformation logic in top-level CTEs", - "path": "models/mart/orders.sql", - "remediation": "Move transformation logic into named top-level CTEs before the terminal SELECT." - } - ] -} -``` - -Faults are ordered deterministically by path, position, code, and content. - -## Inspect a rule - -`rule` prints metadata for any exact built-in or configured custom code. The rule does not need to -be active: - -```bash -sqb kata rule SQBKS101 -``` - -```text -SQBKS101: dependency-import-ctes -Family: structure -Enabled by default: no -Kind: built-in - -dependencies must be isolated in import CTEs - -Remediation: Move each __ref(...) or __source(...) into one named top-level import CTE and reference that CTE from later logic. -``` - -Custom rules also show their source and declared option defaults. - -Inspecting an `SQBKT` rule also prints the effective canonical roots and configured cross-domain -pipeline directory: - -```bash -sqb kata rule SQBKT003 -``` - -## Generate policy skills - -Generate agent guidance from the active rules, options, thresholds, naming vocabulary, SQL test -paths, and scoped deviations: - -```bash -sqb kata skills -``` - -Kata writes the same policy-specific guidance to: - -- `.agents/skills/sqlbuild-kata/SKILL.md` -- `.claude/skills/sqlbuild-kata/SKILL.md` -- `.opencode/skills/sqlbuild-kata/SKILL.md` - -Check committed guidance in CI without rewriting it: - -```bash -sqb kata skills --check -``` - -Install mode refuses to overwrite divergent, malformed, or unowned files. See -[SQLBuild skills](/cli/skills) for the separate general framework guidance command. - -## Exit codes - -| Command | Code | Meaning | -|---------|------|---------| -| `sqb kata` | `0` | No retained faults | -| `sqb kata` | `1` | Faults found or Kata could not evaluate the project | -| `sqb kata rule` | `0` | Exact rule found | -| `sqb kata rule` | `2` | Unknown rule code | -| `sqb kata skills` | `0` | Guidance installed | -| `sqb kata skills --check` | `0` | All guidance is fresh | -| `sqb kata skills --check` | `1` | Guidance is not fresh: missing, stale, divergent, malformed, or unowned | - -## Examples - -```bash -# Evaluate the configured policy -sqb kata - -# Emit machine-readable CI output -sqb kata --json - -# Scope evaluation to marts and their downstream models -sqb kata --select tag:marts+ - -# Inspect an opt-in rule before adopting it -sqb kata rule SQBKJ002 - -# Keep policy-derived agent guidance current -sqb kata skills -sqb kata skills --check -``` diff --git a/cli/lint.mdx b/cli/lint.mdx deleted file mode 100644 index e621c08..0000000 --- a/cli/lint.mdx +++ /dev/null @@ -1,28 +0,0 @@ ---- -title: "lint" -description: "Report native SQL lint diagnostics." ---- - -# sqb lint - -Runs SQLBuild's native, dialect-aware lint rules without changing files. - -```bash -sqb lint [flags] -``` - -| Flag | Description | -|------|-------------| -| `--json` | Print structured diagnostics | -| `--select`, `-s` | Lint selected models | -| `--select-file` | Read selectors from a file | -| `--exclude` | Exclude models from the selected scope | - -```bash -sqb lint -sqb lint --select tag:marts -sqb lint --json -``` - -Lint exits nonzero when active diagnostics remain. See [SQL Linting and Formatting](/concepts/sql-linting) -for rule selection, suppressions, and the boundary between lint, fix, format, and Kata. diff --git a/cli/rules.mdx b/cli/rules.mdx new file mode 100644 index 0000000..e44bb49 --- /dev/null +++ b/cli/rules.mdx @@ -0,0 +1,49 @@ +--- +title: "rules" +description: "List, inspect, or run compiler-integrated Rules and generate project guidance." +--- + +# sqb rules + +Configured Rules run automatically through `sqb compile` and build planning. The `rules` command +provides focused catalogue, inspection, execution, and guidance operations. + +## Usage + +```bash +sqb rules list +sqb rules show +sqb rules run [--select ] [--exclude ] +sqb rules skills [--check] +``` + +Add `--json` after `rules` for machine-readable list or run output: + +```bash +sqb rules --json list +sqb rules --json run SQBRSQL +``` + +Lifecycle progress is written to stderr so stdout remains valid JSON. + +## Commands + +- `list`: print built-in and repository-defined Rules. +- `show`: print metadata for one exact code. +- `run`: compile and evaluate only one exact code or family prefix. +- `skills`: install guidance derived from configured Rules; `--check` verifies freshness without writing. + +`run` accepts normal SQLBuild model selectors. Native built-ins execute before custom Python Rules. + +## Exit codes + +| Command | Code | Meaning | +|---|---:|---| +| `rules list` | 0 | Catalogue printed | +| `rules show` | 0 | Exact code found | +| `rules show` | 2 | Unknown code | +| `rules run` | 0 | No findings | +| `rules run` | 1 | Findings reported or compilation failed | +| `rules run` | 2 | Unknown code or family | +| `rules skills --check` | 0 | Guidance is fresh | +| `rules skills --check` | 1 | Guidance is missing or stale | diff --git a/cli/skills.mdx b/cli/skills.mdx index b2df668..0a8ac3d 100644 --- a/cli/skills.mdx +++ b/cli/skills.mdx @@ -86,17 +86,17 @@ cd waffle-shop # Agent skill files are already installed ``` -## Kata policy skills +## Project Rules skills -`sqb skills` installs general SQLBuild framework guidance. `sqb kata skills` generates -project-specific guidance from the active Kata rules, options, thresholds, naming vocabulary, and -scoped deviations: +`sqb skills` installs general SQLBuild framework guidance. `sqb rules skills` generates +project-specific guidance from the active Rules, options, thresholds, naming vocabulary, and scoped +deviations: ```bash -sqb kata skills -sqb kata skills --check +sqb rules skills +sqb rules skills --check ``` -Use `--check` in CI to detect missing or stale policy guidance without rewriting files. Kata uses -the separate `sqlbuild-kata` skill path and refuses to overwrite divergent or unowned content. See -[Kata SQL Architecture Checks](/concepts/kata) and the [Kata CLI reference](/cli/kata). +Use `--check` in CI to detect missing or stale Rules guidance without rewriting files. Rules use +the separate `sqlbuild-rules` skill path and refuse to overwrite divergent or unowned content. See +[Compiler-integrated Rules](/concepts/rules) and the [Rules CLI reference](/cli/rules). diff --git a/concepts/incremental.mdx b/concepts/incremental.mdx index b19fbc3..ba35c1c 100644 --- a/concepts/incremental.mdx +++ b/concepts/incremental.mdx @@ -305,7 +305,7 @@ max_batches = 100 action = "error" # or "warn" ``` -Project limits support only `error` and `warn`; they never silently cap work. When a model has a nested `microbatch_limit`, the project policy checks the full resolved range first and the model policy then applies. +Project limits support only `error` and `warn`; they never silently cap work. When a model has a nested `microbatch_limit`, the project limit checks the full resolved range first and the model policy then applies. `--max-microbatches N` is an invocation-wide, hard `error` ceiling and an explicit one-run authorization. It takes precedence over project and model limits, applies to models without a declared limit, and never inherits `cap_from_start` or `cap_from_end`. For example, passing a value large enough for an intentional backfill authorizes the full range instead of retaining the model's ordinary-run cap. diff --git a/concepts/kata.mdx b/concepts/kata.mdx deleted file mode 100644 index 8f39ac0..0000000 --- a/concepts/kata.mdx +++ /dev/null @@ -1,372 +0,0 @@ ---- -title: "Kata SQL Architecture Checks" -description: "Enforce opt-in SQL architecture and model-shape policy over your compiled project." ---- - -Kata is SQLBuild's opt-in SQL architecture policy. It compiles the project, then checks model -structure, naming, dependency boundaries, joins, contracts, and test coverage. Built-in checks run -offline: they do not execute warehouse SQL or rewrite source files. Findings have stable codes and -concrete remediations. - -Kata is error-only: every retained finding blocks the command. Use it for conventions that a team -has deliberately adopted, not as a collection of advisory style warnings. - -Tests codify behavioral expectations; Kata codifies architectural expectations for SQL models. -For equivalent boundaries, repository structure, and code-shape checks in Python projects, see -[Fensu](https://docs.fensu.dev/). - -## Where Kata fits - -| Command | Responsibility | -|---------|----------------| -| `sqb compile` | SQL validity, references, inferred columns, contracts, and lineage | -| `sqb lint` / `sqb format` | SQL presentation and formatting | -| `sqb kata` | Repository architecture and model-shape conventions | -| `sqb test` | Transformation behavior | -| `sqb audit` | Data quality against materialized data | - -Kata is a separate command. It is not run automatically by `compile` or `build`. - -## Enable Kata - -Commit the shared policy to `sqlbuild_project.toml`: - -```toml -[kata] -select = ["SQBK"] -``` - -This activates the complete standard policy. Start here, then use `ignore` to switch off conventions -the repository is not ready to enforce. - - -Kata evaluates no rules when `[kata].select` is empty. Prefixes select matching rules that are -enabled by default; exact codes also select individually opt-in rules. All current built-ins are -enabled by default, so `SQBK` selects the complete built-in catalogue. - - -Rule selectors are case-sensitive prefixes. They do not use `*` wildcards: - -- Built-in rules use `SQBK`, such as `SQBKS101`. -- Custom rules use `XSQBK`, such as `XSQBKP001`. -- `select` activates rules; `ignore` removes matching rules from the active policy. -- An exact code activates that rule even when it is opt-in. -- The CLI `--select` and `--exclude` flags scope models, not rules. - -Inspect any built-in or configured custom rule without enabling it: - -```bash -sqb kata rule SQBKS101 -``` - -## Built-in rules - -All current built-ins form the standard policy and are enabled by matching prefixes. - -### Structure - -| Code | Check | -|------|-------| -| `SQBKS000` | Standalone comments belong on the first inner line of a CTE | -| `SQBKS001` | Transformation logic belongs in top-level CTEs | -| `SQBKS002` | The terminal SELECT reads plainly from the final top-level CTE | -| `SQBKS101` | Each `__ref` and `__source` is isolated in one dependency import CTE | -| `SQBKS201` | `SELECT *` is restricted to dependency import CTEs | -| `SQBKS202` | Positional set-operation branches enumerate their columns | -| `SQBKS301` | CTEs are top-level, not nested | -| `SQBKS302` | Recursive CTEs are not permitted | -| `SQBKS401` | View materialization agrees with the `stg_v`, `int_v`, or `mart_v` marker | -| `SQBKS501` | CTE names describe their contents | - -### Layers and model grammar - -| Code | Check | -|------|-------| -| `SQBKL001` | Dependencies flow forward through the layer order | -| `SQBKL101` | Qualified table dependencies use `__ref` or `__source` | -| `SQBKR001` | Model names follow `____[__]` | -| `SQBKR002` | Model layer names agree with their folders | -| `SQBKR201` | Model source suffixes and source dependency names use approved, current tokens | -| `SQBKR301` | Referenced model identifiers follow Kata model-name grammar | -| `SQBKR401` | Models declare `contract enforced` | -| `SQBKR500` | Every model resolves to one configured domain root and level | -| `SQBKR501` | Every model owner is a leaf or a branch, never both | -| `SQBKR502` | Ownership paths stay within `max_subdomain_depth` | -| `SQBKR503` | Compressed underscore-token prefixes do not hide implicit owners | - -### Owner layout - -Kata treats warehouse levels as an axis beneath a genuine domain root. Levels are configurable and -may contain more than one path component: - -```toml -[kata.layout] -levels = ["staging", "intermediate/clean", "intermediate/enriched", "mart"] -domain_roots = ["commerce/orders", "support/tickets"] -``` - -`domain_roots` is optional. Without it, Kata infers the complete domain root as every path component -between `models/` and the configured level. If more than one configured level can interpret a path, -`SQBKR500` reports every candidate instead of guessing. Add explicit roots for those ambiguous -trees. Level paths and explicit domain roots must be normalized, unique, and non-overlapping. - -The standard owner shapes are: - -```text -models///.sql -models////.sql -``` - -At every ownership node, direct models make the node a leaf and child owners make it a branch. A -directory may not contain both. Different branches may terminate at different depths; Kata does not -require empty ceremonial folders merely to make paths the same length. - -`max_subdomain_depth` defaults to one and counts only owners after the configured level. Domain-root -components, composite-level components, declaration roles, and declaration-role buckets do not -count. Projects can raise the non-negative threshold explicitly: - -```toml -[kata.thresholds] -max_subdomain_depth = 2 -``` - -Kata detects ownership hidden in flattened underscore names with a compressed token trie. Unary -token chains remain compound terms, so `order_status/` and `order_status_history/` identify -`order_status` rather than `order`. Real branch points remain explicit: partner annotation -export, annotation validation, and events identify an outer `partner` owner and an inner -`annotation` concern. Detection starts with two siblings by default: - -```toml -[kata.thresholds] -min_shared_owner_prefix_directories = 2 -``` - -Set this threshold to zero to disable the prefix-family check. - -### Joins - -| Code | Check | -|------|-------| -| `SQBKJ001` | Implicit comma joins are not permitted | -| `SQBKJ002` | Cross joins require an exact, reasoned exception | -| `SQBKJ101` | Non-cross joins declare `ON` or `USING` keys | - -### Column naming and types - -| Code | Check | -|------|-------| -| `SQBKN001` | `is_`, `has_`, and `can_` columns are BOOLEAN | -| `SQBKN002` | `*_at`, `*_ts`, and `*_timestamp` columns use timestamp types | -| `SQBKN003` | `*_date` columns are DATE | - -These checks use declared contract columns, not inferred output columns. - -### Decision hygiene - -| Code | Check | -|------|-------| -| `SQBKH001` | Enum comparisons use declared members and normalized operands | -| `SQBKH002` | Non-canonical numeric decisions use named constants | -| `SQBKH101` | Identical enum domains are consolidated | -| `SQBKH201` | Public enum and constant files live under domain folders | -| `SQBKH301` | Declaration role containers are flat or fully grouped | -| `SQBKH302` | Declaration role buckets stay within their configured depth | -| `SQBKH303` | Flat roles and individual buckets stay within file-count caps | -| `SQBKH304` | Buckets use specific concern names rather than generic role names | -| `SQBKH305` | Shared filename prefixes become navigation buckets | - -`SQBKH001` requires direct comparisons to `@enum("").`. Normalize controlled values -upstream rather than wrapping either comparison operand. A direct source-side value may be -normalized in the comparison because the project does not control source casing; the enum member -must still remain unwrapped. - -The declaration-container rules apply to public and private `macros`, `constants`, and `enums` -roles. A container is either flat or every file is grouped into one level of concern buckets: - -```text -_macros/ _macros/ -├── normalise_name.py ├── normalisation/ -└── resolve_match.py │ └── names.py - └── resolution/ - └── matches.py -``` - -Buckets are navigation only. They never change the declaration owner or compiler visibility. -Generic buckets such as `utils`, `common`, `shared`, and `misc` fault. Defaults are: - -```toml -[kata.thresholds] -max_role_container_depth = 1 -max_macro_container_files = 10 -max_constant_container_files = 10 -max_enum_container_files = 10 -min_shared_container_prefix_files = 2 -``` - -### Tests and coverage - -| Code | Check | -|------|-------| -| `SQBKX001` | Non-passthrough models meet the configured audit minimum | -| `SQBKX002` | Non-passthrough models meet the configured SQL test minimum | -| `SQBKX201` | Selected custom rules have statically discoverable public-harness test cases | - -Selecting a custom rule automatically adds `SQBKX201` unless the policy ignores it. This is a -static check for conventional `RuleCase` and `evaluate_rule` usage; it does not execute the tests. -Thresholds default to one and can be set to zero to disable the corresponding minimum: - -```toml -[kata.thresholds] -min_audits_per_model = 1 -min_tests_per_model = 1 -min_custom_rule_test_cases = 1 -``` - -### SQL tests and scenarios - -The `SQBKT` family governs SQL authored under `tests/unit/` and `tests/scenarios/`. It consumes -compiler-resolved test targets and resource ownership; it does not infer ownership from filenames -or repeat compiler diagnostics for malformed tests. - -| Code | Check | -|------|-------| -| `SQBKT001` | Unit tests and scenarios use their compiler-owned canonical roots | -| `SQBKT002` | Unit and scenario filenames identify their subject and behavior | -| `SQBKT003` | Unit tests mirror resolved model, macro, UDF, or table-function ownership | -| `SQBKT004` | Every `TEST` block has an explicit target-aware `subject__expected_behavior` name | -| `SQBKT101` | Scenario descriptions identify a concrete business behavior rather than generic case numbering | - -Select the family independently when adopting these conventions: - -```toml -[kata] -select = ["SQBKT"] -``` - -Every unit-test block, including the only block in a file, has an explicit name: - -```sql -TEST ( - name "stg_orders__excludes_cancelled_orders", -); -``` - -The double underscore immediately following the resolved subject separates it from nonempty -behavior. This keeps identities such as `commerce__mart_v_order` valid as complete subjects. -Single-model subjects match the resolved expected model, direct-mode subjects match the tested -macro, UDF, or table function, and multi-model subjects name the common domain or an explicit -pipeline. Single underscores separate words within either part. Generic values such as `test`, -`works`, `basic`, and `case_1` are rejected without maintaining a verb allowlist. - -Unit filenames use either `test_.sql` or `test___.sql`. When a behavior -suffix is present, it corresponds to the behavior portion; concise prefixes such as -`excludes_cancelled` for `excludes_cancelled_orders` are valid. Scenario filenames omit the -redundant prefix and use `__.sql`: - -```text -tests/unit/staging/test_stg_orders__excludes_cancelled.sql -tests/scenarios/daily_revenue__multiple_orders.sql -``` - -Mirroring uses compiled relationships: - -- A single-model test mirrors the model parent below its compiler-owned model root. -- A multi-model test mirrors the nearest common model-domain parent. -- Models with no meaningful common parent use the configured pipeline directory. -- Macro, UDF, and table-function tests mirror all resolved direct resource owners. -- When ownership cannot be proven from compiler facts, Kata skips mirroring rather than guessing. - -The pipeline directory is relative to `tests/unit/`, normalized, and included in cache and generated -guidance identity. The default is `pipelines`: - -```toml -[kata.sql_tests] -pipeline_directory = "chains/commerce" -``` - -This maps cross-domain tests to `tests/unit/chains/commerce/`. Absolute paths, traversal, repeated -separators, and backslash paths are invalid configuration. - -## Naming policy - -Naming and layer rules can use a closed project vocabulary: - -```toml -[kata] -domains = ["commerce", "support"] -approved_source_tokens = ["web", "partner"] -cte_name_whitelist = ["finalized_rows"] -cte_name_denylist = ["scratch_result"] - -[kata.retired_source_tokens] -legacy_feed = "partner" -``` - -Valid Kata layers are `stg`, `stg_v`, `int_clean`, `int_v`, `int_enriched`, `mart`, and -`mart_v`. Configuration supplies vocabulary to active rules; it does not activate them. When -`SQBKR001` or `SQBKH201` is active, a non-empty `domains` list constrains model or declaration -domains respectively. - -## Exceptions and scoped ignores - -Choose the narrowest mechanism that represents the policy: - -| Mechanism | Scope | Reason required | Stale-checked | -|-----------|-------|-----------------|---------------| -| `ignore` | Disable rules globally | No | No | -| `rule_exceptions` | One exact rule and exact file | Yes | Yes | -| `rule_ignores` | Rule prefixes or codes across path globs | Yes | No | -| `select_star_allow` | Path-glob allowance for `SQBKS201` | Yes | No | - -```toml -[[kata.rule_exceptions]] -rule = "SQBKJ002" -path = "models/mart/commerce__mart__matrix.sql" -reason = "Intentional Cartesian product over a bounded dimension" - -[[kata.rule_ignores]] -rules = ["SQBKS"] -paths = ["models/legacy/**"] -reason = "Legacy migration boundary" - -[[kata.select_star_allow]] -paths = ["models/mart/*_export.sql"] -reason = "Intentional passthrough export" -``` - -An exact exception fails when its active rule no longer produces a fault at that file, prompting -the repository to remove obsolete exceptions. Broad migration boundaries and lone-star allowances -remain reasoned but are intentionally not stale-checked. - -## Cache and CI - -Built-in policies use a persistent cache under `target/kata-cache`. Compiled model content, active -rules, options, thresholds, naming vocabulary, and relevant project files participate in cache -identity. Disable it when diagnosing cache behavior: - -```toml -[kata.cache] -enabled = false -``` - -Run Kata directly in CI. It exits `1` when faults remain: - -```bash -sqb kata -sqb kata --json -``` - -Generate agent guidance from the same resolved policy and verify that committed guidance remains -fresh: - -```bash -sqb kata skills -sqb kata skills --check -``` - -Kata manages `.agents/skills/sqlbuild-kata/SKILL.md`, -`.claude/skills/sqlbuild-kata/SKILL.md`, and `.opencode/skills/sqlbuild-kata/SKILL.md`. It refuses -to overwrite divergent or unowned files. - -See [Custom Kata Rules](/concepts/kata/custom-rules) to encode repository-specific policy and the -[Kata CLI reference](/cli/kata) for command output and exit behavior. diff --git a/concepts/kata/custom-rules.mdx b/concepts/kata/custom-rules.mdx deleted file mode 100644 index 780c45c..0000000 --- a/concepts/kata/custom-rules.mdx +++ /dev/null @@ -1,154 +0,0 @@ ---- -title: "Custom Kata Rules" -description: "Define and test repository-owned SQL architecture rules with the public Kata API." ---- - -Custom Kata rules extend the built-in policy when a repository has domain conventions that cannot -be expressed by configuration alone. They use the same selection, suppression, deterministic -ordering, and remediation output as built-ins. - -Custom rule codes use `XSQBK`. Keep codes stable after adoption because they -become part of configuration, CI output, and exceptions. - -## Define a rule - -```python -from sqlbuild.kata import KataFault, RuleContext, kata - - -@kata( - code="XSQBKO001", - family="orders", - slug="customer-identity", - message="order models must declare a customer_id column", - remediation="Declare customer_id in the MODEL columns contract.", -) -def customer_identity(*, model, ctx: RuleContext) -> list[KataFault]: - if any(column.name == "customer_id" for column in ctx.declared_columns): - return [] - return [ctx.path_fault()] -``` - -The function signature is exactly two keyword-only arguments named `model` and `ctx`. Return an -empty list when the model passes or one or more `KataFault` values when it fails. - -`RuleContext` exposes the compiled model, authored SQL, raw Polyglot AST, references, parsed model -name, materialization, declared columns, audit and test counts, public declarations, active policy, -and fault constructors. Repository files can be read safely through `project_read_text` and -`project_glob`. - -## Load and select rules - -Load repository-owned files or dotted modules from `sqlbuild_project.toml`: - -```toml -[kata] -select = ["XSQBKO001"] -rule_paths = ["kata/rules"] -rule_modules = ["project_kata.rules"] -``` - -A directory in `rule_paths` is scanned recursively for Python files containing `@kata`. Dotted -modules must resolve beneath the project root. Codes must be unique across built-in and custom -rules. - -Custom rules require exact selectors by default. Set `enabled_by_default=True` on the decorator to -include a rule in matching prefix selections. This does not activate Kata when -`[kata].select` is empty. - -## Typed options - -Declare options with `RuleOption.boolean`, `integer`, `string`, `string_list`, or `integer_list`: - -```python -from sqlbuild.kata import KataFault, RuleContext, RuleOption, kata - - -REQUIRED_DOMAIN = RuleOption.string( - name="required_domain", - default="commerce", - description="Domain that owns order models", -) - - -@kata( - code="XSQBKO002", - family="orders", - slug="required-domain", - message="order models must belong to the configured domain", - remediation="Move or rename this model for the configured domain.", - options=(REQUIRED_DOMAIN,), -) -def required_domain(*, model, ctx: RuleContext) -> list[KataFault]: - parts = ctx.name_parts - if parts is not None and parts.domain == ctx.option(REQUIRED_DOMAIN): - return [] - return [ctx.path_fault()] -``` - -Configure options under the exact rule code. Unknown rules, option names, or invalid values fail -configuration: - -```toml -[kata.rule_options.XSQBKO002] -required_domain = "commerce" -``` - -## Test every rule - -Use the public harness so tests exercise normal SQLBuild discovery, compilation, rule loading, and -structured fault evaluation: - -```python -from sqlbuild.kata import RuleCase, evaluate_rule - -from kata.rules.orders import customer_identity - - -def test_missing_customer_identity_faults() -> None: - result = evaluate_rule( - rule=customer_identity, - test_case=RuleCase( - description="missing customer identity faults", - source=( - "MODEL (materialized table);\n\n" - "WITH final AS (SELECT 1 AS order_id)\n" - "SELECT order_id FROM final\n" - ), - path="models/mart/commerce__mart__orders.sql", - expected_fault_count=1, - ), - ) - - assert result.fault_count == 1 -``` - -`RuleCase.files` can add supporting project files and `RuleCase.config` supplies the rule's option -values. Keep conventional `RuleCase` and `evaluate_rule` calls under `tests/` so `SQBKX201` can -count statically discoverable harness cases. This coverage check does not execute the tests, so run -the test suite separately in CI. - -## Execution and caching - -Selected custom rules execute in a bounded Python subprocess with a 30-second timeout. Exceptions -are reported with the rule code and model path, and returned faults rejoin normal suppressions and -deterministic ordering. - -Selecting any custom rule disables the model cache by default. To keep the built-in cache available, -require hermetic custom rules explicitly: - -```toml -[kata.cache] -enabled = true -require_cacheable = true -``` - -Cacheable rules may import supported pure modules such as `collections`, `dataclasses`, `enum`, -`math`, `re`, `typing`, and `sqlbuild.kata`. Use `RuleContext` rather than direct filesystem calls. -SQLBuild validates these constraints before evaluation. - -Custom findings are still recomputed on each invocation. `require_cacheable` preserves the native -model cache around them; it does not cache custom subprocess output. - -Return to [Kata SQL Architecture Checks](/concepts/kata) for built-in rules, selectors, and -exceptions. diff --git a/concepts/models/contracts.mdx b/concepts/models/contracts.mdx index 0470e22..b6c1a77 100644 --- a/concepts/models/contracts.mdx +++ b/concepts/models/contracts.mdx @@ -154,6 +154,6 @@ non-projection uses such as `join_on`, `where`, `group_by`, and `window_order_by ## Related policies -The project default is implicit open-shape validation for models that omit `contract`. A repository can select explicit opt-in contracts with `settings.column_contract_mode = "explicit"` or enable [`SQBKR401`](/concepts/kata#layers-and-model-grammar) to require enforced contracts through Kata architecture policy. +The project default is implicit open-shape validation for models that omit `contract`. A repository can select explicit opt-in contracts with `settings.column_contract_mode = "explicit"` or configure a custom [Rule](/concepts/rules/custom-rules) that requires enforced contracts for selected model families. Contracts also constrain schema-change behavior. For example, `snapshot_schema_change append_new_columns` is incompatible with `contract enforced` because an unannounced appended column would violate the exact declaration. diff --git a/concepts/project-configuration.mdx b/concepts/project-configuration.mdx index 756a998..849fdf1 100644 --- a/concepts/project-configuration.mdx +++ b/concepts/project-configuration.mdx @@ -310,21 +310,19 @@ default_audit_run_scope = "final" - **`staged`** (default for most adapters): Materializes into a staging table, runs audits, then swaps into the target. If audits fail, the production table is untouched. - **`immediate`**: Creates the table directly at the target location. Audits run after materialization. Simpler but no pre-promotion safety net. -## Kata +## Rules -Kata policy belongs in the shared `sqlbuild_project.toml` so local and CI evaluation use the same -architecture rules: +Rules configuration belongs in the shared `sqlbuild_project.toml` so local and CI compilation use +the same checks: ```toml -[kata] -select = ["SQBK"] +[rules] +select = ["SQBRSQL", "SQBRGRAPH", "XSQBRARCH"] ``` -Kata is opt-in. `SQBK` activates every built-in rule; narrower prefixes activate a rule family, and -exact codes activate individual rules. Audit, SQL test, and custom-rule test-case minimums each -default to one and can be overridden under `[kata.thresholds]`. See -[Kata SQL Architecture Checks](/concepts/kata) for the complete rule, configuration, and suppression -reference. +Rules are opt-in. Exact codes activate individual checks and prefixes activate a family. Built-in +codes begin with `SQBR`; repository-defined codes begin with `XSQBR`. See +[Compiler-integrated Rules](/concepts/rules) for configuration, authoring, and suppressions. ## Project variables diff --git a/concepts/rules.mdx b/concepts/rules.mdx new file mode 100644 index 0000000..2552d6e --- /dev/null +++ b/concepts/rules.mdx @@ -0,0 +1,64 @@ +--- +title: "Rules" +description: "Enforce repeatable SQL and project requirements during compilation." +--- + +Rules are configurable compile-time requirements evaluated against SQLBuild's compiled project. +They turn decisions that would otherwise be repeated in code review into deterministic compiler +findings. + +SQLBuild evaluates checks in this order: + +1. mandatory compiler correctness +2. selected native built-in Rules +3. selected custom Python Rules +4. artifact completion + +Rules report findings and never rewrite source. [`sqb format`](/cli/format) remains the separate +source-rewriting command. + +## Built-in and custom Rules + +Native built-in Rules use `SQBR...` codes and cover reusable requirements maintained by SQLBuild. +Projects can define custom Python Rules with project-owned `XSQBR...` codes. Both use the same +selection, findings, exceptions, ordering, and cache system. + +You can adopt and configure custom Rules without maintaining their implementation. Python authoring +details live in [Custom Rules](/concepts/rules/custom-rules). + +## One definition of project validity + +`sqb compile` is authoritative. Build and execution commands use the same compiler path and reject +configured findings before opening a warehouse connection. A successful compile means mandatory +compiler correctness and every selected Rule passed. + +Use `sqb rules` for focused catalogue inspection and development. Passing a focused Rule selection +does not claim that the complete configured Rules set passed. + +## Rules and other checks + +| Capability | Purpose | +|---|---| +| Compiler correctness | Mandatory requirements needed to construct a trustworthy project | +| Rules | Configurable static requirements over compiled project facts | +| Formatting | Canonical source rewriting through `sqb format` | +| Tests | Expected behavior of SQL logic | +| Audits | Warehouse-data quality requirements | +| Runtime checks | Validation of external resources and execution state | + +## Explore Rules + + + + Select exact Rules or families and configure project-owned options. + + + Understand diagnostics, exact exceptions, path ignores, and stale checks. + + + Learn evaluation order, focused execution, and dependency-aware reuse. + + + Author typed Python Rules over compiler-owned facts. + + diff --git a/concepts/rules/configuration-and-selection.mdx b/concepts/rules/configuration-and-selection.mdx new file mode 100644 index 0000000..33e4136 --- /dev/null +++ b/concepts/rules/configuration-and-selection.mdx @@ -0,0 +1,66 @@ +--- +title: "Configuration and selection" +description: "Select built-in and custom Rules by exact code or family prefix." +--- + +Configure built-in and custom Rules together: + +```toml +[rules] +select = ["SQBRSQL", "SQBRGRAPH", "XSQBRARCH"] +ignore = ["SQBRSQL004"] +``` + +An empty `select` disables configurable Rules. It does not disable mandatory compiler correctness. + +## Codes and families + +Built-in codes use `SQBR`, such as `SQBRSQL004`, `SQBRMODEL101`, and +`SQBRGRAPH101`. Custom codes use `XSQBR`, such as +`XSQBRARCH001`. + +A family is always the code with its final three digits removed: + +```text +SQBRSQL004 → SQBRSQL +XSQBRARCH001 → XSQBRARCH +XSQBR001 → XSQBR +``` + +Exact codes select one Rule. Prefixes select every matching Rule. `ignore` uses the same matching +semantics and takes precedence over `select`. + +Use the catalogue to inspect available codes and metadata: + +```bash +sqb rules list +sqb rules show SQBRSQL004 +``` + +## Rule options + +Custom Rules can declare typed options. Configure them by exact code: + +```toml +[rules.rule_options.XSQBRNAME001] +required_prefix = "order" +``` + +Unknown Rule codes, option names, and invalid option values fail configuration. + +## Focused selection + +Run one exact Rule or family while developing or adopting it: + +```bash +sqb rules run SQBRSQL004 +sqb rules run SQBRSQL +sqb rules run XSQBRARCH --select customer_orders +``` + +Focused execution constructs the compiler facts needed by the selected Rules. It does not replace +the complete configured enforcement performed by `sqb compile`. + + + See every `sqb rules` command and exit code. + diff --git a/concepts/rules/custom-rules.mdx b/concepts/rules/custom-rules.mdx new file mode 100644 index 0000000..980ff4e --- /dev/null +++ b/concepts/rules/custom-rules.mdx @@ -0,0 +1,90 @@ +--- +title: "Authoring custom Rules" +description: "Define repository-owned checks with one typed Python API." +--- + +Custom Rules are ordinary Python under `rules/**/*.py`. Only functions decorated with `@rule` +register; helpers, constants, dataclasses, and classes remain ordinary Python. + +## Model-subject Rules + +```python +from sqlbuild.rules import Finding, Model, RuleContext, rule + + +@rule( + code="XSQBRARCH001", + message="Final models must declare an order identifier", + remediation="Declare order_id in the model contract.", +) +def final_order_identifier(*, model: Model, ctx: RuleContext) -> list[Finding]: + declared = {column.name for column in ctx.columns.declared(model)} + return [] if "order_id" in declared else [ctx.finding(subject=model)] +``` + +Annotations determine invocation. The signature must contain exactly one typed `Model` or `Project` +subject and one typed `RuleContext`, all keyword-only. Parameter names and position are not semantic. +A model-subject Rule can still inspect project-wide facts. + +## Project-subject Rules + +Use `Project` when an invariant has no natural model subject: + +```python +from sqlbuild.rules import Finding, Project, RuleContext, rule + + +@rule( + code="XSQBRARCH002", + message="The project must contain a final model directory", + remediation="Add models/final and place final outputs beneath it.", +) +def final_directory(*, ctx: RuleContext, project: Project) -> list[Finding]: + del project + paths = ctx.project.tree.glob("models/final/**") + return [] if paths else [ctx.finding(subject="models")] +``` + +Rules cannot mutate SQL, resources, adapter lowering, or compiler output. They return diagnostics +only. + +## Typed options + +```python +from sqlbuild.rules import Finding, Model, RuleContext, RuleOption, rule + +REQUIRED_PREFIX = RuleOption.string( + name="required_prefix", + default="customer", + description="Required model-name prefix.", +) + + +@rule( + code="XSQBRNAME001", + message="The model name uses the wrong prefix", + remediation="Rename the model with the configured prefix.", + options=(REQUIRED_PREFIX,), +) +def required_prefix(*, model: Model, ctx: RuleContext) -> list[Finding]: + prefix = ctx.option(REQUIRED_PREFIX) + return [] if model.name.startswith(prefix) else [ctx.finding(subject=model)] +``` + +```toml +[rules.rule_options.XSQBRNAME001] +required_prefix = "order" +``` + +Unknown codes, option names, or invalid values fail configuration. + +## Continue + + + + Inspect SQL, graph, contracts, tests, audits, declarations, and project structure. + + + Test findings and keep helper and project observations cacheable. + + diff --git a/concepts/rules/custom-rules/rule-context.mdx b/concepts/rules/custom-rules/rule-context.mdx new file mode 100644 index 0000000..18c35f4 --- /dev/null +++ b/concepts/rules/custom-rules/rule-context.mdx @@ -0,0 +1,71 @@ +--- +title: "RuleContext and compiler facts" +description: "Inspect stable compiler-owned facts and opt into lazy SQL AST access." +--- + +`RuleContext` exposes typed, read-only views over the compiled project: + +- `ctx.sql`: authored and expanded SQL +- `ctx.graph`: compiler-resolved dependencies and dependents +- `ctx.columns`: declared and inferred output columns +- `ctx.contracts`: enforcement and grain facts +- `ctx.tests` and `ctx.audits`: checks associated with a model +- `ctx.declarations`: public and model-scoped enums and constants +- `ctx.project`: compiled resources and deterministic project-tree observations + +## Authored and expanded SQL + +Choose the SQL representation that matches the requirement: + +```python +sql = ctx.sql.for_model(model) + +authored_text = sql.authored.source +expanded_text = sql.expanded.source +``` + +An authoring convention may inspect what a developer wrote. A compiler-output convention may need +the expanded SQL after interpolation, declarations, and macros. + +## Common SQL structures + +Common structures are available through typed source nodes: + +```python +for cte in sql.expanded.ctes(): + ... + +for star in sql.authored.star_projections(): + ... +``` + +## Lazy Polyglot access + +Use the full Polyglot AST only when common projections are insufficient: + +```python +ast = sql.expanded.polyglot_ast() +for node in ast.walk(): + ... +``` + +AST construction is lazy. A path or contract Rule does not pay for SQL parsing merely because AST +access exists elsewhere. + +## Project structure + +Use the compiler-owned tree rather than direct filesystem calls: + +```python +parts = ctx.project.tree.relative_parts(path=model.path, under="models") +models = ctx.project.tree.resources_under("models/orders") +config = ctx.project.tree.read_text("rules/requirements.yaml") +``` + +Paths remain project-relative, and observations participate in cache invalidation. + +## Choose the subject deliberately + +A model-subject Rule is evaluated and cached per model. A project-subject Rule runs once and can +iterate `ctx.project.models`. Use a project subject only when the invariant genuinely needs a +project-wide view; its cache invalidation is intentionally broader. diff --git a/concepts/rules/custom-rules/testing-and-determinism.mdx b/concepts/rules/custom-rules/testing-and-determinism.mdx new file mode 100644 index 0000000..9198c43 --- /dev/null +++ b/concepts/rules/custom-rules/testing-and-determinism.mdx @@ -0,0 +1,62 @@ +--- +title: "Testing and deterministic Rules" +description: "Test custom findings and keep implementations safe for dependency-aware caching." +--- + +## Test Rules + +Use the public harness to exercise discovery, compilation, and Rules evaluation: + +```python +from sqlbuild.rules.testing import RuleCase, evaluate_rule +from rules.final_outputs import final_order_identifier + + +def test_given_missing_order_id_when_evaluating_then_reports_finding() -> None: + result = evaluate_rule( + rule=final_order_identifier, + test_case=RuleCase( + description="missing order identifier", + source='MODEL (description "Orders");\nSELECT 1 AS customer_id\n', + path="models/final/customer_orders.sql", + expected_finding_count=1, + ), + ) + + assert result.finding_count == 1 +``` + +`RuleCase.files` adds supporting project files. `RuleCase.config` supplies option values. Include +passing cases, failing cases, near misses, and exact source-position assertions. + +Repository pytest files remain outside the SQLBuild project's SQL `tests/` directory. + +## Helpers + +Custom Rules can import repository-owned helpers under `rules/` and supported pure Python modules. +Changing an imported helper invalidates the Rules that depend on it. + +Only decorated functions register. Ordinary functions, constants, dataclasses, and classes remain +helpers. + +## Deterministic inputs + +Direct filesystem, environment, clock, randomness, network, and subprocess access is rejected. +Those inputs cannot be reproduced safely by the Rules cache. + +Read supported project text and structure through `ctx.project.tree`: + +```python +text = ctx.project.tree.read_text("rules/requirements.yaml") +matches = ctx.project.tree.glob("models/*/interface/*.sql") +``` + +Both positive and negative observations are tracked. If `glob` returns no paths, adding a matching +path invalidates the cached result. + +## Cache granularity + +Cache identity incorporates the Rule implementation, imported helper closure, configured options, +subject, accessed compiler facts, tracked project observations, dialect, and compatibility +versions. Prefer model subjects for independent per-model checks and project subjects for genuine +cross-project invariants. diff --git a/concepts/rules/execution-and-caching.mdx b/concepts/rules/execution-and-caching.mdx new file mode 100644 index 0000000..0d180f6 --- /dev/null +++ b/concepts/rules/execution-and-caching.mdx @@ -0,0 +1,53 @@ +--- +title: "Execution and caching" +description: "Understand Rule enforcement order, focused runs, and dependency-aware reuse." +--- + +SQLBuild evaluates a project in this order: + +1. discovery, expansion, and mandatory compiler correctness +2. selected native built-in Rules +3. selected custom Python Rules +4. artifact completion + +Native built-ins always run before custom Rules. Structured compile output reports built-in and +custom durations separately, together with cache hits and misses. + +## Authoritative enforcement + +`sqb compile` runs the complete configured Rules set. Build and execution commands use the same +compiler path and reject Rule findings before opening a warehouse connection. + +`sqb rules run` is intentionally focused. Use it to inspect one exact code or family, but run the +normal compiler before treating the complete project as valid. + +## Dependency-aware caching + +Rules cache under `target/rules-cache`. A cache identity includes the Rule code, implementation and +imported helpers, options, invocation subject, accessed compiler facts, dialect, tracked project +observations, and compatibility versions. + +Model-subject Rules support fine-grained reuse. Editing one model invalidates affected model +subjects rather than every Rule invocation. Project-subject Rules run once and are appropriate for +genuinely project-wide invariants, with correspondingly broader invalidation. + +Positive and negative project-tree observations are dependencies. If a Rule observes that a glob +has no matches, adding a matching file invalidates that result. + +## Built-in-only execution + +When no custom Rule is selected, SQLBuild does not initialize the custom Python host. When every +custom result is cached, the host is not launched for evaluation. + +## Agent guidance + +Generate project-specific guidance from the configured Rules: + +```bash +sqb rules skills +sqb rules skills --check +``` + +SQLBuild manages `.agents/skills/sqlbuild-rules/SKILL.md`, +`.claude/skills/sqlbuild-rules/SKILL.md`, and `.opencode/skills/sqlbuild-rules/SKILL.md` without +overwriting divergent or unowned content. diff --git a/concepts/rules/findings-and-exceptions.mdx b/concepts/rules/findings-and-exceptions.mdx new file mode 100644 index 0000000..df2adf7 --- /dev/null +++ b/concepts/rules/findings-and-exceptions.mdx @@ -0,0 +1,52 @@ +--- +title: "Findings and exceptions" +description: "Understand Rule diagnostics and record intentional exceptions safely." +--- + +A Rule finding identifies an exact code, project-relative path, line, column, message, and +remediation. Error findings make compilation fail and prevent artifact completion. + +## Exact exceptions + +Use an exact exception for one known finding. Exceptions require a reason and are stale-checked: + +```toml +[[rules.rule_exceptions]] +rule = "SQBRSQL004" +path = "models/examples/sample_orders.sql" +reason = "This example intentionally demonstrates one sampled row." +``` + +An exception refers to one exact Rule code and path. If the finding disappears, SQLBuild reports +the stale exception so obsolete configuration does not accumulate silently. + +## Path-scoped ignores + +Use a path ignore when a documented project area intentionally follows a different convention: + +```toml +[[rules.rule_ignores]] +rules = ["SQBRSQL"] +paths = ["models/examples/**"] +reason = "Examples retain intentionally minimal SQL." +``` + +Path ignores accept exact codes and family prefixes. Keep their scope narrow and explain why the +project differs from the selected requirement. + +## Mandatory correctness + +Mandatory compiler correctness is not configurable and cannot be suppressed. A project must first +compile into a trustworthy representation before any selected Rule can run. + +## Machine-readable findings + +Use JSON when another tool consumes findings: + +```bash +sqb compile --json +sqb rules --json run SQBRSQL +``` + +Machine-readable output remains on stdout. Lifecycle progress and terminal status are written to +stderr. diff --git a/concepts/sql-linting.mdx b/concepts/sql-linting.mdx deleted file mode 100644 index e19ae50..0000000 --- a/concepts/sql-linting.mdx +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: "SQL Linting and Formatting" -description: "Native SQL diagnostics, safe fixes, and canonical formatting." ---- - -SQLBuild owns SQL lint and formatting behavior. It uses the active adapter dialect and understands -SQLBuild model headers, so no Sqruff or SQLFluff installation is required. - -| Command | Purpose | Changes files by default? | -|---------|---------|---------------------------| -| `sqb lint` | Report correctness and selected convention diagnostics | No | -| `sqb fix` | Apply only fixes SQLBuild can prove safe | Yes | -| `sqb format` | Write SQLBuild's canonical presentation | Yes | - -Use `sqb fix --check` or `sqb format --check` in CI. Add `--diff` to preview changes without writing -them. All three commands accept normal `--select`, `--select-file`, and `--exclude` scopes. - -## Rule policy - -Core rules run by default. Optional rules are enabled in project configuration by exact code or -prefix: - -```toml -[lint] -select = ["SQBL", "SQBL024"] -``` - -Lint codes use the `SQBL` prefix. Formatting owns presentation choices such as capitalization, -indentation, spacing, commas, and canonical SQL spelling; these are not inflated into lint warnings. -Kata remains separate and evaluates repository architecture and naming policy over compiled models. - -## Suppressions - -Suppress a local diagnostic only with a reason: - -```sql --- sqb: ignore SQBL004 because this fixture intentionally samples one row -SELECT value FROM items LIMIT 1 -``` - -Unused or malformed suppressions produce `SQBL000`. Mandatory compiler and model-header faults -cannot be suppressed. Prefer fixing the SQL or narrowing optional policy before adding a suppression. - -## Safe fixes - -`sqb fix` changes only findings with a deterministic equivalent rewrite. Diagnostics that require -schema knowledge or author intent remain reported and are never guessed. Review the diff before -committing, especially when enabling additional optional rules. - -See [`sqb lint`](/cli/lint), [`sqb fix`](/cli/fix), and [`sqb format`](/cli/format). diff --git a/docs.json b/docs.json index 8390adb..9d80444 100644 --- a/docs.json +++ b/docs.json @@ -100,7 +100,15 @@ }, "concepts/snapshots", "concepts/audits", - "concepts/sql-linting", + { + "group": "Rules", + "root": "concepts/rules", + "pages": [ + "concepts/rules/configuration-and-selection", + "concepts/rules/findings-and-exceptions", + "concepts/rules/execution-and-caching" + ] + }, "concepts/testing", "concepts/scenarios", "concepts/selectors", @@ -128,10 +136,11 @@ ] }, { - "group": "Kata", + "group": "Custom Rules", + "root": "concepts/rules/custom-rules", "pages": [ - "concepts/kata", - "concepts/kata/custom-rules" + "concepts/rules/custom-rules/rule-context", + "concepts/rules/custom-rules/testing-and-determinism" ] } ] @@ -188,12 +197,10 @@ "cli/playground", "cli/skills", "cli/compile", - "cli/lint", - "cli/fix", "cli/format", "cli/contract", "cli/scope", - "cli/kata", + "cli/rules", "cli/plan", "cli/build", "cli/load", diff --git a/feature-comparison.mdx b/feature-comparison.mdx index 0bf5655..7160aca 100644 --- a/feature-comparison.mdx +++ b/feature-comparison.mdx @@ -39,8 +39,8 @@ SQLBuild, dbt, and SQLMesh are all SQL pipeline frameworks. They share common gr | Column-level lineage | Compile-time, fast and rich modes | dbt Core: post-hoc via docs; dbt Fusion engine: compile-time | Compile-time | | Column contract validation | Compile-time inference plus runtime enforcement with `contract enforced` | YAML schema contracts at runtime | Schema contracts via plan | | Existing-schema contracts | Online read-only diff and safe repository generation | Codegen/packages | External schema tooling | -| Native lint/fix/format | Dialect-aware coded diagnostics, reasoned suppressions, safe fixes, and canonical formatting | External tools | Built-in formatter plus external linting | -| SQL architecture policy | Opt-in Kata rules over compiled models with coded faults, remediations, suppressions, and custom rules | Project conventions through packages and external tooling | Built-in audits and external linting | +| Rules and formatting | Compiler-integrated native and custom diagnostics, reasoned suppressions, and separate canonical formatting | External tools | Built-in formatter plus external linting | +| Compiler-integrated Rules | Native and custom rules over compiler-owned SQL, models, dependencies, contracts, tests, and project paths | Project conventions through packages and external tooling | Built-in audits and external linting | | SQL transpilation | For local E2E replay into DuckDB | No | For cross-dialect model execution | | Python macros | `@macro()` syntax | No (Jinja only) | SQLMesh macro syntax | | Compiler-enforced declaration scopes | Project, descendant-public, exact-owner-private, and model-private tiers with offline `sqb scope` inspection | No lexical declaration scopes | No lexical declaration scopes | @@ -115,7 +115,7 @@ SQLBuild, dbt, and SQLMesh are all SQL pipeline frameworks. They share common gr | Adapters | DuckDB, MotherDuck, Snowflake, BigQuery, Databricks, PostgreSQL, SQL Server | 30+ (community adapters) | DuckDB, Snowflake, BigQuery, Databricks, Spark, Redshift, Postgres, Trino, MySQL | | State requirements | Stateless by default | manifest.json + target/ | Requires state store (local database or PostgreSQL for production) | | Playground | `sqb playground` | Clone example repo | Example project | -| AI agent skills | General guidance with `sqb skills`; policy-derived guidance with `sqb kata skills` | No | No | +| AI agent skills | General guidance with `sqb skills`; Rules guidance with `sqb rules skills` | No | No | | Execution observability | Immutable schema-versioned lifecycle facts, invocation-local ordering, orchestration context, and typed project sinks | Events/artifacts plus external observability | Plans, state, and external observability | ## Where each tool fits diff --git a/index.mdx b/index.mdx index 2e670ed..96b5726 100644 --- a/index.mdx +++ b/index.mdx @@ -50,8 +50,8 @@ E2E replay. Before any model runs, SQLBuild does static analysis of your project - offline, no warehouse connection needed. - **Catch errors at compile.** SQL syntax, type inference, contract checks, and column lineage all run before execution. A bad reference or a type mismatch fails at compile, with an error that points at the line - not halfway through a warehouse run. -- **Keep SQL consistent natively.** `sqb lint`, `sqb fix`, and `sqb format` share SQLBuild's dialect-aware parser and model semantics, without a separate linter installation. -- **Enforce architecture deliberately.** Opt into [Kata](/concepts/kata) to check model structure, dependency CTEs, layer boundaries, joins, naming, contracts, and test coverage with coded faults and remediations. +- **Turn review decisions into compiler checks.** Configure [Rules](/concepts/rules) over compiler-owned SQL, models, dependencies, contracts, tests, and project paths. +- **Format separately.** `sqb format` owns deterministic source rewriting; Rules remain diagnostic-only. - **Fast, because it's Rust where it counts.** Static analysis runs on [Polyglot](https://github.com/tobilg/polyglot), a Rust SQL engine (MIT, 32+ dialects), so compile stays quick even on large projects. - **Open, not paywalled.** The static analysis is part of the Apache-2.0 core - no proprietary engine, no separate login, and no paid tier gating the smart checks. diff --git a/quickstart.mdx b/quickstart.mdx index 1a1671d..88c2cdc 100644 --- a/quickstart.mdx +++ b/quickstart.mdx @@ -256,7 +256,7 @@ waffle-shop/ ## Next steps - [Models](/concepts/models) - understand `MODEL()` headers and materialization types -- [Kata](/concepts/kata) - opt into repository architecture and model-shape checks +- [Rules](/concepts/rules) - add compile-time SQL, architecture, and model-shape checks - [Functions](/concepts/functions) - SQL UDFs, Python UDFs, and table functions - [Incremental](/concepts/incremental) - learn about cursor-based incremental strategies - [Audits](/concepts/audits) - configure data quality checks