Add ch dictionary primitive

.changeset/dictionary-schema-primitive.md

@@ -0,0 +1,16 @@
+---
+"chkit": patch
+"@chkit/core": patch
+"@chkit/clickhouse": patch
+"@chkit/plugin-pull": patch
+"@chkit/plugin-codegen": patch
+---
+
+Add `dictionary()` as a first-class ClickHouse schema primitive, mirroring `materializedView()` across the full lifecycle: DSL authoring, validation, canonicalization, SQL rendering, migration planning/diff, drift, `check`, destructive-op safety, `pull` introspection, and `codegen` typed interfaces.
+
+- `dictionary({ database, name, attributes, primaryKey, source, layout, lifetime, range?, settings?, comment? })` — attributes support `default`/`expression` (mutually exclusive), `hierarchical`, `bidirectional` (requires `hierarchical`), `injective`, and `isObjectId`. `range: { min, max }` renders `RANGE(MIN ... MAX ...)` for `RANGE_HASHED`/`COMPLEX_KEY_RANGE_HASHED` layouts, and `settings` renders `SETTINGS(...)`.
+- ClickHouse has no `ALTER DICTIONARY`, so any structural change plans a single atomic `CREATE OR REPLACE DICTIONARY`. Dropping a dictionary is treated as destructive and blocked without `--allow-destructive`.
+- Set `renamedFrom` on a dictionary (or pass `--rename-dictionary old_db.old=new_db.new` to `chkit generate`) to rename a dictionary via `RENAME DICTIONARY IF EXISTS ... TO ...` instead of a destructive drop + create.
+- `chkit pull` introspects live dictionaries (including `RANGE`/`SETTINGS` and all attribute modifiers) into typed schema files, preserving ClickHouse's `[HIDDEN]` password redaction on `SOURCE(...)` credentials. A `SOURCE(...)` password change diffs and migrates like any other field change; `chkit generate` warns when a literal password is about to be written into migration SQL as plain text. `chkit pull` warns in two cases: when an introspected password comes back as `[HIDDEN]` (chkit can't recover the real value, so that dictionary's `source` is excluded from future diffs until it's replaced), and when ClickHouse is configured to reveal real passwords on introspection (`display_secrets_in_show_and_select` + `displaySecretsInShowAndSelect`), since that writes a plain-text credential into the generated schema file with no other indication. All warnings print to the console and are included as a `warnings` array in `--json` output.
+- `codegen` generates a typed interface (and optional Zod schema) for each dictionary from its `attributes`, always included regardless of `includeViews`.
+- `ON CLUSTER` mode stamps `ON CLUSTER <name>` onto every dictionary DDL statement, including `CREATE OR REPLACE DICTIONARY` and `RENAME DICTIONARY`.

apps/docs/src/content/docs/cli/generate.md

@@ -21,6 +21,7 @@ chkit generate [flags]
 | `--migration-id <id>` | string | — | Escape hatch: override the default timestamp migration prefix |
 | `--rename-table <mapping>` | string | — | Explicit table rename: `old_db.old_table=new_db.new_table` |
 | `--rename-column <mapping>` | string | — | Explicit column rename: `db.table.old_column=new_column` |
+| `--rename-dictionary <mapping>` | string | — | Explicit dictionary rename: `old_db.old_dict=new_db.new_dict` |
 | `--table <selector>` | string | — | Scope operations to matching tables |
 | `--dryrun` | boolean | `false` | Print the plan without writing any files |
 | `--empty` | boolean | `false` | Scaffold a blank manual migration without diffing the schema |
@@ -63,12 +64,22 @@ An empty match set emits a warning and produces no output.
 chkit detects potential renames through two mechanisms:
 
 1. **Schema metadata** — set `renamedFrom` on your schema definition
-2. **CLI flags** — `--rename-table old_db.old_table=new_db.new_table` and `--rename-column db.table.old_col=new_col`
+2. **CLI flags** — `--rename-table old_db.old_table=new_db.new_table`, `--rename-column db.table.old_col=new_col`, and `--rename-dictionary old_db.old_dict=new_db.new_dict`
 
 CLI flags take priority when both sources specify a mapping for the same object. Rename flags accept comma-separated values for multiple mappings.
 
+A dictionary rename emits a single `RENAME DICTIONARY IF EXISTS ... TO ...` statement instead of a `drop_dictionary` + `create_dictionary` pair — see [Dictionary rename](/schema/dsl-reference/#dictionary-rename).
+
 Validation errors are raised for conflicting, chained, or cyclic rename mappings.
 
+### Dictionary password warnings
+
+A dictionary's `SOURCE(...)` clause is a raw string (see [Credentials in `source`](/schema/dsl-reference/#credentials-in-source)), so any credentials it embeds are written verbatim into the generated migration SQL — ClickHouse has no DDL-level secret substitution. A password change is a real diff like any other field change and produces a `CREATE OR REPLACE DICTIONARY` migration.
+
+`generate` warns when a dictionary being created or replaced this run has a literal `password '...'` in its `SOURCE(...)` — it will land in the committed migration file as plain text. This prints to the console and is included as a `warnings` array in `--json` output.
+
+The one exception: a dictionary whose `source` still carries ClickHouse's `[HIDDEN]` introspection placeholder (written by [`chkit pull`](/plugins/pull/#credential-handling-hidden-passwords) when it can't recover the real password) never produces a migration on its own — chkit doesn't know the real value, so it can't safely diff or render it. Replace `[HIDDEN]` with a real credential in the schema file first.
+
 ### Dryrun mode
 
 With `--dryrun`, the command prints the migration plan (operations with risk levels and SQL) without writing any files. Useful for previewing changes before committing.
@@ -79,7 +90,7 @@ Plans for objects in a database lead with a `create_database` operation (`CREATE
 
 With `--empty`, the command skips the schema diff entirely and writes a blank, timestamped migration stub for you to hand-edit. Use it for DDL that chkit does not model — raw `INSERT`/backfill statements, `OPTIMIZE`, manual dictionary reloads, or one-off data fixes.
 
-The stub carries the standard migration header (with `operation-count: 0`) plus a placeholder comment. The snapshot is left untouched, so an empty migration never absorbs pending schema drift. The `--name` and `--migration-id` flags apply; without `--name`, the file defaults to `manual`. Schema-diff flags (`--table`, `--rename-table`, `--rename-column`, `--dryrun`) are not used in empty mode.
+The stub carries the standard migration header (with `operation-count: 0`) plus a placeholder comment. The snapshot is left untouched, so an empty migration never absorbs pending schema drift. The `--name` and `--migration-id` flags apply; without `--name`, the file defaults to `manual`. Schema-diff flags (`--table`, `--rename-table`, `--rename-column`, `--rename-dictionary`, `--dryrun`) are not used in empty mode.
 
 `chkit migrate` picks the file up like any other migration and applies it in filename order. Write your SQL into the stub *before* applying it — editing a migration after it has run triggers a checksum mismatch.
 
@@ -129,6 +140,12 @@ chkit generate --rename-table old_db.users=new_db.accounts
 chkit generate --rename-column analytics.events.old_name=new_name
 ```
 
+**Explicit dictionary rename:**
+
+```sh
+chkit generate --rename-dictionary old_db.old_dict=new_db.new_dict
+```
+
 ## Exit codes
 
 | Code | Meaning |
@@ -152,7 +169,8 @@ chkit generate --rename-column analytics.events.old_name=new_name
     { "type": "create_database", "key": "database:default", "risk": "safe", "sql": "CREATE DATABASE IF NOT EXISTS default;" },
     { "type": "create_table", "key": "default.users", "risk": "safe", "sql": "CREATE TABLE ..." }
   ],
-  "renameSuggestions": []
+  "renameSuggestions": [],
+  "warnings": []
 }
 ```
 
@@ -167,7 +185,8 @@ chkit generate --rename-column analytics.events.old_name=new_name
   "snapshotFile": "./chkit/meta/snapshot.json",
   "definitionCount": 3,
   "operationCount": 2,
-  "riskSummary": { "safe": 2, "caution": 0, "danger": 0 }
+  "riskSummary": { "safe": 2, "caution": 0, "danger": 0 },
+  "warnings": []
 }
 ```
 

apps/docs/src/content/docs/guides/clickhouse-compatibility.md

@@ -20,6 +20,7 @@ A few schema features depend on the ClickHouse version of your target:
 | [Refreshable materialized views](/schema/refreshable-views/) | Production-ready on **24.10+** (no flag). Experimental and flag-gated on 23.12–24.9. chkit targets 24.10+. |
 | `set` data-skipping index | **ClickHouse 26+** requires the `set(0)` form rather than a bare `set`; chkit emits `set(maxRows)` accordingly. See the [DSL reference](/schema/dsl-reference/). |
 | `uniqueKey` | Renders `UNIQUE KEY` DDL, which is supported on ObsessionDB / ClickHouse Cloud engines but rejected by vanilla `MergeTree`. |
+| [`dictionary()`](/schema/dsl-reference/#dictionary) | DDL `CREATE DICTIONARY` only — supported broadly on 21.x+. chkit does not model XML-config dictionaries or `RENAME`/`SYSTEM RELOAD DICTIONARY`. |
 
 If you target a single-node open-source ClickHouse, prefer the standard `MergeTree` engine family and avoid the Cloud/Shared-only features above.
 

apps/docs/src/content/docs/plugins/codegen.md

@@ -10,6 +10,7 @@ This document covers practical usage of the optional `codegen` plugin.
 ## What it does
 
 - Generates deterministic TypeScript row types from chkit schema definitions.
+- Generates a typed interface (and optional Zod schema) for each `dictionary()` from its `attributes` — dictionaries are always included, regardless of `includeViews`.
 - Optionally generates Zod schemas from the same definitions.
 - Optionally generates typed ingestion functions for inserting rows into ClickHouse tables. Generated ingest helpers gzip-compress request bodies by default and can opt out per call.
 - Optionally generates a self-contained runtime migration module with all migration SQL inlined, for environments without filesystem access (e.g., Cloudflare Workers).
@@ -186,4 +187,4 @@ When `runOnGenerate` is enabled (the default), the migration module is regenerat
 
 - Query-level type inference is not included.
 - Arbitrary SQL expression typing is not included.
-- Views/materialized views are opt-in and emitted conservatively.
+- Views/materialized views are opt-in (`includeViews`) and emitted conservatively (`{ [key: string]: unknown }`). Dictionaries are always included and typed from `attributes`, since their shape is structured rather than an arbitrary query.

apps/docs/src/content/docs/plugins/pull.md

@@ -1,6 +1,6 @@
 ---
 title: Pull Plugin
-description: Introspect live ClickHouse tables, views, and materialized views and generate chkit schema files.
+description: Introspect live ClickHouse tables, views, materialized views, and dictionaries and generate chkit schema files.
 sidebar:
   order: 3
 ---
@@ -11,6 +11,7 @@ This document covers practical usage of the optional `pull` plugin.
 
 - Connects to a live ClickHouse instance and introspects table metadata (columns, engines, indexes, projections, partitioning, TTL, settings).
 - Introspects views and materialized views (including `TO` clause parsing).
+- Introspects dictionaries (attributes — including `HIERARCHICAL`/`BIDIRECTIONAL`/`INJECTIVE`/`IS_OBJECT_ID` modifiers — primary key, `SOURCE`/`LAYOUT`/`LIFETIME`/`RANGE`/`SETTINGS`), preserving ClickHouse's `[HIDDEN]` password redaction — see [Credential handling](#credential-handling-hidden-passwords).
 - Generates a deterministic TypeScript schema file using `@chkit/core` builders.
 - Supports filtering by database and dry-run previews.
 
@@ -106,7 +107,45 @@ export default schema(app_events, app_events_view, app_events_mv)
 
 Tables may also include `uniqueKey`, `ttl`, `settings`, `indexes`, and `projections` when present in the source metadata.
 
+## Credential handling (`[HIDDEN]` passwords)
+
+By default, ClickHouse redacts inline `SOURCE(...)` passwords to `[HIDDEN]` on introspection (`system.tables.create_table_query`, `SHOW CREATE DICTIONARY`), and chkit does not attempt to work around that. When a pulled dictionary's `source` contains `[HIDDEN]`, `chkit pull` prints a console warning (and includes it in a `warnings` array in `--json` output), and the generated file emits the source verbatim with a leading comment:
+
+```ts
+// NOTE: password redacted by ClickHouse — replace '[HIDDEN]' with your credential (e.g. process.env.X).
+const default_users_dict = dictionary({
+  database: "default",
+  name: "users_dict",
+  attributes: [
+    { name: "id", type: "UInt64" },
+    { name: "name", type: "String" },
+  ],
+  primaryKey: ["id"],
+  source: "MYSQL(host 'db' port 3306 user 'reader' password '[HIDDEN]' db 'app' table 'users')",
+  layout: "HASHED()",
+  lifetime: "300",
+})
+```
+
+Replace `[HIDDEN]` with a real credential — typically an environment-variable interpolation, matching how you'd author the dictionary by hand (see [Credentials in `source`](/schema/dsl-reference/#credentials-in-source)):
+
+```ts
+source: `MYSQL(host 'db' port 3306 user 'reader' password '${process.env.MYSQL_PASSWORD}' db 'app' table 'users')`,
+```
+
+For round-trip fidelity without a manual edit, use [named collections](https://clickhouse.com/docs/operations/named-collections) on the ClickHouse side instead of an inline password — chkit does not require this, but it's the ClickHouse-native way to avoid the redaction entirely.
+
+Because a `source` still carrying `[HIDDEN]` is excluded from the diff entirely (see [Credentials in `source`](/schema/dsl-reference/#credentials-in-source)), `chkit generate` won't produce a migration for that dictionary's `source` until you replace the placeholder with a real value.
+
+### Recovering the real password instead
+
+ClickHouse can be configured to skip the redaction and hand back the real password on introspection: enable the server-side `display_secrets_in_show_and_select` setting and grant the connecting user `displaySecretsInShowAndSelect`. The `[HIDDEN]` warning mentions this escape hatch.
+
+If you do this, be aware of the consequence: `chkit pull` has no way to detect that this is happening, or to re-redact the value on your behalf — it just copies through whatever ClickHouse returns. The real password lands in the generated schema file in plain text, same as any dictionary you'd author by hand with an inline credential. `chkit pull` detects this case too and warns that a plain-text password was written to the file, so you don't discover it by accident later.
+
 ## Current limits
 
 - Materialized views without a `TO` clause are skipped.
+- Dictionaries whose `create_table_query` can't be parsed (attributes, primary key, or `SOURCE`/`LAYOUT`/`LIFETIME` missing) are skipped.
+- XML-config dictionaries (not created via DDL) are not introspected.
 - Requires a live ClickHouse connection.