diff --git a/.changeset/stash-sql-edge-skills.md b/.changeset/stash-sql-edge-skills.md new file mode 100644 index 00000000..1b6d8ca7 --- /dev/null +++ b/.changeset/stash-sql-edge-skills.md @@ -0,0 +1,56 @@ +--- +'stash': minor +'@cipherstash/wizard': minor +'@cipherstash/stack': patch +--- + +Two new bundled agent skills for the integrations that don't use an ORM — +`stash-sql` and `stash-edge` (#754). + +Everything a raw-SQL or edge integration needed was reachable only from +`dist/*.d.ts` JSDoc, the Postgres catalog, or experiment: grepping the skills +`stash init` installs for `postgres-js|::jsonb::eql|sql.json|query_text_search` +returned a single hit, in an unrelated code comment. + +**`stash-sql`** — hand-written SQL over `pg` / `postgres-js`, no ORM. The +column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, +`@@`, `@>` each encrypted domain accepts, and against which `eql_v3.query_*` +operand), the storage-vs-query payload distinction, per-driver parameter +binding, recipes for equality / free-text / range / `ORDER BY` / JSON +containment / JSON field selectors, and the `information_schema` drift check. +Two failure modes get their mechanism spelled out: pre-stringifying a payload +on postgres-js double-encodes it into a jsonb *string* scalar, tripping the +domain CHECK with a message naming neither JSON nor encoding; and leaving an +operand as bare `jsonb` silently selects a different operator overload — one +that coerces to the *storage* domain and so rejects the ciphertext-free query +term. + +**`stash-edge`** — the `@cipherstash/stack/wasm-inline` entry for Deno, +Supabase Edge Functions, Cloudflare Workers, and Bun. Import specifier per +runtime, the four mandatory `CS_*` variables and minting them with +`stash env`, how the WASM client surface differs from the native typed client +(no `.audit()`, no `.withLockContext()`, per-item bulk shape, ESM-only), and +the auth-strategy `Result` that must be unwrapped before it reaches +`config.authStrategy`. + +Both carry **the credential-identity rule**, a silent data-loss footgun now +also stated in `stash-cli` (under `env` and `encrypt backfill`) and +`stash-supabase`: EQL index terms derive from the ZeroKMS client key, so rows +written under one credential and queried under another decrypt correctly and +never match a query, with no error. + +`stash-encryption` now states that the two entries' schema types **do not +interchange** — their column classes carry private fields, so TypeScript +compares them nominally and rejects a shared schema module in both directions. +It works at runtime, which makes a type assertion the tempting fix; the +guidance is to author the schema against exactly one entry instead. + +`stash init` / `stash impl` handoffs and the `@cipherstash/wizard` skills +prompt install both skills for the `postgresql` and `supabase` integrations. +Drizzle and Prisma Next get cross-links from their own skills instead, since +those integrations emit correctly-typed operands themselves. + +Also fixes the `@cipherstash/stack/wasm-inline` module JSDoc, which showed +`OidcFederationStrategy.create(...)`'s `Result` being passed straight to +`config.authStrategy` without unwrapping — the same JSDoc the raw-SQL surface +was being reverse-engineered from. diff --git a/AGENTS.md b/AGENTS.md index 943dbb98..bd5c17b0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -85,7 +85,7 @@ If these variables are missing, tests that require live encryption will fail or - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) - `examples/*`: Working apps (basic, prisma, supabase-worker) - `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo). -- `skills/*`: Agent skills (`stash-cli`, `stash-encryption`, `stash-indexing`, `stash-drizzle`, `stash-dynamodb`, `stash-supabase`, `stash-prisma-next`, `stash-supply-chain-security`) +- `skills/*`: Agent skills (`stash-cli`, `stash-encryption`, `stash-indexing`, `stash-sql`, `stash-edge`, `stash-drizzle`, `stash-dynamodb`, `stash-supabase`, `stash-prisma-next`, `stash-supply-chain-security`) ## Agent Skills — these ship to customers @@ -113,6 +113,8 @@ nothing type-checks them, and the damage lands in a customer's repo, not ours. | Drizzle / Supabase / Prisma Next / DynamoDB integrations | `skills/stash-drizzle`, `skills/stash-supabase`, `skills/stash-prisma-next`, `skills/stash-dynamodb` | | The rollout/cutover lifecycle (`packages/migrate`, `stash encrypt *`) | `skills/stash-encryption` and `skills/stash-cli` | | The `@cipherstash/eql` pin, `eql install`/`eql migration` behaviour, or index-related SQL guidance | `skills/stash-indexing` | +| The EQL operator/domain surface (`eql_v3.query_*` casts, predicate forms) | `skills/stash-sql` | +| `packages/stack/src/wasm-inline.ts`, the WASM entry's exports, or `stash env` | `skills/stash-edge` | | pnpm config, CI workflows, dependency policy | `skills/stash-supply-chain-security` | | The durable agent rules themselves | `packages/cli/src/commands/init/doctrine/AGENTS-doctrine.md` | diff --git a/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts b/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts index e4a2e3aa..bc9d6ec4 100644 --- a/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts +++ b/packages/cli/src/commands/init/lib/__tests__/install-skills.test.ts @@ -82,6 +82,28 @@ describe('SKILL_MAP', () => { expect(SKILL_MAP.postgresql).not.toContain('stash-supabase') expect(SKILL_MAP.postgresql).not.toContain('stash-prisma-next') }) + + // #754: the no-ORM path had no source for the raw-SQL binding surface or + // the WASM entry — an integration on it had to reverse-engineer both from + // `dist/*.d.ts` and the Postgres catalog. `postgresql` is that path; + // Supabase shares it (Edge Functions are the flagship WASM-entry use, and + // its migrations/RPC are hand-written SQL). + it.each([ + 'postgresql', + 'supabase', + ] as const)('%s includes the raw-SQL and edge skills', (integration) => { + expect(SKILL_MAP[integration]).toContain('stash-sql') + expect(SKILL_MAP[integration]).toContain('stash-edge') + }) + + // The ORM integrations emit correctly-typed operands themselves, so they + // get cross-links from their own skills rather than the full install. + it.each([ + 'drizzle', + 'prisma-next', + ] as const)('%s does not install the raw-SQL skill', (integration) => { + expect(SKILL_MAP[integration]).not.toContain('stash-sql') + }) }) describe('skillsFor', () => { diff --git a/packages/cli/src/commands/init/lib/install-skills.ts b/packages/cli/src/commands/init/lib/install-skills.ts index acd15818..a57897d4 100644 --- a/packages/cli/src/commands/init/lib/install-skills.ts +++ b/packages/cli/src/commands/init/lib/install-skills.ts @@ -12,10 +12,16 @@ import { findBundledDir } from './bundled-paths.js' */ export const SKILL_MAP: Record = { drizzle: ['stash-encryption', 'stash-drizzle', 'stash-indexing', 'stash-cli'], + // Supabase gets the raw-SQL and edge skills on top of its own: Edge + // Functions are the flagship use of the WASM entry, and Supabase projects + // write hand-written SQL in migrations and RPC even when the app itself + // goes through PostgREST (#754). supabase: [ 'stash-encryption', 'stash-supabase', 'stash-indexing', + 'stash-sql', + 'stash-edge', 'stash-cli', ], 'prisma-next': [ @@ -24,7 +30,16 @@ export const SKILL_MAP: Record = { 'stash-indexing', 'stash-cli', ], - postgresql: ['stash-encryption', 'stash-indexing', 'stash-cli'], + // The no-ORM path: `stash-sql` (binding + predicate forms) and `stash-edge` + // (WASM entry, CS_* credentials) are the two skills this integration has no + // other source for — everything else assumes an ORM emits the operands. + postgresql: [ + 'stash-encryption', + 'stash-indexing', + 'stash-sql', + 'stash-edge', + 'stash-cli', + ], } /** The skills every integration gets — the safe fallback for an unmapped one. */ diff --git a/packages/cli/src/commands/init/lib/setup-prompt.ts b/packages/cli/src/commands/init/lib/setup-prompt.ts index 6380cacf..68fd147c 100644 --- a/packages/cli/src/commands/init/lib/setup-prompt.ts +++ b/packages/cli/src/commands/init/lib/setup-prompt.ts @@ -110,6 +110,10 @@ const SKILL_PURPOSES: Record = { 'Prisma Next-specific patterns: `cipherstash.*` field constructors, migration flow, encrypted query operators', 'stash-indexing': 'index recipes for encrypted columns — the `eql_v3` extractor functional indexes, Supabase/managed-Postgres constraints, EXPLAIN verification', + 'stash-sql': + 'hand-written SQL over `pg` / `postgres-js`: the encrypted predicate matrix, `eql_v3.query_*` operand casts, per-driver parameter binding', + 'stash-edge': + 'the `@cipherstash/stack/wasm-inline` entry for Deno / Supabase Edge Functions / Workers: imports, `CS_*` credentials, the credential-identity rule', 'stash-dynamodb': 'DynamoDB encryption: per-item encrypt/decrypt, HMAC attribute keys, audit logging', 'stash-cli': diff --git a/packages/stack/src/wasm-inline.ts b/packages/stack/src/wasm-inline.ts index 0b7e8966..54c79138 100644 --- a/packages/stack/src/wasm-inline.ts +++ b/packages/stack/src/wasm-inline.ts @@ -64,17 +64,24 @@ * import { OidcFederationStrategy } from "@cipherstash/stack/wasm-inline" * import { cookieStore } from "@cipherstash/auth/cookies" * - * const authStrategy = OidcFederationStrategy.create( + * // `create` returns a Result — UNWRAP it. `config.authStrategy` takes the + * // strategy itself; handing it the Result fails later and opaquely. + * const strategy = OidcFederationStrategy.create( * "crn:ap-southeast-2.aws:my-workspace-id", () => getClerkSessionToken(req), * { store: cookieStore({ request: req, responseHeaders }) }, * ) - * const client = await Encryption({ schemas, config: { authStrategy, clientId, clientKey } }) + * if (strategy.failure) throw new Error(strategy.failure.error.message) + * + * const client = await Encryption({ + * schemas, config: { authStrategy: strategy.data, clientId, clientKey }, + * }) * ``` * * For service-to-service / CI use with a custom token store, build an * `AccessKeyStrategy.create(workspaceCrn, accessKey, { store })` the same - * way (it derives the region from the CRN). Both strategies are - * re-exported from this entry. + * way — same Result-unwrapping, and it derives the region from the CRN. Both + * strategies are re-exported from this entry. An auth strategy and + * `config.accessKey` are mutually exclusive. */ import { withResult } from '@byteslice/result' diff --git a/packages/wizard/src/lib/install-skills.ts b/packages/wizard/src/lib/install-skills.ts index 474bc2bd..25442278 100644 --- a/packages/wizard/src/lib/install-skills.ts +++ b/packages/wizard/src/lib/install-skills.ts @@ -13,14 +13,24 @@ import type { Integration } from './types.js' */ const SKILL_MAP: Record = { drizzle: ['stash-encryption', 'stash-drizzle', 'stash-indexing', 'stash-cli'], + // `stash-sql` / `stash-edge` mirror the CLI's SKILL_MAP — see the comments + // there for why Supabase and the generic (no-ORM) path get them (#754). supabase: [ 'stash-encryption', 'stash-supabase', 'stash-indexing', + 'stash-sql', + 'stash-edge', 'stash-cli', ], prisma: ['stash-encryption', 'stash-indexing', 'stash-cli'], - generic: ['stash-encryption', 'stash-indexing', 'stash-cli'], + generic: [ + 'stash-encryption', + 'stash-indexing', + 'stash-sql', + 'stash-edge', + 'stash-cli', + ], } /** diff --git a/skills/stash-cli/SKILL.md b/skills/stash-cli/SKILL.md index 7cf99621..eda6fe9e 100644 --- a/skills/stash-cli/SKILL.md +++ b/skills/stash-cli/SKILL.md @@ -455,6 +455,8 @@ Backfill **auto-detects the target column's EQL version** from its Postgres doma **Dual-write precondition.** The application must already write both `` and `_encrypted` on every insert and update. Otherwise rows written *during* the backfill land in plaintext only, silently. The first run prompts (interactive) or requires `--confirm-dual-writes-deployed` (non-interactive), then records `dual_writing`. Resumes don't re-prompt. +**Credential precondition — run the backfill with the *application's* credentials.** Backfill encrypts through whichever `CS_*` credentials are in its environment, and EQL index terms derive from the ZeroKMS client key. Backfilling from a laptop on the local device profile, then querying from an app using credentials minted by `stash env`, produces rows that decrypt correctly and **never match a query** — with no error. Export the target environment's `CS_*` values in the shell running the backfill. See [`env`](#env) and `stash-edge` § The Credential-Identity Rule. + | Flag | Description | |---|---| | `--table` / `--column` | Required | @@ -514,6 +516,14 @@ CS_CLIENT_KEY= CS_CLIENT_ACCESS_KEY=CSAK… ``` +> **Every writer of a searchable column must use these same credentials** — +> including `stash encrypt backfill`, seed scripts, and admin tools — or their +> rows decrypt but never match a query. EQL index terms derive from the ZeroKMS +> client key, so a row written under one credential and queried under another +> decrypts correctly and silently fails every search. Mint one credential per +> environment and export it for **every** process that writes that +> environment's data. See `stash-edge` § The Credential-Identity Rule. + Things to know: - **The access key is shown exactly once** — CTS cannot re-reveal it. Pipe the diff --git a/skills/stash-drizzle/SKILL.md b/skills/stash-drizzle/SKILL.md index 05728057..76eff338 100644 --- a/skills/stash-drizzle/SKILL.md +++ b/skills/stash-drizzle/SKILL.md @@ -417,6 +417,8 @@ import { index } from "drizzle-orm/pg-core" Run `ANALYZE ` after the migration applies — an expression index gathers no statistics at `CREATE INDEX` time. For when to create indexes during a rollout (after backfill, before switching reads), engagement rules, and `EXPLAIN` verification, see the `stash-indexing` skill. +> **Dropping to raw SQL?** `db.execute(sql\`…\`)` bypasses the operators this integration emits, so you own the operand casts yourself — an encrypted predicate needs its needle cast to the column's `eql_v3.query_*` domain, and the driver's parameter-binding rules differ between `pg` and `postgres-js`. The `stash-sql` skill is the reference for both. + ## Migrating an Existing Column to Encrypted The hard case: a Drizzle table that already exists in production with live data in a plaintext column you want to encrypt. You can't just change the column type — that would drop the data and break NOT NULL constraints. diff --git a/skills/stash-edge/SKILL.md b/skills/stash-edge/SKILL.md new file mode 100644 index 00000000..2359d39f --- /dev/null +++ b/skills/stash-edge/SKILL.md @@ -0,0 +1,398 @@ +--- +name: stash-edge +description: Run CipherStash encryption on edge and non-Node runtimes with the `@cipherstash/stack/wasm-inline` entry — Deno, Supabase Edge Functions, Cloudflare Workers, and Bun. Covers the import specifier per runtime, the four mandatory `CS_*` variables and minting them with `stash env`, the credential-identity rule (rows written under different credentials decrypt but never match a query), how the WASM client surface differs from the native typed client, and why an EQL v3 schema module cannot be shared across the two entries. Use when adding encryption to a Supabase Edge Function, a Worker, or a Deno service; when a native module fails to load in a deployed runtime; when wiring `CS_*` secrets into an edge deploy; or when encrypted search returns zero rows on the edge but works locally. +--- + +# Encryption on the Edge (WASM entry) + +`@cipherstash/stack` has two runtime entries. The default one binds +`@cipherstash/protect-ffi`, a Node-API native module, and must be loaded by +Node's own `require`. **`@cipherstash/stack/wasm-inline` is the entry for +everywhere else** — it carries the WASM build of the same engine as a base64 +blob inside the JS, so there is no native binding, no separate `.wasm` fetch, +and nothing for a bundler to externalise. + +This skill covers that entry and the deployment shape around it. It is EQL v3 +throughout. For the SQL that actually queries the encrypted columns — the +predicate forms and driver binding rules — see `stash-sql`; edge functions +almost always talk to Postgres over a raw driver, so the two are usually read +together. + +## When to Use This Skill + +- Adding encryption to a Supabase Edge Function, Cloudflare Worker, Deno + service, or Bun app. +- A deployed runtime fails to load the native module (`protect-ffi`), or a + bundler chokes trying to include it. +- Wiring `CS_*` credentials into an edge deploy, or minting them at all. +- Encrypted search works locally but returns **zero rows** in the deployed + function — see [The Credential-Identity Rule](#the-credential-identity-rule-a-silent-data-footgun). +- A schema module shared with Node tooling fails to typecheck against the + edge client. + +## Choosing the Entry + +| Runtime | Entry | Why | +|---|---|---| +| Node server, Next.js server code | `@cipherstash/stack` (+ `/v3`) | Native NAPI is faster; keep `@cipherstash/protect-ffi` external (e.g. Next's `serverExternalPackages`) | +| Supabase Edge Functions | `@cipherstash/stack/wasm-inline` | Deno, V8-only, no native modules | +| Cloudflare Workers | `@cipherstash/stack/wasm-inline` | V8 isolate, no native modules | +| Deno (any) | `@cipherstash/stack/wasm-inline` | No NAPI under Deno's default permissions | +| Bun | `@cipherstash/stack/wasm-inline` | Works, and avoids native-module resolution differences | +| Anywhere bundling server code | `@cipherstash/stack/wasm-inline` | Bundles cleanly; nothing to externalise | + +**The WASM entry is ESM-only.** Its `exports` map has an `import` condition +and no `require` — deliberately, since the runtimes it targets are ESM. A CJS +`require('@cipherstash/stack/wasm-inline')` will not resolve. Node consumers +that need it must be ESM (`"type": "module"` or `.mjs`). + +## Importing It + +### Supabase Edge Functions / Deno — `npm:` specifier + +The Edge runtime resolves `npm:` specifiers at function start; there is no +build step. + +```ts +import { + Encryption, encryptedTable, types, isEncrypted, +} from 'npm:@cipherstash/stack@1.0.0-rc.4/wasm-inline' +``` + +**Pin an exact version.** Deno caches by specifier, so an unpinned import +drifts between deploys. And while the package is on a prerelease line, a +caret range does not do what it looks like: `@^1.0.0` will **not** match +`1.0.0-rc.4`, because semver ranges exclude prereleases unless the range +itself names one. Use the exact version, or a prerelease-bearing range +(`@^1.0.0-rc.4`). Check what is current with `npm view @cipherstash/stack dist-tags`. + +### Deno with an import map + +For a project with a `deno.json`, map the specifier once and import the bare +name everywhere: + +```jsonc +{ + "imports": { + "@cipherstash/stack/wasm-inline": "npm:@cipherstash/stack@1.0.0-rc.4/wasm-inline" + } +} +``` + +```ts +import { Encryption, encryptedTable, types } from '@cipherstash/stack/wasm-inline' +``` + +> **No `--allow-ffi` needed.** The whole point of this entry is that nothing +> native loads. If a Deno process running this entry ever demands an FFI +> permission, something has resolved the native entry instead — check the +> import path before granting anything. + +### Cloudflare Workers / Bun / bundlers — normal install + +```bash +npm install @cipherstash/stack +``` + +```ts +import { Encryption, encryptedTable, types } from '@cipherstash/stack/wasm-inline' +``` + +No `externals`, no `nodeExternals`, no `serverExternalPackages` entry. If a +build config already externalises `@cipherstash/protect-ffi` for the native +entry, that config does not apply here and can be left alone. + +## Credentials + +The edge client takes **all four** `CS_*` values explicitly. There is no +credential discovery: `~/.cipherstash` does not exist in a Worker or an Edge +Function container, and there is no device-code login to fall back on. + +```ts +const client = await Encryption({ + schemas: [users], + config: { + workspaceCrn: Deno.env.get('CS_WORKSPACE_CRN')!, + accessKey: Deno.env.get('CS_CLIENT_ACCESS_KEY')!, + clientId: Deno.env.get('CS_CLIENT_ID')!, + clientKey: Deno.env.get('CS_CLIENT_KEY')!, + }, +}) +``` + +Read them from the platform's environment accessor — `Deno.env.get(...)` on +Deno/Supabase, the `env` binding argument on Workers, `process.env` on Bun. + +### Minting them: `stash env` + +```bash +stash env --name my-app-prod # print the four vars to stdout +stash env --name my-app-prod --write # write .env.production.local (mode 0600) +stash env --name edge-dev --write .env.local +``` + +This creates a fresh ZeroKMS client **and** a CipherStash access key from your +local `stash auth login` session. Things that matter here: + +- **The access key is shown exactly once.** Pipe it straight into the secret + store; it cannot be re-revealed. +- **Stdout is pipe-clean** — only the dotenv block goes to stdout, so + `stash env --name x > prod.env` and pipes into secret-store CLIs are safe. +- Each run mints a **new** credential, and duplicate names are rejected. Use a + distinct `--name` per environment. +- `CS_CLIENT_KEY` and `CS_CLIENT_ACCESS_KEY` are secrets. Never commit them; + put placeholder names in `.env.example` instead. + +### Getting them into the runtime + +```bash +# Supabase — local +supabase functions serve --env-file .env.local my-function + +# Supabase — deployed +stash env --name my-app-prod --write .env.production.local +supabase secrets set --env-file .env.production.local + +# Cloudflare Workers +wrangler secret put CS_CLIENT_KEY # repeat per variable + +# Vercel / other platforms +vercel env add CS_CLIENT_KEY production +``` + +## The Credential-Identity Rule (a silent data footgun) + +> **Every writer of a searchable column must use the same credentials as every +> reader — including `stash encrypt backfill`, seed scripts, and admin tools. +> Rows written under different credentials decrypt correctly but never match a +> query.** + +EQL's searchable-encryption index terms (the `hm`, `op`, `bf` fields in the +stored payload) derive from the **ZeroKMS client key**, not from the workspace +or keyset. Two clients in the same workspace with different `CS_CLIENT_ID` / +`CS_CLIENT_KEY` pairs therefore produce **different terms for the same +plaintext**. + +The consequences are asymmetric, which is what makes this hard to spot: + +- **Decryption still works.** The data key is wrapped through ZeroKMS against + the workspace, so any authorised client in the workspace can decrypt the + row. Round-trip tests pass. +- **Search silently fails.** An equality or match predicate compares the + query term against the stored term. Different client keys, different terms, + no match — and no error. The query returns zero rows exactly as though the + data were absent. + +The classic way to hit it: run `stash encrypt backfill` from a laptop (using +the local device-profile credentials), then query those rows from an Edge +Function using `CS_*` values minted by `stash env`. Every row decrypts. No +search ever matches. + +**What to do:** + +- Mint one credential per *environment*, and use it for **every** process that + touches that environment's data — the app, the backfill, seed scripts, admin + jobs, and one-off scripts alike. +- Before running `stash encrypt backfill` against an environment, export that + environment's `CS_*` values into the shell running it. +- If rows have already been written under the wrong credentials, re-encrypt + them with the correct client: read (decryption still works), then write back + through a client built with the target credentials. + +**Diagnosing it:** if `decrypt` returns the right plaintext but an equality +query on the same row returns nothing, compare the stored term against a +freshly minted one for the same plaintext. Matching plaintext with differing +`hm` values is this bug, not an indexing problem. + +## The Client Surface + +`Encryption` from the WASM entry. **Both entries name the factory +`Encryption`**, so the import path is the only thing that distinguishes them — +which makes a stray `import { Encryption } from '@cipherstash/stack'` easy to +miss and confusing to debug, because the two clients take different config and +different bulk shapes. Check the specifier first whenever an edge client +behaves unexpectedly. + +Every fallible method returns the same `{ data } | { failure }` Result +contract as the native client; unwrap before use. + +```ts +const enc = await client.encrypt('alice@example.com', { table: users, column: users.email }) +if (enc.failure) throw new Error(enc.failure.message) + +const dec = await client.decrypt(enc.data) +if (dec.failure) throw new Error(dec.failure.message) +``` + +Available: `encrypt`, `decrypt`, `isEncrypted`, `encryptQuery`, +`encryptQueryBulk`, `bulkEncrypt`, `bulkDecrypt`, `encryptModel`, +`decryptModel`, `bulkEncryptModels`, `bulkDecryptModels`. + +### How it differs from the native typed client + +| | Native (`@cipherstash/stack`) | WASM (`@cipherstash/stack/wasm-inline`) | +|---|---|---| +| Factory | `Encryption({ schemas })` | `Encryption({ schemas, config })` — same name, different module | +| Schema authoring | `encryptedTable` / `types` from `@cipherstash/stack/v3` | the entry's own re-exports (see below) | +| Config | discovered from env / `~/.cipherstash` | all four `CS_*` passed explicitly | +| Typing | signatures derived from the schema | schema-aware, but not the full typed client | +| `.audit()` | chainable on operations | **not available** | +| `.withLockContext()` | chainable on operations | **not available** — see below | +| `bulkEncrypt` shape | `(plaintexts, { table, column })`, `{ id, plaintext }` envelopes | per-item `{ plaintext, table, column }`, plain index-aligned array | +| Module format | ESM + CJS | **ESM only** | + +**Identity-bound encryption is configured, not chained.** There is no +`.withLockContext()` on this entry. Build an `OidcFederationStrategy` (or +`AccessKeyStrategy` for service-to-service) and pass it as +`config.authStrategy`, so the client is authenticated *as the end user* for +its whole lifetime: + +```ts +import { Encryption, OidcFederationStrategy } from '@cipherstash/stack/wasm-inline' + +// `create` returns a Result — unwrap it. Passing the Result itself as +// `authStrategy` is the easy mistake, and it fails opaquely later. +const strategy = OidcFederationStrategy.create( + workspaceCrn, // 'crn::' + () => getUserJwt(req), // called on every re-federation — Clerk, Supabase Auth, … +) +if (strategy.failure) throw new Error(strategy.failure.error.message) + +const client = await Encryption({ + schemas: [users], + config: { authStrategy: strategy.data, clientId, clientKey }, +}) +``` + +`AccessKeyStrategy.create(workspaceCrn, accessKey)` has the same +Result-returning shape, for service-to-service use with a custom token store. +When you pass an auth strategy, do **not** also pass `config.accessKey` — they +are mutually exclusive and the client rejects the combination. + +Construct a client **per request** when using a user-scoped strategy — a +module-level client would bind whichever user happened to arrive first. + +### The bulk shape differs — don't copy the native form + +```ts +// WASM entry: each entry carries its own table and column. +const out = await client.bulkEncrypt([ + { plaintext: 'a@example.com', table: users, column: users.email }, + { plaintext: 'b@example.com', table: users, column: users.email }, +]) +if (out.failure) throw new Error(out.failure.message) +// out.data is index-aligned; a null/undefined plaintext yields null at that index. +``` + +The model helpers (`encryptModel` / `decryptModel` and their bulk forms) *are* +present on this entry and behave as they do natively — declared columns +encrypted by JS property name, everything else passing through, one ZeroKMS +round trip per call. + +## Schema Modules Do Not Cross Entries + +A schema authored with `@cipherstash/stack/v3` **will not typecheck** +against the WASM entry's `Encryption`, and the reverse fails too: + +``` +Type 'EncryptedTextSearchColumn' is not assignable to type 'AnyEncryptedV3Column'. + Types have separate declarations of a private property 'columnName'. +``` + +The two entries ship independent type bundles, and the column classes carry +private fields — which TypeScript compares **nominally**. The declarations are +identical in shape but not the same declaration, so assignment is rejected in +both directions. + +It works fine at runtime, which is the trap: the tempting fix is +`as never` / `as any` on the schema, which silences a real signal and will +keep silencing it after a genuine schema mismatch appears. + +**Author the schema module against exactly one entry, and use that entry's +client with it.** For a project whose encryption runs on the edge, that means +importing `encryptedTable` and `types` from `@cipherstash/stack/wasm-inline` +in the shared schema module: + +```ts +// schema.ts — the single source of truth for this project's schema +import { encryptedTable, types } from '@cipherstash/stack/wasm-inline' + +export const users = encryptedTable('users', { + email: types.TextSearch('email'), + ssn: types.TextEq('ssn'), +}) +``` + +Node-side code that imports this module must then also build its client from +`@cipherstash/stack/wasm-inline` (which runs on Node perfectly well, just with +the WASM engine rather than the native one) and must be ESM. + +If a project genuinely needs the native client on the server *and* the WASM +client on the edge, keep two schema modules and treat their agreement as +something to test, not something the type system will enforce for you. Column +names and domains must match exactly — they are what the database and the +stored payload's `i` identifier are keyed by. + +## Querying from the Edge + +Edge functions rarely have an ORM, so encrypted search is usually hand-written +SQL over `pg` or `postgres-js`. Mint the search needle with `encryptQuery`, +then bind it as a typed parameter: + +```ts +const term = await client.encryptQuery('alice@example.com', { + table: users, column: users.email, queryType: 'equality', +}) +if (term.failure) throw new Error(term.failure.message) + +// postgres-js — bind the unwrapped term, not the Result +const rows = await sql` + SELECT * FROM users + WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +The predicate forms, the per-driver binding rules, and the query-domain names +are the subject of `stash-sql` — read it before writing the first query. The +two rules that bite immediately: the operand must be **cast to the column's +`eql_v3.query_*` domain**, and on `postgres-js` payloads must be bound with +`sql.json(...)`, never pre-stringified. + +## Troubleshooting + +**`Dynamic require of "..." is not supported` / a native `.node` file in the bundle** +— the native entry got imported. Check every import path resolves to +`@cipherstash/stack/wasm-inline`, including transitive ones from your own +shared modules. + +**`require(...) is not a function` / the specifier won't resolve in CJS** — this +entry is ESM-only. Move the consumer to ESM. + +**Missing `CS_*` at runtime** — the secret store was never populated, or the +function was served without `--env-file`. Validate all four at handler entry +and return an actionable error rather than letting client construction fail +opaquely; the example in `examples/supabase-worker` does exactly this. + +**Encryption works, search returns zero rows** — the credential-identity rule +above. Second most likely: a missing index (see `stash-indexing`) makes it +slow, not empty, so empty results point at credentials or an untyped operand +(`stash-sql`). + +**Search needle rejected** — free-text needles must be at least 3 characters; +shorter ones tokenize to nothing. + +**Cold-start latency** — the inlined WASM module is compiled on first use. +Construct the client at module scope when the auth strategy is not +user-scoped, so it is reused across invocations on a warm isolate. + +## Reference + +- `stash-sql` — the raw-SQL predicate cookbook and driver binding rules. +- `stash-encryption` — schema authoring, the `types.*` domain catalog, and the + rollout/cutover lifecycle. +- `stash-cli` — `stash env`, `stash eql install`, `stash encrypt backfill`. +- `stash-indexing` — indexes on encrypted columns (the DDL is the same + wherever the app runs). +- `stash-supabase` — the PostgREST wrapper, for Supabase apps that are not + writing raw SQL. +- Working example: `examples/supabase-worker` in the `cipherstash/stack` repo. +- Bundling guide: https://cipherstash.com/docs/stack/deploy/bundling diff --git a/skills/stash-encryption/SKILL.md b/skills/stash-encryption/SKILL.md index 31f93990..cd0d940d 100644 --- a/skills/stash-encryption/SKILL.md +++ b/skills/stash-encryption/SKILL.md @@ -190,7 +190,7 @@ The SDK never logs plaintext data. | `@cipherstash/stack/types` | All TypeScript types | | `@cipherstash/stack-drizzle` | Drizzle ORM integration for EQL v3 schemas — the package root, EQL v3 only (see the `stash-drizzle` skill) | | `@cipherstash/stack-supabase` | `encryptedSupabase` wrapper for Supabase — EQL v3 only (see the `stash-supabase` skill) | -| `@cipherstash/stack/wasm-inline` | The **edge** entry — Deno, Bun, Cloudflare Workers, Supabase Edge Functions. Its own `Encryption` factory plus the v3 authoring surface, `EncryptionErrorTypes`, and the WASM build of protect-ffi inlined into the bundle. No native binding, so no bundler externalisation needed. | +| `@cipherstash/stack/wasm-inline` | The **edge** entry — Deno, Bun, Cloudflare Workers, Supabase Edge Functions. Its own `Encryption` factory plus its own copy of the v3 authoring surface, `EncryptionErrorTypes`, and the WASM build of protect-ffi inlined into the bundle. No native binding, so no bundler externalisation needed. **ESM-only, and its schema types do not interchange with the other entries'** — see the `stash-edge` skill. | | `@cipherstash/stack/dynamodb` | `encryptedDynamoDB` — encrypt/write is **EQL v3 only** (`types.*`); decrypt still reads existing v2 items. See the `stash-dynamodb` skill | | `@cipherstash/stack/schema`, `@cipherstash/stack/client`, `@cipherstash/stack/encryption` | Legacy v2 schema builders and client surface — see "Legacy: EQL v2" below | @@ -431,6 +431,9 @@ for (const item of decrypted.data) { > [!IMPORTANT] > The `client` below is a **different client** from the one used everywhere else in this skill. The edge entry has its own `Encryption` factory — the native `Encryption` client's `bulkEncrypt` takes `(plaintexts, { table, column })` and will fail at runtime if given the per-item shape below. Construct the WASM client explicitly: +> [!IMPORTANT] +> **The schema is not shareable between entries either.** Note that `encryptedTable` and `types` are imported *from the WASM entry* below, not from `@cipherstash/stack/eql/v3`. The entries ship independent type bundles whose column classes carry private fields, so TypeScript compares them **nominally**: a schema authored on one entry is rejected by the other's client, in both directions (`Types have separate declarations of a private property 'columnName'`). It works at runtime, which makes `as any` the tempting fix — don't. Author the shared schema module against exactly one entry and build that entry's client from it. See the `stash-edge` skill. + ```typescript // Deno / Workers / Supabase Edge Functions — note the import path import { Encryption, encryptedTable, types } from "@cipherstash/stack/wasm-inline" @@ -567,7 +570,7 @@ All values in the array must be non-null. ### On the Wire: Operators and Ordering -Scalar filters compare through each domain's `eql_v3.*` operators (`col = term`, `col > term`, ...), and `ORDER BY` on an encrypted column goes through the ordering extractors — `eql_v3.ord_term(col)` for OPE-backed (`Ord`/`Search`) domains, `eql_v3.ord_term_ore(col)` for `OrdOre`. The Drizzle v3 integration emits all of this for you (including `asc`/`desc`, which emit `ORDER BY eql_v3.ord_term(col)`). Over Supabase/PostgREST, the adapter's `order()` works on OPE-backed ordering columns (plain `*_ord`, `text_ord`, `text_search`) by sorting on the column's `col->op` term; `OrdOre`-flavour (`*_ord_ore`) domains and columns with no ordering term are rejected. See the `stash-drizzle` and `stash-supabase` skills. These same extractor expressions are also what you index — the functional-index recipes are in the `stash-indexing` skill. +Scalar filters compare through each domain's `eql_v3.*` operators (`col = term`, `col > term`, ...), and `ORDER BY` on an encrypted column goes through the ordering extractors — `eql_v3.ord_term(col)` for OPE-backed (`Ord`/`Search`) domains, `eql_v3.ord_term_ore(col)` for `OrdOre`. The Drizzle v3 integration emits all of this for you (including `asc`/`desc`, which emit `ORDER BY eql_v3.ord_term(col)`). Over Supabase/PostgREST, the adapter's `order()` works on OPE-backed ordering columns (plain `*_ord`, `text_ord`, `text_search`) by sorting on the column's `col->op` term; `OrdOre`-flavour (`*_ord_ore`) domains and columns with no ordering term are rejected. See the `stash-drizzle` and `stash-supabase` skills. These same extractor expressions are also what you index — the functional-index recipes are in the `stash-indexing` skill. Writing the SQL by hand instead (no ORM, `pg` / `postgres-js`)? The predicate matrix, the `eql_v3.query_*` operand casts, and the per-driver parameter-binding rules are in the `stash-sql` skill; running on Deno / Workers / Supabase Edge Functions, the `stash-edge` skill. ## Encrypted JSON (`types.Json`) diff --git a/skills/stash-indexing/SKILL.md b/skills/stash-indexing/SKILL.md index 5e7b46cc..8ed63bf3 100644 --- a/skills/stash-indexing/SKILL.md +++ b/skills/stash-indexing/SKILL.md @@ -257,7 +257,7 @@ Index not being used: - **Drizzle** — `encryptedIndexes(t)` from `@cipherstash/stack-drizzle` derives the recommended indexes for every encrypted column in the table, or declare individual expression indexes in the schema DSL. See `stash-drizzle` § Indexing Encrypted Columns. - **Prisma Next** — Prisma's schema language cannot express functional indexes; the DDL goes in a migration in the adapter's flow. See `stash-prisma-next`. - **Supabase** — a `supabase/migrations/` file; no superuser needed (see above). See `stash-supabase`. -- **Raw SQL / plain PostgreSQL** — the recipes in this skill, in whatever migration tool owns the schema. Never ad-hoc in production. +- **Raw SQL / plain PostgreSQL** — the recipes in this skill, in whatever migration tool owns the schema. Never ad-hoc in production. The predicates those indexes serve are in `stash-sql`. ## When to Create Indexes During an Encryption Rollout @@ -271,3 +271,5 @@ Index not being used: - `stash-encryption` — the `types.*` domain catalog, wire-format operators and ordering, and the rollout/cutover lifecycle. - `stash-cli` — `stash eql install`, `stash db validate` (its "No indexes on an encrypted column" Info finding is resolved by this skill), `stash encrypt backfill` / `cutover`. - `stash-drizzle`, `stash-supabase`, `stash-prisma-next` — per-integration query patterns; index DDL placement per the section above. +- `stash-sql` — the hand-written predicate forms these indexes serve (`pg` / `postgres-js`, no ORM). +- `stash-edge` — the WASM entry, for apps whose queries run on Deno / Workers / Supabase Edge Functions. diff --git a/skills/stash-prisma-next/SKILL.md b/skills/stash-prisma-next/SKILL.md index 632af947..11e8b448 100644 --- a/skills/stash-prisma-next/SKILL.md +++ b/skills/stash-prisma-next/SKILL.md @@ -178,7 +178,10 @@ The `ANALYZE` is part of the recipe — an expression index has no statistics until it runs. Works as a non-superuser role (Supabase included); only the ORE-flavour (`_ord_ore`) ordering opclass is superuser-gated. For the full model — which domains take which index, engagement rules, `EXPLAIN` -verification, rollout timing — see the `stash-indexing` skill. +verification, rollout timing — see the `stash-indexing` skill. For encrypted +predicates written as raw SQL rather than through the `cipherstash:*` +operators — operand casts to `eql_v3.query_*`, per-driver parameter binding — +see the `stash-sql` skill. In a migration, the recipes ride a raw-SQL operation (`rawSql` from `@prisma-next/postgres/migration`) in the migration's `operations`: diff --git a/skills/stash-sql/SKILL.md b/skills/stash-sql/SKILL.md new file mode 100644 index 00000000..3846b3d5 --- /dev/null +++ b/skills/stash-sql/SKILL.md @@ -0,0 +1,365 @@ +--- +name: stash-sql +description: Query EQL v3 encrypted columns from hand-written SQL over `pg` (node-postgres) or `postgres` (postgres-js) — no ORM. Covers the column-domain-to-query-domain operator matrix (which of `=`, `<>`, `<`, `>=`, `@@`, `@>` each encrypted domain accepts), minting search needles with `encryptQuery`, the per-driver parameter-binding rules for encrypted payloads, and the double-encoding failure that trips the domain CHECK with a message naming neither JSON nor encoding. Use when writing INSERT/SELECT against an encrypted column without an ORM, when a predicate returns zero rows or raises "operator does not exist", or when a domain CHECK constraint rejects an encrypted value on write. +--- + +# Raw SQL Against Encrypted Columns (EQL v3) + +An EQL v3 encrypted column is a **Postgres domain over `jsonb`** (`public.eql_v3_text_search`, +`public.eql_v3_bigint_ord`, …). Reading and writing it from raw SQL is two rules: + +1. **Writing** — bind the `Encrypted` payload your client produced as a + *JSON object* parameter. How you do that differs per driver, and getting it + wrong trips a domain CHECK with an unhelpful message. +2. **Querying** — never send a plaintext. Mint a **query term** with + `encryptQuery` and cast it to the column's matching `eql_v3.query_*` + domain. That cast is what selects the right operator overload: leave the + operand as bare `jsonb` and you get a *different* overload, one that + expects a full storage envelope. + +This covers the `pg` and `postgres-js` drivers with no ORM — plain Node +services, Hono, edge functions. If you use Drizzle, Prisma Next, or the +Supabase client, those integrations emit correct operands for you: see +`stash-drizzle`, `stash-prisma-next`, `stash-supabase` instead. + +## When to Use This Skill + +- Writing `INSERT` / `UPDATE` / `SELECT` against an encrypted column by hand. +- A predicate returns zero rows, or errors with `operator does not exist`. +- A write fails with `value for domain eql_v3_… violates check constraint`. +- Choosing the right operator for a column's domain. +- Ordering, ranging, or searching inside an encrypted JSON document. + +## The Two Halves + +Assume `users.email` is a `types.TextEq` column — the domain names below +follow the column, so a `types.TextSearch` column would use +`eql_v3.query_text_search` in exactly the same places. + +```ts +// 1. Encrypt for storage — the payload includes the ciphertext (`c`). +const enc = await client.encrypt('alice@example.com', { table: users, column: users.email }) +if (enc.failure) throw new Error(enc.failure.message) +await sql`INSERT INTO users (email) VALUES (${sql.json(enc.data)})` + +// 2. Mint a search needle — CIPHERTEXT-FREE, terms only. +const term = await client.encryptQuery('alice@example.com', { + table: users, column: users.email, queryType: 'equality', +}) +if (term.failure) throw new Error(term.failure.message) +const rows = await sql` + SELECT * FROM users WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +Storage payloads and query terms are **different shapes with different +domains**. A storage payload carries `c` (the ciphertext); a query term +deliberately omits it and the `eql_v3.query_*` CHECKs *require* its absence. +Binding a storage payload where a query term belongs fails the CHECK, and vice +versa. + +`queryType` is one of `'equality'`, `'freeTextSearch'`, `'orderAndRange'`, +`'searchableJson'`. Omit it only for single-index columns (`types.TextEq`); +be explicit on multi-index domains like `types.TextSearch`. + +## Naming: Column Domain → Query Domain + +Strip `public.`, insert `query_`, move to the `eql_v3` schema: + +``` +public.eql_v3_text_eq → eql_v3.query_text_eq +public.eql_v3_text_search → eql_v3.query_text_search +public.eql_v3_bigint_ord → eql_v3.query_bigint_ord +public.eql_v3_timestamp_ord → eql_v3.query_timestamp_ord +``` + +**One irregular case:** `types.Json` builds `public.eql_v3_json_search`, but +its query domain is `eql_v3.query_json` — not `query_json_search`. + +| Schema factory | Column domain (`public.`) | Query domain (`eql_v3.`) | +|---|---|---| +| `types.TextEq` | `eql_v3_text_eq` | `query_text_eq` | +| `types.TextMatch` | `eql_v3_text_match` | `query_text_match` | +| `types.TextOrd` | `eql_v3_text_ord` | `query_text_ord` | +| `types.TextSearch` | `eql_v3_text_search` | `query_text_search` | +| `types.Eq` | `eql_v3__eq` | `query__eq` | +| `types.Ord` | `eql_v3__ord` | `query__ord` | +| `types.OrdOre` | `eql_v3__ord_ore` | `query__ord_ore` | +| `types.Json` | `eql_v3_json_search` | **`query_json`** | +| `types.Text`, `types.`, `types.Boolean` | `eql_v3_text` / `eql_v3_` / `eql_v3_boolean` | **none — storage only** | + +`` ranges over `Integer`, `Smallint`, `Bigint`, `Numeric`, `Real`, +`Double`, `Date`, `Timestamp`. The storage-only domains carry no query terms +by design — there is no query domain and nothing to search server-side. + +## The Predicate Matrix + +Which operators each column domain accepts against its query domain. Anything +not listed does not exist as an encrypted operator. + +| Column domain | Operators | Query domain operand | +|---|---|---| +| `eql_v3__eq`, `eql_v3_text_eq` | `=` `<>` | `query__eq` / `query_text_eq` | +| `eql_v3__ord`, `eql_v3_text_ord` | `=` `<>` `<` `<=` `>` `>=` | `query__ord` / `query_text_ord` | +| `eql_v3__ord_ore`, `eql_v3_text_ord_ore` | `=` `<>` `<` `<=` `>` `>=` | `query__ord_ore` / `query_text_ord_ore` | +| `eql_v3_text_match` | `@@` | `query_text_match` | +| `eql_v3_text_search` | `=` `<>` `<` `<=` `>` `>=` `@@` | `query_text_search` | +| `eql_v3_json_search` | `@>` | `query_json` | +| `eql_v3_json_entry` (from `col -> 'selector'`) | `=` `<>` `<` `<=` `>` `>=` | any `query__ord` / `query_text_ord` / `query_text_search` | + +Note what is **absent**: there is no `<` on an `_eq` domain, and no `@@` +outside the match-capable text domains. Asking for one raises +`operator does not exist` — which is the good failure. The bad failure is +leaving the operand as bare `jsonb` (see [Traps](#traps)). + +Every operator has a function twin, useful when an operator is awkward to +emit: `eql_v3.eq(col, term)`, `eql_v3.matches(col, term)`, and the comparison +functions. `col = term` and `eql_v3.eq(col, term)` are equivalent. + +## Binding Parameters: The Driver Rules + +**This differs between drivers.** Both encrypted payloads and query terms are +plain JS objects, and the two drivers disagree about how to put a JS object +into a `jsonb`-backed domain. + +### `postgres` (postgres-js) — always `sql.json(...)` + +| Binding form | `INSERT` into a domain column | Query operand with `::jsonb::eql_v3.query_*` | +|---|---|---| +| `${sql.json(payload)}` | ✅ | ✅ | +| `${payload}` (bare object) | ❌ `invalid input syntax for type json` | ✅ | +| `${JSON.stringify(payload)}::jsonb` | ❌ CHECK violation | ❌ CHECK violation | + +**Use `sql.json(...)` in both positions** — it is the only form that works in +both, so there is no reason to track which position you are in. + +```ts +await sql`INSERT INTO users (email) VALUES (${sql.json(enc.data)})` +await sql`SELECT * FROM users + WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +### `pg` (node-postgres) — pass the object + +node-postgres serialises a JS object to JSON exactly once, so all three forms +happen to work. Pass the object and let the driver do it: + +```ts +await client.query('INSERT INTO users (email) VALUES ($1)', [enc.data]) +await client.query( + 'SELECT * FROM users WHERE email = $1::eql_v3.query_text_eq', [term.data]) +``` + +Do not pre-stringify even though `pg` tolerates it — it is the one habit that +silently breaks if the project ever moves to `postgres-js`. + +### The double-encoding failure, precisely + +`${JSON.stringify(payload)}::jsonb` on postgres-js produces: + +``` +value for domain eql_v3_text_search violates check constraint "eql_v3_text_search_check" +``` + +The message names neither JSON nor encoding, which is why this one costs an +afternoon. What happened: the explicit `::jsonb` makes postgres-js infer a +`jsonb` parameter, so it JSON-encodes the value — which was *already* a JSON +string. The result is a jsonb **string scalar**, not an object: + +```sql +SELECT jsonb_typeof($1::jsonb) -- 'string', not 'object' +``` + +Every EQL domain CHECK opens with `jsonb_typeof(VALUE) = 'object'`, so it +fails on the very first clause. Diagnose any CHECK-violation-on-write by +running `jsonb_typeof` on the parameter; `'string'` means double-encoded. + +## Query Recipes + +Assume `sql` is a postgres-js tag; for `pg` use numbered placeholders as above. + +### Equality + +```ts +const term = await client.encryptQuery(email, { + table: users, column: users.email, queryType: 'equality', +}) +await sql`SELECT * FROM users + WHERE email = ${sql.json(term.data)}::jsonb::eql_v3.query_text_eq` +``` + +On a `types.TextSearch` column the cast is `::eql_v3.query_text_search` — the +query domain always matches the *column's* domain, not the query type. + +### Free-text match + +```ts +// `bio` is a types.TextSearch column here; on a types.TextMatch column the +// cast is ::eql_v3.query_text_match. +const term = await client.encryptQuery('needle', { + table: users, column: users.bio, queryType: 'freeTextSearch', +}) +await sql`SELECT * FROM users + WHERE bio @@ ${sql.json(term.data)}::jsonb::eql_v3.query_text_search` +``` + +Match is **one-sided**: a hit may be a false positive, a miss never is. Filter +client-side after decryption if exactness matters — and never build a negated +match (`NOT (bio @@ …)`), which would drop true rows. Needles must be at least +3 characters; shorter ones tokenize to nothing and are rejected. + +### Range and ordering + +```ts +const term = await client.encryptQuery(new Date('2026-01-01'), { + table: events, column: events.createdAt, queryType: 'orderAndRange', +}) +await sql`SELECT * FROM events + WHERE created_at >= ${sql.json(term.data)}::jsonb::eql_v3.query_timestamp_ord + ORDER BY eql_v3.ord_term(created_at) DESC + LIMIT 20` +``` + +**`ORDER BY` must use the extractor form.** `ORDER BY created_at` sorts the +raw encrypted payload — which is neither meaningful nor index-backed. Sorting +on `eql_v3.ord_term(col)` is both. Ordering is available on `_ord`, +`_ord_ore`, and `text_search` columns; use `ord_term_ore` for `_ord_ore`. + +### Encrypted JSON — containment + +```ts +const needle = await client.encryptQuery({ role: 'admin' }, { + table: users, column: users.prefs, queryType: 'searchableJson', +}) +await sql`SELECT * FROM users + WHERE prefs @> ${sql.json(needle.data)}::jsonb::eql_v3.query_json` +``` + +An **object** value produces a containment needle. Note the containment needle +is a bare `{ sv: [...] }` shape with no version field — unlike the scalar +terms, which are full v3 envelopes. Bind it the same way regardless. + +### Encrypted JSON — field selector + +A **string** value produces a JSONPath selector, and v3 has no encrypted-selector +envelope: `encryptQuery` returns the **bare selector-hash string**. Bind it as +the plain text argument of `->` / `->>`, with no domain cast: + +```ts +const sel = await client.encryptQuery('$.role', { + table: users, column: users.prefs, queryType: 'searchableJson', +}) +await sql`SELECT prefs -> ${sel.data} FROM users` +``` + +The extracted value is an `eql_v3_json_entry`, which accepts the ordering +operators — so a field inside an encrypted document can be ranged and ordered: + +```ts +await sql`SELECT * FROM orders + WHERE data -> ${sel.data} >= ${sql.json(term.data)}::jsonb::eql_v3.query_integer_ord + ORDER BY eql_v3.ord_term(data -> ${sel.data})` +``` + +Field-level `=` between extracted entries is **not** supported (an extracted +entry carries no value selector) — use document containment for exact field +equality. + +## Reading Rows Back + +`SELECT` returns the stored payload as an object; hand it straight to +`decrypt` — do not `JSON.parse` it, and do not cast it to `::jsonb` in the +query (see the projection trap below). + +```ts +const [row] = await sql`SELECT id, email FROM users WHERE id = ${id}` +const dec = await client.decrypt(row.email) +if (dec.failure) throw new Error(dec.failure.message) +``` + +For whole rows, `decryptModel` / `bulkDecryptModels` walk the schema and +decrypt every declared column in one ZeroKMS round trip. They match by **JS +property name**, so a raw `SELECT` returning snake_case DB column names will +not match a schema keyed by camelCase properties — alias in the query +(`SELECT last_login AS "lastLogin"`) or decrypt the columns individually. + +## Traps + +**A bare `::jsonb` operand picks a different operator, not a missing one.** +EQL also defines overloads with `jsonb` on the right — and they coerce that +operand to the **storage** domain: + +```sql +-- what `col = $1::jsonb` actually resolves to: +eql_v3.eq_term(a) = eql_v3.eq_term(b::public.eql_v3_text_search) +``` + +The storage domain's CHECK requires the ciphertext key `c`, which query terms +deliberately omit — so binding a query term without the domain cast raises a +CHECK violation rather than doing what you meant. Those overloads exist so you +can compare against a *full storage envelope* (an already-encrypted value). +For a needle from `encryptQuery`, always cast to the `eql_v3.query_*` domain. + +**Cast to the `query_*` domain, not the column domain.** `$1::public.eql_v3_text_eq` +fails for the same reason — the column domain's CHECK requires `c`. + +**The `value::jsonb` projection trap.** `SELECT email::jsonb … ORDER BY email` +folds the cast into the scan and sorts on `(email)::jsonb`, matching no index. +Project the column raw. + +**`GROUP BY` / `DISTINCT` on the raw column** hashes the whole encrypted +payload (1–2 KB per row) and spills. Group on the extractor — +`GROUP BY eql_v3.eq_term(email)` — which is small and deterministic. + +**Predicates are not indexes.** Everything here works without an index and +sequential-scans. Adding the functional index over the extractor is a separate +step — see `stash-indexing`. + +**Every writer needs the same credentials.** Index terms derive from the +ZeroKMS client key, so rows written by a client with different `CS_*` +credentials decrypt correctly but never match a query — silently. This +includes `stash encrypt backfill` and seed scripts. See `stash-edge` § +The Credential-Identity Rule. + +## Troubleshooting + +**`operator does not exist: public.eql_v3_… = eql_v3.query_…`** — the domain +pair has no such operator. Check the [matrix](#the-predicate-matrix): the +column's domain may not support that predicate (e.g. `<` on an `_eq` column), +or the query domain does not match the column's domain. + +**`value for domain … violates check constraint`** on write — double-encoded +payload; run `SELECT jsonb_typeof($1::jsonb)` and see +[above](#the-double-encoding-failure-precisely). On a *query* operand, the +same error usually means a storage payload (with `c`) was bound where a query +term belongs. + +**Zero rows, no error** — in order of likelihood: (1) the rows were written +under different `CS_*` credentials (see the trap above — this is the common +one, and it is completely silent); (2) the column's domain does not carry the +term the predicate needs (a `types.Text` column carries none); (3) a +free-text needle under 3 characters tokenized to nothing. A *missing* domain +cast raises an error rather than returning zero rows, so it is not a candidate +here. + +**Slow but correct** — no index. See `stash-indexing`; confirm with +`EXPLAIN (COSTS OFF)` that the plan shows an `Index Cond` on the extractor +rather than a `Seq Scan`. + +**Checking what a column actually is**, when the schema and the database may +have drifted: + +```sql +SELECT column_name, domain_schema, domain_name + FROM information_schema.columns + WHERE table_name = 'users' AND domain_name LIKE 'eql_v3%'; +``` + +## Reference + +- `stash-encryption` — schema authoring, the `types.*` catalog, `encryptQuery` + and the client API, the rollout/cutover lifecycle. +- `stash-indexing` — functional indexes over the term extractors, and the + `EXPLAIN` checklist. +- `stash-edge` — the WASM entry, `CS_*` credentials, and the + credential-identity rule. +- `stash-cli` — `stash eql install`, `stash db validate`, `stash encrypt backfill`. diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index fdbb2d62..c1041fe1 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -56,6 +56,14 @@ this is also how **Supabase Edge Functions** get credentials in local dev — `stash env --name edge-dev --write` and pass `--env-file`, or `supabase secrets set` them for deploys. +> **One credential per environment, used by everything that writes.** EQL index +> terms derive from the ZeroKMS client key, so rows written by a client with +> different `CS_*` values decrypt correctly but never match a query — silently. +> That includes `stash encrypt backfill`, seed scripts, and Edge Functions. +> Encryption *inside* an Edge Function (Deno, no native modules) uses the +> `@cipherstash/stack/wasm-inline` entry — see the `stash-edge` skill; SQL +> written by hand in a migration or RPC is covered by `stash-sql`. + ### 1. Install EQL v3 on the database ```bash