[4/6] feat(eql-mapper): rewrite SQL to the EQL v3 functional-index form

CHANGELOG.md

@@ -6,6 +6,18 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Changed
+
+- **EQL v3 (searchable encryption)**: Proxy now targets EQL v3. Encrypted columns are declared with self-configuring, typed `jsonb` domains (for example `eql_v3_text_search`, `eql_v3_integer_ord`, `eql_v3_json_search`) that encode both the scalar type and the column's searchable capabilities in the column type itself, replacing EQL v2's opaque `eql_v2_encrypted` composite type and its separate `eql_v2_configuration` table. The bundled `cipherstash-client` is upgraded to 0.42.0 and EQL to 3.0.2. Existing v2-encrypted data and schemas must be migrated to v3.
+
+### Added
+
+- **Encrypted full-text match with `@@`**: The `@@` operator is now supported on encrypted text columns whose domain carries a match (bloom-filter) term, rewritten to the EQL v3 `eql_v3.match_term` form.
+
+### Fixed
+
+- **`LIKE`/`ILIKE` capability checking**: `LIKE` and `ILIKE` on an encrypted column are now gated by the column's token-match capability. Previously these predicates bypassed capability checking and were silently accepted on columns that do not support fuzzy match; they are now rejected with a capability error.
+
 ## [2.2.4] - 2026-06-18
 
 ### Fixed

packages/cipherstash-proxy/src/postgresql/frontend.rs

@@ -27,7 +27,7 @@ use crate::prometheus::{
     STATEMENTS_PASSTHROUGH_TOTAL, STATEMENTS_UNMAPPABLE_TOTAL,
 };
 use crate::proxy::EncryptionService;
-use crate::EqlOutput;
+use crate::{EqlOutput, EqlQueryPayload};
 use bytes::BytesMut;
 use cipherstash_client::encryption::Plaintext;
 use eql_mapper::{self, EqlMapperError, EqlTerm, TypeCheckedStatement};
@@ -649,6 +649,14 @@ where
         let mut encrypted_expressions = vec![];
         for encrypted in encrypted_literals {
             let e = match encrypted {
+                // A JSON selector (RHS of `->`/`->>`, or the `jsonb_path_query`
+                // path) is a bare tokenized-selector hash used directly as `text`
+                // by the eql_v3 functions (`eql_v3."->"(json, text)`). Bind the raw
+                // token: JSON-serializing it (below) would re-quote the bare string
+                // (`"<hash>"`), so it would never match the stored per-entry `s`.
+                Some(EqlOutput::Query(EqlQueryPayload::Selector(s))) => {
+                    Some(Value::SingleQuotedString(s.clone()))
+                }
                 Some(en) => Some(to_json_literal_value(&en)?),
                 None => None,
             };

packages/cipherstash-proxy/src/postgresql/messages/bind.rs

@@ -5,7 +5,7 @@ use crate::postgresql::context::column::Column;
 use crate::postgresql::data::bind_param_from_sql;
 use crate::postgresql::format_code::FormatCode;
 use crate::postgresql::protocol::BytesMutReadString;
-use crate::EqlOutput;
+use crate::{EqlOutput, EqlQueryPayload};
 use crate::{SIZE_I16, SIZE_I32};
 use bytes::{Buf, BufMut, BytesMut};
 use cipherstash_client::encryption::Plaintext;
@@ -84,12 +84,24 @@ impl Bind {
     pub fn rewrite(&mut self, encrypted: Vec<Option<EqlOutput>>) -> Result<(), Error> {
         for (idx, ct) in encrypted.iter().enumerate() {
             if let Some(ct) = ct {
-                let json = serde_json::to_value(ct)?;
-
-                // convert json to bytes
-                let bytes = json.to_string().into_bytes();
-
-                self.param_values[idx].rewrite(&bytes);
+                match ct {
+                    // A JSON selector (`->`/`->>`/`jsonb_path_query`) is a bare
+                    // tokenized-selector hash bound directly as `text`, NOT jsonb.
+                    // Use the raw token: JSON-serializing it re-quotes the bare
+                    // string (`"<hash>"`), which never matches the stored per-entry
+                    // `s`. It must also skip the jsonb version header a binary
+                    // rewrite would prepend — the binary wire form of `text` is
+                    // just its raw bytes, and a leading `0x01` corrupts the
+                    // selector so `->` matches nothing.
+                    EqlOutput::Query(EqlQueryPayload::Selector(s)) => {
+                        self.param_values[idx].rewrite_text(s.clone().into_bytes());
+                    }
+                    // convert json to bytes
+                    _ => {
+                        let bytes = serde_json::to_value(ct)?.to_string().into_bytes();
+                        self.param_values[idx].rewrite(&bytes);
+                    }
+                }
             }
         }
         Ok(())
@@ -156,6 +168,20 @@ impl BindParam {
         self.dirty = true;
     }
 
+    /// Rewrite this param as a bare `text` value, without the jsonb version
+    /// header [`rewrite`] prepends for binary jsonb payloads.
+    ///
+    /// Used for the tokenized selector of a JSON field access (`->`/`->>`/
+    /// `jsonb_path_query`), which is bound as `text`, not jsonb. The binary
+    /// wire form of `text` is simply its raw UTF-8 bytes, so no header is
+    /// added in either format — a stray `0x01` would corrupt the selector and
+    /// stop `->` from matching any stored entry.
+    pub fn rewrite_text(&mut self, bytes: Vec<u8>) {
+        self.bytes.clear();
+        self.bytes.extend_from_slice(&bytes);
+        self.dirty = true;
+    }
+
     pub fn requires_rewrite(&self) -> bool {
         self.dirty
     }

packages/cipherstash-proxy/src/postgresql/messages/data_row.rs

@@ -35,7 +35,7 @@ impl DataRow {
                 .filter(|_| data_column.is_not_null())
                 .and_then(|config| {
                     data_column
-                        .try_into()
+                        .to_eql_ciphertext()
                         .inspect_err(|err| match err {
                             Error::Encrypt(EncryptError::ColumnIsNull) => {
                                 debug!(target: DECRYPT, msg ="ColumnIsNull", ?config);
@@ -179,24 +179,36 @@ impl TryFrom<DataColumn> for BytesMut {
     }
 }
 
-impl TryFrom<&mut DataColumn> for EqlCiphertext {
-    type Error = Error;
-
-    fn try_from(col: &mut DataColumn) -> Result<Self, Error> {
-        // EQL v3 column types (`eql_v3_text_eq`, `eql_v3_integer_ord`, …) are
-        // DOMAINS over `jsonb`, so a value arrives with jsonb's representation.
-        //
-        // EQL v2's `eql_v2_encrypted` was a composite type, which is why this
-        // used to strip a `("…")` wrapper in text and a 12-byte rowtype header
-        // in binary. Neither exists any more — a domain is wire-identical to its
-        // base type.
-        //
-        //   text   — the JSON object itself, no wrapper and no doubled quotes
-        //   binary — a 1-byte jsonb version header followed by the JSON text
-        //
-        // The two are told apart by the leading byte: the version header is
-        // `0x01`, and JSON text for an EQL payload always starts with `{`.
-        let Some(bytes) = &col.bytes else {
+impl DataColumn {
+    /// Parse this column's bytes into an [`EqlCiphertext`].
+    ///
+    /// EQL v3 column types (`eql_v3_text_eq`, `eql_v3_integer_ord`, …) are
+    /// DOMAINS over `jsonb`, so a value arrives with jsonb's representation.
+    ///
+    /// EQL v2's `eql_v2_encrypted` was a composite type, which is why this
+    /// used to strip a `("…")` wrapper in text and a 12-byte rowtype header
+    /// in binary. Neither exists any more — a domain is wire-identical to its
+    /// base type.
+    ///
+    ///   text   — the JSON object itself, no wrapper and no doubled quotes
+    ///   binary — a 1-byte jsonb version header followed by the JSON text
+    ///
+    /// The two are told apart by the leading byte: the version header is
+    /// `0x01`, and JSON text for an EQL payload always starts with `{`.
+    ///
+    /// The JSON is usually a self-describing payload — a scalar `{v,i,c,…}` or
+    /// a SteVec document `{v,k:"sv",i,h,sv}` — and deserialises directly. The
+    /// exception is a JSON field access (`eql_v3."->"(…)` /
+    /// `eql_v3.jsonb_path_query(…)`), whose result is a single
+    /// `eql_v3_json_entry` (`{v,i,h,s,c,op}`) — one SteVec entry merged with
+    /// its document envelope. That has a `c`, so it would masquerade as a
+    /// scalar `Encrypted` payload, but its `c` is an *entry* ciphertext that
+    /// only decrypts with the entry's selector-derived nonce. So when the
+    /// payload is a bare entry (see [`is_json_entry`]) it is reshaped into a
+    /// one-entry SteVec document (see [`json_entry_into_ste_vec_document`]) and
+    /// the ordinary SteVec decrypt path recovers the field value.
+    fn to_eql_ciphertext(&self) -> Result<EqlCiphertext, Error> {
+        let Some(bytes) = &self.bytes else {
             return Err(EncryptError::ColumnCouldNotBeParsed.into());
         };
 
@@ -206,11 +218,58 @@ impl TryFrom<&mut DataColumn> for EqlCiphertext {
             None => return Err(EncryptError::ColumnCouldNotBeParsed.into()),
         };
 
-        serde_json::from_slice(json).map_err(|err| {
-            debug!(target: DECRYPT, error = err.to_string());
-            err.into()
-        })
+        let mut value: serde_json::Value =
+            serde_json::from_slice(json).map_err(log_deserialise_error)?;
+
+        if is_json_entry(&value) {
+            json_entry_into_ste_vec_document(&mut value)?;
+        }
+
+        serde_json::from_value(value).map_err(log_deserialise_error)
+    }
+}
+
+/// Whether a decoded EQL payload is a bare `eql_v3_json_entry` — the result of
+/// a JSON field access (`eql_v3."->"(…)` / `eql_v3.jsonb_path_query(…)`).
+///
+/// A root-level selector `s` is the tell: a scalar `Encrypted` payload has no
+/// selector at all, and a SteVec document carries selectors only inside its
+/// `sv[]` entries, never at the root.
+fn is_json_entry(value: &serde_json::Value) -> bool {
+    value.get("s").is_some()
+}
+
+/// Reshape a single `eql_v3_json_entry` into a one-entry SteVec document.
+///
+/// The entry is `{v,i,h,s,c,op}`: document-envelope fields (`v`, `i`, `h`)
+/// alongside one SteVec entry's fields (`s`, `c`, the optional array marker
+/// `a`, and the optional ordering term `op`). Move the entry fields under
+/// `sv:[{…}]` and tag the object as a SteVec (`k:"sv"`), yielding
+/// `{v,k:"sv",i,h,sv:[{s,c,a?,op?}]}` — the shape an [`EqlCiphertext`] SteVec
+/// document deserialises from and the decrypt path knows how to open.
+fn json_entry_into_ste_vec_document(value: &mut serde_json::Value) -> Result<(), Error> {
+    use serde_json::Value;
+
+    let object = value
+        .as_object_mut()
+        .ok_or(EncryptError::ColumnCouldNotBeParsed)?;
+
+    let mut entry = serde_json::Map::new();
+    for key in ["s", "c", "a", "op"] {
+        if let Some(field) = object.remove(key) {
+            entry.insert(key.to_owned(), field);
+        }
     }
+
+    object.insert("k".to_owned(), Value::String("sv".to_owned()));
+    object.insert("sv".to_owned(), Value::Array(vec![Value::Object(entry)]));
+
+    Ok(())
+}
+
+fn log_deserialise_error(err: serde_json::Error) -> Error {
+    debug!(target: DECRYPT, error = err.to_string());
+    err.into()
 }
 
 #[cfg(test)]

packages/eql-mapper-macros/src/parse_type_decl.rs

@@ -169,7 +169,7 @@ impl Parse for EqlTrait {
         Err(syn::Error::new(
             input.span(),
             format!(
-                "Expected Eq, Ord, TokenMatch or JsonLike while parsing EqlTrait; got: {}",
+                "Expected Eq, Ord, TokenMatch, JsonLike or Contain while parsing EqlTrait; got: {}",
                 input.cursor().token_stream()
             ),
         ))
@@ -477,6 +477,11 @@ impl Parse for SqltkBinOp {
 
         if input.peek(token::At) {
             let _: token::At = input.parse()?;
+            // `@@` (fuzzy match) or `@>` (containment).
+            if input.peek(token::At) {
+                let _: token::At = input.parse()?;
+                return Ok(Self(quote!(::sqltk::parser::ast::BinaryOperator::AtAt)));
+            }
             let _: token::Gt = input.parse()?;
             return Ok(Self(quote!(::sqltk::parser::ast::BinaryOperator::AtArrow)));
         }
@@ -552,7 +557,7 @@ impl Parse for SqltkBinOp {
 
         Err(syn::Error::new(
             input.span(),
-            "Expected an operator corresponding to one of the EQL traits Eq, Ord, TokenMatch or JsonLike".to_string(),
+            "Expected an operator corresponding to one of the EQL traits Eq, Ord, TokenMatch, JsonLike or Contain".to_string(),
         ))
     }
 }

packages/eql-mapper/CONTEXT.md

@@ -4,11 +4,13 @@ Parses SQL, infers a type for every node, and rewrites statements that touch enc
 columns into their EQL v3 equivalents. It knows nothing about the PostgreSQL wire
 protocol, ZeroKMS, or ciphertext — it reasons about types and rewrites syntax.
 
-> **Migration in flight.** The code still emits the EQL v2 surface
-> (`eql_v2_encrypted` casts, `eql_v2.*` calls); the vocabulary below is the **v3
-> target** the type-checker extension is being designed against. See
-> [`docs/adr/`](./docs/adr/) for the load-bearing decisions and
-> `docs/plans/2026-07-20-eql-v3-type-checker-handoff.md` for the impact maps.
+> **v3 migration.** The mapper now emits the EQL **v3** surface — v3 domain casts
+> (`::public.eql_v3_*` for stored values, `::eql_v3.query_*` for operands) and the
+> `eql_v3.*` functional-index form (term-extraction functions, `eql_v3.jsonb_*`,
+> `eql_v3."->"`, `match_term`). No `eql_v2.*` names remain in its output.
+> End-to-end validation against a live database with EQL v3 installed is still
+> pending. See [`docs/adr/`](./docs/adr/) for the load-bearing decisions and
+> `docs/plans/2026-07-20-eql-v3-type-checker-handoff.md` for the original impact maps.
 
 ## Language
 

packages/eql-mapper/docs/adr/0003-v3-rewrite-pipeline.md

@@ -0,0 +1,118 @@
+---
+status: accepted
+---
+
+# The EQL v3 rewrite pipeline: term functions, cast targets, and operand context
+
+ADR-0001 fixes *what* the mapper emits (the `eql_v3.*_term()` functional-index form);
+this ADR fixes *how* the transformation pipeline produces it, because the v2→v3 change
+is structural, not a find-and-replace of names.
+
+## Two contexts, two cast targets
+
+An encrypted value appears in a statement in one of two roles, and they cast differently:
+
+- **Stored value** — an INSERT `VALUES` item or an UPDATE `SET` right-hand side. It casts
+  to the column domain, `'<ciphertext>'::jsonb::public.eql_v3_<token>_<cap>`, and is **not**
+  wrapped in a term function.
+- **Query operand** — the right-hand side of a predicate (`col = $1`, `col > 'x'`). It casts
+  to the query twin, `'<ciphertext>'::jsonb::eql_v3.query_<token>_<cap>`, and the whole
+  predicate is rewritten through term functions (below).
+
+The v2 pipeline did not need this distinction: every encrypted value cast to the single
+opaque `eql_v2_encrypted`, and the opaque type carried its own operators. Under v3 the two
+roles produce different SQL, so the pipeline must know a value's role.
+
+**Decision:** the *type checker* records each encrypted literal/param's role while it walks
+the AST (it already visits the INSERT/UPDATE targets and the predicate operands during
+inference), and the transformation rules read that role. Re-deriving role from AST context
+inside the transform is the fallback if threading it through inference proves awkward, but
+the inference pass is where the context is already known.
+
+## Operator rewriting
+
+A comparison with an encrypted operand is rewritten by wrapping **both** operands in the
+term function the operator's capability selects:
+
+```
+col <op> operand   →   eql_v3.<term>(col) <op> eql_v3.<term>(operand)
+```
+
+The term function is chosen by `(operator, the terms the column's domain stores)` — verified
+against the `eql_v3.eq`/`lt`/… bodies in the installed `cipherstash-encrypt.sql`:
+
+| Operator | Term function |
+|---|---|
+| `=` `<>` | `eq_term` **if the domain stores `hm`**, else `ord_term` (`op`), else `ord_term_ore` (`ob`) |
+| `<` `<=` `>` `>=` | `ord_term` (domain stores `op`) / `ord_term_ore` (domain stores `ob`) |
+| `@@` | `match_term` (domain stores `bf`) |
+
+Two verified subtleties:
+
+- **`=` is not always `eq_term`.** A term-extraction function exists exactly where its term
+  does: `eq_term` only on domains storing `hm` (`_eq`, `_search*`, and — the text exception —
+  `text_ord*`). On an ord-only scalar such as `integer_ord` there is no `hm`, so `eql_v3.eq`
+  itself is `ord_term(a) = ord_term(b)`. The mapper mirrors this: `=`/`<>` fall back to the
+  ordering term when the domain has no `hm`.
+- **`ord_term` vs `ord_term_ore`** is not derivable from the coarse `Ord` trait — it comes
+  from the domain identity (ADR-0002): a `*_ord` / `*_ord_ope` domain stores `op` ⇒
+  `ord_term`; a `*_ord_ore` / `*_search_ore` domain stores `ob` ⇒ `ord_term_ore`.
+
+Which terms a domain stores is recoverable from its typname (with the text `hm` exception);
+the mapper derives them there rather than re-consulting `eql-bindings`.
+
+### Operand cast target — the query twin
+
+The right-hand operand casts to the **query twin** `eql_v3.query_<token>_<cap>` (schema
+`eql_v3`), e.g. a `public.eql_v3_integer_ord` column's operand casts to
+`eql_v3.query_integer_ord`. Verified: the twins exist for every scalar domain, carry the
+**term-only** payload (`{v,i,<terms>}`, no stored ciphertext `c`) that a query value actually
+is, and have their own `ord_term`/`eq_term` overloads. So:
+
+```
+salary > 'x'   →   eql_v3.ord_term(salary) > eql_v3.ord_term('<ct>'::jsonb::eql_v3.query_numeric_ord)
+```
+
+The column operand needs no cast (it is already the domain type); only the query operand is
+cast, and to the twin — **not** the column domain, whose CHECK requires the ciphertext a
+query value does not carry. (`eql_v3.eq(domain, jsonb)` casting to the column domain is a
+separate convenience overload, not what the mapper emits.)
+
+**Selecting the term function *is* the capability check.** A column whose domain provides no
+term function for the operator (e.g. `ORDER BY` / `>` on an `_eq` column, any operator on a
+storage-only `boolean`/`json` column) has no valid rewrite target — that absence *is* the
+capability error the type checker raises. This keeps one mechanism, not a separate bounds
+check bolted on.
+
+## JSON is different (see ADR-0002 amendment)
+
+Encrypted JSON columns (`eql_v3_json_search`) keep `->`/`->>` (JsonLike) **and** `@>`/`<@`
+(Contain) — verified against the installed `cipherstash-encrypt.sql`. `Contain` is therefore
+retained as a JSON-only capability and its rewrite targets the SteVec containment surface
+(`eql_v3.to_ste_vec_query` + `@>`), **not** deleted. `@>`/`<@` on *scalar* encrypted columns
+still have no term/rewrite and so raise.
+
+## Rule inventory under v3
+
+- `CastLiteralsAsEncrypted` / `CastParamsAsEncrypted` — retained; the cast target moves from
+  `eql_v2_encrypted` to the role-appropriate v3 domain (column domain vs query twin). These
+  gain access to each node's domain identity (via `node_types`) to name the target.
+- **`RewriteEqlComparisonOps`** (new) — wraps scalar comparison operands in term functions
+  and performs the capability check. Models its node handling on `RewriteContainmentOps`
+  (`mem::replace` against a throwaway `Value::Null` to preserve operand `NodeKey` identity
+  for the cast rules that run after).
+- `RewriteContainmentOps` — **retargeted, not retired**: `@>`/`<@` on JSON columns rewrite to
+  the v3 SteVec containment surface; on scalar columns they raise.
+- `RewriteStandardSqlFnsOnEqlTypes` — retargeted from `eql_v2.{min,max,jsonb_*}` to the v3
+  surface. Whether some of these become native overload resolution (and the rule shrinks) is
+  gated on v3 shipping operator/function overloads bound to the domains; verify per function.
+- `PreserveEffectiveAliases`, `FailOnPlaceholderChange` — unchanged.
+
+## Consequences
+
+- The pipeline gains a stored-vs-operand notion it did not have; this is the load-bearing new
+  concept, and getting it wrong casts an operand to a column domain (or vice versa).
+- Bound checking goes live here: the term-function selection raising on an absent capability
+  is the user-visible capability error, so the ADR-0001 "let the database do its job" stance
+  is refined — the mapper raises when there is no valid rewrite, rather than emitting SQL that
+  would fail at the database.