Protect labelled customer, employee and person record IDs

CHANGELOG.md

@@ -46,6 +46,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   retain their keys and punctuation. Generic `token` keys remain outside the
   code-field rule; ambiguous password metadata stays fail-closed. Whole Argon2
   encodings no longer split at an inner postal-code candidate (#824).
+- Protect explicitly labelled customer, employee and person/member record identifiers
+  in English/German prose, JSON and logs with exact reversible value capture.
+  Generic record, order, invoice, issue, build and tracking IDs remain outside these rules.
+
 - Plain-text session files on macOS/Linux clear and verify their extended ACL
   before any restore-key bytes are written, including overwrite replacements.
   ACL failures and unsupported platforms (including Windows) refuse
@@ -111,6 +115,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   A new mapping invalidates the cache. Ambiguous runs now return the distinct
   `AmbiguousTokenRun` error, whose display and debug output contain no tokens
   or manifest values.
+- Cross-class identifier trimming discards repeat seeds that extend beyond the
+  shortened owner, preserving fail-closed validation without refusing valid
+  mixed licence and card inputs.
+
+- Preserve first-value repeats when a complete private or unknown cue remains
+  inside a conservative legacy identifier owner, including raw NFC/NFD licence
+  labels. One typed prefix seed retains the owned tail and existing length floor;
+  conservative protection of public prefix copies remains a measured cost.
+
+- Labelled customer, employee and member IDs preserve terminal prose punctuation,
+  stop at complete next-field cues, and protect raw quoted scalars through nested
+  serialization, initial newlines and tab groups. Single and triple delimiters
+  retain doubled quotes and complete multiline values. Unicode opening
+  punctuation, typographic quotes, paired brackets and backticks preserve whole
+  private scalars. Complete CSV cells and enclosing SQL literals distinguish
+  doubled delimiters from JSON escaping. Unpaired scalars inside CSV and SQL
+  literals retain embedded line breaks and literal backslashes. Every close search stays
+  inside its enclosing serialized string, preserving later public fields.
+  JSON record-separator escapes retain the following person-label cue.
+  Unterminated values end before an enclosing serialized string
+  close, at an unescaped record line break, or at input end. Key-looking private
+  suffixes stay protected, and enclosing JSON remains parseable. Unclassified
+  quote runs retain nonempty ownership. Conservative tab prefixes protect repeats
+  under unchanged length floors, including later session turns; their public-byte
+  cost is disclosed in benchmark evidence.
+- Legacy grouped identifier extensions stop at positively recognized complete
+  operational metadata cues without trimming their original regex capture.
+  Private and unknown tails retain conservative ownership until an independent
+  recognizer supplies existing boundary proof. Unicode groups and complete
+  licence connectors retain their original bytes.
 - `gaze-proxy` no longer restores a value captured from a JSON document with its
   JSON escapes. A name captured as `Robert \"Bob\" Smith` from a JSON text block
   or JSON tool result now reaches Anthropic `tool_use.input` as
@@ -145,6 +179,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- Signed session snapshots with deterministic repeat-seed metadata use envelope
+  v8. JSON source spelling alone retains v7; ordinary snapshots remain v6. New
+  readers restore v2–v8. Older readers reject v8 rather than silently dropping
+  repeat protection; upgrade all readers before exchanging these snapshots.
 - **Breaking:** context JSON requires a top-level object. Positional arrays of
   zero through six fields are rejected with `PolicyConfig` by
   `gaze clean --context-json` and daemon clean frames. Replace positional input
@@ -153,9 +191,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 - **Snapshot contract.** `Session` records each token's source spelling and
   adds `source_spelling` and `restore_strict_decoded_text_with_provenance`. A
-  manifest holding a JSON-escaped value exports as `session_blob` envelope v7,
-  which v0.17 and older refuse with `InvalidSnapshotVersion(7)`. Other
-  manifests still export as v6.
+  manifest holding a JSON-escaped value exports as `session_blob` envelope v7
+  unless repeat metadata requires v8. Readers on v0.17 and older refuse both
+  versions. Manifests without either metadata path still export as v6.
 
 ### Benchmark
 

CLAUDE.md

@@ -54,8 +54,9 @@ The browser-MITM project is separate (formerly in-tree `debug-proxy`).
   [`docs/explanation/detection/validator-veto.md`](docs/explanation/detection/validator-veto.md).
 - Repeat-value sweep: after resolve and before the safety net, uncovered copies of rule-found values
   (this document's winners + session manifest entries with a non-learned evidence tier) join the
-  pool and it is resolved again; NER/safety-net values never propagate. `session_blob` is v6
-  (per-entry evidence tier). Contract:
+  pool and it is resolved again; NER/safety-net values never propagate. `session_blob` uses v6 for ordinary values, v7 for JSON source spelling alone, and v8 when
+  deterministic repeat-seed metadata is present (per-entry evidence plus authenticated seed offsets),
+  including entries that also retain JSON source spelling. Older readers reject unsupported versions. Contract:
   [`docs/explanation/detection/manifest-sweep.md`](docs/explanation/detection/manifest-sweep.md).
 - Collision-family policy: cross-class recognizer rivalries declare `[recognizers.collision]` or
   `[policy.custom_recognizers.collision]` metadata and compile into `FamilyPolicyTable`.

UPGRADE.md

@@ -50,17 +50,45 @@ process that writes one.
 
 A session now records whether each value was captured inside a JSON string
 literal in its escaped spelling. A manifest that holds such a value exports as
-envelope version 7. Every other manifest still exports as version 6.
+envelope version 7 unless it also carries repeat metadata (version 8).
+Manifests without either metadata path still export as version 6.
 
 - A v0.17 or older reader refuses a v7 blob with `InvalidSnapshotVersion(7)`.
   It fails closed; nothing is restored.
-- This release imports v2 to v7 blobs. Older blobs restore exactly as before;
-  their values read as plain text.
+- This release imports v2 to v8 blobs. Older blobs restore exactly as before;
+  values without spelling metadata read as plain text.
 
 `gaze-proxy` restores such a value into Anthropic `tool_use.input` as the value
 its escapes spell, not with literal backslashes. Text blocks and other
 plain-text destinations still receive the bytes the client sent.
 
+### Action required: conditional `session_blob` version 8
+
+Field-owned record identifiers now retain exact scalar whitespace and conservative
+repeat prefixes for ambiguous tab groups and complete cues
+inside conservative legacy owners. Legacy prefixes never shorten the original
+capture or declare the next field public. Their ordinary length floor is unchanged.
+A signed snapshot with this metadata uses envelope v8. JSON source spelling alone
+retains v7. Ordinary snapshots remain v6. Readers supporting only v7 and earlier
+reject v8 with `InvalidSnapshotVersion(8)`.
+
+Upgrade every reader before importing newly written v8 snapshots. The new reader
+imports and restores v2–v8 without re-tokenizing old manifests. Seed offsets bind
+to the existing owner value. Import rejects invalid geometry, missing deterministic
+evidence, and seed metadata in an older envelope. Whole-value ownership and
+strict restore remain intact. Conservative prefix matches can protect public
+bytes and retain the existing repeat-length floors.
+
+Single and triple ASCII quote delimiters retain raw content, including doubled
+quotes and complete multiline scalars. Every closing-quote search stops before
+the enclosing serialized string's close at every representable depth.
+Unterminated or inconsistent values end at the nearest enclosing serialized
+string close (excluded, with escape depth preserved), an unescaped LF/CRLF/CR
+record boundary, or input end. Key-looking text and container punctuation inside
+the unfinished value remain private; a following assignment alone is no boundary.
+Unclassified quoting keeps nonempty ownership through that safe boundary. Unquoted tab grouping
+retains its existing conservative public-column cost.
+
 ---
 
 ## v0.16.x → v0.17.0

anvil.yaml

@@ -60,7 +60,7 @@ checks:
     inputs: *rust-inputs
     toolchain: *rust-tools
     commands:
-      - cargo clippy --workspace --all-features --all-targets -- -D warnings
+      - env CARGO_BUILD_JOBS=2 nice -n 10 cargo clippy --workspace --all-features --all-targets -- -D warnings
   fast:
     description: Format and lint before full verification
     reusable: true
@@ -69,7 +69,7 @@ checks:
     toolchain: *rust-tools
     commands:
       - cargo fmt --all -- --check
-      - cargo clippy --workspace --all-features --all-targets -- -D warnings
+      - env CARGO_BUILD_JOBS=2 nice -n 10 cargo clippy --workspace --all-features --all-targets -- -D warnings
   test:
     description: Workspace tests, with at least one executed test
     expensive: true
@@ -78,7 +78,7 @@ checks:
     toolchain: *rust-tools
     commands:
       - python3 -m unittest discover -s scripts/anvil -p 'test_test_*.py'
-      - python3 scripts/anvil/test_nonempty.py cargo test --workspace --all-features
+      - python3 scripts/anvil/test_nonempty.py env CARGO_BUILD_JOBS=2 nice -n 10 cargo test --workspace --all-features
   feature-matrix:
     expensive: true
     locks: [workspace-build]
@@ -114,3 +114,19 @@ checks:
     commands:
       - cargo fmt --all -- --check
       - env CARGO_BUILD_JOBS=2 GAZE_DISABLE_OS_KEYSTORE=1 nice -n 10 python3 scripts/gate/os-keystore-mutation-probe.py --focused
+
+  record-id-focused:
+    description: Focused labelled-record JSON and sweep regression tests
+    inputs: *rust-inputs
+    toolchain: *rust-tools
+    commands:
+      - env CARGO_BUILD_JOBS=2 nice -n 10 cargo test -p gaze-recognizers --lib
+      - env CARGO_BUILD_JOBS=2 nice -n 10 cargo test -p gaze-pii --lib
+      - env CARGO_BUILD_JOBS=2 nice -n 10 cargo test -p gaze-cli --test record_ids
+      - env CARGO_BUILD_JOBS=2 nice -n 10 cargo test -p gaze-cli --test daemon_reversible record_ids
+  record-id-mutations:
+    description: Compiled mutations for record ownership and repeat protection
+    inputs: *rust-inputs
+    toolchain: *rust-tools
+    commands:
+      - env CARGO_BUILD_JOBS=2 nice -n 10 python3 scripts/verify_record_id_scanner_mutations.py

crates/gaze-assembly/src/detector_wiring.rs

@@ -141,6 +141,7 @@ pub(crate) fn register_rulepack_recognizers(
                 pattern_template,
                 capture_groups,
                 complete_labelled_value,
+                labelled_value_mode,
             } => {
                 let pattern =
                     lower_regex_pattern(&recognizer.id, pattern, pattern_template, locale_vocab)?;
@@ -182,7 +183,14 @@ pub(crate) fn register_rulepack_recognizers(
                             .as_ref()
                             .and_then(|context| context.reject_match_regex.as_deref()),
                     )?
+                    .with_unquoted_capture_rejection(
+                        recognizer
+                            .context
+                            .as_ref()
+                            .and_then(|context| context.reject_unquoted_capture_regex.as_deref()),
+                    )?
                     .with_complete_labelled_value(complete_labelled_value)
+                    .with_labelled_value_mode(labelled_value_mode)
                     .with_locale_basis(recognizer.locale_basis)
                     .with_validator_on_fail(
                         recognizer

crates/gaze-cli/tests/daemon_reversible.rs

@@ -728,3 +728,98 @@ fn context_requests_keep_resident_ner_and_its_learned_evidence_tier() {
         "caller-known rule evidence must propagate"
     );
 }
+
+#[test]
+fn padded_record_ids_keep_repeat_protection_on_second_daemon_turn() {
+    let dir = tempdir().unwrap();
+    let policy = dir.path().join("policy.toml");
+    fs::write(&policy, "[session]\nscope = \"conversation\"\n[policy.rulepacks]\nbundled = [\"core\"]\n[[rule]]\nkind = \"default\"\naction = \"tokenize\"\n").unwrap();
+    let mut daemon = Daemon::start(&policy);
+    for (index, padding) in [" ".repeat(32), "\u{00a0}".repeat(16), "\\t".repeat(32)]
+        .into_iter()
+        .enumerate()
+    {
+        let session = format!("padded-record-{index}");
+        let opened = daemon.request(json!({"version":1,"op":"open","session_id":session}));
+        assert_eq!(opened["op"], "open", "{opened}");
+        let text = format!(r#"{{"customerId":"SYN731284{padding}"}}"#);
+        let first =
+            daemon.request(json!({"version":1,"op":"clean","session_id":session,"text":text}));
+        assert!(
+            !first["clean_text"].as_str().unwrap().contains("SYN731284"),
+            "{first}"
+        );
+        let next = daemon.request(json!({"version":1,"op":"clean","session_id":session,"text":"Follow-up about SYN731284"}));
+        assert!(
+            !next["clean_text"].as_str().unwrap().contains("SYN731284"),
+            "{next}"
+        );
+    }
+}
+
+#[test]
+fn mixed_record_ids_keep_json_structure_on_second_daemon_turn() {
+    let dir = tempdir().unwrap();
+    let policy = dir.path().join("policy.toml");
+    fs::write(&policy, "[session]\nscope = \"conversation\"\n[policy.rulepacks]\nbundled = [\"core\"]\n[[rule]]\nkind = \"default\"\naction = \"tokenize\"\n").unwrap();
+    let mut daemon = Daemon::start(&policy);
+    let mut checked = 0;
+    for (index, (seed, text, encoded)) in [
+        (
+            "customerId: 731284",
+            r#"{"731284":true,"total":731284,"copy":"731284"}"#,
+            false,
+        ),
+        (
+            "Customer ID = 731284",
+            r#"{\"731284\":true,\"total\":731284,\"copy\":\"731284\"}"#,
+            true,
+        ),
+        (
+            r#"{"customerId":"SYN\u0037\u0033\u0031\u0032\u0038\u0034"}"#,
+            r#"{"copy":"SYN731284"}"#,
+            false,
+        ),
+        (
+            r#"{"customerId":"SYN731284"}"#,
+            r#"{"copy":"SYN\u0037\u0033\u0031\u0032\u0038\u0034"}"#,
+            false,
+        ),
+    ]
+    .into_iter()
+    .enumerate()
+    {
+        let session = format!("mixed-record-{index}");
+        let opened = daemon.request(json!({"version":1,"op":"open","session_id":session}));
+        assert_eq!(opened["op"], "open", "{opened}");
+        let first =
+            daemon.request(json!({"version":1,"op":"clean","session_id":session,"text":seed}));
+        assert!(first["clean_text"].is_string(), "{first}");
+        let input = format!("log: {text}");
+        let next =
+            daemon.request(json!({"version":1,"op":"clean","session_id":session,"text":input}));
+        let clean = next["clean_text"].as_str().unwrap();
+        let body = clean.strip_prefix("log: ").unwrap();
+        let decoded = if encoded {
+            serde_json::from_str::<String>(&format!("\"{body}\"")).unwrap()
+        } else {
+            body.to_owned()
+        };
+        let parsed: Value = serde_json::from_str(&decoded).unwrap();
+        if index < 2 {
+            assert_eq!(parsed["731284"], true);
+            assert_eq!(parsed["total"], 731284);
+            assert_ne!(parsed["copy"], "731284");
+        } else {
+            assert!(
+                !clean.contains("SYN731284") && !clean.contains(r"SYN\u0037"),
+                "{clean}"
+            );
+        }
+        let restored =
+            daemon.request(json!({"version":1,"op":"restore","session_id":session,"text":clean}));
+        assert_eq!(restored["text"], input, "{restored}");
+        checked += 1;
+    }
+    assert_eq!(checked, 4);
+}