Skip to content

test(trios-chat): behaviour baseline, golden vectors, laws hash and pin tests - #1827

Open
dmitrii-f-t27 wants to merge 5 commits into
mainfrom
chat-t27/w0-baseline-golden-pin
Open

dmitrii-f-t27 wants to merge 5 commits into
mainfrom
chat-t27/w0-baseline-golden-pin

Conversation

@dmitrii-f-t27

@dmitrii-f-t27 dmitrii-f-t27 commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1825

Part of #1812 (Trinity Secure Chat written once in .t27).

Stacked on #1820 and contains its commit (adbe1a1, workspace member). Merge 1820 first.

What it adds

  • crates/trios-chat/golden/: the frozen behaviour baseline that every later swap of generated code must reproduce.
    • test-names/<package>.txt: sorted cargo test -p <package> -- --list names; counts.txt: listed and passed counts.
    • e2e_pass.txt (25/25 PASS), falsifier.stdout, falsifier.exit (1), falsifier.skips (400).
    • vectors.txt: output of the new golden_vectors bin, 3722 lines, identical on two runs, no OS randomness.
  • src/bin/golden_vectors.rs: deterministic vectors (laws hash, padding classes, dest hash, sealed envelope with fixed secrets and nonce, chain steps, DH steps, receive-accept paths, prekey bundle canonical bytes, capability token and tool manifest signing bytes, injection verdicts for every corpus line plus benign strings). ASCII-only source.
  • tests/laws_hash.rs: pins hex(laws_hash) = d8c962d0cfe111bf72f9d59f2296baf015ff5c9f0390cf807b7cc08c706f3a98.
  • T27_PIN (empty table: file, spec path, t27 SHA, t27c built_by, sha256, seal file) and tests/t27_pin.rs (sha256 of each vendored generated file equals its PIN row).
  • T27_EXCEPTIONS.md: 8 files proposed to stay hand-written Rust (glue, SeaORM/tokio I/O, the e2e gate). Marked "proposed, pending owner decision D11".
  • sha2 = "0.10" in the shim's [dev-dependencies] (the rings already use 0.10); Cargo.lock gains one dependency line.

Measured on b9b05c4 + this head (local, cargo 1.94.0, rustc 1.94.0, repository Cargo.lock, DATABASE_URL unset)

trios-chat-cr-chat-00 listed=9 passed=9 failed=0 exit=0
trios-chat-cr-chat-01 listed=476 passed=476 failed=0 exit=0
trios-chat-cr-chat-02 listed=432 passed=431 failed=0 exit=0
trios-chat-cr-chat-03 listed=521 passed=520 failed=0 exit=0
trios-chat-cr-chat-04 listed=395 passed=395 failed=0 exit=0
trios-chat-cr-chat-05 listed=437 passed=436 failed=0 exit=0
trios-chat-cr-chat-06 listed=404 passed=404 failed=0 exit=0
trios-chat-cr-chat-07 listed=381 passed=381 failed=0 exit=0
trios-chat-cr-chat-laws listed=4 passed=4 failed=0 exit=0
trios-chat-br-output listed=4 passed=4 failed=0 exit=0
trios-chat-br-io-chat-05 listed=1 passed=1 failed=0 exit=0
trios-chat-br-io-chat-07 listed=6 passed=6 failed=0 exit=0
trios-chat listed=5 passed=5 failed=0 exit=0
  • trios-chat-br-io-chat-05: its single test returns early without DATABASE_URL, so it checks nothing.
  • trios-chat-br-io-chat-07: measured on 3da28be and added in 4a87e09 (6/6).
  • find crates/trios-chat -name '*.sh' | wc -l: 0.
  • Required checks (no-js-check, Test, Constitutional Enforcement, guard): see the checks of this PR.

No ring source changes. The falsifier is not added as a gate (it exits 1 today; any fix is a separate PR).

L16 rationale: clippy -D warnings already fails at base 8b229e9 and ci.yml skips it; this PR adds tests, one bin and data files.
CODEOWNERS: this PR changes /Cargo.lock; it is merged only after @gHashTag's approval (requested by email).

Dmitrii Fedorov (@dmitrii-f-t27)

The root Cargo.toml lists the 12 trios-chat ring crates but not the shim
crates/trios-chat, so cargo refused to build it and e2e_chat_25,
falsifier_runner and tests/r_chat_guard.rs never ran in CI.
Cargo.lock gains only the trios-chat package entry.

Refs #1814
L16 rationale: clippy -D warnings already fails at base 8b229e9 and ci.yml skips it; this PR changes no Rust source.
…, T27_PIN, T27_EXCEPTIONS (Refs #1825)

Golden files (test names, e2e PASS list, falsifier output, vectors) follow in the next commit after the local run.

L16 rationale: clippy -D warnings already fails at base 8b229e9 and ci.yml skips it; this PR adds no ring logic.
…golden

Test names per package (cargo test -- --list) and pass counts for 12 of the
13 chat packages, e2e_chat_25 PASS list (25/25), falsifier_runner stdout,
exit code (1) and skipped corpus lines (400), and golden_vectors output
(3722 lines, identical on two runs). trios-chat-br-io-chat-07 is not
measured yet and follows in a separate commit.

Refs #1825
L16 rationale: clippy -D warnings already fails at base 8b229e9 and ci.yml skips it; this change adds only data files.
…es, same output)

Refs #1825
L16 rationale: clippy -D warnings already fails at base 8b229e9 and ci.yml skips it; output of golden_vectors is unchanged.
@tri-claw

tri-claw Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Summary by t27-bees

This PR establishes a frozen behavior baseline (golden tests, vectors, and pins) for trios-chat so future generated-code swaps can be verified against a known-good snapshot. It adds a deterministic golden_vectors binary, pins the laws_hash, introduces a T27_PIN table with a verification test, and documents eight proposed hand-written exceptions.

Worth a look

  • crates/trios-chat/golden/falsifier.exit — Contains 1, meaning the falsifier currently fails; the PR explicitly does not gate on it, but any CI that treats non-zero exit as failure will need to handle this.
  • crates/trios-chat/tests/t27_pin.rs:69 — Verifies vendored generated files against T27_PIN; if the table stays empty (as shown) the test passes vacuously and provides no actual pinning until rows are added.
  • crates/trios-chat/T27_EXCEPTIONS.md — Lists 8 files as "proposed, pending owner decision D11"; until that decision is recorded, the exception set is unsettled and the pin test may not cover the intended surface.
  • crates/trios-chat/golden/test-names/trios-chat-br-io-chat-05.txt — Only 1 test listed; the description notes it returns early without DATABASE_URL, so the golden baseline includes a test that checks nothing in environments lacking that variable.
  • crates/trios-chat/Cargo.toml:7 — Adds sha2 = "0.10" to [dev-dependencies]; the workspace already uses sha2 0.10 via the rings, but duplicate version declarations can drift if not managed by a workspace dependency.

Read: 27 files, +10675 -0, a summary only: the diff is larger than 4000 changed lines, so it was not read line by line. Head 4a87e09. Model: openai-compatible/nvidia/nemotron-3-ultra-550b-a55b. Commands: @t27-bees review | summary | help | pause | resume. Free for every repository.

Refs #1825

The ring was not measured in b9b05c4 because the local disk was below the
safe limit. Measured on 3da28be with the repository Cargo.lock, --offline,
DATABASE_URL unset: listed=6 passed=6 failed=0 exit=0.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Freeze the trios-chat behaviour baseline at 8b229e9: test names, e2e PASS list, falsifier output, golden vectors, pin test

1 participant