Repository navigation
gen-c, gen-rust and gen (Zig) are not byte-deterministic: 1, 3 and 2 files differ between two identical runs #3006
Description
Three of the four backends are not byte-deterministic
Same binary, same tree, two consecutive runs over all 650 specs, comparing the
generated output file by file:
gen-c 581/581 files DIFFERING BETWEEN TWO IDENTICAL RUNS = 1
gen-rust 581/581 = 3
gen (Zig) 581/581 = 2
gen-verilog 581/581 = 0
Found while measuring something else: a skeptic sizing a patch's blast radius saw
3 differing files in gen-rust and 2 in gen-c, ran the stock binary against
itself as a control, and got the same wobble — so the difference belonged to
master, not to the patch. Reproduced here independently with a third harness.
gen-verilog is deterministic, which is why the wobble has not been noticed:
the seal system and the emit-bitexact gate are historically pointed at the
Verilog path.
What this does NOT show, measured rather than assumed
tri seals drift was run three times on an unchanged tree and reported
0, 0, 0. So the non-determinism does not currently reach the seal gate, and
this issue does not claim the seals are unreliable. The likely reason is that a
seal stores sha256 of output for a spec, and the handful of wobbling files are
not among the ones whose hashes get recomputed on a given run — but that is a
guess, and it is written here as one.
Why it still matters
Every byte-comparison over generated output is only as good as the generator's
determinism. This repository has several:
.trinity/seals/*.jsonstoregen_hash_c,gen_hash_rust,gen_hash_zig.emit-bitexact-gate.ymlexists to assert that emission is reproducible.t27c corpus --per-specis used to diff one binary against another; the log
already records "the three specs, named" as a technique that depends on the
diff being signal.
A wobble of 1–3 files in 581 is small enough to read as a real change on a day
when it happens to fire, and this repository's own log records the cost of that
mistake more than once — "a gate going red on a change that cannot touch it".
Not established
- Which files wobble, and whether it is the same ones each time. I counted;
I did not name them. - The mechanism. The prior art in this tree points at hash-map iteration order:
Three module forms, and a signature is not a field #2833-era work foundgen-verilog-for-simulationvarying in the order of two
adjacentreg _t27_call_tmp_*declarations, attributed toHashSetiteration.
Whether the C/Rust/Zig wobble is the same cause is unmeasured. - Whether it is load-dependent. Both my run and the skeptic's were on a busy
machine.
The cheapest next step is naming the files: run the pair again and print the
differing basenames rather than the count.
Refs #2988
Boundary
.github/workflows/emit-bitexact-gate.yml
User Scenarios
- Given the t27 compiler built from a fixed commit, when running
t27c corpus --per-spectwice consecutively over all 650 specs, then the generated C, Rust, and Zig outputs for every spec are byte-identical between the two runs.
Requirements
- FR-001: The C backend (gen-c) MUST produce byte-identical output for every spec across repeated invocations of the same binary.
- FR-002: The Rust backend (gen-rust) MUST produce byte-identical output for every spec across repeated invocations of the same binary.
- FR-003: The Zig backend (gen) MUST produce byte-identical output for every spec across repeated invocations of the same binary.
- FR-004: The emit-bitexact gate MUST verify byte-determinism for all four backends (C, Rust, Zig, Verilog) on every CI run.
Success Criteria
t27c corpus --per-specrun twice consecutively produces zero differing files for gen-c, gen-rust, and gen (Zig) —exit 0.github/workflows/emit-bitexact-gate.ymlexecutes and asserts byte-determinism for all backends —exit 0- 650 specs × 3 backends verified deterministic with 0 wobbling files
Activity
gHashTag commentedon Sep 3, 2026
Named: four specs, and largely the same four in every backend
tri emit determinism(PR #3007) does what this issue said the next step was.650 specs, same binary, two runs:
Four specs in total. Two of them wobble in all three non-Verilog backends.
That is a different finding from the one this issue opened with: not three
independent emitter defects, but something shared that four specs reach.
The counts in this issue's title are not a population
The harness that opened this read 1 / 3 / 2. This one reads 4 / 2 / 4 on
the same tree with the same binary. Which files wobble is itself a draw, so a
count taken once measures the draw and not the population. Only the union is
meaningful, and across both measurements it is these four names.
That is the second time this pass a count turned out to be the wrong shape of
answer, and it is the reason the command prints names.
Still not established
earlier finding saw
gen-verilog-for-simulationvarying in the order of twoadjacent
reg _t27_call_tmp_*declarations. Whether these four are the samecause is unmeasured and the command does not guess.
_tmp_pipeline_import.t27andpipeline.t27are neighbours by name and
sacred_verification.t27andzamolodchikov_4d_conjecture.t27are both underspecs/physics/, which issuggestive and nothing more until someone diffs the two emissions of one of
them and reads what moved.
tri seals driftread 0, 0, 0 on threeruns, so this still does not show the seals are unreliable.
The cheapest next step is now one file: take
specs/igla/coder/pipeline.t27,emit it twenty times, and
diffthe outputs that differ. Four names is atractable population; three counts was not.
docs(skill): 429-430 -- a broken corpus is not a control, and a wobbl…
feat(tri): emit determinism -- name the specs whose output wobbles (#…
salvage(queen-3006): commit what the turn left uncommitted
Update emit-bitexact-gate.yml to check all four backends for determinism