You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Audit every diagnostic counter for a cap, and record the answers #3450
Follow-up to #3448, where clang's default -ferror-limit=20 turned out to have been censoring every C error total published for this repository. The obvious next question is whether the other instruments do the same, and the honest answer was that nobody had asked.
Measured, 50 errors planted per language
instrument
reported
verdict
clang (default)
20
truncates at -ferror-limit=20
clang -ferror-limit=0
50
complete
rustc
50
complete
zig build-obj
50
complete
iverilog
50
complete — but two diagnostic lines per error
yosys
1
aborts on the first error
yosys is not capped, it stops: an error count from it is 0 or 1 and means nothing. Only its pass/fail is a measurement. Anything on the dashboard reading "yosys 454 → 455" is an acceptance count, which is valid.
The fixtures were the hard part, and two of five were wrong
Both wrong ones read as findings about the instrument:
the Zig fixture named its functions f16, f32 — which shadow Zig primitives — so it reported 2 errors of an entirely different kind, which looked exactly like a cap;
the Verilog fixture used implicitly declared identifiers, which yosys treats as a warning, so it reported 0 errors, which looked like silence.
tools/diagnostic_capacity.py --self-check now requires each fixture to produce the expected kind of diagnostic before its count is believed.
What is and is not affected
Acceptance counts are safe, and this was checked rather than assumed. Clang's exit code is 1 with and without the limit on a failing file, and 0 on a clean one — so a count of files that compile cannot be censored by a per-file cap on diagnostics. Every "cc accepts N" / "zig test N" / "iverilog accepts N" figure stands.
The repository's own gates are clean. The only tool invoking cc is check_duplicate_agreement.py, and it reads returncode, never a diagnostic count.
The Rust totals are not affected either.3251 → 3223 from #3437 came from rustc, which reports completely.
So the damage was confined to C error totals in my own reports:
tools/diagnostic_capacity.py records the table above and fails if an instrument that reported completely starts truncating — a ratchet on the measuring apparatus rather than on the code. It runs in under a second.
Activity
tools: ask every compiler whether it reports what it finds (#3451)
tools: the capacity fixture must not speak one compiler's dialect (#3452
tools: rank what is actually wrong with the generated C (#3466)
tools: pin the corpus figures with their instrument (#3506)