Skip to content

fix(msan): five "stack-overflow" suites were hiding two uninitialized reads - #2596

Merged
DeusData merged 3 commits into
mainfrom
fix/msan-hidden-uninit-reads
Oct 11, 2026
Merged

DeusData merged 3 commits into
mainfrom
fix/msan-hidden-uninit-reads

Conversation

@DeusData

Copy link
Copy Markdown
Owner

The MSan lane excluded five suites for "stack-overflow under instrumentation". That was never deep recursion. MSan was crashing while it reported real uninitialized reads. This PR fixes the two reads, makes the reports print, and puts the five suites back in the lane. Follows up on the finding in #2583.

What was hiding

suites read fix
grammar_labels, grammar_probe_e, grammar_regression, lang_contract tree_sitter_wit_external_scanner_create() returns void * but had an empty body; the runtime stores the garbage as the scanner payload (UB, C11 6.9.1p12; hidden by the vendored -w) return NULL; plus a row in MANIFEST.md's local-patch table. Upstream bytecodealliance/tree-sitter-wit still has it at 059bfdd1af72
pipeline githistory_coupling_limits_output filled its commits' files and count but never their timestamp, which cbm_compute_change_coupling reads (pass_githistory.c:304) the test's stack array is zero-initialized; production's commits are already zeroed

Why it looked like a stack overflow

  • Every abort was on the main thread. MSan's T<n> is the OS thread id, and it equalled the pid in all five x86-64 aborts (CI job 91868259558, 2026-08-04, the last run without exclusions). The old notes read it as a worker-thread ordinal, so the worker-stack knob, MSAN_ORIGINS and CBM_WORKERS=1 could never change anything.
  • With the default slow unwinder (in this image the MSan-instrumented libunwind), reporting a read exhausted the main thread's 256 MiB stack, and the overflow handler then died with "nested bug in the same thread". fix(rst): a section's text no longer reads an unwritten byte #2583's address and today's are both a few hundred bytes past the 256 MiB limit that msan.sh sets.
  • The two runs below are on the same tree and test, grammar_label_goldens.
MSAN_OPTIONS result
default (slow unwind) stack-overflow on address 0xffffeffffef0 ... T627, then nested bug in the same thread
fast_unwind_on_fatal=1 use-of-uninitialized-value at wit/scanner.c:13:54, with a full stack

Lane changes (scripts/msan.sh)

  • MSAN_OPTIONS defaults to fast_unwind_on_fatal=1. The build already keeps frame pointers.
  • MSAN_EXCLUDE is down to cli, whose install-path failure is a separate cause that is still undiagnosed.
  • Rewritten to match what is true:
    • the known-red block;
    • the header, which still said CI runs without exclusions;
    • the MSAN_ORIGINS and stack comments.
  • The "PARTIAL lane" warning and the exclude-name typo check sat in a branch behind elif true and never ran. They now run in the per-suite path. The check is a case match, not a grep -q pipe, which pipefail can misreport.
  • Comment-only: scripts/ci/msan-lane.sh and the comment above the test-msan job in .github/workflows/_test.yml no longer say the five suites overflow. No job or step changes.

Checks

  • Local arm64 MSan (docker compose … run --rm test-msan, the real msan.sh with its new defaults), on the five formerly excluded suites:

  • macOS ASan, the same five suites: 441 passed, 0 failed.

  • make lint-ci is clean. scripts/security-vendored.sh passes with the two updated checksums (wit/scanner.c and MANIFEST.md).

  • PR CI is the gate. The lane selector puts this PR in T5-core (workflow and vendored paths), so test-msan runs on x86-64.

  • Local legs skipped: the full 3-OS ladder. The C changes are a one-line return in a vendored scanner and a test initializer, so a platform-specific failure is unlikely.

tree_sitter_wit_external_scanner_create() is declared to return void *
but had an empty body. tree-sitter's runtime stores its result as the
scanner payload (parser.c ts_parser__external_scanner_create), so every
WIT parse used an indeterminate value, which C11 6.9.1p12 makes
undefined. The scanner never reads its payload, so ordinary builds were
unaffected, and the vendored build's -w hid -Wreturn-type.

MemorySanitizer reports it at wit/scanner.c:13 from grammar_labels,
grammar_probe_e, grammar_regression and lang_contract. Those four suites
were excluded from the MSan lane because the report itself crashed and
posed as a stack overflow (fixed in the msan.sh commit of this series).

Recorded in MANIFEST.md's local-patch table. Upstream
bytecodealliance/tree-sitter-wit still has the empty body at
059bfdd1af72, the last change to src/scanner.c.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
The test filled 3675 commits' files and count but never their timestamp,
and cbm_compute_change_coupling reads each commit's timestamp to track a
pair's last co-change (pass_githistory.c:304). MemorySanitizer reports
the read; the stack array is now zero-initialized like every other
githistory test's commits. Production is unaffected: its commits come
from calloc and a zeroed parse state.

This was the read behind the pipeline suite's MSan "stack overflow"
(the x86-64 CI log of 2026-08-04 aborted in this test).

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Five suites (grammar_regression grammar_labels pipeline lang_contract
grammar_probe_e) were excluded from the MSan lane for "stack-overflow
under instrumentation". That was never deep recursion. MSan aborted while
REPORTING real uninitialized reads:

- every abort was on the main thread. MSan's "T<n>" is the OS thread id
  and equalled the pid in all five x86-64 aborts (CI job 91868259558);
  the old block read it as a worker-thread ordinal, so the worker-stack
  knob, MSAN_ORIGINS and CBM_WORKERS=1 could never change anything;
- with the default slow unwinder (the image's MSan-instrumented
  libunwind) a report exhausted the main thread's 256 MiB stack, and the
  overflow handler then failed with "nested bug in the same thread";
- with fast_unwind_on_fatal=1 the same test prints the
  use-of-uninitialized-value report instead (measured on one tree and
  test: slow unwind aborts at the stack limit, fast unwind reports
  wit/scanner.c:13).

The two reads are fixed in the previous commits of this series, so:

- MSAN_OPTIONS defaults to fast_unwind_on_fatal=1 (the build already
  keeps frame pointers);
- MSAN_EXCLUDE is down to cli, whose install-path failure is a separate,
  still-undiagnosed cause;
- the known-red block, the header (it still claimed CI runs without
  exclusions) and the MSAN_ORIGINS and stack comments say what is true;
- the partial-lane warning and the exclude-name typo check lived in a
  branch behind `elif true` that never ran. They now run in the
  per-suite path, the check as a case match instead of a grep -q pipe
  that pipefail can misreport;
- the comments in scripts/ci/msan-lane.sh and above the test-msan job
  in _test.yml no longer say the five suites overflow.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData
DeusData merged commit 475db1e into main Oct 11, 2026
73 of 75 checks passed
DeusData added a commit that referenced this pull request Oct 12, 2026
Brings in #2596 (MSan: hidden uninitialized reads) and #2068 (CodeAtlas
web).

README: main's CBM Atlas line and this branch's infrastructure-as-code
line are both kept.

tools/list metadata: main and this branch each stayed under the 20 KiB
discovery budget, but together they reached 20,809 bytes. The branch's
wording is shorter now (search_graph kind/namespace, one hint per trace
mode: 136 bytes less), and the reviewed budget is 21 KiB so the
topology/deploy/run modes keep their usage hints.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant