Skip to content

Collect all case precheck failures without suppressing native execution and reports #610

Description

@LunaStev

Observed on PR #606 head 037baf248a9793be2b0563fb28972b5c4fb9714a (2026-09-12).

The Linux amd64 job runs an all-target source precheck under set -e before its native case runner. One source error stops the entire precheck, skips the native run, and leaves the unconditional report upload with no file.

Observed in the PR #606 Linux amd64 job:

  1. Check every target case stops on the stale helper import in linux/amd64/test13.wave.
  2. Run all Wave cases is skipped.
  3. Upload reports No files were found with the provided path: wave-cases-linux-amd64.json. No artifacts will be uploaded.

The workflow uses:

set -euo pipefail
mapfile -t cases < <(python3 tools/case_manifest.py sources)
for source in "${cases[@]}"; do
  target/release/wavec check "tests/cases/$source"
done

Starting point: cases.yml.

Acceptance:

  • Check every discovered case and aggregate nonzero results with source paths and diagnostics, then return failure if any check fails.
  • Preserve a precheck summary/log artifact even when prechecking fails.
  • Keep the native case run eligible once its compiler/std prerequisites succeeded, or isolate the all-target precheck in a separate job. Do not attempt runtime execution after compiler provisioning/build failed.
  • Preserve the job/workflow failure status and distinguish precheck failure, runtime failure, and tests that never ran; do not manufacture passing case records.
  • Verify with at least two failing sources and a later successful source so fail-fast behavior cannot silently return.

Related: #575 concerns native ARM64 diagnostic-step ordering; #576 concerns the contents of reports produced by the runner. This issue concerns the separate all-target precheck and the missing report caused before the native runner starts. Fixing the obsolete import alone does not correct this workflow behavior.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugA problem that causes incorrect behavior or crashes.ciGitHub CI or CIhelp wantedThe issue requires extra attention or help from others.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions