Skip to content

test(stellar): stabilize fuzz harness and add PR sample - #260

Merged
truthixify merged 4 commits into
wraith-protocol:developfrom
boalambo:fix/stellar-fuzz-harness
Oct 18, 2026
Merged

truthixify merged 4 commits into
wraith-protocol:developfrom
boalambo:fix/stellar-fuzz-harness

Conversation

@boalambo

Copy link
Copy Markdown
Contributor

Summary

Stabilizes the Stellar property fuzz harness and adds a quick sample for pull request checks.

Changes

✅ Fix scheduled fuzz timeout

  • Increased testTimeout from 20 minutes to 60 minutes in vitest.config.ts
  • The 100,000-run fuzz campaign (~6.5min execution + reporting overhead) was timing out during result reporting

✅ Add PR sample test

  • New test:fuzz:sample script runs 100 iterations with fixed seed 12345
  • Fast enough for pull request checks while still exercising all 12 properties
  • Deterministic results for consistent CI behavior

✅ Failure replay instructions

  • Added onFailure callback to print the failing seed and exact replay command
  • Added console output when using fixed seed to show replay instructions
  • Makes debugging property test failures straightforward

✅ Update CI workflow

  • PRs now run pnpm test:fuzz:sample after the main test suite
  • Scheduled nightly job continues to run the full 100k-run random campaign
  • Maintains deep fuzzing coverage without slowing down PR feedback

Testing

The changes have been implemented and committed. The CI will verify:

  • Normal test suite passes
  • Stellar property sample (100 runs, seed 12345) passes on PRs
  • Full fuzz campaign (100k runs) passes on nightly schedule

Related Issue

Closes #247

- Increase testTimeout to 60 minutes to accommodate 100k-run fuzz campaign
- Add test:fuzz:sample script with fixed seed for quick PR checks
- Add onFailure callback to print seed and replay instructions on failure
- Update CI to run sample on PRs, keep full campaign scheduled nightly
@boalambo
boalambo requested a review from truthixify as a code owner October 17, 2026 11:33
@drips-wave

drips-wave Bot commented Oct 17, 2026

Copy link
Copy Markdown

@boalambo Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@truthixify truthixify left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bounded PR fuzz sample passes all 12 property tests and keeps the large run in nightly CI. Approved.

@truthixify truthixify left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two regressions remain in the harness change. package.json now contains duplicate test:leak keys, so one definition is silently discarded. Also onFailure is not a fast-check Parameters option, so the new callback is ignored and does not print the promised replay command. Remove the duplicate script and use fast-check's supported reporter/run-details API (including seed and counterexample path) with a deliberately failing test that proves the replay output is emitted.

- Remove duplicate test:leak script in package.json
- Replace invalid onFailure callback with valid reporter parameter
- Add FC_PATH support for replay with counterexample path
- Add proof test (gated by FC_PROOF_TEST env var) to demonstrate replay output
- Add explicit type annotation to proof test parameter
@boalambo

Copy link
Copy Markdown
Contributor Author

Two regressions remain in the harness change. package.json now contains duplicate test:leak keys, so one definition is silently discarded. Also onFailure is not a fast-check Parameters option, so the new callback is ignored and does not print the promised replay command. Remove the duplicate script and use fast-check's supported reporter/run-details API (including seed and counterexample path) with a deliberately failing test that proves the replay output is emitted.

Fixed in 2c5ecfe. Removed the duplicate test:leak (my "Accept both" merge mistake), replaced onFailure with fast-check's reporter, and added a gated proof test (FC_PROOF_TEST=1). Sample run passes 12/12, and the proof test fails with exit 1 and prints the seed (42), path (0:0) and replay command. Ready for re-review.

@boalambo
boalambo requested a review from truthixify October 18, 2026 12:50

@truthixify truthixify left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The revision removes the duplicate script, replaces the unsupported fast-check onFailure option with a valid reporter, and carries seed plus counterexample path into replay. The fixed-seed sample (12 properties), build, and public type tests all pass locally.

@truthixify
truthixify merged commit 27e6f36 into wraith-protocol:develop Oct 18, 2026
25 checks passed
@boalambo
boalambo deleted the fix/stellar-fuzz-harness branch October 18, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Wave 10] Stabilize the Stellar fuzz harness and add a pull-request sample

2 participants