Status against the Flakiness Report feature checklist.
| # | Feature | Status | Notes |
|---|---|---|---|
| 1 | Report metadata | Supported | Commit, project, CI URL, optional config path, timestamps, producer, runner, and runtime are populated. |
| 2 | Environment metadata | Supported | Environment name and OS name/version/arch come from @flakiness/sdk. |
| 3 | Multiple environments | N/A | node:test has no project/environment abstraction within one run. |
| 4 | Custom environments (FK_ENV_*) |
Supported | Parsed by ReportUtils.createEnvironment(). |
| 5 | Test hierarchy / suites | Supported | File and named suites are preserved. Test-owned subtests use an anonymous suite. Node does not emit children hidden by a skipped suite. |
| 6 | Per-attempt reporting | Partial | Node 24.7+ exposes persisted rerun indexes. Prior attempts without details are synthesized with zero duration; replayed durations are retained when Node exposes them. Historical timestamps and worker indexes are not exposed. |
| 7 | Per-attempt timeout | Partial | The effective timeout is recoverable and populated for timed-out attempts. Node does not expose it for successful attempts. |
| 8 | Test steps | N/A | node:test has subtests, not a separate native step API. |
| 9 | Expected status | Supported | Node's expectFailure maps to expectedStatus: "failed" and a fail annotation. |
| 10 | Attachments | N/A | No native attachment API is exposed to reporters. |
| 11 | Step-level attachments | N/A | No steps or attachments. |
| 12 | Timed stdio | Partial | Test-scoped test:log events are timed when available. Current stdout/stderr events are file-scoped and are not guessed onto tests. |
| 13 | Annotations | Supported | skip, todo, expectFailure, and test diagnostics map to annotations with descriptions and locations where available. |
| 14 | Tags | Supported | Native Node tags are consumed on versions that emit them; trailing @tag title tokens are the compatibility fallback. |
| 15 | parallelIndex |
Supported | Derived from concurrently active process-isolated test-file workers. |
| 16 | FLAKINESS_TITLE |
Supported | An explicit factory option takes precedence. |
| 17 | FLAKINESS_OUTPUT_DIR |
Supported | Defaults to flakiness-report. |
| 18 | Sources | Supported | All in-repository locations are embedded with context through the SDK. |
| 19 | Error snippets | N/A | Node does not expose its terminal formatter's snippets to custom reporters. Sources and error locations are emitted instead. |
| 20 | Errors support | Supported | Error objects, primitive/object throws, aggregate errors, stacks, and in-repository locations are handled. |
| 21 | Unattributed errors | Supported | File loading, root after-hook, post-test asynchronous, interrupted file, process, and otherwise unowned run failures are retained. Node 20 and Node 22.9 omit structured locations from global diagnostics. |
| 22 | Source locations | Supported | Tests, suites, errors, and annotations receive git-relative POSIX locations. |
| 23 | Auto-upload | Supported | GitHub OIDC, access tokens, custom endpoint, and disable-upload are delegated to the SDK. A forced interrupted exit can only persist locally. |
| 24 | CPU / RAM telemetry | Supported | CPU average/max/count and RAM series/total bytes are sampled through the SDK. |
| 25 | Duplicate-name handling | Supported | Duplicates fail by default or can be renamed with FLAKINESS_DUPLICATES=rename; affected attempts receive dupe annotations. |
- Node 20 and 22 lack stable test and parent identity fields. The reporter reconstructs hierarchy from declaration order and nesting when those fields are absent.
- Recent Node releases add
testId,parentId, nativetags,expectFailure, persisted rerun metadata, and interruption events at different version boundaries. The reporter consumes each field when emitted. - Node.js
mainalso addsentryFileand test-scopedtest:log; both are consumed when available. - Node 22.10+ per-file summary boundaries preserve the importing entry file for tests declared in shared modules, including concurrent process-isolated runs. Node 20 and Node 22.9 cannot distinguish that ownership and do not guess.
- Watch cycles are reported independently. The output directory always contains the latest completed cycle.