Skip to content

fix(ci): add vitest to Web leg and retry transient audit failures - #169

Closed
sun-970 wants to merge 11 commits into
bytefolk:mainfrom
sun-970:fix/ci-web-vitest-and-audit-retry
Closed

sun-970 wants to merge 11 commits into
bytefolk:mainfrom
sun-970:fix/ci-web-vitest-and-audit-retry

Conversation

@sun-970

@sun-970 sun-970 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Problem and result

Refs #122 for CI/release-readiness follow-up only; this does not complete or publish that release. The Web CI job previously omitted unit tests, and transient registry failures could fail the audit step. This PR runs the Web tests and retries only recognized transient audit failures, with three attempts (two retries) per threshold. Real findings, unknown errors, missing processes, signals and timeouts remain failures.

The current follow-up also preserves successful audit output, including findings below the configured threshold. Before this fix a status-0 audit discarded stdout/stderr; the new regression reproduced that loss. Audit thresholds remain production/moderate and all-dependencies/high, with a 60-second process limit per attempt and 10-second backoffs.

Changes and provenance

  • Run npm test in the Web CI job and collect audit-retry.test.mjs with the application tests.
  • Invoke npm through Node and its CLI path on Windows, use portable backoff, and keep deterministic retry, error, signal, timeout and process-fixture coverage.
  • Preserve reports from both successful audits; document attempts accurately under [Unreleased].
  • Integrate the exact lockfile refresh from chore(web): refresh audited development dependencies #192, source commit 11e02e21ef2c3dbd2dae26e4376872e54e78ecb5, cherry-picked with -x as a7c17ec. It fixes the common audited baseline: js-yaml 4.3.2, Vitest/@vitest packages 4.1.11, and postcss-selector-parser 6.1.4. The old lockfile reproduced four findings including the high-severity js-yaml advisory; this was a real dependency finding, not a transient error.
  • Fix the Windows evidence batch helper to call npm run audit and return the saved status on the same line as endlocal. A native Windows process regression checks successful and failing audits and replays both original failure modes as negative controls. It runs in the existing Node 24 Windows job.
  • Follow-up implementation head: 5c3a4a75ab96bed4c046c0822ebc76a87f215cda. Web validation was executed at d91deff; the subsequent commit changes only the Windows helper, its process test and its CI invocation.

Validation ledger

From web/, Node 24.13.0, npm 11.6.2, public npm registry, checked-in lockfile. Linux aarch64 validation uses an exact git archive of the implementation head and dependencies installed with npm ci; no service or credential is required by these tests.

Acceptance Command Expected / observed
Locked dependencies install npm ci --registry=https://registry.npmjs.org --fetch-timeout=45000 PASS, 401 packages installed, zero audit findings
Both audit thresholds remain enforced npm run audit --registry=https://registry.npmjs.org PASS, both thresholds succeed on attempt 1, both reports printed, zero vulnerabilities
Audit and application regressions npm test PASS on Linux, 7 files / 104 tests, including 37 audit tests
Successful reports remain visible npx --no-install vitest run audit-retry.test.mjs with the new regression before the implementation fix RED as expected: one failed / 36 passed; empty stdout contradicted the expected audit reports
Static validation and production build npm run lint && npm run typecheck && npm run build PASS on Linux and macOS; existing large-chunk build warning remains
Windows evidence helper node --test scripts/test_win_audit_verify.mjs in the Windows CI job PASS, 4/4 on real Windows, including both original failure-mode negative controls; job evidence
Diff integrity git -c core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol diff --check PASS; the Windows .bat intentionally retains CRLF

The original PR's process fixture (78b3d2d) also hits its 5-second CLI timeout on this macOS host with the same dependencies. The follow-up reproduces that host-dependent result; the unmodified timeout tests pass on Linux. The test was not weakened. A first macOS tar transport included AppleDouble metadata and caused six extra parse failures; the final Linux result above uses a clean exact Git archive (7 test files), not that invalid transport.

Boundaries and pending evidence

  • Exact-head GitHub CI passed on 5c3a4a75ab96bed4c046c0822ebc76a87f215cda: all 16 distinct non-skipped check names succeeded; CodeQL remains skipped under the existing fork policy. Component CI, Agent-memory acceptance, security, and PR Policy all passed. Final-head Web CI reports 104 tests including 37 audit regressions; Windows CI reports four helper regressions. These are CI results, not independent approval.
  • Real Windows audit execution has not been repeated in this follow-up. The committed process fixtures and Windows launch-path tests are distinct from real Windows execution.
  • API, CLI/MCP contracts, authorization, storage and migration semantics are unaffected. No new Go executable was executed on macOS. No governance rule, audit threshold or test was disabled.
  • No release, merge, tag or formal approval is part of this follow-up. Reverting the two follow-up commits restores the preceding branch head; dependencies must still satisfy the audit gate. Temporary validation trees can be discarded; no user data or services are mutated.

The Web CI job ran npm audit but never ran npm test, leaving the six
vitest files under web/src/ without pipeline coverage. The audit step
also failed the entire Web leg on the first transient registry error
(503, timeout, ECONNRESET) with zero retries.

- ci.yml: add "Run unit tests" step (npm test) after audit
- audit-retry.mjs: distinguish network errors (retry up to 3x with
  10s backoff) from real vulnerability reports (fail immediately)
- web/package.json: route audit script through audit-retry.mjs
@waterbro-8

Copy link
Copy Markdown
Collaborator

Not a review vote, and nothing here is an approval, a request-changes, or an acceptance — it is a local execution report, offered because this head has never had CI run on it (see "CI state" below), so the third Test plan box cannot be checked by anyone reading the thread yet.

Disclosure first: I opened #165/#168 and I posted today on #158 about the same Audit dependencies failure, so I have a view on this problem. I have no code in this diff, and the diagnosis below is mine independently, not a review of yours.

What I ran, and its provenance

Pulled 42c57328364 from your fork as a tarball, then in the extracted tree git init + git add -A + git write-tree:

  • whole tree 64f825f6381604f48738f5b3d9cf75b847a9aa46equal to GET /repos/bytefolk/mem/commits/42c57328364.commit.tree.sha;
  • web subtree 724517431707bbbfa97a5913155dcb6b1adcb724equal to the same path entry at your head.

So the files I exercised are byte-identical to this PR's head. Node 24.19.0, vitest 4.1.10, Linux. node_modules is hardlinked from a local web/ install whose package.json is identical to main@7a194f1eba4 (checked), so the dependency graph under test is the one CI would install, not a private variant.

CI state on this head

GET /repos/bytefolk/mem/commits/42c57328364/check-runs0 check runs. A fork head needs a maintainer to approve the pending workflow run, there is no API for that (gh run has no approve subcommand; GET …/actions/runs/<id>/approve returns 404 — the route does not exist), and it is a Web-UI click only. So - [ ] CI verify workflow passes on this PR is not stale, it is genuinely un-executed.

Confirmed: the premise is live, and it is not a vulnerability

  • main@7a194f1eba4's own CI run 33854785355 (attempt 1) = failure, with exactly one failed job Web and one failed step Audit dependencies.
  • The log text is npm warn audit network timeout at: https://registry.npmjs.org/-/npm/v1/security/advisories/bulknpm error audit endpoint returned an error. No advisory is reported. The same two lines are what build(deps): bump google.golang.org/grpc from 1.82.1 to 1.83.2 in /server #158 was red for, and that diff is server/go.mod + server/go.sum only, so it cannot reach the web dependency graph.
  • I re-ran the failed job while writing this (gh run rerun 33854785355 --failed): attempt 2 = completed / success, closed 2026-09-06T03:12:27Z, with no change to main. Same head, same code, different outcome — which is the definition of "transient" rather than "broken", and is the evidence your Context section asserts.

Confirmed: your classification actually catches that failure, and it is fail-closed on exhaustion

Ran node audit-retry.mjs in web/ with the real npm, real registry, and NPM_CONFIG_FETCH_TIMEOUT=1 to force the same timeout:

  • attempt 1/2/3 each logged hit a network error — will retry, ~10 s apart, then exhausted 3 attempts, then the final pass-through run → exit 1.

Two other failure modes land in the same bucket, which I did not expect and is worth knowing: with NPM_CONFIG_REGISTRY pointed at a closed port the underlying reason is connect ECONNREFUSED, and at an unresolvable host it is getaddrinfo ENOTFOUND — neither string is in NETWORK_PATTERNS, yet both are retried, because npm appends audit endpoint returned an error to stderr on every endpoint failure and that one is in the list. So classification does not depend on enumerating errnos. Both also exit 1 after exhaustion.

And with a stub npm that emits # npm audit report plus a non-zero exit, the script logs found real vulnerabilities — not retrying, runs exactly one attempt, echoes the report to stdout, and exits 1. That branch is load-bearing, not decorative.

Confirmed: the vitest step closes a real hole, and it is green here

web/package.json already declares "test": "vitest run" and vitest ^4.1.10, and main's web job (ci.yml:231-266) goes install → audit → lint → typecheck → build with no npm test; the two npm test steps in main's CI are working-directory: npm, not web. So the existing unit tests never execute in CI. On this head: 6 test files passed / 67 tests passed / 0 failed. The new step would be green, and it is coverage the repo does not have today.

One defect, measured: a non-numeric exit status becomes exit 0

result.status is null when the child is killed by a signal or cannot be spawned. Both non-success returns in runWithRetry are return result.status;, so exitCode becomes null, null !== 0 is true, and the script finishes with process.exit(null) — which Node coerces to 0. Control measurement: node -e 'process.exit(null)'rc=0; node -e 'process.exit(undefined)'rc=0.

npm behaviour this head same file with return result.status ?? 1; at both sites
killed by SIGKILL mid-audit exit 0 exit 1
cannot be spawned at all (ENOENT) exit 0 exit 1
real vulnerability report exit 1 exit 1
network error, 3 attempts exhausted exit 1 exit 1

Why that matters here rather than being theoretical: the step it replaces is run: npm audit … && npm audit …, which Actions executes through a shell, and a shell reports a signal-killed child as non-zero — measured sh -c '/bin/kill -KILL $$'rc=137. So on the path where a runner OOM-kills npm, the old step was red and the new one is green. That is a flip from fail-closed to fail-open on the repository's dependency-security gate, introduced by the file that is supposed to make that gate reliable.

The fix is the shape the file already uses one block lower: the exhaustion path returns final.status ?? 1. Carrying the same ?? 1 to the two return result.status; sites turns both rows red, which is what the mutant column above measures. (Guard on the mutation itself, because an edit that silently fails to land produces the same "fix works" story: diff -q pristine vs mutant must differ, and it did — 2 sites replaced.)

Two notes I would not block on

  • spawnSync("sleep", [String(BACKOFF_MS / 1000)]) finds no sleep on Windows, so the backoff degrades to zero delay there. The web job is ubuntu-24.04, so no CI consequence today; only relevant if someone runs npm run audit locally on Windows and wonders why it hammers the endpoint three times instantly.
  • When npm audit exits 0, the wrapper returns before touching stdout, so a report for findings below the threshold is no longer printed. Measured: stub with # npm audit report on stdout and exit 0 produced empty captured stdout. The gate decision is unchanged (--audit-level still governs the exit code, same as the old && chain), but the old inline step showed that text in the log and this one hides it. One process.stdout.write(stdout) on the success path would keep the visibility.

Sequencing, not a criticism of this PR

CHANGELOG.md is touched by five open PRs at once — #140, #162, #165, #168, #169 — all inserting under ## [Unreleased]. Today only #140 reports mergeable_state=dirty; the other four are clean, which just means they are racing. Whichever lands second onward will need the same one-hunk resolution (keep both sides, ### Fixed entries are not mutually exclusive), so it is worth expecting a rebase rather than treating the next conflict as a defect in whoever hits it.

Happy to re-run anything above on a new head; the trees are laid out so a re-run is one command.

@PeterGuy326 PeterGuy326 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking review findings:\n\n1. executes a fourth after the loop has already exhausted . That makes the attempt count and the logged evidence disagree, and it can extend CI beyond the stated retry budget. Remove the extra invocation or make the intended total/retry count explicit and test it.\n\n2. The new retry helper has no script-level test for: transient failure then success, vulnerability failure without retry, non-network failure without retry, and exhaustion. The PR body leaves CI verification unchecked and the fork PR currently has no check run, so this cannot be approved yet.\n\nPlease update the helper and add deterministic coverage, then obtain a fresh CI run on the PR head.

@PeterGuy326 PeterGuy326 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking review findings:

  1. web/audit-retry.mjs:87-91 executes a fourth npm audit after the loop has already exhausted MAX_ATTEMPTS = 3. That makes the attempt count and the logged evidence disagree, and it can extend CI beyond the stated retry budget. Remove the extra invocation or make the intended total/retry count explicit and test it.

  2. The new retry helper has no script-level test for: transient failure then success, vulnerability failure without retry, non-network failure without retry, and exhaustion. The PR body leaves CI verification unchecked and the fork PR currently has no check run, so this cannot be approved yet.

Please update the helper and add deterministic coverage, then obtain a fresh CI run on the PR head.

@wadrzl

wadrzl commented Sep 6, 2026

Copy link
Copy Markdown

PR #169 验证结果

分支fix/ci-web-vitest-and-audit-retry

验证情况

项目 结果
npm test(vitest) ✅ 6 个文件,67 个用例全部通过
audit-retry.mjs 重试逻辑 ✅ 正常工作,3 次重试 + 10s 退避均按预期执行
npm audit 最终结果 ⚠️ 失败,但原因是本地镜像源(npmmirror)不支持 audit 接口,非代码问题

结论

代码改动没有问题,测试覆盖完整,重试机制符合预期。在 GitHub CI(使用官方 npm registry)环境下 audit 应能正常通过。可以合并。

@waterbro-8

Copy link
Copy Markdown
Collaborator

Addendum to my earlier comment on this PR (5556561886, posted 2026-09-06 03:13:20Z from this same account). Not a review vote, not an approval, not an acceptance, and nothing below changes that comment's conclusions — it is five more things I measured, one of which corrects how benign I described the CHANGELOG situation. Disclosure: #168 is my own PR, so item 4 is not a neutral observation; read it as mine.

1. The no-retry branch does fire on real npm output — but one string is carrying it

I built a genuine advisory fixture instead of a hand-written stub: npm i axios@0.21.1 in a scratch package, npm 12.0.2, --registry=https://registry.npmjs.org, then replayed the captured bytes through your script with a stub npm on PATH.

  • npm audit --audit-level=highrc 1, 3458 B on stdout (starting # npm audit report, summary line 1 high severity vulnerability), 0 B on stderr.
  • Your script: exactly 1 invocation, found real vulnerabilities — not retrying, exit 1. Intent preserved.

I then tested all three VULNERABILITY_PATTERNS against that real output. Only npm audit report matches. found \d+ vulnerabilities and vulnerabilities found match nothing npm 12.0.2 prints — its summary is 1 high severity vulnerability (singular, no found prefix). So two of the three entries are dead weight and the whole "fail immediately instead of retrying" decision rests on a single header string that npm has no contract to preserve. Not asking you to change it; it is the reason for item 3.

2. When npm cannot be spawned, the step goes green and the log does not say why

Extending the non-numeric-exit item above with what actually lands in the log. Two runs of audit-retry.mjs on this head:

npm behaviour wrapper exit wrapper stdout wrapper stderr
child killed by SIGKILL 0 0 B … (attempt 1/3) then … failed with unrecognized error — not retrying.
not on PATH (PATH=/usr/bin:/bin) 0 0 B the same two lines, verbatim

result.error and result.signal are never read, so the one thing that would name the cause (spawnSync npm ENOENT, signal: 'SIGKILL') is discarded, and the message printed says "unrecognized error" about a failure that result describes exactly. Checked that the fix does not over-correct: with return result.status ?? 1; at both sites, SIGKILL → rc 1 and the real-advisory replay is still rc 1. Printing result.error ?? result.signal is the half that makes the next person's diagnosis not-a-mystery.

3. This file is invisible to every gate in the repo, so "add a test" is not a one-liner

Ran the other web legs on this head with web/node_modules installed: npm run lint rc 0, npm run typecheck rc 0, npm run build rc 0 — none of them look at audit-retry.mjs:

  • lint is eslint . --ext .ts,.tsx.mjs excluded;
  • tsconfig.app.json is include: ["src"] and tsconfig.node.json is include: ["vite.config.ts"] → a root-level .mjs is outside both;
  • vitest — the step this PR is adding to CI — is include: ['src/**/*.{test,spec}.{ts,tsx}'], so web/audit-retry.test.mjs would not be collected; and the module has no exports, so a test placed inside src/ cannot import the classifiers either.

So items 1 and 2 are unenforceable as the file stands. The short version: export { isNetworkError, isVulnerabilityReport } behind a guarded main() call, plus one web/src/lib/*.test.ts pinning (a) null status → non-zero, (b) the replayed advisory bytes → one attempt. Then the CI step you just added is what enforces them.

4. Correcting myself: the CHANGELOG merge will not conflict, and that is the problem

My comment above said landing second "will need the same one-hunk resolution". I have now executed it. Three-way merge, real merge base 7a194f1eba4, #169 42c57328364 x my #168 8a92baa680b: git merge-file -p --diff3rc 0, zero conflict markers. The merged file reads:

 8  ## [Unreleased]
10  ## [0.1.2] - 2026-09-06
18  ### Fixed
20  - CI Web job now runs vitest unit tests ...   <- your entry

## [Unreleased] is left empty and your entry is filed inside ## [0.1.2], which #168's body defines as "exactly 7 merged commits, the newest being 7a194f1" — #169 is not one of them. Because the merge is textually clean, no one gets prompted. If #168 lands first, the resolution is to move this block back under ## [Unreleased], not "keep both sides"; if #169 lands first, #168's commit list needs an 8th entry. Neither is a defect in this PR — I am flagging it because mergeable_state=clean is what my earlier sentence leaned on, and it means less than I wrote.

Re-checked the contention set live for the record: of the 10 open PRs, CHANGELOG.md is touched by #140 / #162 / #165 / #168 / #169 (only #140 reports dirty), while .github/workflows/ci.yml is touched by #169 alone — so unlike the CHANGELOG, the workflow edit has no competitor.

5. What this head has as evidence, as of 05:49Z

GET /repos/bytefolk/mem/actions/runs?head_sha=42c57328364… → 3 runs (CI, Validate Agent memory, PR Policy), all completed / action_required at run_attempt 1. So the number of CI checks that have executed against this head is still zero — what my first comment said at 03:13Z, still true 2.5 h later. pulls/169 reports mergeable=true / mergeable_state=blocked.

There is now also 5557259936, a local verification report plus a merge-readiness verdict. Its two numbers (vitest 6 files / 67 tests; 3 retries with the 10 s backoff) reproduce exactly in my runs, so that execution is real. Two limits on what it can show, stated as measurements:

  • Against registry.npmmirror.com, npm prints { error: '[NOT_IMPLEMENTED] /-/npm/v1/security/* not implemented yet' } and appends npm error audit endpoint returned an error. That last string is in NETWORK_PATTERNS, so every run against that mirror necessarily goes retry → retry → retry → exit 1. The mirror path cannot reach the pass branch and cannot reach the real-vulnerability branch, so it is not evidence that the gate still gates. On registry.npmjs.org against your own web graph: both commands pass on attempt 1, exit 0 (item 1 and the last paragraph below) — that is the leg CI will take.
  • "Retry logic works as expected" measures the retry counter. The counter is not where the risk in this file sits; item 2 is, and it needs npm to produce no result object at all, which no ordinary run reaches.

I am reporting what the head does and does not have executed, not adjudicating anyone's authority to comment on it.

Corroborations, so the earlier numbers are not single-sourced

npm ci in the web job carries no --omit=dev, so vitest is installed; "test": "vitest run" is non-watch; on this head npm test = 6 files / 67 tests / 0 failed in 1.54 s; main's two npm test steps are both working-directory: npm (ci.yml:294, :344), i.e. the web leg genuinely had no unit-test execution. Happy path of your wrapper on the real web dependency graph with NPM_CONFIG_REGISTRY=https://registry.npmjs.org: both commands pass on attempt 1, exit 0 — and raw npm prints found 0 vulnerabilities (24 B) that the wrapper does not forward. Worth one process.stdout.write(stdout) on the success path. Related measured fact: the exit-1 and exit-0 runs of the same axios fixture produce byte-identical stdout (3458 B), which is why stdout text cannot be the gate signal and --audit-level's exit code has to stay the only authority.

@sun-970

sun-970 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor Author

已在钉钉平台组通知有权限的协作者(PeterGuy326、Bindy-lbb、jnn111、waterbro-8)去 approve 这 3 条 workflow runs。等待 CI 结果。

spawnSync returns null status when npm is killed by signal or not found
on PATH. Three return sites in runWithRetry passed null through to
process.exit, which Node treats as 0 — failing the audit gate silently.

Add `?? 1` at the two unguarded sites (:67, :85) and remove the
redundant 4th spawnSync after loop exhaustion (review finding #1),
reusing the last loop result with the same `?? 1` guard. All three
sites now aligned.
@waterbro-8

Copy link
Copy Markdown
Collaborator

Verification notes on #169this is not a review vote and not an acceptance. I submitted neither APPROVED nor REQUEST_CHANGES here, I'm not asking for changes as a gate, and I touched nothing else (no merge, no label, no assignee, no update branch). Findings below are stated with the evidence level each one has: E1 = I executed it, E3 = read from the diff only.

Read live at 2026-09-08 10:5x local: head 65da42bc2725, base 7a194f1eba4, 2 commits / 4 files / +116 −1, mergeable=true, mergeable_state=behind (main has since moved to 2986fe38175), 15/15 check names success on this head (latest round per name), and the three workflow runs on this head are completed / success at attempt=2. So the "fresh CI run on the PR head" half of the existing review feedback is satisfied.

E1: what this PR genuinely fixes

  1. The vitest leg is real new coverage. In web/: 6 test files, 67 tests, duration 2.65 s, rc=0. Grepping the pre-PR CI job logs, vitest never appears in any leg — so before this PR the frontend had tests that CI never ran. That alone is worth having.
  2. The ?? 1 added in the second commit is load-bearing. I fed the script an npm that gets SIGKILLed (result.status === null): current head exits rc=1; with the three ?? 1 reverted, the identical injection exits rc=0. Before that commit, "npm never actually ran" was reported as an audit pass.
  3. Classification and backoff behave, and every non-zero path fails closed. Vulnerability output → no retry; network output → 3 attempts with two 10.0 s sleeps, then fail; unrecognized output → fail. I also measured why the script can't just be replaced by npm config: with fetch-retries=5 a plain registry GET produces 6 requests while the audit POST /-/npm/v1/security/advisories/bulk produces 1 — npm's built-in retry does not cover the audit endpoint. The script is not redundant.

Findings

F1 (E1, and the thing the existing review asked for): web/audit-retry.mjs has no test at all. Grepping the whole repo for audit-retry hits only the file itself and the web/package.json script line; there is no web/tests directory. So a wrong classification table, an inverted retry count, or a dropped ?? 1 would all ship green. This PR just added a vitest leg that would pick the file up for free — web/audit-retry.test.mjs covering the three branches (vuln → single attempt, network → 3 attempts, unknown → fail) is a small addition, and the commit that added ?? 1 is exactly the regression a test would have caught.

F2 (E1): the premise in the PR description is overstated — the retry addresses 1 of 4 sampled failures. I read the four red Audit dependencies job logs: 1 high (09-02), 2 (1 low, 1 high) (09-02), 2 (1 low, 1 high) (09-03) are genuine finding-driven reds that no retry can fix; only 09-04 was network timeout at .../security/advisories/bulk (it had one low finding too). Worth narrowing the description to "the transient class" rather than implying the red steps were mostly flakes.

On the earlier comment here reporting "3 retries + 10 s backoff executed as expected": I reproduced that and it is not a misobservation — I pointed npm at registry.npmmirror.com, it emits npm error audit endpoint returned an error, which is one of NETWORK_PATTERNS, so the retry loop does engage. Two caveats I'd rather state than gloss: (a) that is a permanent 404 on a mirror-configured machine, whereas the CI failure is a self-healing 300 s timeout — the same branch is being exercised by two different mechanisms, so it doesn't tell us the retry survives the real one; (b) the accompanying claim that test coverage is complete doesn't hold — see F1.

F2b (E1, side effect of the same shape): on a machine whose registry doesn't serve audit, npm run audit goes from "fails immediately" to "sleeps 20 s, tries 3 times, reports the same error". Not a blocker; mentioning it so nobody is surprised.

F3 (E1): a second ### Fixed inside [Unreleased]. On this head the section order is Changed(10) → Fixed(16) → Security(23) → Fixed(52), where main has Changed/Security/Fixed. scripts/render_release_notes.sh will emit two ### Fixed headings with Security between them, and validate_release_version.sh asserts version strings and link endpoints, not section presence — so neither gate goes red. This repo's precedent for the same collision (#137, #140) is: merge the bullet into the existing ### Fixed list rather than create a second heading.

F4 (E1, on real Windows): this turns a working audit gate into a permanently red one on win32, and Linux CI cannot see it. On this machine's Windows side (node.exe v24.16.0), staging #169's own web/package.json + package-lock.json and running the head script directly:

[audit-retry] ... — production dependencies (moderate threshold) (attempt 1/3)
[audit-retry] production dependencies (moderate threshold) failed with unrecognized error — not retrying.
rc=1   (0.21 s, one attempt)

spawnSync("npm", ...) is ENOENT on win32 (npm.cmd is EINVAL, and spawnSync("sleep", ...) is also ENOENT). A failed spawn leaves stderr empty, so classification lands on "unrecognized" and the script exits on attempt 1 — it does not even reach the retry path, and the message points at the wrong cause. Same directory, same arguments, only the spawn changed to process.execPath + npm's npm-cli.js:

status=0  stdout: "found 0 vulnerabilities"

Before this PR, npm run audit worked on Windows because npm resolves its own cmd shim; after it, the gate is always red there while Linux stays green. Suggested shape:

const cli = process.env.npm_execpath;
result = spawnSync(cli ? process.execPath : "npm", cli ? [cli, ...args] : args, {...});
// and for the backoff, drop spawnSync("sleep"):
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, BACKOFF_MS);

Evidence-level honesty: the Atomics.wait swap I measured on win32 (1505 ms actual sleep), and the node+npm-cli.js spawn above I measured on win32 — but I could not drive it through a real npm run audit on Windows, because my WSL→Windows harness swallows npm_execpath (the variable arrives undefined), so I passed the CLI path by argv instead. The npm_execpath half needs one run from a Windows terminal to be E1 end-to-end.

F5 (E3): the retry budget is unbounded relative to the job timeout. Worst case is 3 × 300 s (npm's default fetch-timeout) + 20 s backoff = 15m20s inside a web job declared timeout-minutes: 20. One slow audit converts a single red step into a cancelled job, losing lint / type-check / build for that run too. continue-on-error is the wrong lever (it would make a security gate decorative); either bound the call (timeout: 120_000) or move Audit to be the last step in the job.

F6 (E3, prose): the description says "3 retries"; the implementation is 3 attempts = 2 retries. And Refs #122 points at the p0 release-prep issue, which has nothing to do with audit flakiness — there is currently no issue in the repo about it, which is probably why the link had to borrow an unrelated number.

State, for whoever rebases next

main has moved to 2986fe38175 (the two bytefolk-* workflow files) since this branch was cut, so #169 is now behind. On updating: the only file with a positional hazard is CHANGELOG.md (F3 — the fix there also resolves the collision), and note that on a fork head the new CodeQL leg is gated off, so its absence from a future run set is the fork policy, not a defect in this diff.

One process fact so nobody wastes a cycle: the two CHANGES_REQUESTED reviews sit on the previous head 42c5732836 and, in this repo, pushing does not dismiss CHANGES_REQUESTED (unlike APPROVED — #169 demonstrated both behaviors). Regardless of how clean the next revision is, that gate is @PeterGuy326's to clear.

PeterGuy326 and others added 3 commits September 8, 2026 11:19
The 60s ATTEMPT_TIMEOUT_MS kills npm before its default 300s
fetch-timeout fires, so the retry logic never sees "network timeout"
for slow registry hangs. Adding --fetch-timeout=45000 to both audit
args lets npm report the timeout within the 60s budget, making
isNetworkError catch it and trigger retries as designed.
- Assert vite.config.ts includes audit-retry.test.mjs so removing it
  fails the suite instead of silently dropping 35 tests.
- Assert --fetch-timeout < spawn timeout so the dead-zone fix stays
  enforced if either constant drifts.
Captures git head, npm_execpath probe, full npm run audit output,
and exit code. Must be run from cmd.exe (not WSL) to prove the
win32 branch in audit-retry.mjs:117 receives npm_execpath.
Windows cmd.exe requires CRLF; LF-only caused every line to be
misread as an unknown command.
@PeterGuy326

Copy link
Copy Markdown
Collaborator

HOLD — no approval on this follow-up yet. I re-ran web/audit-retry.test.mjs on current head 78b3d2daa1b71230572fe64ecd249f110b210580: 36/36 passed, but gh pr checks reports no checks for this head. The prior changes-requested reviews are at the old commit, so please run the repository CI against this exact head before requesting a fresh approval.

@PeterGuy326

Copy link
Copy Markdown
Collaborator

Current-head CI approval readback (2026-09-10)

Head: 78b3d2daa1b71230572fe64ecd249f110b210580. The original pending-approval runs on this head have been approved and real check-runs now exist (17 at this snapshot). No old-head run was approved. Auto-merge remains disabled; this is not a code approval or merge-ready declaration.

Failures at this snapshot:

The Web job reached the real dependency audit and failed on the high-severity js-yaml advisory (GHSA-2883-xcg3-v3hh), not a transient registry error. The separate audited-dependency candidate is #192.

@PeterGuy326

Copy link
Copy Markdown
Collaborator

Canonical DRAFT successor: #198, preserving this exact head 5c3a4a7 and its complete author ancestry/tree. Current main 2986fe3 is already included. This uses the existing same-repository CodeQL path without changing security permissions or fork restrictions. This original PR remains open and unchanged; no review approval or merge is implied. Fresh Linux validation and the exact-head ledger are in #198.

@sun-970

sun-970 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by @PeterGuy326's #198 which covers the same changes on current main. Closing per the '本轮先不合并' directive and to avoid duplicate review surface.

@sun-970 sun-970 closed this Sep 10, 2026
PeterGuy326 added a commit that referenced this pull request Sep 17, 2026
#198)

## Tracking record and provenance

Refs #122 — bounded CI/release-readiness follow-up, not completion or
publication
of that release. Canonical draft successor to #169, with the original
author's
`sun-970` commits and maintainer follow-ups retained unchanged in
ancestry.
The original PR stays open; its reviews and discussion remain
authoritative
history, not an approval of this successor.

- Original/final source head:
`5c3a4a75ab96bed4c046c0822ebc76a87f215cda`.
- Successor head: `5c3a4a75ab96bed4c046c0822ebc76a87f215cda`.
- Source and successor tree: `9d8a127375fd8e945113e06ae88983a1328b9fae`.
- Current canonical main: `2986fe38175f54d99f15dd38a498708c6ecd88cd`,
already
an ancestor. A normal merge reports `Already up to date`; no empty
commit,
  cherry-pick reconstruction, rebasing, or force push was used.
- The audited lockfile retains #192's provenance from
`11e02e21ef2c3dbd2dae26e4376872e54e78ecb5` (`cherry-pick -x` in
ancestry).

## Summary and acceptance criteria

The Web CI job must run its unit tests. Recognized transient npm audit
failures
may retry, but vulnerabilities, unknown errors, failed spawns, signals
and
timeouts must remain failures. Successful audit output must remain
visible,
including below-threshold findings. The Windows evidence helper must
complete
its report and preserve a nonzero audit exit status.

This successor changes no source file relative to #169's exact source
head.
It exists because the canonical security workflow skips the entire
CodeQL job
on fork PRs while main requires three language-specific CodeQL contexts.
A
same-repository candidate executes the existing supported path; no
security
workflow, permission, required check, audit threshold, or fork
restriction is
modified or waived.

Changed files against main: `.github/workflows/ci.yml`, `CHANGELOG.md`,
`scripts/test_win_audit_verify.mjs`, `scripts/win-audit-verify.bat`,
`web/audit-retry.mjs`, `web/audit-retry.test.mjs`, `web/package.json`,
`web/package-lock.json`, `web/vite.config.ts`.

## Validation ledger

Environment: isolated Linux aarch64, Linux 6.8, GNU find 4.9.0/coreutils
9.4,
Bash 5.2.21, Node 24.13.0/npm 11.6.2; checksum-verified official Node
archive.
Fresh Git clone from an exact local bundle, two CPUs, GOMAXPROCS=2 and
GOFLAGS=-p=1. Tests use disposable fixtures, not production data or
secrets.

| Command/check | Expected | Actual |
| --- | --- | --- |
| `git diff --exit-code 5c3a4a7 HEAD` |
Exact source tree | PASS, no differences |
| `git merge-base --is-ancestor 2986fe3
HEAD` | Current main included | PASS |
| From root: `bash scripts/test_release_guards.sh` | Release guards fail
closed | PASS |
| `bash scripts/test_release_helpers_compat.sh` | Collection
compatibility | PASS |
| `bash scripts/test_validate_release_action_pins_compat.sh` | Official
pinned actions and compatibility | PASS |
| `bash scripts/validate_release_version.sh 0.1.1` | Version surfaces
agree | PASS |
| From `web/`: `npm ci --registry=https://registry.npmjs.org
--fetch-timeout=45000` | Locked install | PASS, 401 installed packages |
| `npm run audit --registry=https://registry.npmjs.org` | Both unchanged
thresholds pass | PASS, zero vulnerabilities, both reports printed |
| `npm test` | Application and audit regressions pass | PASS, 104 tests
/ 7 files, including 37 audit regressions |
| `npm run lint` / `npm run typecheck` / `npm run build` | Static
validation and production build pass | PASS, existing large-chunk
warning only |
| `git -c
core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol
diff --check 2986fe3 HEAD` | Clean
diff, intentional batch CRLF retained | PASS |
| Canonical successor CI and CodeQL | All required named contexts
execute and pass | PENDING at draft creation; no all-green claim |

## Tests, coverage and known limits

The original successful-report regression was red before the fix (1
failed /
36 passed). Existing exact-source [Web
CI](https://github.com/bytefolk/mem/actions/runs/34438230117/job/102747885693)
passed 104 tests including 37 audit regressions. Existing real
[Windows
CI](https://github.com/bytefolk/mem/actions/runs/34438230117/job/102747885554)
passed four native batch-helper regressions, including negative controls
for
both original defects. These are source-head evidence, not independent
review
or substitute results for the new canonical workflow execution.

Coverage percentage was not remeasured. A real-registry Windows audit
has not
been requalified; its helper tests use local npm.cmd fixtures. The
unchanged
five-second process fixture timeout reproduces on both pre-follow-up and
fixed
heads on the managed macOS host; Linux is the validated environment. The
existing large-chunk build warning remains. Full Linux process/database
and
browser acceptance, and native Windows execution, are delegated to
unchanged
CI; no temporary Go executable runs on macOS.

## Risk, rollback and review

Build/CI/dependency maintenance; no application API, CLI/MCP
authorization,
storage or migration contract changes. `[Unreleased]` already documents
the
behavior. The PR-triggered workflows reference no custom repository
secrets;
CodeQL retains only its existing security-events upload permission. No
release,
npm publication, Scorecard dispatch, tag, merge, or formal approval is
performed.
Rollback is to leave this draft unmerged; no deployed behavior changes.

Draft until the exact candidate has required CI plus independent review.
The
Owner reports independent review of the fix deltas is already in
progress;
this description does not represent that review as completed. Automated
assistance produced validation and the maintainer follow-up; the
original
human-authored history remains intact.

---------

Co-authored-by: liyuanyang <liyuanyang@users.noreply.github.com>
Co-authored-by: waterbro-8 <318569545+waterbro-8@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants