Conversation
The Web CI job ran npm audit but never ran npm test, leaving the six vitest files under web/src/ without pipeline coverage. The audit step also failed the entire Web leg on the first transient registry error (503, timeout, ECONNRESET) with zero retries. - ci.yml: add "Run unit tests" step (npm test) after audit - audit-retry.mjs: distinguish network errors (retry up to 3x with 10s backoff) from real vulnerability reports (fail immediately) - web/package.json: route audit script through audit-retry.mjs
|
Not a review vote, and nothing here is an approval, a request-changes, or an acceptance — it is a local execution report, offered because this head has never had CI run on it (see "CI state" below), so the third Disclosure first: I opened #165/#168 and I posted today on #158 about the same What I ran, and its provenancePulled
So the files I exercised are byte-identical to this PR's head. Node 24.19.0, vitest 4.1.10, Linux. CI state on this head
Confirmed: the premise is live, and it is not a vulnerability
Confirmed: your classification actually catches that failure, and it is fail-closed on exhaustionRan
Two other failure modes land in the same bucket, which I did not expect and is worth knowing: with And with a stub Confirmed: the vitest step closes a real hole, and it is green here
One defect, measured: a non-numeric exit status becomes exit 0
Why that matters here rather than being theoretical: the step it replaces is The fix is the shape the file already uses one block lower: the exhaustion path returns Two notes I would not block on
Sequencing, not a criticism of this PR
Happy to re-run anything above on a new head; the trees are laid out so a re-run is one command. |
PeterGuy326
left a comment
There was a problem hiding this comment.
Blocking review findings:\n\n1. executes a fourth after the loop has already exhausted . That makes the attempt count and the logged evidence disagree, and it can extend CI beyond the stated retry budget. Remove the extra invocation or make the intended total/retry count explicit and test it.\n\n2. The new retry helper has no script-level test for: transient failure then success, vulnerability failure without retry, non-network failure without retry, and exhaustion. The PR body leaves CI verification unchecked and the fork PR currently has no check run, so this cannot be approved yet.\n\nPlease update the helper and add deterministic coverage, then obtain a fresh CI run on the PR head.
PeterGuy326
left a comment
There was a problem hiding this comment.
Blocking review findings:
-
web/audit-retry.mjs:87-91executes a fourthnpm auditafter the loop has already exhaustedMAX_ATTEMPTS = 3. That makes the attempt count and the logged evidence disagree, and it can extend CI beyond the stated retry budget. Remove the extra invocation or make the intended total/retry count explicit and test it. -
The new retry helper has no script-level test for: transient failure then success, vulnerability failure without retry, non-network failure without retry, and exhaustion. The PR body leaves CI verification unchecked and the fork PR currently has no check run, so this cannot be approved yet.
Please update the helper and add deterministic coverage, then obtain a fresh CI run on the PR head.
PR #169 验证结果分支: 验证情况
结论代码改动没有问题,测试覆盖完整,重试机制符合预期。在 GitHub CI(使用官方 npm registry)环境下 audit 应能正常通过。可以合并。 |
|
Addendum to my earlier comment on this PR ( 1. The no-retry branch does fire on real
|
npm behaviour |
wrapper exit | wrapper stdout | wrapper stderr |
|---|---|---|---|
child killed by SIGKILL |
0 | 0 B | … (attempt 1/3) then … failed with unrecognized error — not retrying. |
not on PATH (PATH=/usr/bin:/bin) |
0 | 0 B | the same two lines, verbatim |
result.error and result.signal are never read, so the one thing that would name the cause (spawnSync npm ENOENT, signal: 'SIGKILL') is discarded, and the message printed says "unrecognized error" about a failure that result describes exactly. Checked that the fix does not over-correct: with return result.status ?? 1; at both sites, SIGKILL → rc 1 and the real-advisory replay is still rc 1. Printing result.error ?? result.signal is the half that makes the next person's diagnosis not-a-mystery.
3. This file is invisible to every gate in the repo, so "add a test" is not a one-liner
Ran the other web legs on this head with web/node_modules installed: npm run lint rc 0, npm run typecheck rc 0, npm run build rc 0 — none of them look at audit-retry.mjs:
lintiseslint . --ext .ts,.tsx→.mjsexcluded;tsconfig.app.jsonisinclude: ["src"]andtsconfig.node.jsonisinclude: ["vite.config.ts"]→ a root-level.mjsis outside both;- vitest — the step this PR is adding to CI — is
include: ['src/**/*.{test,spec}.{ts,tsx}'], soweb/audit-retry.test.mjswould not be collected; and the module has no exports, so a test placed insidesrc/cannot import the classifiers either.
So items 1 and 2 are unenforceable as the file stands. The short version: export { isNetworkError, isVulnerabilityReport } behind a guarded main() call, plus one web/src/lib/*.test.ts pinning (a) null status → non-zero, (b) the replayed advisory bytes → one attempt. Then the CI step you just added is what enforces them.
4. Correcting myself: the CHANGELOG merge will not conflict, and that is the problem
My comment above said landing second "will need the same one-hunk resolution". I have now executed it. Three-way merge, real merge base 7a194f1eba4, #169 42c57328364 x my #168 8a92baa680b: git merge-file -p --diff3 → rc 0, zero conflict markers. The merged file reads:
8 ## [Unreleased]
10 ## [0.1.2] - 2026-09-06
18 ### Fixed
20 - CI Web job now runs vitest unit tests ... <- your entry
## [Unreleased] is left empty and your entry is filed inside ## [0.1.2], which #168's body defines as "exactly 7 merged commits, the newest being 7a194f1" — #169 is not one of them. Because the merge is textually clean, no one gets prompted. If #168 lands first, the resolution is to move this block back under ## [Unreleased], not "keep both sides"; if #169 lands first, #168's commit list needs an 8th entry. Neither is a defect in this PR — I am flagging it because mergeable_state=clean is what my earlier sentence leaned on, and it means less than I wrote.
Re-checked the contention set live for the record: of the 10 open PRs, CHANGELOG.md is touched by #140 / #162 / #165 / #168 / #169 (only #140 reports dirty), while .github/workflows/ci.yml is touched by #169 alone — so unlike the CHANGELOG, the workflow edit has no competitor.
5. What this head has as evidence, as of 05:49Z
GET /repos/bytefolk/mem/actions/runs?head_sha=42c57328364… → 3 runs (CI, Validate Agent memory, PR Policy), all completed / action_required at run_attempt 1. So the number of CI checks that have executed against this head is still zero — what my first comment said at 03:13Z, still true 2.5 h later. pulls/169 reports mergeable=true / mergeable_state=blocked.
There is now also 5557259936, a local verification report plus a merge-readiness verdict. Its two numbers (vitest 6 files / 67 tests; 3 retries with the 10 s backoff) reproduce exactly in my runs, so that execution is real. Two limits on what it can show, stated as measurements:
- Against
registry.npmmirror.com, npm prints{ error: '[NOT_IMPLEMENTED] /-/npm/v1/security/* not implemented yet' }and appendsnpm error audit endpoint returned an error. That last string is inNETWORK_PATTERNS, so every run against that mirror necessarily goes retry → retry → retry → exit 1. The mirror path cannot reach the pass branch and cannot reach the real-vulnerability branch, so it is not evidence that the gate still gates. Onregistry.npmjs.orgagainst your ownwebgraph: both commands pass on attempt 1, exit 0 (item 1 and the last paragraph below) — that is the leg CI will take. - "Retry logic works as expected" measures the retry counter. The counter is not where the risk in this file sits; item 2 is, and it needs npm to produce no result object at all, which no ordinary run reaches.
I am reporting what the head does and does not have executed, not adjudicating anyone's authority to comment on it.
Corroborations, so the earlier numbers are not single-sourced
npm ci in the web job carries no --omit=dev, so vitest is installed; "test": "vitest run" is non-watch; on this head npm test = 6 files / 67 tests / 0 failed in 1.54 s; main's two npm test steps are both working-directory: npm (ci.yml:294, :344), i.e. the web leg genuinely had no unit-test execution. Happy path of your wrapper on the real web dependency graph with NPM_CONFIG_REGISTRY=https://registry.npmjs.org: both commands pass on attempt 1, exit 0 — and raw npm prints found 0 vulnerabilities (24 B) that the wrapper does not forward. Worth one process.stdout.write(stdout) on the success path. Related measured fact: the exit-1 and exit-0 runs of the same axios fixture produce byte-identical stdout (3458 B), which is why stdout text cannot be the gate signal and --audit-level's exit code has to stay the only authority.
|
已在钉钉平台组通知有权限的协作者(PeterGuy326、Bindy-lbb、jnn111、waterbro-8)去 approve 这 3 条 workflow runs。等待 CI 结果。 |
spawnSync returns null status when npm is killed by signal or not found on PATH. Three return sites in runWithRetry passed null through to process.exit, which Node treats as 0 — failing the audit gate silently. Add `?? 1` at the two unguarded sites (:67, :85) and remove the redundant 4th spawnSync after loop exhaustion (review finding #1), reusing the last loop result with the same `?? 1` guard. All three sites now aligned.
|
Verification notes on #169 — this is not a review vote and not an acceptance. I submitted neither APPROVED nor REQUEST_CHANGES here, I'm not asking for changes as a gate, and I touched nothing else (no merge, no label, no assignee, no Read live at 2026-09-08 10:5x local: head E1: what this PR genuinely fixes
FindingsF1 (E1, and the thing the existing review asked for): F2 (E1): the premise in the PR description is overstated — the retry addresses 1 of 4 sampled failures. I read the four red On the earlier comment here reporting "3 retries + 10 s backoff executed as expected": I reproduced that and it is not a misobservation — I pointed npm at F2b (E1, side effect of the same shape): on a machine whose registry doesn't serve audit, F3 (E1): a second F4 (E1, on real Windows): this turns a working audit gate into a permanently red one on win32, and Linux CI cannot see it. On this machine's Windows side (
Before this PR, const cli = process.env.npm_execpath;
result = spawnSync(cli ? process.execPath : "npm", cli ? [cli, ...args] : args, {...});
// and for the backoff, drop spawnSync("sleep"):
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, BACKOFF_MS);Evidence-level honesty: the F5 (E3): the retry budget is unbounded relative to the job timeout. Worst case is 3 × 300 s (npm's default F6 (E3, prose): the description says "3 retries"; the implementation is 3 attempts = 2 retries. And State, for whoever rebases nextmain has moved to One process fact so nobody wastes a cycle: the two CHANGES_REQUESTED reviews sit on the previous head |
The 60s ATTEMPT_TIMEOUT_MS kills npm before its default 300s fetch-timeout fires, so the retry logic never sees "network timeout" for slow registry hangs. Adding --fetch-timeout=45000 to both audit args lets npm report the timeout within the 60s budget, making isNetworkError catch it and trigger retries as designed.
- Assert vite.config.ts includes audit-retry.test.mjs so removing it fails the suite instead of silently dropping 35 tests. - Assert --fetch-timeout < spawn timeout so the dead-zone fix stays enforced if either constant drifts.
Captures git head, npm_execpath probe, full npm run audit output, and exit code. Must be run from cmd.exe (not WSL) to prove the win32 branch in audit-retry.mjs:117 receives npm_execpath.
Windows cmd.exe requires CRLF; LF-only caused every line to be misread as an unknown command.
|
HOLD — no approval on this follow-up yet. I re-ran |
|
Current-head CI approval readback (2026-09-10) Head:
Failures at this snapshot: The Web job reached the real dependency audit and failed on the high-severity js-yaml advisory (GHSA-2883-xcg3-v3hh), not a transient registry error. The separate audited-dependency candidate is #192. |
|
Canonical DRAFT successor: #198, preserving this exact head 5c3a4a7 and its complete author ancestry/tree. Current main 2986fe3 is already included. This uses the existing same-repository CodeQL path without changing security permissions or fork restrictions. This original PR remains open and unchanged; no review approval or merge is implied. Fresh Linux validation and the exact-head ledger are in #198. |
|
Superseded by @PeterGuy326's #198 which covers the same changes on current main. Closing per the '本轮先不合并' directive and to avoid duplicate review surface. |
#198) ## Tracking record and provenance Refs #122 — bounded CI/release-readiness follow-up, not completion or publication of that release. Canonical draft successor to #169, with the original author's `sun-970` commits and maintainer follow-ups retained unchanged in ancestry. The original PR stays open; its reviews and discussion remain authoritative history, not an approval of this successor. - Original/final source head: `5c3a4a75ab96bed4c046c0822ebc76a87f215cda`. - Successor head: `5c3a4a75ab96bed4c046c0822ebc76a87f215cda`. - Source and successor tree: `9d8a127375fd8e945113e06ae88983a1328b9fae`. - Current canonical main: `2986fe38175f54d99f15dd38a498708c6ecd88cd`, already an ancestor. A normal merge reports `Already up to date`; no empty commit, cherry-pick reconstruction, rebasing, or force push was used. - The audited lockfile retains #192's provenance from `11e02e21ef2c3dbd2dae26e4376872e54e78ecb5` (`cherry-pick -x` in ancestry). ## Summary and acceptance criteria The Web CI job must run its unit tests. Recognized transient npm audit failures may retry, but vulnerabilities, unknown errors, failed spawns, signals and timeouts must remain failures. Successful audit output must remain visible, including below-threshold findings. The Windows evidence helper must complete its report and preserve a nonzero audit exit status. This successor changes no source file relative to #169's exact source head. It exists because the canonical security workflow skips the entire CodeQL job on fork PRs while main requires three language-specific CodeQL contexts. A same-repository candidate executes the existing supported path; no security workflow, permission, required check, audit threshold, or fork restriction is modified or waived. Changed files against main: `.github/workflows/ci.yml`, `CHANGELOG.md`, `scripts/test_win_audit_verify.mjs`, `scripts/win-audit-verify.bat`, `web/audit-retry.mjs`, `web/audit-retry.test.mjs`, `web/package.json`, `web/package-lock.json`, `web/vite.config.ts`. ## Validation ledger Environment: isolated Linux aarch64, Linux 6.8, GNU find 4.9.0/coreutils 9.4, Bash 5.2.21, Node 24.13.0/npm 11.6.2; checksum-verified official Node archive. Fresh Git clone from an exact local bundle, two CPUs, GOMAXPROCS=2 and GOFLAGS=-p=1. Tests use disposable fixtures, not production data or secrets. | Command/check | Expected | Actual | | --- | --- | --- | | `git diff --exit-code 5c3a4a7 HEAD` | Exact source tree | PASS, no differences | | `git merge-base --is-ancestor 2986fe3 HEAD` | Current main included | PASS | | From root: `bash scripts/test_release_guards.sh` | Release guards fail closed | PASS | | `bash scripts/test_release_helpers_compat.sh` | Collection compatibility | PASS | | `bash scripts/test_validate_release_action_pins_compat.sh` | Official pinned actions and compatibility | PASS | | `bash scripts/validate_release_version.sh 0.1.1` | Version surfaces agree | PASS | | From `web/`: `npm ci --registry=https://registry.npmjs.org --fetch-timeout=45000` | Locked install | PASS, 401 installed packages | | `npm run audit --registry=https://registry.npmjs.org` | Both unchanged thresholds pass | PASS, zero vulnerabilities, both reports printed | | `npm test` | Application and audit regressions pass | PASS, 104 tests / 7 files, including 37 audit regressions | | `npm run lint` / `npm run typecheck` / `npm run build` | Static validation and production build pass | PASS, existing large-chunk warning only | | `git -c core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol diff --check 2986fe3 HEAD` | Clean diff, intentional batch CRLF retained | PASS | | Canonical successor CI and CodeQL | All required named contexts execute and pass | PENDING at draft creation; no all-green claim | ## Tests, coverage and known limits The original successful-report regression was red before the fix (1 failed / 36 passed). Existing exact-source [Web CI](https://github.com/bytefolk/mem/actions/runs/34438230117/job/102747885693) passed 104 tests including 37 audit regressions. Existing real [Windows CI](https://github.com/bytefolk/mem/actions/runs/34438230117/job/102747885554) passed four native batch-helper regressions, including negative controls for both original defects. These are source-head evidence, not independent review or substitute results for the new canonical workflow execution. Coverage percentage was not remeasured. A real-registry Windows audit has not been requalified; its helper tests use local npm.cmd fixtures. The unchanged five-second process fixture timeout reproduces on both pre-follow-up and fixed heads on the managed macOS host; Linux is the validated environment. The existing large-chunk build warning remains. Full Linux process/database and browser acceptance, and native Windows execution, are delegated to unchanged CI; no temporary Go executable runs on macOS. ## Risk, rollback and review Build/CI/dependency maintenance; no application API, CLI/MCP authorization, storage or migration contract changes. `[Unreleased]` already documents the behavior. The PR-triggered workflows reference no custom repository secrets; CodeQL retains only its existing security-events upload permission. No release, npm publication, Scorecard dispatch, tag, merge, or formal approval is performed. Rollback is to leave this draft unmerged; no deployed behavior changes. Draft until the exact candidate has required CI plus independent review. The Owner reports independent review of the fix deltas is already in progress; this description does not represent that review as completed. Automated assistance produced validation and the maintainer follow-up; the original human-authored history remains intact. --------- Co-authored-by: liyuanyang <liyuanyang@users.noreply.github.com> Co-authored-by: waterbro-8 <318569545+waterbro-8@users.noreply.github.com>
Problem and result
Refs #122 for CI/release-readiness follow-up only; this does not complete or publish that release. The Web CI job previously omitted unit tests, and transient registry failures could fail the audit step. This PR runs the Web tests and retries only recognized transient audit failures, with three attempts (two retries) per threshold. Real findings, unknown errors, missing processes, signals and timeouts remain failures.
The current follow-up also preserves successful audit output, including findings below the configured threshold. Before this fix a status-0 audit discarded stdout/stderr; the new regression reproduced that loss. Audit thresholds remain production/moderate and all-dependencies/high, with a 60-second process limit per attempt and 10-second backoffs.
Changes and provenance
npm testin the Web CI job and collectaudit-retry.test.mjswith the application tests.[Unreleased].11e02e21ef2c3dbd2dae26e4376872e54e78ecb5, cherry-picked with-xasa7c17ec. It fixes the common audited baseline: js-yaml 4.3.2, Vitest/@vitest packages 4.1.11, and postcss-selector-parser 6.1.4. The old lockfile reproduced four findings including the high-severity js-yaml advisory; this was a real dependency finding, not a transient error.call npm run auditand return the saved status on the same line asendlocal. A native Windows process regression checks successful and failing audits and replays both original failure modes as negative controls. It runs in the existing Node 24 Windows job.5c3a4a75ab96bed4c046c0822ebc76a87f215cda. Web validation was executed atd91deff; the subsequent commit changes only the Windows helper, its process test and its CI invocation.Validation ledger
From
web/, Node 24.13.0, npm 11.6.2, public npm registry, checked-in lockfile. Linux aarch64 validation uses an exactgit archiveof the implementation head and dependencies installed withnpm ci; no service or credential is required by these tests.npm ci --registry=https://registry.npmjs.org --fetch-timeout=45000npm run audit --registry=https://registry.npmjs.orgnpm testnpx --no-install vitest run audit-retry.test.mjswith the new regression before the implementation fixnpm run lint && npm run typecheck && npm run buildnode --test scripts/test_win_audit_verify.mjsin the Windows CI jobgit -c core.whitespace=blank-at-eol,blank-at-eof,space-before-tab,cr-at-eol diff --checkThe original PR's process fixture (
78b3d2d) also hits its 5-second CLI timeout on this macOS host with the same dependencies. The follow-up reproduces that host-dependent result; the unmodified timeout tests pass on Linux. The test was not weakened. A first macOS tar transport included AppleDouble metadata and caused six extra parse failures; the final Linux result above uses a clean exact Git archive (7 test files), not that invalid transport.Boundaries and pending evidence
5c3a4a75ab96bed4c046c0822ebc76a87f215cda: all 16 distinct non-skipped check names succeeded; CodeQL remains skipped under the existing fork policy. Component CI, Agent-memory acceptance, security, and PR Policy all passed. Final-head Web CI reports 104 tests including 37 audit regressions; Windows CI reports four helper regressions. These are CI results, not independent approval.