Skip to content

test(review): log each fake call atomically so an aborted fake leaves no cut-off line - #722

Merged
moq-bot[bot] merged 2 commits into
mainfrom
quest/m0/review-test-partial-log
Nov 14, 2026
Merged

moq-bot[bot] merged 2 commits into
mainfrom
quest/m0/review-test-partial-log

Conversation

@kixelated

@kixelated kixelated commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes the flaky rejected-login review test at its root and finishes the quest, deleting it.

Caused by

  • moq-dev/quest#621 - ran the two reviewer seats concurrently and aborted the sibling on the first failure, so a fake CLI can die mid-append, while the tests kept reading one shared append-only log; the fake's non-atomic logging dates from #271.

#706 is not listed: its diff only loosened the status-check count (the abort can stop the sibling's check before it logs, which is a real race) and touched neither the fake nor the log reader, so it left this defect rather than introducing it.

Summary

Each fake CLI call in test/review.test.ts now writes its record to its own .tmp file in the FAKE_LOG directory and renames it to .json once whole; calls() reads only the .json files, sorted by their microsecond start time. A fake killed mid-write leaves only a .tmp, so a cut-off line can no longer reach JSON.parse. #706's 1..2 bound on status checks stays: that count is a real race (the abort may stop the sibling before it spawns or logs), not a flake, and the reproduction below hit a different failure.

No public API or wire change; test code only. Dropped: the shared calls.log file. test/agent.test.ts has a similar append-only fake log, left alone since nothing kills its fakes mid-write.

Validation

  • Reproduced before the fix: making the fake write its log line in two halves with a 1.5 s pause between (the kill lands in the gap) failed all five a login the API rejects fails as logged out cases with SyntaxError: JSON Parse error: Expected '}' at s.calls() (the reported cut-off line).
  • With the fix and the same interrupted write (into the .tmp), all five cases pass.
  • test/review.test.ts passed 93/93 three runs in a row; just check and just test pass.

Follow-ups

None.

Models

Size: XS
Implementer: Claude Code, claude-sonnet-5-5, high effort

(Written by Sonnet 5.5 with high effort)

🤖 Generated with Claude Code

Review

Rewritten by quest review after each round.

  • Head: 9374666

  • Base: origin/main at 4ab5dae

  • Risk: low, Only test fake logging and a development plan change, so any breakage is confined to the test suite and a revert fully undoes it without affecting users or dependent repositories.

  • Classified: low by Codex, gpt-6.1-sol, low effort, session 01a1d7a1-3aea-7da3-add7-aea559e6f621

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1d7a1-4aaa-7692-9d9d-f81bd3e03f49, round 1

  • Score: 9/10, Atomic per-call log files address the logging race without changing existing assertions; no actionable regressions were found by inspection, though tests were not run as instructed.

  • Rounds: 1; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

  • Reviewer: Codex, gpt-6.1-sol, low effort, shadow

  • Session: 01a1d7a1-4aaa-7502-aa1f-e2e15e7a8d1c, round 1

  • Score: 9/10, Per-call files and atomic rename prevent partial JSON reads, with no actionable defects found by inspection; tests were not run as instructed.

  • Rounds: 1; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

kixelated and others added 2 commits November 13, 2026 17:12
… no cut-off line

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Fix the flaky rejected-login review test at its root test(review): log each fake call atomically so an aborted fake leaves no cut-off line Nov 14, 2026
@kixelated
kixelated marked this pull request as ready for review November 14, 2026 01:17
@kixelated
kixelated enabled auto-merge November 14, 2026 01:18

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes: the full diff of the test-only fix for the cut-off fake log line.

  • Atomic per-call logging: each fake CLI call in test/review.test.ts writes <µs-start>-<pid>.tmp into the FAKE_LOG directory and renames it to .json, so an aborted fake leaves no partial record.
  • Reader: calls() lists only .json files and sorts them. Names are fixed-width microsecond timestamps, so the lexicographic sort matches call order.
  • Setup: FAKE_LOG is now a directory created in setup(). No stale calls.log references or unused imports remain.
  • Quest file: the finished quest quest/m0/review-test-partial-log.md is deleted.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@moq-bot
moq-bot Bot added this pull request to the merge queue Nov 14, 2026
@coderabbitai

coderabbitai Bot commented Nov 14, 2026

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

Fake reviewer CLI invocations are now written to separate JSON files through temporary files that are renamed after writing. The test fixture creates a call-log directory and reads sorted JSON entries, ignoring temporary and other files. The document describing the rejected-login test plan was removed.

Merge Risk: Minimal · up to 93746

This change makes fake CLI call logging in the review tests atomic, which removes a flaky failure mode. It does not touch production code, and no merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names the main change: each fake CLI call in the review test is now logged atomically, so an aborted fake cannot leave a cut-off line. It matches the changeset, which writes each call to a .…
Description check Passed The description is specific and relevant. It explains the root cause, the per-call .tmp-then-rename logging, the reproduction and validation steps, and the deleted quest document. It matches the delet…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the main change: each fake CLI call in the review test is now logged atomically, so an aborted fake cannot leave a cut-off line. It matches the changeset, which writes each call to a .tmp file and renames it to .json.

Full details: Description check

Explanation

The description is specific and relevant. It explains the root cause, the per-call .tmp-then-rename logging, the reproduction and validation steps, and the deleted quest document. It matches the deleted plan file and the test/review.test.ts changes.

Finishing Touches

Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0171cfeb-4861-404b-822e-ada1a4247bf7
Commits

Reviewing files that changed from the base of the PR and between 4ab5dae and 9374666.

Files selected for processing: 2

  • quest/m0/review-test-partial-log.md
  • test/review.test.ts
Files with no reviewable changes: 1

  • quest/m0/review-test-partial-log.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

Merged via the queue into main with commit a5b1225 Nov 14, 2026
23 checks passed
@moq-bot
moq-bot Bot deleted the quest/m0/review-test-partial-log branch November 14, 2026 01:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant