Skip to content

chore: Stabilize timing-sensitive Go CLI tests that intermittently fail in CI - #1311

Merged
hatayama merged 1 commit into
v3-betafrom
feature/hatayama/clean-up-go-cli-tests
Jun 12, 2026
Merged

hatayama merged 1 commit into
v3-betafrom
feature/hatayama/clean-up-go-cli-tests

Conversation

@hatayama

Copy link
Copy Markdown
Owner

Summary

  • Go CLI test runs no longer fail intermittently on loaded CI machines due to overly tight real-time test margins.

User Impact

  • Before: the build-cli CI job could fail randomly even when no Go code changed (most recently on PR chore: Remove Editor-freezing and obsolete tests from the Unity test suite #1310, where a busy-retry test surfaced a dial i/o timeout instead of the expected busy error), forcing re-runs and blocking unrelated PRs.
  • After: the four timing-sensitive tests use margins wide enough to be deterministic under CI load, so a red build-cli job means a real regression again.

Changes

  • Connection retry busy test: widened the retry window from 30ms to 500ms so at least one busy response always lands inside the window, and stubbed the Unity process probe to block until the context is done — the dial deadline is a separate timer that can fire microseconds before the retry context reports expiry, so an instant probe skipped the busy-masking guard (this exact failure was reproduced locally before the fix).
  • Accepted-ack retry tests: widened from a 20ms window vs 60ms server delay to 200ms vs 600ms, keeping the 3x ratio that defines their semantics.
  • IPC client accept-timeout test: widened from 50ms vs 150ms to 250ms vs 750ms for the same reason.
  • Test-only changes; no production code paths are modified.

Verification

  • go test ./internal/cli/ -run 'TestSendWithTransientConnectionRetry' -count=30 -race (30/30 pass)
  • go test ./internal/unityipc/ -run 'TestSendWithProgressOutcome' -count=30 -race (30/30 pass)
  • scripts/check-go-cli.sh (full pass, exit 0)

Four tests relied on sub-100ms real-time windows that a loaded CI
machine cannot honor; the busy-after-retry-window test already failed
in CI with a dial i/o timeout surfacing instead of the busy RPC error.

- Raise the busy test retry window from 30ms to 500ms so at least one
  busy response always lands inside the window, and stub the Unity
  process probe to block until the context is done: the dial deadline
  is a separate timer that can fire microseconds before retryContext
  reports expiry, so an instant probe would skip the busy-masking guard
- Widen the accepted-ack tests from a 20ms window vs 60ms server delay
  to 200ms vs 600ms, keeping the 3x ratio that defines their semantics
- Widen the IPC client accept timeout test from 50ms vs 150ms to
  250ms vs 750ms for the same reason

Verified with -count=30 -race stress runs on both packages and a full
scripts/check-go-cli.sh pass.
@coderabbitai

coderabbitai Bot commented Jun 12, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8f28341c-78e9-4fc6-a6b4-6e3a5e718920

📥 Commits

Reviewing files that changed from the base of the PR and between f650787 and df6c611.

📒 Files selected for processing (2)
  • cli/internal/cli/connection_retry_test.go
  • cli/internal/unityipc/client_test.go

📝 Walkthrough

Walkthrough

This PR updates transient connection and IPC client test timing assumptions across two test files to reduce CI flakiness. Retry timeout windows are widened, server response delays are increased, and a context-aware probe injection is added to ensure timeout and busy-response behavior remains deterministic under load.

Changes

Connection and IPC retry timing robustness

Layer / File(s) Summary
Accepted request retry timeout tests
cli/internal/cli/connection_retry_test.go
Two tests (TestSendWithTransientConnectionRetryDoesNotCancelAcceptedRequestAtRetryTimeout and TestSendWithTransientConnectionRetryKeepsRetryTimeoutBeforeAcceptedAck) increase retry timeout to 200ms and server delays to 600ms so dial, request write, and acknowledgment complete within the window while final response handling occurs after timeout.
Busy response retry behavior
cli/internal/cli/connection_retry_test.go
TestSendWithTransientConnectionRetryReturnsBusyAfterRetryWindow expands retry timeout to 500ms and injects a context-blocking Unity process probe that waits for context expiration, ensuring busy masking logic observes the expired context; cleanup restores original probe and timing.
Client accept timeout hardening
cli/internal/unityipc/client_test.go
TestSendWithProgressOutcomeWaitsForFinalResponseAfterDispatchAckWithoutAcceptTimeout increases server delay between acknowledgment and final response, and raises acceptTimeout with comments explaining the margin accommodates CI load while proving accepted requests persist beyond the accept timeout.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically identifies the main change: stabilizing timing-sensitive Go CLI tests to fix CI flakiness, which directly aligns with the changeset.
Description check ✅ Passed The description thoroughly documents the changes, impact, and verification, all directly related to the test stabilization work in the changeset.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/hatayama/clean-up-go-cli-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Re-trigger cubic

@hatayama
hatayama merged commit 325aba4 into v3-beta Jun 12, 2026
8 checks passed
@hatayama
hatayama deleted the feature/hatayama/clean-up-go-cli-tests branch June 12, 2026 01:28
RyanXie123 pushed a commit to RyanXie123/unity-cli-loop that referenced this pull request Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant