Skip to content

docs(ops): the canary's exit 3 needs both attempts stopped - #3303

Merged
lilyshen0722 merged 1 commit into
mainfrom
cto/d230-canary-doc-followup
Nov 2, 2026
Merged

lilyshen0722 merged 1 commit into
mainfrom
cto/d230-canary-doc-followup

Conversation

@lilyshen0722

Copy link
Copy Markdown
Contributor

This is the docs follow-up promised on #3295. It answers CodeRabbit's body point at 733b09f, which arrived after that PR entered the queue.

docs/runbooks/litellm-guardrails.md said the canary exits 3 "if the guard stops one." The script exits 3 only when the guard stops a turn on both attempts, because one stop alone is inconclusive. The sentence now says that.

🤖 Generated with Claude Code

https://claude.ai/code/session_015AKgpQ2pr6xRMhN5mcfoHF

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015AKgpQ2pr6xRMhN5mcfoHF

@lilyshen0722 lilyshen0722 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HEAD: 2dd1a1a — GATE: PASS

One review round (D218), Tier 0, a one-line doc change checked against the source on main. In scripts/guard-name-canary.js, exitCodeFor returns BLOCKED_EXIT (3) when a check's verdict is blocked. The verdict docblock just below it says "Blocked needs both attempts blocked", and a single stop reads as inconclusive (2). The runbook sentence now says the same.

GATE: PASS

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between bdd549c and 2dd1a1a.

Files selected for processing: 1
  • docs/runbooks/litellm-guardrails.md

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 3 remain after this review.

Walkthrough

The runbook now documents that the guard-name canary exits with code 3 only when both attempts stop. A stop on one attempt is inconclusive. Deploy Dev still runs the canary after verification and rolls back on exit code 3.

Changes: Guardrail canary documentation

Layer / File(s) Summary
Canary result and deployment response
docs/runbooks/litellm-guardrails.md
The runbook changes the exit-3 condition to require stops on both attempts. It describes one stop as inconclusive and retains the documented Deploy Dev rollback on exit code 3.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Change: Other

Possibly related issues

  • Team-Commonly/commonly#541 — The runbook documents the canary outcome for LiteLLM guardrail blocking, but does not change enforcement behavior.
Merge Risk: Minimal · up to 2dd1a

The canary behavior is documented consistently with the deployment rollback flow.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely states the main documentation change: exit 3 requires both canary attempts to be stopped.
Description check Passed The description directly explains the documentation correction and its relationship to the canary behavior.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Finishing Touches
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@lilyshen0722
lilyshen0722 added this pull request to the merge queue Nov 2, 2026
Merged via the queue into main with commit e0d2b2e Nov 2, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant