Skip to content

docs: RISK.md declares which changes are high risk - #9194

Merged
kixelated merged 2 commits into
mainfrom
quest/m1/risk-policy
Nov 19, 2026
Merged

kixelated merged 2 commits into
mainfrom
quest/m1/risk-policy

Conversation

@kixelated

@kixelated kixelated commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator

Implements part of quest/m1/risk-policy.md and rewrites it to what remains.

Problem

quest review gives every PR a risk tier since moq-dev/quest#716, but a repository without a RISK.md records every change as medium. That makes moq's tiers useless, and the maintainer's week of reading tiers (moq-dev/quest's risk-tier quest) cannot cover this repository.

Approach

  • RISK.md in quest's format (pinned d4df7d5): prose the classifier reads, plus a ## High list of paths whose every change is high.
    • High: an API break or a wire change (public Rust, JS, and bindings API, frozen wire versions incl. -wip the next cut publishes, hang catalog JSON, container formats, CLI flags, config keys, env vars); what a token grants; when an agent may merge or post without the maintainer; release packaging and publish scripts.
    • Held by path: RELEASE.md, version.json, .release-plz.toml, sh/version.py, sh/api-diff/, sh/gh/release.sh. quest already holds RISK.md, .github/workflows/, and .github/actions/.
    • Medium: shipped code that keeps its API and wire, merge-gate selection (sh/dispatch.sh, sh/changed.sh), other agent instructions. Low: tests, benchmarks, TLA+ models, quest plans, examples, demos, docs.
  • One definition of breaking: the questline now cites Releases for API break and wire change instead of restating both lists; RISK.md cites it too.
  • Pointers: CONTRIBUTING.md's landing gate and REVIEW.md point at RISK.md.
  • Quest rewritten to what remains: the high tier's full suite (just gate) and the classes that land without a pick. quest's format has no place for either yet; moq-dev/quest's guardrails quest adds both, so the quest now waits on that alone. Sized XS.

Impact

No public API or wire change. quest review tiers on PRs reviewed after this merges change from a flat medium to the classifier's tier. Nothing acts on a tier yet.

Alternatives

  • Hold public API and wire paths by pattern (rs/*/src/**, rs/hang/src/catalog/): rejected. Most of the tree is public API, so a path rule would make nearly every PR high. Whether a change breaks is judged from its diff, which is the classifier's job.
  • Hold the per-language publish scripts and packaging/ by path: rejected for prose. They are spread across sh/<lang>/ with tests beside them, so a glob would also catch test edits.
  • Add local sections for the full-suite command and classes now: rejected. The quest says to take the format from quest and add nothing local, and the guardrails quest defines that format.

Decisions

Taken without the maintainer (AFK); each is easy to undo in review:

  1. Auth (rs/moq-auth, js/auth, the relay's token checks) is high when it changes what a token grants. The Goal did not list it. A widening there is a vulnerability in every relay after the next cut.
  2. Changes to when an agent may push, merge, post, or spend without the maintainer are high, mirroring moq-dev/quest's own RISK.md. Other agent-instruction edits are medium.
  3. Merge-gate selection (sh/dispatch.sh, 47 merges last month) is medium, not held high: it changes too often, and a miss reaches main, not users.
  4. RELEASE.md is held by path: agents follow it to cut and backport (0 merges last month).
  5. The questline's two restated definitions of breaking were replaced by a link to RELEASE.md, which already defines both. This is the "one source the others cite" the quest's Plan asked for.

Validation

  • quest check (with and without --base origin/main) passes. Appending a malformed ## High item (/bad) made it fail with RISK.md:56: ... starts with /, confirming it parses this file and accepts all 6 items.
  • Real classifier (quest review --base origin/quest/m1/risk-policy --harness codex --effort low, Codex gpt-6.1-sol low, about 15.9k input tokens), on throwaway local branches based on this branch, deleted afterward:
    • Rename the relay's --drain-timeout / MOQ_DRAIN_TIMEOUT: high, "CLI flags and environment variables are explicitly high risk".
    • Reword an integration test's doc comment: low, "only an integration test comment changes ... a revert fully undoes it".
    • Change the relay cache governor's interval from 5s to 2s: medium, "changes shipped relay behavior ... reverting the constant fully undoes the change".
  • Held-path share: of 2,322 merges to main since 2026-10-19, 163 touched quest's built-in paths and 166 touched those plus this file's ## High list (git pathspec count), so the list adds about 0.1%. The classifier's share comes on top.
  • quest gate just check passes.
  • For the remaining quest: the Nightly Gate job (just gate, no Rust cache) took 10m51s on 2026-11-18, against the scoped queue median of 5.5 minutes. Earlier runs' job data returned HTTP 502.

Follow-ups

None new. The rest of the work stays in quest/m1/risk-policy.md, waiting on moq-dev/quest's guardrails quest.

Models

  • Size: S
  • Implementer: Claude Code, claude-opus-5-5, high effort

🤖 Generated with Claude Code

(Written by Claude Opus 5.5 with high)

Review

Rewritten by quest review after each round.

  • Head: ab15a7c

  • Base: origin/main at 8e1605c

  • Risk: high, it changes RISK.md, which is always high: it sets the tier of every later change

  • Classified: high by the always-high paths

  • Reviewer: Codex, gpt-6.1-sol, medium effort

  • Session: 01a1f4bb-fd27-73f1-99fc-803c27b44938, round 1

  • Score: 9/10, The risk policy matches existing release definitions, remaining guardrail work is explicitly deferred, and no actionable defects or blockers were identified; builds and tests were not run as instructed.

  • Rounds: 1; first round 0; findings 0: fixed 0, refuted 0, open 0

  • Findings: none

kixelated and others added 2 commits November 19, 2026 08:45
Quest-Claim: f34b4ba427abd058
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title Declare which changes are high risk docs: RISK.md declares which changes are high risk Nov 19, 2026
@kixelated
kixelated marked this pull request as ready for review November 19, 2026 16:55

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Approach: sound. It adds a RISK.md in quest's format and makes RELEASE.md#releases the one definition of an API break and a wire change.

Reviewed changes

Docs only. I confirmed that RELEASE.md defines both terms, so the new links resolve to what the text says.

  • RISK.md: sets the tier prose and a six-path ## High list.
  • quest/README.md: replaces the two restated definitions of breaking with links to RELEASE.md#releases.
  • CONTRIBUTING.md and REVIEW.md: point at RISK.md.
  • quest/m1/risk-policy.md: cut down to the work that remains, now waiting only on the guardrails quest.

Pullfrog  | View workflow run | Using claude-sonnet-5-5 | 𝕏

@coderabbitai

coderabbitai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The changes add RISK.md, which defines risk tiers, high-risk paths, and release-related effects. CONTRIBUTING.md and REVIEW.md direct readers to the policy, and quest review is documented as recording a diff-based tier in the PR body. The quest README refers to Releases for API-break and wire-change definitions. The risk-policy quest document updates its goal, related text, and waiting checklist.

Merge Risk: Minimal · up to ab15a

This change adds a risk-tier policy document and links to it from the contributor and review guides. It changes no runtime behavior and is safe to merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "docs: RISK.md declares which changes are high risk" matches the main change: a new RISK.md that defines high-risk changes and paths. It is specific and concise, and it is accurate to the ch…
Description check Passed The description explains the problem (repositories without RISK.md record every change as medium), the approach (new RISK.md with high, medium, and low tiers and held paths), the pointers in CONTRIBUT…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "docs: RISK.md declares which changes are high risk" matches the main change: a new RISK.md that defines high-risk changes and paths. It is specific and concise, and it is accurate to the changeset.

Full details: Description check

Explanation

The description explains the problem (repositories without RISK.md record every change as medium), the approach (new RISK.md with high, medium, and low tiers and held paths), the pointers in CONTRIBUTING.md and REVIEW.md, the quest rewrite, validation, and decisions. It is directly related to the changeset.

Finishing Touches

Simplify code

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 599bc715-9a8a-4dda-8eb2-883cd7bf55e1
Commits

Reviewing files that changed from the base of the PR and between d44c498 and ab15a7c.

Files selected for processing: 5

  • CONTRIBUTING.md
  • REVIEW.md
  • RISK.md
  • quest/README.md
  • quest/m1/risk-policy.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer pick (2026-11-19, /quest-complete): ✅ /quest-merge at ab15a7c (rejected: read it first; iterate on the tiers).

(Written by Claude Opus 5.5 with high)

@kixelated
kixelated added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit c6e0b15 Nov 19, 2026
10 checks passed
@kixelated
kixelated deleted the quest/m1/risk-policy branch November 19, 2026 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant