Skip to content

ci: prove affected drill mutations on pull requests - #5599

Merged
kixelated merged 3 commits into
mainfrom
quest/m2/drill-mutation-gate
Oct 16, 2026
Merged

kixelated merged 3 commits into
mainfrom
quest/m2/drill-mutation-gate

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

An applying mutation can silently stop reintroducing its bug. Local --apply-only checks cannot catch that, so sensitivity failures previously surfaced only after merge in nightly.

Approach

Add a stable Drill sensitivity job to the existing Check workflow for PRs and merge groups. Select mutations from their patch target paths, changed patches, or the drill/harness/selector changing. Reuse sensitivity.sh with mutation-name arguments and pin MOQ_SHAPER_SEED=1. An empty selection reports success before Nix installation or snapshot compilation.

The job uses job-level continue-on-error while socket timing remains uncontrolled. No repository protection setting changes. Full nightly sensitivity and local apply-only checks remain intact.

Impact

  • Public API and wire: none.
  • PR/merge-group CI gains an advisory behavioral sensitivity signal, with a stable name suitable for later protection.

Alternatives

Running cold mutation snapshots in every developer's just check, replacing the full nightly, and making this required before controlled time were rejected in the quest.

Validation

Nix just check passed. Six selector/recipe regressions passed, including a recipe argument-forwarding regression verified red then green. Real pinned-seed subscriber-leaks-broadcasts validation passed its baseline and failed its mutated snapshot with the declared leak reason. Committed-head selection reports needed=false and execution skips; PR-base selection reports true. GitHub workflow lint and quest checks passed. Unit scope tests run in existing nightly tooling coverage.

Follow-ups

After the existing Controlled time quest moves drills off host timing, remove continue-on-error and ask the maintainer to require Drill sensitivity. Remaining mutations run in the selected PR job and full nightly; this local validation ran one actual mutation. Completed quest and index reference removed.

(Written by GPT-6)

kixelated and others added 3 commits October 15, 2026 11:43
Co-Authored-By: GPT-6 <noreply@openai.com>
Co-Authored-By: GPT-6 <noreply@openai.com>
@kixelated
kixelated marked this pull request as ready for review October 15, 2026 23:45
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T23:48:27.494413Z 86e9511 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 16 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 2ff1d585-62ce-4d11-985b-b21a30c5e59b
📥 Commits

Reviewing files that changed from the base of the PR and between 9f1f7d1 and 86e9511.

📒 Files selected for processing: 8
  • .github/justfile
  • .github/workflows/check.yml
  • quest/m2/README.md
  • quest/m2/drill-mutation-gate.md
  • test/drill/README.md
  • test/drill/pr.py
  • test/drill/pr.test.py
  • test/justfile
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary:

  • Merged main and resolved the quest/m2/README.md conflict by keeping the new Biome entry and dropping this quest's (completed) line. No code changes.
  • The previous red Interop (relay connection N (subscriber) idled out instead of closing) was unrelated: the same failure hit other PRs that day, and fix(relay): log every connection's close from Connection::run #5602 fixed it on main. The merge picks up that fix.
  • Codex reviewed the merged head with no findings. CodeRabbit was rate-limited.
  • Marked ready and enabled auto-merge. Drill sensitivity stays advisory (continue-on-error), and no repository protection settings were changed.

(Written by Opus 5.5)

@kixelated
kixelated added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit af5848c Oct 16, 2026
13 checks passed
@kixelated
kixelated deleted the quest/m2/drill-mutation-gate branch October 16, 2026 00:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant