diff --git a/RISK.md b/RISK.md index be31857d..a6ffbb70 100644 --- a/RISK.md +++ b/RISK.md @@ -3,28 +3,50 @@ `quest review` reads this file from the base to set each PR's risk tier, as the README's Risk tiers section describes. -`quest` runs development in moq-dev's and Kixel's repositories. A merge to -`main` that changes what `scripts/release-plan.sh` ships publishes a -release, which contributors and the maintainer's hosts install. A revert -publishes another, but what agents did under the bad one stays done. +`quest` runs development in moq-dev's and Kixel's repositories. What merges +to `main` ships in the next release, which contributors and the maintainer's +hosts install, and a revert ships in the one after. A bug that agents act on +while its release is live is a risk every behavior change carries, and +medium covers it, including a bug in a record, comment, or upload `quest` +already writes. What a revert cannot undo is what a change newly makes +`quest` or its agents do beyond the repository: merge, post, upload, or +spend where they did not before. What others rely on here: - The rules and skills under `assets/`, which every agent session in those - repositories loads. A change to when an agent may push, merge, post, or - spend without asking is high. + repositories loads. - The `quest/core` and `quest/github` exports, which Kixel Code imports. - A change to what they export or return is at least medium, and high when - it changes how a record or status is read. -- The records `quest` writes into PR bodies and the output of +- The records `quest` writes into PR bodies and sends, and the output of `quest status`, `quest usage`, and `quest unblocked`, which skills, Kixel Code, and scripts parse. - The reviewer's sandbox and the credentials `quest` passes to the agents it starts, which keep a change from instructing its own reviewer or reaching a secret. +A change is high when it: + +- breaks a consumer of an export, a record, or that output: it removes or + renames one, changes what an existing field or line means or how it is + written, makes a reader refuse what the previous release wrote, or adds + what a consumer must now supply, such as a required member of an + interface it implements or an argument it passes. Such a change marks + its title with `!`, though a `!` for a CLI flag alone is not this. +- changes when an agent may push, merge, post, or spend without asking, + in `assets/` or in the code that acts on it. +- changes the reviewer's sandbox or the credentials an agent gets, or + sends a new kind of data off the host or into a public comment, such as + a new upload destination or command output it now posts. New fields in + a record `quest` already sends are not this. + +A change is medium when behavior others rely on changes and a revert +undoes it, such as an export, an optional field, or an output line that +existing consumers keep parsing, a new or changed command or option, a bug +fix, a skill or audit's instructions that leave remote-write authority +alone, or a routine dependency or pin bump. + Tests, plans under `quest/`, and documentation change nothing others rely -on. +on, so they are low. ## High diff --git a/quest/m0/benchmark/guardrails.md b/quest/m0/benchmark/guardrails.md index 0d6d293e..d06879d1 100644 --- a/quest/m0/benchmark/guardrails.md +++ b/quest/m0/benchmark/guardrails.md @@ -59,6 +59,14 @@ tiers are trusted, not merging every pick on its own. Decided 2026-11-19 [moq-dev/quest#786](https://github.com/moq-dev/quest/pull/786)): ✅ keep this evidence here (rejected: drop). +Decided 2026-11-21 (maintainer, /quest-run): a week of recorded tiers did +not match their sense of risk, since high took 55% of PRs and nearly every +code PR. [moq-dev/quest#794](https://github.com/moq-dev/quest/pull/794) +narrowed high in `RISK.md` to breaking changes to exports, records, or +output, new remote writes, uploads, or spending, merge authority, +credentials, and the held paths. Re-tiered, the week's 56 tiered PRs went +from 32 high to 22, with no low PR moving. + This quest edits the shipped skills and rules, which the maintainer asked for by planning it. @@ -97,11 +105,6 @@ Action, which edits a workflow file: recommended to keep that class automatic, since the actions are pinned by SHA, Dependabot waits out a cooldown, and Pullfrog approves the head. -## Required - -- [Read a week of risk tiers before guardrails act on them](/quest/m0/benchmark/risk-tier.md) - - the maintainer's verdict that the recorded tiers can be trusted - ## Related - [Routing evidence](/quest/m0/routing-evidence/README.md) - its shadow diff --git a/quest/m0/benchmark/risk-tier.md b/quest/m0/benchmark/risk-tier.md deleted file mode 100644 index a278fc8c..00000000 --- a/quest/m0/benchmark/risk-tier.md +++ /dev/null @@ -1,35 +0,0 @@ -# [XS] Read a week of risk tiers before guardrails act on them - -## Goal - -The maintainer has read at least a week of the risk tiers `quest review` -recorded, against what merged and what later broke, and either trusts them -for [tier guardrails](/quest/m0/benchmark/guardrails.md) or has fixed what -set them wrong: a path in a repository's `RISK.md`, its prose on what is -customer-facing, or the classifier's instructions. - -## Plan - -`quest review` sets each head's tier since -[#716](https://github.com/moq-dev/quest/pull/716) and records it in the PR -body's `## Review` section. `quest usage --since ` shows each PR's -tier and reason, whether a reviewer raised it, and a rollup per tier of -PRs, merges, reviewer raises, and later fixes (`## Caused by`). A tier -reviewers often raise, or a low PR a later one fixed, points at what to -change. A fix lands in this quest's PR; the reading starts over only if -the fix changes most tiers. - -A repository with no `RISK.md` records every change medium, so the reading -covers only the repositories that have one. - -## Waiting - -- Maintainer: read a week of recorded tiers once a release with #716 runs - on the hosts, and say whether they match your sense of risk; no agent - can judge that. Check with the tier column and the By risk tier rollup - of `quest usage --since `. - -## Related - -- [Development as a benchmark](/quest/m0/benchmark/README.md) - the - decisions this builds on diff --git a/vitest.config.ts b/vitest.config.ts index b3489c41..e3ea1ddc 100644 --- a/vitest.config.ts +++ b/vitest.config.ts @@ -20,7 +20,13 @@ export default defineConfig({ // with a key or program the fixtures cannot reach; each fixture sets its identity. // Git for Windows maps "/dev/null" to its null device but rejects os.devNull's "\\.\nul". // An agent session's ID would log the gates tests run as that session's waits (src/cli/waits.ts). - env: { GIT_CONFIG_GLOBAL: "/dev/null", CLAUDE_CODE_SESSION_ID: "", CODEX_THREAD_ID: "" }, + // The developer's own records.json would send reads to Kixel Quest where it serves this repository. + env: { + GIT_CONFIG_GLOBAL: "/dev/null", + CLAUDE_CODE_SESSION_ID: "", + CODEX_THREAD_ID: "", + XDG_CONFIG_HOME: "/nonexistent/quest-test-config", + }, ...(live ? { include: ["test/live/**/*.test.ts"], testTimeout: 30 * 60_000, hookTimeout: 60_000 } : {