Skip to content

docs(rfc): propose per-goal limits, drift signals and a frozen done check - #37649

Merged
buluoray merged 1 commit into
mainfrom
crew/slack/rfc-goal-limits-drift-done-check
Nov 16, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/slack/rfc-goal-limits-drift-done-check

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Part of #37361

Problem / Motivation

Goal: agree a design, before code lands, that lets a Bot's owner bound one goal: count-based drift signals that stop new work, owner-set limits the Bot cannot raise, and a done check the owner approves and the Bot cannot change.

In #37361 one Bot with one goal ran for six days: 175 merge requests (99 in one day), 249 pushes after open, 16 workers at once against a plan of two or three, 62 review re-checks, 9 duplicate closes, and open issues going from 46 to 108. A maintainer triaged it as a design-level change and pointed it at the RFC process. On main, nothing limits or watches a goal: concurrency_cap is written by the Bot's own work_ledger_record action=goal, the goal's item total has no cap, and the patrol has no exit tied to the goal being done.

Why it matters

A Bot left to run unattended can create work for itself faster than it closes it, and the owner cannot see where it stands without reading the Bot's chat. Every signal in #37361 was a count that could be read by day two.

Not a goal

What changed (motivation → approach → change)

Backwards compatibility

Compatible: docs only; no behavior changes.

Removes nothing: one new Markdown file and one new index row.

Tests

Docs only; there is no code to test.

  • bash scripts/docs-lint.sh: all documentation checks passed; python3 scripts/docs_lint.py --strict-identifiers reports no finding in either changed file.
  • BRAND_BASE_REF=origin/main python3 scripts/check_brand_name.py: no misspellings in the added lines.
  • scripts/check_comment_history.py scans src/kiro_crew and test only; this diff touches neither.
  • The internal-host denylist in scripts/verify_no_internal_host.py, run over both files: no hits.
  • Every cited file and line was re-read at 2041e8681c.
  • Proof: not provable -- docs or config only

Manual verification

N/A: documentation only.

Screenshots / video

Why no screenshot: no markup change; Markdown files only.

Related Issues

Part of #37361. Refs #37538, #12693, #6338, #37381, #35488.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 15, 2026 17:22
@buluoray
buluoray requested a review from CrysisDeu November 15, 2026 17:22
@buluoray
buluoray enabled auto-merge (squash) November 15, 2026 17:22
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision 437f1715b0f5d8bad215b30f6cac5e22eda1d20f ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 437f1715b0f5d8bad215b30f6cac5e22eda1d20f touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 437f1715b0f5d8bad215b30f6cac5e22eda1d20f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

This is a docs-only draft RFC, and its storage claim checks out: member_goals.py says member-records/ is OS-masked in every mode. Two notes below, about the RFC contradicting itself.

Notes

  • Open decision 7 recommends amending rfc-work-ledger-concurrency-cap.md §7.2. The header lists §7.2 under "Builds on, and does not change", and the description says the RFC "does not contradict" it. Either take §7.2 off that list, or make decision 7 conditional on a separate amendment.
  • The description says "Section 10 has six open decisions", but §10 lists seven. The one left out, decision 7, is the one that changes an accepted RFC, so a maintainer reading only the description misses it.

[DESIGN-REVIEWED] 437f171

Description read

[DESCRIPTION-READ] 05ff3f92f08976b322de51fffa9656d0446b843c0e0c1a5190d9c6b39c52d314

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 437f1715b0f5d8bad215b30f6cac5e22eda1d20f and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 437f171

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 437f1715b0f5d8bad215b30f6cac5e22eda1d20f false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 437f1715b0f5d8bad215b30f6cac5e22eda1d20f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 437f171

Verdict parsed from the review's SHA-scoped output markers for commit 437f1715b0f5d8bad215b30f6cac5e22eda1d20f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 437f1715b0f5d8bad215b30f6cac5e22eda1d20f false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

KiroCrewCaptain commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider estimates a 10% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 404 lines, more than the 400 it may approve on its own.

This PR adds documentation only: a new draft RFC (rfc-goal-limits-and-done-check.md) and one draft row in the RFC index. It changes no code, so it cannot regress behaviour or lose data, and no test is owed.

Trust boundary. The RFC adds owner-only fields to a Bot's goal record. Its claim about where that record lives checks out: the member_goals.py docstring says the member-records/ directory is masked by the OS sandbox in every mode, and the Bot's file tools are refused there. The design keeps those fields off every Bot tool, and the Bot's only way to touch the done check is a change card the owner confirms, the same pattern set_key_results uses today. The Bot cannot raise any limit, because the store takes the lower of the owner's limit and the Bot's own concurrency_cap.

Citations. The ones I spot-checked match the head commit:

  • concurrency_cap in the mcp_work.py schema at line 759.
  • DEFAULT_GOAL_CONCURRENCY_CAP resolves to 32.
  • In the goal-conductor SKILL.md, lines 241, 433, 781, 834 and 900 say what the RFC quotes.

CI. All checks are green, and every automated review lane passed or was skipped.

Remaining issues. The internal inconsisten

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 437f1715b0 · 404 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@buluoray
buluoray force-pushed the crew/slack/rfc-goal-limits-drift-done-check branch from c256114 to 437f171 Compare November 15, 2026 19:00
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: passed Eligible automated validation passed for the current revision readiness: checking Automated validation is still running labels Nov 15, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

This adds a new concept, so it needs a design sign-off from a maintainer before it can merge. A maintainer will post the decision here.

Review result: no blocking findings on this head. The PR changes docs only, a new draft RFC and one README index row, so no code or test rule applies. We checked the security claims against the code: member-records/ is a sealed leaf, the Bot can write its own concurrency_cap, DEFAULT_GOAL_CONCURRENCY_CAP is 32, and the four evaluator kinds exist. Of 15 line citations we checked, all point at the right code, with one off by a single line. One thing done well: every proposed limit refuses work in the store with a code, so none of them is a prompt notice the Bot can ignore. The rollout phases also have exit criteria a test can fail.

Optional:

  • The "what already shipped" baseline is stale. feat(mate): let the owner cap how many workers a Bot keeps open #37381 (the per-Bot max_workers cap, bot_worker_cap_reached) is already in 2041e8681c, the commit the RFC says it audited. Yet §2.2 row 1, §5 row 1 and the README row call it missing or open. fix(goal-conductor): park findings that no Done when condition needs #37648 and fix(goal-conductor): treat an owner's remark as a suggestion until the owner makes it a rule #37651 have also merged since then, so §2.2 rows 10 and 11 and §5 rows 4 and 5 need updating. Re-auditing against current main and bumping audited-at would fix all of these at once.
  • The header lists rfc-work-ledger-concurrency-cap.md §7.2 under "Builds on, and does not change", but decision 7 amends §7.2 if it is accepted. The PR body also says §10 has six open decisions, but it lists seven.
  • The new limits, signals, done_check and parked[] fields sit at the per-Bot record level. The RFC does not say how a work ledger goal (a fresh goal_id) names the member goal it serves. Without that link, the "per goal" limits in §9 cannot be attributed to a goal. Either define the key or declare the fields per-Bot. If they become per-Bot, the per-goal workers_at_once limit could be dropped in favor of the shipped max_workers.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review input on head 437f1715b0f5. This is an RFC, so this is a comment, not an approval.

The core move is sound: every limit and signal refuses work in the store with a code, so none of them depends on the Bot following a prompt rule. The rollout phases have exit criteria a test can fail. docs-lint.sh and test/test_pr_quality_gates.py (97 passed) are clean locally, and all CI checks pass on this head.

Things to fix in the text

  1. Stale baseline. Since audited-at: 2041e8681c, #37381, #37648 and #37651 have merged, and the branch crew/slack/goal-cost-on-goals-tab is now #37665, merged today. §2.2 rows 1, 10, 11, 12, §5 and the README row still call these open or unshipped. Re-audit against current main and bump audited-at.
  2. Self-contradiction (also raised by the Design Review). The header lists rfc-work-ledger-concurrency-cap.md §7.2 under "Builds on, and does not change", but open decision 7 recommends amending it. Either move §7.2 out of that list or make decision 7 conditional.
  3. §4.2 step 2 refuses a session_create from "the Bot's chat or its workers" when a goal's signal fires. session_create is not tied to a goal, so a Bot with two goals, or an owner asking the Bot for something unrelated, would also be stopped. Say how a session is attributed to a goal, or scope the refusal to the goal's board and its workers only.

Open decisions

  1. Default lines. I would not stop by default on every signal at once. Phase 1 already measures the counts; ship stop-by-default only for signals whose source phase 1 confirmed and whose line held on real goals, and show the rest only. A busy but healthy goal (for example a review patrol) can cross "PRs per day" or "pushes after open" without drifting.
  2. Which issues count. Agree: only issues the goal's sessions opened or closed.
  3. Bot proposes a higher limit. Agree: no proposal tool. The stop report saying what is spent is enough.
  4. Hours. Agree: wall clock minus held time.
  5. No done check. Agree: run as today and show "No done check".
  6. Credits before #35488. Agree: wait.
  7. Plan concurrency as a user-set cap. I lean no. The plan is text the Bot writes, and one go-ahead covers every round, so treating its number as "set by the user" blurs the owner-only rule §4.1 builds on. §4.3 already gives the owner-only bound: prefill workers_at_once from the plan's number and let the owner confirm it on the Goals tab. That needs no amendment to §7.2, and #37652 could then write that confirmed value instead.

A Kiro Crew team member must sign off on this design before it merges; this review does not accept it.

AI-assisted, sent by Bolin Review bot

@buluoray
buluoray merged commit c2f1b32 into main Nov 16, 2026
101 checks passed
@buluoray
buluoray deleted the crew/slack/rfc-goal-limits-drift-done-check branch November 16, 2026 01:29
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants