Skip to content

fix(goal-conductor): treat an owner's remark as a suggestion until the owner makes it a rule - #37651

Merged
buluoray merged 1 commit into
mainfrom
crew/slack/owner-remark-suggestion
Nov 15, 2026
Merged

buluoray merged 1 commit into
mainfrom
crew/slack/owner-remark-suggestion

Conversation

@buluoray

Copy link
Copy Markdown
Contributor

Part of #37361

Problem / Motivation

Goal: A Bot treats an owner's passing remark as a suggestion for that case, and asks once, naming the cost, before it turns the remark into a rule for every change.

An owner wrote "maybe even a separate security reviewer, for the credentials" once. The Bot made it a security review, with re-checks, for every change that touches credentials, roles, the API, install or deploy. It never asked what that rule costs.

Why it matters

A remark that silently becomes a rule for every change multiplies the work on every later change. The owner only finds out days later, when the Bot's work has grown past what it planned.

Not a goal

  • No change to learn_add's code or to how saved lessons are injected.
  • No change to the Bot conductor prompt in crewmate_prompts.json; it says nothing about lessons.
  • No goal limits, drift signals or done-check from the same issue; other PRs carry those.

What changed (motivation → approach → change)

The goal-conductor skill told the Bot to save a person's correction as a lesson, and said nothing about remarks. A remark therefore had no default reading, and the Bot read "maybe" as a rule.

The skill's Patrol section now says that an owner's remark applies to the case it was said about unless the owner says it is a rule. Before the Bot turns a remark into a rule for every change, it asks the owner once and names what the rule adds to each change. The same check comes before saving a remark with learn_add: the Bot saves it as a rule only when the owner said so or agreed, and keeps the scope the owner gave.

Backwards compatibility

Compatible: skill text only. A correction the owner gives still becomes a lesson as before; only an unconfirmed remark waits for one question.

Removes nothing: the diff adds one paragraph to src/kiro_crew/builtin_skills/goal-conductor/SKILL.md and one test file.

Tests

test/test_conductor_owner_remark_suggestion.py pins the three rules in the Patrol section: a remark is a suggestion unless the owner says it is a rule; ask once and name the cost before a rule for every change; the same check before learn_add. Mutation-checked: removing the paragraph turns all three red, and removing each phrase turns its own test red. Determinism: the file passed 5 repeats and a reversed test order.

Proof: PROVEN test/test_conductor_owner_remark_suggestion.py::test_owner_remark_is_a_suggestion_unless_the_owner_says_it_is_a_rule, test/test_conductor_owner_remark_suggestion.py::test_bot_asks_once_and_names_the_cost_before_a_rule_for_every_change, test/test_conductor_owner_remark_suggestion.py::test_learn_add_gets_the_same_check (head 9ef5b17)

Manual verification

N/A — the change is skill wording, and the test reads that wording from the shipped skill file.

Related Issues

Part of #37361

Sibling sweep

Swept: git grep -n -i "as a lesson|learn_add" -- src/kiro_crew/builtin_skills src/kiro_crew/crewmate_prompts.json
Sibling: src/kiro_crew/builtin_skills/goal-conductor/SKILL.md:461 -- changed here
Sibling: src/kiro_crew/builtin_skills/kirocrew-dev/dashboard-template/agent-spec.md:119 -- not affected -- that agent is told never to call learn_add

Pattern harvest

Not generalizable: the defect is one missing default in the conductor's instructions, not a code shape a lint rule can match.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@buluoray
buluoray requested review from a team and KiroCrewCaptain as code owners November 15, 2026 17:39
@buluoray
buluoray requested a review from bolichen97 November 15, 2026 17:39
@buluoray
buluoray enabled auto-merge (squash) November 15, 2026 17:39
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 9ef5b1773afdbbc135e0640c9b29427a04d71b70 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 9ef5b1773afdbbc135e0640c9b29427a04d71b70 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design concerns: the change adds one paragraph of skill text, placed right after the "save a correction as a lesson" paragraph that caused the problem, and it removes nothing.

Notes

  • The case in the description is a crew-wide security review, and the skill also has a ### Goal changes mid-flight section (line 833). Consider a one-line cross-reference there, so a remark that would widen the plan gets the same ask-once check.

[DESIGN-REVIEWED] 9ef5b17

Description read

[DESCRIPTION-READ] bc8500dfb175aa172bbd86417a9422f8eef2d4a3b9fbb5e361474e7cd27e8f30

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 9ef5b1773afdbbc135e0640c9b29427a04d71b70 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks: the change adds one paragraph to the conductor skill, its defect is tracked in #37361, and it does not touch the explicit-correction path.

What this change ships

Inventory (4 items) — 4 justified

Intent: stop a goal-conductor Bot from turning an owner's one-off "maybe" into a costly standing rule for every change. This is a FIX.

  1. The Bot reads an owner's passing remark as applying only to the case it was said about. — justified
  2. A new step: the Bot asks the owner once, naming the extra work per change, before making a remark a rule for every change. — justified
  3. The Bot saves a remark with learn_add only when the owner said it is a rule or agreed, and keeps the owner's scope. — justified
  4. A new test pins that wording in the shipped skill's Patrol section. — justified

[FIRST-PRINCIPLES-REVIEWED] 9ef5b17

Description read

[DESCRIPTION-READ] 61aa7c5bd61967d58f111b622ff310d0232cdd717677d7ec3f01980182f34662

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9ef5b1773afdbbc135e0640c9b29427a04d71b70 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 9ef5b17

Verdict parsed from the review's SHA-scoped output markers for commit 9ef5b1773afdbbc135e0640c9b29427a04d71b70.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9ef5b1773afdbbc135e0640c9b29427a04d71b70 false-positive: <reason, 20 to 500 characters>

@github-actions

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9ef5b1773afdbbc135e0640c9b29427a04d71b70 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9ef5b17

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9ef5b1773afdbbc135e0640c9b29427a04d71b70 false-positive: <reason, 20 to 500 characters>

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: Approve

Why: the decider estimates a 6% chance that this change introduces a defect, within the 12% limit for approving on its own, and every check passes.

The PR adds one paragraph to the Patrol section of goal-conductor/SKILL.md. It says an owner's passing remark is a suggestion for that case unless the owner says it is a rule. Before making it a rule for every change, the Bot asks once and names the cost, and the same check applies before saving the remark with learn_add. The existing correction-as-lesson path is unchanged. The new test reads the shipped skill and slices it between '### Patrol' (line 401) and '### Close the round' (line 739). The paragraph (lines 465–471) falls inside that range, so the test fails if the paragraph is removed. It imports _BUILTIN_SKILLS_DIR from kiro_crew.skills, which re-exports it and which seven other tests import the same way. The parity tests in test_mate_conductor_parity.py only check that anchor phrases are present, so added text cannot break them. I found no skill size or snapshot limit. No other skill uses the 'as a lesson' wording, and docs/decisions has no ruling on remarks or lesson saving. The change is wording only: no data, trust or lifecycle impact.

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 9ef5b1773a · 46 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 15, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

This PR is part of #37361, and that issue is tracked as a design-level change to how goal-driven Bots run. This adds a new concept, so it needs a design sign-off from a maintainer before it can merge. A maintainer will post the decision here.

Review result on this head: no blocking findings. The diff adds one paragraph to the Patrol section of goal-conductor/SKILL.md and adds one new test file. Nothing is removed. I checked the test's assertions against main and against this head. All four pinned phrases are missing on main and present here, so reverting the paragraph turns all three tests red. The tests only read the shipped skill file, with no clock, network or global state. No other open PR adds the same wording, and nothing in docs/decisions/ covers how a remark or a rule should be handled. Well done: the paragraph sits right after the "save a correction as a lesson" sentence it qualifies, and it covers the learn_add path too.

Optional:

  • feat(prompts): one reroute-first ask policy for every crew kind #37609 limits questions in the same skill to ask cases (a) to (e). Whichever PR merges second should check that the new "ask the owner once" wording still reads as case (e), and that it parks instead of asking in a turn nobody typed.
  • test_learn_add_gets_the_same_check could also assert "keep the scope the owner gave" and "save it as a rule only when the owner said so or agreed when you asked". Right now those clauses could be dropped without any test going red.

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving 9ef5b1773a.

  • The new Patrol paragraph is scoped well: a remark stays a suggestion for its own case, the Bot asks once and names the cost before widening it, and the same check sits in front of learn_add. A real correction still becomes a lesson as before.
  • Ran test/test_conductor_owner_remark_suggestion.py locally: 3 passed. CI is green on this head.

AI-assisted, sent by Bolin Review bot

@buluoray
buluoray merged commit 4b346e1 into main Nov 15, 2026
98 checks passed
@buluoray
buluoray deleted the crew/slack/owner-remark-suggestion branch November 15, 2026 19:12
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants