Skip to content

feat(conductor): check each worker's new transcript for drift on every patrol turn - #36302

Merged
bolichen97 merged 1 commit into
mainfrom
conductor-drift-check
Nov 13, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
conductor-drift-check

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A goal conductor notices a worker heading the wrong way while it works, and steers it, instead of finding out at acceptance.

Owner's ask: "类似的问题,conductor没有办法主动对worker的工作纠错。我们目前其实有N轮一定苏醒,但是苏醒后它应该主动检查下worker的工作,是否偏离方向" (the conductor has no way to correct a worker's work; it already wakes at least every N quiet rounds, and on waking it should check whether each worker has drifted).

Today the goal-conductor skill's patrol step 5 says session_read_message is "never as the routine cycle read", and Cost discipline repeats it. So a conductor sees only each worker's own work_report claims. A worker that builds something its item does not name, or widens scope, looks on track until its done fails acceptance.

Why it matters

Every off-direction item costs a whole worker run plus a re-plan round. The patrol already wakes (on blocked/question/done, a stale/orphaned flag, and the forced floor turn after a quiet run), so the conductor is awake at the right moments. It just never looks.

Not a goal

  • No new code, tool, field or monitor behaviour. Prose only.
  • No change to acceptance: verdicts still come from accept_eval, never from a transcript.
  • No liveness reading: stale/orphaned stay the liveness signal.
  • Not the round-planning text near "two or three items per round" or the concurrency prose in spec_prompts.py (sibling PR).

What changed (motivation → approach → change)

Gap → the conductor reads only claims → on every patrol turn, read each open worker's NEW lines from a stored cursor and steer when they drift.

  • goal-conductor/SKILL.md Patrol step 5: a "Check each worker for drift" paragraph. Per open item: session_read_message with since = the item's cursor kept in the session ledger's next, limit: 10; no new rows means skip; more than 10 new rows (or cursor_unavailable) means read the newest 10 without since. Compare against the item's title, acceptance and the goal. Off direction: session_send with steer: true plus action=decide. On track: silent. The cost is stated as the RFC bounds it: at most two reads per open item per patrol turn (the since read, plus the newest 10 only on overflow), so at most 20 rows per item. A remote-dispatched item is skipped on a patrol wake, because reading and steering it need an owner turn (the skill's Remote dispatch section).
  • Round-close next now carries each item's drift cursor. Cost discipline names the drift read as an allowed use.
  • spec_prompts.py read_message_detail (shared by the root conductor charter and the crewmate conductor's conductor.patrol): the same rule in short, including what to do with no stored cursor (no since, read the newest 10), since the crewmate conductor is not taught the session-ledger cursor store. The crewmate golden fixture follows.

Why not a monitor judge. monitor_start/monitor_update judge can take worker chat keys as targets and screen their new assistant lines with a wake_when like "a worker's new lines show it working outside its item". Checked in autonudge_service/gate.py and judge_tick.py, it cannot carry this alone:

  1. It only answers wake/quiet on a QUIET probe tick; a report wake is never screened, so drift on a woken turn still needs the conductor's own read.
  2. It sees two 500-char criteria and the transcript tail, not each item's title or acceptance (up to 6 items).
  3. It cannot steer; a wake still needs the read to say what is wrong.
  4. Targets change with every dispatch, so it adds a monitor_update per dispatch.

So the rule above is the smaller complete change; the judge is not adopted.

RFC. rfc-conductor-work-ledger.md replaced the transcript-reading patrol for liveness and verdicts, and keeps "session_send to ask and session_read_message to read the answer… The ledger removes the polling, not the conversation" (§ around line 722). This read is neither liveness nor a verdict, and it is bounded by the cursor. The product-shape decision is docs/request-for-change/rfc-conductor-drift-check.md (accepted), merged in #36360; this PR is rebased onto it.

Backwards compatibility

Compatible: prose only; no tool, schema, field or stored format changes.

Removes nothing: the "never as the routine cycle read" sentence is replaced by a bounded, purpose-limited read; every existing use of session_read_message stays allowed.

Tests

test/fixtures/crewmate_prompts/ledger.golden.txt updated to the new shared rule (a reading aid). test/test_agent_refactor_spec_bytes.py: the kirocrew-conductor.json and kirocrew-ledger-conductor.json digests in all 8 scenarios move to the bytes CI printed for this change (the goldens are CI's bytes and cannot be regenerated locally); no other spec file changed. test/test_agent_refactor_contracts.py pins _CONDUCTOR_SYSTEM_PROMPT's digest; it moves to the new prompt's (2eabb56eb275fc24). The spec-byte goldens were regenerated locally: on the unchanged base the local run matched CI's goldens (8 passed), so its digests are CI's. Every test file that names the conductor prompt, the goal-conductor skill or spec_prompts (111 files) was run locally on this head; all pass except two in test_slack_code_channels_binding.py that only fail when run in a mixed batch (the file alone: 60 passed). Ran against the worktree's src: test_mate_conductor_parity.py, test_crewmate_prompts.py, test_conductor_agent.py, test_conductor_skills_steer_corrections.py, test_goal_conductor_skill_grants.py, test_needs_human_is_checked.py, test_work_ledger_pending_budget.py, test_mate_conductor_agent.py, test_team_lead_agent.py, test_ledger_conductor_agent.py: 224 passed.

Manual verification

N/A — prompt and skill prose; the parity and prompt tests cover the shared-rule assembly.

Related Issues

Closes #36299

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 146cf024a09286ceb968bf10dec2dcbb2708e11d touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 146cf024a09286ceb968bf10dec2dcbb2708e11d — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the accepted drift-check RFC defines this change, the read is capped by the cursor and limit: 10, and the tradeoff against using a monitor judge is spelled out.

Notes

  • The shared read_message_detail rule never tells the crewmate conductor where to store a cursor, so on every patrol turn it reads the same newest 10 rows. It can't skip a quiet worker, and it may steer twice on lines it has already judged. The cost stays bounded, so this is a note, not a blocker.

[DESIGN-REVIEWED] 146cf02

Description read

[DESCRIPTION-READ] acfa1a46e34efa6bc9fbe796b33166e871f4441407c75117c44d3e13f53e6739

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 146cf024a09286ceb968bf10dec2dcbb2708e11d — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The accepted base RFC rfc-conductor-drift-check.md records the change to the patrol's default, and the diff follows its section 5 rollout item by item.

What this change ships

Inventory (7 items) — 7 justified

Intent: a goal conductor notices a worker going the wrong way while it works and steers it back, instead of finding out at acceptance. ADDITION.

  1. On every patrol turn, the floor turn included, the goal conductor reads each open local worker's new transcript lines (since cursor, limit: 10) — justified
  2. When the new lines drift from the item, the conductor sends a correction with steer: true and records it with action=decide; on track it stays silent — justified
  3. The round-close next record now carries each item's drift cursor — justified
  4. When a worker wrote more than 10 rows, or the cursor is refused (cursor_unavailable), the conductor reads the newest 10 once — justified
  5. Remote-dispatched items are skipped on a patrol wake — justified
  6. The shared read_message_detail rule teaches the same check to the root conductor and the Bot (crewmate) conductor, with a no-cursor fallback — justified
  7. Cost discipline now allows the drift read — justified

[FIRST-PRINCIPLES-REVIEWED] 146cf02

Description read

[DESCRIPTION-READ] 886aa7f8c9dfea67e1c118cb7e4d5108b9ac2e55489ba281bb45112f0ed760bc

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 146cf024a09286ceb968bf10dec2dcbb2708e11d and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 146cf02

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 146cf024a09286ceb968bf10dec2dcbb2708e11d false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 146cf024a09286ceb968bf10dec2dcbb2708e11d — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 146cf02

Verdict parsed from the review's SHA-scoped output markers for commit 146cf024a09286ceb968bf10dec2dcbb2708e11d.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 146cf024a09286ceb968bf10dec2dcbb2708e11d false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Nov 13, 2026
@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from b7a7e12 to 84f5927 Compare November 13, 2026 03:31
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 13, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider raised 2 possible problems and estimates a 68% chance that this change introduces a defect; a person should check them.

This prose change adds a per-turn transcript drift read to every conductor. That reverses the work-ledger RFC's stated design ("replaces the patrol cycle … instead of a per-item transcript read with a stored cursor"). The PR says the RFC that records the new decision (#36360) is on its base, but the file is not in the repo at this head, and the work-ledger RFC is not updated. CONTRIBUTING requires an RFC for a changed default. Separately, the shared short rule and the root charter's round-close text don't tell the conductor to store or write back the cursor, while the skill does. On those paths the cursor is lost at reset. Request changes until the RFC is on the base (or a maintainer overrides) and the charter's cursor handling matches the skill.

Things to check (possible problems the decider raised; at this confidence many turn out to be fine):

  • This is a product-shape change to the default conductor patrol that reverses the accepted work-ledger design, and no RFC or maintainer override is on the base. The work-ledger RFC is left stale. (src/kiro_crew/builtin_skills/goal-conductor/SKILL.md, medium)
  • Cursor handling is only fully specified in the skill. A conductor following its charter or the crewmate prompt loses the cursor at each reset, so it re-reads the newest tail or skips rows. The stated goal is met on some paths only, and the charter now disagrees with the skill. (src/kiro_crew/agent_parts/spec_prompts.py, medium)

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit 84f59271be · 72 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from 84f5927 to 3676926 Compare November 13, 2026 04:19
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 13, 2026
@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from 3676926 to f143dc4 Compare November 13, 2026 07:40
@iamwhatever

Copy link
Copy Markdown
Contributor Author
  • fixed span=5256f01201e4 in f143dc4

The product-shape decision is now on the base: docs/request-for-change/rfc-conductor-drift-check.md (status accepted), merged in #36360, records the per-patrol drift read of each open local worker.
This head is rebased onto it, and the skill text now matches the RFC bounds: at most two reads (20 rows) per item per patrol turn, and remote-dispatched items are skipped on a patrol wake.
The read is neither liveness nor a verdict; stale/orphaned and accept_eval are unchanged.

@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Nov 13, 2026
@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from f143dc4 to 66827ab Compare November 13, 2026 07:53
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from 66827ab to 1d4a5c5 Compare November 13, 2026 08:35
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 13, 2026
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 13, 2026
@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from 1d4a5c5 to 0c774b5 Compare November 13, 2026 10:02
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 13, 2026
@iamwhatever
iamwhatever force-pushed the conductor-drift-check branch from 0c774b5 to 146cf02 Compare November 13, 2026 12:40
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention merge conflict Branch has merge conflicts with its base — author must resolve before merge labels Nov 13, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. I read the whole diff on this head. All checks are green.

Within the signed RFC (#36360)

  • Text only: read_message_detail in spec_prompts.py, goal-conductor step 5, the next field and "Cost discipline". The rest is the regenerated goldens and spec hashes.
  • No new tool, route or storage. The drift check uses the existing session_read_message, with since and limit: 10, so it goes through that tool's own target check and redaction. A correction is a session_send steer: true plus work_ledger_record action=decide. The cursor lives in the conductor's own session ledger next.
  • Bounded as signed: at most two reads per open item per patrol turn, so at most 20 rows per item. Patrol turns come only from a report, a flag or the quiet-run floor. A remote item is skipped on a patrol wake.
  • Nothing new is refused or returned less, and there is no regex.

Notes, not blockers

  • Cursor format in next. The skill says "its drift cursor (step 5)" but gives no spelling. Different conductors will write it differently, and a later turn has to read it back from free text. One fixed form in the step-1 next list, for example cursor=<total> on each item's line, would make the read-back reliable.
  • The 6-item bound comes from the board, not this text. The 20-rows-per-item cap is in the skill, but the number of items read is "each OPEN local item", which today is limited by concurrency_cap (6). #36304 (signed) moves that default to 32. Once it is built, a patrol turn could read up to 32 × 20 rows. If the RFC's 6-item ceiling is meant to hold, the skill should say "at most 6 items per turn, oldest first".
  • The Bot (mate) conductor prompt hashes are unchanged, so the drift check reaches only the Ultra Mode and ledger conductors for now. That is fine if the RFC scopes it that way.

Direction signed by maintainer (bolichen97) as RFC #36360.

@bolichen97
bolichen97 merged commit 701e45f into main Nov 13, 2026
155 of 158 checks passed
@bolichen97
bolichen97 deleted the conductor-drift-check branch November 13, 2026 15:06
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 13, 2026
buluoray pushed a commit that referenced this pull request Nov 15, 2026
…y patrol turn (#36302)

Co-authored-by: Joe Guo <zejiangg@amazon.com>
(cherry picked from commit 701e45f)
buluoray pushed a commit that referenced this pull request Nov 15, 2026
…y patrol turn (#36302)

Co-authored-by: Joe Guo <zejiangg@amazon.com>
(cherry picked from commit 701e45f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Goal conductor never checks a worker's transcript for drift on patrol

3 participants