Skip to content

test(conductor): guard Ultra Mode's team lead reach of shared conductor rules - #36529

Merged
bolichen97 merged 1 commit into
mainfrom
test/ultra-team-lead-skill-parity
Nov 13, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
test/ultra-team-lead-skill-parity

Conversation

@iamwhatever

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A test fails when Ultra Mode's team lead stops reaching a shared conductor rule.

Ultra Mode runs kirocrew-team-lead (crew_mode.DEFAULT_CREW_MODE_AGENT). Its system prompt carries none of _LEDGER_CONDUCTOR_RULES. It reaches them only through two skills: team-lead says "Read goal-conductor first", and goal-conductor/SKILL.md holds the procedure. test/test_mate_conductor_parity.py checks only the plain conductor against the Bot's conductor, so nothing guards the Ultra path. Its docstring also still names kirocrew-conductor as the Ultra agent.

Why it matters

A shared rule can be added or dropped for the Bot and the plain conductor while the skill never gets it, and Ultra silently runs without it. Three open PRs add or change shared rules right now (#36481 host_pressure, #36312 resource_wave, #36302 drift in read_message_detail).

Not a goal

What changed (motivation → approach → change)

Ultra reads shared rules only through skills → so the test keys each shared rule to short anchor phrases in goal-conductor/SKILL.md, not to full text, and ordinary rewording does not break it.

  • ULTRA_SKILL_ANCHORS: rule key → anchor phrases. ULTRA_GAPS: rule key → (anchor, reason) for rules the skill does not carry today.
  • Every _LEDGER_CONDUCTOR_RULES key must sit in exactly one of the two maps. A new key (e.g. host_pressure, resource_wave) fails until it is mapped: that is the extension point.
  • Mapped anchors must be in the skill. A gap whose anchor appears in the skill is reported as stale.
  • The chain is checked: Ultra's default agent is the team lead, its prompt names team-lead and goal-conductor, the team-lead skill says "read goal-conductor first", and the installed spec maps both skills.
  • Docstring fixed: kirocrew-conductor is the plain and child conductor; the team lead section is described.

Two gaps on main today: dependent_calls and patrol_requested have no counterpart in either skill.

Backwards compatibility

Compatible: test-only; no product code changes.

Removes nothing: the diff adds tests and rewords a docstring.

Tests

  • test_every_shared_rule_has_an_ultra_counterpart_or_a_named_gap: each shared key is mapped or gapped, and no entry names a missing key.
  • test_goal_conductor_skill_carries_each_mapped_rule: every anchor is in the skill.
  • test_every_ultra_gap_is_still_a_gap: a closed gap is reported.
  • test_the_anchor_check_sees_a_lost_rule: positive control, a missing anchor is caught by key.
  • test_ultra_mode_reaches_goal_conductor_through_team_lead: the prompt → skill → skill chain and the spec's skill mapping.

Proof: not provable -- test-only change

Determinism: the whole file passed 24/24 three times in a row, and once with all 24 node ids in reversed order (-n0). The new tests read checked-in text and one spec written under tmp_path; no clock, network or shared state. test/test_flake_pattern_ratchet.py: 68 passed.

Manual verification

N/A — unit coverage sufficient; the tests read checked-in prompt and skill text and one installed spec in tmp_path.

Related Issues

Closes #36501

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@iamwhatever
iamwhatever requested review from a team and KiroCrewCaptain as code owners November 13, 2026 09:15
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 13, 2026
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision aea80a723860a6da02be33f6a0a2b057d8eb49b1 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision aea80a723860a6da02be33f6a0a2b057d8eb49b1 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of aea80a723860a6da02be33f6a0a2b057d8eb49b1 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change only adds tests and covers a real unguarded path. No product code changes and nothing is taken away.

Notes

  • Anchor phrases are matched by plain substring, so a skill could keep the phrase but invert its meaning and still pass. That is fine for catching a dropped rule, but don't count it as proof the rule's meaning holds.

[DESIGN-REVIEWED] aea80a7

Description read

[DESCRIPTION-READ] 2a09f5f8e1a63c1e57eac0a68eac8be82d4c23daef44ce416315bbc86e6657f6

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed aea80a723860a6da02be33f6a0a2b057d8eb49b1 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] aea80a7

Verdict parsed from the review's SHA-scoped output markers for commit aea80a723860a6da02be33f6a0a2b057d8eb49b1.

False positive or not applicable? A repository writer can comment:
/ai-review override fable aea80a723860a6da02be33f6a0a2b057d8eb49b1 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of aea80a723860a6da02be33f6a0a2b057d8eb49b1 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] aea80a7

False positive or not applicable? A repository writer can comment:
/ai-review override gpt aea80a723860a6da02be33f6a0a2b057d8eb49b1 false-positive: <reason, 20 to 500 characters>

@iamwhatever
iamwhatever force-pushed the test/ultra-team-lead-skill-parity branch from 7387421 to d9c146b Compare November 13, 2026 09:25
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Rebased: main moved (docs-lint index fix #36504 landed) since this head ran (d9c146b -> aea80a7); no touched file changed. CI re-running.

@iamwhatever
iamwhatever force-pushed the test/ultra-team-lead-skill-parity branch from d9c146b to aea80a7 Compare November 13, 2026 09:40
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Nov 13, 2026

@bolichen97 bolichen97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the full diff. Test-only, for #36501. It adds a guard that Ultra Mode's kirocrew-team-lead still reaches every shared _LEDGER_CONDUCTOR_RULES entry through the team-lead -> goal-conductor skill chain.

  • Every shared rule key must be either in ULTRA_SKILL_ANCHORS, with anchor phrases present in goal-conductor/SKILL.md, or in ULTRA_GAPS with a reason. No key may be in both, and no entry may name a rule that does not exist.
  • A closed gap must be moved, so the gap list cannot go stale.
  • A positive control shows that a lost anchor is reported by its key.
  • A last test pins the prompt -> team-lead -> goal-conductor chain and that the installed team-lead spec maps both skills.

No product code changes. LGTM.

@bolichen97
bolichen97 merged commit ae706ae into main Nov 13, 2026
99 of 103 checks passed
@bolichen97
bolichen97 deleted the test/ultra-team-lead-skill-parity branch November 13, 2026 14:07

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, security checklist all-NO, no reviewer-lane findings, readiness signal fresh for this head SHA, declared category matches the diff. Category: adds parity tests for the team lead's reach of shared conductor rules; test file only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: passed Eligible automated validation passed for the current revision

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: no test guards that Ultra Mode's team lead reaches the shared conductor rules

3 participants