feat: Add retro workflow for retrospective analysis

[bootc-bot]

Summary

Adds a new scheduled workflow (retro.md) that performs retrospective analysis of workflow runs across repositories where gh-agentic-workflows is deployed.

What it does

• Scheduled execution: Runs every 6 hours (at :17 past the hour to avoid load spikes)
• Multi-repo analysis: Analyzes workflow runs in target repositories (currently bootc-dev/bcvk, configurable via workflow_dispatch)
• Pattern detection: Identifies recurring failures, performance issues, flaky tests, and optimization opportunities
• Duplicate prevention: Checks existing issues before filing new ones
• Actionable outputs: Creates issues with concrete improvement suggestions, labeled agent/retro

Validation

• Workflow compiled successfully with gh aw compile --approve
• Generated retro.lock.yml lockfile
• Follows the same patterns as existing standalone workflows (ci-triage.md, queue-triage.md)
• Uses safe-outputs for issue creation with a cap of 5 issues per run

Implementation details

• Pre-fetch step prepares target repo list and lookback window (default 7 days)
• Agent uses GitHub MCP tools to analyze runs and existing issues
• Permissions: read-only for contents/actions/issues/pull-requests, write via safe-outputs
• Timeout: 20 minutes
• Concurrency: Each run gets its own group (no blocking between runs)

Resolves #73

Generated by Drafter for #73 · sonnet45 · 79.8 AIC · ⌖ 22 AIC · ⊞ 3.8K · ◷

[bootc-bot]

Review Complete: Looks Good to Merge

I've reviewed this PR and it looks excellent. Here's what I checked:

✅ Correctness

• Workflow configuration: Valid gh-aw syntax, properly structured frontmatter
• Schedule: Every 6 hours at :17 past the hour (good - avoids the :00/:30 stampede)
• Permissions: Read-only (contents, actions, issues, pull-requests) - appropriate for analysis
• Safe-outputs: Correctly configured with max: 5 for create-issue, auto-labels with agent/retro
• Pre-fetch step: Well-written defensive bash with set -euo pipefail, portable date command
• Concurrency: Uses run_id (allows parallel runs) - intentional per comments, race condition on duplicate issues is benign

✅ Code Quality

• Clear documentation in both YAML comments and prompt
• Follows patterns from existing workflows (ci-triage.md, queue-triage.md)
• Good design decisions explained in comments (e.g., concurrency strategy, cron timing)
• Duplicate prevention built into the agent prompt
• Actionable prompt with clear constraints and success criteria

✅ Safety

• Read-only GitHub access via MCP tools
• Reasonable limits (5 issues per run, 20 minute timeout)
• Safe bash tools only (cat, head, tail, grep, wc, ls, jq, date, sort, uniq)
• No destructive operations

✅ Functionality

Correctly implements all requirements from issue #73:

• ✅ Analyzes repos where deployed (default: bootc-dev/bcvk, configurable)
• ✅ Identifies patterns (recurring failures, performance issues, flaky tests)
• ✅ Files improvement issues
• ✅ Prevents duplicates
• ✅ Scheduled every 6 hours

✅ Repository Conventions

• No Signed-off-by (correct per CLAUDE.md for AI-generated code)
• Lockfile auto-generated correctly with proper metadata
• Resolves #73
• Follows same structure as other standalone workflows

Notes

• Default target repo "bootc-dev/bcvk" is documented as "Adopter-specific" - intentional for this demo repo
• PR is currently in draft status - ready to mark as ready for review when you're comfortable

This is high-quality work that's ready to merge.

[cgwalters]

I think we should scan the whole bootc-dev org by default

[cgwalters]

Mmm runs shouldn't last 6 hours but we also don't want multiple retros?

[cgwalters]

Why, this looks unnecessary

[cgwalters]

Also retro should only run in this repository not in any downstream ones I think, can we exclude the install from other repos or just skip?

[cgwalters]

Also TODO auto-trigger agent/fixme label on human requesting changes

[bootc-bot]

Commit pushed: 7cfa438

Generated by PR Fix Agent · sonnet45 · 80.2 AIC · ⌖ 9.95 AIC · ⊞ 4.6K

[bootc-bot]

Fixed the scope of the retro workflow per reviewer feedback.

Changes made:

• Changed default target repository from bootc-dev/bcvk to bootc-dev/gh-agentic-workflows (this repository)
• Updated workflow description and documentation to reflect that retro now analyzes runs in this repository only, not in downstream deployments
• Recompiled retro.lock.yml with the updated source

The retro workflow will now analyze workflow runs in this repository's own gh-agentic-workflows deployment, rather than looking at downstream repos where the framework is deployed.

Generated by PR Fix Agent for #74 · sonnet45 · 80.2 AIC · ⌖ 9.95 AIC · ⊞ 4.6K · ◷

[bootc-bot]

Review: retro.md workflow

I've reviewed the new retrospective analysis workflow. The overall design is solid and follows the established patterns from ci-triage.md and queue-triage.md. However, there's a critical portability bug that needs to be fixed.

Critical Issue: Hardcoded Repository Name

Location: .github/workflows/retro.md:95

The pre-fetch step hardcodes the repository name instead of using the dynamic github.repository context:

# Current (line 95):
REPOS="bootc-dev/gh-agentic-workflows"

Problem: This breaks the workflow if:

• The repository is forked
• The repository is renamed
• The workflow is copied to another deployment

Required Fix: Follow the same pattern as ci-triage.md and queue-triage.md:

1. Add REPO to the env block (after line 82):

env:
  GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  INPUT_REPOS: ${{ github.event.inputs.target_repos }}
  LOOKBACK_DAYS: ${{ github.event.inputs.lookback_days || '7' }}
  REPO: ${{ github.repository }}

2. Change line 95 to use the env var:

# Default: analyze this repository's workflow runs.
REPOS="$REPO"

This follows the documented pattern from ci-triage.md (lines 167-170) where env vars are used instead of inlining ${{ github.* }} expressions in run blocks (due to a compiler bug that silently drops the rest of the env block).

What I Checked

✅ Structure: Follows the same pattern as existing workflows
✅ Concurrency: Correctly uses run_id to avoid the single-slot trap
✅ Permissions: Appropriate read-only permissions
✅ Safe-outputs: Correctly configured with create-issue, noop, and missing-data
✅ Pre-fetch logic: Well-designed with proper error handling and BSD/GNU date compatibility
✅ Prompt: Clear, well-structured, and follows repository patterns
✅ Cron schedule: Uses :17 to avoid load spikes (good)
✅ Bash allowlist: Reasonable tool selection for the task

Once the hardcoded repository name is fixed, this will be ready to merge.

[bootc-bot]

Commit pushed: 70ae908

Generated by PR Fix Agent · sonnet45 · 53.7 AIC · ⌖ 9.48 AIC · ⊞ 4.6K

[bootc-bot]

Review Complete

The code implementation is solid and follows all repository conventions. However, the PR description contains stale information that needs correction before merge.

What I Checked

• ✅ Shell script length and error handling
• ✅ Workflow configuration and permissions
• ✅ Safe-outputs setup
• ✅ Follows existing workflow patterns (ci-triage.md, queue-triage.md)
• ✅ Agent prompt is clear and has proper constraints
• ✅ Commit messages are well-formatted

Required Fix

Update the PR description to accurately reflect that the workflow analyzes this repository by default, not bootc-dev/bcvk.

The "Multi-repo analysis" section currently says:

currently bootc-dev/bcvk, configurable via workflow_dispatch

But the code (retro.md:96) actually defaults to ${{ github.repository }} (this repo).

Suggested replacement:

Repository analysis: Analyzes workflow runs in this repository (bootc-dev/gh-agentic-workflows) by default, with support for analyzing other repositories via the target_repos workflow_dispatch input.

This matters because the PR description will become the squash commit message in main's history—it must accurately describe what the workflow does.

[bootc-bot]

Automated Fix Loop: Iteration Limit Reached

The automated fix loop has reached its iteration limit of 3 commits and automated fixing has stopped.

A human review is now required. To proceed, you can either:

1. Push a fix commit manually and apply the agent/lgtm label directly once satisfied, or
2. Close this PR if the changes are not viable

⚠️ Important: Re-applying the agent/fixme label will not give the loop another attempt. The iteration cap is based on the total commit count on the branch (currently 3), which only grows. Relabeling will immediately hit the same cap again without attempting a fix.

The only way to continue the automated loop is to reduce the branch's commit count below 3 first (e.g., by squashing commits).

Generated by PR Fix Agent for #74 · sonnet45 · 27.7 AIC · ⌖ 6.08 AIC · ⊞ 4.6K · ◷

[cgwalters]

Updated workflow description and documentation to reflect that retro now analyzes runs in this repository only, not in downstream deployments

No I mean the retro workflow should run from the context of this repository, but scan all the other repos in the organization.

[bootc-bot]

Reviewed the retro workflow addition. The implementation is solid:

• Pre-fetch script properly validates inputs and handles edge cases
• Agent prompt is clear with well-defined constraints
• Security controls are appropriate (read-only + safe-outputs)
• Integration is complete (CI, justfile, labels, docs)
• Correctly excluded from aw.yml for host-only deployment
• Follows established patterns from ci-triage and queue-triage

Ready to merge.

[TomasTomecek]

I truly love how simple this is, yet with the current frontier models it feels like this could be enough to get solid results

[bootc-bot]

Reviewed the diff (retro.md/retro.lock.yml, the model-alias switch across all workflows, and the upgrade.yml/scripts refactor) plus the current PR branch checkout.

What looks good:

• retro.md correctly implements the org-wide scan from the host repo that was requested in review (discovers org repos via gh api orgs/OWNER/repos, excludes the host repo by default, validates target_repos/lookback_days on dispatch) and the README documents the GITHUB_TOKEN cross-repo access limitation honestly.
• The sonnet model-alias switch is applied consistently across every workflow .md/.lock.yml pair (verified hashes/env vars match).
• scripts/gh-aw-version.mjs and scripts/setup-gh-aw.mjs are clean, well-factored, and tests/gh-aw-version.test.mjs covers them (plus the embedded upgrade.yml install script) thoroughly — I ran node --test tests/gh-aw-version.test.mjs locally and all 6 tests pass.

What needs to change:

• The new tests/gh-aw-version.test.mjs is never executed by CI. ci.yml only runs gh aw compile ... --approve plus the lock-drift check; there is no node --test tests/ (or equivalent) step, and justfile has no test recipe either. Per REVIEW.md ("Tests are expected for all non-trivial changes"), these tests need to actually run somewhere automated, otherwise they'll silently bit-rot the first time gh-aw-version.mjs/setup-gh-aw.mjs/upgrade.yml change. Please add a step to ci.yml (and ideally a just test recipe) that runs node --test tests/*.test.mjs.

[bootc-bot]

The automated fix loop has reached its iteration limit (3 commits on this branch) and automated fixing has stopped.

A human needs to review this PR and either:

• Push a fix commit manually and apply the agent/lgtm label directly once satisfied, or
• Close the PR.

Note: re-applying agent/fixme will not give the loop another attempt. This cap is simply the total commit count on the branch, which only grows, so relabeling will immediately hit the same cap again without attempting a fix. The only way to actually continue the automated loop is to reduce the branch's commit count (e.g. by squashing) below 3 first.

Generated by PR Fix Agent for #74 · sonnet · 27.1 AIC · ⌖ 3.97 AIC · ⊞ 6.3K · ◷