Skip to content

[deep-report] engine-example-counter undercounts nested 'engine: {id: ...}' form, making engine-parity trend data unreliable #55132

Closed

Description

@github-actions

Description

claude-code-user-docs-review's engine-example-counter sub-agent (.github/workflows/claude-code-user-docs-review.md:168-180) counts engine usage across example workflows by scanning .github/workflows/*.md for literal occurrences of engine: claude, engine: copilot, engine: codex, engine: custom. This misses the nested-mapping form of the engine: field (engine:\n id: copilot\n ...), which is common in the repo — e.g. .github/workflows/delight.md declares engine:\n id: copilot\n copilot-sdk: true.

Verified live counts in this repo:

  • Literal engine: claude / engine: copilot (single-line form): 37 / 34 files
  • Nested engine:\n id: claude / engine:\n id: copilot form: 24 / 101 additional files

The literal-string counter therefore undercounts Copilot examples by roughly 3x and Claude examples by ~65%, missing the majority of nested-form declarations entirely. This directly explains the "Trend note" flagged by the workflow's own most recent report (discussion #55075): "engine-example counts shifted sharply since 2026-08-22, claude 39→60, copilot 39→138 — worth confirming whether this reflects real example growth or a counting-methodology change." It's the latter — the count is unreliable and swings based on which YAML form authors happen to use, not real example growth/decline.

Proposed Fix

Update the engine-example-counter agent's counting logic to also match the nested engine:\n id: <name> form (e.g. via a YAML-aware parse or a regex covering both forms), so engine-example counts and the Copilot-vs-Claude parity trend this workflow reports on are accurate and stable across runs.

Expected Impact

Fixes a data-quality bug in a recurring report — the "Engine & Tool Matrix" and parity trend note are currently misleading rather than informative.

Suggested Agent

general-purpose — fix the counting logic in the engine-example-counter sub-agent definition.

Estimated Effort

Quick (< 1 hour)

Data Source

DeepReport Intelligence analysis, discussion #55075 ([claude-code-user-docs-review] Claude Code User Documentation Review — 2026-08-23) "Trend note". Verified live via grep counts against .github/workflows/*.md.

Generated by 🔬 Deep Report · agent · 212.6 AIC · ⌖ 15.7 AIC · ⊞ 12.4K · ◷

  • expires on Aug 25, 2026, 10:35 AM UTC-08:00

Activity

  1. github-actions commented on Aug 23, 2026

    @github-actions
    ContributorAuthor

    🍪 Issue Monster selected this for Copilot

    I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

    If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

    Om nom nom! 🍪

    🍪 Om nom nom by Issue Monster · cop10 · 3.26 AIC · ⌖ 2.08 AIC · ⊞ 17.7K · ◷

  2. self-assigned this
    on Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions