Skip to content

[ADR-0007] Record the markets pilot's measured baseline and post-collapse cost in the package (§1, §7, §11) #2327

Open

Description

@don-petry

Why

The ADR-0007 markets pilot (#1729) is finalized. Its measured cost and final status are recorded in the #1729 comment #1729 (comment). The package doc docs/initiatives/agent-ingress-collapse-markets.md still has three gaps:

  • §1 does not carry the baseline run counts, even though AC5 cites §1.
  • §7 states the skip-billing ruling but has no measurement.
  • §11's gap list predates the merge.

This is a docs-only change. Copy the figures below verbatim. Do not re-measure.

Do this

1. §1 Baseline (AC5)

After the stub/pin table, add a "Measured baseline (billed runs)" subsection with:

  • Window: 2026-09-15T00:00:00Z..2026-09-22T00:00:00Z, 7 days, bounded at both ends. Each count is a per-workflow total_count from actions/workflows/<stub>.yml/runs?created=<window>.
Stub Workflow runs
dev-lead.yml 584
pr-review-mention.yml 330
pr-auto-review.yml 208
pr-review.yml 187
ci-failure-analyst.yml 33
Total 1342

2. §7 Skip-billing (AC10)

Keep the existing ruling text. Add a "Measured (post-collapse)" subsection.

Method:

  • Post-collapse window: 2026-10-10T01:16:00Z..2026-10-14T01:16:00Z, 4 days, from the markets#513 merge.
  • agent-ingress.yml had 382 runs. Each of the five deleted stubs had 0.
  • A job counts as billed when runner_name is non-empty. Billed minutes are per job, each rounded up to a whole minute.
  • Job sample: 1340 of the 1342 baseline runs (2 returned no jobs), and all 382 post-collapse runs.

Per-day figures:

Role Billed jobs/day, before → after Billed min/day, before → after
dev-lead 44.4 → 41.8 60.3 → 49.8
pr-auto-review 29.7 → 23.8 29.7 → 24.8
pr-review 4.7 → 4.5 4.7 → 4.5
pr-review-mention 0 → 0.25 0 → 0.25
ci-failure-analyst 0 → 0 0 → 0
Total 78.9 → 70.2 (−11%) 94.7 → 79.2 (−16%)

Other figures to include:

  • Workflow runs/day: 191.7 → 95.5 (−50%).
  • Jobs evaluated/day, including skipped: 358.6 → 626.0 (+75%). These are all kind (a): 0 jobs concluded skipped with a runner assigned, in either window.
  • PRs opened: 5 in the baseline window (0.7/day) and 7 in the post-collapse window (1.75/day). Post-collapse traffic was not lighter.
  • Raw totals: baseline 552 billed jobs and 663 billed minutes; post-collapse 281 billed jobs and 317 billed minutes.

Conclusion to state: the collapse is cost-neutral to slightly cheaper for markets. The per-event if: filters skip union events before a runner is assigned. The main saving is pr-auto-review: the old stub gave a runner to every run (208/208), while the ingress job is billed for 95 of 382.

3. §11 Honest gap list

Bring each row to its final state and cite the closing item:

Add one row:

Do NOT

  • Change §3 (it must stay byte-identical to markets main), any ADR, scripts, or tests.
  • Re-measure, or alter the figures.

Acceptance

  • §1, §7 and §11 carry the content above.
  • bats tests/agent_ingress_markets_package.bats tests/fleet_stub_registry.bats stays green. In particular, the §3-to-fixture byte-identity test must not be affected.
  • Only docs/initiatives/agent-ingress-collapse-markets.md changes.

Refs #1729, #2177, combsmith/markets#506


Generated by Claude Code

Activity

  1. github-actions commented on Oct 14, 2026

    @github-actions
    Contributor

    Thanks for the detailed write-up and for linking the measurements from #1729. This is a docs-only update to §1, §7 and §11 of the markets package doc, and the figures and constraints are clearly specified. One question before work starts: should the new §11 row for AC7 (ci-failure-analyst live execution) link the maintainer's acceptance comment on #1729 directly, or is a plain reference to #1729 enough?

  2. don-petry commented on Oct 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #2327 — [ADR-0007] Record the markets pilot's measured baseline and post-collapse cost in the package (§1, §7, §11)

    Scope

    Docs-only: record the measured baseline (§1), the post-collapse skip-billing measurement (§7), and the final gap-list state (§11) in docs/initiatives/agent-ingress-collapse-markets.md, copying the issue's figures verbatim.

    Implementation Checklist

    Tests to Write

    • None — documentation-only change (TDD exception). Existing suites must stay green.

    Test Command

    bats tests/agent_ingress_markets_package.bats tests/fleet_stub_registry.bats and bash .dev-lead/scripts/dev-lead-lint.sh

  3. don-petry commented on Oct 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete — PR #2328

    The implementation for issue #2327 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.

    Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupdocumentationDocumentation changes

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions