Skip to content

ci: delivery metrics (lead time, red time, change failure, empty bee runs) defined in specs/ci/delivery_metrics.t27 #6600

Closed

Description

@gHashTag

Item H of epic #6595: delivery metrics in the DORA style, plus why bee runs end empty.

Measured read-only on 2026-10-05 over the window 2026-09-28T18:57Z .. 2026-10-05T18:57Z (gh, the ruleset t27-master-protection, check runs on 428 first-parent master commits, /queen/public-board, /queen/status, /queen/public-activity, and every queen-* branch on the remote). Headline numbers:

  • 442 PRs merged (63.1 per day). Lead time from open to merge, median / p90: bees 6 min / 18.5 h (n=153), agents 26 min / 7.7 h (n=276), owner 374 h / 394 h (n=5), all 18 min / 15.6 h.
  • Required checks (validate, check-linked-issue, parse-ratchet): 0 h red on master. Non-required: 9 checks red more than 48 h of the 168 h window (Scorecard analysis and spec-guards for the whole window, Corpus ratchet 101.5 h, fpga-conformance 99.6 h, duplicate-bodies 84.9 h).
  • 0 reverts; 40 of 442 merged PRs (9.0%) were named by a fix PR merged within 24 h.
  • 58 open PRs, 19 (32.8%) older than 7 days.
  • 417 of 745 queen-* branches touched in the window (56.0%) end with no commit of the bee's and no diff.

Boundary

specs/ci/delivery_metrics.t27, its generated gen/c/ci/delivery_metrics.c, and one docs/now/ entry with the measured table. No tri command, no new script, nothing in BrowserOS.

User Scenarios & Testing

  • A reader asks "how fast do we merge, and how often do we break master?" and finds the definitions in one spec and today's numbers in its tests.
  • A future report or dashboard reads the definitions (lead time, red time, change failure, empty bee run) and alert thresholds from the spec instead of restating them.
  • Negative controls: a value one step past each threshold alerts; a cancelled or skipped run does not turn a check red; a fix PR 24 h + 1 min after the merge, or one that only says Closes #M, is not a change failure.

Requirements

  • Define lead time, author kind (bee / agent / owner / bot), red time per check and for the required union, change failure, and an empty bee run.
  • Alert thresholds: lead p90 > 24 h; required red > 60 min per 7 days; any check red > 48 h per 7 days; change failure > 15%; open PRs older than 7 days > 25%; empty bee branches > 20%; fewer than 1 merge per day.
  • Tests use today's measured numbers. Only t27 plus generated code.

Success Criteria

  • t27c test-report specs/ci/delivery_metrics.t27 on the t27c lab: FAIL 0, and a mutated measured constant gives FAIL 1.
  • validate, check-linked-issue, parse-ratchet, own-language and Corpus ratchet green on the PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions