pr-automation-budget reads the login only: it escalates waiting Dependabot PRs, and agent activity posted as the owner is invisible to it #2224

The per-PR automation budget (#926) decides who is "automated" from the login alone. That is wrong in two directions, and both showed up on the same PR.

Problem 1: the Dependabot merge pipeline is counted as an agent

The budget escalates Dependabot PRs that are doing nothing wrong, and turns their auto-merge off. A Dependabot PR that waits behind a busy main reaches the cap of 10 after three or four routine branch updates.

The counter classifies by login only. compute_pr_automation_cycles (scripts/lib/pr-automation-budget.sh:64-81) counts every commit, issue comment and review whose author ends in [bot] or is listed in AUTOMATION_BOT_LOGINS, newer than the latest non-bot event. gather_pr_automation_events (:265-292) feeds it {when, login} pairs and nothing else. It never asks whether the actor is one of the fleet's agents.

A Dependabot PR is built to merge with no human on it, so there is no human event and every bot event since the PR opened counts:

• It starts at 3: Dependabot's commit, the petry-projects-dependabot-automrg[bot] approval, and CodeRabbit's summary.
• Each time dependabot-rebase brings the branch up to date it adds 2: the App's update-branch merge commit, and the re-approval dependabot-automerge posts on the new head.

The next pr-review run then calls enforce_pr_budget (scripts/review-one-pr.sh:995). pr_automation_escalate posts the exhaustion comment, adds needs-human-review and disables auto-merge (pr-automation-budget.sh:342 → disable_auto_merge_for_hold, scripts/lib/hold-label.sh:122).

dependabot-rebase merges only PRs that have auto-merge enabled, so every later sweep logs Skipping — auto-merge not enabled. The PR sits green and unmerged until something else moves its head and dependabot-automerge turns auto-merge back on.

A branch update is caused by a push to main. It is not an agent reacting to another agent, which is the non-converging loop the breaker exists to stop (#860).

Line numbers are at 4f6e2c38. The counting logic is the same on pr-review/v1-next, pr-review/v1-stable and dev-lead/v139-stable.

Problem 2: agent comments posted as the owner re-arm the budget

dev-lead, the personas and auto-rebase post as don-petry. That login does not end in [bot] and is not in AUTOMATION_BOT_LOGINS, so every one of their comments is read as a human interaction and moves the reset point. A machine action resets the budget, which is the one thing #926 says must never happen.

On #176 it took 67 seconds. pr-review escalated at 04:34:01Z with the count at 10. dev-lead's hold notice (<!-- dev-lead-hold-notice label=needs-human-review -->, posted as don-petry) landed at 04:35:08Z and took the count to 0. pr-review run 37884264390 then went past the budget gate and ran a full review of a PR that had just been escalated.

When dev-lead itself is the escalator, the <!-- pr-automation-budget exhausted --> comment is posted as don-petry too, so the breaker's own escalation re-arms it.

The effect across the fleet: on the 54 PRs open on 2026-10-10 the highest count was 4. Measured over each PR's whole life (332 agent-authored PRs, see "Calibration" below), 4.8% reached the cap of 10 at some point, almost always through third-party review bots. The agents' own activity is nearly invisible to the breaker.

Evidence

combsmith/broodminder-data#176 (SonarSource/sonarqube-scan-action 8.2.2 → 8.3.0), all checks green throughout. The ten events the counter saw, replayed with the real library against the live PR:

Eight of the ten are the Dependabot merge pipeline itself. None is a fleet agent.

• 10-09 04:21Z: dev-lead runs start logging [suppress] PR #176 … exhausted its per-PR automation budget.
• 10-09 04:34:01Z: pr-review run 37883350479 (pr-review/v1-next at f34a765) logs PR #176 reached MAX_PR_AUTOMATION_CYCLES=10, posts the comment, adds the label and disables auto-merge.
• 10-10 10:31Z: the scheduled sweep logs Skipping — auto-merge not enabled.
• 10-10 11:21Z: a push to main makes the sweep update the branch again; dependabot-automerge re-enables auto-merge at 11:25Z; the PR merges at 16:43Z. Held for 36 hours.

The same escalation hit three earlier Dependabot PRs: #1991 (2026-10-01, cleared when the label was removed 16 minutes later), combsmith/TalkTerm#510 (2026-09-21, held 38 hours) and combsmith/TalkTerm#249 (2026-10-02, merged by hand 17 hours later). No open Dependabot PR carries the hold today.

Acceptance criteria

Dependabot pipeline:

• The Dependabot pipeline's own events do not count on a Dependabot PR. When the PR's author is dependabot[bot], events by dependabot[bot] and by the dependabot-automerge App (petry-projects-dependabot-automrg[bot]) are left out of the count. The login list is one overridable variable with those two as the default; an empty value turns the exemption off.
• They do not reset the budget either. An exempt event is still a machine event. Only a human event moves the reset point, as today.
• Both keys are required. The same App login on a PR authored by anyone else counts as it does today. Every other bot login on a Dependabot PR (donpetry-bot, github-actions[bot], repair-pr-approvals, any other *[bot]) counts as it does today, so an agent that loops on a Dependabot PR still trips the breaker at 10.
• Fail closed. If the PR author cannot be read, nothing is exempt and the count is what it is today.

Agent comments under a human login:

• A marked agent comment is an agent event: it counts, and it never resets the budget. An issue comment or review whose body carries one of our automation markers is an agent event whatever its login. The marker set is the registered list in scripts/lib/maintainer-comment-gate.sh (_MAINTAINER_GATE_AGENT_MARKERS) plus this breaker's own <!-- pr-automation-budget marker, minus maintainer-resolve (a maintainer posts that one through maintainer-resolve-comment.sh, so it is a human act). A test keeps the two lists in step.
• The cap is set from measurement. MAX_PR_AUTOMATION_CYCLES defaults to 50, with the numbers behind it recorded beside the default and in docs/agentic-interaction-model.md. See "Calibration" below.
• The runaway detector's cycle threshold follows. RUNAWAY_MAX_CYCLES defaults to 25, half the cap, so the daily report names a PR well before the breaker stops it.
• An unmarked comment by the same login still resets, so the maintainer's own comment re-engages the PR exactly as it does now.
• Bodies stay out of the event list. The marker test runs where the comments are read; gather_pr_automation_events returns {when, login} plus flags, never bodies.

Both:

• Decided in one place, pure. Each rule is a function with no network calls. gather_pr_automation_events supplies the author, so every consumer inherits the rule without its own change: enforce_pr_budget (review-one-pr.sh, dev-lead-fix-ci.sh, dev-lead-fix-reviews.sh), pr_resume_suppressed (dev-lead-resume.sh, dev-lead-retry.sh) and qa-lead-pr-gate.sh.
• No event is dropped. pr_stall_scan.sh and pr_mergeready_scan.sh read .when from every gathered event to measure idle time; flagged events stay in the list.
• bats tests in tests/test_pr_automation_budget.bats:
  • the ten fix: catch GitHub API 429s silently swallowed in review-one-pr.sh #176 events above with PR author dependabot[bot] → count 2, not exhausted;
  • the same ten events with PR author don-petry → count 10, exhausted;
  • a Dependabot PR with ten donpetry-bot events → exhausted;
  • a Dependabot PR with a human event followed by App merge commits and approvals → count 0, and the reset point stays at the human event;
  • empty or missing PR author → no exemption;
  • the login-list override is honoured, and an empty list disables the exemption;
  • the ten fix: catch GitHub API 429s silently swallowed in review-one-pr.sh #176 events followed by a marked hold notice from don-petry → count 11 (the notice counts and does not reset);
  • an agent looping under a human login, with no bot login involved, exhausts the budget at the cap and not one event earlier;
  • the default cap is 50, and the threshold tests follow the default instead of a literal;
  • the same events followed by an unmarked don-petry comment → count 0;
  • the reset point is the last unmarked human event when marked and unmarked comments are mixed;
  • every marker family is recognised, and a human's own body is not (plain text, <!-- maintainer:not-a-finding -->, <!-- maintainer-resolve, an unclosed marker, another tool's marker);
  • through gather_pr_automation_events with a stubbed gh: the fix: catch GitHub API 429s silently swallowed in review-one-pr.sh #176 history is not exhausted on a Dependabot PR and is exhausted on anyone else's; an unreadable PR author exempts nothing and is not an API failure; dev-lead's hold notice and its own escalation comment do not re-arm the budget.
• The feat: implement issue #846 — [Phase 2] Cache the LSP index and enforce the 30s cold-start SLA with auto-skip #860 guard still trips. The storms in tests/test_pr_runaway_regression.bats are scaled to the cap in force (they were written as 12 events against a cap of 10) and both scenarios still exhaust the budget and escalate once.
• Visible. When events are flagged, one log line per kind says how many and why. The library's header comment states the rule.

Calibration (maintainer decision, 2026-10-10)

Marked agent comments count, and the cap is raised so that only extreme cases are blocked.

The sample is 332 agent-authored PRs: every PR created in the org from 2026-09-10 to 2026-10-10 (249 merged, 30 closed unmerged) plus every PR still open (53). For each one the metric is the peak of the running count over the PR's life, which is the number the breaker compares with the cap.

Share of the 332 PRs that would reach the cap at least once:

• 50 sits above every PR that merged (highest 35) with room to spare. The one PR above it is feat: implement issue #1713 — [#1269 split 1/2] shadow_mode input + total PR-output suppression (inert, nothing dispatched) #1730, which peaked at 93 automated actions between two human interactions.
• feat: implement issue #846 — [Phase 2] Cache the LSP index and enforce the 30s cold-start SLA with auto-skip #860 peaks at 1,487 under the same rule. Its ack storm ran at about one comment every 9 seconds, so it reaches 50 in under ten minutes of a 4.5-hour storm.
• With the new rule and cap, none of the 54 PRs open today is at the cap (highest is 24). At the old cap of 10, 19 of them would be held.

Two limits of the measurement. PRs that were escalated at 10 under main were paused there, so their later peaks are lower than they would have been with a higher cap. And the percentiles describe a month in which pr-review and several review bots were often rate-limited.

Out of scope

• Commits and review-thread replies made under the owner login still reset the budget. A dev-lead commit carries no marker and has the same author and committer as a commit the maintainer pushes. A thread reply arrives as a review with an empty body. Telling these apart needs a marker on the commit side, or reading every review comment on every budget check. The calibration above is measured with these resets in place; closing this gap later will raise the counts and the cap must be measured again.

• Third-party advisory bots still count. coderabbitai[bot] and sonarqubecloud[bot] were 2 of the 10 on fix: catch GitHub API 429s silently swallowed in review-one-pr.sh #176, and SonarCloud posts a new comment per head. After this fix a Dependabot PR still gains about 1 per branch update, so it would need roughly eight updates to trip instead of three. Whether those bots should count on any PR is a wider decision about the budget.

• The branch-update churn itself (Merge protections are unsatisfiable at N>1 open PRs: one merge to main forces a full re-review of every other PR — ~154 forced re-reviews in 48h, zero merges #1864).

• dependabot-rebase skipping PRs whose auto-merge is off. That workflow lives in petry-projects/.github.

Related: #926, #946, #1864, combsmith/broodminder-data#176.

Filed from a Claude Code session at the maintainer's request, 2026-10-10. Problem 2 and the calibration added the same day at the maintainer's request. All times UTC.