Repository navigation
[release/17.0] Raise PR Documentation Check max-turns from 50 to 80 - #22005
Closed
Mitch Denny (mitchdenny) wants to merge 1 commit into
Closed
Mitch Denny (mitchdenny) wants to merge 1 commit into
Mitch Denny (mitchdenny) wants to merge 1 commit into
Conversation
## Failure PR Documentation Check runs were failing with `429 Maximum LLM invocations exceeded (50 / 50)` (`failureClass=invocation_cap_exceeded`). Failure triage mislabelled them as provider rate limits. **14 cap hits** between 2026-10-02 and 2026-10-20, e.g. [39318742372](https://github.com/microsoft/aspire/actions/runs/39318742372), [39362924453](https://github.com/microsoft/aspire/actions/runs/39362924453), [36976170750](https://github.com/microsoft/aspire/actions/runs/36976170750), [37537997917](https://github.com/microsoft/aspire/actions/runs/37537997917), [38540541504](https://github.com/microsoft/aspire/actions/runs/38540541504) (duplicates #20679, #20768, #20826, #20863, #20941, #20994, #21023, #21056). Capped runs were doing real work. For example, 39362924453 was cut off after 5m32s with `+120 -7` already staged. Fixes #21157 ## Change - `pr-docs-check.md`: `max-turns: 50` → `80`, with a comment pointing at #21157. - `pr-docs-check.md` prompt: "Execution budget and completion" now says `80-invocation limit` instead of `50-invocation limit`. The early-finalization target (invocation 35) is unchanged. - `pr-docs-check.lock.yml` recompiled with the pinned gh-aw **v0.89.17**. Apart from the header `body_hash`, the only changes are the two rails `max-turns` compiles to: ```diff - printf '%s\n' "{… \"maxRuns\":50 …}" > "${RUNNER_TEMP}/gh-aw/awf-config.json" + printf '%s\n' "{… \"maxRuns\":80 …}" > "${RUNNER_TEMP}/gh-aw/awf-config.json" - GH_AW_MAX_TURNS: 50 + GH_AW_MAX_TURNS: 80 ``` ## Confidence - **Root cause — high.** The harness logs say `LLM invocation cap saturated` / `Maximum LLM invocations exceeded (50 / 50)`, not a provider 429. - **Fix correctness — high.** 80 gives ~60% headroom. Agent p90 was 461s in the capped window (Oct 6–20, 13 hits / 53 runs). It has been 169s since then (0 hits / 74 runs), after `auto` model routing changed. Since the model is `auto` and the CLI is unpinned, the pressure can return. - **Regression risk — low.** Runaway protection is unchanged: the 25M `maxEffectiveTokens` proxy rail, the 20-minute job timeout, the prompt's no-retry rule for deterministic failures, and outcome validation. The worst case is more invocations on rare long runs. Those runs already burned ~156–167 AIC before failing and being retried by hand. - **Validation — compile and actionlint.** `gh aw compile pr-docs-check` (v0.89.17): `✓ … 1 succeeded`. `actionlint` on the lock file reports the same 3 pre-existing findings as `main` (`concurrency.queue`, `copilot-requests`), nothing new. `auto-sec/publication-guard.js` passes. This isn't verified in a live run yet. Urgency is low because there have been no hits since Oct 20. - **CI:** the repo's `Compile, lint, and test agentic workflows`, `Lint handwritten workflows` and `zizmor` checks pass on this PR. **How we stop this recurring:** keep the turn cap as a backstop with headroom above the observed p90, and have failure triage classify `invocation_cap_exceeded` as a turn-cap problem instead of an engine 429. <details><summary>Source diff (<code>pr-docs-check.md</code>)</summary> ```diff diff --git a/.github/workflows/pr-docs-check.md b/.github/workflows/pr-docs-check.md index f03f126..154552f 100644 --- a/.github/workflows/pr-docs-check.md +++ b/.github/workflows/pr-docs-check.md @@ -45,7 +45,9 @@ if: >- # The prompt reserves the final calls for completion, queues the notification # before terminal PR creation, and prohibits retrying deterministic failures. # Keep the cap as a backstop; incomplete output still fails outcome validation. -max-turns: 50 +# Raised from 50 to 80 after large docs PRs hit "Maximum LLM invocations +# exceeded (50 / 50)" (#21157). +max-turns: 80 checkout: # Check out aspire.dev EXACTLY ONCE, as the current workspace, because that is ``` </details> **🤖 Posted by AspireAgent (AI Agent)** --------- Co-authored-by: davidfowl <95136+davidfowl@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> (cherry picked from commit cfda6e1)
14 tasks
Mitch Denny (mitchdenny)
marked this pull request as ready for review
November 10, 2026 04:27
Member
Author
|
Approved for inclusion in release/17.0. Do not merge yet. |
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-08-pr21846
branch
from
November 10, 2026 05:29
92adb38 to
cd1e383
Compare
Contributor
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 22005Or
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 22005" |
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-08-pr21846
branch
from
November 10, 2026 07:07
cd1e383 to
92adb38
Compare
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-08-pr21846
branch
from
November 10, 2026 07:12
92adb38 to
cd1e383
Compare
Mitch Denny (mitchdenny)
force-pushed
the
mitchdenny-backport-17-0-08-pr21846
branch
from
November 10, 2026 09:17
cd1e383 to
92adb38
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Backport of #21846 to
release/17.0(cherry-picked from cfda6e1 with no modifications). Layer 8 of 55 in the release/17.0 backport stack; stacked on #22004.Backport recommendation: No — Workflows run from main.
Backport of #21846
Checklist
<remarks />and<code />elements on your triple slash comments?