Repository navigation
test(monitor): pin the shared provider-error vocabulary - #14079
Conversation
…sumer classify_provider_error_text decides the ProviderErrorKind that the GitLab and Azure DevOps call sites raise, and no test in the tree references the function. That leaves the marker set asserted nowhere, so narrowing it for one provider's benefit is green for the others. Pin each marker's kind once at the classifier, pin the unrecognised-text fall-through to TRANSIENT, and add one throttling-refusal case per provider through its own call site.
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS When #13759 lands its [DESIGN-REVIEWED] bbb7f73 |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
CI disclosure: the single red lane on this head is inherited from
That is #14060, which records the same deterministic assertion on a pristine This PR is test-only and changes two files -- Board on CI checks out the pull-request merge ref, so re-running that single lane once #14060's fix is on |
|
The second commit is a merge of The lanes on the previous head resolved against the pull-request merge commit for the OLD base The reviewed content is unchanged. For the record, none of the previous head's other reds were findings:
|
What is the problem?
classify_provider_error_textinsrc/kiro_crew/monitoring/pull_request.pyis theone classifier that turns a provider CLI's stderr into a
ProviderErrorKind. Twocall sites raise
PullRequestProviderErrorcarrying its answer:src/kiro_crew/monitoring/gitlab_merge_request.py:214src/kiro_crew/monitoring/azure_devops_pull_request.py:196No test in the tree references the function. On
be067a3503,grep -rn classify_provider_error_text test/returns nothing, so every marker inits four buckets and its
TRANSIENTfall-through are asserted nowhere. Theper-provider error tests that do exist cover one kind each -- GitLab
HTTP 404and Azure DevOps
HTTP 403-- and neither provider has a rate-limit case.Two details of the issue as filed do not hold against current
main, and theynarrow what this change does:
mainare("http 429", "rate limit", "too many requests", "throttled").secondary rateand
abuse detectionare not among them, andgit log -S'secondary rate' -- src/kiro_crew/monitoring/pull_request.pyisempty. That widening lives on the open branch of feat(monitor): the gh-pr reader fetches, the wake judge decides #13759, which edits this same
function.
src/kiro_crew/probes/gh_pr.pydoes not call this classifier onmain. TheGitHub monitors classify through
src/kiro_crew/monitoring/github_provider_errors.pyinto
src/kiro_crew/taskq/adapters/github.py, whose own marker tuple carriessecondary rateandabuse detectionand is pinned intest/test_dependency_signals.py. The shared classifier therefore has twoconsumers, not three.
The coverage gap is real either way, and it is why the widening on #13759 can land
or be reverted with nothing going red.
Why this issue matters to the user
The kind is what a GitLab or Azure DevOps watch reports and retries on.
TRANSIENTis the fall-through, so a marker that stops matching does not fail loudly: the
refusal is reported as a generic transient error and retried on that basis. With no
assertion anywhere, a marker removed or re-bucketed for one provider's benefit
changes how the other providers report refusals, and the regression surfaces only
as a mis-reported error on a provider nobody was testing.
How our fix solves it
Test-only, no
src/change -- which also keeps this clear of #13759's edits to thesame function.
Pinned once at the classifier, where every consumer meets it:
TRANSIENTfall-through.The table is spelled out as literals rather than read back from the module, because
a table derived from the set under test agrees with any set.
On top, one throttling-refusal case per provider, through that provider's own call
site:
test_gitlab_throttling_refusal_is_rate_limitedandtest_azure_throttling_refusal_is_rate_limited. Both use wording that carries nostatus code and no
rate limitphrase, so the sharedthrottledmarker is theonly thing that can yield
RATE_LIMITED, and the case fails if that marker goesaway. The Azure case also has to survive that call site's missing-extension branch,
which reads the same text first.
Deliberately out of scope: the pins catch a marker removed or re-bucketed, which is
the regression named. A marker ADDED without a pin still does not go red; catching
that needs the buckets promoted to named constants in
src/, which would collidewith #13759. The retryability question the issue raises is untouched.
What tests we did
Both changed files,
-n0, isolatedKIROCREW_HOME:test/test_pull_request_readiness.py test/test_source_provider_pull_request_monitors.py-> 119 passed. 21 of those nodes are new: 18 vocabulary rows, the fall-through,
and the two provider cases.
Nine mutations of
src/kiro_crew/monitoring/pull_request.py, each caught. Everymutation ran behind a control run of the same two files reporting
119 passedunmutated, and behind a
git hash-objectcomparison proving the file actuallychanged; the file was restored from a pristine copy held outside the repository, not
with
git checkout.throttledthrottledrow + both provider casesrate limitAPI rate limit exceededrowhttp 429HTTP 429rowtoo many requestsToo Many RequestsrowTRANSIENTrevoked tokenrevoked token suppliedrowdoes not existdoes not existrowaccess deniedaccess deniedrowAUTHORIZATIONEvery one of those failures is a new test; no pre-existing test fired for any
mutation, which is the coverage gap measured directly.
Also clean:
black --check(2 files unchanged),flake8(no output),scripts/check_comment_history.pyagainst basebe067a3503(passed, after its own--testselftest passed), and prepare-prpreflight.pyreportingSTATUS: READY.Any other suggestions on the work
secondary rateandabuse detection. With these pins in place that widening arrives as a declaredchange to a covered function, and wants two more rows in the table here.
src/kiro_crew/taskq/adapters/github.py.Two marker sets now answer the same question in two places; whether they should
converge is a design question this change does not take.
private regex rather than on these kinds -- is a behaviour change needing its own
pins, and stays deferred.
Closes #14074