Skip to content

fix(test): repair the two backend tests main went red on - #17907

Closed
bolichen97 wants to merge 2 commits into
mainfrom
fix/main-red-doc-ratchet-17900
Closed

bolichen97 wants to merge 2 commits into
mainfrom
fix/main-red-doc-ratchet-17900

Conversation

@bolichen97

@bolichen97 bolichen97 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Main's two red backend tests (#17900) pass again, each keeping its meaning.

Two backend tests fail on main:

  • test/test_http_server_bind_ratchet.py::test_bare_http_server_constructions_match_the_recorded_count finds 16 bare HTTPServer((host, port), ...) constructions against a recorded 15.
  • test/test_session_control_doc.py::test_every_tabulated_refusal_code_is_one_the_source_raises says the page lists remote_target_unsupported, which session_control.py no longer raises.

Why it matters

Every PR's Backend Tests (Linux and Windows) go red on these two tests, so no PR can show a clean CI run. Fixing only one leaves the PR red on the other.

Not a goal

  • Changing the ratchet's recorded count or what it scans.
  • Tidying the other 15 ratcheted sites.

What changed (motivation → approach → change)

#17450 landed after two changes it had not seen, so two tests went red on main.

Red test Cause Change
server-bind ratchet #16728 added a bare ThreadingHTTPServer stub in test/test_per_tool_mcp_deny_single_binary.py, a 16th site against a recorded 15 give that stub the server_bind override the ratchet's docstring prescribes (no host lookup), so the count stays 15
session-control doc #17662 replaced remote_target_unsupported with relay_archive_read_only, but the refusal table in the session-control help page (docs/session-control.md in the package) still lists the old code drop that row; the page already describes relay_archive_read_only

Backwards compatibility

Compatible: one test stub and one doc row change; no product code changes.

Reader: kiro_crew docs/session-control.md refusal table -- chat (agent help page) -- the removed code has not been raised since #17662, so no caller can get it; test_every_tabulated_refusal_code_is_one_the_source_raises pins the table to the source.

Tests

No new tests: the two red tests are the regression check. On main they fail (2 failed, 10 passed). On this branch test_session_control_doc.py and test_http_server_bind_ratchet.py give 12 passed, and test_per_tool_mcp_deny_single_binary.py passes with its 4 real-adapter cases skipped (they need an installed opencode binary). The flake-pattern ratchet passes.

Manual verification

N/A — unit coverage sufficient: the two tests that went red are the check.

Screenshots / video

N/A — no frontend file changed.

Related Issues

Closes #17900

Pattern harvest

Rule candidate: review-prompt
Pattern: a long-running PR that adds an exact-count guard should rebase and re-run it right before merge, since sibling merges move the count.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

#16728 added a bare ThreadingHTTPServer((host, port), ...) stub in
test_per_tool_mcp_deny_single_binary.py after #17450's ratchet
recorded 15 sites, so test_bare_http_server_constructions_match_the_
recorded_count fails on main with 16. Give that stub the server_bind
override the ratchet asks for: the count stays 15 and the stub skips
the host lookup that can stall a bind.

Part of #17900
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

One stale remote_target_unsupported mention is still there, at docs/feature-map/README.md:238. No test pins that page, so it stays green but wrong.

Inventory (2 items) — 2 justified

Intent: get main's two red backend tests (#17900) passing again, each with its meaning intact. This is a FIX.

The workflow's authentic.patch covers 80 files because it diffs against the current main tip. The branch's own change, measured from merge-base a2ca1b77f, is the 2 files below. The other 78 files are base drift, not this PR.

  1. The opencode deny test's fake model server no longer looks up the host name when it binds. It now uses the server_bind override that the ratchet docstring and testing-conventions prescribe, so the ratchet count stays at 15 — justified
  2. The session-control help page drops the remote_target_unsupported refusal row, which the source stopped raising in feat(remote-crew): delete the turn relay; old relay chats are read-only #17662 — justified

Checks behind the verdict:

  • Already exists? I grepped TCPServer\.server_bind\(self\) and got 3 hits: plumb_cpu.py:62, test_decisions_local_runtime_real_spawn.py:68, and this PR. No shared test helper exists, so the PR adds no second version of one.
  • Unfixed siblings? I grepped remote_target_unsupported across the repo and got exactly 1 remaining hit, the feature-map line above. It is outside the Goal, and deleting it would be a one-word removal.

[FIRST-PRINCIPLES-REVIEWED] 946052d

Description read

[DESCRIPTION-READ] 9bd1f7986e864fe4d3f8b3136830d9eb12ceaa68247f9bf4b87b7e7daf113b47

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ✅ PASS

UX-level review of 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Nothing to check.

[UX-REVIEWED] 946052d

Description read

[DESCRIPTION-READ] c356a238f834ac7089c9f3b4c2845375be3d6c9983e4e32847603a0cb77ca272

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Check docs/feature-map/README.md:238: it still says remote-crew reload targets are refused with remote_target_unsupported, a code the source no longer raises.

[DESIGN-REVIEWED] 946052d

Description read

[DESCRIPTION-READ] b2002cd7d5158f5860a6e8216bf9335ec1d82107f0fb794101bdcaa76ea3be8a

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 946052d

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 946052d

Verdict parsed from the review's SHA-scoped output markers for commit 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 946052dc84d8ac8a095f86e1bcf5b6e2d2c0c3c4: <one-sentence reason>

#17662 replaced remote_target_unsupported with relay_archive_read_only,
but the session-control page still tabulated the old code, so
test_every_tabulated_refusal_code_is_one_the_source_raises fails on
main. Drop that row; the page already describes relay_archive_read_only.

Closes #17900
@bolichen97 bolichen97 changed the title test: keep the server-bind ratchet at 15 for the opencode deny stub fix(test): repair the two backend tests main went red on Oct 7, 2026
@bolichen97

Copy link
Copy Markdown
Contributor Author

Goal widened at the maintainer's request: this PR now also drops the stale remote_target_unsupported row, so it fixes both #17900 tests and can go green by itself. The two halves split across two PRs left each red on the test only the other fixed.

@iamwhatever

Copy link
Copy Markdown
Contributor

Main has two reds: test/test_session_control_doc.py::test_every_tabulated_refusal_code_is_one_the_source_raises and test/test_http_server_bind_ratchet.py::test_bare_http_server_constructions_match_the_recorded_count. Each single fix is still red on the other test, so neither can go green by itself:

PR Fixes Red on
#17903 doc row bind ratchet
#17906 bind ratchet (budget 15 → 16) doc test
#17907 bind ratchet (subclass, budget stays 15) doc test
#17909 both (design-tweak _Server.server_bind + doc row) —

We are backing #17909, which carries both fixes and keeps the ratchet at 15. This PR stays open until #17909 merges.

@bolichen97

Copy link
Copy Markdown
Contributor Author

Superseded by #17909, which fixed both main-red tests and has merged; main now passes test_session_control_doc and test_http_server_bind_ratchet. Closing; the branch is kept.

@bolichen97 bolichen97 closed this Oct 8, 2026
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 8, 2026
@bolichen97
bolichen97 deleted the fix/main-red-doc-ratchet-17900 branch November 20, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

main red: session-control doc and HTTPServer ratchet tests fail after #17450

2 participants