Repository navigation
test: pin the lapsed hop port hold on its socket, not the port number - #33884
Conversation
The lease-lapse test proved the release by binding the freed port once from another process. The freed port is an ordinary ephemeral port, so on a busy runner another process can be handed it before the probe runs, and the probe reads EADDRINUSE although our socket is closed. Assert on the socket the hold owned instead, as the release test does: capture it while held, lapse the lease through an injected clock, and require it closed once the reaper drops the hold. The external probe stays as a containment check through the bounded wait the release test uses, whose default becomes FALLBACK_TIMEOUT_SECS; the lease-stands test's one-shot probe on a just-freed port uses the same wait. Closes #32220
|
[operator: bolichen97] Determinism evidence for this head ( Forced condition, a throwaway plugin loaded with """Throwaway -p plugin, never committed: forces the race issue #32220 names.
Right after the port under test is freed, a short-lived competitor in this process
binds it (plain bind, no SO_REUSEADDR) for HOLD_SECS, the way another process handed
the freed ephemeral port would. A one-shot probe then reads REFUSED 98; a bounded
wait reads BOUND once the competitor goes.
"""
import socket
import threading
import pytest
HOLD_SECS = 1.0
FORCED = []
def _compete(port):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
s.bind(("127.0.0.1", port))
except OSError:
s.close()
return
FORCED.append(port)
threading.Timer(HOLD_SECS, s.close).start()
@pytest.fixture(autouse=True)
def _forced_competitor(request, monkeypatch):
from kiro_crew.instances import hop_port_guard as m
name = request.node.name
if name == "test_the_port_is_free_again_once_the_lease_lapses":
orig = m.HopPortGuard.release
def release(self, port):
with self._lock:
held = port in self._held
orig(self, port)
if held:
_compete(port)
monkeypatch.setattr(m.HopPortGuard, "release", release)
elif name == "test_a_second_process_is_refused_while_the_lease_stands":
mod = request.module
orig_free = mod._free_port
calls = []
def free_port():
port = orig_free()
calls.append(port)
if len(calls) == 1:
_compete(port)
return port
monkeypatch.setattr(mod, "_free_port", free_port)
yield
def pytest_terminal_summary(terminalreporter):
terminalreporter.write_line(f"forced_competitor: competitor bound {len(FORCED)} port(s)")On the parent On this head: Negative control ( """Throwaway -p plugin, never committed: negative control for the socket pin.
release() drops the hold from the held set but closes its socket 2 s later, off the
lock. The lapse test must fail by name at its socket assertion, not pass.
"""
import threading
import pytest
@pytest.fixture(autouse=True)
def _late_close(request, monkeypatch):
from kiro_crew.instances import hop_port_guard as m
if request.node.name == "test_the_port_is_free_again_once_the_lease_lapses":
def release(self, port):
with self._lock:
entry = self._held.pop(port, None)
if entry is None:
return
try:
self._selector.unregister(entry[0])
except (KeyError, ValueError):
pass
threading.Timer(2.0, entry[0].close).start()
monkeypatch.setattr(m.HopPortGuard, "release", release)
yield |
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Test-only change. It checks the guard's own socket instead of the port number, which removes the race with other processes that caused the flake. No design-level concerns. [DESIGN-REVIEWED] 0f2fb38 Description read[DESCRIPTION-READ] 8c9899a1a88f1ce04cc6a3ec5a36508f449c2e1c914090ed78061924bd814967 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
buluoray
left a comment
There was a problem hiding this comment.
Approved at 0f2fb38. Test-only deflake for #32220: the lapse test now checks that the reaper closed the hold's own socket (test/test_hop_port_guard.py:180-200), and both one-shot BOUND probes use the bounded wait helper (test/test_hop_port_guard.py:150-151, :57-58). src/kiro_crew/instances/hop_port_guard.py:338-344 is unchanged behaviour. Two mutations each turn the named test red.
Problem / Motivation
Goal:
test_the_port_is_free_again_once_the_lease_lapsesstops failing unrelated PRs when another process briefly takes the port the hop port guard just gave back.The test failed twice on Linux CI, on PRs that do not touch the hop port guard, always at the same line:
Why it matters
A red backend shard blocks PR Readiness for whoever hits it and costs a rerun each time.
Not a goal
What changed (motivation → approach → change)
The test waited until the guard held no port, then asked a separate process to bind that port once. The freed port is an ordinary ephemeral port, so on a busy runner another process can be handed it in the gap before the probe runs. That process holds it for a moment, and the single probe reads
EADDRINUSE(errno 98) although our socket is closed.The test asserts on the guard's own socket instead of the port number, the way
test_release_frees_it_for_a_legitimate_reconnectalready does. It captures the hold's socket while the lease stands, lapses the lease through themanual_clockfixture, and requires that socket closed once the reaper drops the hold. The bind from another process stays as a containment check through_wait_for_bind_from_another_process, whose default bound isFALLBACK_TIMEOUT_SECS, so a brief competitor is waited out. The same one-shot probe on a just-freed port intest_a_second_process_is_refused_while_the_lease_stands(its "port was free to start" check) goes through the same wait.Backwards compatibility
Compatible: test-only change.
Removes nothing: the lapse test checks the same release on the socket it owned, and the refusal assertions are unchanged.
Tests
test/test_hop_port_guard.py:test_the_port_is_free_again_once_the_lease_lapsestest_a_second_process_is_refused_while_the_lease_standstest_release_frees_it_for_a_legitimate_reconnectFALLBACK_TIMEOUT_SECSboundProof: not provable -- test-only change
Determinism proof (testing-conventions D12):
-pplugin (not committed) lets a second socket bind the just-freed port for 1 s, right after the guard releases it and right after_free_port()returns in the lease-stands test. On the parent2caa69b7a6both tests go red with the CI error (assert 'REFUSED 98' == 'BOUND', andport was not free to start); on this head both pass.release()drops the hold but closes its socket 2 s later makes the lapse test fail by name atthe lapsed hold was dropped with its socket still open.-n0. The whole file passes 16 of 17-n0runs; the 17th timed out (over 120 s, host load about 200) in the teardown oftest_a_descriptor_limit_is_owed_rather_than_raised, which this diff does not touch. 10/10 at-n 4 --dist loadgroup --max-worker-restart=2besidetest/test_instances_port_allocator.pyandtest/test_instances_forwarder_hints.py.Manual verification
N/A — test-only change; CI runs the shard.
Related Issues
Closes #32220
Sibling sweep
Swept: git grep -n '== "BOUND"' -- test/test_hop_port_guard.py
Sibling: test/test_hop_port_guard.py:151 -- changed here
Sibling: test/test_hop_port_guard.py:225 -- changed here (its containment wait takes the helper's default bound)
Sibling: test/test_hop_port_guard.py:386 -- not affected -- the port stays bound by the test's own socket for the whole check, so it is never a freed port
Sibling: test/test_hop_port_guard.py:156 -- not affected -- this and the other REFUSED probes target a port the guard holds, which no competitor can take
Pattern harvest
Rule candidate: review-prompt
Pattern: a test proves a socket was released by binding its freed port number once, racing every other process the kernel may hand that ephemeral port to
Checklist
Contribution License Agreement
Contributions are accepted under the project's Apache-2.0 license.