Skip to content

test(protocols): keep the frame-layer-limit timing bound stable on PyPy (#2160) - #2162

Merged
JarryShaw merged 1 commit into
mainfrom
fix/2160-frame-layer-timing-bound
Oct 25, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/2160-frame-layer-timing-bound

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • test — tests only

Description

Closes #2160.

  • Cause (measured, not reproduced): the old bound never failed locally, in 30 fresh PyPy 3.10 runs, under 5M–40M-object heaps, or in a long tests/protocols process. PyPy's GC hooks show each deep parse triggers 2–3 full-collection steps where a short parse triggers none. That cost grows with heap size: deep IPv6 took 8.1, 13.6 and 25.0 ms at 0, 10M and 40M live objects, and 9.4–10.0 ms at every size with the collector off. A CI worker's heap is the likely source of the 125 ms in run 40161694146. Runner CPU steal is the other candidate.
  • Fix, test only: best() runs gc.collect() and disables the collector while timing, then restores it, as timeit does. The bound deep < 4 * short + 0.05 is unchanged.
  • Results with the fix: PyPy's worst ratio was 1.74 fresh and 1.23 with a 40M heap. The timing test passed 30/30 on PyPy and 30/30 on CPython.
  • Regression guard: new fake-clock TimingBoundTests check that the collector's cost stays out of the timing, that the GC state is restored even when the parse raises, and that a linear or quadratic per-layer cost still breaks the bound. Without the fix the first test fails (0.1375 != 0.0125).
  • Tests: the whole file passes on CPython 3.14 and PyPy 3.10 (16 tests, 53 subtests each). tests/test_tier_guard.py: 133 passed.

Not in scope: the bound is loose. A quadratic walk past the limit is caught for IPv6 but not for IPv4 or 802.1Q, where the 0.05 s slack dominates. Tightening it needs measurements across the CI matrix, so it is tracked separately.

…Py (#2160)

On PyPy each parse of a deep frame sets off major-collection steps that a
short frame's parse does not, and their cost grows with the worker's heap,
so the best of three deep parses can still carry it. Time both frames after
a gc.collect() with the collector disabled, as timeit does; the bound itself
is unchanged.

best() and bounded() move to module level, and TimingBoundTests drives them
with a made-up clock: the collector's share is kept out, the collector state
is restored, and a linear or quadratic parse still breaches the bound.
@JarryShaw JarryShaw added bug Issues reporting a defect (set by the bug report template; a default, not an assessment) test Pull requests that add or correct tests (test: subject prefix) labels Oct 25, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 25, 2026
@JarryShaw JarryShaw added the review: running A cross-review is in flight against the current head - no verdict yet label Oct 25, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at 185daec00 (Sonnet review).

  • GC state restored: best() captures gc.isenabled() before collecting and re-enables in finally only if it was enabled. This holds on a normal return, on an exception from parse, and when the GC was already off on entry. test_the_collector_is_left_as_it_was covers both entry states. A mutant that always re-enables fails it, and the addCleanup stops any leak into later tests.
  • The guard fails without the fix: removing gc.disable() gives 0.1375 != 0.0125. The guard drives the real module-level best() and bounded(), with only the clock injected.
  • CPython is unchanged: bounded() is deep < 4 * short + 0.05, the same as the old inline check, and the parse is still inside the timed span.
  • Tests: CPython 3.14 ran the file 3 times, 16 passed each time; tests/test_tier_guard.py 133 passed. The file is tier ALL.
  • UNVERIFIED: the PyPy 3.10 measurements, because no PyPy was available to the reviewer. The PR body says the cause was measured but not reproduced.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 25, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 91.55% (unit tier, Python 3.15, 185daec00, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 19599 880 2618 866 92.12%
pcapkit/corekit 2589 75 816 37 96.48%
pcapkit/dumpkit 282 3 102 3 98.44%
pcapkit/foundation 2971 23 1036 25 98.80%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 17418 219 4574 185 98.14%
pcapkit/toolkit 869 36 298 10 95.20%
pcapkit/utilities 504 7 146 5 98.15%
pcapkit/vendor 4432 1988 1018 167 51.87%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Ready to merge at 185daec00. Merging closes #2160.

  • Review: Sonnet GOOD TO GO. The GC state is restored on every path, and the guard fails without the fix.
  • CI: all 62 CheckRuns passed, none failed or pending. Required checks passed is SUCCESS, and the KTLO legs show no warnings, including the pypy3.10 leg that raised test: the frame-layer-limit timing bound flakes on PyPy 3.10 (KTLO) #2160.
  • Ordering: I merged this PR into main and ran tests/protocols plus the root modules under the plain unittest ordering leg: 1903 tests, 0 failures, 0 errors.
  • Conflicts: none. Only the one test file changed.

@JarryShaw
JarryShaw merged commit 0ccb582 into main Oct 25, 2026
66 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 25, 2026
@JarryShaw
JarryShaw deleted the fix/2160-frame-layer-timing-bound branch October 25, 2026 14:20
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 25, 2026
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Issues reporting a defect (set by the bug report template; a default, not an assessment) test Pull requests that add or correct tests (test: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

test: the frame-layer-limit timing bound flakes on PyPy 3.10 (KTLO)

1 participant