Skip to content

test: the frame-layer-limit timing bound flakes on PyPy 3.10 (KTLO) #2160

Closed

Description

@JarryShaw

What fails: on main 4c3d30163, Unit Tests run 40161694146 warns that Python pypy3.10 (KTLO) had failing tests. The CI report on Discussion #2057 records the same warning. The failing test is:

SUBFAILED(shape="'IPv6 in IPv6'") tests/protocols/test_frame_layer_limit_1610_unit.py::FrameLayerLimitTests::test_no_recursion_error_at_any_depth_and_the_time_stays_bounded
AssertionError: 0.13758993199996894 not less than 0.10007906800005913 : (0.012519767000014781, 0.13758993199996894)

It asserts deep < 4 * short + 0.05. Here short and deep are the best wall-clock times for a 66-layer frame and a deep frame. On this run short was 0.0125 s and deep was 0.1376 s.

Flaky, not a regression: this is the first failure of this test on the pypy3.10 leg in the last 12 main Unit Tests runs. They run from ad71ef592 through 4c3d30163. Nothing in that range touches the frame-layer limit code. A fixed absolute slack of 0.05 s on top of a short-frame ratio is presumably too tight for PyPy's JIT warm-up and GC pauses on a shared runner. That is an inference; I have not reproduced it.

Expected: a bounded-time check that holds on every interpreter the matrix runs, while still failing on a real superlinear regression. The KTLO leg is non-blocking, but it must stay green without warnings.

Next step: a worker will measure the spread on PyPy locally and fix the bound. Measuring per-layer cost, warming up first, or scaling the slack per implementation would each work, and the measurement decides. The fix will come with a test that shows the old bound fails under the measured noise.

Activity

  1. added
    bugIssues reporting a defect (set by the bug report template; a default, not an assessment)
    testPull requests that add or correct tests (test: subject prefix)
    on Oct 25, 2026
  2. added
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 25, 2026
  3. JarryShaw commented on Oct 25, 2026

    @JarryShaw
    OwnerAuthor

    #2162 fixes this in the test only: best() collects, then disables the GC while timing. The CI failure wasn't reproduced locally. The measurements point at PyPy's full-collection steps, which grow with heap size; details are in the PR.

  4. removed
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 25, 2026
  5. added this to the PyPCAPKit v1.5 milestone on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIssues reporting a defect (set by the bug report template; a default, not an assessment)testPull requests that add or correct tests (test: subject prefix)

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions