Skip to content

docs: regenerate the engine benchmark with Python 3.15 measured - #2109

Merged
JarryShaw merged 1 commit into
mainfrom
docs/engine-benchmark-315
Oct 23, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
docs/engine-benchmark-315

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • docs — documentation only changes

Description of your pull request and other information

The engine tables in docs/source/index.rst were out of date now that 3.15 is released.

  • Support table: Engine Support by Python Version still read "3.15 remains inferred" and had two footnotes, one saying no 3.15 interpreter was available and one saying tshark was missing. The 3.15 column and the pyshark row are now measured, and they agree with main's Unit Tests run 40010064464 engine jobs. 3rdparty.rst's "pyshark on 3.13 is unchecked" now says it works.
  • Speed table: this was a 2026-09-17 hand run on an M2 laptop with no 3.15 column. It is now the Docker suite's output (examples/benchmark/run.sh, 3.10–3.15, exit 0), pasted unedited. The numbers come from an x86_64 Linux host, so they are not comparable with the old figures.
  • Suite fixes:
    • The released python:3.15-slim-bookworm image (3.15.1) is pinned by digest as a default row.
    • typing-extensions is now installed for Python < 3.11, since the 3.10 image failed with ModuleNotFoundError.
    • report.py underlines headings at index.rst's level.
  • Tests: test_docs_epilog_links_1936_unit and test_docs_pcap_module_link_1948_unit get narrow exemptions for the generated snippets. Each failed 1 case before the exemption, and both pass now.

Noticed, not fixed: pcap_ct takes 0.27 ms per packet on 3.15 against about 0.117 on 3.12–3.14, consistently across three passes. Filed separately.

@JarryShaw JarryShaw added docs Pull requests that change documentation only (docs: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 23, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at e94d74ff2 (Sonnet review). One fix is needed; it is being made on this branch.

  • Required: the Engine Comparison prose in docs/source/index.rst still says the default engine "takes about 0.2 milliseconds per packet". The table a few lines below now measures 0.4879 ms, with passes ranging 0.42–0.52.
  • Passes:
    • Support table: matches main's Unit Tests run 40010064464 cell for cell on 3.10–3.15 (Scapy and PCAP_CT supported throughout, PyShark unsupported from 3.14, PyPCAP not installable and PyPCAPFile unsupported from 3.12).
    • Speed tables: unedited report.py output.
    • Test exemptions: narrow; only the generated rows are exempted.
    • Harness: test_harness.py gives 116 passed, and the typing-extensions marker matches pyproject.toml.
  • UNVERIFIED: the image digest against the registry, and the pcapkit and dpkt rows, which have no Engines job.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 23, 2026
- python-images.txt: pin the released python:3.15-slim-bookworm (3.15.1)
  in place of the 3.15 rc, and make it a default-tier row.
- requirements-common.txt: pin typing-extensions for 3.10; since
  pcapkit's compat imports it unconditionally there, the 3.10 pypcap
  virtualenv failed to import pcapkit and the 3.10 image no longer built.
- report.py: underline the snippet headings with `-`, the level they take
  in docs/source/index.rst, so the snippets paste verbatim.
- index.rst: replace the hand-measured macOS Test Environment/Test Results
  with a full examples/benchmark/run.sh run (3.10-3.15, cfabfb2), and
  drop the "3.15 inferred" and "tshark absent" footnotes from the support
  table now that 3.15 and pyshark are measured.
- 3rdparty.rst: pyshark on 3.13 is checked, not "expected to work".
- tests: exempt the generated snippets' engine/pcap literals from the
  rst_epilog link checks (#1936, #1948).

Benchmark self-tests and the docs-pin tests pass.
@JarryShaw
JarryShaw force-pushed the docs/engine-benchmark-315 branch from e94d74f to 970796e Compare October 23, 2026 20:30
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at 970796ee6. The only change since the review's e94d74ff2 is the two-line fix it asked for (git diff e94d74ff2 970796ee6: docs/source/index.rst, 2+/2−).

  • Old: "takes about 0.2 milliseconds per packet, slower than most of the third-party engines below."
  • New: "is slower than every third-party engine below except PyShark, as Test Results measures."
  • Checked against the table: pcapkit runs 0.43–0.52 ms per packet. That is behind pypcapfile, dpkt, pypcap, pcap_ct (0.27 even on 3.15) and scapy, and ahead of pyshark only (~31 ms).
  • Tests: the docs-link tests (1936, 1948) and test_conventions_doc_claims pass. Every other point of the review stands.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 23, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 91.54% (unit tier, Python 3.15, 970796ee6, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 19599 880 2618 866 92.12%
pcapkit/corekit 2565 71 812 37 96.56%
pcapkit/dumpkit 282 3 102 3 98.44%
pcapkit/foundation 2930 23 1022 25 98.79%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 17414 219 4574 185 98.14%
pcapkit/toolkit 869 36 298 10 95.20%
pcapkit/utilities 473 7 136 5 98.03%
pcapkit/vendor 4432 1988 1018 167 51.87%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Ready to merge at 970796ee6.

  • Review: GOOD TO GO, after the stale "0.2 ms" sentence was fixed.
  • CI: every blocking check passed (Required checks passed SUCCESS). The 3.8–3.10 KTLO legs carry no warnings.
  • Ordering: the tests/project plain-unittest ordering leg passed on a scratch merge into main: 2180 tests, 0 failures.

If #2112 merges first, the 3.15 pcap_ct cell (0.2702) goes stale. I'll re-measure it in a follow-up.

@JarryShaw
JarryShaw merged commit eef5aea into main Oct 23, 2026
62 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 23, 2026
@JarryShaw
JarryShaw deleted the docs/engine-benchmark-315 branch October 23, 2026 21:57
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 23, 2026
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Pull requests that change documentation only (docs: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant