Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: Engine Benchmark

# #2125: times every extraction engine on every interpreter the engines are
# tested on, so the docs' Engine Support speed table no longer goes stale
# between hand runs (#2109). Natively, through examples/benchmark/native.sh
# rather than run.sh: a hosted runner is already a fresh VM per job, so
# containers would add a build per version and no isolation, and a native run
# makes macOS and Windows runners candidates too. One job measures every
# tested on, so the speed tables of docs/source/engines.rst have a weekly
# measurement to be copied from rather than waiting on a hand run (#2109).
# Natively, through examples/benchmark/native.sh rather than run.sh: a hosted
# runner is already a fresh VM per job, so containers would add a build per
# version and no isolation, and a native run makes macOS and Windows runners
# candidates too. One job measures every
# version, since the per-version table's absolute times compare only within
# one machine: by default `ubuntu-26.04-arm`, the fastest runner that
# measures every environment and engine. What a native run does not pin --
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/ci-timing.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
report:
name: CI timing report
runs-on: ubuntu-latest
# Three nested limits, so a slow week ends in a smaller report rather than
# Nested limits, so a slow week ends in a smaller report rather than
# none (#1930): the script stops reading jobs at --budget-minutes and writes
# what it has; the step's cap sits far enough above it for an in-flight
# request to finish its retries; this one leaves room for setup and uploads.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ jobs:
# posts nothing: its findings belong to the pull request, and the alerts the
# report lists are the default branch's. The only job holding
# `security-events: read` or `discussions: write`, and it runs no project
# code but the two util/ scripts.
# code but util/codeql_report.py and util/discussion_report.py.
discussion:
name: Post the CodeQL report
needs: [ analyze ]
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/create-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -816,8 +816,8 @@ jobs:
python -m pip download -r conda/requirements.txt -d conda/wheels

# With `activate-environment: base`, `python-version` installs that Python
# into base, next to conda itself. conda-forge's conda has no py315 build
# yet, so on 3.15 that fails with "RemoveError: Conda cannot remove
# into base, next to conda itself. conda-forge's conda has no py315
# build, so on 3.15 that fails with "RemoveError: Conda cannot remove
# itself" (#1633, measured with conda 26.7.2). So 3.15 leaves base on its
# own Python: `conda build --python` below builds against 3.15 regardless,
# measured to build and import pypcapkit-*-py315_0. Drop the exception
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cron-conda.yml
Original file line number Diff line number Diff line change
Expand Up @@ -551,7 +551,7 @@ jobs:
- macos-26-intel
- ubuntu-latest
# #2060: win-64, uploaded with the anaconda-client CLI as the Intel
# leg is, now that its runs have built and smoke-tested it green.
# leg is, once its runs had built and smoke-tested it green.
- windows-latest
# #1633: the supported tier, from .github/python-versions.toml through
# `conda-update` (#1660); create-release.yml's ``conda`` job says why
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/experimental.yml
Original file line number Diff line number Diff line change
Expand Up @@ -608,9 +608,9 @@ jobs:
if-no-files-found: ignore
retention-days: 7

# #2060: the unit tier on Windows before the win-64 conda package is
# published, so its first release has had a test run. One leg, on `latest`,
# the newest CPython the conda builds target. Non-blocking by `experimental`'s
# #2060: the unit tier on Windows, the platform of the win-64 conda package,
# added so that its first release had a test run. One leg, on `latest`, the
# newest CPython the conda builds target. Non-blocking by `experimental`'s
# mechanics: every step that can fail is `continue-on-error`, the long ones
# have caps under the job's, `Experimental summary` reports each failure as a
# warning, and the job-level guard covers a lost runner. None of
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -185,9 +185,9 @@ jobs:
# YAML anchors need actionlint 1.7.10 or later.
#
# The `-ignore` hides a false positive on `concurrency.queue`, a documented
# key (project-status.yml uses it) no actionlint release knows yet
# (rhysd/actionlint#657, fixed by rhysd/actionlint#654). Drop it once a
# release with the fix is pinned.
# key (project-status.yml uses it) that the pinned actionlint does not
# know (rhysd/actionlint#657, fixed by rhysd/actionlint#654). Drop it once
# a release with the fix is pinned.
- name: actionlint
id: actionlint
if: ${{ !cancelled() }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -283,8 +283,8 @@ jobs:
path: ${{ runner.temp }}/junit*.xml
retention-days: 14

# #1672: the last step of every KTLO matrix job but the job summary's
# two, util/leg_summary.py (#1967). Each step above that can fail on a
# #1672: the last step of every KTLO matrix job before the job-summary
# steps, util/leg_summary.py (#1967). Each step above that can fail on a
# KTLO leg has an `id` and `continue-on-error: ${{ matrix.ktlo == true }}`,
# so its failure leaves the job at success; this step makes it visible
# again, one warning and one job-summary line per failed step, from each
Expand Down
14 changes: 7 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,17 +158,17 @@ The job drives them through the `Makefile` rather than restating their flags (`v
variables, so a check cannot come out clean locally and red in CI because the two drifted.

**Those steps are advisory, not a gate.** Each carries `continue-on-error: true`, so a finding lands
as a non-blocking annotation and a red linter will not fail your pull request; none of them is
clean. The workflow's header records what each tool would need before
its `continue-on-error` line could be deleted. Read the job's run summary, where each step writes
as a non-blocking annotation and a red linter will not fail your pull request. The workflow's
header records which of them are clean and what each of the others would need before its
`continue-on-error` line could be deleted. Read the job's run summary, where each step writes
its verdict, and try not to add to the numbers.

`isort` is the exception and is local-only. It appears in `cron-vendor.yml` as a formatter that
rewrites the regenerated constants, not as a check, so nothing verifies import ordering on a pull
request.
`isort` is the exception: no `Lint` step runs it. It appears in `cron-vendor.yml` as a formatter
that rewrites the regenerated constants, and `tests/project/test_isort_clean.py`, which the `test`
extra's `isort` lets run, fails on a file whose imports `make isort` would reorder.

One trap: `make vermin` redirects its report into `temp/vermin.txt` rather than to your terminal,
and vermin exits 1 because `vermin.ini` sets `targets = 3.8` against the 3.11 minimum it detects.
so when vermin finds code that misses `vermin.ini`'s `targets = 3.8` and exits 1,
Make gives up at the redirect with `make: *** [vermin] Error 1`, the report left in the file and the line
that would have opened it never reached. `make vermin-ci` runs the same flags to stdout, which is why
CI uses it and why it is easier to read at a desk. Running the lot before you push still saves a
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ coverage: samples
pipenv run coverage run -m pytest -q
pipenv run coverage report

# The engine speed table in docs/source/index.rst -- every Python version
# The engine speed tables in docs/source/engines.rst -- every Python version
# examples/benchmark/python-images.txt marks `default`, one image each,
# measured in containers so the host does not affect the
# result. Needs docker, and nothing else: deliberately not run through pipenv,
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,5 +96,5 @@ reference. Pages worth knowing by name:
| [How to ...](https://jarryshaw.github.io/PyPCAPKit/demo.html) | Worked examples, library and CLI |
| [Extensions](https://jarryshaw.github.io/PyPCAPKit/ext.html) | Registering your own protocols, engines and dumpers |

Release history is in [CHANGELOG.md](CHANGELOG.md), and contribution guidelines
are in [CONTRIBUTING.md](CONTRIBUTING.md).
The newest changelog entry is in [CHANGELOG.md](CHANGELOG.md), which links to the
full history, and contribution guidelines are in [CONTRIBUTING.md](CONTRIBUTING.md).
4 changes: 2 additions & 2 deletions examples/benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,8 +266,8 @@ numbers, and it should not need overriding:
instead, which is closer to what a user gets.)
- `dpkt`, `scapy`, `pyshark` and `pypcapfile` are pure Python. `lxml`,
`pyshark`'s only compiled dependency, publishes manylinux wheels for aarch64 on
every interpreter in the matrix (cp310 through cp314), so no column needs a compiler
for it.
every interpreter this was checked on (cp310 through cp314), so none of those
columns needs a compiler for it.
- `pypcap` is the only thing that compiles, and nothing in it is
architecture-specific: its pre-generated `pcap.c` contains no arch guards or
inline assembly, and its `setup.py` branches on `sys.maxsize > 2**32` rather
Expand Down
7 changes: 4 additions & 3 deletions examples/generators/dispatch.py
Original file line number Diff line number Diff line change
Expand Up @@ -298,9 +298,10 @@ def _link_payload(code: 'int') -> 'bytes':
from pcapkit.protocols.application.rarp import RARP

# ARP/RARP/InARP/DRARP all report their alias from the wire ``oper``
# field rather than from the dispatching EtherType (link/arp.py:191-202
# per #496), so ``oper`` has to land in the RARP range (3, 4) or this
# would silently probe ARP instead while still "passing".
# field rather than from the dispatching EtherType (``ARP.read`` in
# pcapkit/protocols/link/arp.py, per #496), so ``oper`` has to land in
# the RARP range (3, 4) or this would silently probe ARP instead while
# still "passing".
return bytes(RARP(oper=3))
if code == EtherType.Customer_VLAN_Tag_Type:
from pcapkit.protocols.link.c_tag import C_Tag
Expand Down
4 changes: 2 additions & 2 deletions examples/generators/options.py
Original file line number Diff line number Diff line change
Expand Up @@ -1288,8 +1288,8 @@ def _pcapng_context() -> 'tuple[Any, Any]':
type=Enum_BlockType.Section_Header_Block, block={})
# ``Protocol.info`` is annotated as the generic data model for the protocol,
# so a checker cannot see that a Section Header Block's is the one ``Context``
# wants. The engine does exactly this, at
# pcapkit/foundation/engines/pcapng.py:181.
# wants. The engine does exactly this, in ``PCAPNG.run`` of
# pcapkit/foundation/engines/pcapng.py.
section_info = section.info # type: Any
context = Context(section_info)
interface = PCAPNG(num=1, sct=1, ctx=context,
Expand Down
9 changes: 5 additions & 4 deletions util/_actions.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,11 @@
repository root, it imports ``util._actions``.

The scripts run on the image's ``/usr/bin/python3``, or on a leg's own
interpreter, which may be 3.7, GraalPy or a free-threaded build, and
:file:`util/pip_install.py` is the step that installs anything at all. So this
module is standard library only and keeps to 3.7's syntax; a project test,
:file:`tests/project/test_actions_helper_2047.py`, holds it to both.
interpreter, which may be an end-of-life KTLO one, GraalPy or a free-threaded
build, and :file:`util/pip_install.py` is the step that installs anything at
all. So this module is standard library only and keeps to 3.7's syntax; a
project test, :file:`tests/project/test_actions_helper_2047.py`, holds it to
both.

The escaping is GitHub's, from the "Workflow commands for GitHub Actions"
reference (https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-commands)
Expand Down
20 changes: 11 additions & 9 deletions util/bump_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@
back out of the tree, tags ``v<version>``, and publishes to PyPI and Anaconda.

So in this repository a bump is not a preparatory step that a release later
follows -- the bump *is* what causes the release, minutes later and with no human
in between. That fact decides what this script is responsible for updating, and
it is the reason ``date-released`` below is in scope rather than left to a
release step that does not exist.
follows -- the bump *is* what causes the release, with nothing in between but the
``github-release`` environment's one approval (#887). That fact decides what this
script is responsible for updating, and it is the reason ``date-released`` below
is in scope rather than left to a release step that does not exist.

``CITATION.cff``
----------------
Expand All @@ -40,9 +40,8 @@

Measured over the thirty most recent releases, the bump commit's UTC calendar date
equals the PyPI upload's UTC calendar date **thirty times out of thirty**, median
gap three minutes, and no bump in the project's history has taken more than a day
to publish. The weekly cron fires at 10:05 UTC and publishes at 10:07, twelve
hours clear of a date boundary. Hence UTC via :func:`datetime.datetime.now` with an
gap three minutes. The weekly cron fires at 10:05 UTC, twelve hours clear of a
date boundary. Hence UTC via :func:`datetime.datetime.now` with an
explicit timezone, not :meth:`datetime.date.today`: seven of those thirty bumps
were made late evening in US-Eastern, where a naive local date would have been a
day behind the publish it was describing.
Expand All @@ -54,8 +53,11 @@
leaving the field stale, since a stale date is wrong in *every* one of the other
eleven cases; it is an argument for eventually anchoring both fields to
:file:`.github/workflows/create-release.yml`, which runs only when a publish is
actually going out. Until then this is the better of the two available answers, and
the ``version`` half is guarded by a test that fails when the file drifts from
actually going out. That would also give the right date to a release approved on
a later day than its bump, which this script dates by the bump: ``1.5.0b9``'s
Create Release run was approved three days after it was created (#1634). Until
then this is the better of the two available answers, and the ``version`` half
is guarded by a test that fails when the file drifts from
``pcapkit.__version__`` -- which also covers the version changes made by hand,
which never run this script at all. Those are about a quarter of the commits
that have moved ``__version__`` on ``main`` over the project's life, and a larger
Expand Down
7 changes: 4 additions & 3 deletions util/discussion_report.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# -*- coding: utf-8 -*-
"""Post a run's report to its job's thread in a Discussions category (#2030).

The ``discussion`` job of :file:`.github/workflows/ci-timing.yml` and of
:file:`.github/workflows/cron-vendor.yml` runs, from the checkout's root and on
the runner's ``python3``::
The ``Post the report to Discussions`` step of each workflow's ``discussion``
job, :file:`.github/workflows/ci-timing.yml`'s and
:file:`.github/workflows/cron-vendor.yml`'s among them, runs, from the
checkout's root and on the runner's ``python3``::

python3 util/discussion_report.py --category NAME --title TITLE --report FILE \\
--mode scheduled|push --status STATUS
Expand Down
5 changes: 3 additions & 2 deletions util/leg_summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
``continue-on-error`` that holds on that leg, so its failure leaves the job, and
``required-checks``, at success. This script makes the failure visible again. The
``KTLO summary`` steps of :file:`.github/workflows/unit-tests.yml`'s ``test``,
``integration`` and ``pypcap-parity`` jobs and :file:`.github/workflows/python-compatibility.yml`'s
``compatibility`` job, and the ``Experimental summary`` step of
``integration`` and ``pypcap-parity`` jobs, :file:`.github/workflows/python-compatibility.yml`'s
``compatibility`` job and :file:`.github/workflows/create-release.yml`'s ``pypi`` job, and the
``Experimental summary`` step of
:file:`.github/workflows/experimental.yml`'s ``experimental`` job, each run::

/usr/bin/python3 util/leg_summary.py ktlo # or: experimental
Expand Down
4 changes: 2 additions & 2 deletions util/python_versions.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
``python3 util/python_versions.py --github-output``, which appends one step
output per name below to ``$GITHUB_OUTPUT``. So it is standard library only:
:mod:`tomllib` needs 3.11, and each image it runs on has a newer ``python3`` --
3.12.3 on ``ubuntu-latest`` (24.04), 3.14.4 on 26.04, which that label moves to
from 2026-10-19, and 3.14.7 on ``macos-latest`` (macOS 26 arm64), per
3.12.3 on Ubuntu 24.04, 3.14.4 on 26.04, which ``ubuntu-latest`` names from
2026-10-19, and 3.14.7 on ``macos-latest`` (macOS 26 arm64), per
actions/runner-images' READMEs at 2c421200c.

``latest``
Expand Down
2 changes: 1 addition & 1 deletion util/release_conda.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"""The conda legs' steps, in both conda jobs (#888, #1822, #2044).

``conda`` in :file:`.github/workflows/create-release.yml` and ``conda-dist`` in
:file:`.github/workflows/cron-conda.yml` run it in three forms, each with
:file:`.github/workflows/cron-conda.yml` run it in the forms below, each with
``PCAPKIT_CONDA_OS``, the leg's ``matrix.os``, and ``PCAPKIT_PYTHON_VERSION``, its
``matrix.python-version``, where it reads them:

Expand Down
2 changes: 1 addition & 1 deletion util/release_published.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"""Whether a version's files are all on PyPI, and all on Anaconda (#888, #1633, #1922).

:file:`.github/workflows/create-release.yml`'s ``version_check`` job runs it,
after setup-python, in two forms:
after setup-python, in these forms:

``python3 util/release_published.py pypi /tmp/pypi_release.json``
The step "Check PyPI publish completeness". It expects a wheel for each
Expand Down
2 changes: 1 addition & 1 deletion util/release_tags.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"""Which commit a release tag names, against the commit this run was created for (#1634, #1748).

:file:`.github/workflows/create-release.yml` runs it from the checkout's root,
on the runner's own ``python3``, in two forms:
on the runner's own ``python3``, in these forms:

``python3 util/release_tags.py check``
The step "Check the release tags name this run's commit" of the ``github``,
Expand Down
18 changes: 9 additions & 9 deletions util/run_unittest_leg.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,18 +50,17 @@
:file:`.github/workflows/unit-tests.yml`'s ``unittest-ordering`` job calls
once per entry of its own matrix -- the leg list lives in that workflow, not
as a module-level constant here. :mod:`tests.protocols` is the largest leg;
its timings, on different bases (a local serial run, and CI before and
after it was sharded), are tabulated in that workflow's own comment rather than
restated here. At the other end sit
its CI timings, before and after it was sharded, are in that workflow's own
comment rather than restated here. At the other end sit
:mod:`tests.dumpkit` and :mod:`tests.interface`, small legs within a few
seconds of each other in wall time (~39s and ~38s). Neither small leg is
:mod:`tests.const` (~268s), which is larger by test count as well.

Peak RSS was measured for the three cheapest legs only, via
:func:`resource.getrusage` on ``RUSAGE_CHILDREN`` in this venv:
:mod:`tests.cli` 152 MiB, :mod:`tests.dumpkit` 237 MiB, :mod:`tests.interface`
305 MiB. No figure was taken for the larger legs, and that workflow comment's
table carries wall times only, no RSS -- but the largest of the three is still
305 MiB. No figure was taken for the larger legs, and that workflow comment
gives wall times only, no RSS -- but the largest of the three is still
some 90x short of 29 GB, which is enough to place the OOM on running
*everything* in one process rather than on any one directory.

Expand Down Expand Up @@ -102,8 +101,8 @@
A stalled leg: ``--stall-dump SECONDS`` (or ``PCAPKIT_UNITTEST_STALL_DUMP``)
arms :func:`faulthandler.dump_traceback_later`, so a leg still running after
that long writes every thread's stack to stderr once, then carries on. The
default, 1080s, is just under the 20-minute step cap, so a leg about to be
killed says where it was first; ``0`` disables it.
default, 1080s, is just under the ``unittest-ordering`` step's 20-minute cap,
so a leg about to be killed says where it was first; ``0`` disables it.

Memory: every purge-then-import leaves the previous generation as cyclic
garbage, and the interpreter's own collector intermittently falls behind --
Expand Down Expand Up @@ -162,8 +161,9 @@
#: this script exists to catch.
_EXCLUDED_ROOT_MODULES = frozenset({'tests.test_tier_guard_xdist'})

#: Seconds before a still-running leg dumps every thread's stack: just
#: under the 20-minute step cap, so the dump lands before the kill.
#: Seconds before a still-running leg dumps every thread's stack: just under
#: the ``unittest-ordering`` step's 20-minute cap, so the dump lands before the
#: kill.
DEFAULT_STALL_DUMP = 1080

#: Environment override for :data:`DEFAULT_STALL_DUMP`; ``--stall-dump`` wins.
Expand Down
Loading
Loading