Repository navigation

fix(ipv6): CALIPSO make() writes a corrupt length on overflow and raises raw errors for a bad pad #2098

Workflow file for this run

.github/workflows/unit-tests.yml at c228007

	name: Unit Tests

	on:
	push:
	branches: [main]
	pull_request:
	branches: [main]
	workflow_call:
	inputs:
	gate-only:
	description: >-
	Run only the single-version gate job instead of the full matrix.
	Callers that merely need a pass/fail verdict before shipping set
	this to true: the full matrix already runs once per commit from
	this workflow's own push and pull_request triggers, so running it
	again per caller tests the same commit several times over.
	type: boolean
	required: false
	default: false

	permissions:
	contents: read

	# Only pull-request runs may be superseded. Keying every event on the ref (as
	# this did) coalesces all of main's pushes into one group, and GitHub cancels a
	# pending run whenever a newer one joins the group -- `cancel-in-progress:
	# false` protects a run that has already started, never one still queued. Merging
	# two PRs a minute apart therefore left the first commit with no unit tests at
	# all: measured on run 34970682137 for e0f0160a3, which finished `cancelled` with
	# zero jobs one second after the next push's run was created. Keying non-PR
	# events on the run id gives each commit its own group.
	#
	# `github.workflow` is the caller's workflow name inside a called workflow, and
	# `github.run_id` is the caller's run id too, so the literal `unit-tests-` prefix
	# is what stops a called run from sharing a group with the caller that invoked it
	# -- if they shared one, the callee would sit pending behind its own caller and
	# deadlock.
	concurrency:
	group: unit-tests-${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref \|\| github.run_id }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	jobs:
	test:
	# `github.event_name` is NOT 'workflow_call' inside a called workflow --
	# the github context is inherited from the caller, so it reads
	# 'pull_request', 'push', 'schedule' or 'workflow_run'. Guarding on it (as
	# this workflow used to) therefore ran the whole matrix in every caller
	# and skipped the gate everywhere. An input is the only reliable way for a
	# reusable workflow to know how it was invoked.
	name: Python ${{ matrix.python-version }}
	if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
	needs: [changes]
	runs-on: ubuntu-latest
	timeout-minutes: 45
	strategy:
	fail-fast: false
	matrix:
	python-version:
	# 3.15 deliberately excluded: ruleset 23497679's required checks
	# only cover 3.10-3.14, so a 3.15 leg here cannot block a merge and
	# was pure queue contention -- see python-compatibility.yml for its
	# (non-blocking, schedule-only) replacement.
	- "3.10"
	- "3.11"
	- "3.12"
	- "3.13"
	- "3.14"
	# Exactly one leg runs under coverage, so measuring it adds no leg to
	# the queue. 3.14 because it had the most headroom under the 45-minute
	# cap on main (15.6, 15.8 and 16.8 minutes
	# in runs 37393532176, 37392964804 and 37391296686, against 12.8-22.0
	# for the other four), and because coverage's sys.monitoring core,
	# which measures branches only from 3.14 on, is its cheapest tracer.
	include:
	- python-version: "3.14"
	coverage: true

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: ${{ matrix.python-version }}
	cache: pip

	# The DPKT extra is here so the tests gated on it (test_dpkt_unit.py,
	# test_misc.py, test_extraction.py, test_core.py) actually execute
	# instead of skipping -- see #729. It is a small, pure-Python
	# dependency (measured clean installs on 3.10-3.14), so it costs
	# nothing per leg and adds no new job or leg, unlike Scapy below.
	#
	# crypto (cryptography) and NGAP (pycrate) join it for the same reason,
	# covering two of #738's remaining gates -- HAS_CRYPTO (test_esp_unit.py,
	# 14 methods) and HAS_PYCRATE (test_ngap_unit.py, 10 methods), both plain
	# unit-tier. `cli` (emoji) is deliberately NOT added here: its only
	# HAS_EMOJI-gated tests are in tests/integration/test_cli_subprocess.py,
	# which this job ignores wholesale, so installing it here would unlock
	# nothing. cryptography is the one compiled wheel of the two; measured
	# clean installs on 3.10-3.14 (manylinux wheels, no build step). pycrate
	# is pure Python but its distribution is large (~238MB, see the NGAP
	# extra's comment in pyproject.toml) -- still cheap in wall time, just not
	# "a second" cheap.
	#
	# PyPCAPFile (pypcapfile) is deliberately NOT here. It used to be kept off
	# every job in this file: on 3.10/3.11 (where its
	# `python_version < '3.12'` marker lets it install), installing it turned
	# 7 of the 10 HAS_PYPCAPFILE-gated methods this comment used to count
	# into failures, against a real bug in pcapkit/toolkit/pypcapfile.py's
	# handling of pypcapfile 0.12.0's `IP.src`/`IP.dst`. #743 and #746 fixed
	# that bug, and #751 gave the now-safe extra a home: the dedicated
	# `engine-tests` and `pypcap-parity` jobs below install it (15
	# HAS_PYPCAPFILE-gated methods between them), deliberately apart from
	# this job rather than added to it -- see either job's own comment for
	# why.
	- name: Install package and test dependencies
	run: \|
	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test,DPKT,crypto,NGAP]'

	# 7.10 is the first release with `[run] patch = ["subprocess"]`, which
	# .github/coverage.toml relies on to measure the xdist workers.
	- name: Install coverage
	if: ${{ matrix.coverage }}
	run: python -m pip install 'coverage>=7.10'

	# `-n auto` resolves to whatever this runner reports, and nothing else in
	# the log says what that was -- print it so the number `-n auto` picked
	# is a measured fact instead of an assumption about runner size.
	- name: Report available parallelism
	run: \|
	nproc
	python -c "import os; print('cpu_count', os.cpu_count())"

	- name: Run unit tests
	id: tests
	# `--dist load` is pinned explicitly rather than left to default: a
	# future change to xdist's own default, or someone "optimising" to
	# `loadfile`/`loadscope`, cannot silently change how this suite is
	# distributed. The coverage leg runs the identical selection.
	env:
	COVERAGE: ${{ matrix.coverage }}
	run: \|
	set -- -q -n auto --dist load \
	--ignore=tests/integration \
	--ignore-glob='*_runtime.py' \
	--ignore-glob='*_regression.py'
	if [ "$COVERAGE" = "true" ]; then
	# Logs every measured process and its parent, for the check below.
	COVERAGE_DEBUG=pid,process COVERAGE_DEBUG_FILE="$PWD/coverage-processes.log" \
	python -m coverage run --rcfile=.github/coverage.toml -m pytest "$@"
	else
	python -m pytest "$@"
	fi

	# The table goes to the job summary. The numbers also go to an artifact
	# that coverage-comment.yml renders as a PR comment once this run
	# completes; that comment cannot be posted from here, because this file is
	# also a reusable workflow and its callers grant only `contents: read`.
	# `!cancelled()` so a failing test still reports what it measured, and
	# says the tests failed in the first line.
	- name: Report coverage
	if: ${{ !cancelled() && matrix.coverage }}
	shell: bash
	env:
	SHA: ${{ github.sha }}
	TESTS_OUTCOME: ${{ steps.tests.outcome }}
	run: \|
	set -euo pipefail
	# #1387: a worker killed before its atexit save leaves no .coverage.*
	# file, and combine then reports the rest as the whole. So fail, and
	# post no number, unless the controller and every xdist worker (its
	# direct children in the log above) left one.
	python - <<'PY'
	import glob, os, re, sys
	parent, cmd = {}, {}
	log = 'coverage-processes.log'
	for line in open(log, errors='replace').readlines() if os.path.exists(log) else []:
	if m := re.match(r' (\d+)\.\w+: New process(: cmd\| parent pid): (.)', line):
	(parent if m[2] == ' parent pid' else cmd)[m[1]] = m[3]
	root = [p for p, c in cmd.items() if "'run', '--rcfile=.github/coverage.toml'" in c]
	workers = sorted(p for p, q in parent.items() if root and q == root[0])
	saved = set(re.findall(r'\.pid(\d+)\.', ' '.join(glob.glob('.coverage.*'))))
	missing = [p for p in root + workers if p not in saved]
	print(f'controller {root}, {len(workers)} xdist workers, missing data: {missing}')
	if len(root) != 1 or not workers or missing:
	why = f'no data file from pid {missing}' if missing else 'the process log is missing or unreadable'
	with open(os.environ['GITHUB_STEP_SUMMARY'], 'a') as summary:
	print(f'Coverage not reported: {why} (#1387); a total would under-count.',
	file=summary)
	print(f'::error::Coverage data is incomplete: {why}.')
	sys.exit(1)
	PY
	python -m coverage combine --rcfile=.github/coverage.toml
	python -m coverage report --rcfile=.github/coverage.toml
	python -m coverage html --rcfile=.github/coverage.toml -d coverage-html
	python -m coverage json --rcfile=.github/coverage.toml -o coverage.json
	mkdir -p coverage-comment
	# Writes the numbers as JSON for coverage-comment.yml, which renders
	# its own Markdown from them rather than posting text from this run.
	python - <<'PY' >> "$GITHUB_STEP_SUMMARY"
	import collections, json, os
	data = json.load(open('coverage.json'))
	keys = ('num_statements', 'missing_lines', 'num_branches', 'num_partial_branches',
	'covered_lines', 'covered_branches')
	packages = collections.defaultdict(collections.Counter)
	for path, info in data['files'].items():
	parts = path.split('/')
	name = '/'.join(parts[:2]) if len(parts) > 2 else 'pcapkit (top level)'
	packages[name].update({k: info['summary'][k] for k in keys})
	def pct(c):
	total = c['num_statements'] + c['num_branches']
	return 100.0 * (c['covered_lines'] + c['covered_branches']) / total if total else 100.0
	rows = [[name, c['num_statements'], c['missing_lines'], c['num_branches'],
	c['num_partial_branches'], round(pct(c), 2)] for name, c in sorted(packages.items())]
	total = round(data['totals']['percent_covered'], 2)
	tests = os.environ['TESTS_OUTCOME']
	json.dump({'total': total, 'tests': tests, 'packages': rows},
	open('coverage-comment/coverage-summary.json', 'w'))
	status = '' if tests == 'success' else f' -- tests {tests}, so this is partial'
	print(f"Coverage: {total:.2f}%{status} (unit tier, Python 3.14, `{os.environ['SHA'][:9]}`)")
	print()
	print('\| Package \| Statements \| Missed \| Branches \| Partial \| Cover \|')
	print('\|---\|--:\|--:\|--:\|--:\|--:\|')
	for row in rows:
	print('\| `{}` \| {} \| {} \| {} \| {} \| {:.2f}% \|'.format(*row))
	print()
	print('Per-file detail: the `coverage-html` artifact of this run.')
	PY

	- name: Upload coverage HTML report
	if: ${{ !cancelled() && matrix.coverage }}
	uses: actions/upload-artifact@v7
	with:
	name: coverage-html
	path: coverage-html/
	retention-days: 30

	- name: Upload coverage summary
	if: ${{ !cancelled() && matrix.coverage }}
	uses: actions/upload-artifact@v7
	with:
	name: coverage-summary
	path: coverage-comment/coverage-summary.json
	retention-days: 7

	integration:
	name: Integration Python ${{ matrix.python-version }}
	if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
	needs: [changes]
	runs-on: ubuntu-latest
	timeout-minutes: 45
	strategy:
	fail-fast: false
	matrix:
	python-version:
	# See the `test` job above for why 3.15 is excluded here.
	- "3.10"
	- "3.11"
	- "3.12"
	- "3.13"
	- "3.14"

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: ${{ matrix.python-version }}
	cache: pip

	# The Scapy extra is there for examples/generators/pcap.py and legacy.py,
	# which build their captures with scapy; examples/generators/pcapng.py needs
	# nothing beyond the standard library and pcapkit itself, which it uses to
	# parse each fixture back. DPKT joins it so the two dpkt-gated methods
	# living under tests/integration/ (test_engine_parity.py,
	# test_engine_runtime.py) actually run here instead of skipping -- see
	# #729; same reasoning as the `test` job above.
	#
	# `cli` (emoji) joins it for one of #738's remaining gates. This job's
	# selection (tests/_tiers.py's fixture_tier_paths()) always includes the
	# whole of tests/integration/ -- which is where every HAS_EMOJI-gated
	# method lives (test_cli_subprocess.py, 14 methods). `crypto` and `NGAP`
	# are deliberately NOT added here: test_esp_unit.py and test_ngap_unit.py
	# are plain unit-tier and never enter this job's selection, so installing
	# cryptography/pycrate here would unlock nothing -- they are already
	# covered, across all five Python versions, by the `test` job above.
	# PyPCAPFile is NOT added here either, even though this job's selection
	# also reaches test_new_engine_parity_runtime.py's 6 HAS_PYPCAPFILE
	# methods -- see the `test` job's comment above for the bug that used to
	# make installing it here a regression, now fixed by #743/#746. #751's
	# `pypcap-parity` job below covers those 6 methods instead, on the same
	# fixture-tier selection as this job but in a venv of its own.
	- name: Install package, test and generator dependencies
	run: \|
	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test,Scapy,DPKT,cli]'

	# `shell: bash` rather than the default, because it runs with -o pipefail:
	# without it the `tee` below would swallow the generator's exit status.
	- name: Regenerate sample captures
	shell: bash
	run: \|
	# Keep a failure here legible as what it is -- the fixtures could not
	# be built, so no test has run yet and nothing is wrong with the code
	# under test.
	if ! python examples/generators/make_samples.py 2>&1 \| tee "$RUNNER_TEMP/make-samples.log"; then
	echo "::error title=Sample fixture generation failed::examples/generators/make_samples.py could not rebuild examples/captures/. This is a fixture-generation failure, not a test failure -- the test suite has not run."
	exit 1
	fi

	# examples/generators/pcapng.py fetches two captures from the Wireshark
	# repository and synthesises stand-ins when the download fails. The
	# stand-ins parse, and tests/protocols/test_pcapng_regression.py only
	# asserts that extraction succeeds, so a degraded run goes green and
	# is otherwise indistinguishable from a clean one. Say which happened.
	if grep -q '(download unavailable)' "$RUNNER_TEMP/make-samples.log"; then
	echo "::warning title=Sample fixtures degraded::The upstream Wireshark captures were unreachable, so synthesised stand-ins were used. The PCAP-NG tier ran, but not against the upstream bytes."
	grep '(download unavailable)' "$RUNNER_TEMP/make-samples.log"
	else
	echo "Upstream Wireshark captures were used, matching their pinned SHA-256 digests."
	fi

	# See the `test` job above for why this step exists.
	- name: Report available parallelism
	run: \|
	nproc
	python -c "import os; print('cpu_count', os.cpu_count())"

	# The ``test`` job above already ran every unit-tier test for real, on this
	# same commit. Re-running them here (a bare ``pytest -q`` used to) bought
	# no signal and cost ~21.75 minutes per Python version -- ~130 minutes per
	# PR push across the matrix (see #715). So this selects only the
	# fixture-dependent tier plus the handful of unit-tier modules whose
	# capture read is skipped, not exercised, in the `test` job -- see
	# tests/_tiers.py's module docstring and fixture_tier_paths() for why a
	# module can be unit-tier by path and still belong here. The selection is
	# not spelled out as literal flags: it is asked of tests/_tiers.py
	# directly (FIXTURE_TIER_SELECTOR), so this step and that module cannot
	# drift the way two independent copies of the same list could.
	#
	# A failure to compute the selection intentionally does not fall back to
	# a bare `pytest -q` -- that would silently re-introduce the whole-suite
	# duplication this change exists to remove, with nothing in the log to
	# say so. `--dist load` is pinned explicitly; see the `test` job above
	# for why.
	- name: Run full test suite
	shell: bash
	run: \|
	if ! selection=$(python -c "from tests._tiers import fixture_tier_paths; print(' '.join(fixture_tier_paths()))"); then
	echo "::error title=Could not compute the fixture-tier selection::tests._tiers.fixture_tier_paths() failed -- see the traceback above. Refusing to fall back to a bare 'pytest -q', which would silently re-run the entire suite instead of failing loudly."
	exit 1
	fi
	if [ -z "$selection" ]; then
	echo "::error title=Fixture-tier selection is empty::tests._tiers.fixture_tier_paths() returned nothing, which cannot be right -- tests/integration alone should always be part of it. Refusing to run pytest with no arguments."
	exit 1
	fi
	echo "Fixture-dependent selection: $selection"
	python -m pytest -q -n auto --dist load $selection

	# Per-engine coverage for the third-party capture engines (#845): every
	# (Python version, engine) pair is checked, and a failure -- or a correctly
	# asserted decline -- names the one engine and interpreter responsible. Each
	# pair ("cell") resolves to one of three expected outcomes, decided by the
	# cell script's `expect` case below (anything not listed there is
	# `supported`):
	#
	# * `supported` -- the engine installs via its normal pyproject.toml
	# extra and its dedicated unit-tier tests must
	# actually pass, proving this engine ran.
	# * `unsupported` -- the engine is installable on this interpreter but
	# its own PYTHON_CEILING means it declines at run
	# time (PyShark on 3.14+; PyPCAPFile on 3.12+, whose
	# pip marker would otherwise stop it ever installing
	# here -- so the cell installs the raw distribution
	# past that marker, deliberately, rather than via the
	# extra). The same test files run; their own
	# version-aware assertions (e.g.
	# PyPCAPFile's test_unsupported_reason_tracks_the_running_interpreter)
	# are what prove the decline actually fired and named
	# the interpreter, rather than merely skipping.
	# * `not-installable` -- PyPCAP on 3.12+: a C-extension build failure, not
	# a runtime refusal (no wheel; pcap.c does not
	# compile against the 3.12+ C API -- PyPCAP carries
	# no PYTHON_CEILING at all, so there is no decline to
	# assert). The cell still installs the raw
	# distribution past pyproject.toml's marker to prove
	# the failure is real rather than assumed, and treats
	# that failure as the expected, passing outcome --
	# distinctly logged via `::notice`, never silently
	# green and never red for the wrong reason. If it
	# ever starts building, the cell fails loudly.
	#
	# One job per engine loops over the six interpreters, each in a fresh venv,
	# rather than one job per cell (6 Pythons x 6 engines): the queue, not the
	# runners, is the bottleneck, and 36 jobs would each pay checkout, setup and
	# apt for a minute of work. One job per engine rather than per Python
	# because the engine is what varies the environment: the apt packages
	# are per engine (installed once per job instead of six times), and pcap-ct
	# and pypcap both install a top-level `pcap` module, so a per-Python job
	# looping over engines would need a fresh venv per engine anyway while also
	# carrying every engine's system packages. It also keeps `matrix.engine` as
	# the one matrix axis, which is what tests/_dependency_gates.py's
	# _engine_matrix_variants() reads this job as. The cell script below is one
	# cell's install-and-test logic, run once per interpreter as its own process
	# (so `set -e` holds inside it), under a `::group::` named for the cell, with
	# every annotation titled by engine and Python version and a per-cell row in
	# the job summary.
	#
	# 3.15 is experimental-prerelease per #845's ruling: present so a regression
	# is visible, but never blocking. It runs as its own step with step-level
	# `continue-on-error: true`, so a 3.15 failure shows as a failed step and an
	# error annotation while the job -- and `required-checks`, which reads this
	# job's result -- stays green. The 3.10-3.14 cells share one step, and any
	# one of them failing fails the job.
	#
	# Scoped deliberately out (#845): per-engine integration
	# coverage (needs regenerated captures, so it stays in aggregate in
	# `integration` and `pypcap-parity`), and the multi-engine HAS_RUNTIME gate
	# plus the incidental crypto/NGAP/vendor coverage the pre-#845 shared venv
	# carried.
	engine-tests:
	name: Engines (${{ matrix.engine }})
	if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
	needs: [changes]
	runs-on: ubuntu-latest
	timeout-minutes: 30
	env:
	ENGINE: ${{ matrix.engine }}
	strategy:
	fail-fast: false
	matrix:
	engine:
	- DPKT
	- Scapy
	- PyShark
	- PyPCAPFile
	- PyPCAP
	- PCAP_CT

	steps:
	- uses: actions/checkout@v7

	# All six interpreters in one step; each is then reachable as
	# `python3.X`. setup-python does not ship 3.15 as a release build yet, so
	# `allow-prereleases` is needed for it to resolve at all (as in
	# python-compatibility.yml's compatibility-nightly job); for 3.10-3.14 a
	# GA release exists and wins.
	- uses: actions/setup-python@v7
	with:
	python-version: \|
	3.10
	3.11
	3.12
	3.13
	3.14
	3.15
	allow-prereleases: true

	# Only the engines that need a system package get one:
	#
	# tshark: PyShark is a wrapper around it and does no parsing of its own.
	# The debconf pre-seed stops tshark's postinst blocking on the "allow
	# non-superusers to capture packets" prompt.
	#
	# libpcap0.8: PCAP_CT's ctypes loader calls find_library("pcap") at run
	# time; without it, PCAP_CT.unsupported_reason() degrades the engine to
	# the default parser instead of running it.
	#
	# build-essential + libpcap-dev: PyPCAP is the one engine that compiles
	# (pcap.c against libpcap's headers), on every cell -- including the
	# `not-installable` ones, deliberately, so that failure is a genuine
	# compile failure against the 3.12+ C API rather than merely "no
	# compiler was present to try".
	#
	# `timeout-minutes` bounds this step because an apt stall in it used to
	# eat the whole job: per #974, a PCAP_CT cell fetched its InRelease
	# metadata and then went silent for 29m52s until the job's own 30-minute
	# cap killed it, which reports `cancelled` and so fails `Required checks
	# passed` -- an unrelated mirror turning a green PR red. Stalls in
	# `apt-get update` have had the same all-or-nothing shape.
	#
	# 10 minutes, with one retry inside it. These are whole-step wall times,
	# both attempts included, so they bound no single attempt. On 2026-10-07
	# this step failed at 141s (job 112943258387, #1306, PyShark,
	# 18:17:51Z-18:20:12Z) and the parity one at 155s (job 112948578234,
	# #1307, 18:31:41Z-18:34:16Z, 27.9 MB of archives with
	# `azure.archive.ubuntu.com` logging `Ign:` every 30s). Across the runs
	# sampled, failures ran 91s to 194s, most at 187-194s: the band the old
	# 45s/90s bounds cut off. So 60s update, 180s install (~27 MB at >=0.15
	# MB/s), two attempts, 480s worst case -- only 120s under the 600s step
	# cap, and steps past 200s are on record, so the cap rather than that
	# margin is the hang detector. `dpkg --configure -a` precedes the retry.
	#
	# Deliberately not passing `-o Acquire::Retries=3 -o
	# Acquire::http::Timeout=30`, which #974 proposed: both are already the
	# defaults in noble's apt (2.8.3: `Acquire::Retries` 3 in
	# apt-pkg/acquire-item.cc, `TimeOut(30)` in methods/basehttp.cc), so they
	# would change nothing while reading like protection. Their being in
	# force is also why the #974 stall ran to 29m52s: whatever hung is not
	# something apt's per-`select()` data timeout governs, which is why the
	# bound here is `timeout(1)` around the whole command.
	- name: Install this engine's system packages
	timeout-minutes: 10
	run: \|
	set -eu
	apt_install() {
	for attempt in 1 2; do
	if sudo timeout 60 apt-get update \
	&& sudo DEBIAN_FRONTEND=noninteractive timeout 180 apt-get install -y --no-install-recommends "$@"; then
	return 0
	fi
	echo "::warning title=apt attempt $attempt of 2 failed (${ENGINE})::apt-get update or install failed or exceeded its per-attempt bound; retrying once."
	sudo dpkg --configure -a \|\| true
	done
	return 1
	}
	if [ "$ENGINE" = "PyShark" ]; then
	echo "wireshark-common wireshark-common/install-setuid boolean false" \| sudo debconf-set-selections
	apt_install tshark
	elif [ "$ENGINE" = "PCAP_CT" ]; then
	apt_install libpcap0.8
	elif [ "$ENGINE" = "PyPCAP" ]; then
	apt_install build-essential libpcap-dev
	fi

	# The per-cell script: one (Python version, engine) cell, start to
	# finish, in a venv of its own. Written to a file so each cell runs as a
	# separate `bash` process -- `set -e` is suspended inside a function
	# called from an `if`, so an inline function would let a failed install
	# carry on silently.
	#
	# `expect` decides not just the assertion but how the engine gets
	# installed: `supported` uses the ordinary pyproject.toml extra, whose
	# own marker (if any) already resolves to "install" on this Python
	# version; `unsupported` and `not-installable` install the raw
	# distribution directly, deliberately bypassing whichever marker would
	# otherwise have skipped it silently (#845 finding 2).
	#
	# `set +e` is scoped to the one install command whose failure is
	# sometimes expected. Upgrading pip and editable-installing this repo's
	# `test` extra failing is never expected on any cell, so those run under
	# the script's `set -eu` (#845's cross-review: running them unchecked
	# once let a broken base install on a `not-installable` cell read as
	# "expected").
	#
	# Runs the SAME test files whether `expect` is `supported` or
	# `unsupported`: the difference is in what those files assert once they
	# see this interpreter's version (PyPCAPFile's
	# test_unsupported_reason_tracks_the_running_interpreter, PyShark's
	# test_the_reason_tracks_the_running_interpreter), which is what proves
	# the decline fired. `not-installable` cells run no tests.
	- name: Write the per-cell script
	run: \|
	cat > "$RUNNER_TEMP/engine-cell.sh" <<'CELL'
	set -eu
	PY="$1"
	CELL_NAME="$ENGINE on Python $PY"

	case "$ENGINE:$PY" in
	PyShark:3.14\|PyShark:3.15) EXPECT=unsupported ;;
	PyPCAPFile:3.12\|PyPCAPFile:3.13\|PyPCAPFile:3.14\|PyPCAPFile:3.15) EXPECT=unsupported ;;
	PyPCAP:3.12\|PyPCAP:3.13\|PyPCAP:3.14\|PyPCAP:3.15) EXPECT=not-installable ;;
	*) EXPECT=supported ;;
	esac
	echo "Cell: $CELL_NAME, expecting '$EXPECT'"

	"python$PY" -m venv "$RUNNER_TEMP/venv-$PY"
	# shellcheck disable=SC1090
	. "$RUNNER_TEMP/venv-$PY/bin/activate"
	python --version

	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test]'

	# RAW_PKGS is an array, not a string, so PCAP_CT's two-package case
	# ("pcap-ct libpcap") reaches pip as two argv entries.
	case "$ENGINE" in
	DPKT) EXTRA=DPKT; RAW_PKGS=(dpkt); IMPORT_NAME=dpkt ;;
	Scapy) EXTRA=Scapy; RAW_PKGS=(scapy); IMPORT_NAME=scapy ;;
	PyShark) EXTRA=PyShark; RAW_PKGS=(pyshark); IMPORT_NAME=pyshark ;;
	PyPCAPFile) EXTRA=PyPCAPFile; RAW_PKGS=(pypcapfile); IMPORT_NAME=pcapfile ;;
	PyPCAP) EXTRA=PyPCAP; RAW_PKGS=(pypcap); IMPORT_NAME=pcap ;;
	PCAP_CT) EXTRA=PCAP_CT; RAW_PKGS=(pcap-ct libpcap); IMPORT_NAME=pcap ;;
	*) echo "::error::unknown engine $ENGINE"; exit 1 ;;
	esac

	set +e
	if [ "$EXPECT" = "supported" ]; then
	output=$(python -m pip install -e ".[$EXTRA]" 2>&1)
	else
	output=$(python -m pip install "${RAW_PKGS[@]}" 2>&1)
	fi
	status=$?
	set -e
	echo "$output"

	if [ "$status" -ne 0 ] && [ "$EXPECT" != "not-installable" ]; then
	echo "::error title=$ENGINE failed to install on Python $PY::This cell expects '$EXPECT', which requires a successful install. pip exited $status."
	exit 1
	fi
	if [ "$status" -eq 0 ] && [ "$EXPECT" = "not-installable" ]; then
	echo "::error title=$ENGINE now installs on Python $PY::This cell is recorded as not-installable in this job's expect case (and in pyproject.toml's marker for $ENGINE). It installed cleanly -- update both rather than leaving this green for the wrong reason."
	exit 1
	fi
	if [ "$status" -ne 0 ]; then
	# A not-installable cell's point is a genuine C-API compile
	# failure, not a flaky index: grep for pip's own retry/network
	# vocabulary, and require the absence of a compile marker
	# alongside it. A bare "5xx or 429" number scan is not used,
	# because pip prints "(NNN kB)" download sizes and compiler
	# diagnostics quote line numbers in that range (#845's
	# cross-review), which would make a permanent false red.
	if echo "$output" \| grep -qiE \
	'read timed out\|connection (reset\|refused\|aborted)\|temporary failure in name resolution\|max retries exceeded\|could not fetch url\|newconnectionerror\|readtimeouterror\|httpsconnectionpool\|retrying \(retry\(\|http error (5[0-9][0-9]\|429)\|status code 429\|too many 429 error responses\|no matching distribution found' \
	&& ! echo "$output" \| grep -qiE \
	'failed building wheel\|error: command .* failed with exit code\|subprocess-exited-with-error'; then
	echo "::error title=$ENGINE's install failure on Python $PY looks transient, not a compile failure::pip exited $status, but the output matches a network/index failure pattern rather than the C-API compile failure this cell records as expected. Re-run the job rather than trusting this as a genuine 'not-installable' result."
	exit 1
	fi
	echo "::notice title=$ENGINE is not installable on Python $PY (expected)::pip exited $status, matching this cell's recorded expectation ('not-installable'), and the output does not match a transient-failure pattern. No tests run for this cell -- this is a deliberate skip, not a pass."
	echo "\| $PY \| $ENGINE \| $EXPECT \| not installable (expected) \|" >> "$GITHUB_STEP_SUMMARY"
	exit 0
	fi

	# #845 finding 2: a `supported` cell whose extra resolves to nothing
	# still exits 0 and every test would then skip, so require the
	# module this cell exists to exercise.
	if ! python -c "import $IMPORT_NAME" 2>/dev/null; then
	echo "::error title=$ENGINE installed but $IMPORT_NAME does not import on Python $PY::pip exited 0 but the module this cell exists to exercise is not importable -- a green install that unlocked nothing."
	exit 1
	fi

	case "$ENGINE" in
	DPKT) TEST_PATHS="tests/toolkit/test_dpkt_unit.py" ;;
	Scapy) TEST_PATHS="tests/toolkit/test_scapy_unit.py tests/foundation/engines/test_scapy_engine.py tests/protocols/transport/test_sctp_unit.py" ;;
	PyShark) TEST_PATHS="tests/toolkit/test_pyshark_unit.py tests/foundation/engines/test_pyshark_engine.py" ;;
	PyPCAPFile) TEST_PATHS="tests/toolkit/test_pypcapfile_unit.py tests/foundation/engines/test_pypcapfile_engine.py" ;;
	PyPCAP) TEST_PATHS="tests/toolkit/test_pypcap_unit.py tests/foundation/engines/test_pypcap_engine.py tests/foundation/engines/test_pcap_backend.py" ;;
	PCAP_CT) TEST_PATHS="tests/toolkit/test_pcap_ct_unit.py tests/foundation/engines/test_pcap_ct_engine.py tests/foundation/engines/test_pcap_backend.py" ;;
	esac
	echo "Running: $TEST_PATHS"
	# shellcheck disable=SC2086
	if ! python -m pytest -q -n auto --dist load $TEST_PATHS; then
	echo "::error title=$ENGINE tests failed on Python $PY::See the 'Python $PY ($ENGINE)' group in this step's log."
	exit 1
	fi
	echo "\| $PY \| $ENGINE \| $EXPECT \| passed \|" >> "$GITHUB_STEP_SUMMARY"
	CELL
	{
	echo "### Engine cells: $ENGINE"
	echo
	echo "\| Python \| Engine \| Expect \| Result \|"
	echo "\| --- \| --- \| --- \| --- \|"
	} >> "$GITHUB_STEP_SUMMARY"

	# See the `test` job above for why this step exists.
	- name: Report available parallelism
	run: \|
	nproc
	python -c "import os; print('cpu_count', os.cpu_count())"

	# Every cell runs even after one fails, so one red run reports all of
	# them; the step fails at the end if any did.
	- name: Run the 3.10-3.14 cells
	shell: bash
	run: \|
	failed=()
	for py in 3.10 3.11 3.12 3.13 3.14; do
	echo "::group::Python $py ($ENGINE)"
	if ! bash "$RUNNER_TEMP/engine-cell.sh" "$py"; then
	failed+=("$py")
	echo "\| $py \| $ENGINE \| \| FAILED \|" >> "$GITHUB_STEP_SUMMARY"
	fi
	echo "::endgroup::"
	done
	if [ "${#failed[@]}" -ne 0 ]; then
	echo "::error title=$ENGINE failed on Python ${failed[*]}::See each cell's group in this step's log and the job summary."
	exit 1
	fi

	# Non-blocking by #845's ruling -- see this job's comment above.
	- name: Run the 3.15 cell (experimental, non-blocking)
	shell: bash
	continue-on-error: true
	run: \|
	echo "::group::Python 3.15 ($ENGINE)"
	if ! bash "$RUNNER_TEMP/engine-cell.sh" 3.15; then
	echo "\| 3.15 \| $ENGINE \| \| FAILED (non-blocking) \|" >> "$GITHUB_STEP_SUMMARY"
	echo "::endgroup::"
	echo "::error title=$ENGINE failed on Python 3.15 (non-blocking)::3.15 is experimental per #845; this does not fail the job."
	exit 1
	fi
	echo "::endgroup::"

	# Whether upstream PyPCAP is worth building in CI at all -- #751's own
	# instruction was "try to build and if the CI is not a good suit, then we
	# ripe it". This job is that attempt: a C toolchain plus libpcap headers, on
	# the two Python versions its own marker allows
	# ("pypcap; python_version < '3.12'", pyproject.toml:157). If a clean run of
	# this job's install step goes red -- not a flake -- the fix is deleting this
	# job and returning HAS_PYPCAP to
	# tests/_dependency_gates.DEPENDENCY_GATE_EXCLUSIONS with that run linked as
	# the reason, per the ruling.
	#
	# Kept apart from `engine-tests` above for two reasons: it needs a compiler
	# that job has no other reason to carry, and its `pcap` module would collide
	# with pcap-ct's if both were installed into the one venv (see
	# `engine-tests`'s own comment on that). The HAS_PYPCAP-gated module this job
	# targets, tests/foundation/engines/test_new_engine_parity_runtime.py, reads
	# generated captures (arp.pcap, tcp.pcap, ipv4.pcap -- its own module
	# docstring), so this mirrors the `integration` job's fixture-tier selection
	# and Scapy/DPKT/cli baseline rather than `test`'s ignore-shape one, and pays
	# that job's full run cost a second time on top -- accepted here rather than
	# discovered, since tests/_dependency_gates.py's guard has no cheaper of its
	# three known selection shapes for reaching one fixture-dependent module (see
	# that module's own docstring). The same module also carries 6 of
	# HAS_PYPCAPFILE's 15 gated methods, which come along for free once
	# PyPCAPFile is on this job's install line too.
	#
	# The matrix is deliberately just 3.10 and 3.11, not the full five: PyPCAP's
	# marker and PyPCAPFile's marker are both "python_version < '3.12'", so
	# 3.12-3.14 would install neither extra here and pay this job's full
	# fixture-tier run for zero new coverage.
	pypcap-parity:
	name: PyPCAP/PyPCAPFile parity Python ${{ matrix.python-version }}
	if: ${{ inputs.gate-only != true && needs.changes.outputs.code != 'false' }}
	needs: [changes]
	runs-on: ubuntu-latest
	timeout-minutes: 45
	strategy:
	fail-fast: false
	matrix:
	python-version:
	- "3.10"
	- "3.11"

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: ${{ matrix.python-version }}
	cache: pip

	# pypcap ships no wheel: it compiles pcap.c against libpcap, so both the
	# headers and the shared library have to be present before pip is asked
	# to build it.
	#
	# tshark joins them per #845's finding 1: this job installs pyshark and
	# its fixture-tier selection (see this job's own comment above) is the
	# only place tests/integration/test_engine_runtime.py's real
	# ``engine='pyshark'`` extraction is reachable at all --
	# `engine-tests`'s own selection ignores tests/integration and
	# *_runtime.py wholesale. Without tshark, PyShark.unsupported_reason()
	# declines the engine, the extractor silently falls back to the default
	# parser, and the test's assertions used to pass on that fallback's
	# output -- a vacuous pass, invisible in skip counts. The debconf
	# pre-seed and DEBIAN_FRONTEND mirror `engine-tests`'s "Install this
	# engine's system packages" step exactly, for the same reason: tshark's
	# postinst otherwise blocks on the "allow non-superusers to capture
	# packets" prompt.
	#
	# `timeout-minutes` and the retry for the reason given on `engine-tests`'s
	# own install step -- see the note there, which this step shares
	# wholesale, including the 10-minute cap (under this job's 45) and the
	# 60s/180s bounds. It was not the leg that stalled in #974, but it has
	# since failed the same way, at 155s, on 2026-10-07: #1307's job
	# 112948578234, the second instance cited in that note.
	- name: Install libpcap headers, a C toolchain, and tshark
	timeout-minutes: 10
	run: \|
	set -eu
	echo "wireshark-common wireshark-common/install-setuid boolean false" \| sudo debconf-set-selections
	for attempt in 1 2; do
	if sudo timeout 60 apt-get update \
	&& sudo DEBIAN_FRONTEND=noninteractive timeout 180 apt-get install -y --no-install-recommends build-essential libpcap-dev tshark; then
	exit 0
	fi
	echo "::warning title=apt attempt $attempt of 2 failed::apt-get update or install failed or exceeded its per-attempt bound; retrying once."
	sudo dpkg --configure -a \|\| true
	done
	exit 1

	- name: Install package, test and generator dependencies
	run: \|
	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test,Scapy,DPKT,cli,PyShark,PyPCAP,PyPCAPFile]'

	# See the `integration` job above for why this step is shaped the way it
	# is.
	- name: Regenerate sample captures
	shell: bash
	run: \|
	if ! python examples/generators/make_samples.py 2>&1 \| tee "$RUNNER_TEMP/make-samples.log"; then
	echo "::error title=Sample fixture generation failed::examples/generators/make_samples.py could not rebuild examples/captures/. This is a fixture-generation failure, not a test failure -- the test suite has not run."
	exit 1
	fi

	if grep -q '(download unavailable)' "$RUNNER_TEMP/make-samples.log"; then
	echo "::warning title=Sample fixtures degraded::The upstream Wireshark captures were unreachable, so synthesised stand-ins were used. The PCAP-NG tier ran, but not against the upstream bytes."
	grep '(download unavailable)' "$RUNNER_TEMP/make-samples.log"
	else
	echo "Upstream Wireshark captures were used, matching their pinned SHA-256 digests."
	fi

	# See the `test` job above for why this step exists.
	- name: Report available parallelism
	run: \|
	nproc
	python -c "import os; print('cpu_count', os.cpu_count())"

	# See the `integration` job above for why this is asked of
	# tests/_tiers.py directly rather than spelled out as literal flags.
	- name: Run full test suite
	shell: bash
	run: \|
	if ! selection=$(python -c "from tests._tiers import fixture_tier_paths; print(' '.join(fixture_tier_paths()))"); then
	echo "::error title=Could not compute the fixture-tier selection::tests._tiers.fixture_tier_paths() failed -- see the traceback above. Refusing to fall back to a bare 'pytest -q', which would silently re-run the entire suite instead of failing loudly."
	exit 1
	fi
	if [ -z "$selection" ]; then
	echo "::error title=Fixture-tier selection is empty::tests._tiers.fixture_tier_paths() returned nothing, which cannot be right -- tests/integration alone should always be part of it. Refusing to run pytest with no arguments."
	exit 1
	fi
	echo "Fixture-dependent selection: $selection"
	python -m pytest -q -n auto --dist load $selection

	# GitHub issue #981: two test modules
	# (tests/protocols/application/test_http_unit.py and
	# tests/test_base_class_contract.py) each pass alone, and pytest -- the
	# runner every job above uses -- reports the combined run green, while
	# plain unittest over the same two modules in one process fails four
	# subTest cases. pytest-subtests is not why -- it is not even installed in
	# this project (absent from the `test` extra in pyproject.toml), and plain
	# pytest already reports a failed subTest as its own top-level SUBFAILED
	# entry rather than folding it into a passing parent. The real cause is
	# tests/conftest.py's autouse restore_module_table fixture, which
	# reconciles the cross-module pcapkit reimport pollution #981 is about --
	# but only under pytest, since plain unittest never loads conftest.py at
	# all. Every job above runs under pytest, so that reconciliation is why
	# none of them can see this shape of defect -- invisible by construction
	# once the fixture is doing its job, not by oversight. This job exists
	# solely to make that visible, by using a different test runner --
	# unittest.TextTestRunner -- that never loads conftest.py and so never
	# gets the reconciliation that hides the defect from pytest.
	#
	# A cheaper, narrower alternative: `pytest --noconftest` over just those two
	# modules, in that order, disables that same autouse fixture directly and
	# reproduced the identical four subTest failures in about 70s on the pre-fix
	# tree, under the pytest this CI already installs. Whether that single
	# invocation would have been sufficient instead of this dedicated job and its
	# per-directory matrix was not evaluated when this job was written --
	# recorded here rather than justified after the fact; see
	# util/run_unittest_leg.py's own docstring for the fuller version of both
	# this and the paragraph above.
	#
	# util/run_unittest_leg.py (its own docstring has the full reasoning) runs
	# one tests/ subdirectory per matrix cell (for protocols, one shard of it,
	# see #1029 below), paired with every root-level
	# tests/test_*.py module, directory first and root second -- the order
	# #981's own reproduction needs, since the defect is an earlier module
	# polluting a later one. Running the whole suite in one process is not an
	# option (it OOMs at 29 GB on the machine this was diagnosed on); a matrix
	# of one leg per top-level directory (protocols, since #1029, being two
	# cells) is the affordable approximation, and it
	# is sized for wall time rather than for memory. Peak RSS was measured for
	# the three cheapest legs only -- cli 152 MiB, dumpkit 237 MiB, interface
	# 305 MiB (resource.getrusage on RUSAGE_CHILDREN, this venv) -- leaving the
	# largest of the three some 90x short of 29 GB; no figure was taken for
	# tests/protocols, the largest leg. Measured wall times (this venv, serial):
	# cli 18s, const 268s, dumpkit 39s, foundation 272s, interface 38s, project
	# 70s, protocols 847s (before the #1029 split), toolkit 97s, utilities 42s,
	# vendor 104s, one leg per
	# matrix cell -- see `timeout-minutes` below for why the job-level budget is
	# no longer 20.
	#
	# GitHub issue #1029: the protocols cell measured 30.4 minutes on a CI
	# runner against this job's 45-minute cap (#1029 measured the next slowest
	# leg at 6.2 minutes on CI), so it is now two cells: `protocols/internet` and the rest of
	# tests/protocols (its other five subdirectories and the modules sitting
	# directly in it), the latter expressed as `protocols` with
	# `--exclude protocols/internet`. run_unittest_leg.py takes a sub-path and
	# repeatable --exclude for this; every other cell is untouched.
	#
	# Measured on CI (GitHub Actions wall time per cell, one unit throughout):
	#
	# leg run 37334173854 run 37341197811
	# (#1031, unsharded) (#1032, sharded)
	# protocols 27.8m -
	# protocols/internet - 8.2m
	# protocols (rest) - 7.6m
	# foundation (control) 6.2m 6.4m
	# const (control) 6.0m 6.1m
	#
	# The baseline is 27.8m from a same-hour run, deliberately not #1029's
	# headline 30.4m (its quiet-runner re-run, taken after a window of
	# cancelled runs). The controls moved by 0.1-0.2m between the two runs, so
	# the runners compare. Two cells rather than three because these already
	# balance (8.2m against 7.6m) and the next slowest unsharded leg is
	# foundation at 6.4m: a third cell could save under two minutes of wall time
	# and would cost more co-running module pairs. No local per-module timings
	# are recorded here; the figure above is the one that matters, and the
	# pre-#1029 local serial table above is left as it was measured.
	#
	# What sharding costs: modules in different cells no longer share a process,
	# so pollution from one to the other is invisible again. Counts below were
	# derived with leg_modules() on main at 30714cd51 plus this change: 51
	# modules in tests/protocols (12 under internet, 39 in the rest, of which 10
	# sit directly in tests/protocols). "Pairs" are unordered, C(n,2), since a
	# single serial run realises each pair in one order. Before: 1275 pairs
	# co-ran in one process. After: 807 (internet x internet 66, rest x rest
	# 741); 468 do not. The 255 pairs of a protocols module followed by a
	# root-level module (51 x 5 roots) all survive, since every cell still runs
	# the root modules after its own. The 10 modules sitting directly in
	# tests/protocols stay in the rest cell: moving them into internet would keep
	# 637 pairs instead of 807 and lengthen internet, the slower cell already.
	#
	# tests/vendor was first left out of ``leg`` because running it this way
	# surfaced an instance of the very defect this job exists to catch (issue
	# #985): a stale VendorRuntimeWarning generation made assertWarnsRegex fail
	# in tests/vendor/test_vendor_snapshot_restore_unit.py. That was fixed by
	# re-resolving the class fresh per test, so the directory is now a leg (297
	# tests in the leg, 118 of them tests/vendor's own, measured on this venv).
	#
	# Two directories are deliberately absent from ``leg``, for reasons that
	# are not this job's to fix:
	#
	# * tests/corekit -- already known and already accepted, not a new
	# finding. tests/_support.py's own purge_modules() docstring measures
	# and documents exactly this: "python -m unittest discover -s
	# tests/corekit fails 5 of that class's identity subtests, because
	# test_module sorts first and purges". pytest is "the supported
	# runner" for precisely this reason -- tests/conftest.py's autouse
	# restore_module_table fixture reconciles the module-identity drift
	# that plain unittest has no way to see coming. Running this leg
	# against tests/corekit would be permanently red for a characteristic
	# the suite's own documentation already treats as expected, not a
	# regression this job would be reporting.
	# * tests/integration -- needs generated captures this script does not
	# build, and is already run whole, under pytest, by the `integration`
	# job above. The exclusion lives here, in this matrix, and nowhere else:
	# util/run_unittest_leg.py does not refuse the directory, it collects
	# nothing from it. tests._tiers.is_unit_tier() rejects all 12 of its
	# test_*.py files, so leg_modules('integration') returns an empty tuple
	# and the leg would run the root-level modules alone -- green, while
	# covering none of tests/integration. Adding it back here would buy that
	# nothing, not a failure.
	#
	# Not added to ``required-checks`` below: a brand-new job earns a place in
	# the required set once its own track record justifies it, which is the
	# maintainer's call to make separately from landing it.
	unittest-ordering:
	name: Plain unittest ordering (${{ matrix.label \|\| matrix.leg }})
	# Not on pull requests: it is not a required check, and its 11 cells
	# would be a sixth of every PR push's jobs. It still runs on every push
	# to main, which is the run that tests what actually lands.
	if: ${{ inputs.gate-only != true && github.event_name != 'pull_request' }}
	runs-on: ubuntu-latest
	# 45, not 20: `timeout-minutes` is job-level, so it has to cover
	# checkout/setup/install (~39s) as well as the leg itself, and this is
	# the only test job in this file with no xdist underneath it, so its
	# protocols leg (847s serial, measured above) carries the full cost
	# alone. A review pass measured medium legs at 1.22-1.31x that figure on
	# a contended machine, putting protocols at 17-18.5 minutes, and one
	# contended run of it hit 1275s outright -- over the old 20-minute cap.
	# 45 matches the `test`/`integration`/`pypcap-parity` jobs' own cap
	# rather than inventing a new number (`engine-tests` is 30, but runs
	# under xdist, which this job deliberately does not). 45 did stop fitting
	# comfortably (#1029: protocols at 30.4 minutes on CI), so `protocols` is
	# now split into sub-legs, as foreseen -- see the matrix comment above. The
	# cap is left at 45: CI measured the slowest cell, protocols/internet, at
	# 8.2m (run 37341197811), so even a slowdown like #1029's window leaves it
	# far inside the cap: that window's runs were cancelled at 45.3m or more,
	# so it ran at least 45.3/27.8, about 1.6x (a lower bound, the runs were
	# cut off), which would put the 8.2m cell near 13m. Lowering the cap
	# would only trade headroom for a tighter timeout. Raising it to 60
	# instead was the one-line alternative; it would have kept every
	# cross-module pair co-running but left the slowest cell at 28 minutes.
	timeout-minutes: 45
	strategy:
	fail-fast: false
	matrix:
	leg:
	- cli
	- const
	- dumpkit
	- foundation
	- interface
	- project
	- protocols/internet
	- toolkit
	- utilities
	- vendor
	include:
	# The remainder of tests/protocols -- see the #1029 note above.
	- leg: protocols
	exclude: --exclude protocols/internet
	label: protocols (rest)

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: "3.14"
	cache: pip

	# DPKT/crypto/NGAP join `test` for the same reason the `test` job above
	# installs them: this leg's root-level modules and its own directory's
	# modules both reach gates that would otherwise skip silently rather
	# than run.
	- name: Install package and test dependencies
	run: \|
	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test,DPKT,crypto,NGAP]'

	- name: Report available parallelism
	run: \|
	nproc
	python -c "import os; print('cpu_count', os.cpu_count())"

	# `--verbose` names every test as it runs, so a slow or hung cell
	# shows where it was. The step cap of 20 is about twice the slowest
	# cell's measured maximum (9.8 minutes; the other shard 8.9), and sits
	# under the job's 45 so a hang reports as this step timing out.
	- name: Run tests/${{ matrix.leg }} and the root-level modules under plain unittest
	timeout-minutes: 20
	run: python util/run_unittest_leg.py ${{ matrix.leg }} ${{ matrix.exclude }} --verbose

	# ``CHANGELOG.md`` is generated from the newest entry under
	# ``docs/source/changelog/`` by ``util/changelog_md.py``, so it falls out of step
	# the moment an entry is edited without regenerating it. That is worth its own
	# job rather than leaving it to review, because ``Create Release`` feeds the file
	# to the GitHub Release body: a drifted copy is not merely wrong in the tree, it
	# is published.
	#
	# Deliberately not gated on ``gate-only``, unlike the four jobs above
	# (`test`, `integration`, `engine-tests`, `pypcap-parity`), and that is the
	# point of putting it here at all. ``create-release.yml`` calls this
	# workflow as its release gate, so an ungated job runs on the release path and
	# the release body cannot be built from a file that has drifted. The matrix is
	# skipped per caller because it is expensive and already ran for the commit;
	# this is neither expensive nor covered elsewhere on that path.
	#
	# ``tests/project/test_changelog_md.py`` asserts the same identity, so the
	# matrix covers it too -- but that test ``skipTest``s when either file is
	# absent, and reports a mismatch as two 17 KB strings. ``--check`` cannot skip,
	# exits 1 when the file is missing, and prints a unified diff.
	changelog:
	name: Changelog drift
	runs-on: ubuntu-latest
	timeout-minutes: 5

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: "3.14"

	# No install step: the generator imports argparse, difflib, pathlib, re, sys
	# and typing and nothing else, so it runs against a bare interpreter.
	# Verified with ``python -S``.
	- name: Check CHANGELOG.md is in step with its source entry
	shell: bash
	run: \|
	if ! python util/changelog_md.py --check; then
	echo "::error title=CHANGELOG.md has drifted::CHANGELOG.md is generated from the newest entry under docs/source/changelog/ and no longer matches it. Regenerate it with 'python util/changelog_md.py' and commit the result. The unified diff above shows what moved."
	exit 1
	fi

	# Which of a pull request's changed files are code. This decides
	# whether the expensive legs run at all (see `required-checks` for how their
	# skips are then judged), so it fails safe: anything it cannot classify is
	# code.
	#
	# A file is docs iff it is under `docs/` or ends in `.md`, and is not under
	# `.github/`, `pcapkit/`, `tests/`, `util/` or `examples/` -- so a Markdown
	# file in one of those (a PR template, a test fixture) is code, and so is
	# every other file (`pyproject.toml`, `setup.py`, ...). The diff is the PR
	# merge commit against its first parent, which is exactly what merging would
	# add; `--no-renames` lists both sides of a rename, so moving a code file
	# under `docs/` is still a code change.
	#
	# Only this workflow's own `pull_request` run is classified. Every other
	# event reports `code: true`, including a `workflow_call` from a caller
	# triggered by a pull request: inside a called workflow `github.event_name`
	# is the caller's, but `github.workflow` is the caller's name too (see the
	# concurrency comment at the top), and no caller is named "Unit Tests".
	changes:
	name: Classify changed files
	if: ${{ inputs.gate-only != true }}
	runs-on: ubuntu-latest
	timeout-minutes: 5
	outputs:
	code: ${{ steps.classify.outputs.code }}

	steps:
	- uses: actions/checkout@v7
	if: ${{ github.event_name == 'pull_request' && github.workflow == 'Unit Tests' }}
	with:
	fetch-depth: 2

	- name: Classify the diff
	id: classify
	env:
	OWN_PR: ${{ github.event_name == 'pull_request' && github.workflow == 'Unit Tests' }}
	run: \|
	python3 - <<'PY'
	import os
	import subprocess

	CODE_DIRS = ('.github/', 'pcapkit/', 'tests/', 'util/', 'examples/')

	def is_docs(path):
	if path.startswith(CODE_DIRS):
	return False
	return path.startswith('docs/') or path.endswith('.md')

	code = 'true'
	if os.environ['OWN_PR'] != 'true':
	print("Not this workflow's own pull_request run: everything runs.")
	else:
	# -z: names are NUL-terminated, so a name holding whitespace
	# (or one git would otherwise quote) stays one path.
	diff = subprocess.run(['git', 'diff', '--name-only', '-z', '--no-renames', 'HEAD^1', 'HEAD'],
	capture_output=True, text=True)
	files = [path for path in diff.stdout.split('\0') if path]
	if diff.returncode != 0 or not files:
	print(f'::warning title=Could not list the changed files::git diff exited {diff.returncode} '
	f'with {len(files)} files: {diff.stderr.strip()}. Treating the diff as code.')
	else:
	code_files = [path for path in files if not is_docs(path)]
	print(f'{len(files)} changed files, {len(code_files)} of them code:')
	for path in files:
	print(f" {'code' if path in code_files else 'docs'} {path}")
	if not code_files:
	code = 'false'
	print(f'code={code}')
	with open(os.environ['GITHUB_OUTPUT'], 'a', encoding='utf-8') as file:
	print(f'code={code}', file=file)
	PY

	# The docs-only replacement for `test`, running every test that reads
	# docs/ or a Markdown file, since a docs-only diff can still break those:
	# tests/project and the root-level modules, plus the modules elsewhere under
	# tests/ named in the second run step. tests/project/test_unit_tests_queue.py
	# finds every module outside those two that names a docs/ or Markdown path
	# and fails until it is listed there, so a new reader cannot go unrun. Runs
	# only when `changes` says the diff is docs-only, so it never duplicates
	# `test` on a code change. It uses
	# util/run_unittest_leg.py's `project` leg -- tests/project plus the
	# root-level modules, with the extras `unittest-ordering`'s `project` cell
	# runs green on main -- rather than a pytest invocation of its own, which
	# tests/_dependency_gates.py would then have to model as a new selection
	# shape.
	project-tests:
	name: Project tests (docs-only)
	needs: [changes]
	if: ${{ inputs.gate-only != true && needs.changes.outputs.code == 'false' }}
	runs-on: ubuntu-latest
	timeout-minutes: 20

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: "3.14"
	cache: pip

	- name: Install package and test dependencies
	run: \|
	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test,DPKT,crypto,NGAP]'

	- name: Run tests/project and the root-level modules under plain unittest
	timeout-minutes: 15
	run: python util/run_unittest_leg.py project --verbose

	- name: Run the other tests that read docs/ or Markdown
	timeout-minutes: 5
	run: python -m unittest -v tests.corekit.test_sentinel_exports_unit

	# The verdict a shipping workflow asks for: the full suite, fixtures and
	# all, on one interpreter. It is deliberately not the matrix -- the matrix
	# has already run against this very commit, from this workflow's own push or
	# pull_request trigger, before anything reaches a release path.
	gate:
	name: Gate (full suite, Python 3.14)
	if: ${{ inputs.gate-only == true }}
	runs-on: ubuntu-latest
	timeout-minutes: 45

	steps:
	- uses: actions/checkout@v7

	- uses: actions/setup-python@v7
	with:
	python-version: "3.14"
	cache: pip

	# DPKT joins Scapy here for the same reason as the `integration` job
	# above -- this is the full, unfiltered suite, so it is the one place
	# where skipping the dpkt-gated tests would be most misleading. crypto,
	# cli and NGAP join it for the same reason and cover the three of #738's
	# remaining gates this change fixes, since an unfiltered `pytest` run (no
	# --ignore, no tier selection) reaches every one of them regardless of
	# which tier they live in -- unlike the `test` and `integration` jobs
	# above, which each cover only the subset their own selection reaches.
	# PyPCAPFile is deliberately NOT added here -- see the `test` job's
	# comment above for the bug that used to make installing it a
	# regression, now fixed by #743/#746. This job runs on 3.14 only, where
	# PyPCAPFile's marker resolves to nothing regardless, so adding it here
	# would be a no-op that misleadingly suggests the extra is exercised by
	# `gate`; #751's `engine-tests` and `pypcap-parity` jobs cover it for
	# real, on the 3.10-3.14 (and 3.10-3.11) legs where it actually installs.
	- name: Install package, test and generator dependencies
	run: \|
	python -m pip install -U pip setuptools wheel
	python -m pip install -e '.[test,Scapy,DPKT,crypto,cli,NGAP]'

	# See the integration job above for why this step is shaped the way it is.
	- name: Regenerate sample captures
	shell: bash
	run: \|
	if ! python examples/generators/make_samples.py 2>&1 \| tee "$RUNNER_TEMP/make-samples.log"; then
	echo "::error title=Sample fixture generation failed::examples/generators/make_samples.py could not rebuild examples/captures/. This is a fixture-generation failure, not a test failure -- the test suite has not run."
	exit 1
	fi

	if grep -q '(download unavailable)' "$RUNNER_TEMP/make-samples.log"; then
	echo "::warning title=Sample fixtures degraded::The upstream Wireshark captures were unreachable, so synthesised stand-ins were used. The PCAP-NG tier ran, but not against the upstream bytes."
	grep '(download unavailable)' "$RUNNER_TEMP/make-samples.log"
	else
	echo "Upstream Wireshark captures were used, matching their pinned SHA-256 digests."
	fi

	# See the `test` job above for why this step exists.
	- name: Report available parallelism
	run: \|
	nproc
	python -c "import os; print('cpu_count', os.cpu_count())"

	# Deliberately still the whole suite, unlike the `integration` job above:
	# this is the shipping gate's own independent re-verification (see the
	# job comment above), not the same accidental duplication #715 measured
	# for `integration`. `gate` does not run per PR push -- only via
	# `create-release.yml`'s `gate-only: true` -- so it is not part of the
	# ~130 minutes/push #715 measured, and partitioning it would trade away
	# the "trust nothing, re-run everything" property this job exists for.
	# Whether `gate` should run at all on a commit the matrix already
	# covered is #715's separate item 2 ("stop the gate re-running on main
	# pushes"), a trigger-frequency question, not a selection one -- out of
	# scope here. `--dist load` is pinned explicitly; see the `test` job
	# above for why.
	- name: Run full test suite
	run: python -m pytest -q -n auto --dist load

	# Ruleset 23497679's required_status_checks requires six contexts: this
	# job's `Required checks passed`, and five for `Compat Python 3.10`-`3.14`
	# from the `compatibility` job in `.github/workflows/python-compatibility.yml`.
	# This job aggregates the gating jobs of this file, the ones its `needs:`
	# lists, because rulesets match check names literally and support no
	# wildcard: requiring each matrix cell by name would mean hand-editing the
	# ruleset every time a matrix changes. On a docs-only pull request it
	# accepts `skipped` for the four code legs (see below). The `Compat` legs
	# are required on their own because a job cannot `needs:` a job in another
	# workflow file, so nothing here can stand in for
	# them.
	#
	# `if: ${{ always() && inputs.gate-only != true }}` and the explicit
	# per-job check below are both required, not decorative:
	#
	# * Without `always()`, this job would itself be skipped -- not
	# failed -- the instant any job it `needs:` failed or skipped, per
	# "Using jobs in a workflow" > "Defining prerequisite jobs": "If a job
	# fails or is skipped, all jobs that need it are skipped unless the
	# jobs use a conditional expression that causes the job to continue."
	# And a skipped required check is not automatically a blocked merge:
	# GitHub's own "Troubleshooting required status checks" page lists
	# `skipped` alongside `success` and `neutral` as a passing check
	# status, and its "Handling skipped but required checks" table says
	# plainly that when "A job depends on a failed job", the result is
	# "The dependent job is skipped and may not block merging" -- exactly
	# the silently-worse-than-no-gate failure this job exists to prevent.
	# That page's own recommended fix is this job's shape: "Use always()
	# with needs for required checks that depend on other jobs."
	# * `inputs.gate-only != true` mirrors the jobs this needs: below.
	# They are all skipped by their own `if:` whenever this workflow is
	# called with `gate-only: true` (from create-release.yml), so without
	# this guard every needs.*.result below would read `skipped` on that
	# path and this job would report a spurious failure -- on a release
	# call, not a PR, which is not where ruleset 23497679 applies anyway.
	# * The check itself inspects every dependency explicitly and prints each
	# one's result before failing, so a red run names which leg broke
	# instead of leaving that to be re-discovered from the four jobs' own
	# logs.
	#
	# Note for whoever adds a second caller of this reusable workflow: under
	# `workflow_call`, GitHub prefixes a called workflow's check names with the
	# calling job's own name (`<caller job> / Required checks passed`), which
	# would not match a ruleset entry naming just `Required checks passed`.
	# Harmless today -- `create-release.yml`'s only call passes
	# `gate-only: true`, which skips this job via the guard above before that
	# renaming would ever matter -- but a future caller that does NOT set
	# `gate-only: true` would produce a differently-named, unrequired check.
	#
	# Deliberately NOT `needs: gate`: `gate` only ever runs when this workflow
	# is called with `gate-only: true` and is otherwise skipped by its own
	# `if:`, so depending on it would leave this job permanently skipped on
	# every ordinary PR -- the one case this job has to actually gate.
	# `changelog` is deliberately left out too: it is a drift check, not a
	# gating leg, and folding it in would widen what merging requires -- a
	# change for the maintainer to decide, not one to slip in here.
	# `test`, `integration`, `engine-tests` and `pypcap-parity` are the gating
	# legs; `changes` and `project-tests` are needed only for the docs-only
	# path below.
	#
	# `engine-tests`'s `unsupported` and `not-installable` cells are not a gap
	# here: the cell script exits 0 for a cell that declined exactly as
	# expected, so they contribute to that job's `success` like any other cell.
	#
	# Nor is its 3.15 cell. It is a step with step-level `continue-on-error:
	# true` inside each engine's job, whose effect on the job's result is
	# documented (a step's `conclusion` is its result after `continue-on-error`
	# is applied), so the job, and this gate, read `success` while the failed
	# step stays visible. Not a job-level `continue-on-error`: GitHub does not
	# document whether `needs.<job>.result` sees such a job's failure as
	# `failure` or `success`, so a 3.15-only regression could block every merge.
	#
	# The docs-only path. On a pull request whose diff touches nothing but
	# `docs/*` and `.md` files outside the code directories (the `changes`
	# job's classifier), `test`, `integration`, `engine-tests` and
	# `pypcap-parity` are skipped by their own `if:`, and `project-tests` runs
	# instead. A job-level `if:` rather than workflow-level `paths-ignore`,
	# because `paths-ignore` would not run this workflow at all and leave this
	# required context pending forever. So `skipped` is accepted for those four
	# legs only when `changes` succeeded and said `code == 'false'`, and then
	# `project-tests` must have succeeded. Every other combination keeps the
	# strict rule: `changes` failing, its output missing, or any leg skipped
	# while `code` is `true` fails this gate. `changes` reports `code: true` for
	# every event but this workflow's own `pull_request`, so `push` and
	# `workflow_call` runs never take this path.
	required-checks:
	name: Required checks passed
	needs: [changes, test, integration, engine-tests, pypcap-parity, project-tests]
	if: ${{ always() && inputs.gate-only != true }}
	runs-on: ubuntu-latest
	timeout-minutes: 5

	steps:
	- name: Check every required job succeeded
	env:
	CHANGES: ${{ needs.changes.result }}
	CODE: ${{ needs.changes.outputs.code }}
	TEST: ${{ needs.test.result }}
	INTEGRATION: ${{ needs.integration.result }}
	ENGINE_TESTS: ${{ needs.engine-tests.result }}
	PYPCAP_PARITY: ${{ needs.pypcap-parity.result }}
	PROJECT_TESTS: ${{ needs.project-tests.result }}
	run: \|
	set -eu
	echo "changes: $CHANGES (code: '$CODE')"
	echo "test: $TEST"
	echo "integration: $INTEGRATION"
	echo "engine-tests: $ENGINE_TESTS"
	echo "pypcap-parity: $PYPCAP_PARITY"
	echo "project-tests: $PROJECT_TESTS"

	failed=0
	if [ "$CHANGES" != "success" ] \|\| { [ "$CODE" != "true" ] && [ "$CODE" != "false" ]; }; then
	echo "::error title=changes did not classify the diff::result was '$CHANGES' and code was '$CODE'; without a classification nothing may be skipped."
	CODE=true
	failed=1
	fi

	# Docs-only (code == false): a leg may be skipped, and project-tests
	# is what must have passed. Otherwise every leg must have succeeded.
	if [ "$CODE" = "false" ]; then
	allowed="success skipped"
	[ "$PROJECT_TESTS" = "success" ] \|\| { echo "::error title=project-tests did not succeed::result was '$PROJECT_TESTS'; on a docs-only diff it is the test that has to pass."; failed=1; }
	else
	allowed="success"
	fi
	for pair in "test=$TEST" "integration=$INTEGRATION" "engine-tests=$ENGINE_TESTS" "pypcap-parity=$PYPCAP_PARITY"; do
	job="${pair%%=*}"
	result="${pair#*=}"
	case " $allowed " in
	" $result ") ;;
	*) echo "::error title=$job did not succeed::result was '$result'; accepted here: $allowed (code: '$CODE')."; failed=1 ;;
	esac
	done

	if [ "$failed" -ne 0 ]; then
	echo "::error title=Required checks did not all succeed::See the per-job lines above. A skipped or cancelled dependency is only a pass on a docs-only diff; see this job's own comment."
	exit 1
	fi
	echo "All required jobs reported an accepted result."

Navigation Menu

Repository navigation

fix(ipv6): CALIPSO make() writes a corrupt length on overflow and raises raw errors for a bad pad #2098

Workflow file

fix(ipv6): CALIPSO make() writes a corrupt length on overflow and raises raw errors for a bad pad #2098

Uh oh!

Workflow file for this run