Skip to content

fix(protocols): bound the IPv6 extension-header chain at 32 headers (#1604) - #1608

Merged
JarryShaw merged 1 commit into
mainfrom
fix/ipv6-ext-header-chain-bound
Oct 10, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/ipv6-ext-header-chain-bound

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Description of your pull request and other information

Closes #1604 for the default engine and pypcapfile. The ruling's bound for dpkt, scapy and pyshark follows in #1609.

  • Cause. Inside IPv4, each extension header dissects the next one recursively (ipv6_opts.py:324 → Internet._import_next_layer → ProtocolBase._parse_next_layer, about 11 frames a header). The RecursionError was caught by beholder (decorators.py:128), which only logs it and leaves the rest Raw. So TCP was lost after 79 headers on 3.11, and after 85 to 90 on 3.14. IPv6's own walk is a loop with no bound.
  • Fix. EXTENSION_HEADER_LIMIT = 32 in pcapkit/protocols/internet/internet.py. Internet._next_layer_class hands each header its depth in the chain. Past 32 the rest stays Raw, with one ProtocolWarning that names the bound. Any other layer between headers starts the count again. IPv6's walk stops at the same depth.
  • pypcapfile. _past_extension_headers and _upper_layer stop at 32 too.
  • dpkt, scapy, tshark. These walk the chain in their own parsers. Measured, they find TCP past 32: dpkt at any length, scapy up to about 245 layers, tshark up to about 496. They are Gap rows citing fix(toolkit): the dpkt, scapy and pyshark engines do not stop an extension-header chain at 32 #1609 in the new test_ext_chain_limit_agreement_1604_unit.py. One more row cites fix(toolkit): the dpkt and scapy engines drop TCP behind an IPv6 extension header carried in IPv4 #1606: dpkt and scapy dissect no extension header carried in IPv4, at any length.
  • The two tunnel tests in test_pypcapfile_ext_chain_limit_1604_unit.py were updated for fix(toolkit): key tunnelled TCP by the innermost IP header (#1581) #1605's tunnel walk, which now reads TCP through IP-in-IP when the innermost layer is IPv4; TCP over IPv6 is still left out.
  • Round trips stay byte-exact (frame.data, and from_data(info.to_dict())) at 32, 33 and 1,000 headers.

@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Oct 10, 2026
@JarryShaw
JarryShaw force-pushed the fix/ipv6-ext-header-chain-bound branch from e6d87c6 to d130f15 Compare October 10, 2026 15:53
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: pending No verdict for the current head - never reviewed, or the head moved since the last one labels Oct 10, 2026
@github-actions

github-actions Bot commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

Coverage: 89.72% (unit tier, Python 3.14, 254034005, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 3 20 3 95.16%
pcapkit/const 18858 956 2366 870 91.04%
pcapkit/corekit 2366 67 760 35 96.48%
pcapkit/dumpkit 275 3 100 3 98.40%
pcapkit/foundation 2883 147 1010 36 94.07%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17115 227 4570 192 98.05%
pcapkit/toolkit 907 123 322 6 85.44%
pcapkit/utilities 494 4 146 4 98.75%
pcapkit/vendor 4409 2342 1006 157 43.25%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at d130f15c8. This was a Sonnet cross-review; the author ran on Opus.

All nine claims are confirmed.

  • The bound holds: 32 headers still reach TCP and 33 or more do not. This was checked over IPv4 and IPv6, on HBH, Opts, Route, Frag, AH, HIP and mixed chains. Each cut raises one ProtocolWarning naming the bound.
  • No RecursionError up to 20,000 headers (IPv6 jumbogram). Every size takes 4–9 ms, against 3.81 s on main at 20,000.
  • Round trips are byte-exact at 1, 32, 33 and 1,000 headers.
  • pypcapfile matches the default engine in all 30 IPv4 cases.
  • No regressions: 291 protocol tests and 9,612 subtests give identical results on both trees.
  • The new tests fail on main: 41 failed, 16 passed; the passing ones are boundary guards, mutation-checked.

I reproduced the 32/33/1,000 counts for default, dpkt and scapy myself.

Non-blocking:

  • ESP, 253 or 254 as the 33rd header counts toward the bound. An ESP that main dissected after 32 headers is now Raw. RFC 8200 lists ESP as an extension header, so this reads as intended.
  • The limit is imported by value. ipv6.py and pypcapfile.py import EXTENSION_HEADER_LIMIT by name, so patching the module attribute only moves the IPv4 path. The constant isn't public.
  • Deep nesting is still unbounded. The count restarts at each tunnel, so deep IP-in-IP nesting still exhausts the stack silently, as it does on main. Filed as fix(protocols): deep layer nesting of any protocol exhausts the recursion limit and loses TCP silently #1610.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 10, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Hold this until #1605 merges: combined with #1605, two of this PR's tests fail. I merged both heads (143d691ed + d130f15c8, clean textual merge) and ran the affected files on Python 3.11 and 3.14. All pass except tests/toolkit/test_pypcapfile_ext_chain_limit_1604_unit.py, which gives 2 failed / 6 passed on both versions:

Both are test expectations that predate #1605; the code is unaffected. Merge #1605 first. I'll then rebase this PR onto main with those two tests updated and get the delta re-reviewed. Relabelled review: needs-changes until then.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 10, 2026
…1604)

- Inside IPv4, each extension header dissected the next recursively, so a long
  chain hit the recursion limit and beholder left the rest raw, silently, at a
  depth that varied by interpreter; IPv6's own walk had no bound at all.
  EXTENSION_HEADER_LIMIT (32) now caps both: past it the rest stays Raw, with
  one ProtocolWarning.
- pypcapfile's _past_extension_headers and _upper_layer stop at the same depth.
- dpkt, scapy and tshark walk the chain in their own parsers; where they
  disagree is recorded as Gap rows in a new agreement module (#1609, #1606).
@JarryShaw
JarryShaw force-pushed the fix/ipv6-ext-header-chain-bound branch from d130f15 to 2540340 Compare October 10, 2026 16:52
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 10, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 254034005. This is the round-3 delta review by Sonnet; the author ran on Opus.

This head is one commit on main (d6f627481). Its tree is identical to the re-stacked commit I checked earlier. Only test_pypcapfile_ext_chain_limit_1604_unit.py differs from the reviewed d130f15c8. The review confirmed four things:

  • The two updated tests still catch a missing bound. With both limit checks removed: 14 failed / 7 passed on Python 3.11 and 3.14. Either check removed alone still fails both rewritten tests.
  • The bound holds on both sides of a tunnel. The cases were 4in4, 4in6, 6in4 and 6in6, with 0, 31, 32 or 33 headers before the tunnel and inside it. Default finds TCP only when both sides are 32 or fewer, and pypcapfile agrees in all 64 cases.
  • No test depends on the removed tunnel warning text.
  • No regressions. The toolkit, pypcapfile and engine-agreement selection passes on both Pythons. The only additions over main are the PR's own tests.

I corrected the body's new bullet: #1605 reads TCP through IP-in-IP only when the innermost layer is IPv4.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 10, 2026
@JarryShaw
JarryShaw merged commit ee47dbe into main Oct 10, 2026
39 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 10, 2026
@JarryShaw
JarryShaw deleted the fix/ipv6-ext-header-chain-bound branch October 10, 2026 22:59
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 10, 2026
JarryShaw added a commit that referenced this pull request Oct 14, 2026
…1721)

Third-party engines are best effort (#1687), so the pypcapfile code that
exists only to match the default engine comes out:

- revert #1682, #1600, #1575 and #1535, and the pypcapfile hunks of #1655,
  #1608 and #1605: reading loopback and raw-IP link types, the frame layer
  bound, the VLAN and tunnel walks, keying tunnelled TCP by the innermost
  header, stepping over AH and extension headers, and checking options
  with the default engine's parsers
- keep the helpers the dpkt and scapy adapters import, unchanged, until
  their own reverts drop them; their docstrings no longer say pypcapfile
  asks the default engine's parsers too
- re-land #1576, #1577, #1590 and #1591 as guards in _network and
  _transport, #1513's once-per-capture warning for TCP straight over
  IPv6, and #1667's packet2dict fix: an undecoded frame is reported as
  its octets, not as hex text twice as long
- keep #1569's record reader and #1690's packet2timestamp
- delete the three test modules those reverts added, the pypcapfile
  timestamp agreement class, and #1682's link-type warning test; the
  re-landed guards' tests assert constants
- drop what the reverts removed from the docs, READ_LINKTYPES included,
  and say that TCP behind a VLAN tag, in a tunnel or behind an
  extension header (an IPv6 one, or AH in IPv4) is now left out
  without a warning

The pypcapfile modules, test_new_engine_parity_runtime.py,
test_engine_runtime.py, the timestamp module and the project guards pass
on 3.11 (pypcapfile 0.12.0) and 3.14.
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

fix(protocols): a chain of ~80+ IPv6 extension headers exhausts recursion and leaves the payload undissected

1 participant