Skip to content

fix(protocols): a payload layer parsed from a short stream rebuilds default-filled octets - #1171

Merged
JarryShaw merged 1 commit into
mainfrom
fix/1170-short-payload-raw
Oct 7, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/1170-short-payload-raw

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner
  • Searched for similar pull requests
  • Followed the coding style (make pylint, make mypy, make isort) — run on the 3 changed source files; no new messages vs main
  • make test passes, and a test case covers the change — per module only, listed below; not the full make test
  • Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible — N/A, added centrally after the wave

What is the purpose of your pull request?

  • fix — corrects a defect

Description of your pull request and other information

Closes #1170. All three _import_next_layer (ProtocolBase, Internet, IPv6) now share ProtocolBase._parse_next_layer. No captured octets gives NoPayload. Fewer than the next layer's __length_hint__ gives Raw, unless the parsed header fits anyway (the L2TPv2 hint, 16, is its longest form, and a 12-octet ZLB stays L2TPv2).

case before after
IPv6 header only (40) IPv6:UDP:Raw, rebuilds 48 IPv6, 40
IPv4 + 1 / 4 UDP octets rebuilds 28 / 28 21 / 24 (Raw)
test.pcapng frame 3 (18 IPv4 octets) Ethernet:IPv4, rebuilds 20 Ethernet:Raw, byte for byte

Frame 3 changes chain, so the #1166 and #1169 tests that pinned its zero-filled IPv4 rebuild now pin the Raw payload instead.

from_data sweep over examples/captures/ (1612 frames). IPv4 header/full mismatches go from 1/1362 to 0/1361, UDP stays 0/103, TCP 0/1259, IPv6 0 / 183 (pre-existing). No new mismatches. The only chain change is frame 3.

Tests: the new tests/protocols/test_next_layer_short_header_unit.py gives 7 passed; on main, 5 fail. Also ran ipv4 truncated (3 passed), declared-length (6), internet_unit, protocol_base, ipv4/ipv6/tcp_udp/pcapng_regression, and tests/project (389 passed, 1 skipped). Ordering legs: protocols (rest) shows the same 2F/17E before and after (all pre-existing); protocols/internet is 468, 0 failures, before and after.

…efault-filled octets

Every _import_next_layer picked NoPayload only for a declared length of
zero, so a payload captured short of the next layer's header was parsed
into zero-filled fields that from_data then rebuilt. One shared helper,
ProtocolBase._parse_next_layer, now backs the ProtocolBase, Internet and
IPv6 implementations: no captured octets gives NoPayload, and fewer than
the next layer's __length_hint__ gives Raw, unless the header it parsed
fits anyway (the L2TPv2 hint is its longest form).

test.pcapng frame 3, cut inside the IPv4 header, is now Ethernet:Raw, so
the two tests that pinned its zero-filled IPv4 rebuild are updated.

Closes #1170
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Coverage: 88.89% (unit tier, Python 3.14, 402d851c8, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18757 1034 2342 845 90.24%
pcapkit/corekit 1878 91 580 22 94.34%
pcapkit/dumpkit 142 0 42 0 100.00%
pcapkit/foundation 2442 102 852 39 94.38%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 15707 188 3974 163 98.19%
pcapkit/toolkit 487 65 144 1 85.42%
pcapkit/utilities 429 4 122 4 98.55%
pcapkit/vendor 4409 2359 1006 158 42.84%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review verdict on 402d851c8: GOOD TO GO (ran on Sonnet; author Opus). Held for the owner's decision on #1170, and labelled breaking.

  • Rebuilds match what was captured: IPv6 header-only 40 → 40; IPv4 + 1/4/7 UDP octets → 21/24/27; Ethernet + 18 IPv4 octets → 32 (it was 34 on main).
  • The .length-fits exception is sound. A 12-octet AH, a 20-octet ARP and a 12-octet L2TPv2 ZLB all stay parsed. Bogus short headers (TCP offset 0/3, AH length 0, ESP/SCTP/UDP/VLAN short) all become Raw.
  • Merged with main 748c10dce, both ordering legs pass: 838 tests and 468 tests, 0/0. The sweep over 1612 frames changes only test.pcapng frame 3. extract on http.pcap is about 2.5% slower.
  • The new test fails without the fix: 5 of 7 methods.
  • Follow-ups, not blocking:
    • IPv6_Route (hint 4, needs 8) and MH (hint 6, needs 8) understate their __length_hint__, so 4–7 octet bodies still fabricate.
    • ApplicationBase has its own _import_next_layer (application.py:132), which this PR does not route through the shared guard.
    • L2TPv2.from_data drops the L flag and the length field (c802000c… → 8802…). This also happens on main.

@JarryShaw JarryShaw added breaking Breaks public-facing behaviour or API (apply alongside the type label) review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 7, 2026
@JarryShaw
JarryShaw merged commit 8565b74 into main Oct 7, 2026
42 checks passed
@JarryShaw
JarryShaw deleted the fix/1170-short-payload-raw branch October 7, 2026 03:25
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 7, 2026
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

fix(protocols): a payload layer parsed from a short stream rebuilds default-filled octets

1 participant