Skip to content

fix(pcapng)!: keep an SHB or IDB shorter than its fixed fields as captured (#1422) - #1425

Merged
JarryShaw merged 1 commit into
mainfrom
fix/1422-pcapng-short-shb-idb
Oct 8, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/1422-pcapng-short-shb-idb

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Oct 8, 2026 •

Copy link
Copy Markdown
Owner

Closes #1422

Byte-exact rebuild (PCAPNG.from_data(PCAPNG(raw).info).data), before → after, both byte orders: SHB 16→28 ⇒ 16, 20→28 ⇒ 20, 24→28 ⇒ 24; SHB 12 raised ProtocolError ⇒ 12; IDB 12→20 ⇒ 12, 16→20 ⇒ 16. Full SHB (28) and IDB (20) unchanged.

  • SHB < 28 / IDB < 20: the body is kept as block_raw and written back by make(), as fix(pcapng)!: keep malformed journal entries and short packet blocks as captured (#1413, #1414) #1420 does for EPB/PB. Fields the block holds are decoded, the rest read as zero. Contract change: such blocks gain block_raw and no longer report fields over-read from the next block.
  • A 12-octet SHB has no room for its magic. It takes the byte order in which its Block Total Length reads as 12, the only way to read it. A bad magic on any other length still raises ProtocolError.

Tests: new test_pcapng_short_shb_idb_1422_unit.py (8 tests, 60 subtests). Without the fix, 47 subtests and the extraction test fail. Rebased onto 949b0a900, which empties that stale KNOWN_GAPS allowlist; test_bitfield_coverage_unit.py now passes on this head. tests/project: 404 passed, 1 skipped.

@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 8, 2026
…tured (#1422)

- SHB below 28 octets and IDB below 20 read their fixed fields past the
  Block Total Length and rebuilt at the full fixed size. The body is now
  kept as `block_raw`, which make() writes back at the declared length,
  as #1420 does for EPB/PB; fields the block holds are decoded, the rest
  read as zero, and options are empty.
- The schemas skip the length/length2 comparison for such blocks; the
  reader compares the real trailing copy itself.
- A 12-octet SHB has no room for its Byte-Order Magic, so it takes the
  byte order in which its Block Total Length reads as 12. A bad magic on
  any other length still raises ProtocolError.
- New regression module for #1422.
@JarryShaw
JarryShaw force-pushed the fix/1422-pcapng-short-shb-idb branch from 136d42c to 08dfbb0 Compare October 8, 2026 17:57
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) breaking Breaks public-facing behaviour or API (apply alongside the type label) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Coverage: 89.45% (unit tier, Python 3.14, 08dfbb05f, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18791 968 2352 869 90.90%
pcapkit/corekit 2038 89 634 25 94.76%
pcapkit/dumpkit 234 2 84 2 98.74%
pcapkit/foundation 2637 98 928 48 94.78%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 16615 221 4474 185 98.04%
pcapkit/toolkit 539 74 168 3 84.58%
pcapkit/utilities 429 4 122 4 98.55%
pcapkit/vendor 4409 2342 1006 157 43.25%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 08dfbb05f (Sonnet cross-review; the author ran on Opus). CI: 38 ok, 0 failed, CLEAN.

  • Byte-exact, re-derived by me with PCAPNG.from_data(PCAPNG(raw).info).data, in both byte orders:
    • PR: SHB 12/16/20/24/28 and IDB 12/16/20 all rebuild at their own length.
    • main (949b0a900): SHB 12 raises ProtocolError, SHB 16/20/24 rebuild as 28, IDB 12/16 rebuild as 20.
  • 12-octet SHB inference: 0x0000000c reads as 12 in exactly one byte order. Other trailing lengths (0, 8, 13, 0x0C000000), a mixed-order trailer, and a bad magic on 16–28 octets all still raise ProtocolError. The fallback is only reachable from the SHB post_process.
  • No over-read: in short SHB/IDB → IDB/SHB → EPB files, both byte orders, the EPB parses with interface_id 0 and captured_len 60.
  • Tests bite: with the two source files reverted to main, 48 fail. With the PR, 8 tests and 60 subtests pass.
  • Neighbours: 63 pcapng-touching modules across tests/ run one per process, plus test_bitfield_coverage_unit (3 passed). All green.
  • The ! is justified: new block_raw key, same shape as fix(pcapng)!: keep malformed journal entries and short packet blocks as captured (#1413, #1414) #1420.
  • Python 3.10: no 3.11+ API used.

Note, not blocking: a 12-octet IDB has no link type, so it reads as linktype=0, and a later EPB on that interface decodes as NULL, not Ethernet. The file carries nothing better. A 16-octet IDB's snaplen=0 is treated as "no limit"; the checks at pcapng.py:1201, 2190, 4226 and 4285 guard with if snaplen.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 8, 2026
@JarryShaw
JarryShaw merged commit a8cc025 into main Oct 8, 2026
41 checks passed
@JarryShaw
JarryShaw deleted the fix/1422-pcapng-short-shb-idb branch October 8, 2026 18:20
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 8, 2026
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

fix(pcapng): an SHB or IDB shorter than its fixed fields rebuilds at full size

1 participant