Skip to content

fix(protocol): raw items in a make() list build a packet; str enum names resolve per argument - #1352

Merged
JarryShaw merged 1 commit into
mainfrom
fix/protocol-make-1308-1312
Oct 7, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/protocol-make-1308-1312

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner
  • Searched for similar pull requests
  • Followed the coding style — I ran pylint, mypy and isort on protocol.py; they report nothing new in the changed lines
  • make test passes, and a test case covers the change — I ran only the two unittest legs, tests/project and the touched modules (below)
  • Added a changelog entry — N/A, added centrally

What is the purpose of your pull request?

  • fix — corrects a defect

Closes #1308
Closes #1312

#1312. unpack() reused the made schema, so raw bytes items reached read() and crashed with AttributeError.

  • When it re-parses: only if make() was given raw or Schema items, and the made schema still holds raw bytes where parsing yields a schema. The parsed header then replaces the made one.
  • Malformed items: if the octets do not parse, it raises ProtocolError chained from the parse error. It catches only BaseError and struct.error.
  • Which protocols: on main this crash hit 8 protocols: HIP, IPv4, TCP, HOPOPT, IPv6_Opts, MH (including a bare MH()), SCTP and IPv6_Route.
  • Fuzz: 1,500 seeded random inputs per protocol produce 0 non-library exceptions.

#1308. _make_index_namespace locates the _make_index call with ast and resolves against that call's first argument. All 111 call sites pass a plain name. 12 of them, in hip.py, pass loop variables rather than parameters (group, suite, cipher, mode, reg, tf). When the first argument is not a parameter, or the source is unavailable, it falls back to main's all-arguments check. So x = b; self._make_index(x) behaves as on main.

Behaviour change: test_make_index_str_unit pinned the ambiguous-name raise, so I updated it.

Cost: best of 3, 10,000 constructions each:

main (4849d16e4) branch
TCP() 89.4 µs 93.0 µs
IPv4(protocol=253) 109.9 µs 116.0 µs

The host load average was about 6 during the runs.

Evidence:

  • New tests: test_protocol_make_unit.py gives 25 failed, 4 passed against main's protocol.py, and 12 passed with the fix.
  • Unit legs: protocols/internet 529 tests, 0 failures, 0 errors; the rest of protocols 918 tests, 0 failures, 0 errors.
  • tests/project: 389 passed, 1 skipped.
  • Fixtures: info is identical to main on all 23 fixtures.

@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 7, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review verdict on 2a7ca1c49: NEEDS CHANGES (Sonnet reviewed; author Opus). There is one blocker, and I re-derived it myself.

1. The broad except in unpack() keeps #1312's crash for malformed raw items. When the re-parse fails, the made schema is kept with raw bytes in it, and read() then raises the original error. IPv4(options=[b'\x02', b'\x94\x04\x00\x00'], protocol=253) gives AttributeError: 'bytes' object has no attribute 'type' at ipv4.py:621. The ProtocolError: Field data resolved to a negative length that the except swallowed was the right answer. The reviewer's fuzz of 1,500 inputs per protocol ended in AttributeError for IPv4 497 times, HOPOPT 199, TCP 284, IPv6_Opts 164 and MH 27.

Fix: when the re-parse fails and raw items are present, raise a ProtocolError (chained from the parse error), and narrow the except to the library's parse errors.

2. Not blocking. _holds_raw() walks the whole schema on every made construction, which adds about 11% even with no raw items: TCP with no options goes from 93–96 µs to 105–109 µs. Short-circuit it, for example with a flag set only when raw items are supplied. Separately, a first argument that is not a parameter (x = b; self._make_index(x)) now resolves nothing, where main resolved it. All 12 such sites in hip.py pass namespace=, so nothing in-tree breaks, but the PR body's "all 111 pass a plain name" should say that 12 of them are not parameters.

Held up:

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 7, 2026
…mes resolve per argument

- #1312: unpack() read the schema make() returned, so raw bytes items
  (options, parameters, chunks, MH data) reached read() and crashed with
  AttributeError. When make() was given raw or Schema items, a made schema
  holding raw bytes where parsing its packed octets yields a schema is
  replaced by the parsed one; octets that do not parse raise ProtocolError.
  A bare MH() now constructs.
- #1308: _make_index_namespace() locates the _make_index call in the
  caller's source and resolves the name against that call's own argument,
  so two arguments holding the same member name no longer raise
  "ambiguous member name". The all-arguments check remains the fallback
  when the source is unavailable or the first argument is not a parameter.

test_make_index_str_unit pinned the old ambiguous-name raise and is
updated; new tests in tests/protocols/test_protocol_make_unit.py.
@JarryShaw
JarryShaw force-pushed the fix/protocol-make-1308-1312 branch from 2a7ca1c to f345fe8 Compare October 7, 2026 20:45
@JarryShaw

Copy link
Copy Markdown
Owner Author

Round-2 verdict on f345fe84e: GOOD TO GO. I re-ran my round-1 probe on this head myself. The head is rebased onto 4849d16e4.

  • Blocker fixed. IPv4(options=[b'\x02', b'\x94\x04\x00\x00'], protocol=253) now raises ProtocolError: IPv4: malformed raw item given to make(), chained from the parse error. It no longer raises AttributeError. The except is narrowed to BaseError and struct.error. A seeded fuzz of 1,500 inputs across all 8 protocols gives 0 non-library exceptions, where the old code gave 323 AttributeErrors.
  • Valid raw items still build. IPv4 Router Alert gives 24 B, a TCP MSS gives 24 B, and a bare MH() gives 8 B.
  • Hot path. The schema walk now runs only when make() was given bytes or a Schema. The cost over main falls from about 11% to about 4–6%: 89.4 µs to 93.0 µs for TCP(), and 109.9 µs to 116.0 µs for IPv4(protocol=253), both measured at load ≈6.
  • Loop-variable regression fixed. It now falls back to the all-arguments check, so x = b; self._make_index(x) behaves as it does on main. The 12 hip.py sites are named in the body.
  • Tests and checks. The previous head fails 8 of the new tests. The legs give 529/0 and 918/0. Fixture info is identical to main.

One residual gap, stated in the PR body. A tuple-form option whose maker returns raw bytes would not set the flag. No such maker is known in-tree.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Coverage: 89.31% (unit tier, Python 3.14, f345fe84e, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18777 966 2348 869 90.90%
pcapkit/corekit 1949 83 606 19 94.99%
pcapkit/dumpkit 189 1 68 1 99.22%
pcapkit/foundation 2443 102 852 39 94.39%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 16076 187 4216 158 98.26%
pcapkit/toolkit 539 74 168 3 84.58%
pcapkit/utilities 429 4 122 4 98.55%
pcapkit/vendor 4409 2359 1006 158 42.84%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 66950b1 into main Oct 7, 2026
37 checks passed
@JarryShaw
JarryShaw deleted the fix/protocol-make-1308-1312 branch October 7, 2026 23:02
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 7, 2026
JarryShaw added a commit that referenced this pull request Oct 8, 2026
…o longer raises the wrapped error (#1395)

#1344 pads the IPv4 option area once at the end, so the schema now
accepts octets read() then rejects (an option whose length runs past the
area), and the reader's bare ProtocolError escaped the wrap #1352 put
around the schema re-parse only. unpack() now wraps the read as well when
the made header holds raw items, and the wrapped message carries the
original one. The same escape is fixed for TCP, HOPOPT, IPv6-Opts, MH,
SCTP and HIP.

Closes #1394
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

1 participant