Skip to content

refactor(foundation): route DTLS reassembly via toolkit and analysis (#2496) - #2528

Merged
JarryShaw merged 2 commits into
mainfrom
refactor/2496-dtls-toolkit-routing
Nov 1, 2026
Merged

JarryShaw merged 2 commits into
mainfrom
refactor/2496-dtls-toolkit-routing

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else
  • const — regenerates constant or registry tables, members keeping their numeric values

Description of your pull request and other information

Closes #2496 (points 1 and 3 of reassembly-integration.rst).

  • DTLS.read() no longer takes reassembly/flow/frame_num/frame_timestamp and imports nothing from pcapkit.foundation.
  • New pcapkit.toolkit.pcap.dtls_reassembly(frame) and pcapkit.toolkit.pcapng.dtls_reassembly(frame), following the tcp_segment precedent. The buffer ID is the IP+UDP endpoints plus the epoch.
  • New Extractor(dtls=...) and extract(dtls=...), plus a 'dtls' entry in Extractor.__reassembly__, fed by the PCAP and PCAP-NG engines. The other engines turn it off with a warning.
  • New pcapkit.foundation.analysis.dtls_handshake parses the reassembled body.
  • The feat(foundation): reassemble fragmented DTLS handshake messages #2451 tests pass, plus a new module covering extraction over PCAP and PCAP-NG with out-of-order and interleaved fragments.

Open questions are below; the Sonnet review is in flight.

…2496)

- DTLS.read drops its reassembly/flow/frame arguments and no longer
  imports pcapkit.foundation.reassembly; fragment bodies stay raw
- add toolkit.pcap/pcapng.dtls_reassembly, which read each frame's
  fragmented handshake messages keyed on the nearest IP/UDP endpoints
- add Extractor/extract(dtls=...), ReassemblyManager/ReassemblyData.dtls;
  the default engines feed it, the other engines disable it with a warning
- add pcapkit.foundation.analysis.dtls_handshake to decode a reassembled
  message as an unfragmented one
- SSL._make_tls_handshake_messages drops the slice-back the in-place
  decode needed
- docs: analysis page, toolkit/engine tables, pep.rst, convention page

Tests: DTLS reassembly, toolkit, engine and extraction modules pass.
@JarryShaw JarryShaw added refactor Restructuring for its own sake — neither a fix nor a new capability (refactor: prefix) needs: decision Waiting on the maintainer to decide — not blocked by other work review: running A cross-review is in flight against the current head - no verdict yet labels Nov 1, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Decisions needed:

  1. Oversized messages. dtls_handshake decodes through a synthetic cleartext record, so a message over the record's 16-bit length stays raw. Options: (a) accept that; (b) add a public message-level decoder on SSL/DTLS, which rule 1 allows. Recommendation: (b), as a follow-up.
  2. foundation.analysis is a single module for now. Options: keep it, or make it a package that mirrors foundation/reassembly. Recommendation: keep it until a second user exists.
  3. The dtls= flag sits after tcp=, which moves buffer_size and every later positional parameter by one. That is breaking for positional callers. Options: (a) keep it grouped; (b) append it after context, so nothing moves. Recommendation: (b).
  4. IP/TCP parity (a deferred Datagram.packet, and __protocol_type__ on the DTLS reassembly class). Recommendation: a follow-up issue.
  5. SSL._make_tls_handshake_messages drops feat(foundation): reassemble fragmented DTLS handshake messages #2451's slice-back for bodies longer than fragment_length. It never shipped, and its only producer is removed. Recommendation: accept.

@JarryShaw JarryShaw added the review: running A cross-review is in flight against the current head - no verdict yet label Nov 1, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author
  1. (b)
  2. split it, mirrors reassembly
  3. (a) - the feat class is not shipped in releases yet. not real breaking changes.
  4. follow-up issue
  5. check RFC for specs. if none, accept.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Owner rulings, paraphrased and reconciled:

  1. (b): add a public message-level decoder on SSL/DTLS, so a message over a record's 16-bit length still decodes. It goes in this PR, and dtls_handshake uses it.
  2. Split foundation.analysis into a package mirroring foundation/reassembly (analysis/dtls.py), against my recommendation to keep a single module.
  3. (a): keep dtls= grouped after tcp=. The owner does not count the positional shift as a real breaking change, because the feature is not in a release yet, so no breaking label.
  4. Follow-up issue: refactor(reassembly): bring DTLS reassembly to IP/TCP parity (deferred packet, __protocol_type__) #2531, the IP/TCP parity items.
  5. SSL._make_tls_handshake_messages: check RFC 6347 and RFC 9147 for whether a fragment's body must equal fragment_length. If they specify it, keep the slice-back (or validate the length); if not, accept the removal.

These go into one fix round on this branch once the Sonnet review in flight lands, so that its findings join the same commit. The fix round is pushed once queued runs drop below 20.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review (Sonnet) at baf1cddf5: NEEDS CHANGES. The design is sound: DTLS.read() imports nothing from pcapkit.foundation, the toolkit wiring follows IP/TCP, and the analysis function reproduces the old decoded bodies exactly. The commit breaks two pinned tests:

  • tests/toolkit/test_pcap_tcp_segment_public_1570_unit.py pins pcap.__all__;
  • the RFC-constant sweep flags _MAX_RECORD_FRAGMENT = 0xFFFF.

foundation.analysis.dtls_handshake is also not exported, and the convention page states a count. The fix round now running covers these together with the owner's rulings 1, 2 and 5. One review note on ruling 3: with dtls= after tcp=, a positional buffer_size integer from an existing caller lands in dtls and reads as truthy. It stays as ruled; say so if it should move after context instead.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Nov 1, 2026
…2496)

- add DTLS.read_handshake, a public decoder for one handshake message
  outside any record; dtls_handshake uses it, so a message past a
  record's 16-bit length decodes, and _MAX_RECORD_FRAGMENT goes
- split pcapkit.foundation.analysis into a package mirroring reassembly
  (analysis/dtls.py), exported from pcapkit.foundation and pcapkit.all
- SSL._make_tls_handshake_messages cuts a whole-message fragment body to
  the fragment's range again, and refuses any body that does not yield
  exactly fragment_length octets
- docs: analysis and DTLS pages, convention page without counts or timed
  wording, references to the defining module
- tests: pin dtls_reassembly in toolkit.pcap.__all__; cover the decoder,
  the fragment-body rule and the exports
@JarryShaw

Copy link
Copy Markdown
Owner Author

Fix round at 26c5110c0, cross-reviewed (Sonnet): GOOD TO GO.

  • Ruling 1: a public DTLS.read_handshake now decodes reassembled messages. A 672-datagram differential against the old path differs only on the two messages over 16 bits, which now decode.
  • Ruling 2: foundation.analysis is a package mirroring foundation/reassembly.
  • Extractor: output for fragmented DTLS over pcap and pcapng is byte-identical.
  • Tests: coverage of the changed lines is full. The tests and fixes from the review all pass.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Nov 1, 2026
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Coverage: 93.96% (unit tier, Python 3.15, 26c5110c0, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 20087 144 2710 125 98.78%
pcapkit/corekit 2605 74 818 37 96.52%
pcapkit/dumpkit 281 3 100 3 98.43%
pcapkit/foundation 3243 27 1136 32 98.65%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 19879 209 5222 204 98.35%
pcapkit/toolkit 953 36 342 10 95.68%
pcapkit/utilities 502 7 144 5 98.14%
pcapkit/vendor 5009 2222 1148 182 52.31%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit b2bd2f8 into main Nov 1, 2026
63 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Nov 1, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Nov 1, 2026
JarryShaw added a commit that referenced this pull request Nov 1, 2026
#2490, #2575)

#2541's exception table was built on a base without #2528 and #2542, so
main fails the member-order guard: #2528 put DTLS back in order, and #2542
renamed the IGMP classes to IGMPv3. Drop DTLS and re-key the IGMP entries.

#2509 merged before #2521 required a wrapper for every class-level map, so
Radiotap.__proto__ has none. Nothing dispatches by it yet; excuse it until
#2575 adds Radiotap.register and register_radiotap.
@JarryShaw

Copy link
Copy Markdown
Owner Author

I've added the breaking label after the merge. The new dtls parameter sits between tcp and buffer_size in Extractor.__init__ and extract(). Those parameters are positional-or-keyword in v1.5.1, so a caller passing buffer_size or anything after it positionally now gets a different meaning. ReassemblyManager, which is in v1.5.1's __all__, also gains a required dtls argument. The central changelog (#2483) marks this change breaking to match.

@JarryShaw JarryShaw added the breaking Breaks public-facing behaviour or API (apply alongside the type label) label Nov 1, 2026
JarryShaw added a commit that referenced this pull request Nov 1, 2026
- Added: one's-complement checksum verification and computation (#2543),
  and the IPX and SCTP registration wrappers (#2521).
- Folded into unreleased entries: LinuxSLL/FRAD/Netlink registries (#2527),
  the IGMP class names (#2542), TLS record chaining and the SSLv2 trial's
  warnings (#2537), DTLS reassembly via Extractor and toolkit (#2528), and
  IP_Pseudo's walk at any depth (#2543). ICMP drops its stale checksum line.
- Breaking: make() checksums, DRARP's module (#2540), IPv6 overlapping
  fragments (#2569), TCP_GAP_LIMIT replacing TCP.__window__ (#2579), and
  Extractor/extract's positional dtls (#2528).
- Changed: member order (#2490), conventions (#2542, #2521, #2570), CI -n
  logical (#2572). Fixed: AppType.get (#2568), trial warnings (#2519, #2577).
- CHANGELOG.md regenerated; util/changelog_md.py --check passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) refactor Restructuring for its own sake — neither a fix nor a new capability (refactor: prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Route DTLS reassembly through a toolkit helper, like IP/TCP

1 participant