[JarryShaw]

• Searched for similar pull requests

• Followed the coding style. I ran pylint and mypy on reassembly/tcp.py: mypy shows no new errors, and pylint adds only too-many-branches (17/12) on TCP.reassembly. I did not run isort project-wide.

• make test passes, and a test case covers the change. The modules are listed below.

• Changelog entry. N/A, added centrally after the wave.

• fix — corrects a defect

---

Closes #1354
Closes #1355
Closes #1356

• fix(reassembly): TCP reports a zero-filled gap as COMPLETE when a segment arrives below the first one seen #1354: submit judges each ACK bucket from its own Fragment.gap instead of the shared hdl.

  • Any gap entry makes the bucket PARTIAL.
  • The RFC 815 update now fills every hole a segment spans.
  • Without a SYN, hdl gets a -sys.maxsize sentinel hole below the first segment, so a later segment below it leaves a real hole.

• fix(reassembly): TCP has no modulo-2**32 sequence arithmetic, so a wrapped sequence number allocates about 4 GiB #1355: sequence numbers are unwrapped mod 2³² relative to the end of the bucket they continue, following RFC 9293 §3.4. Datagram.conflict is reported mod 2³².

• fix(reassembly): TCP submits the buffer on FIN/RST, so data reordered behind the FIN is split off #1356: a FIN cuts hdl back to the FIN's sequence number. The buffer is submitted once no hole is left, or else at end of capture.

  • RST still submits at once. RFC 9293 §3.10.7.4 flushes the receiver's queues and enters CLOSED.

• Bound: TCP.__window__, 16 MiB, overridable by subclassing.

  • What it limits: a merge may not zero-fill a gap wider than this. The gap is measured on the side where the segment lands: from the bucket's end to the segment's start, or from the segment's end to the bucket's start.
  • Why a fixed cap: Packet carries no advertised window, so that cannot be used. RFC 7323's 2³⁰ maximum would still allow a 1 GiB pad. Linux's default tcp_rmem ceiling is 6 MiB, so 16 MiB leaves plenty of room above real windows.
  • What happens instead of padding: the old buffer is submitted and a new one is opened.
    • No SYN for that buffer: the segment starts a new stream. This covers a reused 4-tuple, where N*5 at 0x30000000 previously allocated 768 MiB.
    • SYN captured: this is the same stream with data lost in between. The lost range goes into the gap list of the old bucket and of the new one, so both are PARTIAL and neither is COMPLETE. The new buffer keeps the SYN header and still counts as having seen the SYN.
    • I record the loss in Fragment.gap, because that is already where completion is decided, so no data-class field is needed.
    • A gap entry can now lie outside raw, so a merge also removes the newly received range from that entry.
  • Tests, all with bytearray hooked:
    • the reused 4-tuple with no SYN;
    • segments about 2³¹ away;
    • a far segment below the buffer;
    • a SYN stream with a 20 MiB hole, after it and below it, in strict and non-strict mode;
    • the missing range shrinking as data arrives.

  The three SYN tests fail on the previous head with a false COMPLETE.

• Limitation, documented in the TCP docstring: this applies only to a buffer begun without a SYN. A FIN counts the stream as complete from the lowest octet seen. Data that arrives later, below everything seen before that FIN, therefore opens a new buffer, and the first buffer has already been reported COMPLETE although it is not. Example: [1000 A*5, FIN 1005, 990 B*10] gives B*10 and A*5.

• Repro: /tmp/rtissues/foundation-repro.py check_1 to check_3 print REPRO ×3 on main and NOREPRO ×3 here.

• Capture regression: I dumped every datagram (completion, ids, index, payload sha1, conflicts) for 23 captures × {default, dpkt} × strict on/off. All 92 keys are identical to main, and 28 of them are non-empty (7 captures). My earlier figure of 184 also counted scapy and pyshark, but their output matched default key for key, so they are not independent runs.

• Tests:

  • New module: 16 passed.
  • test_tcp.py: 26 passed. Its submit test seeded holes into hdl, which pinned the old mechanism, so they now go into Fragment.gap.
  • Every other module under tests/foundation/reassembly/, plus tests/integration/test_reassembly_*, traceflow/test_tcp.py and interface/test_core.py: all pass.
  • tests/project: 389 passed.
  • run_unittest_leg.py foundation under a file-size cap: 499 tests. The only failures are the 3 already on main (fix(dumpkit): JSON output of DPKT/Scapy frames grows without bound and fills the disk #1372, Errno 27).

• Revert proof against main: with main's tcp.py written over the source, the original 9 tests gave 11 failed, 3 passed. Only test_rst_aborts_at_once passes on both.

• No breaking change: the data-class signatures are unchanged. Only the private _update_hole_descriptors changed signature.

[JarryShaw]

Cross-review verdict on f2243517b: NEEDS CHANGES (Sonnet reviewed; author Opus). All three fixes work — ~5,000 fuzz cases with 0 wrong COMPLETE/PARTIAL (the same oracle fails 201/300 on main), and 0 capture differences — but two things need addressing.

1. A FIN with an open hole now keeps the buffer, and later data on the reused 4-tuple merges into it. Probe: SYN, A*5, FIN at 11, then N*5 at seq 0x30000000 with no SYN. main gives two datagrams; here the N segment joins the old buffer and allocates ~768 MiB of zero pad (bytearray(805306357)). Rare — it needs a lost SYN, a lost segment and a reused 4-tuple — but it is new, unbounded allocation. Bound it: treat a segment far outside the buffer's window (e.g. beyond the advertised window, or a configurable cap) as a new stream rather than padding to it.
2. Document the known limit. With no SYN, data arriving later but below everything seen before a covering FIN opens a new buffer, and the first is reported falsely COMPLETE ([1000 A*5, FIN 1005, 990 B*10] → B*10 and A*5). The PR mentions it nowhere — not tcp.py, data/tcp.py or the body.

Held up: wraps at 2³²−1, within and across buckets, and straddling retransmissions; conflicts reported mod 2³²; RST submits at once per RFC 9293 §3.10.7.4; the revert proof gives 11 failed / 3 passed. Nit: a segment ~2³¹ away still allocates ~2 GiB, which is no worse than main, but the same bound would fix it.

[JarryShaw]

Round-2 verdict on b6bb34edb: NEEDS CHANGES (Sonnet reviewed; author Opus). The round-1 blocker is fixed: the reused-4-tuple probe now gives N*5 COMPLETE and A*5 PARTIAL, with a 5-byte allocation. But the new bound creates the same class of defect #1354 is about.

The 16 MiB bound also splits a stream whose SYN was captured, and reports both halves COMPLETE. After a SYN, A*1000 and then B*1000 20 MiB later yields two COMPLETE datagrams, A and B. The loss is invisible, although the SYN proves this is one stream with a hole. main gave one PARTIAL datagram. The code never checks for a SYN. The __window__ comment says the split is for "a reused 4-tuple whose SYN was not captured", and the Warning: block says a capture with the SYN is unaffected, so both promise a check that isn't there.

Restrict the split to buffers with no SYN seen. Where a split is unavoidable, report both parts PARTIAL, never COMPLETE. Add a test for the SYN-captured 20 MiB hole.

Held up:

• Boundaries: W−1 and W pad, W+1 splits. Wrap-crossing and in-window retransmissions merge. A stale old-ACK segment 48 MiB past its bucket no longer pads.
• Long transfers: 40 MiB in-order transfers stay one datagram.
• Docstring: the example reproduces exactly.
• Captures: 92 capture keys (23 × default/dpkt × strict) are identical to main. The body's "184 keys" did not reproduce; please state what it counts.
• Tests: the reassembly, traceflow and integration modules all pass.

Nit: the backward check compares the segment's end, so it is 5 bytes asymmetric with the forward one.

[JarryShaw]

Round-3 verdict on c5bbadf97: GOOD TO GO (Sonnet reviewed; author Opus). This resolves round 2's finding that a SYN-captured split was reported as COMPLETE.

• The SYN split. SYN, A*1000, then B*1000 20 MiB later now gives two PARTIAL halves in both strict and non-strict mode. Each half keeps the SYN header, and the hooked bytearray never allocates more than 1 MiB.
• Refilling the hole in 1 MiB chunks gives COMPLETE whether the chunks arrive in order, in reverse or shuffled. No stale gap entry is left behind. The largest allocation is 11.5 MB, inside the 16 MiB window.
• Bookkeeping. These all leave correct gap lists:
  • four splits in a row;
  • a retransmission overlapping the split boundary;
  • a base sequence crossing 2³²;
  • FIN or RST right after a split.
• Fuzzing. A new 17,000-case fuzz (windows 16, 64 and 200) found 0 exceptions, 0 wrong COMPLETE runs and no gap entry over real bytes. The round-1 oracle found 0 wrong in 5,000 cases.
• Unchanged behaviour. The no-SYN reused-4-tuple path is unchanged. The 92 capture keys are identical to main.
• Revert proof: 4 tests fail. Legs: reassembly 95, traceflow 21, integration 15 + 5, all passing. I re-ran test_tcp_reassembly_roundtrip_unit myself: 16/0.

Nits, not blocking:

• A half emitted at split time stays PARTIAL even if the hole is later refilled within the window.
• A reach-back prepend can leave a duplicate gap entry; it is harmless, but the Fragment.gap docstring doesn't mention it.
• A FIN far past the buffer end submits a bucket with no data.
• In-window loss between the SYN and the first data is still reported COMPLETE. That also happens on main, and the Warning: block slightly overstates this case.

[github-actions]

Coverage: 88.99% (unit tier, Python 3.14, c5bbadf97, Unit Tests run success)

Package	Statements	Missed	Branches	Partial	Cover
pcapkit (top level)	104	4	20	4	93.55%
pcapkit/const	18777	1032	2348	843	90.27%
pcapkit/corekit	1949	84	606	20	94.91%
pcapkit/dumpkit	189	1	68	1	99.22%
pcapkit/foundation	2495	103	876	40	94.45%
pcapkit/interface	112	7	40	5	92.11%
pcapkit/protocols	16013	206	4178	177	98.06%
pcapkit/toolkit	539	74	168	3	84.58%
pcapkit/utilities	429	4	122	4	98.55%
pcapkit/vendor	4409	2359	1006	158	42.84%

Per-file detail: the coverage-html artifact of this run.