Repository navigation
fix(ftp,httpv1): keep the raw command case, HTTP/1 whitespace and field octets, and parse whole multi-line FTP replies - #1347
Conversation
|
Cross-review verdict on 1.
The reviewer also got 2. #1303 never checks the RFC 959 closing line. Per §4.2, a multi-line reply ends at a line with the same code followed by SP. Here everything after a
Accept the segment only when it ends at the first Not blocking: raw lines are matched to headers by position, so removing or inserting one header normalises every unedited header after it. Matching by content would keep them exact. Held up:
|
…ld octets, and parse whole multi-line FTP replies - FTP requests keep the command token as received (`raw_cmmd`); `make` writes it whenever it differs from `cmmd` only in case (#1333). - A multi-line FTP reply arriving whole in one segment parses when it ends at its first `<code> SP` line; the lines after the first are kept verbatim in `lines` and rebuilt (#1303). - HTTP/1 keeps the start line (`receipt.raw_line`) and each field line, obs-folds included (`raw_header`); `make` writes them whenever they read back as the given values, matching field lines by content, so whitespace and non-UTF-8 field values rebuild byte-exactly (#1302, #1333). A raw line carrying a CR or LF its values lack raises `ProtocolError`. Parsed values are unchanged. Closes #1302 Closes #1303 Closes #1333
d104be9 to
5017fb9
Compare
|
Round-2 verdict on
|
|
Coverage: 88.98% (unit tier, Python 3.14,
Per-file detail: the |
make pylint,make mypy,make isort): ran on the four changedpcapkit/files. mypy and isort are clean. pylint reports only the 4 warningsmainalready has.make testpasses, and a test case covers the change: ran the new module, 10 FTP/HTTP modules,tests/projectand theprotocols --exclude protocols/internetordering leg (910 tests, 0 failures)docs/source/changelog/and regeneratedCHANGELOG.md, if the change is user-visible — N/A, added centrallyWhat is the purpose of your pull request?
fix— corrects a defectDescription of your pull request and other information
Per the #1333 ruling, the original token or line is kept next to the normalised value, and
makewrites it whenever it still reads back as that value. A value that has been edited is built from the edit.Request.raw_cmmd, soquit\r\nrebuilds as itself whilecmmdstaysCommand.QUIT. A whole multi-line reply in one segment now parses (fix(ftp): a complete multi-line reply in one segment does not parse #1303), but only if it ends at its first<code> SPline (RFC 959 §4). A missing closing line, a different code, or anything after the closing line still raisesProtocolError. The lines after the first are kept verbatim in the newResponse.lines.receipt.raw_lineandraw_header, with one field line per header and obs-folds kept. This makes the fix(httpv1): non-UTF-8 octets in HTTP/1 header values are re-encoded as UTF-8 on rebuild #1302 latin-1 field value and all four fix(ftp,httpv1): command case and HTTP/1 whitespace are normalised on rebuild #1333 whitespace cases byte-exact.headersby content, in order, so adding or removing a header leaves the others as received.ProtocolError. The CRLF of an obs-fold is the only exception.Before → after, via
from_data(info).data:quit→QUIT;220-a\r\n220 b→ProtocolError;X-Name: caf\xe9→caf\xc3\xa9;200 \tOK//a\t//a\x0c→ stripped;Host:a→Host: a;X: a\r\n b→X: a b. All 11 probes now rebuild byte-exactly.infois unchanged apart from the new keys:examples/captures/*(frames and reassembled streams), and 189 bytes literals from the application tests, parsed as FTP and as HTTP/1.origin/mainwith the new keys dropped, every layer is identical.220-a\r\n220 b\r\n, which now parses where it used to raise.Fails without the fix: the new module gives 12 passed / 48 subtests with the fix. Against
mainthe first revision failed 31 / passed 5. Against that revision (d104be9cf) the new review tests fail 16 subtests: the 4 injection shapes, 3raw_lineshapes, 4 unclosed or over-long replies, 2 content-matching cases and 3 badlinesvalues.Rebuilds: all 471 capture layers rebuild byte-exactly, and none raise.
Contract changes:
Request.raw_cmmd,Response.lines,RequestHeader/ResponseHeader.raw_lineandHTTP.raw_header.make()kwargs with defaults.FTP.makeraisesProtocolErrorwhenlinesare not a closing tail of bytes for amorereply.HTTP.makeraisesProtocolErroron the CR/LF case above._make_datadict, so I added the new keys to them intest_ftp_unit.pyandtest_http_unit.py.Closes #1302
Closes #1303
Closes #1333