Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions tests/protocols/application/test_ftp_http1_raw_unit.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# -*- coding: utf-8 -*-
"""FTP and HTTP/1.* keep what they normalise. C.f. #1302, #1303, #1333.

- #1302: HTTP/1.* field values decoded with a guessed charset were encoded back
as UTF-8. The field lines are now kept as received (``raw_header``).
- #1303: a multi-line FTP reply arriving whole in one segment did not parse. The
lines after the first are now kept as received (``lines``).
- #1302: HTTP/1.* field values decoded with a guessed charset must not be
encoded back as UTF-8. The field lines are kept as received (``raw_header``).
- #1303: a multi-line FTP reply arriving whole in one segment parses. The lines
after the first are kept as received (``lines``).
- #1333: the FTP command case and the HTTP/1.* whitespace -- around start-line
tokens, around the field colon, and in ``obs-fold`` -- were normalised on
rebuild. The parsed values stay normalised for lookup, and the original
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
one.
- #1241: text decoded with a detected charset was encoded back as UTF-8, so
non-UTF-8 octets in FTP arguments and HTTP/1.* start lines changed on rebuild.
The charset is now recorded on the data whenever UTF-8 would not reproduce the
The charset is recorded on the data whenever UTF-8 would not reproduce the
octets.

"""
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
# -*- coding: utf-8 -*-
"""FTP reply codes and HTTP status codes parse and rebuild for every three digits.

GitHub issue #1443: :class:`~pcapkit.const.ftp.return_code.ReturnCode` resolved
only 100 to 659 and :class:`~pcapkit.const.http.status_code.StatusCode` only
100 to 599, so a reply such as ``999 hi`` or a status line such as
``HTTP/1.1 999 X`` raised :exc:`~pcapkit.utilities.exceptions.EnumValueError`
and could not be parsed at all. Both wire fields are three digits, so every
value from 000 to 999 now resolves, to ``Unassigned`` inside the registry's
range and ``Unknown`` outside it, and the packet rebuilds byte for byte.
GitHub issue #1443: guards against
:class:`~pcapkit.const.ftp.return_code.ReturnCode` resolving only 100 to 659 and
:class:`~pcapkit.const.http.status_code.StatusCode` only 100 to 599, which made a
reply such as ``999 hi`` or a status line such as ``HTTP/1.1 999 X`` raise
:exc:`~pcapkit.utilities.exceptions.EnumValueError` and fail to parse at all.
Both wire fields are three digits, so every value from 000 to 999 resolves, to
``Unassigned`` inside the registry's range and ``Unknown`` outside it, and the
packet rebuilds byte for byte.
``make()`` also writes the code as three digits, so ``099`` keeps its zero.

Every case builds its own octets in memory and reads no capture. Classes are
Expand Down
23 changes: 12 additions & 11 deletions tests/protocols/application/test_ftp_unit.py
Original file line number Diff line number Diff line change
Expand Up @@ -116,20 +116,20 @@ def test_command_get_is_case_insensitive(self) -> None:
# form raised too.
self.assertIs(Command('retr'), Command.RETR)

# GitHub issue #860: a genuinely unknown command no longer registers
# GitHub issue #860: a genuinely unknown command does not register
# at all -- the owner's ruling is that ``get``/``_missing_`` have no
# way to supply ``feat``/``desc``/``type``/``conf``, so minting one
# would register a permanently hollowed-out member; only
# ``register()`` can do that properly. The *value* keeps the
# caller's own casing (unchanged from before #860, and the same
# convention :class:`~pcapkit.const.ftp.command.FEATCode` already
# used) -- only the *name* is canonicalised -- so a repeated call
# with the same casing is *equal* but never *identical* (nothing is
# cached to be identical to any more), while a different-case call
# is a genuinely different value and is correctly *not* equal: on
# ``main`` minting's cache silently returned the first casing seen
# for every later call regardless of case, which is exactly the
# "registered enum out of an unrecognised value" #860 removes.
# caller's own casing (the same convention
# :class:`~pcapkit.const.ftp.command.FEATCode` uses) -- only the
# *name* is canonicalised -- so a repeated call with the same casing
# is *equal* but never *identical* (nothing is cached to be
# identical to), while a different-case call is a genuinely
# different value and is correctly *not* equal. A minting cache
# would return the first casing seen for every later call
# regardless of case, which is exactly the "registered enum out of
# an unrecognised value" #860 rules out.
unknown = Command.get('xyzw')
self.assertEqual(unknown._name_, 'XYZW')
self.assertEqual(unknown, 'xyzw')
Expand All @@ -146,7 +146,8 @@ def test_command_get_is_case_insensitive(self) -> None:
def test_ftp_read_parses_a_lowercase_request(self) -> None:
"""The case-insensitivity is reachable from wire data: ``ftp.py``
compiles ``FTP_REQUEST`` with :data:`re.I` and passes the match
verbatim, so a lowercase request used to crash the parse -- #582.
verbatim, so the lookup must accept a lowercase request rather than
crash the parse -- #582.
"""
from pcapkit.const.ftp.command import Command
from pcapkit.protocols.application.ftp import FTP, Type
Expand Down
25 changes: 13 additions & 12 deletions tests/protocols/application/test_http_from_data_1154_unit.py
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
# -*- coding: utf-8 -*-
"""``HTTP.from_data`` rebuilds the message it was parsed from. C.f. #1154.

:meth:`HTTP._make_data <pcapkit.protocols.application.http.HTTP._make_data>`
chose the versioned class from a ``version`` key that neither HTTP/1.* nor
HTTP/2 data carries, so every ``HTTP.from_data(http.info)`` raised
``ProtocolError: invalid HTTP version: 0``. It now dispatches on the class of
the data, and passes the ``version`` that :meth:`HTTP.make
<pcapkit.protocols.application.http.HTTP.make>` dispatches on.
Guards against :meth:`HTTP._make_data
<pcapkit.protocols.application.http.HTTP._make_data>` choosing the versioned
class from a ``version`` key that neither HTTP/1.* nor HTTP/2 data carries,
which made every ``HTTP.from_data(http.info)`` raise ``ProtocolError: invalid
HTTP version: 0``. It dispatches on the class of the data, and passes the
``version`` that :meth:`HTTP.make <pcapkit.protocols.application.http.HTTP.make>`
dispatches on.

Behind that, :meth:`httpv2.HTTP._make_data
<pcapkit.protocols.application.httpv2.HTTP._make_data>` returned the frame's
``length``, which ``ProtocolBase.__init__`` takes as the number of octets to
parse back -- the payload alone, nine short of the frame -- so every HTTP/2
``from_data`` raised as well. ``make`` computes the Length itself, so the key
is no longer returned.
<pcapkit.protocols.application.httpv2.HTTP._make_data>` must not return the
frame's ``length``, which ``ProtocolBase.__init__`` takes as the number of
octets to parse back -- the payload alone, nine short of the frame -- so every
HTTP/2 ``from_data`` would raise as well. ``make`` computes the Length itself,
so the key is not returned.

The frames are the Scapy-built ones of
``test_http_guess_http2_1141_unit.LEGAL_FRAMES``, embedded as bytes.
Expand Down Expand Up @@ -93,7 +94,7 @@ def test_dispatcher_rebuilds_http2_frames(self) -> None:
self._round_trip(HTTP, bytes.fromhex(octets))

def test_httpv2_rebuilds_its_own_frames(self) -> None:
"""``httpv2.HTTP.from_data`` no longer parses back only Length octets."""
"""``httpv2.HTTP.from_data`` parses back the whole frame, not only Length octets."""
from pcapkit.protocols.application.httpv2 import HTTP as HTTPv2

for name, octets in HTTP2_FRAMES:
Expand Down
25 changes: 13 additions & 12 deletions tests/protocols/application/test_http_guess_http2_1141_unit.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
# -*- coding: utf-8 -*-
"""HTTP's version guess takes a bare frame for HTTP/2 only if a sender could write it. C.f. #1141.

Since #1138 an HTTP/2 frame's Length counts the payload alone
(:rfc:`9113#section-4.1`), so any nine octets whose Length fits the buffer are
self-consistent. :meth:`HTTP._guess_version
An HTTP/2 frame's Length counts the payload alone (:rfc:`9113#section-4.1`,
#1138), so any nine octets whose Length fits the buffer are self-consistent.
:meth:`HTTP._guess_version
<pcapkit.protocols.application.http.HTTP._guess_version>` falls through to an
HTTP/2 trial parse when neither the preface nor an HTTP/1 start line matches, and
that trial accepted most zero-heavy binary: nine zero octets came back as
``HTTP/2 type=0``, a ``DATA`` frame on stream 0, which :rfc:`9113#section-6.1`
makes a protocol error.
HTTP/2 trial parse when neither the preface nor an HTTP/1 start line matches. A
trial that accepts any self-consistent header would take most zero-heavy binary:
nine zero octets would come back as ``HTTP/2 type=0``, a ``DATA`` frame on stream
0, which :rfc:`9113#section-6.1` makes a protocol error.

The guess now requires a frame type :rfc:`9113` defines, no flag that type
The guess requires a frame type :rfc:`9113` defines, no flag that type
leaves undefined, a clear reserved bit, and a stream that type may be sent on.
Explicit ``version=2`` parsing is not affected, and is asserted unchanged here.
Explicit ``version=2`` parsing is not subject to the guess, as asserted here too.

The legal frames in :data:`LEGAL_FRAMES` were built with Scapy 2.7.0's
:mod:`scapy.contrib.http2` (``H2Frame`` over each frame class) and are embedded
Expand Down Expand Up @@ -70,8 +70,9 @@
('ORIGIN, not an RFC 9113 type', '0000000c0000000000'),
)

#: The seeded fuzz. Pre-fix, this corpus produced 566 HTTP/2 labels, of which
#: 249 were ``DATA`` (most on stream 0) and 317 frames of unknown type.
#: The seeded fuzz. Measured without the guess's checks, this corpus produced
#: 566 HTTP/2 labels, of which 249 were ``DATA`` (most on stream 0) and 317
#: frames of unknown type.
FUZZ_SEED = 1141
FUZZ_COUNT = 5000
PRE_FIX_HTTP2_LABELS = 566
Expand Down Expand Up @@ -111,7 +112,7 @@ def test_nine_zero_octets_are_not_guessed_as_http2(self) -> None:
self._guess(bytes(9))

def test_explicit_version_still_parses_nine_zero_octets(self) -> None:
"""Explicit HTTP/2 parsing stays RFC-exact: only the guess is tightened."""
"""Explicit HTTP/2 parsing is RFC-exact: the checks apply to the guess only."""
from pcapkit.protocols.application.http import HTTP

http = HTTP(bytes(9), 9, version=2)
Expand Down
11 changes: 6 additions & 5 deletions tests/protocols/application/test_http_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,13 +104,14 @@ def test_malformed_http_payload_falls_back_to_raw(self) -> None:
self.assertEqual(tcp.payload.name, 'Unknown')

# The registry key that selected the protocol whose parse then failed --
# TCP port 80 here. It used to be :data:`None`, because ``beholder`` was
# the one path to :class:`Raw` that did not forward ``alias``; that made a
# *registered* number less informative than an unregistered one, which
# reaches Raw through ``_import_next_layer`` and keeps its enumeration.
# TCP port 80 here. Guards against it reading :data:`None`, as it would if
# ``beholder`` did not forward ``alias`` on its path to :class:`Raw`; that
# would make a *registered* number less informative than an unregistered
# one, which reaches Raw through ``_import_next_layer`` and keeps its
# enumeration.
# ``Data_Raw.protocol`` is "the original enumeration of this protocol", and
# 80 is what this payload arrived as, so the value belongs here. The
# protochain still reads ``Raw``: a plain :obj:`int` has no ``name`` to
# protochain reads ``Raw`` all the same: a plain :obj:`int` has no ``name`` to
# render, and only an enumeration key -- SCTP's payload protocol
# identifier, say -- shows through.
self.assertEqual(tcp.payload.info.protocol, 80)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,10 @@

The predicate :meth:`HTTP._guess_version
<pcapkit.protocols.application.http.HTTP._guess_version>` uses to recognise an
HTTP/1.* start line was ``httpv1._test_start_line``, imported lazily by the
dispatcher in ``http.py``. It is now
:func:`~pcapkit.protocols.application.http.test_start_line`, and the patterns it
HTTP/1.* start line is :func:`~pcapkit.protocols.application.http.test_start_line`,
not a private ``httpv1._test_start_line`` imported lazily, and the patterns it
shares with the HTTP/1.* parser -- ``_RE_METHOD``, ``_RE_VERSION``,
``_RE_STATUS`` and ``_HTTP2_PREFACE_HEADER`` -- moved with it, so ``httpv1.py``
``_RE_STATUS`` and ``_HTTP2_PREFACE_HEADER`` -- live beside it, so ``httpv1.py``
imports them from ``http.py`` rather than keeping a copy of its own.

The predicate is imported inside each test, never at module level: pytest
Expand Down
Loading
Loading