Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and
- **a breaking change to** `ExtensionHeader`: it is generated from IANA's IPv6 Extension Header Types registry rather than the Protocol Numbers registry's flag, and the two disagree on 147 (`BIT-EMU`) ([#925](https://github.com/JarryShaw/PyPCAPKit/issues/925), [#926](https://github.com/JarryShaw/PyPCAPKit/pull/926)).
- the TCP and UDP `AppType` registries load on first use rather than at import ([#1538](https://github.com/JarryShaw/PyPCAPKit/issues/1538), [#1560](https://github.com/JarryShaw/PyPCAPKit/pull/1560)).
- **a breaking change to** `LLACode`: `Wilcard` is spelled `Wildcard` and `MH` is `MN`, the names [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) gives codes 0 and 2, with no alias for either old name ([#1848](https://github.com/JarryShaw/PyPCAPKit/issues/1848), [#1872](https://github.com/JarryShaw/PyPCAPKit/pull/1872)).
- **a breaking change to** `tcp.flags.Flags`: each member is its wire mask in the flags field, `FIN` `0x001` up to `AE` `0x100`, rather than `1 << offset` in IANA's most-significant-first bit numbering, which put `FIN` at `0x8000`. `int(tcp.connection)` of a SYN/ACK is `0x012`, as the header carries it, not `0x4800`, and `Flags(0x12)` is `SYN | ACK`. The range guard narrows to the 12-bit flags field, `0 <= value <= 0xFFF`. Code that stored or compared the old integers has to change; membership tests and names are unaffected ([#2310](https://github.com/JarryShaw/PyPCAPKit/issues/2310)).

#### Fixed

Expand Down
8 changes: 8 additions & 0 deletions docs/source/changelog/1.5.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,14 @@ Changed
* **a breaking change to** :class:`~pcapkit.const.mh.lla_code.LLACode`: ``Wilcard`` is
spelled ``Wildcard`` and ``MH`` is ``MN``, the names :rfc:`5568` gives codes 0 and 2,
with no alias for either old name (:issue:`1848`, :pr:`1872`).
* **a breaking change to** :class:`tcp.flags.Flags <pcapkit.const.tcp.flags.Flags>`:
each member is its wire mask in the flags field, ``FIN`` ``0x001`` up to ``AE``
``0x100``, rather than ``1 << offset`` in IANA's most-significant-first bit numbering,
which put ``FIN`` at ``0x8000``. ``int(tcp.connection)`` of a SYN/ACK is ``0x012``,
as the header carries it, not ``0x4800``, and ``Flags(0x12)`` is ``SYN | ACK``. The
range guard narrows to the 12-bit flags field, ``0 <= value <= 0xFFF``. Code that
stored or compared the old integers has to change; membership tests and names are
unaffected (:issue:`2310`).

Fixed
~~~~~
Expand Down
28 changes: 14 additions & 14 deletions pcapkit/const/tcp/flags.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,40 +26,40 @@ class Flags(EnumRegistry, IntFlag):
"""[Flags] TCP Header Flags"""

#: Reserved for future use [:rfc:`9293`]
Reserved_4 = 1 << 4
Reserved_4 = 1 << 11

#: Reserved for future use [:rfc:`9293`]
Reserved_5 = 1 << 5
Reserved_5 = 1 << 10

#: Reserved for future use [:rfc:`9293`]
Reserved_6 = 1 << 6
Reserved_6 = 1 << 9

#: AE (Accurate ECN) [:rfc:`9768`]
AE = 1 << 7
AE = 1 << 8

#: CWR (Congestion Window Reduced) [:rfc:`3168`]
CWR = 1 << 8
CWR = 1 << 7

#: ECE (ECN-Echo) [:rfc:`3168`]
ECE = 1 << 9
ECE = 1 << 6

#: Urgent Pointer field is significant (URG) [:rfc:`9293`]
URG = 1 << 10
URG = 1 << 5

#: Acknowledgment field is significant (ACK) [:rfc:`9293`]
ACK = 1 << 11
ACK = 1 << 4

#: Push Function (PSH) [:rfc:`9293`]
PSH = 1 << 12
PSH = 1 << 3

#: Reset the connection (RST) [:rfc:`9293`]
RST = 1 << 13
RST = 1 << 2

#: Synchronize sequence numbers (SYN) [:rfc:`9293`]
SYN = 1 << 14
SYN = 1 << 1

#: No more data from sender (FIN) [:rfc:`9293`]
FIN = 1 << 15
FIN = 1 << 0

@classmethod
def _validate_value(cls, value: 'Any') -> 'None':
Expand All @@ -80,7 +80,7 @@ def _validate_value(cls, value: 'Any') -> 'None':
unaffected.

"""
if not (isinstance(value, int) and 0 <= value <= 0xFFFF):
if not (isinstance(value, int) and 0 <= value <= 0xFFF):
raise EnumValueError(f'{value!r} is not a valid {cls.__name__}')

@classmethod
Expand All @@ -91,6 +91,6 @@ def _missing_(cls, value: 'int') -> 'Flags':
value: Value to get enum item.

"""
if not (isinstance(value, int) and 0 <= value <= 0xFFFF):
if not (isinstance(value, int) and 0 <= value <= 0xFFF):
raise ValueError(f'{value!r} is not a valid {cls.__name__}')
return super()._missing_(value)
16 changes: 9 additions & 7 deletions pcapkit/dumpkit/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -220,16 +220,18 @@ def render_enum(o: 'enum.Enum | aenum.Enum') -> 'str':
plausible member name, so a consumer splitting the rendering on ``::``
cannot tell it from a member genuinely so named -- and ``NONE`` *is* a
declared name elsewhere in the library. And the defect is not confined to
zero: ``Flags(1)``, ``Flags(8)`` and ``Flags(15)`` are every bit as
nameless, so a guard written against ``value == 0`` would fix one case
and leave the rest.
zero: a registry whose field has undeclared bits, such as
:class:`~pcapkit.const.mh.binding_update_flag.BindingUpdateFlag`, is
every bit as nameless at ``1`` or ``8``, so a guard written against
``value == 0`` would fix one case and leave the rest.

The fallback is the value's own decimal spelling, which is what the
enumeration libraries themselves already use for an undeclared residue:
``Flags(2057).name`` is ``'ACK|9'``, naming the declared bit and giving
the leftovers as one number. A wholly-undeclared value is that same
rendering with no declared bit to precede it, so ``Flags(9)`` becomes
``'Flags::9 [9]'`` and ``Flags(0)`` becomes ``'Flags::0 [0]'``. It also
``BindingUpdateFlag(32777).name`` is ``'A|9'``, naming the declared bit
and giving the leftovers as one number. A wholly-undeclared value is that
same rendering with no declared bit to precede it, so
``BindingUpdateFlag(9)`` becomes ``'BindingUpdateFlag::9 [9]'`` and
``Flags(0)`` becomes ``'Flags::0 [0]'``. It also
cannot be mistaken for a member name, since a Python identifier may not
begin with a digit.

Expand Down
6 changes: 3 additions & 3 deletions pcapkit/protocols/transport/tcp.py
Original file line number Diff line number Diff line change
Expand Up @@ -528,9 +528,9 @@ def read(self, length: 'Optional[int]' = None, **kwargs: 'Any') -> 'Data_TCP':
# never runs, and ``Enum_Flags.SYN in self._flags`` would raise ``TypeError:
# argument of type 'int' is not a container or iterable``. A segment with any flag
# set would mask that. :class:`Enum_Flags` is an ``aenum.IntFlag`` whose
# ``_missing_`` accepts any value from ``0`` to ``0xFFFF``, so ``Enum_Flags(0)`` is
# a valid flagless member that compares equal to ``0`` and ORs as usual. This is the type :attr:`connection`
# advertises.
# ``_missing_`` accepts any value in the 12-bit flags field, ``0`` to ``0xFFF``,
# so ``Enum_Flags(0)`` is a valid flagless member that compares equal to ``0`` and
# ORs as usual. This is the type :attr:`connection` advertises.
_flag = Enum_Flags(0)
for key, val in schema.flags.items():
if val == 1:
Expand Down
13 changes: 9 additions & 4 deletions pcapkit/vendor/tcp/flags.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,13 @@ class Flags(Vendor):
"""TCP Header Flags"""

#: Value limit checker. The registry indexes *bit offsets* 4 through 15,
#: but the members it generates are ``1 << offset``, so the value domain a
#: lookup has to accept is the 16-bit field those bits live in -- a
#: counted from the most significant end of the 16-bit Data Offset/flags
#: word, and the members it generates are their wire masks,
#: ``1 << (15 - offset)`` (:issue:`2310`). So the value domain a lookup has
#: to accept is the low 12 bits of that word, the flags field -- a
#: composite such as ``SYN | ACK`` is a legitimate value and only the
#: composite path reaches :meth:`~pcapkit.const.tcp.flags.Flags._missing_`.
FLAG = 'isinstance(value, int) and 0 <= value <= 0xFFFF'
FLAG = 'isinstance(value, int) and 0 <= value <= 0xFFF'
#: Link to registry.
LINK = 'https://www.iana.org/assignments/tcp-parameters/tcp-header-flags.csv'

Expand Down Expand Up @@ -146,7 +148,10 @@ def process(self, data: 'list[str]') -> 'list[str]': # type: ignore[override]
name = DATA.get(int(code), dscp.split(' (')[0]).replace('Reserved for future use', 'Reserved')
renm = self.rename(name or 'Unassigned', code, original=dscp)

pres = f"{renm} = 1 << {code}"
# NOTE: IANA numbers the bits from the most significant end of the
# 16-bit Data Offset/flags word, so the wire mask of offset ``n`` is
# ``1 << (15 - n)`` -- FIN, at offset 15, is the low bit (#2310).
pres = f"{renm} = 1 << {15 - int(code)}"
sufs = f'#: {desc}'

# if len(pres) > 74:
Expand Down
43 changes: 22 additions & 21 deletions tests/const/test_const_enum_builtin_parity.py
Original file line number Diff line number Diff line change
Expand Up @@ -543,7 +543,7 @@ class ConstFlagCompositeTests(unittest.TestCase):
``FLAG = 'isinstance(value, int) and 4 <= value <= 15'`` checker in
:mod:`pcapkit.vendor.tcp.flags` would have done had the template ever
interpolated it: those are *bit offsets*, and the values they generate are
``1 << offset``.
the wire masks ``1 << (15 - offset)`` (#2310).
"""

def setUp(self) -> None:
Expand All @@ -565,19 +565,16 @@ def test_tcp_flag_composites_resolve(self) -> None:
every = 0
for member in Flags:
every |= int(member)
self.assertEqual(every, 0xFFF0)
self.assertEqual(every, 0xFFF)
self.assertEqual(int(Flags(every)), every)

# One past the 16-bit field the registry indexes bits of.
for beyond in (0x10000, -1, -0x10000, UNRESOLVABLE):
# One past the 12-bit flags field, where the data offset begins, and
# the values outside any width.
for beyond in (0x1000, 0x10000, -1, -0x1000, UNRESOLVABLE):
with self.subTest(value=beyond):
with self.assertRaises(ValueError):
Flags(beyond)

# An unassigned low bit is in range and still composes, rather than
# being rejected: bits 0-3 of that field are the data offset.
self.assertEqual(int(Flags(1)), 1)

def test_the_other_flag_registry_composes(self) -> None:
"""``CommandType`` composes; ``TransportProtocol`` is not in this group.

Expand Down Expand Up @@ -717,11 +714,11 @@ def test_the_guard_leaves_the_other_lookup_paths_alone(self) -> None:
broken either would fail rather than merely go unmeasured.

The fallback is probed with a two-part check: a *registered* default
(``1 << 14``, ``SYN``) is genuinely consulted and resolves, while
(``1 << 1``, ``SYN``) is genuinely consulted and resolves, while
``Flags.get(UNRESOLVABLE, 0)`` raises the same original-key error as
no default at all. ``0`` is in-bounds for ``Flags``'s own
``_missing_`` (``0 <= value <= 0xFFFF``), but it is not a *registered*
``Flags`` member (``Flags``'s smallest declared value is ``1 << 4``),
``_missing_`` (``0 <= value <= 0xFFF``), but it is not a *registered*
``Flags`` member (``Flags``'s smallest declared value is ``1 << 0``),
and GitHub issue #864 restricts ``default`` to a plain
``_value2member_map_`` lookup -- the same accepted cost as an
unassigned integer range, here on an :class:`~aenum.IntFlag` registry
Expand All @@ -748,12 +745,12 @@ def test_the_guard_leaves_the_other_lookup_paths_alone(self) -> None:

# Integer paths, which do.
self.assertIs(TransportProtocol.get(1), TransportProtocol.tcp)
self.assertIs(Flags.get(1 << 14), Flags.SYN)
self.assertIs(Flags.get(1 << 1), Flags.SYN)

# And the fallback the guard's ValueError is what triggers: GitHub issue
# #584's ``get(key, default)``, which an EnumError would have walked past.
# A registered default is genuinely consulted and resolves.
self.assertIs(Flags.get(UNRESOLVABLE, 1 << 14), Flags.SYN)
self.assertIs(Flags.get(UNRESOLVABLE, 1 << 1), Flags.SYN)
# ``0`` is not a registered ``Flags`` member (GitHub issue #864), so
# it does not resolve either -- the original key's error propagates,
# the same as omitting the default outright.
Expand Down Expand Up @@ -1227,26 +1224,30 @@ def test_the_tcp_flags_checker_covers_the_value_domain_not_the_bit_offsets(self)
"""The template's checker, and why a bit-offset one could not be kept.

A ``Flags.FLAG`` of ``4 <= value <= 15`` covers the *offsets* the IANA
registry indexes, while the members it generates are ``1 << offset``.
The template never interpolated it, so it was never wrong in practice;
interpolating it unchanged would have rejected every composite, every
single member above bit 3, and ``Flags(0)``.
registry indexes, while the members it generates are their wire masks,
``1 << (15 - offset)`` (#2310). The template never interpolated it, so
it was never wrong in practice; interpolating it unchanged would have
rejected ``FIN``, ``SYN``, every member from ``ACK`` up, most
composites, and ``Flags(0)``.
"""
from pcapkit.vendor.tcp.flags import Flags as VendorFlags

checker = VendorFlags.FLAG
self.assertIn('0xFFFF', checker)
self.assertRegex(checker, r'<= 0xFFF$')

from pcapkit.const.tcp.flags import Flags

old = 'isinstance(value, int) and 4 <= value <= 15'
for member in Flags:
value = int(member)
with self.subTest(member=member.name):
self.assertTrue(eval(checker, {'value': value}), # pylint: disable=eval-used
f'the checker rejects its own member {member.name}={value}')
self.assertFalse(eval('isinstance(value, int) and 4 <= value <= 15',
{'value': value}), # pylint: disable=eval-used
f'the old checker accepted {member.name}={value}')
for value in (0, int(Flags.FIN), int(Flags.SYN | Flags.ACK), int(Flags.CWR)):
with self.subTest(value=hex(value)):
self.assertTrue(eval(checker, {'value': value})) # pylint: disable=eval-used
self.assertFalse(eval(old, {'value': value}), # pylint: disable=eval-used
f'the old checker accepted {value:#x}')

def test_self_check_of_the_repr_percent_format_walk(self) -> None:
""":func:`percent_format_lines_in_repr` against its own fixtures.
Expand Down
34 changes: 18 additions & 16 deletions tests/const/test_const_registry_protocol.py
Original file line number Diff line number Diff line change
Expand Up @@ -201,25 +201,27 @@ class _PreConversionFlags(IntFlag):
hand-copied ``get`` (omitted here -- irrelevant to ``_missing_``) and this
``_missing_``, copied verbatim. Kept inline rather than fetched from git
history so :class:`TCPFlagsConversionTests` runs offline and the
comparison is exact rather than approximate.
comparison is exact rather than approximate. The member values and the
range check track the generated module's, wire masks in the 12-bit flags
field since #2310.
"""

Reserved_4 = 1 << 4
Reserved_5 = 1 << 5
Reserved_6 = 1 << 6
AE = 1 << 7
CWR = 1 << 8
ECE = 1 << 9
URG = 1 << 10
ACK = 1 << 11
PSH = 1 << 12
RST = 1 << 13
SYN = 1 << 14
FIN = 1 << 15
Reserved_4 = 1 << 11
Reserved_5 = 1 << 10
Reserved_6 = 1 << 9
AE = 1 << 8
CWR = 1 << 7
ECE = 1 << 6
URG = 1 << 5
ACK = 1 << 4
PSH = 1 << 3
RST = 1 << 2
SYN = 1 << 1
FIN = 1 << 0

@classmethod
def _missing_(cls, value: 'Any') -> 'Any':
if not (isinstance(value, int) and 0 <= value <= 0xFFFF):
if not (isinstance(value, int) and 0 <= value <= 0xFFF):
raise ValueError(f'{value!r} is not a valid {cls.__name__}')
return super()._missing_(value)

Expand Down Expand Up @@ -276,14 +278,14 @@ def test_missing_resolves_identically_to_the_pre_conversion_shape(self) -> None:
:class:`~aenum.Flag` machinery exactly as it did when ``Flags``
inherited from ``IntFlag`` directly. Swept rather than sampled at a
few boundary values, across every declared member, every boundary of
the 16-bit field the range check bounds, and several composites,
the 12-bit field the range check bounds, and several composites,
so a widened or narrowed range check would be caught here rather
than only in the composite-specific tests in
``tests/const/test_const_enum_builtin_parity.py``.
"""
from pcapkit.const.tcp.flags import Flags

probes = [0, 1, 0xF, 0x10, 0xFFFF, 0x10000, -1, -0x10000,
probes = [0, 1, 0xF, 0x10, 0xFFF, 0x1000, 0xFFFF, 0x10000, -1, -0x1000,
1 << 70, 0xFFF0]
for member in _PreConversionFlags:
probes.append(int(member))
Expand Down
Loading
Loading