diff --git a/docs/source/contributing/conventions/const-vendor-pairing.rst b/docs/source/contributing/conventions/const-vendor-pairing.rst index 20b0831d65..c7af54ed65 100644 --- a/docs/source/contributing/conventions/const-vendor-pairing.rst +++ b/docs/source/contributing/conventions/const-vendor-pairing.rst @@ -4,22 +4,41 @@ Const-Vendor Pairing --------------------- **Every module under** :mod:`pcapkit.const` **is generated by a matching crawler -under** :mod:`pcapkit.vendor`. A registry copied from an IANA, IETF or similar -web-published table is never hand-maintained -- the crawler re-scrapes the source and -regenerates the module, so a hand edit to the generated file is silently discarded on -the next run. :issue:`2441`, raised in cross-review and knowingly deferred rather than -fixed. +under** :mod:`pcapkit.vendor`, with no exception. A registry copied from an IANA, +IETF or similar web-published table is never hand-maintained -- the crawler +re-scrapes the source and regenerates the module, so a hand edit to the generated +file is silently discarded on the next run. :issue:`2441`, raised in cross-review +and knowingly deferred rather than fixed. -The Exception -~~~~~~~~~~~~~ +The ``DATA`` Attribute, for a Registry With No Web Source +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -**A source that is not a web-crawlable table is the one legitimate reason to hand-write -a const module.** :class:`~pcapkit.const.linux.arphrd.ARPHRD`'s source is the Linux -kernel header ````, not an IANA page, and its docstring says so plainly --- *"Only the values that pcapkit names are listed"* -- rather than claiming full -coverage from a registry :mod:`pcapkit.vendor` could crawl. The ``EnumRegistry`` shape -it was moved to on :pr:`2443` is a structural decision for API uniformity with every -other registry, not a claim that a crawler produces the file. +A source that is not a web-crawlable table does not excuse a module from this +pairing -- it changes what the crawler fetches. :class:`~pcapkit.vendor.default.Vendor` +reads its registry two ways: :meth:`~pcapkit.vendor.default.Vendor._request` either +``GET``\ s :attr:`~pcapkit.vendor.default.Vendor.LINK`, or, when a subclass overrides +:meth:`~pcapkit.vendor.default.Vendor.request` to take no arguments, returns a +module-level ``DATA`` literal instead and never touches the network. +:mod:`pcapkit.vendor.ipx.packet` and :mod:`pcapkit.vendor.ipx.socket` are the +precedent: both read a hand-transcribed ``DATA`` dict because their source, +Novell's and Xerox's own IPX socket/packet-type tables, went offline (:issue:`507`, +:issue:`518`). + +:class:`~pcapkit.const.linux.arphrd.ARPHRD` follows this pattern: its source is the +Linux kernel header ````, C source +under the kernel's own versioning rather than an IANA page, so +:mod:`pcapkit.vendor.linux.arphrd` reads a ``DATA`` dict of +``{code: (name, description)}`` transcribed from the header instead of fetching a +URL. The crawler's :meth:`~pcapkit.vendor.default.Vendor.process` renders that dict +into the same enumeration members, the same ``FLAG`` bound and the same +``_missing_`` body the committed :mod:`pcapkit.const.linux.arphrd` carries -- +running it is a no-op against the file in the tree, exactly as a web-scraped +crawler's run is. The explanatory prose that once lived in the hand-maintained +``const`` module -- which values are listed and why, the cross-reference to +:class:`~pcapkit.const.arp.hardware.Hardware` -- now lives in the crawler's own +module docstring and comments, alongside the ``DATA`` literal it documents; the +generated ``const`` module carries only the plain, templated docstring every +other generated module does. Known Violation ~~~~~~~~~~~~~~~ @@ -30,4 +49,6 @@ IANA registry and claim to name every value IANA assigns, with no :mod:`pcapkit.vendor` crawler behind either one -- no ``pcapkit/vendor/tls/`` directory exists, and neither class appears in :data:`pcapkit.vendor.__all__`. This was flagged in that PR's own review and merged anyway on the stated reasoning that the -crawler would follow later. Tracked as :issue:`2495`. +crawler would follow later. Tracked as :issue:`2495`. Unlike ARPHRD, TLS's source +*is* a live, web-published IANA table, so its eventual crawler takes the normal +``LINK``-and-scrape shape, not the ``DATA`` attribute. diff --git a/pcapkit/const/linux/arphrd.py b/pcapkit/const/linux/arphrd.py index f449b55333..0bad7cc306 100644 --- a/pcapkit/const/linux/arphrd.py +++ b/pcapkit/const/linux/arphrd.py @@ -1,15 +1,12 @@ # -*- coding: utf-8 -*- -"""Linux ARP Hardware Types +# pylint: disable=line-too-long,consider-using-f-string +"""Linux ARP hardware types ============================== .. module:: pcapkit.const.linux.arphrd -This module contains the constant enumeration for the Linux ``ARPHRD_*`` -hardware types of ````, the ``arphrd_type`` of the Linux -"cooked" capture encapsulations. Only the values that pcapkit names are listed. -The rest of the numbering is IANA's, see -:class:`~pcapkit.const.arp.hardware.Hardware`, which differs for most -non-Ethernet types. +This module contains the constant enumeration for **Linux ARP hardware types**, +which is automatically generated from :class:`pcapkit.vendor.linux.arphrd.ARPHRD`. """ @@ -49,14 +46,21 @@ class ARPHRD(EnumRegistry, IntEnum): @classmethod def _validate_value(cls, value: 'Any') -> 'None': - """Reject ``value`` unless it is an ``arphrd_type`` (a 16-bit value). + """Reject ``value`` unless it passes the range check :meth:`_missing_` makes. Overrides :meth:`~pcapkit.corekit.enum.EnumLookup._validate_value` - so that :meth:`~pcapkit.corekit.enum.EnumRegistry.register` refuses - the values :meth:`_missing_` refuses. + with that check and the message it raises, so + :meth:`~pcapkit.corekit.enum.EnumRegistry.register` refuses exactly the + values :meth:`~pcapkit.corekit.enum.EnumLookup.get` refuses, in the + same words (:issue:`1816`). + + Args: + value: Value about to be looked up or minted. Raises: - EnumValueError: If ``value`` is not an integer in ``0..0xFFFF``. + EnumValueError: If ``value`` fails that check. A + :exc:`ValueError`, so an ``except ValueError`` caller is + unaffected. """ if not (isinstance(value, int) and 0x0000 <= value <= 0xFFFF): @@ -64,14 +68,11 @@ def _validate_value(cls, value: 'Any') -> 'None': @classmethod def _missing_(cls, value: 'int') -> 'ARPHRD': - """Resolve an unnamed ``arphrd_type`` to an unregistered member. + """Lookup function used when value is not found. Args: value: Value to get enum item. - Raises: - ValueError: If ``value`` is not a 16-bit integer. - """ if not (isinstance(value, int) and 0x0000 <= value <= 0xFFFF): raise ValueError('%r is not a valid %s' % (value, cls.__name__)) diff --git a/pcapkit/vendor/__init__.py b/pcapkit/vendor/__init__.py index 0e2d4b698f..9febaf3742 100644 --- a/pcapkit/vendor/__init__.py +++ b/pcapkit/vendor/__init__.py @@ -114,6 +114,7 @@ def __getattr__(name: 'str') -> 'typing.NoReturn': from pcapkit.vendor.ipv6 import * from pcapkit.vendor.ipx import * from pcapkit.vendor.l2tp import * + from pcapkit.vendor.linux import * from pcapkit.vendor.mh import * from pcapkit.vendor.ngap import * from pcapkit.vendor.ospf import * @@ -153,6 +154,8 @@ def __getattr__(name: 'str') -> 'typing.NoReturn': 'IPX_Packet', 'IPX_Socket', # L2TP 'L2TP_Type', + # Linux + 'Linux_ARPHRD', # MH 'MH_Packet', 'MH_Option', 'MH_DNSStatusCode', 'MH_ACKStatusCode', 'MH_MNIDSubtype', 'MH_StatusCode', 'MH_EnumeratingAlgorithm', diff --git a/pcapkit/vendor/linux/__init__.py b/pcapkit/vendor/linux/__init__.py new file mode 100644 index 0000000000..16c02ba732 --- /dev/null +++ b/pcapkit/vendor/linux/__init__.py @@ -0,0 +1,21 @@ +# -*- coding: utf-8 -*- +# pylint: disable=unused-import +""":mod:`pcapkit.const.linux` Vendor Crawlers +=============================================== + +.. module:: pcapkit.vendor.linux + +This module contains all vendor crawlers of the Linux-specific encapsulations, +e.g. :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`. Available crawlers +include: + +.. list-table:: + + * - :class:`Linux_ARPHRD ` + - Linux ARP Hardware Types + +""" + +from pcapkit.vendor.linux.arphrd import ARPHRD as Linux_ARPHRD + +__all__ = ['Linux_ARPHRD'] diff --git a/pcapkit/vendor/linux/arphrd.py b/pcapkit/vendor/linux/arphrd.py new file mode 100644 index 0000000000..87a8052ee1 --- /dev/null +++ b/pcapkit/vendor/linux/arphrd.py @@ -0,0 +1,107 @@ +# -*- coding: utf-8 -*- +"""Linux ARP Hardware Types +============================== + +.. module:: pcapkit.vendor.linux.arphrd + +This module contains the vendor crawler for **Linux ARP Hardware Types**, +which is automatically generating :class:`pcapkit.const.linux.arphrd.ARPHRD`. + +""" + +import collections +import sys +from typing import TYPE_CHECKING + +from pcapkit.vendor.default import Vendor + +if TYPE_CHECKING: + from collections import Counter + +__all__ = ['ARPHRD'] + +############################################################################### +# NOTE: this crawler does not crawl. Its source is the Linux kernel header +# ````, which assigns ``ARPHRD_*`` values in C source under the +# kernel's own versioning, not an IANA or IETF table :mod:`pcapkit.vendor` could +# scrape. Per the const-vendor-pairing convention, a hand-maintained registry +# still gets a crawler, reading :data:`~pcapkit.vendor.linux.arphrd.DATA` +# instead of fetching a URL -- the same shape as +# :mod:`pcapkit.vendor.ipx.packet` and :mod:`pcapkit.vendor.ipx.socket`. +# +# pcapkit names only the ``ARPHRD_*`` values its Linux "cooked" capture +# dissector (:class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`) actually +# matches on, not every value the header defines. The rest of the ARP +# hardware-type numbering is IANA's own +# (:class:`~pcapkit.const.arp.hardware.Hardware`), which differs for most +# non-Ethernet types. +############################################################################### + +#: ARPHRD hardware-type registry, transcribed from ```` for the +#: values pcapkit's Linux "cooked" capture dissector names. +#: +#: Maps an ``arphrd_type`` to the enumeration name it takes and the comment +#: rendered above it. Names go through :meth:`~Vendor.rename`, so the generated +#: member is the :meth:`~Vendor.safe_name` of the first element. +DATA = { + 0: ('NETROM', 'NET/ROM pseudo.'), + 770: ('FRAD', 'Frame Relay Access Device'), + 778: ('IPGRE', 'GRE over IP'), + 803: ('IEEE80211_RADIOTAP', 'IEEE 802.11 with radiotap header'), + 823: ('IP6GRE', 'GRE over IPv6'), + 824: ('NETLINK', 'Kernel Netlink socket'), +} # type: dict[int, tuple[str, str]] + + +class ARPHRD(Vendor): + """Linux ARP hardware types [````]""" + + #: Value limit checker. + FLAG = 'isinstance(value, int) and 0x0000 <= value <= 0xFFFF' + + def request(self) -> 'dict[int, tuple[str, str]]': # type: ignore[override] # pylint: disable=arguments-differ + """Fetch registry data. + + Returns: + Registry data (:data:`~pcapkit.vendor.linux.arphrd.DATA`). + + """ + return DATA + + def count(self, data: 'dict[int, tuple[str, str]]') -> 'Counter[str]': # type: ignore[override] + """Count field records. + + Args: + data: Registry data. + + Returns: + Field recordings. + + """ + return collections.Counter(self.safe_name(name) for name, _ in data.values()) + + def process(self, data: 'dict[int, tuple[str, str]]') -> 'tuple[list[str], list[str]]': # type: ignore[override] + """Process registry data. + + Args: + data: Registry data. + + Returns: + Enumeration fields and missing fields. + + """ + enum = [] # type: list[str] + miss = [ + "return cls._unregistered_member(value, 'Unassigned')", + ] + + for code, (name, desc) in data.items(): + pval = str(code) + renm = self.rename(name, pval) + + enum.append(f'#: {self.wrap_comment(desc)}\n {renm} = {pval}') + return enum, miss + + +if __name__ == '__main__': + sys.exit(ARPHRD()) # type: ignore[arg-type] diff --git a/tests/vendor/test_vendor_missing_body_unit.py b/tests/vendor/test_vendor_missing_body_unit.py index 4c5ead5346..da0487a3fe 100644 --- a/tests/vendor/test_vendor_missing_body_unit.py +++ b/tests/vendor/test_vendor_missing_body_unit.py @@ -85,7 +85,7 @@ #: or under a different variable name entirely is skipped by construction too, #: same as one built by ``.append()``. Those are covered behaviourally, not #: statically, by :file:`tests/const/test_const_enum_lookup.py`. -EXPECTED_CRAWLERS_WITH_CONSTANT_MISS_LINES = 35 +EXPECTED_CRAWLERS_WITH_CONSTANT_MISS_LINES = 36 def _constant_miss_lines() -> 'dict[str, list[str]]':