diff --git a/docs/source/pcapkit/corekit/fields/byteorder.rst b/docs/source/pcapkit/corekit/fields/byteorder.rst new file mode 100644 index 0000000000..f926519914 --- /dev/null +++ b/docs/source/pcapkit/corekit/fields/byteorder.rst @@ -0,0 +1,15 @@ +Byte-Order Helpers +================== + +.. module:: pcapkit.corekit.fields.byteorder + +:mod:`pcapkit.corekit.fields.byteorder` contains the field callback that reads +a capture file's byte order out of the packet data. It is shared by the PCAP +frame header (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the PCAP-NG +blocks (:mod:`~pcapkit.protocols.schema.misc.pcapng`). + +Auxiliary Functions +------------------- + +.. autofunction:: pcapkit.corekit.fields.byteorder.byteorder_callback +.. autofunction:: pcapkit.corekit.fields.byteorder.packet_byteorder diff --git a/docs/source/pcapkit/corekit/fields/index.rst b/docs/source/pcapkit/corekit/fields/index.rst index 5626ea7bd6..e8b3e5a49b 100644 --- a/docs/source/pcapkit/corekit/fields/index.rst +++ b/docs/source/pcapkit/corekit/fields/index.rst @@ -15,6 +15,7 @@ describe the structure of protocol headers. ipaddress collections misc + byteorder Every field class is a :class:`~pcapkit.corekit.fields.field.FieldBase` subclass, responsible for parsing and/or formatting the field value. The class diff --git a/docs/source/pcapkit/protocols/misc/byteorder.rst b/docs/source/pcapkit/protocols/misc/byteorder.rst deleted file mode 100644 index 1e6a1eb5b6..0000000000 --- a/docs/source/pcapkit/protocols/misc/byteorder.rst +++ /dev/null @@ -1,15 +0,0 @@ -Byte Order - Shared Byte-Order Helpers -====================================== - -.. module:: pcapkit.protocols.schema.misc.byteorder - -:mod:`pcapkit.protocols.schema.misc.byteorder` contains the field callback that -reads a capture file's byte order out of the packet data. It is shared by the -PCAP frame header (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the -PCAP-NG blocks (:mod:`~pcapkit.protocols.schema.misc.pcapng`). - -Auxiliary Functions -------------------- - -.. autofunction:: pcapkit.protocols.schema.misc.byteorder.byteorder_callback -.. autofunction:: pcapkit.protocols.schema.misc.byteorder.packet_byteorder diff --git a/docs/source/pcapkit/protocols/misc/index.rst b/docs/source/pcapkit/protocols/misc/index.rst index b9a6310a74..2ae424fe85 100644 --- a/docs/source/pcapkit/protocols/misc/index.rst +++ b/docs/source/pcapkit/protocols/misc/index.rst @@ -16,6 +16,5 @@ indication of empty payload, the PCAP header classes, and the PCAP-NG pcap pcapng - byteorder raw null diff --git a/docs/source/pcapkit/protocols/misc/pcapng.rst b/docs/source/pcapkit/protocols/misc/pcapng.rst index a1559bb13f..bc76982bc0 100644 --- a/docs/source/pcapkit/protocols/misc/pcapng.rst +++ b/docs/source/pcapkit/protocols/misc/pcapng.rst @@ -512,9 +512,9 @@ Auxiliary Functions ------------------- The byte order of every block but the Section Header Block is read by -:func:`~pcapkit.protocols.schema.misc.byteorder.byteorder_callback`, which is +:func:`~pcapkit.corekit.fields.byteorder.byteorder_callback`, which is shared with the PCAP frame header and documented in -:mod:`pcapkit.protocols.schema.misc.byteorder`. +:mod:`pcapkit.corekit.fields.byteorder`. .. autofunction:: pcapkit.protocols.schema.misc.pcapng.shb_byteorder_callback .. autofunction:: pcapkit.protocols.schema.misc.pcapng.pcapng_block_selector diff --git a/examples/generators/endian.py b/examples/generators/endian.py index 622321e2e9..4074dac116 100644 --- a/examples/generators/endian.py +++ b/examples/generators/endian.py @@ -13,7 +13,7 @@ That is how GitHub issue #605 survived: ``Frame.unpack`` seeded the byte order under the key ``bytesorder`` where ``byteorder_callback`` -- then in :file:`pcapkit/protocols/schema/misc/pcap/frame.py`, now in -:file:`pcapkit/protocols/schema/misc/byteorder.py` -- reads ``byteorder``, so the +:file:`pcapkit/corekit/fields/byteorder.py` -- reads ``byteorder``, so the lookup always missed and always fell back to :data:`sys.byteorder`. Reading a big-endian capture then byte-swapped ``ts_sec``, ``ts_usec``, ``incl_len`` and ``orig_len`` -- and because ``incl_len`` is the payload length, the first record diff --git a/pcapkit/protocols/schema/misc/byteorder.py b/pcapkit/corekit/fields/byteorder.py similarity index 83% rename from pcapkit/protocols/schema/misc/byteorder.py rename to pcapkit/corekit/fields/byteorder.py index 07196ba097..49fc9a5870 100644 --- a/pcapkit/protocols/schema/misc/byteorder.py +++ b/pcapkit/corekit/fields/byteorder.py @@ -1,11 +1,14 @@ # -*- coding: utf-8 -*- -"""byte-order helpers shared by the PCAP and PCAP-NG schemas +"""byte-order field helpers -:mod:`pcapkit.protocols.schema.misc.byteorder` holds the field callback that -reads a capture file's byte order out of the packet data. The record header of -a classic PCAP file (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the +:mod:`pcapkit.corekit.fields.byteorder` holds the field callback that reads a +capture file's byte order out of the packet data. The record header of a +classic PCAP file (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the blocks of a PCAP-NG file (:mod:`~pcapkit.protocols.schema.misc.pcapng`) both use -it. +it. It acts on a :class:`~pcapkit.corekit.fields.numbers.NumberField` and on the +``__packet__`` key :class:`~pcapkit.corekit.fields.misc.SchemaField` hands a +nested schema, so it lives with the fields rather than with either schema +(:issue:`1711`). """ diff --git a/pcapkit/corekit/fields/misc.py b/pcapkit/corekit/fields/misc.py index a60f386687..d235c0ca52 100644 --- a/pcapkit/corekit/fields/misc.py +++ b/pcapkit/corekit/fields/misc.py @@ -567,7 +567,7 @@ def nested_packet_context(packet: 'dict[str, Any]') -> 'dict[str, Any]': reserved ``__packet__`` key, for a callback that needs to name the outer schema specifically rather than whichever schema happens to declare a given field -- see - :func:`pcapkit.protocols.schema.misc.byteorder.packet_byteorder` and + :func:`pcapkit.corekit.fields.byteorder.packet_byteorder` and :meth:`~pcapkit.protocols.schema.misc.pcapng.BlockType.post_process` for why that distinction matters, and note that both already hand-roll this exact fallback and so are unaffected by (and do not diff --git a/pcapkit/protocols/schema/misc/pcap/frame.py b/pcapkit/protocols/schema/misc/pcap/frame.py index 1c07adfdc4..64e8a7ac1e 100644 --- a/pcapkit/protocols/schema/misc/pcap/frame.py +++ b/pcapkit/protocols/schema/misc/pcap/frame.py @@ -4,9 +4,9 @@ from typing import TYPE_CHECKING +from pcapkit.corekit.fields.byteorder import byteorder_callback from pcapkit.corekit.fields.misc import PayloadField from pcapkit.corekit.fields.numbers import UInt32Field -from pcapkit.protocols.schema.misc.byteorder import byteorder_callback from pcapkit.protocols.schema.schema import Schema, schema_final __all__ = ['Frame'] diff --git a/pcapkit/protocols/schema/misc/pcapng.py b/pcapkit/protocols/schema/misc/pcapng.py index 8765cf0ca9..3c4f48dd6f 100644 --- a/pcapkit/protocols/schema/misc/pcapng.py +++ b/pcapkit/protocols/schema/misc/pcapng.py @@ -17,6 +17,7 @@ from pcapkit.const.pcapng.secrets_type import SecretsType as Enum_SecretsType from pcapkit.const.pcapng.verdict_type import VerdictType as Enum_VerdictType from pcapkit.const.reg.linktype import LinkType as Enum_LinkType +from pcapkit.corekit.fields.byteorder import byteorder_callback, packet_byteorder from pcapkit.corekit.fields.collections import OptionField from pcapkit.corekit.fields.ipaddress import (IPv4AddressField, IPv4InterfaceField, IPv6AddressField, IPv6InterfaceField) @@ -26,7 +27,6 @@ from pcapkit.corekit.fields.strings import (BitField, BytesField, DecodedString, PaddingField, StringField) from pcapkit.corekit.multidict import MultiDict, OrderedMultiDict -from pcapkit.protocols.schema.misc.byteorder import byteorder_callback, packet_byteorder from pcapkit.protocols.schema.schema import EnumSchema, Schema, schema_final from pcapkit.utilities.exceptions import FieldValueError, ProtocolError, stacklevel from pcapkit.utilities.logging import SPHINX_TYPE_CHECKING @@ -177,7 +177,7 @@ def shb_byteorder_callback(field: 'NumberField', packet: 'dict[str, Any]') -> 'N one would otherwise inherit the *previous* section's byte order. The magic is therefore also written back as ``packet['byteorder']``, which is what the SHB's own options -- read by - :func:`~pcapkit.protocols.schema.misc.byteorder.byteorder_callback`, after this field -- + :func:`~pcapkit.corekit.fields.byteorder.byteorder_callback`, after this field -- resolve their byte order from. A Section Header Block of 12 octets has no room for the magic, and its diff --git a/tests/protocols/misc/test_schema_byteorder_1519_unit.py b/tests/corekit/test_fields_byteorder_1519_unit.py similarity index 65% rename from tests/protocols/misc/test_schema_byteorder_1519_unit.py rename to tests/corekit/test_fields_byteorder_1519_unit.py index 5f545dc96d..93748be3ef 100644 --- a/tests/protocols/misc/test_schema_byteorder_1519_unit.py +++ b/tests/corekit/test_fields_byteorder_1519_unit.py @@ -3,24 +3,37 @@ GitHub issue #1519. The PCAP frame header and the PCAP-NG blocks each had a ``byteorder_callback``, and the PCAP-NG one additionally fell back to an -enclosing ``__packet__``. One helper now serves both, in -:mod:`pcapkit.protocols.schema.misc.byteorder`, with the PCAP-NG body: a +enclosing ``__packet__``. One helper now serves both, with the PCAP-NG body: a superset that gives the frame header the same answer on every packet the library hands it, since :meth:`Frame.pack ` and :meth:`Frame.unpack ` always seed ``byteorder``. This pins the helper, its ``__packet__`` fallback, and that both schemas use it. +GitHub issue #1711 then moved the helper from +``pcapkit.protocols.schema.misc.byteorder`` to +:mod:`pcapkit.corekit.fields.byteorder`, with no re-export left behind. This +also pins that the old path no longer imports, and that the new module reaches +no higher than :mod:`pcapkit.corekit`. + """ +import ast import importlib +import pathlib import struct import sys import unittest from tests._support import reimport_once_per_class -SHARED = 'pcapkit.protocols.schema.misc.byteorder' +SHARED = 'pcapkit.corekit.fields.byteorder' + +#: Where the helper lived before #1711; removed, not re-exported. +OLD_PATH = 'pcapkit.protocols.schema.misc.byteorder' + +#: Source of :data:`SHARED`, read with :mod:`ast` rather than imported. +MODULE = pathlib.Path(__file__).resolve().parents[2] / 'pcapkit' / 'corekit' / 'fields' / 'byteorder.py' #: The byte order that is not the host's, so a fallback to the host is visible. OTHER = 'big' if sys.byteorder == 'little' else 'little' @@ -85,6 +98,30 @@ def test_frame_header_reads_the_byte_order_it_is_given(self) -> None: self.assertEqual((header.ts_sec, header.ts_usec), (1, 2)) self.assertEqual(header.pack(dict(packet)), raw) + def test_the_old_import_path_no_longer_imports(self) -> None: + """#1711: the move leaves no re-export at the old path.""" + with self.assertRaises(ModuleNotFoundError) as ctx: + importlib.import_module(OLD_PATH) + self.assertEqual(ctx.exception.name, OLD_PATH) + + package = importlib.import_module(OLD_PATH.rpartition('.')[0]) + self.assertFalse(hasattr(package, 'byteorder')) + + def test_the_module_imports_nothing_from_foundation_or_protocols(self) -> None: + """``corekit`` sits below both, so the helper may not reach up to either.""" + tree = ast.parse(MODULE.read_text(encoding='utf-8')) + imported = [] + for node in ast.walk(tree): + if isinstance(node, ast.Import): + imported.extend(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + self.assertEqual(node.level, 0, f'relative import at line {node.lineno}') + imported.append(node.module or '') + self.assertTrue(imported) + for name in imported: + with self.subTest(module=name): + self.assertFalse(name.startswith(('pcapkit.foundation', 'pcapkit.protocols')), name) + if __name__ == '__main__': unittest.main() diff --git a/tests/corekit/test_fields_misc_packet_context.py b/tests/corekit/test_fields_misc_packet_context.py index 38f2b220cd..d7b73d10fe 100644 --- a/tests/corekit/test_fields_misc_packet_context.py +++ b/tests/corekit/test_fields_misc_packet_context.py @@ -179,7 +179,7 @@ def test_nested_schema_reads_enclosing_field_by_name_and_does_not_leak_writes(se def test_pcapng_byteorder_consumer_still_works_with_both_shapes(self) -> None: """The one existing, hand-rolled ``__packet__`` fallback is unaffected. - ``packet_byteorder`` (``pcapkit/protocols/schema/misc/byteorder.py:25``) + ``packet_byteorder`` (``pcapkit/corekit/fields/byteorder.py:29``) is called both with a plain dict built by hand -- as several tests and :func:`~pcapkit.foundation.engines.pcapng` construct -- and with what ``SchemaField`` now actually builds. Both must keep working, and diff --git a/tests/protocols/misc/pcap/test_frame_endian_runtime.py b/tests/protocols/misc/pcap/test_frame_endian_runtime.py index a9d32af581..f7a07c7b49 100644 --- a/tests/protocols/misc/pcap/test_frame_endian_runtime.py +++ b/tests/protocols/misc/pcap/test_frame_endian_runtime.py @@ -13,7 +13,7 @@ :meth:`Frame.unpack ` seeded the order under the key ``bytesorder``, while ``byteorder_callback`` -- then in :file:`pcapkit/protocols/schema/misc/pcap/frame.py`, now in -:file:`pcapkit/protocols/schema/misc/byteorder.py` -- reads ``byteorder``, so the +:file:`pcapkit/corekit/fields/byteorder.py` -- reads ``byteorder``, so the ``.get()`` never found the key and always fell back to the host's order -- the sibling :meth:`Frame.pack ` eleven lines earlier spelled it correctly, which is what marks it as a slip diff --git a/tests/protocols/misc/pcap/test_header_frame_unit.py b/tests/protocols/misc/pcap/test_header_frame_unit.py index e1b0916b1d..5fa536862e 100644 --- a/tests/protocols/misc/pcap/test_header_frame_unit.py +++ b/tests/protocols/misc/pcap/test_header_frame_unit.py @@ -448,7 +448,7 @@ def test_frame_header_is_read_in_the_files_byte_order(self) -> None: seeded the file's byte order under the key ``bytesorder``, where ``byteorder_callback`` -- then in :file:`pcapkit/protocols/schema/misc/pcap/frame.py`, now in - :file:`pcapkit/protocols/schema/misc/byteorder.py` -- reads ``byteorder``, + :file:`pcapkit/corekit/fields/byteorder.py` -- reads ``byteorder``, so the lookup always missed and always fell back to :data:`sys.byteorder`. On a little-endian host reading a little-endian capture that is the right answer by coincidence, which is why every