Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/source/pcapkit/corekit/fields/byteorder.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
Byte-Order Helpers
==================

.. module:: pcapkit.corekit.fields.byteorder

:mod:`pcapkit.corekit.fields.byteorder` contains the field callback that reads
a capture file's byte order out of the packet data. It is shared by the PCAP
frame header (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the PCAP-NG
blocks (:mod:`~pcapkit.protocols.schema.misc.pcapng`).

Auxiliary Functions
-------------------

.. autofunction:: pcapkit.corekit.fields.byteorder.byteorder_callback
.. autofunction:: pcapkit.corekit.fields.byteorder.packet_byteorder
1 change: 1 addition & 0 deletions docs/source/pcapkit/corekit/fields/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ describe the structure of protocol headers.
ipaddress
collections
misc
byteorder

Every field class is a :class:`~pcapkit.corekit.fields.field.FieldBase`
subclass, responsible for parsing and/or formatting the field value. The class
Expand Down
15 changes: 0 additions & 15 deletions docs/source/pcapkit/protocols/misc/byteorder.rst

This file was deleted.

1 change: 0 additions & 1 deletion docs/source/pcapkit/protocols/misc/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,5 @@ indication of empty payload, the PCAP header classes, and the PCAP-NG

pcap
pcapng
byteorder
raw
null
4 changes: 2 additions & 2 deletions docs/source/pcapkit/protocols/misc/pcapng.rst
Original file line number Diff line number Diff line change
Expand Up @@ -512,9 +512,9 @@ Auxiliary Functions
-------------------

The byte order of every block but the Section Header Block is read by
:func:`~pcapkit.protocols.schema.misc.byteorder.byteorder_callback`, which is
:func:`~pcapkit.corekit.fields.byteorder.byteorder_callback`, which is
shared with the PCAP frame header and documented in
:mod:`pcapkit.protocols.schema.misc.byteorder`.
:mod:`pcapkit.corekit.fields.byteorder`.

.. autofunction:: pcapkit.protocols.schema.misc.pcapng.shb_byteorder_callback
.. autofunction:: pcapkit.protocols.schema.misc.pcapng.pcapng_block_selector
Expand Down
2 changes: 1 addition & 1 deletion examples/generators/endian.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
That is how GitHub issue #605 survived: ``Frame.unpack`` seeded the byte order
under the key ``bytesorder`` where ``byteorder_callback`` -- then in
:file:`pcapkit/protocols/schema/misc/pcap/frame.py`, now in
:file:`pcapkit/protocols/schema/misc/byteorder.py` -- reads ``byteorder``, so the
:file:`pcapkit/corekit/fields/byteorder.py` -- reads ``byteorder``, so the
lookup always missed and always fell back to :data:`sys.byteorder`. Reading a
big-endian capture then byte-swapped ``ts_sec``, ``ts_usec``, ``incl_len`` and
``orig_len`` -- and because ``incl_len`` is the payload length, the first record
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# -*- coding: utf-8 -*-
"""byte-order helpers shared by the PCAP and PCAP-NG schemas
"""byte-order field helpers

:mod:`pcapkit.protocols.schema.misc.byteorder` holds the field callback that
reads a capture file's byte order out of the packet data. The record header of
a classic PCAP file (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the
:mod:`pcapkit.corekit.fields.byteorder` holds the field callback that reads a
capture file's byte order out of the packet data. The record header of a
classic PCAP file (:mod:`~pcapkit.protocols.schema.misc.pcap.frame`) and the
blocks of a PCAP-NG file (:mod:`~pcapkit.protocols.schema.misc.pcapng`) both use
it.
it. It acts on a :class:`~pcapkit.corekit.fields.numbers.NumberField` and on the
``__packet__`` key :class:`~pcapkit.corekit.fields.misc.SchemaField` hands a
nested schema, so it lives with the fields rather than with either schema
(:issue:`1711`).

"""

Expand Down
2 changes: 1 addition & 1 deletion pcapkit/corekit/fields/misc.py
Original file line number Diff line number Diff line change
Expand Up @@ -567,7 +567,7 @@ def nested_packet_context(packet: 'dict[str, Any]') -> 'dict[str, Any]':
reserved ``__packet__`` key, for a callback that needs to name the
outer schema specifically rather than whichever schema happens to
declare a given field -- see
:func:`pcapkit.protocols.schema.misc.byteorder.packet_byteorder` and
:func:`pcapkit.corekit.fields.byteorder.packet_byteorder` and
:meth:`~pcapkit.protocols.schema.misc.pcapng.BlockType.post_process`
for why that distinction matters, and note that both already
hand-roll this exact fallback and so are unaffected by (and do not
Expand Down
2 changes: 1 addition & 1 deletion pcapkit/protocols/schema/misc/pcap/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

from typing import TYPE_CHECKING

from pcapkit.corekit.fields.byteorder import byteorder_callback
from pcapkit.corekit.fields.misc import PayloadField
from pcapkit.corekit.fields.numbers import UInt32Field
from pcapkit.protocols.schema.misc.byteorder import byteorder_callback
from pcapkit.protocols.schema.schema import Schema, schema_final

__all__ = ['Frame']
Expand Down
4 changes: 2 additions & 2 deletions pcapkit/protocols/schema/misc/pcapng.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from pcapkit.const.pcapng.secrets_type import SecretsType as Enum_SecretsType
from pcapkit.const.pcapng.verdict_type import VerdictType as Enum_VerdictType
from pcapkit.const.reg.linktype import LinkType as Enum_LinkType
from pcapkit.corekit.fields.byteorder import byteorder_callback, packet_byteorder
from pcapkit.corekit.fields.collections import OptionField
from pcapkit.corekit.fields.ipaddress import (IPv4AddressField, IPv4InterfaceField,
IPv6AddressField, IPv6InterfaceField)
Expand All @@ -26,7 +27,6 @@
from pcapkit.corekit.fields.strings import (BitField, BytesField, DecodedString, PaddingField,
StringField)
from pcapkit.corekit.multidict import MultiDict, OrderedMultiDict
from pcapkit.protocols.schema.misc.byteorder import byteorder_callback, packet_byteorder
from pcapkit.protocols.schema.schema import EnumSchema, Schema, schema_final
from pcapkit.utilities.exceptions import FieldValueError, ProtocolError, stacklevel
from pcapkit.utilities.logging import SPHINX_TYPE_CHECKING
Expand Down Expand Up @@ -177,7 +177,7 @@ def shb_byteorder_callback(field: 'NumberField', packet: 'dict[str, Any]') -> 'N
one would otherwise inherit the *previous* section's byte order. The magic
is therefore also written back as ``packet['byteorder']``, which is what the
SHB's own options -- read by
:func:`~pcapkit.protocols.schema.misc.byteorder.byteorder_callback`, after this field --
:func:`~pcapkit.corekit.fields.byteorder.byteorder_callback`, after this field --
resolve their byte order from.

A Section Header Block of 12 octets has no room for the magic, and its
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,37 @@

GitHub issue #1519. The PCAP frame header and the PCAP-NG blocks each had a
``byteorder_callback``, and the PCAP-NG one additionally fell back to an
enclosing ``__packet__``. One helper now serves both, in
:mod:`pcapkit.protocols.schema.misc.byteorder`, with the PCAP-NG body: a
enclosing ``__packet__``. One helper now serves both, with the PCAP-NG body: a
superset that gives the frame header the same answer on every packet the
library hands it, since :meth:`Frame.pack
<pcapkit.protocols.misc.pcap.frame.Frame.pack>` and :meth:`Frame.unpack
<pcapkit.protocols.misc.pcap.frame.Frame.unpack>` always seed ``byteorder``.
This pins the helper, its ``__packet__`` fallback, and that both schemas use it.

GitHub issue #1711 then moved the helper from
``pcapkit.protocols.schema.misc.byteorder`` to
:mod:`pcapkit.corekit.fields.byteorder`, with no re-export left behind. This
also pins that the old path no longer imports, and that the new module reaches
no higher than :mod:`pcapkit.corekit`.

"""

import ast
import importlib
import pathlib
import struct
import sys
import unittest

from tests._support import reimport_once_per_class

SHARED = 'pcapkit.protocols.schema.misc.byteorder'
SHARED = 'pcapkit.corekit.fields.byteorder'

#: Where the helper lived before #1711; removed, not re-exported.
OLD_PATH = 'pcapkit.protocols.schema.misc.byteorder'

#: Source of :data:`SHARED`, read with :mod:`ast` rather than imported.
MODULE = pathlib.Path(__file__).resolve().parents[2] / 'pcapkit' / 'corekit' / 'fields' / 'byteorder.py'

#: The byte order that is not the host's, so a fallback to the host is visible.
OTHER = 'big' if sys.byteorder == 'little' else 'little'
Expand Down Expand Up @@ -85,6 +98,30 @@ def test_frame_header_reads_the_byte_order_it_is_given(self) -> None:
self.assertEqual((header.ts_sec, header.ts_usec), (1, 2))
self.assertEqual(header.pack(dict(packet)), raw)

def test_the_old_import_path_no_longer_imports(self) -> None:
"""#1711: the move leaves no re-export at the old path."""
with self.assertRaises(ModuleNotFoundError) as ctx:
importlib.import_module(OLD_PATH)
self.assertEqual(ctx.exception.name, OLD_PATH)

package = importlib.import_module(OLD_PATH.rpartition('.')[0])
self.assertFalse(hasattr(package, 'byteorder'))

def test_the_module_imports_nothing_from_foundation_or_protocols(self) -> None:
"""``corekit`` sits below both, so the helper may not reach up to either."""
tree = ast.parse(MODULE.read_text(encoding='utf-8'))
imported = []
for node in ast.walk(tree):
if isinstance(node, ast.Import):
imported.extend(alias.name for alias in node.names)
elif isinstance(node, ast.ImportFrom):
self.assertEqual(node.level, 0, f'relative import at line {node.lineno}')
imported.append(node.module or '')
self.assertTrue(imported)
for name in imported:
with self.subTest(module=name):
self.assertFalse(name.startswith(('pcapkit.foundation', 'pcapkit.protocols')), name)


if __name__ == '__main__':
unittest.main()
2 changes: 1 addition & 1 deletion tests/corekit/test_fields_misc_packet_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -179,7 +179,7 @@ def test_nested_schema_reads_enclosing_field_by_name_and_does_not_leak_writes(se
def test_pcapng_byteorder_consumer_still_works_with_both_shapes(self) -> None:
"""The one existing, hand-rolled ``__packet__`` fallback is unaffected.

``packet_byteorder`` (``pcapkit/protocols/schema/misc/byteorder.py:25``)
``packet_byteorder`` (``pcapkit/corekit/fields/byteorder.py:29``)
is called both with a plain dict built by hand -- as several tests and
:func:`~pcapkit.foundation.engines.pcapng` construct -- and with what
``SchemaField`` now actually builds. Both must keep working, and
Expand Down
2 changes: 1 addition & 1 deletion tests/protocols/misc/pcap/test_frame_endian_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
:meth:`Frame.unpack <pcapkit.protocols.misc.pcap.frame.Frame.unpack>` seeded the
order under the key ``bytesorder``, while ``byteorder_callback`` -- then in
:file:`pcapkit/protocols/schema/misc/pcap/frame.py`, now in
:file:`pcapkit/protocols/schema/misc/byteorder.py` -- reads ``byteorder``, so the
:file:`pcapkit/corekit/fields/byteorder.py` -- reads ``byteorder``, so the
``.get()`` never found the key and always fell back to the host's order -- the
sibling :meth:`Frame.pack <pcapkit.protocols.misc.pcap.frame.Frame.pack>`
eleven lines earlier spelled it correctly, which is what marks it as a slip
Expand Down
2 changes: 1 addition & 1 deletion tests/protocols/misc/pcap/test_header_frame_unit.py
Original file line number Diff line number Diff line change
Expand Up @@ -448,7 +448,7 @@ def test_frame_header_is_read_in_the_files_byte_order(self) -> None:
seeded the file's byte order under the key ``bytesorder``, where
``byteorder_callback`` -- then in
:file:`pcapkit/protocols/schema/misc/pcap/frame.py`, now in
:file:`pcapkit/protocols/schema/misc/byteorder.py` -- reads ``byteorder``,
:file:`pcapkit/corekit/fields/byteorder.py` -- reads ``byteorder``,
so the lookup always missed and always fell back to
:data:`sys.byteorder`. On a little-endian host reading a little-endian
capture that is the right answer by coincidence, which is why every
Expand Down
Loading