feat(protocols)!: verify and compute one's-complement checksums (#2470)

docs/source/contributing/pep.rst

@@ -56,8 +56,7 @@ with DTLS: :class:`~pcapkit.protocols.application.ssl.SSL` is the shared
 record-layer base of :class:`~pcapkit.protocols.application.tls.TLS` and
 :class:`~pcapkit.protocols.application.dtls.DTLS` -- see the `DTLS`_ section
 below. Neighbor Discovery (:rfc:`4861`, the ``NDP`` above) and Multicast
-Listener Discovery (:rfc:`9777`), both carried in ICMPv6, are not implemented,
-nor is ICMP/ICMPv6 checksum computation or verification (:issue:`2470`).
+Listener Discovery (:rfc:`9777`), both carried in ICMPv6, are not implemented.
 :class:`~pcapkit.protocols.link.vlan.VLAN` is the closer precedent for a pair
 whose *layout* is identical -- it holds the whole of the tag, and
 :class:`~pcapkit.protocols.link.c_tag.C_Tag` and
@@ -819,11 +818,13 @@ These protocols parse a checksum or CRC field —
 :class:`~pcapkit.protocols.application.ospf.OSPF`,
 :class:`~pcapkit.protocols.transport.sctp.SCTP`,
 :class:`~pcapkit.protocols.transport.tcp.TCP` and
-:class:`~pcapkit.protocols.transport.udp.UDP` — and of them only SCTP checks
-whether the value is *right*, through
+:class:`~pcapkit.protocols.transport.udp.UDP` — and SCTP was the first to
+check whether the value is *right*, through
 :attr:`SCTP.checksum_valid <pcapkit.protocols.transport.sctp.SCTP.checksum_valid>`.
-For the others the field is recorded and never questioned, so a corrupted
-capture parses as cleanly as an intact one.
+IPv4, OSPF, TCP and UDP check it too, as do ICMP and ICMPv6, as the
+one's-complement half below describes. For HIP, HOPOPT, IPv6-Opts, IPX and MH the
+field is recorded and never questioned, so a corrupted capture parses as cleanly
+as an intact one.
 
 **These are different problems and they want separating**, because only one
 of them is cryptographic and the distinction decides what is actually hard.
@@ -862,8 +863,10 @@ owner, and each lands on a precedent the library already has:
   large capture, and `Maybe Even Faster?`_ above is a standing concern, so it is
   a flag rather than unconditional behaviour.
 * **A wrong checksum is a warning, not a hard failure, and the outcome is
-  recorded on the protocol class.** The warning belongs in
-  :class:`~pcapkit.utilities.warnings.ProtocolWarning`, and the recording has an
+  recorded on the protocol class.** The warning is a
+  :class:`~pcapkit.utilities.warnings.ChecksumWarning`, a
+  :class:`~pcapkit.utilities.warnings.ProtocolWarning` of its own so a filter can
+  silence it alone (:issue:`2470`), and the recording has an
   exact precedent in :class:`~pcapkit.protocols.internet.esp.ESPStatus`: an
   enumeration of outcomes carried on the parsed result, so a caller can ask
   after the fact rather than having to have been capturing warnings at the
@@ -879,19 +882,49 @@ owner, and each lands on a precedent the library already has:
   what the construction side needs in order to *emit* a correct checksum, and
   the two paths then share one definition instead of agreeing by coincidence.
 
-One case still wants a decision at implementation time, since it is about
-wording rather than design: **checksum offload**. A capture taken on the sending
+One case was left to implementation time, since it is about wording rather
+than design: **checksum offload**. A capture taken on the sending
 host routinely contains checksums the NIC had not computed yet, so the field is
 zero or garbage on the wire and "invalid" is the wrong word for it. Whatever
 status enumeration this grows should be able to say *not computed* distinctly
 from *wrong*, or it will cry wolf on the commonest capture there is —
 :class:`~pcapkit.protocols.internet.esp.ESPStatus`'s ``NO_SA`` member is the
 model for how to spell "expected, not an error".
 
-Sequenced for **wave 2 or 3**. The cryptographic half could be done sooner since
-ESP has already laid the groundwork. The checksum half builds on the IP
-pseudo-header class, :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`
-(:issue:`2471`).
+**The one's-complement half is done** (:issue:`2470`, :issue:`2472` to
+:issue:`2475`), on :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`
+(:issue:`2471`) and :mod:`pcapkit.protocols.misc.checksum.checksum`, settled as
+below: the toggles by the owner on :issue:`2471`, the rest at implementation,
+as the offload question above asked; the owner accepted the toggles and the
+offload heuristic on :issue:`2470`.
+
+* **Every checksum path is togglable, and verifying and computing default apart.** Parsing
+  verifies only under :envvar:`PCAPKIT_CHECKSUM_VERIFY`, off by default;
+  construction computes a checksum the caller leaves out unless
+  :envvar:`PCAPKIT_CHECKSUM_COMPUTE` turns it off.
+* **The toggle is an environment variable**, with a per-call override only where
+  a caller needs one. A checksum given to ``make()`` is written as given, which is
+  what keeps a rebuild byte for byte; there is no
+  :class:`~pcapkit.foundation.extraction.Extractor` or per-instance parameter.
+* **The IP layer owns the pseudo-header.** :class:`~pcapkit.protocols.internet.ipv4.IPv4`
+  and :class:`~pcapkit.protocols.internet.ipv6.IPv6` verify their upper layer's
+  checksum once parsed, and compute it for an upper layer constructed as their
+  payload; ``make()`` of TCP, UDP and ICMPv6 also takes the ``pseudo`` itself.
+* **Offload is** :attr:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus.NOT_COMPUTED`,
+  modelled on ``NO_SA``: a zero IPv4 or TCP checksum, or a TCP or UDP one holding
+  the pseudo-header's sum alone, as Linux offload leaves it, never warns. So does
+  a zero UDP checksum over IPv4 (:rfc:`768`), but not over IPv6, where it is
+  mandatory (:rfc:`8200#section-8.1`).
+* **A fragment is**
+  :attr:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus.NO_PSEUDO_HEADER`,
+  atomic ones aside: TCP, UDP, ICMP and ICMPv6 checksums cover all of the
+  datagram's data, which it carries part of. OSPF's Packet Length says what its
+  checksum covers, so it is verified when the fragment holds that much, and
+  :attr:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus.TRUNCATED`
+  otherwise.
+
+The cryptographic half is sequenced for **wave 2 or 3**, and could be done sooner
+since ESP has already laid the groundwork.
 
 Deterministic Construction Arguments
 ------------------------------------
@@ -1097,8 +1130,7 @@ is the abstract base :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` and
 :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` share for selector composition
 (:issue:`2436`); each implements its own message-type parsing independently rather than
 through a shared method body. The two are distinguished by whether they carry pseudo-header
-checksums (ICMPv6 does; ICMPv4 does not) and parse without checksum computation or verification
-(:issue:`2470`).
+checksums (ICMPv6 does; ICMPv4 does not), both verified and computed (:issue:`2470`).
 
 **IGMP — done within wave 2** (:issue:`2439`, :issue:`2448`).
 The three concrete classes :class:`IGMPv1 <pcapkit.protocols.internet.igmpv1.IGMP>`,
@@ -1271,14 +1303,13 @@ files what it merely suspects transfers the work rather than doing it, and this
 project has already had to correct a finding whose count and whose diagnosis were
 both wrong when re-derived.
 
-**Queued for wave 2 or 3 — checksum and integrity verification.** See `Checksum
-and Integrity Verification`_ above. It splits in two, and the halves are not
-equally blocked: the cryptographic half (HIP's ``HIP_MAC``/``RVS_HMAC``/
-``RELAY_HMAC`` and its signature parameters) could start whenever, since
+**Queued for wave 2 or 3 — cryptographic integrity verification.** See `Checksum
+and Integrity Verification`_ above. HIP's ``HIP_MAC``/``RVS_HMAC``/``RELAY_HMAC``
+and its signature parameters could start whenever, since
 :class:`~pcapkit.protocols.internet.esp.ESP` has already established both the
 keying-context channel and the report-rather-than-raise policy. The
-one's-complement half (IPv4, TCP, UDP, ICMP/ICMPv6, OSPF) is :issue:`2470` and
-:issue:`2472` to :issue:`2475`.
+one's-complement half (IPv4, TCP, UDP, ICMP/ICMPv6, OSPF) is **done**
+(:issue:`2470`, :issue:`2472` to :issue:`2475`).
 
 **IP pseudo-header — done** (:issue:`2471`).
 :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` is the

docs/source/pcapkit/protocols/internet/icmpv6.rst

@@ -33,6 +33,7 @@ Octets      Bits        Name                    Description
    .. automethod:: make
 
    .. automethod:: _make_data
+   .. automethod:: _verify_pseudo_checksum
 
    .. automethod:: _read_data_type_none
    .. automethod:: _read_data_type_destination_unreachable

docs/source/pcapkit/protocols/internet/ipv4.rst

@@ -89,6 +89,7 @@ Octets      Bits        Name                    Description
       :no-value:
 
    .. automethod:: __index__
+   .. automethod:: __post_init__
 
 Header Schemas
 --------------

docs/source/pcapkit/protocols/internet/ipv6.rst

@@ -44,6 +44,7 @@ Octets      Bits        Name                    Description
    .. automethod:: _import_next_layer
 
    .. automethod:: __index__
+   .. automethod:: __post_init__
 
    .. automethod:: _read_ip_hextet
 

docs/source/pcapkit/protocols/misc/checksum.rst

@@ -83,3 +83,40 @@ Data Models
 .. autoclass:: pcapkit.protocols.data.misc.checksum.ip_pseudo.IP_Pseudo
    :members:
    :show-inheritance:
+
+Internet Checksum
+-----------------
+
+.. module:: pcapkit.protocols.misc.checksum.checksum
+
+:mod:`pcapkit.protocols.misc.checksum.checksum` contains what the
+one's-complement Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP,
+ICMP, ICMPv6 and OSPF shares. Parsing verifies a checksum only if
+:data:`~pcapkit.utilities.config.CHECKSUM_VERIFY` is on, and records the
+outcome as ``checksum_status``; construction computes one left out unless
+:data:`~pcapkit.utilities.config.CHECKSUM_COMPUTE` is off (:issue:`2471`).
+A mismatch warns with a :exc:`~pcapkit.utilities.warnings.ChecksumWarning`,
+which a filter can silence alone (:issue:`2470`).
+
+A checksum over the IP pseudo-header is verified and computed by the enclosing
+:class:`~pcapkit.protocols.internet.ipv4.IPv4` or
+:class:`~pcapkit.protocols.internet.ipv6.IPv6` layer, which builds the
+:class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, rather than by
+the upper layer reaching back to its parent. So an upper layer constructed
+alone, or behind another header such as an IPv6 extension header or an
+Authentication Header, has its checksum computed only if given the ``pseudo``
+to compute it over, and one parsed alone records no outcome. ICMP's checksum,
+over the message alone, is verified as it is parsed, unless the IP layer it is
+parsed from is a fragment.
+
+.. autoclass:: pcapkit.protocols.misc.checksum.checksum.ChecksumStatus
+   :members:
+   :show-inheritance:
+
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.internet_checksum
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.compute_checksum
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.partial_checksum
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.verify_checksum
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.pseudo_header
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.verify_upper
+.. autofunction:: pcapkit.protocols.misc.checksum.checksum.fragment_payload

docs/source/pcapkit/protocols/transport/tcp.rst

@@ -50,6 +50,7 @@ Octets      Bits        Name                    Description
    .. automethod:: make
 
    .. automethod:: _make_data
+   .. automethod:: _verify_pseudo_checksum
 
    .. automethod:: _read_tcp_options
    .. automethod:: _read_mode_donone

docs/source/pcapkit/protocols/transport/udp.rst

@@ -30,6 +30,7 @@ Octets      Bits        Name                    Description
    .. automethod:: make
 
    .. automethod:: _make_data
+   .. automethod:: _verify_pseudo_checksum
 
    .. autoattribute:: __proto__
       :no-value:

docs/source/pcapkit/utilities/config.rst

@@ -4,8 +4,8 @@ Configuration
 .. module:: pcapkit.utilities.config
 
 :mod:`pcapkit.utilities.config` holds every tunable limit of :mod:`pcapkit`
-(:issue:`1726`). Each limit is read from a ``PCAPKIT_*`` environment variable
-once, when the module is imported:
+(:issue:`1726`), and the checksum `flags`_. Each is read from a ``PCAPKIT_*``
+environment variable once, when the module is imported. For a limit:
 
 * A value that is not an integer, or is below the variable's minimum, warns with
   a :exc:`~pcapkit.utilities.warnings.ConfigWarning` and falls back to the
@@ -134,13 +134,43 @@ Limits
 .. autodata:: pcapkit.utilities.config.MAX_ZERO_PAD_SHORTFALL
 .. autodata:: pcapkit.utilities.config.ZERO_PAD_BUDGET_RATIO
 
+Flags
+-----
+
+The checksum flags (:issue:`2471`) take the values :func:`read_flag` accepts,
+and a bad value warns and falls back to the default.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Variable
+     - Default
+     - Effect
+   * - :envvar:`PCAPKIT_CHECKSUM_VERIFY`
+     - off
+     - parsing verifies each checksum and records the outcome
+   * - :envvar:`PCAPKIT_CHECKSUM_COMPUTE`
+     - on
+     - construction computes a checksum the caller leaves out
+
+.. envvar:: PCAPKIT_CHECKSUM_VERIFY
+
+   Sets :data:`~pcapkit.utilities.config.CHECKSUM_VERIFY`.
+
+.. envvar:: PCAPKIT_CHECKSUM_COMPUTE
+
+   Sets :data:`~pcapkit.utilities.config.CHECKSUM_COMPUTE`.
+
+.. autodata:: pcapkit.utilities.config.CHECKSUM_VERIFY
+.. autodata:: pcapkit.utilities.config.CHECKSUM_COMPUTE
+
 .. _config-readers:
 
 Readers
 -------
 
-The limits above are read with these, and a module that keeps a variable of its
-own reads it with them too, so that a bad value is reported the same way.
+The variables above are read with these, and a module that keeps a variable of
+its own reads it with them too, so that a bad value is reported the same way.
 
 .. autofunction:: pcapkit.utilities.config.read_limit
 .. autofunction:: pcapkit.utilities.config.read_flag

docs/source/pcapkit/utilities/warnings.rst

@@ -132,6 +132,10 @@ configured separately, through the :mod:`logging` module::
    :no-members:
    :show-inheritance:
 
+.. autoexception:: pcapkit.utilities.warnings.ChecksumWarning
+   :no-members:
+   :show-inheritance:
+
 .. autoexception:: pcapkit.utilities.warnings.AttributeWarning
    :no-members:
    :show-inheritance:

pcapkit/protocols/application/ospf.py

@@ -58,6 +58,13 @@
    length from the payload and dropped those octets; it takes them as
    ``packet_length`` and ``trailer`` (:issue:`1249`, :issue:`1455`).
 
+.. versionchanged:: 1.6.0
+
+   The OSPFv2 checksum is computed when
+   :meth:`~pcapkit.protocols.application.ospf.OSPF.make` is given none, and
+   verified on read under :data:`~pcapkit.utilities.config.CHECKSUM_VERIFY`
+   (:issue:`2475`).
+
 """
 import ipaddress
 from typing import TYPE_CHECKING, cast
@@ -74,6 +81,7 @@
 from pcapkit.protocols.schema.application.ospf import OSPF as Schema_OSPF
 from pcapkit.protocols.schema.application.ospf import \
     CryptographicAuthentication as Schema_CryptographicAuthentication
+from pcapkit.utilities import config
 from pcapkit.utilities.exceptions import ProtocolError
 
 if TYPE_CHECKING:
@@ -214,6 +222,21 @@ def read(self, length: 'Optional[int]' = None, **kwargs: 'Any') -> 'Data_OSPF':
             ospf.__update__([
                 ('trailer', schema.trailer),
             ])
+
+        # NOTE: The checksum leaves out the 8-octet Authentication field at
+        # octet 16, and is unused under cryptographic authentication (RFC 2328).
+        if config.CHECKSUM_VERIFY and ospf.version == 2:
+            from pcapkit.protocols.misc.checksum.checksum import ChecksumStatus, verify_checksum  # isort: skip # pylint: disable=import-outside-toplevel
+            if ospf.autype == Enum_Authentication.Cryptographic_authentication:
+                status = ChecksumStatus.NOT_COMPUTED
+            else:
+                end = max(ospf.len, self.length)
+                status = verify_checksum(self.alias, self._data[:16] + self._data[self.length:end],
+                                         ospf.chksum, length=end - 8)
+            ospf.__update__([
+                ('checksum_status', status),
+            ])
+
         # OSPF carries no next-protocol field -- the body is LSAs and packet-type
         # specific fields, which pcapkit does not dissect -- so dispatch on the
         # -1 sentinel, as ARP does, rather than on a code read off the wire;
@@ -229,7 +252,7 @@ def make(self,
              packet_length: 'Optional[int]' = None,
              router_id: 'IPv4Address | str | bytes | bytearray' = '0.0.0.0',  # nosec: B104
              area_id: 'IPv4Address | str | bytes | bytearray' = '0.0.0.0',  # nosec: B104
-             checksum: 'bytes' = b'\x00\x00',
+             checksum: 'Optional[bytes]' = None,
              auth_type: 'Enum_Authentication | StdlibEnum | AenumEnum | str | int' = Enum_Authentication.No_Authentication,
              auth_type_default: 'Optional[int]' = None,
              auth_type_namespace: 'Optional[dict[str, int] | dict[int, str] | Type[StdlibEnum] | Type[AenumEnum]]' = None,  # pylint: disable=line-too-long
@@ -252,7 +275,11 @@ def make(self,
                 packet, or pass those as ``trailer`` instead.
             router_id: Router ID.
             area_id: Area ID.
-            checksum: Checksum.
+            checksum: Checksum; if :data:`None` and
+                :data:`~pcapkit.utilities.config.CHECKSUM_COMPUTE` is on, computed
+                for OSPFv2 over the packet but its authentication field
+                (:rfc:`2328#appendix-A.3.1`); zero otherwise, as cryptographic
+                authentication leaves it (:rfc:`2328#appendix-D.4.3`).
             auth_type: Authentication type.
             auth_type_default: Default value for ``auth_type`` if not specified.
             auth_type_namespace: Namespace for ``auth_type``.
@@ -303,19 +330,27 @@ def make(self,
               or not 0 <= packet_length <= 0xFFFF):
             raise ProtocolError(f'OSPF: invalid packet length: {packet_length!r}')
 
-        return Schema_OSPF(
+        schema = Schema_OSPF(
             version=version,
             type=type_,  # type: ignore[arg-type]
             length=packet_length,
             router_id=router_id,
             area_id=area_id,
-            checksum=checksum,
+            checksum=b'\x00\x00' if checksum is None else checksum,
             auth_type=auth_type_,  # type: ignore[arg-type]
             auth_data=data,
             payload=payload,
             trailer=trailer,
         )
 
+        if (checksum is None and config.CHECKSUM_COMPUTE and version == 2
+                and auth_type_ != Enum_Authentication.Cryptographic_authentication):
+            from pcapkit.protocols.misc.checksum.checksum import compute_checksum  # isort: skip # pylint: disable=import-outside-toplevel
+            # NOTE: Over the packet but the Authentication field, as read() checks it (#2475).
+            octets = schema.pack()
+            schema.checksum = compute_checksum(octets[:16] + octets[self.length:max(packet_length, self.length)])
+        return schema
+
     ##########################################################################
     # Data models.
     ##########################################################################

pcapkit/protocols/data/application/ospf.py

@@ -12,6 +12,7 @@
 
     from pcapkit.const.ospf.authentication import Authentication
     from pcapkit.const.ospf.packet import Packet
+    from pcapkit.protocols.misc.checksum.checksum import ChecksumStatus
 
 __all__ = ['OSPF', 'CryptographicAuthentication']
 
@@ -71,6 +72,9 @@ class OSPF(Protocol):
         #: Octets captured past the Packet Length, present only when there are
         #: any.
         trailer: 'bytes'
+        #: Outcome of verifying :attr:`chksum`, present only for OSPFv2 when
+        #: :data:`~pcapkit.utilities.config.CHECKSUM_VERIFY` is on.
+        checksum_status: 'ChecksumStatus'
 
         def __init__(self, version: 'int', type: 'Packet', len: 'int', router_id: 'IPv4Address',
                      area_id: 'IPv4Address', chksum: 'bytes', autype: 'Authentication') -> 'None': ...  # pylint: disable=super-init-not-called,unused-argument,line-too-long,multiple-statements,redefined-builtin