feat: report prior-support clipping in search.summary (#1478)

The Clipper (#1477) counts how often it fires -- n_clipped_lane_steps is
accumulated per-lane in AbstractMultiStartGradient and written to
search_internal -- but the count never reached the artefact a user actually
reads to find out what a search did. search.summary said nothing about
clipping at all.

The channel already existed: search_summary_from_samples reads
samples.samples_info and emits the NaN counters and their rates, guarded on
the key so searches without them are unaffected. Two things were missing from
it rather than a mechanism:

- n_clipped_lane_steps never reached samples_info. It stopped at
  search_internal, so nothing downstream could see it.
- n_constrained_lane_steps did reach samples_info but was never emitted, so
  the trapped-lane counter from #1475 has been invisible in search.summary
  since it shipped. Now reported.

Clipping is reported in three cases, and the distinction between the last two
is the point:

- No clipper (ClipperNone, or a search predating the Clipper): emits nothing.
  The default path's summary is unchanged, which matters because this file is
  read by tooling and sits in every archived run's output.
- Clipped and counted (MultiStartGradient): it enforces the constraint itself
  every step via Clipper.project, so it knows how often it fired and reports
  the count and the rate, denominated by n_starts * total_steps like the NaN
  rates beside it.
- Clipped but not observable (LBFGS and the bound-supporting scipy methods):
  declarative, handing optimize.Bounds to scipy and letting scipy enforce, so
  project is never called and no mask exists. Reporting 0 there would read as
  "the clipper never fired" when it means "this search cannot know", so it
  says "not measured (bounds enforced by scipy)" instead.

The clipper is published as its class NAME rather than a bool, so the summary
can say which strategy ran and a later strategy needs no schema change. The
count is per-LANE, not per-coordinate -- a lane clipped in three parameters on
one step is one clipped lane-step -- matching how the counters beside it read,
so all four stay directly comparable. It is restored from search_internal as a
lifetime total, so a resumed run reports the whole run's clipping rather than
the current process's share.

Verified end-to-end against the search.summary files four real searches wrote,
not just the formatting helper in isolation: LBFGS default (no clipping lines),
LBFGS clipped ("not measured"), MultiStart default (no clipping lines),
MultiStart clipped (Clipped Lane-Steps = 414, rate 0.958).

One behaviour change to note: multi-start summaries gain a Constrained
Lane-Steps line they did not have before. Everything else is additive and
gated.


Claude-Session: https://claude.ai/code/session_01FzF2XmKQaqZRWZfZMxvTNR

Co-authored-by: Claude <noreply@anthropic.com>

Author: Jammy2211

autofit/non_linear/search/mle/bfgs/search.py

@@ -354,6 +354,15 @@ def samples_via_internal_from(
 
         samples_info = {
             "total_iterations": total_iterations,
+            # Prior-support enforcement (PyAutoFit#1477). The name only, with
+            # deliberately NO ``n_clipped_lane_steps`` alongside it: this search
+            # is declarative, handing ``optimize.Bounds`` to scipy and letting
+            # scipy enforce, so ``Clipper.project`` is never called, no mask is
+            # produced and there is nothing to count. Writing a ``0`` here would
+            # read as "the clipper never fired" when it means "this search
+            # cannot know"; the summary renders the absent key as "not measured"
+            # instead.
+            "clipper": type(self.clipper).__name__,
             "time": self.timer.time if self.timer else None,
         }
 

autofit/non_linear/search/mle/multi_start_gradient/search.py

@@ -1168,6 +1168,21 @@ def samples_via_internal_from(
             "n_grad_nan_lane_steps": int(
                 search_internal.get("n_grad_nan_lane_steps", 0)
             ),
+            # Prior-support enforcement (PyAutoFit#1477). The clipper's NAME
+            # rather than a bool, so the summary can say which strategy ran and
+            # so a later strategy needs no schema change here.
+            #
+            # ``n_clipped_lane_steps`` is per-LANE, matching the counters above:
+            # a lane clipped in three parameters on one step is one clipped
+            # lane-step, which keeps all four directly comparable. It is
+            # restored from ``search_internal`` as a lifetime total, so a
+            # resumed run reports the whole run's clipping and not just the
+            # current process's share — the same reasoning as the ``.get``
+            # defaults above.
+            "clipper": type(self.clipper).__name__,
+            "n_clipped_lane_steps": int(
+                search_internal.get("n_clipped_lane_steps", 0)
+            ),
             # Auto-convergence outcome: whether the run stopped on the plateau
             # check ("converged") or exhausted the ``n_steps`` ceiling
             # ("max_steps"), the settings that produced it, and the global-best

autofit/text/text_util.py

@@ -112,6 +112,61 @@ def result_info_from(samples) -> str:
     return "".join(results)
 
 
+#: The clipper that enforces nothing. A run using it is the default, unclipped
+#: path, and reports no clipping lines at all — see ``_clipper_summary_from``.
+_CLIPPER_NONE = "ClipperNone"
+
+
+def _clipper_summary_from(samples_info) -> [str]:
+    """
+    The ``search.summary`` lines describing prior-support enforcement, or none.
+
+    Three cases, and the distinction between the last two is the point:
+
+    - **No clipper configured** (``ClipperNone``, or a search that predates the
+      ``Clipper`` and writes no ``clipper`` key). Emits **nothing**. The default
+      path's summary is unchanged, byte for byte, which matters because this
+      file is read by tooling and by every existing run's archived output.
+    - **Clipped, and counted** (``MultiStartGradient``). It enforces the
+      constraint itself via ``Clipper.project`` on every step, so it knows
+      exactly how often it fired and reports the count and the rate.
+    - **Clipped, but not observable** (``LBFGS`` and the other bound-supporting
+      scipy methods). These are *declarative*: they hand ``optimize.Bounds`` to
+      scipy and let scipy enforce, so ``project`` is never called, no mask is
+      produced and there is nothing to count. Reporting ``0`` here would be a
+      lie of the worst kind available — it reads as "the clipper never fired"
+      when it means "this search cannot know". It says so instead.
+
+    The count is per-LANE, not per-coordinate: a lane clipped in three
+    parameters on one step is one clipped lane-step. That is deliberate, and it
+    matches how the NaN and constrained counters above are read, so the four are
+    directly comparable.
+    """
+    clipper = samples_info.get("clipper")
+
+    if clipper is None or clipper == _CLIPPER_NONE:
+        return []
+
+    line = [f"Clipper = {clipper}\n"]
+
+    if "n_clipped_lane_steps" not in samples_info:
+        line.append(
+            "Clipped Lane-Steps = not measured (bounds enforced by scipy)\n"
+        )
+        return line
+
+    n_clipped = int(samples_info["n_clipped_lane_steps"])
+    line.append(f"Clipped Lane-Steps = {n_clipped}\n")
+
+    lane_steps = int(samples_info.get("n_starts", 0)) * int(
+        samples_info.get("total_steps", 0)
+    )
+    if lane_steps > 0:
+        line.append(f"Clipped Lane-Step Rate = {n_clipped / lane_steps}\n")
+
+    return line
+
+
 def search_summary_from_samples(samples) -> [str]:
     line = [f"Total Samples = {samples.total_samples}\n"]
     if hasattr(samples, "total_accepted_samples"):
@@ -143,6 +198,16 @@ def search_summary_from_samples(samples) -> [str]:
         line.append(f"Value-NaN Lane-Steps = {n_value_nan}\n")
         line.append(f"Gradient-NaN Lane-Steps = {n_grad_nan}\n")
 
+        # The trapped-lane counter (PyAutoFit#1475). It reached ``samples_info``
+        # when it shipped but was never emitted here, so the one artefact a user
+        # reads to find out what the search did did not report it. Keyed
+        # separately from the NaN counters because a ``search_internal`` written
+        # before it existed has no such key, and a zero it never wrote must not
+        # be reported as a measured zero.
+        if "n_constrained_lane_steps" in samples_info:
+            n_constrained = int(samples_info["n_constrained_lane_steps"])
+            line.append(f"Constrained Lane-Steps = {n_constrained}\n")
+
         # ``n_starts * total_steps`` is the number of lane-steps actually taken.
         # A search that died before its first step has a zero denominator, so
         # the rates are omitted rather than reported as a division error.
@@ -153,6 +218,8 @@ def search_summary_from_samples(samples) -> [str]:
             line.append(f"Value-NaN Lane-Step Rate = {n_value_nan / lane_steps}\n")
             line.append(f"Gradient-NaN Lane-Step Rate = {n_grad_nan / lane_steps}\n")
 
+    line += _clipper_summary_from(samples_info=samples_info)
+
     if samples.time is not None:
         line.append(f"Time To Run = {dt.timedelta(seconds=float(samples.time))}\n")
         line.append(

test_autofit/text/test_text_util.py

@@ -213,3 +213,155 @@ def test__search_summary__nan_rates_omitted_when_no_lane_steps_taken(model):
 
     assert "Value-NaN Lane-Steps = 0\n" in summary
     assert "Rate" not in summary
+
+
+def test__search_summary__clipper_none_emits_nothing(model):
+    """
+    The default path must be untouched. A ``ClipperNone`` run reports no
+    clipping lines at all -- this file is read by tooling and sits in every
+    archived run's output, so the unclipped summary stays byte-identical.
+    """
+    samples = _gradient_samples(
+        model=model,
+        samples_info={
+            "total_samples": 10,
+            "time": None,
+            "n_starts": 8,
+            "total_steps": 100,
+            "n_resurrections": 0,
+            "n_value_nan_lane_steps": 0,
+            "n_grad_nan_lane_steps": 0,
+            "clipper": "ClipperNone",
+            "n_clipped_lane_steps": 0,
+        },
+    )
+
+    summary = "".join(text_util.search_summary_from_samples(samples=samples))
+
+    assert "Clipper" not in summary
+    assert "Clipped" not in summary
+
+
+def test__search_summary__clipper_counted_reports_count_and_rate(model):
+    """
+    ``MultiStartGradient`` enforces the constraint itself every step, so it
+    knows how often the clipper fired and reports the count and the rate.
+    """
+    samples = _gradient_samples(
+        model=model,
+        samples_info={
+            "total_samples": 10,
+            "time": None,
+            "n_starts": 8,
+            "total_steps": 100,
+            "n_resurrections": 0,
+            "n_value_nan_lane_steps": 0,
+            "n_grad_nan_lane_steps": 0,
+            "clipper": "ClipperPriorBox",
+            "n_clipped_lane_steps": 40,
+        },
+    )
+
+    summary = "".join(text_util.search_summary_from_samples(samples=samples))
+
+    assert "Clipper = ClipperPriorBox\n" in summary
+    assert "Clipped Lane-Steps = 40\n" in summary
+
+    # 40 / (8 * 100), the same lane-step denominator as the NaN rates.
+    assert "Clipped Lane-Step Rate = 0.05\n" in summary
+
+
+def test__search_summary__clipper_without_count_says_not_measured(model):
+    """
+    ``LBFGS`` is declarative -- it hands bounds to scipy and never calls
+    ``project``, so there is no mask and nothing to count. A ``0`` here would
+    read as "never fired" when it means "cannot know", so it must not appear.
+    """
+    samples = _gradient_samples(
+        model=model,
+        samples_info={
+            "total_samples": 10,
+            "time": None,
+            "clipper": "ClipperPriorBox",
+        },
+    )
+
+    summary = "".join(text_util.search_summary_from_samples(samples=samples))
+
+    assert "Clipper = ClipperPriorBox\n" in summary
+    assert "Clipped Lane-Steps = not measured (bounds enforced by scipy)\n" in summary
+    assert "Clipped Lane-Steps = 0" not in summary
+    assert "Clipped Lane-Step Rate" not in summary
+
+
+def test__search_summary__clipper_absent_for_other_searches(model):
+    """
+    A search predating the ``Clipper`` writes no ``clipper`` key, and must be
+    completely unaffected -- the same invariant the NaN counters hold.
+    """
+    samples = _gradient_samples(
+        model=model,
+        samples_info={
+            "total_samples": 10,
+            "time": None,
+            "n_starts": 8,
+            "total_steps": 100,
+            "n_resurrections": 0,
+            "n_value_nan_lane_steps": 0,
+            "n_grad_nan_lane_steps": 0,
+        },
+    )
+
+    summary = "".join(text_util.search_summary_from_samples(samples=samples))
+
+    assert "Clipper" not in summary
+    assert "Clipped" not in summary
+
+
+def test__search_summary__constrained_lane_steps_reported(model):
+    """
+    The trapped-lane counter (PyAutoFit#1475) reached ``samples_info`` when it
+    shipped but was never emitted, so the artefact a user reads to find out what
+    the search did did not report it.
+    """
+    samples = _gradient_samples(
+        model=model,
+        samples_info={
+            "total_samples": 10,
+            "time": None,
+            "n_starts": 8,
+            "total_steps": 100,
+            "n_resurrections": 0,
+            "n_value_nan_lane_steps": 0,
+            "n_grad_nan_lane_steps": 0,
+            "n_constrained_lane_steps": 667,
+        },
+    )
+
+    summary = "".join(text_util.search_summary_from_samples(samples=samples))
+
+    assert "Constrained Lane-Steps = 667\n" in summary
+
+
+def test__search_summary__constrained_absent_when_never_written(model):
+    """
+    A ``search_internal`` written before the trapped-lane counter existed has no
+    such key. A zero it never wrote must not be reported as a measured zero --
+    ``0`` and "not written" are different findings.
+    """
+    samples = _gradient_samples(
+        model=model,
+        samples_info={
+            "total_samples": 10,
+            "time": None,
+            "n_starts": 8,
+            "total_steps": 100,
+            "n_resurrections": 0,
+            "n_value_nan_lane_steps": 0,
+            "n_grad_nan_lane_steps": 0,
+        },
+    )
+
+    summary = "".join(text_util.search_summary_from_samples(samples=samples))
+
+    assert "Constrained" not in summary