Skip to content

fix: handle rejected final samples in test mode 1 #1462

Description

@Jammy2211

Overview

PYAUTO_TEST_MODE=1 runs a real reduced sampler, but its one-sample finalization can reconstruct a point that Fitness already rejected with FitException. This surfaced in PyAutoHeart run 31441556729 after PyAutoGalaxy's physical ellipticity guard correctly rejected an invalid PowerLaw profile. The finalization lifecycle must preserve the same resample boundary as likelihood evaluation so smoke validation is deterministic and downstream chaining receives a valid result.

Plan

  • Detect the rejected-final-sample condition specifically in reduced real-sampler test mode.
  • Replace it with a bounded, valid representative test-mode sample suitable for result creation and chaining.
  • Raise a clear error if no valid representative can be found; continue to propagate all non-FitException failures and preserve normal-mode behavior.
  • Add deterministic regression coverage for an invalid first/only candidate and run focused plus full PyAutoFit tests.
Detailed implementation plan

Affected Repositories

  • PyAutoFit (primary)

Branch Survey

Repository Current Branch Dirty?
./PyAutoFit main clean

Suggested branch: feature/test-mode-fit-exception-finalization

Implementation Steps

  1. Update autofit/non_linear/search/abstract_search.py so final result construction in TEST_MODE=1 validates the sampler's summary instance before persisting completion.
  2. On FitException only, obtain a valid representative vector using a bounded test-mode resampling path, build a sentinel-scored sample set, and log why the rejected sampler point was replaced.
  3. If bounded resampling produces no valid model instance, raise a clear FitException rather than reconstructing the rejected point or looping indefinitely.
  4. Keep production searches and test modes 2/3 unchanged, and do not catch unrelated exceptions.
  5. Add deterministic lifecycle regression tests in test_autofit/non_linear/search/test_abstract_search.py, including propagation guards.
  6. Run the focused tests and the full test_autofit/ suite before shipping.

Key Files

  • autofit/non_linear/search/abstract_search.py — finalization and test-mode representative fallback.
  • test_autofit/non_linear/search/test_abstract_search.py — deterministic regression coverage.
  • autofit/non_linear/search/updater.py — existing FitException boundary used to establish the lifecycle leak; change only if implementation evidence requires it.

Original Prompt

Click to expand starting prompt

TEST_MODE=1 must not finalize rejected samples

Type: bug
Target: PyAutoFit
Repos:

  • @PyAutoFit
    Difficulty: small
    Autonomy: supervised
    Priority: urgent

Original request (verbatim)

OK do this quickly then albeit why did it not cause issues before I guess the guard didn't exist so yeah FitException handling in test mode

Problem

Fresh post-merge release validation run 31441556729 passed 660 scripts and all TestPyPI install checks, but autolens_workspace/scripts/imaging/features/extra_galaxies/slam.py failed when PYAUTO_TEST_MODE=1 produced an unphysical PowerLaw.ell_comps candidate. ModelParameterException is now correctly both ValueError and FitException, so likelihood evaluation rejects the candidate, but the reduced search subsequently attempts to materialize a rejected sample during finalization/update and lets the exception escape.

Goal

Make reduced real-sampler test mode honor FitException through the complete search lifecycle. A rejected first or only candidate must be resampled or yield a clear no-valid-sample outcome; it must never be reconstructed as a result. Keep the fix generic to PyAutoFit and deterministic under test.

Evidence

  • Heart run: https://github.com/PyAutoLabs/PyAutoHeart/actions/runs/31441556729
  • Failure: ell_comps=(-0.9257683911051445, -0.5502774378326348), magnitude 1.0769644249264025.
  • Release profile sets PYAUTO_TEST_MODE=1; Nautilus.apply_test_mode() sets n_like_max=1.
  • The configured independent truncated-Gaussian ellipticity priors retain a small but non-zero invalid unit-disk tail, so rerunning may pass but cannot make the lifecycle bug safe.

Constraints

  • Do not redesign scientific priors or narrow workspace prior support.
  • Do not catch arbitrary exceptions; preserve the existing FitException boundary.
  • Add a deterministic regression covering an invalid first/only reduced-mode candidate.
  • No live release; validate the focused path and the PyAutoFit test suite before shipping.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions