refactor: collapse the 10 run_smoke.py copies onto one Hands runner

[Jammy2211]

Overview

Ten copies of .github/scripts/run_smoke.py are vendored across the workspace
repos in three structurally different variants. Re-measured 2026-08-24: there is
zero functional drift inside any variant — but that is the receipt for three
manual N-repo sweeps paid to restore it (#185 env-resolver fork, #226/#227
per-script timeout, the jupyter-guard fix), not evidence the copy-per-repo design
is safe. The HowTo tier needed none of those sweeps because it holds no logic.
This task collapses the other two variants onto that same shape: a PyAutoHands-owned
runner with per-repo config only.

It also closes the prompt's blocking question. The autolens_workspace_test
timeout/kill divergence — "the only place consolidation destroys behaviour" — was
already resolved by promotion into build_util (#226/#227 -> 52408a84);
all ten copies now import timeout_for and kill_group.

Plan

• Record the decision the prompt gates on: the timeout/kill behaviour was
  promoted to everyone, in-tree, and no repo keeps a documented divergence.
• Add an allowlist mode (--list <file>) to PyAutoHands autohands/run_python.py,
  so an opt-in smoke_tests.txt workspace can use the same runner as the opt-out
  no_run.yaml HowTo repos.
• Collapse the four *_workspace_test copies (198 lines) to HowToLens-shaped
  delegators (~75 lines).
• Promote the notebook leg — regenerate-and-retry recovery, the jupyter-missing
  non-abort path, and the check ordering around the skip guard — into PyAutoHands,
  then collapse the three user-workspace copies (356 lines).
• Fix docs/internals.md, which is stale on its own inventory ("nine copies, five
  distinct revisions - they have drifted"; omits HowToFit; the drift claim is false).

Detailed implementation plan

Work Classification

Library (PyAutoHands is the primary and owns both new capabilities); the seven
workspace collapses follow as one-file PRs once each phase's Hands PR has merged.

Worktree root

~/Code/PyAutoLabs-wt/smoke-runner-delegation/ (created later by /start_library)

Affected Repositories

• PyAutoHands (primary) — run_python.py allowlist mode, notebook leg, docs/internals.md
• autofit_workspace_test, autogalaxy_workspace_test, autolens_workspace_test, autocti_workspace_test (phase 1)
• autofit_workspace, autogalaxy_workspace, autolens_workspace (phase 2)

Branch Survey

Measured 2026-08-24 by git ls-remote (web-github session; no local worktrees).
Every repo below still carries the merged sweep branch
claude/backport-per-script-timeout-r3w1sv, which is the evidence for the
promotion decision.

Repository	Current Branch	Dirty?
PyAutoHands	main	n/a (no local checkout)
autofit_workspace_test	main	n/a
autogalaxy_workspace_test	main	n/a
autolens_workspace_test	main	n/a
autocti_workspace_test	main	n/a
autofit_workspace	main	n/a
autogalaxy_workspace	main	n/a
autolens_workspace	main	n/a

Suggested branch: feature/smoke-runner-delegation

Measured state (2026-08-24)

Variant	Repos	Lines	Intra-variant drift
workspace	autofit_workspace, autogalaxy_workspace, autolens_workspace	356	none - byte-identical
workspace_test	autofit/autogalaxy/autolens/autocti _test	198	comment-only (2 docstring lines)
HowTo	HowToLens, HowToGalaxy, HowToFit	75	one constant (PROJECT =)

Prior-task steps verified rather than redone:

• Skip-guard (step 1): done. is_clean_skip_exit appears in exactly the three
  notebook-capable copies and nowhere else — correct, since the other seven never
  shell out to jupyter.
• _BUILD_DIR (step 4): done. autogalaxy_workspace is byte-identical to its siblings.
• Timeout/kill (step 2): answered by promotion. TIMEOUT_SECS /
  BUILD_SCRIPT_TIMEOUT in all ten copies; timeout_for + kill_group imported
  from build_util with guarded local fallbacks; _kill_group in zero copies.

The blocker is discovery model, not behaviour

run_python.py is opt-out: recursive discovery under a directory, minus
config/build/no_run.yaml, with no notebook leg. Both other variants are
opt-in allowlists (smoke_tests.txt, smoke_notebooks.txt). That mismatch —
not any behavioural divergence — is why the seven copies still carry logic. So
each phase is a PyAutoHands feature first, per-repo edits second.

Implementation Steps

Phase 1 — workspace_test (4 repos, 198 -> ~75 lines)

1. autohands/run_python.py: add --list <file>. When given, read the allowlist
   (strip blanks and # comments) and run exactly those paths relative to
   directory, instead of find_scripts_in_folder. should_skip against
   no_run.yaml still applies, so an allowlisted script that is also no_run-listed
   is skipped with its reason recorded — do not let the allowlist override an
   explicit exclusion. --report-dir semantics are unchanged.
2. Cover it in PyAutoHands' own test suite: allowlist honoured, ordering preserved,
   no_run intersection skipped-with-reason, missing allowlist file is a clear
   error, absent flag leaves discovery untouched.
3. Replace each _test repo's .github/scripts/run_smoke.py with a delegator
   modelled on HowToLens/.github/scripts/run_smoke.py: PROJECT, the
   build_util import with the sibling-checkout fallback, and a subprocess.run
   into run_python.py with --list smoke_tests.txt --report-dir test-results.
   --report-dir is load-bearing (without it run_python.py never propagates
   failure and the gate is vacuously green).
4. Correct docs/internals.md:183: ten copies, three variants, no live drift;
   add HowToFit; drop the "five distinct revisions - they have drifted" claim.

Phase 2 — workspace (3 repos, 356 -> ~75 lines)

5. Promote the notebook leg into PyAutoHands. build_util.execute_notebook
   already carries the skip guard and execute_notebooks_in_folder already
   exists, so this is a notebook allowlist CLI leg plus the recovery path — not a
   rewrite. What must move across intact from the workspace copy:
   • regenerate_notebook — regenerate the single failing notebook from its
     source .py via py_to_notebook into a temp dir and retry once; the
     on-disk notebooks/ tree is never modified.
   • JUPYTER_MISSING_RC = 127 — a missing jupyter is a per-entry failure,
     never an abort and never a PASS.
   • The staged-copy-at-workspace-root trick in execute_notebook, so kernel cwd
     is the repo root and relative dataset/ paths resolve.
   • Check ordering, which is load-bearing: jupyter-missing and TIMEOUT_RC
     are both returned before is_clean_skip_exit is consulted, and a timeout
     is never retried (retrying burns a second full cap for the same result).
6. Cover each of those four in PyAutoHands' test suite before any workspace is touched.
7. Replace the three workspace copies with delegators passing both allowlists
   (smoke_tests.txt, smoke_notebooks.txt).

Phase 3 — HowTo (3 repos): no work. Already the end state.

Verification

• Each phase's PyAutoHands PR merges first; the workspace PRs follow (library-first gate).
• Per repo, the collapsed runner must produce the same PASS/FAIL/TIMEOUT verdicts
  and the same === Smoke test summary: n/m passed === line on that repo's real
  allowlist as the copy it replaces.
• No repo loses behaviour: per-script timeout, process-group kill, exit code 124
  distinct from a signal, notebook regenerate-and-retry, jupyter-missing non-abort.

Key Files

• PyAutoHands/autohands/run_python.py — allowlist mode (phase 1), notebook leg (phase 2)
• PyAutoHands/autohands/build_util.py — timeout_for :25, kill_group :92, run_capped :104, py_to_notebook :157, should_skip :286, is_clean_skip_exit :325, execute_notebook :359, execute_notebooks_in_folder :472, execute_script :514, execute_scripts_in_folder :622
• PyAutoHands/docs/internals.md :183 — the stale copy inventory
• HowToLens/.github/scripts/run_smoke.py — the 75-line reference shape
• autolens_workspace/.github/scripts/run_smoke.py — the 356-line notebook variant being promoted
• autolens_workspace_test/.github/scripts/run_smoke.py — the 198-line script-only variant

Related history

• PyAutoHands#185 — env-resolver unification; its step 7 explicitly deferred
  wrapper unification as "a hygiene follow-up". This is that follow-up.
• PyAutoHands#226/fix: resolve the per-script timeout parent-side and keep timeout output #227 (52408a84) — timeout_for promotion, the decision this
  task was gating on.
• PyAutoMind/complete/2026/08/smoke-runner-jupyter-guard.md — the third sweep.
• PyAutoMind/complete/2026/08/howto-smoke-all-tutorials.md — the opt-out rollout
  that produced the HowTo delegator shape.

Original Prompt

Click to expand starting prompt

run_smoke.py: three runner variants across 10 repos, no sync mechanism

Type: maintenance
Target: ci
Repos:

• @PyAutoHands
  Difficulty: medium
  Autonomy: supervised
  Priority: normal
  Status: draft
  Filed: 2026-07-25 (backfilled from git)

Re-scoped 2026-08-05. The original finding below is preserved for history
but its two headline claims are now measured to be wrong: step 1 is already
done, and "9 copies in 5 revisions" conflates three structurally different
programs. Read the correction block first — it is the current statement of
the task.

Original finding (2026-07-25, during the notebook skip-guard fix)

Every workspace's PR smoke gate runs its own copy of
.github/scripts/run_smoke.py. There are 9 copies across the workspace
repos in 5 distinct revisions — they have already drifted:
autofit_workspace + autolens_workspace share one revision; HowToLens +
HowToGalaxy + autofit_workspace_test share another; autogalaxy_workspace,
autogalaxy_workspace_test, autolens_workspace_test and autocti_workspace_test
are each unique. PyAutoHeart's reusable smoke-tests.yml deliberately leaves
the runner in the workspace, but nothing keeps the copies aligned.

Immediate consequence: PyAutoHands#198 taught the authoritative executor
(autohands/build_util.py::execute_notebook) to treat a clean SystemExit: 0
notebook exit as a PASS (the optional-dep skip-guard idiom), but the 9 smoke
copies still carry their own execute_notebook and keep reporting the
spurious FAIL until each adopts it. Adoption is ~2 lines per repo (they
already import env_config and build_util.py_to_notebook from PyAutoHands);
the exact snippet + full copy inventory is documented in
PyAutoHands docs/internals.md.

Correction (2026-08-05, measured during the jupyter-guard fix)

Measured across the checkouts, not inferred. There are 10 copies, not 9,
and they are three different programs, not five revisions of one:

Variant	Repos	Lines	Notebook leg	is_clean_skip_exit
workspace	autofit_workspace, autogalaxy_workspace, autolens_workspace	~266	yes	already adopted
workspace_test	autofit_workspace_test, autogalaxy_workspace_test, autocti_workspace_test	113	no	n/a
workspace_test + timeout	autolens_workspace_test	193	no	n/a
HowTo	HowToLens, HowToGalaxy, HowToFit	75	no	n/a

Consequences for the original task, in order of how much they change it:

1. Step 1 is done, and was never applicable beyond three repos. The
   skip-guard is adopted in all three notebook-capable copies. The other seven
   have no execute_notebook at all — they never shell out to jupyter, so
   there is no spurious FAIL for them to fix. The acceptance criterion "a
   notebook exiting via the skip-guard passes every workspace's PR smoke gate"
   is already met, because only three gates run notebooks.
2. The variants differ by feature, not only by drift. workspace_test
   exposes load_smoke_scripts/run_one and no notebook machinery;
   autolens_workspace_test additionally carries TIMEOUT_SECS
   (BUILD_SCRIPT_TIMEOUT) and a _kill_group process-group kill that no
   other copy has. That is a capability, not staleness — a naive
   "make them byte-identical" would delete it.
3. The HowTo tier is already the proposed end-state. Those three are
   75-line delegators (PROJECT = "howtolens", straight into PyAutoHands
   build_util) — the thin-wrapper design step 2 asks whether to build
   already exists in-tree as a working precedent to copy.
4. Real remaining drift inside the workspace variant is now two lines:
   autofit_workspace and autolens_workspace are byte-identical; autogalaxy's
   only divergence is an unused _BUILD_DIR intermediate variable. The
   jupyter-guard fix (autolens_workspace#470) landed the identical patch in all
   three, so it did not widen this.

Task (re-scoped)

1. Roll the 2-line skip-guard adoption across the copies. Done — verify
   and close out, do not redo.
2. Decide the shared-module question per variant, not globally, using the
   HowTo delegator as the reference shape:
   • Is one PyAutoHands-owned runner with per-repo config the right target, or
     two (notebook-capable and script-only)?
   • Does autolens_workspace_test's timeout/kill behaviour get promoted to
     everyone, or does that repo keep a documented divergence? Answer this
     before writing any code — it is the only place consolidation destroys
     behaviour.
3. Implement whichever shape is chosen, one PR per repo.
4. Drop the vestigial _BUILD_DIR line in autogalaxy_workspace if the
   workspace variant is not being replaced wholesale.

Acceptance

• A stated, written decision on the timeout/kill divergence — promoted or
  documented-as-intentional — before any repo is touched.
• Each variant is either a thin wrapper over a PyAutoHands-owned module, or
  carries a documented reason why it diverges.
• No repo loses behaviour it has today; autolens_workspace_test still
  enforces its per-script timeout.

---

Re-measurement + decision (2026-08-24)

Re-measured from every repo's main before planning. The 2026-08-05
correction is itself now stale: its blocking question has been answered
in-tree, and the drift it measured is gone.

Variant	Repos	Lines	Intra-variant drift, measured
workspace	autofit_workspace, autogalaxy_workspace, autolens_workspace	356	none — byte-identical
workspace_test	autofit/autogalaxy/autolens/autocti _test	198	comment-only (2 docstring lines); autogalaxy ≡ autolens byte-identical
HowTo	HowToLens, HowToGalaxy, HowToFit	75	one constant (PROJECT =)

Ten copies, three variants, zero functional divergence inside any variant.
Every blocker the earlier correction named is closed:

• The timeout/kill divergence was resolved by promotion, not documentation.
  timeout_for and kill_group now live in PyAutoHands
  autohands/build_util.py (fix: per-script timeout budget so full-dataset jax_grad scripts are measurable #226/fix: resolve the per-script timeout parent-side and keep timeout output #227 → 52408a84); all ten copies read
  BUILD_SCRIPT_TIMEOUT and import both with guarded local fallbacks. The
  sweep branch claude/backport-per-script-timeout-r3w1sv is still present on
  every affected repo. _kill_group exists in zero copies. Task step 2's
  "answer this before writing any code" is therefore already answered:
  promoted to everyone.
• Step 1 (skip-guard) verified: is_clean_skip_exit appears in exactly the
  three notebook-capable copies and nowhere else — correct, since the other
  seven never shell out to jupyter.
• Step 4 (_BUILD_DIR) is done — autogalaxy_workspace is byte-identical to
  its two siblings.

The decision (task step 2, per variant)

Full delegation to a PyAutoHands-owned runner, staged. The HowTo tier is the
target shape and already exists in-tree.

The zero-drift measured above is not evidence the copy-per-repo design is safe —
it is the receipt for three manual N-repo sweeps that were needed to restore
it (env-resolver fork PyAutoHands#185, per-script timeout #226/#227, the
jupyter-guard fix). The HowTo tier needed none of those three sweeps,
precisely because it holds no logic: PROJECT plus a subprocess.run into
autohands/run_python.py. That is the argument for consolidating, and it is a
maintenance-cost argument, not a drift argument.

The real blocker is discovery model, not behaviour: run_python.py is
opt-out (recursive discovery minus config/build/no_run.yaml) and has no
notebook leg, while both other variants are opt-in allowlists
(smoke_tests.txt, smoke_notebooks.txt). Consolidation is therefore a
PyAutoHands feature first and per-repo edits second.

Phase 1 — workspace_test (4 repos, 198 → ~75 lines).
Add an allowlist mode to autohands/run_python.py (--list <file>, taking
precedence over recursive discovery; no_run.yaml still applies). Then replace
each _test copy with a HowToLens-shaped delegator. Nothing is promoted that
isn't already in build_util — run_one is execute_script plus
timeout_for/kill_group, all three already there.

Phase 2 — workspace (3 repos, 356 → ~75 lines).
Promote the notebook leg into PyAutoHands: the regenerate-from-source-and-retry
recovery, the JUPYTER_MISSING_RC = 127 non-abort path, and the ordering
invariant that a missing jupyter and a TIMEOUT_RC are both checked before
is_clean_skip_exit. build_util.execute_notebook already carries the
skip-guard and execute_notebooks_in_folder already exists, so this is a
notebook-allowlist CLI leg plus the recovery, not a rewrite. Then collapse the
three workspace copies.

Phase 3 — HowTo (3 repos). No work. Already the end state; the audit
confirms it.

Also surfaced

PyAutoHands/docs/internals.md:183 is stale on its own inventory: it says
"nine copies, five distinct revisions — they have drifted", omits HowToFit
(which gained a copy with the opt-out HowTo smoke rollout), and the drift claim
is now false. Correct it as part of Phase 1.

Acceptance (restated against the decision)

• The timeout/kill decision is written down above: promoted, with the
  in-tree evidence. No repo keeps a documented divergence on that axis.
• After Phase 2, every one of the ten copies is a thin wrapper over a
  PyAutoHands-owned module; the only per-repo content is PROJECT and the
  allowlist paths.
• No repo loses behaviour: the per-script timeout, the process-group kill, the
  124 timeout exit code, the notebook regenerate-and-retry recovery, and the
  jupyter-missing non-abort path all survive as build_util/CLI behaviour and
  are asserted by PyAutoHands' own test suite before any workspace is collapsed.

[Jammy2211]

Phases 1 & 2 (Hands legs) shipped — #261 merged

PyAutoHands#261 merged → c0e2e53. CI green on all three matrix legs
(3.12 / 3.13 / 3.14), tenant-firewall step green in each.

Hands now carries everything the seven vendored copies need to become
HowTo-shaped delegators:

run_python.py --list FILE	opt-in script lists (phase 1)
run.py --list FILE	opt-in notebook lists (phase 2)
run.py --no-write-back	execute a throwaway copy, leave the committed tree clean
run.py --retry-from DIR	one regenerate-from-source retry (stale-notebook recovery)
build_util.files_from_list / regenerate_notebook	the two new primitives

16 new tests; suite 388 passed, 5 skipped, 0 failed. The 38 existing
notebook and timeout tests pass untouched, which is what makes the
execute_notebook split safe. Both CLI legs were also exercised end-to-end
against fixture workspaces — allowlist order preserved, no_run winning over
the list, a stale notebook recovering and being recorded once under its
original path, a missing entry failing without stopping the run, the committed
notebook byte-identical afterwards, and exit codes propagating.

Two things changed versus the plan in the issue body

• --no-write-back was needed, and is not what the plan described.
  run_notebook.py writes executed outputs back in place — right for
  generation, where the outputs are the product; wrong for a PR gate, which must
  not dirty the tree it is testing. Naively promoting the notebook leg would have
  made every smoke run mutate notebooks/. Because the kernel cwd is already
  pinned to the repo root, the flag supersedes the workspace copy's
  staged-copy-at-root trick rather than porting it.
• JUPYTER_MISSING_RC did not need promoting. That guard exists because the
  workspace copy shells out to a bare jupyter, so an absent toolchain raised
  FileNotFoundError out of main() and aborted with no summary line.
  execute_notebook invokes sys.executable run_notebook.py, which always
  exists — the failure mode is structurally absent here. One planned promotion
  item dissolved on inspection.

docs/internals.md is also corrected: it claimed "nine copies, five distinct
revisions — they have drifted" and omitted HowToFit.

Still open — deliberately

This issue stays open. The seven workspace collapses have not started:

• Phase 1b — autofit_workspace_test, autogalaxy_workspace_test,
  autolens_workspace_test, autocti_workspace_test (198 → ~75 lines each)
• Phase 2b — autofit_workspace, autogalaxy_workspace, autolens_workspace
  (356 → ~75 lines each)
• Phase 3 — HowTo ×3: no work, already the end state

Each is a one-file PR. They were gated on #261 merging, which has now happened,
so they are unblocked. Acceptance criterion "every one of the ten copies is a
thin wrapper" is not met until they land.

---

Generated by Claude Code

[Jammy2211]

Shipped — all ten copies collapsed

2085 lines → 1127, and most of what remains is the docstring explaining why
each file delegates. Of autolens_workspace_test's 77 lines, 32 are that
docstring.

Repo	PR	Change
PyAutoHands	#261	--list opt-in lists + the notebook leg
PyAutoHands	#262	allowlist authoritative — the 13-script finding
PyAutoHands	#263	regenerate_notebook resolves by relative path
autolens_workspace_test	#273	198 → 77
autogalaxy_workspace_test	#111	198 → 77
autofit_workspace_test	#90	198 → 77
autocti_workspace_test	#16	198 → 77
autofit_workspace	#147	356 → 119
autogalaxy_workspace	#226	356 → 119
autolens_workspace	#500	356 → 119
HowToLens / HowToGalaxy / HowToFit	—	already the end state

Acceptance, against the criteria in the issue

• A written decision on the timeout/kill divergence. It was already answered
  by promotion in fix: per-script timeout budget so full-dataset jax_grad scripts are measurable #226/fix: resolve the per-script timeout parent-side and keep timeout output #227 before this task began — all ten copies imported
  timeout_for and kill_group, and _kill_group existed in zero. Recorded,
  not re-litigated.
• Every variant is a thin wrapper over a PyAutoHands-owned module. All ten
  now are; none carries a documented divergence.
• No repo loses behaviour. Proven, not asserted — see below.

Two bugs found by measuring before writing any delegator

#262 — the no_run-wins rule would have deleted 13 scripts from coverage.
#261 shipped "an explicit exclusion is the more specific intent." Measured
against the real repos, that would have silently skipped 9 scripts in
autogalaxy_workspace_test, 2 in autolens_workspace_test, 1 in autofit_workspace
and 1 in autolens_workspace — all of which run in smoke today, since the
vendored runners never opened no_run.yaml. The two files are policy for
different runs. The failure mode is the dangerous kind: CI would have stayed
green. All 13 verified running and passing in production afterwards.

#263 — regenerate_notebook resolved the source by bare filename, dropping
the subdirectory, so the stale-notebook recovery was dead for every real
notebook and could find the wrong source on a name collision. The existing
tests missed it because their fixture was flat — the one layout where it is
invisible.

Verification

• Env resolution is identical: all 128 listed scripts across all seven repos
  resolved under both the old (relative) and new (absolute) path form and
  diffed — 0 differences, in env and in args.
• The two-leg exit code is the worst of both, checked in all three pass/fail
  combinations.
• Every merge was judged from the CI log, not the green tick. Each repo's
  "Running N listed scripts/notebooks" was checked against its allowlist:
  11/11, 22/22, 35/35, 3/3, 8/8 + 2/2, 14/14 + 2/2, 35/35 + 2/2.

Traps recorded for next time

--report-dir is load-bearing (without it the gate is vacuously green);
run_notebook.py writes executed outputs back in place, which is right for
generation and wrong for a PR gate; JUPYTER_MISSING_RC did not need promoting;
and a bare off/on/yes entry in no_run.yaml parses as a YAML boolean and
crashes should_skip.

Mind record: complete/2026/08/smoke-runner-delegation.md.

One thing left undone: the merged claude/smoke-copy-drift-ci-docs-ozntvv
branches across nine repos were not deleted — the authoring session's git proxy
refuses delete refspecs. All are proven merged into main.

---

Generated by Claude Code

[Jammy2211]

Correction to the close-out comment above: the line total is 2085 → 890
(3×119 + 4×77 + 3×75), not 2085 → 1127.

1127 came from a verification loop that re-read autolens_workspace's
run_smoke.py from the CDN seconds after #500 merged and got the stale
356-line file, counting that repo at its pre-collapse size. Verified against
origin/main: the file is 119 lines.

The collapse and every other claim in that comment stand — only the arithmetic
was wrong. Corrected in the Mind record
(complete/2026/08/smoke-runner-delegation.md).

---

Generated by Claude Code