Skip to content

ci: gate create-release.yml's publishers on a dev-release version - #2452

Merged
JarryShaw merged 2 commits into
mainfrom
feat/gate-release-on-dev-version
Oct 30, 2026
Merged

JarryShaw merged 2 commits into
mainfrom
feat/gate-release-on-dev-version

Conversation

@JarryShaw

@JarryShaw JarryShaw commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

Please follow the guide below

  • Searched for similar pull requests
  • Followed the coding style (make pylint, make mypy, make isort) -- no pcapkit/ file is touched, so these targets are no-ops for this change; confirmed by running them (pre-existing pcapkit/ findings only).
  • make test passes, and a test case covers the change -- see test results below.
  • Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible -- N/A, centralised in chore(docs): centralize 1.6.0 changelog updates in one place, not per-feature-PR #2449: this is a CI/workflow-only change with no pcapkit/ library behaviour.

What is the purpose of your pull request?

  • ci — workflows or build tooling

Description of your pull request and other information

Risk. Owner ruling, 2026-10-30: during active development on a not-yet-released
version (the current wave-2 effort targeting 1.6.0), pcapkit.__version__ should carry
a PEP 440 dev-release suffix (X.Y.Z.devN) as a work-in-progress signal. But
.github/workflows/create-release.yml had no gate on it, and it can be triggered
without a human pushing a v* tag: the workflow_run trigger fires on every "Vendor
Update" completion (a scheduled weekly cron that bumps vendor-registry constants).
version_check already computed PCAPKIT_PRERELEASE (is_prerelease), but that output
was only ever read for display (the GitHub Release's prerelease: flag) and the conda
label — never to skip a publish step. So a dev version left in __version__ by an
in-progress feature branch could be tagged, released on GitHub, and published to PyPI
and conda by the next scheduled Vendor Update, with only github-release's single reviewer approval in the path.

Fix. version_check's "Get Version" step now also computes PCAPKIT_DEV_RELEASE
from packaging.version.Version(pcapkit.__version__).is_devrelease, exposed as a job
output. Each of the four jobs that actually tags or publishes something -- github
(the GitHub Release and v* tag), tag (the conda-* tag and a commit to main),
pypi, and conda -- gets a new top-level && conjunct:
needs.version_check.outputs.PCAPKIT_DEV_RELEASE != 'true'. It is unconditional,
carrying none of the other clauses' startsWith(github.ref_name, 'v') exemption, so it
also blocks a human hand-pushing a v* tag that happens to name a dev-release version.

Why is_devrelease, not is_prerelease. This project has shipped real,
intentionally-tagged pre-1.x betas before (e.g. 1.5.0b14), and is_prerelease is
True for those too -- a blanket is_prerelease gate would silently block a legitimate,
deliberately-tagged beta release, which is not what this ruling is about.
is_devrelease is True only for a PEP 440 dev release (X.Y.Z.devN) specifically, and
stays False for alpha/beta/rc and for a normal/final version, so a real pre-release can
still ship while a dev-in-progress version cannot.

version_check itself keeps running unconditionally (it is read-only, as its own
comment already documents) -- only the four publish-side jobs needed the new gate.

Tests. tests/project/test_release_gates.py gets: new rows in gate_rows() (for
tag/pypi/conda) and GITHUB_ROWS (for github) covering
PCAPKIT_DEV_RELEASE=true on both main and a v* ref, and =null (an unset output
lets the publishers through: their != 'true' fails open, matching the existing
PCAPKIT_RELEASED_ELSEWHERE convention; a fail-closed == 'false' was deliberately left
out); a new text-layer assertion,
test_all_four_publishers_gate_on_dev_release, checked directly against all four real
conditions; a script-source assertion pinning is_devrelease over is_prerelease
(test_dev_release_evidence_reads_is_devrelease_not_is_prerelease); and a MUTATIONS
row for the dropped conjunct. I verified these actually catch a regression by
temporarily removing the new conjunct from the pypi job alone: both the evaluator
layer (gate_rows) and the text layer (test_all_four_publishers_gate_on_dev_release)
independently failed, naming pypi.

docs/source/contributing/releasing.rst gets a short paragraph documenting the
convention in "The One Manual Step" section, where the devN bump counter is already
discussed.

Test plan

  • actionlint 1.7.12 (the version this repo's own lint.yml pins) over create-release.yml -- clean
  • python -c "import yaml; yaml.safe_load(...)" over create-release.yml -- parses, and the four jobs' resolved if: text matches what was intended
  • Regression check: removing the new conjunct from one job (pypi) makes two independent tests fail, naming that job

unit-tests and release_status.py. The unit-tests release gate takes the same unconditional PCAPKIT_DEV_RELEASE != 'true' conjunct, so a dev version doesn't spend a test run it can't publish from. util/release_status.py gains one case: when the flag is 'true' and every publisher skipped, it exits 0 with a quiet "Nothing to release" notice saying this run's version carries a PEP 440 dev-release suffix. A publisher that ran, or a flag that is empty or unset, still reaches normal reconciliation and the #888 error. Reconciliation keeps its eager all([...]), so every evidence variable is read, matching the bash reference in test_release_status_scripts_1967.py.

  • Release-status parity and gate tests (the 28 tests/project files that read create-release.yml, release_status or releasing.rst): 706 passed, 4 skipped, 5303 subtests.
  • New parity cases kill three mutants that survived the previous head: the generator all(...), a dev_release_skipped() without its every-publisher-skipped check, and == 'true' → != 'false'.

@JarryShaw JarryShaw added ci Pull requests that change CI or workflow configuration (ci: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 30, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verified directly: the is_devrelease (not is_prerelease) distinction is correctly implemented, and all four publish jobs (github, tag, pypi, conda) have needs.version_check.outputs.PCAPKIT_DEV_RELEASE != 'true' correctly placed in their if: blocks — read each one directly, not just trusted the report. YAML parses clean. tests/project/test_release_gates.py: 129 passed/468 subtests, matching exactly. Given this touches release-critical infrastructure with an extensive issue history, dispatching a focused cross-review before calling it done.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 30, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review (different model, adversarial) came back NEEDS CHANGES. The if: gate logic itself is verified correct — 32,768-state evaluation confirms it blocks every dev form and 0 non-dev states — but it's incomplete:

  1. release_status isn't dev-aware, so every Create Release run on a .devN main prints a false "Stranded partial release (fix(ci): a half-finished release leaves a v* tag that makes every retry skip silently #888)" error instead of a clean no-op. Needs PCAPKIT_DEV_RELEASE wired into its env plus a dev-aware branch, with matching test updates.
  2. The new outputs: wiring has no regression test — a typo'd output name or an is_prerelease mix-up would silently disable or over-trigger the gate with nothing catching it (the repo already guards an analogous output elsewhere; this needs the same treatment).
  3. releasing.rst incorrectly says bump_version.py removes the .devN suffix — it only bumps the counter; leaving dev state needs a hand edit, which the docs should say.

Also should-fix: the "no human in the path" framing overstates it (reviewer approval already gates github-release), the ~20-min release test gate still runs on every dev push though no publisher could use it, and a stale comment has the right conclusion for the wrong reason.

Dispatching a fix to this branch now.

A scheduled Vendor Update can trigger create-release.yml with no human
starting the run. version_check already computed PCAPKIT_PRERELEASE
(is_prerelease), but nothing read it to stop a publish, so a version
carrying a PEP 440 dev-release suffix (X.Y.Z.devN, the owner's
work-in-progress marker during active development) could be tagged and
published by that automatic trigger alone.

- version_check's "Get Version" step adds PCAPKIT_DEV_RELEASE, reading
  is_devrelease rather than is_prerelease: the latter is also true for
  a deliberately-tagged alpha/beta/rc this project has shipped for real
  (e.g. 1.5.0b14), and gating on it would silently block one.
- github, tag, pypi and conda each add
  needs.version_check.outputs.PCAPKIT_DEV_RELEASE != 'true' as a
  top-level && conjunct, unconditionally, and unit-tests (the ~20-minute
  release test gate) now adds the same conjunct -- a dev version can
  never publish, so running that gate for one is wasted time (#1052).
- util/release_status.py: a dev release with every publisher skipped
  for it now reports a quiet "Nothing to release" notice instead of
  falling into the #888 stranded-partial-release error; wired via a new
  PCAPKIT_DEV_RELEASE entry in release_status's env.
- Reworded four "no human in the path" overstatements (create-release.yml
  x2, releasing.rst, a test docstring) to acknowledge github-release's
  reviewer approval -- still worth the gate, since a reviewer can miss a
  stray .devN, and a re-run that skips github asks for no fresh approval.
- releasing.rst: util/bump_version.py has no option to drop the dev
  suffix -- on a dev version it only advances the devN counter -- so
  corrected the claim that it bumps to the final version string, and
  noted that running it on a dev version just produces another one.
- tests/project/test_release_gates.py: new rows in gate_rows(),
  GITHUB_ROWS and UNIT_TESTS_ROWS covering PCAPKIT_DEV_RELEASE, a
  character-for-character wiring check between version_check's outputs
  and every consumer, a check that no job's gate reads PCAPKIT_PRERELEASE
  by mistake, and a fixed test comment whose reasoning had the risk of
  an unset gate output backwards for a publisher job.
- tests/project/test_release_status_scripts_1967.py: new cases covering
  the dev-release notice and the regression check that the same skip
  pattern with PCAPKIT_DEV_RELEASE=false still raises #888.

Build: coverage run -m pytest tests/project/ -k "release or workflow or
conda_dist" (641 passed, 1 pre-existing unrelated skip); isort, pylint
(10.00/10 on util/release_status.py) and actionlint 1.7.12 all clean.
@JarryShaw
JarryShaw force-pushed the feat/gate-release-on-dev-version branch from 12889d6 to a842da0 Compare October 30, 2026 15:15
@JarryShaw

Copy link
Copy Markdown
Owner Author

Fixed and verified directly (new head a842da007): release_status.py now exits 0 with a quiet "nothing to release" notice on a dev-release run with all publishers skipped, instead of the false #888 error — and the real stranded-release error still fires unchanged when dev=false. Added the output-wiring regression test and the is_devrelease-not-is_prerelease guard. Fixed the releasing.rst factual error and the "no human in the path" overstatement. unit-tests itself now also skips on a dev release. 131 passed/481 subtests on test_release_gates.py (up from 129/468, exactly the new coverage), isort/pylint/mypy/actionlint all clean.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 30, 2026
JarryShaw added a commit that referenced this pull request Oct 30, 2026
#2458)

Mirror create-release.yml's #2452 dev-release gate in cron-conda.yml, which
has its own independent, schedule/workflow_run-reachable route to publishing
with no check against __version__ carrying a PEP 440 .devN suffix.

- conda-update's Get Version step computes PCAPKIT_DEV_RELEASE from
  packaging.version.Version(...).is_devrelease, deliberately not
  is_prerelease, which is also true for a real alpha/beta/rc this project has
  shipped and must not block.
- conda-tag's `if:` adds the clause as a top-level conjunct; conda-dist's adds
  it inside its publishing branch only, so a build-only manual run still
  builds and smoke-tests a dev version without uploading it.
- discussion's "Post the report to Discussions" step now tells "nothing
  pushed" apart from "a dev-release build was pushed but Tag/Upload were
  withheld on purpose", instead of silently dropping both lines; the stale
  "Both skip unless a build was pushed" comment is corrected.
- Comments describing #2452's create-release.yml gate as already merged are
  reworded to "proposed ... (not yet merged)".
- test_conda_dev_release_gate_2458.py gains two inputs (1.6.0rc1.dev2,
  1.6.0.post1.dev0) that close two narrowed-predicate mutations a
  cross-review found surviving; test_cron_reports_2030.py gains coverage for
  the new report line.

actionlint 1.7.12 (shellcheck 0.10.0): zero findings. Both test files: 55
passed, 118 subtests passed, including the new report and mutation-closing
cases; pylint/mypy clean on the touched files (`pcapkit` itself is
untouched, so neither applies per lint.yml's own invocation).
JarryShaw added a commit that referenced this pull request Oct 30, 2026
#2458)

Mirror create-release.yml's #2452 dev-release gate in cron-conda.yml, which
has its own independent, schedule/workflow_run-reachable route to publishing
with no check against __version__ carrying a PEP 440 .devN suffix.

- conda-update's Get Version step computes PCAPKIT_DEV_RELEASE from
  packaging.version.Version(...).is_devrelease, deliberately not
  is_prerelease, which is also true for a real alpha/beta/rc this project has
  shipped and must not block.
- conda-tag's `if:` adds the clause as a top-level conjunct; conda-dist's adds
  it inside its publishing branch only, so a build-only manual run still
  builds and smoke-tests a dev version without uploading it.
- discussion's "Post the report to Discussions" step now tells "nothing
  pushed" apart from "a dev-release build was pushed but Tag/Upload were
  withheld on purpose", instead of silently dropping both lines; the stale
  "Both skip unless a build was pushed" comment is corrected.
- Comments describing #2452's create-release.yml gate as already merged are
  reworded to "proposed ... (not yet merged)".
- test_conda_dev_release_gate_2458.py gains two inputs (1.6.0rc1.dev2,
  1.6.0.post1.dev0) that close two narrowed-predicate mutations a
  cross-review found surviving; test_cron_reports_2030.py gains coverage for
  the new report line, plus three rows closing a fourth review round's
  remaining gaps -- CONDA_CHANGED=false with DEV_RELEASE=true, DEV_RELEASE=
  false, and DEV_RELEASE='' on a pushed build -- each confirmed to kill its
  own narrowed-predicate mutant of the explanation line's elif guard.

actionlint 1.7.12 (shellcheck 0.10.0): zero findings. Both test files: 43
passed, 105 subtests passed, including the new report and mutation-closing
cases; pylint/mypy clean on the touched files (`pcapkit` itself is
untouched, so neither applies per lint.yml's own invocation).
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Coverage: 94.44% (unit tier, Python 3.15, 0c56095b6, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 19609 133 2620 120 98.84%
pcapkit/corekit 2605 74 818 37 96.52%
pcapkit/dumpkit 281 3 100 3 98.43%
pcapkit/foundation 3070 24 1072 26 98.79%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 17593 193 4678 179 98.32%
pcapkit/toolkit 924 36 334 10 95.55%
pcapkit/utilities 502 7 144 5 98.14%
pcapkit/vendor 4449 1854 1020 167 55.11%

Per-file detail: the coverage-html artifact of this run.

JarryShaw added a commit that referenced this pull request Oct 30, 2026
#2458)

Mirror create-release.yml's #2452 dev-release gate in cron-conda.yml, which
has its own independent, schedule/workflow_run-reachable route to publishing
with no check against __version__ carrying a PEP 440 .devN suffix.

- conda-update's Get Version step computes PCAPKIT_DEV_RELEASE from
  packaging.version.Version(...).is_devrelease, deliberately not
  is_prerelease, which is also true for a real alpha/beta/rc this project has
  shipped and must not block.
- conda-tag's `if:` adds the clause as a top-level conjunct; conda-dist's adds
  it inside its publishing branch only, so a build-only manual run still
  builds and smoke-tests a dev version without uploading it.
- discussion's "Post the report to Discussions" step now tells "nothing
  pushed" apart from "a dev-release build was pushed but Tag/Upload were
  withheld on purpose", instead of silently dropping both lines; the stale
  "Both skip unless a build was pushed" comment is corrected.
- Comments describing #2452's create-release.yml gate as already merged are
  reworded to "proposed ... (not yet merged)".
- test_conda_dev_release_gate_2458.py gains two inputs (1.6.0rc1.dev2,
  1.6.0.post1.dev0) that close two narrowed-predicate mutations a
  cross-review found surviving; test_cron_reports_2030.py gains coverage for
  the new report line, plus three rows closing round 3's remaining gaps --
  CONDA_CHANGED=false with DEV_RELEASE=true, DEV_RELEASE=false, and
  DEV_RELEASE='' on a pushed build -- each confirmed to kill its own
  widened-predicate mutant of the explanation line's elif guard.

actionlint 1.7.12 (shellcheck 0.10.0): zero findings. Both test files: 43
passed, 105 subtests passed, including the new report and mutation-closing
cases; pylint clean on both files; mypy clean on test_cron_reports_2030.py
(the 2458 file carries the same pre-existing attr-defined/unused-ignore
pattern as sibling tests, not CI-gated).
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES. Cross-review on Sonnet at head a842da007. It's a relaunch: the first attempt on Fable died on a rate limit.

The dev-version gate itself works as claimed. Every PEP 440 dev spelling is flagged and 1.5.1/1.5.0b14/rc/.post are not. With every publisher skipped, release_status.py exits 0 with the quiet notice, and the real 1.5.1 partial-publish shape (run 41098750345) still exits 1 with the #888 error for every non-'true' flag value.

  1. Revert util/release_status.py:222 to all([reconciled(...) for ...]). The PR turns the eager list into a short-circuiting generator, and neither the body nor the commit message mentions it. I confirmed the edit in the diff. It breaks parity with REFERENCE_STATUS in test_release_status_scripts_1967.py: across 82,944 states, base and head disagree in 13,680, all with an unset evidence variable. In 13,536 of those, base exits 1 (<VAR>: unbound variable, as the bash reference does) and head exits 0 with a warning. There's no production impact, since Actions sets every env: key, but it's an undisclosed contract change. It likely came from pylint R1729, and CI's pylint doesn't cover util/.
  2. Pin the guard with tests. Two mutants survive all 166 tests: dropping the every-publisher-skipped requirement from dev_release_skipped(), and == 'true' → != 'false'. Add a dev-flag case with a pypi failure (must not be quiet) and the 1.5.1 shape with the flag empty or unset (must exit 1), and add PCAPKIT_DEV_RELEASE to sampled_status_cases().
  3. Refresh the PR body. Its counts are stale, and it omits the unit-tests gate, the release_status.py change and the docs correction.

Not blocking: the publishers' != 'true' checks fail open if Get Version's output is empty, where == 'false' would fail closed. And on a v* tag push the notice still says "main's version".

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 30, 2026
…dev-release guard (#2452)

Against a842da0:

- release_status.py reconciles with all([...]) again, as before this PR:
  the generator short-circuited at the first unreconciled publisher and
  skipped reading later evidence, so an unset one exited 0 where the bash
  reference's set -u exits 1. A NOTE says why the list is deliberate.
- The dev-release notice says "this run's version", not "main's version",
  which was wrong on a v* tag push.
- New status cases kill three surviving mutants: a failure followed by a
  skipped publisher with unset evidence (generator), a dev flag with pypi
  failed (dropping the every-publisher-skipped check), and run
  41098750345's 1.5.1 shape with the flag empty and unset (== 'true' to
  != 'false'). sampled_status_cases() now draws PCAPKIT_DEV_RELEASE.
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verified directly (new head 0c56095b6, one commit on top of a842da007). util/release_status.py:226 is back to the eager all([...]), with a note explaining why. The new parity and guard cases each kill a mutant that survived the previous head (generator all, the dropped every-publisher-skipped check, == 'true' → != 'false'), and the notice no longer says "main's version". PR body updated. The == 'false' fail-closed change was deliberately not taken: it would flip the documented null-runs rows in three truth tables, so it gets its own decision rather than riding along here. Sonnet re-review dispatched.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Oct 30, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO (Sonnet, 0c56095b6). Parity with the bash reference holds across 364k states; every workflow mutant and the three targeted script mutants are killed; the 1.5.1 shape still exits 1 with #888. The two body errors it found (the "no human in the path" overstatement, and the fail-open null row described as "never a release") are fixed in the body.

Merge only after 1.5.1 publishes: this edits create-release.yml, so merging it now restarts the release chain.

@JarryShaw JarryShaw removed the review: running A cross-review is in flight against the current head - no verdict yet label Oct 30, 2026
@JarryShaw JarryShaw added the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 30, 2026
@JarryShaw
JarryShaw merged commit 2d23e48 into main Oct 30, 2026
65 checks passed
@JarryShaw
JarryShaw deleted the feat/gate-release-on-dev-version branch October 30, 2026 20:46
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 30, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant