Skip to content

ci: report Experimental Tests and Python Compatibility to Discussions, with step summaries (#2030) - #2071

Merged
JarryShaw merged 1 commit into
mainfrom
ci/2030-shared-reporter
Oct 23, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
ci/2030-shared-reporter

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • ci — changes to CI configuration files and scripts

Description of your pull request and other information

For #2030, per the rulings there.

  • Shared reporter: the new util/check_suite_report.py replaces coverage-comment.yml's inline Write the run report. Unit Tests keeps its discussion job. The new ci-reports.yml relays Experimental Tests and Python Compatibility through workflow_run, with the same steps; a test holds the two identical. The three tested workflows gain no write permission.
  • Step summaries: each summary-writing step also copies its summary to $RUNNER_TEMP (through _actions, JOB_SUMMARY_COPY or tee). A final pair of always() steps uploads it as job-summary-<job>-<index>. The reporter folds each into a <details> block, trimmed to fit the comment limit with a link to the run.
  • Push rule: a green push run warns only for a failed or cancelled job, or a warning on a (KTLO) or (experimental) leg. Scheduled runs still post every run, and the recovery post is unchanged.
  • pip retries: Retrying … after connection is ignored on every leg.
  • No post when every gated job was skipped. By-design lists are empty and pinned.
  • Measured against main's inline step: a passing job's warning and a pip retry now give success. A KTLO warning still gives warning.
  • Results: the project leg ran 1980 tests with 0 failures. actionlint is clean, and the 21 reader modules pass after the rebase.

Two choices to confirm. I recommend keeping both.

  1. Engine KTLO cells: warnings titled (KTLO, non-blocking) also count as non-blocking. That covers the KTLO cells inside the blocking Engines (…) jobs, which would otherwise go silent on push.
  2. Two relays: Unit Tests stays in coverage-comment.yml, rather than moving all three into ci-reports.yml.

Not verifiable before a live run: gh run download --pattern layout, actions: read sufficing for the relay, and how <details> renders.

@JarryShaw JarryShaw added enhancement Issues requesting a new capability (set by the feature request template) ci Pull requests that change CI or workflow configuration (ci: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 23, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at da6a4a38e (Sonnet cross-review).

Security

  • unit-tests.yml gains no permission or secret use.
  • ci-reports.yml is permissions: {}. Its job sparse-checks-out the default branch's scripts, never the triggering run's code, and runs only for push and scheduled runs, so a fork PR can't feed a post.
  • Artifacts are read as capped UTF-8 text only. The status marker is appended last and read last, so a summary can't spoof the thread state.

Behaviour

  • The reviewer put 13 Unit Tests push scenarios through main's inline reporter and the script. Status, title and body are byte-identical apart from the intended changes: a passing blocking job's annotation, a pip retry, and an empty-title runner note on a push run now give success.
  • A green run where every gated job was skipped no longer posts.
  • The (KTLO, non-blocking) title rule is needed: Engines (…) jobs can't be seen by the job-name rule.

Summaries: 400 fuzzed runs (multi-byte text, </details>, 200 KB bodies) all stayed within the comment limit.

Unverified until a live post: a summary containing &#64;name might render as a mention, because the @ guard only catches a literal @. Only users with write access can trigger these runs.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 23, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 91.52% (unit tier, Python 3.15, da6a4a38e, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 19599 881 2618 867 92.11%
pcapkit/corekit 2565 71 812 37 96.56%
pcapkit/dumpkit 282 3 102 3 98.44%
pcapkit/foundation 2930 23 1022 25 98.79%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 17414 219 4574 185 98.14%
pcapkit/toolkit 869 36 298 10 95.20%
pcapkit/utilities 473 7 136 5 98.03%
pcapkit/vendor 4431 1995 1018 167 51.73%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Ready to merge at da6a4a38e: the review is GOOD TO GO and CI is complete (56 passed, 0 failed). The ordering leg ran 2026 tests with 0 failures. The 3.8–3.10 KTLO legs show 0 failing tests, and the PR is based on current main (fd0d2e724). The two choices in the description (the engine KTLO title rule, and keeping two relays) are still yours to confirm; I recommend keeping both. #2069 and #2070 also add to the SCRIPTS tuple, so whichever merges after this will need a small rebase, which I'll do.

…, with step summaries (#2030)

util/check_suite_report.py now writes the check-suite report coverage-comment.yml's discussion job wrote inline, and ci-reports.yml, a workflow_run relay, runs the same steps for Experimental Tests and Python Compatibility, so none of the three gains a write scope.

- Push runs: an annotation counts only on a (KTLO) or (experimental) leg, or one titled (KTLO, non-blocking); a passing blocking job's no longer posts. Scheduled runs count every annotation and still post every run.
- pip's 'Retrying ... after connection' lines never count, on any leg.
- A run whose gated legs all skipped (current or changes.code false) posts nothing.
- Each job's summary is copied to job-summary.md (util/_actions.py's JOB_SUMMARY_COPY, or tee), named and uploaded on always(); the report folds each into a <details> block, trimmed to fit with a link to the run.
@JarryShaw
JarryShaw force-pushed the ci/2030-shared-reporter branch from da6a4a3 to d1103d7 Compare October 23, 2026 04:04
@JarryShaw

Copy link
Copy Markdown
Owner Author

Rebased onto main (69562f99d), da6a4a38e → d1103d7da, to resolve the conflict with #2069. The only conflict was SCRIPTS and RUNS in test_actions_helper_2047, where both sides added names. The resolution is their sorted union (#2069's update_python_versions plus check_suite_report from here). git patch-id --stable is identical outside that file. The helper, KTLO-guard, workflow-script, docs-claims and python-versions tests pass on the rebased tree. The old head's CI was green (56 passed, 0 failed), with 0 KTLO failures. Only that test tuple changed, so you can bypass checks and merge.

@JarryShaw
JarryShaw merged commit 27288aa into main Oct 23, 2026
21 checks passed
@JarryShaw
JarryShaw deleted the ci/2030-shared-reporter branch October 23, 2026 04:05
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 23, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 23, 2026
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix) enhancement Issues requesting a new capability (set by the feature request template)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant