tri epic-ci: which checks of an epic's pull requests turned red since the last snapshot (scripts/tri_loop/epic_ci.py) #6087

Context

tri epic-ci is a new loop helper: it reads two snapshots of pull requests' checks -- now, and the snapshot a watch kept from its previous run -- and prints per pull request how many checks are red, pending and green, then which checks turned red since the previous snapshot.

Why: the review-loop watch (cron ticks on epics #5906 and #5933, 2026-10-04) has one recurring question, "which of the epic's pull requests went newly red since the last tick?", and answered it by hand each time from gh pr view --json statusCheckRollup output. A fixed card with an exit code turns that into one command whose change is visible as a changed line.

The input is what gh already prints, so no GitHub call is made by the tool:

for n in 5970 6011; do gh pr view $n -R gHashTag/t27 --json number,headRefOid,statusCheckRollup; done > now.json

The tool only reads files. It never calls GitHub itself, so it runs anywhere, including in a test.

What to write

1. scripts/tri_loop/epic_ci.py -- the tool. Line 2 of the file is its one-line summary in the form """tri epic-ci -- <summary> (that is what tri loop-help prints). Standard library only.
   • --now PATH (required) and --previous PATH (optional). Each file holds either one JSON array of pull-request objects, or pull-request objects one after another (pretty-printed or not), as repeated gh pr view N --json number,headRefOid,statusCheckRollup prints them. Read them with json.JSONDecoder().raw_decode in a loop, skipping whitespace between values.
   • Each statusCheckRollup entry is one check:
     • __typename StatusContext: the name is context; state SUCCESS is green; FAILURE or ERROR is red; anything else is pending.
     • otherwise (a CheckRun): the name is name; status other than COMPLETED is pending; conclusion SUCCESS, NEUTRAL or SKIPPED is green; FAILURE, TIMED_OUT, CANCELLED, ACTION_REQUIRED or STARTUP_FAILURE is red; anything else is pending.
   • Output:

     prs: <count>
     #<number> <first 7 of headRefOid> red=<r> pending=<p> green=<g>[ [red: <red check names, sorted, unique, comma-separated>]]
     ...one line per pull request, ascending by number...
     newly red: <...>
     

     The [red: ...] part is printed only when r > 0.
   • newly red: a check is newly red when it is red now and was not red on the same pull request number in --previous (whatever head the previous snapshot had). Printed as #<number> <check name> items, ascending by number then name, joined with , ; none when there are none; NOT ESTABLISHED (no --previous given) without --previous.
   • Exit 1 when at least one check is newly red; 0 otherwise (including without --previous); 2 when a file cannot be read or does not parse as JSON (and it prints why).
2. scripts/ci/fixtures/epic_ci/now.json -- exactly:

   [
    {"number": 5970, "headRefOid": "1111111aaaaaaa", "statusCheckRollup": [
      {"__typename": "CheckRun", "name": "parse-ratchet", "status": "COMPLETED", "conclusion": "SUCCESS"},
      {"__typename": "CheckRun", "name": "spec-guards", "status": "COMPLETED", "conclusion": "FAILURE"},
      {"__typename": "CheckRun", "name": "test-ratchet", "status": "IN_PROGRESS", "conclusion": ""},
      {"__typename": "StatusContext", "context": "GitGuardian Security Checks", "state": "SUCCESS"}]},
    {"number": 6011, "headRefOid": "2222222bbbbbbb", "statusCheckRollup": [
      {"__typename": "CheckRun", "name": "parse-ratchet", "status": "COMPLETED", "conclusion": "SUCCESS"},
      {"__typename": "CheckRun", "name": "check", "status": "COMPLETED", "conclusion": "TIMED_OUT"},
      {"__typename": "CheckRun", "name": "coverage", "status": "COMPLETED", "conclusion": "SKIPPED"},
      {"__typename": "StatusContext", "context": "ci/railway", "state": "ERROR"}]}
   ]
   

3. scripts/ci/fixtures/epic_ci/previous.json -- exactly (two objects one after another, not an array):

   {"number": 5970, "headRefOid": "0000000aaaaaaa", "statusCheckRollup": [
     {"__typename": "CheckRun", "name": "spec-guards", "status": "COMPLETED", "conclusion": "FAILURE"},
     {"__typename": "CheckRun", "name": "test-ratchet", "status": "COMPLETED", "conclusion": "SUCCESS"}]}
   {"number": 6011, "headRefOid": "2222222bbbbbbb", "statusCheckRollup": [
     {"__typename": "CheckRun", "name": "check", "status": "IN_PROGRESS", "conclusion": ""}]}
   

4. scripts/ci/test_epic_ci_names_newly_red_checks.py -- runs the tool on the fixtures in the three ways below and on a missing file and a file holding {not json; asserts the exact lines and exit codes. It finds the tool and the fixtures relative to its own file (Path(__file__).resolve().parents[2] is the repository root), never by an absolute path.

Behaviour the test asserts

--now scripts/ci/fixtures/epic_ci/now.json --previous scripts/ci/fixtures/epic_ci/previous.json prints exactly the following and exits 1:

prs: 2
#5970 1111111 red=1 pending=1 green=2 [red: spec-guards]
#6011 2222222 red=2 pending=0 green=2 [red: check, ci/railway]
newly red: #6011 check, #6011 ci/railway

--now scripts/ci/fixtures/epic_ci/now.json alone prints the same first three lines, then newly red: NOT ESTABLISHED (no --previous given), and exits 0.

--now and --previous both scripts/ci/fixtures/epic_ci/now.json print the same first three lines, then newly red: none, and exit 0.

A missing --now file, and a --now file holding {not json, exit 2.

User Scenarios & Testing

• Given the previous tick saw check still running on hooks: pre-commit trusts a PATH tri only if it has hooks pre-commit (#6009) #6011 and now it timed out, When the watch runs tri epic-ci --now now.json --previous previous.json, Then the last line names #6011 check and the exit code is 1.
• Given ci: every check is a .t27 card -- schema, generator, three pilots (P1 of #5933) #5970's spec-guards was red in the previous snapshot and is still red on a new head, When the same command runs, Then spec-guards is listed in ci: every check is a .t27 card -- schema, generator, three pilots (P1 of #5933) #5970's [red: ...] part but not under newly red.

Requirements

• FR-001: The tool MUST only read the files it is given (no network call).
• FR-002: The tool MUST classify CheckRun and StatusContext entries exactly as listed above.
• FR-003: The tool MUST exit 1 when a check is newly red, 0 otherwise, and 2 on an unreadable or non-JSON file.
• FR-004: The test MUST locate the tool and fixtures relative to its own path, never by an absolute path.

Original checks (kept for history; the Queen does not run these)

• test -f scripts/tri_loop/epic_ci.py && echo present prints present
• grep -c 'tri epic-ci -- ' scripts/tri_loop/epic_ci.py prints at least 1
• grep -c '"number"' scripts/ci/fixtures/epic_ci/now.json prints 2
• grep -c '"number"' scripts/ci/fixtures/epic_ci/previous.json prints 2
• test -f scripts/ci/test_epic_ci_names_newly_red_checks.py && echo present prints present
• grep -c -F 'newly red: #6011 check, #6011 ci/railway' scripts/ci/test_epic_ci_names_newly_red_checks.py prints at least 1
• grep -c workspace scripts/ci/test_epic_ci_names_newly_red_checks.py prints 0
• python3 scripts/ci/test_epic_ci_names_newly_red_checks.py exits 0
• python3 scripts/tri_loop/epic_ci.py --now scripts/ci/fixtures/epic_ci/now.json --previous scripts/ci/fixtures/epic_ci/previous.json exits 1

Original file list

• scripts/tri_loop/epic_ci.py
• scripts/ci/test_epic_ci_names_newly_red_checks.py
• scripts/ci/fixtures/epic_ci/now.json
• scripts/ci/fixtures/epic_ci/previous.json

Refs #5776 (review-speed plan row B31). Epic: #5906.

Queen-runnable restatement (2026-10-05)

The original asks for a hand-written Python tool, a Python test and two fixtures under scripts/; under the only-t27 rule (specs/policy/own_language.t27) a bee may not add .py files. What survives in t27 is the tool's decision logic as a spec with tests: how one check is classified, when a check is newly red, and which exit code results. The file reading stays out of scope until t27 can generate the CLI.

Boundary

• specs/tools/epic_ci.t27

User Scenarios & Testing

• Given master has no epic-ci logic in t27, when specs/tools/epic_ci.t27 lands, then module ToolsEpicCi classifies a CheckRun and a StatusContext entry into red, pending or green exactly as the lists above say, using named numeric codes for statuses and conclusions.
• Given the fixture of this issue (ci: every check is a .t27 card -- schema, generator, three pilots (P1 of #5933) #5970: spec-guards red before and after; hooks: pre-commit trusts a PATH tri only if it has hooks pre-commit (#6009) #6011: check pending before and TIMED_OUT now, ci/railway ERROR now), when the spec's tests run, then ci: every check is a .t27 card -- schema, generator, three pilots (P1 of #5933) #5970 counts red=1 pending=1 green=2, hooks: pre-commit trusts a PATH tri only if it has hooks pre-commit (#6009) #6011 counts red=2 pending=0 green=2, spec-guards is not newly red, both hooks: pre-commit trusts a PATH tri only if it has hooks pre-commit (#6009) #6011 checks are newly red, and the exit code is 1; with no previous snapshot the exit code is 0.

Requirements

• FR-001: The spec MUST define pub fn classify_check_run(completed: bool, conclusion: u32) -> u32 and pub fn classify_status_context(state: u32) -> u32 returning named codes for red, pending and green, following FR-002 of the original.
• FR-002: The spec MUST define pub fn is_newly_red(red_now: bool, red_before: bool) -> bool and pub fn exit_code(newly_red: u32, has_previous: bool, unreadable: bool) -> u32 returning 2 for an unreadable input, 1 when at least one check is newly red, and 0 otherwise (including with no previous snapshot).
• FR-003: Tests MUST encode the fixture of this issue with concrete values, including the SKIPPED-is-green and TIMED_OUT-is-red cases.
• FR-004: The change MUST NOT add or modify non-t27 hand-written code (specs/policy/own_language.t27); no .py file and no fixture under scripts/.

Success Criteria

• grep -c 'module ToolsEpicCi' specs/tools/epic_ci.t27 prints 1
• grep -cE 'pub fn (classify_check_run|classify_status_context|is_newly_red|exit_code)\(' specs/tools/epic_ci.t27 prints 4
• grep -cE '^[[:space:]]*test ' specs/tools/epic_ci.t27 prints at least 4
• t27c spec-status specs/tools/epic_ci.t27 prints IMPLEMENTED
• t27c test-report specs/tools/epic_ci.t27 2>&1 | grep -c BLOCKED prints 0