Repository navigation
t27b coverage: real tests for 13 cron cards in specs/crons (launchd, t27 and trinity) #8220
Copy link
Copy link
Closed
Labels
Description
Part of #6063 (t27b coverage), which is item C10 of #6488 ("coverage counted without vacuous passes").
What is wrong
On the t27b lab, 26 cron cards of specs/crons pass vacuously (0 runtime asserts) under the reference and t27b: they have no test, invariant or bench block, which also breaks L4 TESTABILITY. This issue covers the other 13 (the first 13 are in the batch before): the four mac-trios-* launchd cards, two t27-* and seven trinity-* GitHub Actions cards.
What the tests check
A cron card is data, but specs/crons/README.md states its schema, and the tests hold each card to it:
IDis<host>/<repo>/<slug>: it opens withHOST, thenREPO; the slug is one path segment.HOSTis one of the five hosts.ON_FAILUREandCONTROLcome from their vocabularies. An emptyRUNS(the control link) carries its reason inRUNS_NOTE.- A
SCHEDULEis a five-field cron expression. - A GitHub Actions card's
SERVICEis its workflow file,.github/workflows/<slug>.yml. - A launchd card is
code-only. A launch agent'sSERVICEis its Labelai.t27.<x>, itsIDislaunchd/<repo>/<x>, and itsINTERVAL_MSis whole seconds. A crontab line hasSERVICEcrontaband runs in the zone of the Mac (Asia/Bangkok).
Earlier attempt
See #6721: an open PR of an earlier attempt that adds tests to 6 of these 13 cards. Its tests restate each constant, which this lane rules out; it is superseded, not built on.
Done when
- each card has these tests, passing under the reference (
t27c test-report, 0 vacuous) and under t27b; - each is sealed;
- the ledger rows that move come from
blessof the branch run; - the signed lane receipts compare as IMPROVED_ONLY.
User Scenarios
- Given a cron card in
specs/crons/with notest,invariant, orbenchblock, whent27c test-reportruns, then the card passes vacuously with 0 runtime asserts and violates L4 TESTABILITY. - Given the 13 specified cron cards (four
mac-trios-*launchd, twot27-*, seventrinity-*GitHub Actions), when each card has schema-validating tests added, thent27c test-reportshows 0 vacuous passes for these cards and all tests pass under both the reference and t27b. - Given the test implementation avoids restating constants (per the lane rule that supersedes t27b checked: test blocks for 7 specs in specs/crons (6 of 8) #6721), when the cards are sealed and the branch is blessed, then the ledger rows move and signed lane receipts compare as IMPROVED_ONLY.
Requirements
- FR-001: Each of the 13 cron cards MUST have at least one
testblock that exercises its schema constraints (ID format, vocabularies, SCHEDULE, SERVICE, INTERVAL_MS) so thatt27c test-reportreports 0 vacuous passes for these cards. - FR-002: Tests for GitHub Actions cards MUST assert that
SERVICEequals.github/workflows/<slug>.ymland thatON_FAILUREandCONTROLvalues come from their declared vocabularies. - FR-003: Tests for launchd cards MUST assert
code-onlynature,SERVICEequals the Labelai.t27.<x>,IDformatlaunchd/<repo>/<x>, andINTERVAL_MSis whole seconds; crontab-line cards MUST assertSERVICEiscrontaband zone isAsia/Bangkok. - FR-004: All 13 cards MUST pass under the reference (
t27c test-report) and under t27b with no vacuous passes. - FR-005: Each card MUST be sealed; the ledger updates MUST come from
blessof the branch run; signed lane receipts MUST compare as IMPROVED_ONLY.
Boundary
- specs/crons/README.md
- specs/tools/tri/test.t27
- specs/tools/trinity/tri/test.t27
- specs/tools/trinity/tri/bench.t27
- specs/fpga/coverage.t27
- specs/queen/control.t27
- specs/account/repo.t27
- specs/tools/mcp/trinity.t27
- cli/t27b/bench/bench.py
- scripts/tri_loop/t27b.py
Reactions are currently unavailable
Activity