Skip to content

t27b coverage: real tests for 13 cron cards in specs/crons (launchd, t27 and trinity) #8220

Closed

Description

@gHashTag

Part of #6063 (t27b coverage), which is item C10 of #6488 ("coverage counted without vacuous passes").

What is wrong

On the t27b lab, 26 cron cards of specs/crons pass vacuously (0 runtime asserts) under the reference and t27b: they have no test, invariant or bench block, which also breaks L4 TESTABILITY. This issue covers the other 13 (the first 13 are in the batch before): the four mac-trios-* launchd cards, two t27-* and seven trinity-* GitHub Actions cards.

What the tests check

A cron card is data, but specs/crons/README.md states its schema, and the tests hold each card to it:

  1. ID is <host>/<repo>/<slug>: it opens with HOST, then REPO; the slug is one path segment. HOST is one of the five hosts.
  2. ON_FAILURE and CONTROL come from their vocabularies. An empty RUNS (the control link) carries its reason in RUNS_NOTE.
  3. A SCHEDULE is a five-field cron expression.
  4. A GitHub Actions card's SERVICE is its workflow file, .github/workflows/<slug>.yml.
  5. A launchd card is code-only. A launch agent's SERVICE is its Label ai.t27.<x>, its ID is launchd/<repo>/<x>, and its INTERVAL_MS is whole seconds. A crontab line has SERVICE crontab and runs in the zone of the Mac (Asia/Bangkok).

Earlier attempt

See #6721: an open PR of an earlier attempt that adds tests to 6 of these 13 cards. Its tests restate each constant, which this lane rules out; it is superseded, not built on.

Done when

  • each card has these tests, passing under the reference (t27c test-report, 0 vacuous) and under t27b;
  • each is sealed;
  • the ledger rows that move come from bless of the branch run;
  • the signed lane receipts compare as IMPROVED_ONLY.

User Scenarios

  • Given a cron card in specs/crons/ with no test, invariant, or bench block, when t27c test-report runs, then the card passes vacuously with 0 runtime asserts and violates L4 TESTABILITY.
  • Given the 13 specified cron cards (four mac-trios-* launchd, two t27-*, seven trinity-* GitHub Actions), when each card has schema-validating tests added, then t27c test-report shows 0 vacuous passes for these cards and all tests pass under both the reference and t27b.
  • Given the test implementation avoids restating constants (per the lane rule that supersedes t27b checked: test blocks for 7 specs in specs/crons (6 of 8) #6721), when the cards are sealed and the branch is blessed, then the ledger rows move and signed lane receipts compare as IMPROVED_ONLY.

Requirements

  • FR-001: Each of the 13 cron cards MUST have at least one test block that exercises its schema constraints (ID format, vocabularies, SCHEDULE, SERVICE, INTERVAL_MS) so that t27c test-report reports 0 vacuous passes for these cards.
  • FR-002: Tests for GitHub Actions cards MUST assert that SERVICE equals .github/workflows/<slug>.yml and that ON_FAILURE and CONTROL values come from their declared vocabularies.
  • FR-003: Tests for launchd cards MUST assert code-only nature, SERVICE equals the Label ai.t27.<x>, ID format launchd/<repo>/<x>, and INTERVAL_MS is whole seconds; crontab-line cards MUST assert SERVICE is crontab and zone is Asia/Bangkok.
  • FR-004: All 13 cards MUST pass under the reference (t27c test-report) and under t27b with no vacuous passes.
  • FR-005: Each card MUST be sealed; the ledger updates MUST come from bless of the branch run; signed lane receipts MUST compare as IMPROVED_ONLY.

Boundary

  • specs/crons/README.md
  • specs/tools/tri/test.t27
  • specs/tools/trinity/tri/test.t27
  • specs/tools/trinity/tri/bench.t27
  • specs/fpga/coverage.t27
  • specs/queen/control.t27
  • specs/account/repo.t27
  • specs/tools/mcp/trinity.t27
  • cli/t27b/bench/bench.py
  • scripts/tri_loop/t27b.py

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions