Repository navigation
t27b coverage: real tests for 9 function cards in specs/functions (analytics to crm) #8242
Description
Part of #6063 (t27b coverage), which is item C10 of #6488 ("coverage counted without vacuous passes").
What is wrong
On the t27b lab, 18 function cards of specs/functions pass vacuously (0 runtime asserts) under the reference and t27b: they have no test, invariant or bench block, which also breaks L4 TESTABILITY. This issue covers the first 9 in file order: analytics-sales-advise, analytics-skills-detect, broadcast-message-send, client-telemetry-daily, club-invoice-abandoned-watch, content-detailed-script-generate, content-scenario-clips-generate, content-scripts-generate, crm-proactive-sweep.
What the tests check
specs/functions/README.md states the schema of a function card and the rules its generator enforces. The tests hold each card to them:
IDis<domain>-<object>-<verb>, withDOMAINfrom the README's vocabulary as its first part.TRIGGERdecides the rest. An event function has anEVENTandCRON = "", and its canonicalEVENTnames the same three parts asID(<domain>/<object>.<verb>). A cron function has a five-fieldCRON,EVENT = ""and noLEGACY_EVENTS.ON_FAILURE,PROBE_RESULT,CONTROLand everySIDE_EFFECTSvalue come from their vocabularies; no side effect is listed twice.KIND,REPOandTZare as the schema states.SAFE_PROBEagrees withPROBE_RESULT.""means no probe was sent (skipped, ornot-deployedwithGUARD = unknown). A JSON object means the probe reached the function (COMPLETED, orFAILED-at-guardwith a namedGUARD).SERVICEisfile:line.
Cards that break rule 1 (see #7911) keep rules 2 to 5 and skip rule 1.
Earlier attempt
See #6733: an open PR of an earlier attempt that adds tests to 7 of these 9 cards. Its tests restate each constant, which this lane rules out; it is superseded, not built on.
Done when
- each card has these tests, passing under the reference (
t27c test-report, 0 vacuous) and under t27b; - each is sealed;
- the ledger rows that move come from
blessof the branch run; - the signed lane receipts compare as IMPROVED_ONLY.
User Scenarios
- Given the 9 function cards in
specs/functionshave notest,invariant, orbenchblocks and pass vacuously, when proper runtime tests are added to each card, then each card executes at least one assert and passes under both the reference compiler (t27c test-report) and t27b with zero vacuous passes.
Requirements
- FR-001: Each of the 9 function cards MUST contain at least one
test,invariant, orbenchblock that performs runtime assertions. - FR-002: The added tests MUST adhere to the function card schema and vocabulary rules documented in
specs/functions/README.md(ID format, TRIGGER semantics, vocabularies forON_FAILURE,PROBE_RESULT,CONTROL,SIDE_EFFECTS,SAFE_PROBEagreement,SERVICElocation). - FR-003: All 9 cards MUST pass
t27c test-reportwith 0 vacuous passes and also pass under t27b. - FR-004: Each card MUST be sealed after tests are added.
- FR-005: Ledger row updates MUST originate from
blessof the branch run, and signed lane receipts MUST compare as IMPROVED_ONLY.
Boundary
- specs/functions/analytics-sales-advise.t27
- specs/functions/analytics-skills-detect.t27
- specs/functions/broadcast-message-send.t27
- specs/functions/client-telemetry-daily.t27
- specs/functions/club-invoice-abandoned-watch.t27
- specs/functions/content-detailed-script-generate.t27
- specs/functions/content-scenario-clips-generate.t27
- specs/functions/content-scripts-generate.t27
- specs/functions/crm-proactive-sweep.t27
- specs/functions/README.md
Activity
Merge master into vac/functions-1; carry the tests onto #8023's card (C…
Merge master into vac/functions-1 (Closes #8242)
specs(functions): real tests for 9 function cards, analytics to crm (C…