You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
t27b vs t27c: the two runtime-assert counters disagree (string vs bool/int) #8074
t27c test-report (the reference) and t27b count runtime asserts differently, so the two backends disagree on which specs are vacuous:
t27c counts a string comparison against a constant as a runtime assert, and does not count a bool or integer comparison.
t27b counts the bool or integer comparison, and does not count the string comparison.
Example: specs/crons/999-multibots-telegraf-log-monitor.t27 has 4 string asserts and 1 bool assert. The reference reports 4 runtime asserts, t27b reports 1.
Effect: a spec can be vacuous in one backend and checked in the other (the specs in #8062 are vacuous only in t27b). The honest metric t27b_pass_where_reference_passes / reference_pass and the pass_vacuous ledger state both depend on this count.
Boundary
The decision of what counts as a runtime assert, written once in a .t27 spec, used by both counters.
Requirements
One rule, stated in a .t27 spec with tests, decides whether an assert is a runtime assert (string, bool, integer, float).
t27c and t27b report the same runtime-assert count on the same spec.
Success Criteria
On specs/crons/999-multibots-telegraf-log-monitor.t27, t27c test-report and t27b print the same runtime-assert count.
The lab run lists 0 specs that are vacuous in one backend and checked in the other.
Found by the vacuous-spec cleanup lane (#8052, #8054). Refs #6063
User Scenarios
Given a spec with string comparisons against constants and bool/integer comparisons, when t27c and t27b both run test-report on that spec, then both backends report the identical runtime-assert count.
Requirements
FR-001: A single .t27 spec MUST define the rule for what constitutes a runtime assert (string, bool, integer, float).
FR-002: t27c MUST read the rule from that spec and count runtime asserts accordingly.
FR-003: t27b MUST read the rule from that spec and count runtime asserts accordingly.
FR-004: Both backends MUST produce the same runtime-assert count for any given spec.
Success Criteria
t27c test-report specs/crons/999-multibots-telegraf-log-monitor.t27 and t27b specs/crons/999-multibots-telegraf-log-monitor.t27 print the same runtime-assert count (exit 0 for both, counts equal)
Lab run lists 0 specs that are vacuous in one backend and checked in the other (0 specs divergent)
Activity