Skip to content

fix(test): stop a fixed-date fixture colliding with a pinned relative age - #14390

Closed
bolichen97 wants to merge 1 commit into
mainfrom
fix/crew-webview-containment-retained-date
Closed

bolichen97 wants to merge 1 commit into
mainfrom
fix/crew-webview-containment-retained-date

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Frontend Tests (1) stops failing on main and on every PR under it, on a test that was red about nothing.

Frontend Tests (1) is red on main right now, and has been failing every open PR that rebases onto it. The failing case is
src/test/CrewWebviewContainment.test.tsx > crew webview containment > dates the retained document by the version on screen, not by the record that replaced it:

AssertionError: expected 'Couldn't refresh the dashboard. Show…' not to contain '3d ago'

Production code is correct. The test compares a FIXED fixture instant
(2026-09-04T05:16:00) against the LIVE clock, so the fixture's rendered age grows
without bound, while the republish offset it is checked against stayed pinned at
three days. On the day the fixture reached 23d ago, that string came to CONTAIN
3d ago, and expect(text).not.toContain(newWhen) failed.

Why it matters

It is a date bomb, not a one-off: the collision recurs every ten days — 3d, 13d,
23d, 33d — for as long as the fixture stays fixed. Each time, it reds a required
check on main and on every PR underneath it, and it reds it with a message that
points at the banner's copy rather than at the offsets that actually collided, so the
cost is a fresh investigation every time.

Not a goal

  • Changing CrewWebview.tsx or any other production code. The component renders the
    right two ages; only the test's comparison was wrong.
  • Re-pinning the clock for this test (the { now } approach
    IssueRadarCrewPage.test.tsx uses). The component calls fmtRelative without
    now, so pinning it here would need fake timers across an async
    waitFor/QueryClient flow — much wider than the defect.
  • Changing the fixture instant that the rest of this file shares.

What changed (motivation → approach → change)

Before After
Republish offset 3 days → 3d ago 5 hours → 5h ago, a different UNIT from the fixture's age
Guard above the assertions the two ages are UNEQUAL the two ages do not CONTAIN one another, either way

The assertions read one whole rendered sentence with toContain, so the relation they
need is non-containment; the guard asserted inequality, which a suffix relation
satisfies, and so handed the colliding pair straight to the assertion below it.

Moving the republish to hour scale makes the rendered unit letter differ (5h vs
23d, and 1mo/1y as the fixture ages further), so neither string can ever be a
substring of the other. Hours were chosen over seconds or minutes for the reason the
original offset was day-scale: the test computes its own rendering a moment after the
component's, and a finer scale can roll over between the two.

Asserting mutual non-containment is what turns a recurrence into an immediate, local
failure. Restoring the 3-day offset now fails on the guard —
expected '23d ago' not to contain '3d ago' — next to the two offsets a reader
would edit, instead of twenty lines lower in an assertion about banner copy.

Backwards compatibility

Compatible: test-only change, one file, no production code and no exported behavior.

Tests

The changed test is itself the test. Verified both directions:

  • With the fix: CrewWebviewContainment.test.tsx 44 passed; together with
    CrewWebviewContainmentCopy.test.ts, 60 passed.
  • Proving the guard catches the class: restoring the 3-day offset fails on the new
    guard with AssertionError: expected '23d ago' not to contain '3d ago' at the
    guard's own line, not in the assertion below.
  • tsc -p tsconfig.app.json, eslint on the changed file, and
    npm run i18n:check are all green.

Manual verification

N/A — the defect and the fix are both in an automated test, and the failure was
reproduced and cleared by running it.

Screenshots / video

Why no screenshot: test-only change; no component, layout, copy or rendered
output is touched.

Related Issues

no linked issue: found while driving PR #14369, whose Frontend Tests (1) inherited
this red from main.

Pattern harvest

Rule candidate: review-prompt
Pattern: a relative-time string compared with toContain, guarded only by
inequality — "23d ago" is unequal to "3d ago" and still contains it, so any test
that pairs a fixed fixture date with a clock-relative offset can collide as the
fixture ages. The general rule: when an assertion uses substring containment, the
non-vacuity guard must assert non-containment, not inequality.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

… age

`CrewWebviewContainment.test.tsx`'s retained-document test compared a FIXED
fixture instant (2026-09-04) against the LIVE clock, so the fixture's rendered
age grows without bound while the republish offset stayed pinned at three days.
On the day the fixture reached `23d ago` that string came to CONTAIN `3d ago`,
and the test's `expect(text).not.toContain(newWhen)` failed against correct
production code. It recurs every ten days (3d, 13d, 23d, 33d …).

The guard above the assertions only asserted the two ages were UNEQUAL, which a
suffix relation satisfies, so it passed the colliding pair straight through to
an assertion that reads one whole sentence with `toContain`.

- Republish at hour scale (5h), so the rendered UNIT differs from the fixture's
  day/month age and neither string can be a substring of the other. Hours do not
  tick within a test run, which is what the original day-scale offset was for.
- Assert the relation the test actually needs: mutual non-containment, not
  inequality. Restoring the 3-day offset now fails on that guard
  (`expected '23d ago' not to contain '3d ago'`), beside the offsets a reader
  edits, instead of in the assertion twenty lines below.

No production code changes.
@bolichen97

Copy link
Copy Markdown
Contributor Author

Intent: Clear Frontend Tests (1) on main by fixing the one test whose non-vacuity guard let a fixed fixture date and a clock-pinned relative age stand in a substring relation, so the failure stops recurring every ten days.
Not a goal: Touching CrewWebview.tsx or any production code; re-pinning the component's clock with fake timers; changing the fixture instant the rest of the file shares.

@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 173d6d80efe5acba61fa3d356821e8dd01d9c738 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Verify no sibling test pairs the shared fixed fixture with a clock-relative offset the same way; this guard covers only this pair.

[DESIGN-REVIEWED] 173d6d8

@github-actions

Copy link
Copy Markdown
Contributor

UX Review (Fable 5) — ✅ PASS

UX-level review of 173d6d80efe5acba61fa3d356821e8dd01d9c738 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

UX-Verdict: PASS

Nothing to check.

[UX-REVIEWED] 173d6d8

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 173d6d80efe5acba61fa3d356821e8dd01d9c738 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 173d6d8

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 173d6d80efe5acba61fa3d356821e8dd01d9c738: <one-sentence reason>

@bolichen97

Copy link
Copy Markdown
Contributor Author

This PR and #14381 each hold the only fix for the other's remaining red, so neither can reach a green PR Readiness on its own.

PR Frontend Tests (1) Backend Tests (3.12 / Windows) (3)
#14390 (this one — the test fix) green, all 4 shards red: prepare-pr/SKILL.md 68015 > 68000
#14381 (the SKILL.md trim to 67994) red: CrewWebviewContainment expected '23d ago' not to contain '3d ago' green

Both reds are inherited from main — #14181 grew SKILL.md past its ratchet at 2026-09-26 22:18 PDT, and the same commit's template change is unrelated to the test collision. Verified directly: #14381's Frontend Tests (1) log ends in exactly the assertion this PR fixes (1 failed | 598 passed), and this PR's own Frontend Tests shards are all green, so each fix demonstrably works.

Nothing here needs another rebase: the two PRs are already on the same base and rebasing either one re-lands on the other's red.

The cycle breaks in one of three ways, and all three are a maintainer's call:

  1. Fold the 15-byte SKILL.md trim into this PR, making it self-sufficiently green; ci: add the Intent Lock check for a PR's frozen goal #14381 then goes green on its next rebase, with a prose conflict in SKILL.md to resolve since it edits the same file.
  2. The reverse — add this test fix to ci: add the Intent Lock check for a PR's frozen goal #14381 and close this PR.
  3. Merge one of them past the inherited red, which needs an override this branch will not ask for.

@bolichen97 option 1 is the smallest and the one I would take, but it widens this PR's stated goal, so I am not doing it unprompted.

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed 173d6d80efe5acba61fa3d356821e8dd01d9c738 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 173d6d8

Verdict parsed from the review's SHA-scoped output markers for commit 173d6d80efe5acba61fa3d356821e8dd01d9c738.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 173d6d80efe5acba61fa3d356821e8dd01d9c738: <one-sentence reason>

@bolichen97

Copy link
Copy Markdown
Contributor Author

Closing: superseded on main by #14398 (fix(ci): unbreak the two base-side gates red on every open PR), which landed at a83d2258a2's ancestry and fixes the same failure a different way — it asserts on the whole interpolated sentence (band(shownWhen) / band(newWhen) built from en.json) instead of the bare relative stamp, so "23d ago" can no longer contain "3d ago".

Same root cause, same class of fix, so there is nothing left for this PR to carry. Frontend Tests (1) is green on current main; verified locally on a clean origin/main worktree (CrewWebviewContainment.test.tsx, 44 tests, all passing).

For the record, the diagnosis this PR carried: the case compared a fixed fixture date against a republish offset pinned to the live clock, so the fixture's age grew without bound and reached a value whose rendering (23d ago) is a suffix of the pinned one (3d ago). It recurs every 10 days (3d / 13d / 23d / 33d), and the inequality guard could not catch it because it asserted the two strings were not equal — which a suffix relationship satisfies. #14398 closes that hole at the assertion site.

@bolichen97 bolichen97 closed this Sep 27, 2026
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 27, 2026
@bolichen97
bolichen97 deleted the fix/crew-webview-containment-retained-date branch November 20, 2026 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant