Skip to content

make three timing-dependent tests assert behaviour instead of wall-clock bounds - #237

Merged
mauricecarrier7 merged 3 commits into
mainfrom
fix/timing-dependent-tests
Oct 1, 2026
Merged

mauricecarrier7 merged 3 commits into
mainfrom
fix/timing-dependent-tests

Conversation

@mauricecarrier7

Copy link
Copy Markdown
Contributor

What

Three toolkit tests failed on CI because of wall-clock timing, not because of a code defect. Each one now waits for the event it is about, or uses an injected clock.

Test What it depended on Now
AudiobookNetworkServiceTest.testDownloadProgressWithEmptyTracks A 1 s asyncAfter on main had to fire within a 2 s wait. On iOS 26.5 it did not (runs of 8.0 s and 17.9 s). Waits on whenPendingWorkDrains, a new internal hook on DefaultAudiobookNetworkService that runs a block after the queue's pending barriers and their main-queue hops. It then asserts that nothing was published.
AudiobookPlaybackModelAutosaveTests.test_saveSuppression_whenItLapses_theNextDueTickWrites 20 s of simulated ticks had to be delivered inside a real 0.4 s suppression window. AudiobookPlaybackModel reads the window through wallClock, which defaults to Date(). The test pins that clock while the window holds and then advances it.
LCPResourceLoaderStallTests.testContentInfo_WhenLengthLookupFailsLate_FinishesWithoutLength A 0.4 s lookup had to outlast a 0.1 s stall timeout. Its 2-byte range also raced the data phase's 0.1 s timer. It failed after 37.7 s on ios-core's macos-15 run 36911638959. The lookup parks until the test releases it. The test first checks that the request is still open after five stall timeouts. The request asks for content information only, and the final wait is a 30 s hang bound.

Why

ios-core is moving CI to Xcode 26.6 and iOS 26.5, and its toolkit-tests job failed on these tests. The third test also failed on the current macos-15 runner, so it is not specific to iOS 26.

Production changes

Two small seams, both with unchanged default behaviour:

  • DefaultAudiobookNetworkService.whenPendingWorkDrains(_:): internal and only called by tests.
  • AudiobookPlaybackModel.wallClock: defaults to Date(). Only the save-suppression window reads it. The other suppression windows still call Date() directly.

Verification

  • Full suite, run locally on an iOS 26.2 simulator with Xcode 26.3: Executed 315 tests, with 1 test skipped and 0 failures, ** TEST SUCCEEDED **.
  • The three tests ran 10 iterations each (30 executions, 0 failures).
  • Each fixed test was checked against a deliberately broken version of the behaviour it covers. All five changes made it fail on a named assertion:
    • The empty service publishes .downloadComplete at startup: the network test fails because it received [downloadComplete].
    • A suppressed tick moves the autosave baseline: XCTUnwrap fails because the first due tick did not write.
    • The suppression check is removed: the premise assertion fails.
    • A stall timer is added to the content-information phase: XCTAssertNil(outcome) fails with transferStalled.
    • A failed lookup reports a length of 0: XCTAssertNil(contentLength) fails.

Not done: the other suppression windows in AudiobookPlaybackModel still read Date() directly, and I did not change any other wall-clock tests in the suite.

testDownloadProgressWithEmptyTracks asserted that nothing was published by
waiting 1 s on main with a 2 s timeout. On iOS 26.5 CI the main queue did not
run that block within 2 s (runs of 8.0 s and 17.9 s), so the test failed while
the service published nothing.

The test now waits on whenPendingWorkDrains, a new internal hook that runs a
block after every barrier already on the service's queue and the main-queue
hops those blocks scheduled. The 30 s timeout is a hang bound, not the window
being measured.
test_saveSuppression_whenItLapses_theNextDueTickWrites needed 20 s of
simulated ticks to be delivered within a real 0.4 s window. On a slow run the
window lapsed part way through and the premise assertion failed.

AudiobookPlaybackModel reads the suppression window through wallClock, which
defaults to Date(), so production behaviour is unchanged. The test pins the
clock while the window should hold and advances it to lapse the window.
testContentInfo_WhenLengthLookupFailsLate_FinishesWithoutLength relied on a
0.4 s lookup outlasting a 0.1 s stall timeout, and its 2-byte data range raced
the data phase's own 0.1 s stall timer. On ios-core's macos-15 run 36911638959
it took 37.7 s and reported the request unfinished after the 5 s bound.

The lookup now parks until the test releases it, after at least five stall
timeouts have passed with the request still open. The request asks for
content information only, so no data-phase timer is involved, and the final
wait is a 30 s hang bound.
@mauricecarrier7
mauricecarrier7 merged commit 9a80250 into main Oct 1, 2026
3 checks passed
@mauricecarrier7
mauricecarrier7 deleted the fix/timing-dependent-tests branch October 1, 2026 21:38
mauricecarrier7 added a commit to ThePalaceProject/ios-core that referenced this pull request Oct 1, 2026
Picks up ThePalaceProject/ios-audiobooktoolkit#237. That PR rewrites three
toolkit tests that failed this branch's toolkit-tests job on iOS 26.5, and
macos-15, because they depended on wall-clock windows. It also brings in #235
and #236, which change only the toolkit's own CI workflow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants