Skip to content

Flaky test: TestWatcherStartIsIdempotent — 'watcher loop did not start' under CI load #834

Description

@HuggeK

TestWatcherStartIsIdempotent (go/internal/configreload) failed on CI with:

--- FAIL: TestWatcherStartIsIdempotent (1.01s)
    watcher_test.go:267: watcher loop did not start
FAIL	github.com/srcfl/ftw/go/internal/configreload	3.836s

Evidence it is a flake, not a regression:

Likely mechanism — baseline drift in the goroutine count. The test samples
watcherLoopCount() as a baseline, calls Start(), then waits for the count to
exceed the baseline:

baselineLoops := watcherLoopCount()
w.Start()
waitForWatcherLoop(t, baselineLoops)

The count is derived from runtime goroutine stacks, which is global. A watcher
from a previously-run test that is still winding down when the baseline is taken
will decrement the count when it exits — masking the new loop's increment, so
waitForWatcherLoop times out even though the loop started. The failed run's log
supports concurrent watcher activity: a stray
WARN config reload failed err="yaml: …" from another test's watcher appears
mid-test.

Possible fixes: count loops per-watcher (injected counter/callback on the
loop's entry/exit) instead of scanning global goroutine stacks; or make Stop()
join the loop goroutine and have each test fully stop its watcher before the
next starts; or compare against a floor (>= baseline+1 with retries tolerant
of concurrent exits).

Related in spirit to #833 (TestLinuxBindingConcurrentMarkerTransition) — a
second timing-sensitive test that can single-handedly red a run.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions