fix(test): watchdog silent stand-down on missing heartbeat + teardown heartbeats - #317
Merged
Conversation
… heartbeats The out-of-process watchdog treated a missing heartbeat file as healthy (!mtime → return forever), so a CI runner reclaiming the tmpdir file left the guard silently disarmed — the 2026-08-17 teardown hang burned the full 15m step timeout with no kill despite --progress arming the watchdog. Track missing-since and kill after the timeout instead. Also heartbeat the teardown phases (disposeApps per-app, cleanup, complete): the 2026-08-17 hang sat after 'summary pass=229' with zero attribution; the next hang names the teardown stage it froze in.
This was referenced Aug 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes 首个 #316 交付物(挂起可诊断化);#316 根因继续开放。
问题
2026-08-17 CI:httpapi exerciser 在
summary pass=229后静默挂起 15 分钟被 step timeout 杀掉(run 31994077328,同 commit 孪生 run 绿)。 已武装进程外看门狗,但全程未开火。根因之一(已修)
看门狗仔脚本把心跳文件缺失当健康:
CI runner 回收 tmpdir 文件即解除武装。本机独立验证复现:正常场景 3 秒开火 ✅;删除心跳文件后静默不作为(漏洞)→ 修复后 3 秒开火 ✅。
修复
验证
bun typecheck✅--mode effect --include dag --progress:exit 0、22 pass、零误触发 ✅last activity: <none>✅对 #316 的意义
下一个真实挂起将被看门狗在 120s 内终结并带出现场归因(哪个 app 的 dispose、哪段 cleanup),根因从现场钉死,不再依赖低命中率复现。