Repository navigation
spec(queen): key_failover.t27 -- a refused key moves the bee to the next key, then the next pool (Closes #9993) - #9998
Merged
Conversation
…ext key, then the next pool (Closes #9993) A bee is pinned to one key at dispatch. On z.ai 429 code 1305 the SDK retried that key three times and the turn closed after 0 steps: 17 of 93 turns on trios-bee-runner-2 on 2026-10-17, while 33 other keys stood by. The card classifies a refusal (error_kind), rests or closes the key (cooldown_seconds, key_closes), picks the next free key and never the one that failed (next_free), moves pools when three distinct keys said 1305 (pool_overloaded), bounds a turn at 8 attempts (step) and charges the issue only for a bad request (charges_issue). Railway lab: test-report 9/9, 112 runtime asserts, 0 vacuous, 4 invariants; tri mutate spec 92/93 killed, 0 survived, 1 hang (the k = k mutant's own loop). Seal saved there. Wiring into trios goes through t27b ts (#9994). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gHashTag
enabled auto-merge (squash)
October 17, 2026 15:50
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9993. Part of #9976.
Why
The Railway logs of trios-bee-runner-2 for 2026-10-17 13:00-15:18Z show 17 of 93 bee turns ending after 0 steps.
resolveWorkerProvider). The SDK retries that same key three times, and the turn closes as "cause undetermined" while 33 other keys in two pools stand by.The card
error_kind(status, zai_code): classifies a refusal as transient, quota, auth or a bad request. The code wins over the status; the quota list is the runtime's closed list.cooldown_seconds,key_closes: a transient refusal rests the key for 15 s, doubled per strike, at most 240 s. A quota or auth refusal closes the key for 30 min. 1302 and 1305 never close a key.next_free: the next free key, wrapping, and never the key that just failed. The same rule picks the next pool. A list longer than a mask is refused, not truncated.pool_overloaded: once 3 distinct keys of one pool answer 1305, the model is overloaded and the bee moves pools.step: at most 8 attempts per turn, thenprovider_overloaded.charges_issue: only a bad request charges the issue.Evidence (Railway lab, master build)
t27c test-report: 9 tests, 112 runtime asserts, 0 FAIL, 0 vacuous, 4 invariants proved.tri mutate spec --jobs 12 --timeout 240: 92 of 93 killed and 0 survived.next_freerefuse an oversized list, and pinning the cap tobase * 16.k = kmutant's own endless loop..trinity/seals/queen_QueenKeyFailover.json; its spec hash matches this file.Not here
The wiring into trios. That goes through
t27b ts(#9994), which emits the card as TypeScript with function bodies. #7872 stays blocked by the own-language gate.🤖 Generated with Claude Code