sdk/python: bound a claim by the caller's deadline - #212
Merged
Merged
Conversation
Every control-plane request took a fresh client timeout, redirect targets included, so the LangChain toolkit's five-minute promise bought 300s per black-holed candidate: two of them cost 600s before the tool's own budget check ran. Claims now accept an absolute time.monotonic() deadline that bounds each request by what is left and refuses once it is spent, and the toolkit passes the deadline it already computes for the run. Measured against a node that redirects to two addresses which accept TCP and never answer, client timeout 2s: 4.01s and an APIError before, 0.30s and a TimeoutError with a 0.3s deadline.
Keep early transport failures as APIError for redirect fallback, but surface TimeoutError once an absolute claim deadline is exhausted so toolkit calls retain their bounded cutoff result. Keep the deadline tests ahead of their helper.
recording_claim already records every claim body a fake node sees; the deadline test rebuilt it inline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #182.
Client._requestgave every control-plane request a freshclient.timeout, and the redirect walk on a warm miss is a loop of such requests. The LangChain toolkit promises a five-minute wall clock persandbox_execand, since #179, honours it for the run — but not for the claim: two black-holed redirect candidates cost 600 s with a 300 s client timeout, so the tool could exceed its promise on a cluster with unreachable peers.Claims now take an absolute
time.monotonic()deadline:Client.new,Template.newandCheckpoint.newaccept keyword-onlydeadline, threaded through_claim_from,_post_jsonand the redirect walk into_request_requestbounds each attempt bymin(timeout, deadline - now)and raisesTimeoutErroronce the deadline is spent, before the request is built —_try_eachand_redirect_fallbackonly catchAPIError, so the walk stops there instead of trying the next candidateconn.remaining_timeoutthe dial path already used, now shared rather than duplicated (it gained the operation name for the error message)CocoonToolkit._execpasses the deadline it already computes, throughsandbox()into_claimMeasured against a node that redirects to two addresses which accept TCP and never answer, client timeout 2 s:
Default behavior is unchanged: without a
deadlineevery request still gets the fulltimeout.Tests:
test_claim_refuses_a_spent_deadline(the node sees no request at all),test_claim_deadline_bounds_the_redirect_walk(two black holes, fails in 0.3 s instead of 2 x 2 s), plustest_exec_claims_inside_the_call_budgetandtest_exec_reports_a_claim_that_outlives_the_budgeton the toolkit.conftestgained ablack_holefixture — a listening socket that never accepts.Left alone deliberately:
Client.lookupscatters its probes concurrently undermin(_PEERS_TIMEOUT, timeout)= 5 s, so it cannot run away the same way, and the OpenAI adapter makes no wall-clock promise over its claim (itstimeoutis caller-supplied and applies torun).Gates:
ruff format --checkandruff checkclean insdk/python,sdk/openaiandsdk/langchain;pytest195 passed insdk/pythonand 8 insdk/langchain.sdk/openai's suite does not collect on this machine (openai-agentsis not installed); it is untouched by this change and CI runs it.