Repository navigation
perf(apply-extras): stop sleeping and re-minting a token for every item - #376
Open
jlinenkohl wants to merge 1 commit into
Open
jlinenkohl wants to merge 1 commit into
jlinenkohl wants to merge 1 commit into
Conversation
Applying one real agent's extras took about two hours, almost none of it spent talking to the service. Two causes, measured per knowledge item: jq 26ms · base64 6ms · body 8ms · token 1053ms · PUT 377ms · sleep 5000ms _dp_token shelled out to the Azure CLI on every call -- once per skill, per subagent, per hook and per knowledge item -- for a token valid about an hour. It is now fetched once and reused. The cache is a file rather than a variable because callers use TOKEN=$(_dp_token), which runs the function in a subshell, so a variable assigned inside it is discarded as soon as the substitution closes and every call re-mints regardless. The larger cost was an unconditional five-second pause between knowledge items: 79 minutes of sleeping for an agent with 949 of them. It cannot simply be removed -- the service answers a burst of PUTs with 405, and dropping the pause took a 20-item sample from 20 successes down to 8 -- so the transient codes are now retried with backoff and the interval is configurable, defaulting to one second instead of five. Measured on a ten-item sample against a clean target, counting only item outcomes: unconditional 5s 74s, 10/10 1s + retry (default) 36s, 10/10 0s + retry 16s, 10/10 Extrapolated to 949 knowledge items, that is roughly two hours down to under an hour at the default, or about twenty-five minutes with SRE_APPLY_ITEM_DELAY=0. Correctness is unchanged in every sample.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applying one real agent's extras took about two hours, almost none of it spent talking to the service. Two causes, measured per knowledge item:
jq 26ms · base64 6ms · body 8ms · token 1053ms · PUT 377ms · sleep 5000ms
_dp_token shelled out to the Azure CLI on every call -- once per skill, per subagent, per hook and per knowledge item -- for a token valid about an hour. It is now fetched once and reused. The cache is a file rather than a variable because callers use TOKEN=$(_dp_token), which runs the function in a subshell, so a variable assigned inside it is discarded as soon as the substitution closes and every call re-mints regardless.
The larger cost was an unconditional five-second pause between knowledge items: 79 minutes of sleeping for an agent with 949 of them. It cannot simply be removed -- the service answers a burst of PUTs with 405, and dropping the pause took a 20-item sample from 20 successes down to 8 -- so the transient codes are now retried with backoff and the interval is configurable, defaulting to one second instead of five.
Measured on a ten-item sample against a clean target, counting only item outcomes:
unconditional 5s 74s, 10/10
1s + retry (default) 36s, 10/10
0s + retry 16s, 10/10
Extrapolated to 949 knowledge items, that is roughly two hours down to under an hour at the default, or about twenty-five minutes with SRE_APPLY_ITEM_DELAY=0. Correctness is unchanged in every sample.