Repository navigation
fix(agents): name Gemini's system prompt file by its content so the prompt cache holds - #388
Merged
Merged
Conversation
…the prompt cache holds across turns The Gemini CLI writes each global context file's path into the model's system instruction. The run settings named the AgentEnv system prompt file with a new UUID each turn, so the first request of every turn changed the instruction's text and missed the provider's prompt cache for the whole conversation. In a 30-turn GridSim desk run that was 5.3M of 5.6M uncached input tokens and ~80% of its cost. The file is now named by a hash of the prompt; turns sharing one keep it until the last of them ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| def _prompt_file(home: Path, system_prompt: str) -> Iterator[Path]: | ||
| """The system prompt as a global context file named by its content: the CLI writes the file's path into the | ||
| model's system instruction, so a name new each turn would miss the provider's prompt cache on every turn.""" | ||
| path = home / f"agentenv-system-prompt-{hashlib.sha256(system_prompt.encode()).hexdigest()[:32]}.md" |
There was a problem hiding this comment.
The new [:32] hardcodes how much of the prompt hash goes into the filename. The repository requires magic numbers to use descriptive names rather than inline values. Name this length before merging so readers can tell what it controls.
Rule Used: Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. (source)
Learned From
scaleapi/scaleapi#126388
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/agents/agents/gemini/gemini_config.py
Line: 105
Comment:
**Hash length lacks a name**
The new `[:32]` hardcodes how much of the prompt hash goes into the filename. The repository requires magic numbers to use descriptive names rather than inline values. Name this length before merging so readers can tell what it controls.
**Rule Used:** Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. ([source](https://app.greptile.com/scale-ai/-/custom-context?memory=002e0051-41ad-46c1-9098-47433c580150))
**Learned From**
[scaleapi/scaleapi#126388](https://github.com/scaleapi/scaleapi/pull/126388)
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--- Context from: <path> ---).run_settingswrote the AgentEnv system prompt toagentenv-system-prompt-<uuid4>.md, a new name every turn, so the first request of each turn changed the start of the system instruction. That request missed the provider's prompt cache for the entire conversation.Evidence
A 30-turn Gemini 4 Argon run of the GridSim desk, metered per call at a proxy:
cachedContentTokenCount0 for the whole context, which grew to 250k tokens.Tests
test_each_run_gets_its_servers_role_and_system_prompt_in_settings_of_its_ownnow runs the same prompt in a third turn and asserts it gets the same context file name.test_turns_sharing_a_system_prompt_share_its_file_until_the_last_ends.tst/unitand the protocol tests: 11715 passed. The one failure,process_scope_socket_test::test_an_armed_time_trigger_keeps_firing_after_a_client_leaves, is timing-sensitive under xdist and passes 3/3 alone; it is unrelated.🤖 Generated with Claude Code
The behavior appears sound, but the hash length must be named before merging to satisfy the repository rule.
Fix with agent prompt
Summary
Names Gemini's system-prompt file by its content so repeated turns keep the same path.
Reviews (1) · Last reviewed commit: "fix(agents): Gemini's system prompt file..." · Reviewed by Greptile