Skip to content

fix(agents): name Gemini's system prompt file by its content so the prompt cache holds - #388

Merged
earakely-scale merged 1 commit into
mainfrom
edgararakelyan/gemini-stable-prompt-file
Oct 29, 2026
Merged

earakely-scale merged 1 commit into
mainfrom
edgararakelyan/gemini-stable-prompt-file

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • The Gemini CLI writes every global context file's path into the model's system instruction (--- Context from: <path> ---). run_settings wrote the AgentEnv system prompt to agentenv-system-prompt-<uuid4>.md, a new name every turn, so the first request of each turn changed the start of the system instruction. That request missed the provider's prompt cache for the entire conversation.
  • The file is now named by a SHA-256 of the prompt, so it keeps one path from turn to turn. Turns running at once with the same prompt share the file, and an in-process reference count removes it when the last of them ends. The per-run settings file is unchanged: it is still one per run and removed at run end.

Evidence

A 30-turn Gemini 4 Argon run of the GridSim desk, metered per call at a proxy:

  • Inside a turn, calls hit the cache.
  • Every turn's first call reported cachedContentTokenCount 0 for the whole context, which grew to 250k tokens.
  • Those calls account for 5.3M of the run's 5.6M uncached input tokens.
  • Re-pricing the same calls with turn starts cached gives $2.41 instead of $12.47.
  • Between the run's first and last request, the only difference in the system instruction is the file's UUID.

Tests

  • test_each_run_gets_its_servers_role_and_system_prompt_in_settings_of_its_own now runs the same prompt in a third turn and asserts it gets the same context file name.
  • New: test_turns_sharing_a_system_prompt_share_its_file_until_the_last_ends.
  • Both fail on main and pass here.
  • tst/unit and the protocol tests: 11715 passed. The one failure, process_scope_socket_test::test_an_armed_time_trigger_keeps_firing_after_a_client_leaves, is timing-sensitive under xdist and passes 3/3 alone; it is unrelated.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 4/5

The behavior appears sound, but the hash length must be named before merging to satisfy the repository rule.

Fix All in CursorFindings

  1. P2 Hash length lacks a name ▶
Fix with agent prompt
### Issue 1
src/agent_env/agents/agents/gemini/gemini_config.py:105
The new `[:32]` hardcodes how much of the prompt hash goes into the filename. The repository requires magic numbers to use descriptive names rather than inline values. Name this length before merging so readers can tell what it controls.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Names Gemini's system-prompt file by its content so repeated turns keep the same path.

  • Gemini reuses one system-prompt file for matching prompts.

Reviews (1) · Last reviewed commit: "fix(agents): Gemini's system prompt file..." · Reviewed by Greptile

…the prompt cache holds across turns

The Gemini CLI writes each global context file's path into the model's system instruction. The run settings
named the AgentEnv system prompt file with a new UUID each turn, so the first request of every turn changed the
instruction's text and missed the provider's prompt cache for the whole conversation. In a 30-turn GridSim desk
run that was 5.3M of 5.6M uncached input tokens and ~80% of its cost. The file is now named by a hash of the
prompt; turns sharing one keep it until the last of them ends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
def _prompt_file(home: Path, system_prompt: str) -> Iterator[Path]:
"""The system prompt as a global context file named by its content: the CLI writes the file's path into the
model's system instruction, so a name new each turn would miss the provider's prompt cache on every turn."""
path = home / f"agentenv-system-prompt-{hashlib.sha256(system_prompt.encode()).hexdigest()[:32]}.md"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Hash length lacks a name

The new [:32] hardcodes how much of the prompt hash goes into the filename. The repository requires magic numbers to use descriptive names rather than inline values. Name this length before merging so readers can tell what it controls.

Rule Used: Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. (source)

Learned From
scaleapi/scaleapi#126388

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agent_env/agents/agents/gemini/gemini_config.py
Line: 105

Comment:
**Hash length lacks a name**

The new `[:32]` hardcodes how much of the prompt hash goes into the filename. The repository requires magic numbers to use descriptive names rather than inline values. Name this length before merging so readers can tell what it controls.

**Rule Used:** Store magic numbers as class or instance variables with descriptive names rather than using them inline in the code. ([source](https://app.greptile.com/scale-ai/-/custom-context?memory=002e0051-41ad-46c1-9098-47433c580150))

**Learned From**
[scaleapi/scaleapi#126388](https://github.com/scaleapi/scaleapi/pull/126388)

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Cursor Fix in Claude Code Fix in Codex

@earakely-scale
earakely-scale merged commit e0d249d into main Oct 29, 2026
25 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/gemini-stable-prompt-file branch October 29, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant