Skip to content

daily: answer knowledge questions directly instead of verifying each claim with tools - #89

Merged
killpanda merged 2 commits into
mainfrom
claude/project-thread-jfh5sl
Nov 2, 2026
Merged

killpanda merged 2 commits into
mainfrom
claude/project-thread-jfh5sl

Conversation

@killpanda

@killpanda killpanda commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Requested by jack mini · project thread

Why

Session 20261102-183112-c44a83 (daily mode, deepseek-flash) asked a conceptual question: "Is Rust shadowing a good design, or just syntactic sugar?" The agent made 9 serial LLM calls and 9 tool calls (46s) to write and compile demo programs before it answered. The follow-up question took 3 more rounds.

The trace's reasoning shows the cause. In round 1 the model first thinks "I could answer directly". Then it writes: "the user likes 可验证的证据 … Let me check if rustc is available and run a quick demo". After each result it adds one more check ("also verify that mut fails…", "also check unused warnings…"), with one tool call per round.

  • The daily prompt had no "answer knowledge questions directly" rule. It only had a rule to search for current facts.
  • The memory snapshot's user preference ("一步一步推进、每步给可验证的证据") is meant for work tasks, but the model read it as a rule for every reply.
  • The prompt told the model to use web_search, but the tool was not registered. The model provider is a local gateway (magpie), so no DeepSeek key resolved, and the tool was dropped without any message.

Changes

  • src/prompt.rs: the daily guidelines now say to answer from knowledge directly. They say to use tools only when the answer depends on files, the machine, the web or current facts, or when the user asks. When tools are used, independent calls go together in one step. The prompt names web_search only when it is registered.
  • src/memory.rs: the <memory> header now says to apply each entry where it fits the current request.
  • src/tools/memory.rs: the tool description now says to patch an existing entry instead of adding a near-duplicate.
  • src/commands.rs + TUI/headless: in daily mode, if web_search is enabled but missing, a warning shows at startup and when daily mode is turned on. The warning says how to fix it: [web_search] api_key, limbo auth, or DEEPSEEK_API_KEY. To turn the warning off, set enabled = false.

Test

cargo test (107 passed, new tests for the daily prompt and the search notice), cargo fmt --check and cargo clippy --all-targets all pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_014BrrmTSo67QjKWqpBLxeWZ

…claim with tools

A daily-mode session asked a conceptual Rust question and the agent spent
9 serial tool calls (writing and compiling demo programs) before answering,
then 3 more on the follow-up. The reasoning shows why: the prompt had no
rule for answering from knowledge, and the memory snapshot's "wants
verifiable evidence at each step" preference was read as applying to
every reply.

- prompt: daily guidelines now say to answer knowledge questions directly,
  use tools only when the answer depends on files, the machine, the web or
  current facts, and batch independent calls in one step
- prompt: only name web_search when it is registered
- memory: the snapshot header says entries apply where they fit the request
- memory tool: patch an existing entry instead of adding a near-duplicate

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BrrmTSo67QjKWqpBLxeWZ
@killpanda killpanda self-assigned this Nov 2, 2026

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

View in Graphify

Graphify reviewed this change.

No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.

Not checked: tests were not run; no formal proof of the changed code.


Graphify review — findings

Tightens daily-mode behaviour so plain questions get answered directly instead of triggering a tool call per claim. The daily system prompt now tells the model to answer from its own knowledge unless the answer depends on files, this machine, the web or current facts, and to batch independent tool calls into one step. The "search for current facts" line names web_search only when that tool is registered, and otherwise falls back to web_fetch alone. The memory snapshot header now says to apply each entry where it fits, so a stored preference about running work tasks doesn't turn a plain question into a task. The memory tool description also tells the model to patch an existing same-subject entry rather than add a near-duplicate. Bumps the version to 0.4.1.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 84 functions depend on the 34 functions this change touches.

Health — grade A; 5 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

  • run_task() — 5 callers, 4 callees (medium)
  • real_main() — 1 callers, 18 callees (high)
  • build_system_prompt_with() — 2 callers, 6 callees (medium)
  • manual_compaction_appends_record() — 0 callers, 7 callees (medium)
  • crash_mid_batch_is_repaired_on_resume() — 0 callers, 6 callees (medium)

Verification — 84 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 66 function(s) in the blast radius were not formally verified this run

web_search registers only when a DeepSeek key resolves. With a non-deepseek
model provider (e.g. a local gateway) and no [web_search] api_key, auth
entry or DEEPSEEK_API_KEY, the tool was left out with no sign, and daily
sessions quietly lost search. Show a warning in the TUI and the headless
REPL at startup and when daily mode is turned on, with how to fix it or
turn it off ([web_search] enabled = false silences it).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014BrrmTSo67QjKWqpBLxeWZ
@killpanda
killpanda merged commit 2cddc43 into main Nov 2, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants