Repository navigation
fix(harness): clamp model-supplied limits in session_search / session_history - #3382
Conversation
session_search 的 maxResults 无上界——工具在 ToolResultEvictionConfig 的 DEFAULT_EXCLUDED_TOOLS 里(理由"bounded results"),无下流裁剪, 模型传 maxResults=100000 即可扫描全部 session 条目并产生无界结果, 还放大 I/O(每个 session 文件全量加载解析)。 - MAX_RESULTS_CEILING=100,sessionSearch 的 maxResults 与 sessionHistory 的 lastN(同款洞)都经 Math.min 钳制 - ToolParam 描述同步标注 capped at 100 - KeywordMatcher 随 upstream main 同步引入(matchMode 依赖) - 测试:150 条命中下 maxResults=100000 只返回 100;lastN=100000 钳到 100;默认 limit 不受影响 Fixes agentscope-ai#3270
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
oss-maintainer
left a comment
There was a problem hiding this comment.
Summary
Adds a MAX_RESULTS_CEILING = 100 clamp for the model-supplied maxResults / lastN in session_search / session_history, updates the two tool parameter descriptions, and adds SessionSearchToolCeilingTest. The clamp is placed correctly and maxResults/lastN really are the only size knobs the model controls here, so this is the right fix for #3270. Verdict: two things to fix before merge — one test that cannot fail, and one inaccurate justification comment (see inline).
Findings
- [Warning]
SessionSearchToolCeilingTest.java:95—session_historyoutput never contains entry ids, so thesplit("m\d+")count is always 0 andemitted <= 100passes even with the clamp removed. - [Info]
SessionSearchTool.java:50— the "nothing downstream trims" rationale only holds forsession_search;session_historyis not inToolResultEvictionConfig.DEFAULT_EXCLUDED_TOOLS.
Suggestions
// instead of counting ids that are never printed
assertTrue(out.contains("showing last 100)"), () -> "history must be clamped to 100, got: " + out);Also worth pinning the boundary explicitly (lastN=99 → 99, lastN=101 → 100) so a future edit to the ceiling constant is caught.
Note: no license/cla status context was found on the head commit of this PR, so I'm leaving this as a comment rather than an approval — please confirm the CLA is signed (or that the check is not required for this repo) and I'll re-review.
Automated review by github-manager-bot
| 100000); | ||
|
|
||
| // Exactly 100 message blocks, not 150: count the entry ids. | ||
| long emitted = out.split("m\\d+", -1).length - 1; |
There was a problem hiding this comment.
This assertion is vacuous — the test cannot detect a missing/incorrect lastN ceiling.
sessionHistory never prints entry ids: it emits "[role]: content" lines only (SessionSearchTool.java:242-250), and the seeded text is "needle item N", which contains no m<digit> token. So out.split("m\\d+", -1).length - 1 is 0 regardless of the limit, and emitted <= 100 passes even with the clamp deleted.
The header line already carries the truth — assert on it instead:
assertTrue(out.contains("showing last 100)"), () -> "history must be clamped to 100, got: " + out);An explicit 100-vs-101 boundary case (e.g. lastN=99 still returns 99) would pin the behaviour down further.
| /** | ||
| * Hard ceiling for model-supplied result limits. The tool is excluded from | ||
| * {@code ToolResultEvictionConfig} on the "bounded results" rationale, so nothing | ||
| * downstream trims an oversized result — the clamp has to happen here (#3270). |
There was a problem hiding this comment.
The stated rationale only holds for session_search.
ToolResultEvictionConfig.DEFAULT_EXCLUDED_TOOLS lists memory_search, memory_get and session_search — session_history is not in it, so oversized history results are still evictable downstream. The clamp itself is still the right fix here (bounding at the source is better than evicting after the fact), but the comment claiming "nothing downstream trims" an oversized session_history result is inaccurate and will mislead the next reader.
Could you either drop session_history from that sentence, or add it to DEFAULT_EXCLUDED_TOOLS if the intent is for the clamp to be the only bound?
- vacuous 断言修复:split("m\d+") 计数永远为 0(输出不含 entry id),
钳制被删测试仍绿;改为断言头部的 showing last 100),并补 lastN=99/101
边界用例(同时覆盖 Codecov 的 2 个 partial 分支)
- MAX_RESULTS_CEILING 注释修正:session_history 不在
DEFAULT_EXCLUDED_TOOLS(下游可驱逐),"nothing downstream trims"
仅对 session_search 成立;钳制仍是正确做法(源头限界优于事后驱逐,
且同时限制扫描量)
|
Both items addressed in a193e3e, plus the CLA confirmation:
|
oss-maintainer
left a comment
There was a problem hiding this comment.
Summary
Clamps model-supplied session_search maxResults and session_history lastN to MAX_RESULTS_CEILING=100 (fixing #3270) and updates both tool param descriptions. The clamp is correct and overflow-safe (null/non-positive inputs still fall back to the 10/20 defaults), and the rewritten SessionSearchToolCeilingTest is now genuinely load-bearing: it asserts real output substrings ("Found 100 matches", "showing last 100)" — both verified against the actual format strings in SessionSearchTool), so it fails without the clamp, and the previously suggested 99/101 boundary case was added as its own test.
Both concerns from the previous review are resolved: the earlier Warning (history test counted entry ids that are never printed, so it could not fail) is fixed by asserting the clamped header, and the suggested boundary pinning (lastN=99 -> 99, lastN=101 -> 100) was implemented. The earlier Info-level comment inaccuracy (the javadoc still says "nothing downstream trims an oversized result", but session_history is not in ToolResultEvictionConfig.DEFAULT_EXCLUDED_TOOLS, so eviction would trim it) was left as-is; it is cosmetic and not re-reported as a finding. As before, no CLA status context was found on the head commit, so please confirm the CLA check separately before merge.
No blocking findings this round — both previously raised concerns are resolved.
Automated review by github-manager-bot
a193e3e to
35fd915
Compare
…iling # Conflicts: # agentscope-harness/src/main/java/io/agentscope/harness/agent/tool/SessionSearchTool.java
35fd915 to
865915d
Compare
Fixes #3270
What
SessionSearchTool's model-supplied limits are now clamped to a hard ceiling of 100 (MAX_RESULTS_CEILING), closing the same hole #3267/#3267 closed formemory_search:session_searchmaxResults— clamped viaMath.min, so a model passing 100000 no longer scans-and-collects unbounded results;session_historylastN— same hole, same clamp (called out in the PR as a sibling fix);ToolParamdescriptions now state "capped at 100" so the model knows;KeywordMatchersynced frommain(matchMode dependency of the touched code path).The clamp also cuts the I/O amplification the report flagged: the collection loop stops at the clamped limit instead of walking every entry of every parsed
SessionTree.Tests
New
SessionSearchToolCeilingTest(realWorkspaceManager+ real offloaded session tree, 150 needle entries):maxResults=100000→Found 100 matches(not 150);lastN=100000→ history clamped to 100;Module suite matches the upstream baseline (0 failures, only the two known Windows symlink-privilege environment errors).