diff --git a/.gitignore b/.gitignore index dc899b59..c6e11c1d 100644 --- a/.gitignore +++ b/.gitignore @@ -15,4 +15,6 @@ apps/web/tsconfig.tsbuildinfo /.worktrees/ __pycache__/ *.pyc +# Disposable query/output caches can be created from any working directory. +**/.compass/cache/ .DS_Store diff --git a/CHANGELOG.md b/CHANGELOG.md index b639b0ea..eee21ef3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,16 @@ ## Unreleased +- Default typed query and MCP text to compact, source-located answers. Keep + uncertainty visible and expose full paged audit detail with `--verbose` or + `--evidence`; raw machine contracts are unchanged. +- Enforce UTF-8 output budgets with exact continuations across finite commands. + Add immutable saved-output reading via `compass output`; reject partial + machine output and budgets on unbounded streams. +- Cache complete native query responses by verified graph, Program IR, request + limits, profiles and semantic mode, with bounded disposable storage and + corruption fallback. Reuse one pinned engine across CLI page widening. + ## 0.4.1 - 2026-10-01 - Match engineering business vocabulary through bounded synonyms and witnessed diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 63508c76..16046c79 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -1665,3 +1665,29 @@ independently of the bounded path ledger. The resolver also preserves qualified external Rust receivers established by source factory return contracts, subject to the existing inference policy. An absent method on a source-local return type remains unresolved. + +## Compact query text and completed-output budgets + +Human-readable typed query defaults now omit audit fields and redundant ledgers. +Use `--verbose` or `--evidence` for full paged status and record provenance. +No-match/candidate pages are capped at 199 approximate tokens. Text is a display +surface; raw `compass.query/1`, discovery, Agent View and MCP structured content +retain their schemas and semantic records. MCP text uses the same compact +presentation. Keep evidence mode unchanged when continuing a cursor; incompatible +prefixes fail explicitly. + +`--budget N` is a completed-output ceiling of `4*N` UTF-8 bytes across streams, +including newline/continuation. Supported range is 32–65536. It no longer selects +legacy query traversal; `--traverse` or `--page` does. Native query pagers keep +whole entries or report the budget required for one entry. Other completed +answers use content-addressed `compass.saved-output/1` records and the additive +`compass output ID --offset BYTE --budget N` command. This reader does not repeat +command side effects. Oversized machine output fails rather than publish partial +JSON/JSONL; streaming commands/events reject finite budgets before starting. + +Saved answers are disposable, work-directory-local and independently bounded. +Response cache v1 is also disposable: checksum-checked typed complete records +are bound to graph/Program IR, executable version, planner/ranker profiles, +complete request and semantic mode. A changed response-cache meaning requires a +new cache filename/version; missing/corrupt caches do not change native results. +Neither cache rewrites published graphs or historical realizations. diff --git a/MIGRATION.md b/MIGRATION.md index b509b12b..1dee4a31 100644 --- a/MIGRATION.md +++ b/MIGRATION.md @@ -5,6 +5,22 @@ sidecars. Its output root now preserves the familiar flat artifact shape so file-based workflows can transition while Compass's snapshot and store layout remains visible and clearly owned. +## Compact output and budgets + +Typed query text is compact by default. Select `--verbose` or `--evidence` to +recover audit status and per-record provenance. JSON contracts are unchanged. +Keep that mode fixed between cursor pages; restart the query when switching. + +`query --budget N` now bounds output instead of selecting legacy traversal. +Add `--traverse` or `--page` if the old relevance renderer is required. +Budgets accept 32–65536 approximate tokens and include continuation instructions. +Follow `compass output ID --offset BYTE --budget N` from the same directory for +commands that save their completed output. Saved answers can be evicted; the +reader never repeats an action. Oversized machine output fails intact; omit the +budget or read the saved record with a large enough budget. Streaming events, +watch, REPL and serve reject this finite-output control; budgeted init needs `--yes`. +No graph rebuild is required for compact text or disposable response caching. + ## Graph rebuilds and query resolution To enable full bounded document/prose recall, rebuild with diff --git a/PERFORMANCE.md b/PERFORMANCE.md index 4b7d5252..b85cf842 100644 --- a/PERFORMANCE.md +++ b/PERFORMANCE.md @@ -1498,3 +1498,59 @@ unrelated host work and ordinary OS caches remained uncontrolled. Three runs per binary on one known repository do not establish general performance or an approved release baseline. All six graphs still omit one node and 45 edges; normal release qualification remains required. + +## Compact query text and complete-response caching + +The October 2, 2026 replay uses unchanged `suite_natural.toml` and `suite_v2.toml` +oracles over the same pinned Cobra, Flask, Gson, Zod and Axum source trees and +frozen graphs. Both native binaries use the normal optimized workspace release +profile and Rust 1.97.1. The installed comparator remains Graphify 0.9.67; its +natural-suite observations are the archived paired run, and the standard suite +was replayed against the same archived graph inputs. + +| Fixed panel | Native before | Native after | Comparator | Mean native before tokens | Mean native after tokens | Mean comparator tokens | +| --- | ---: | ---: | ---: | ---: | ---: | ---: | +| 25 natural questions | 24/25 | 24/25 | 17/25 | 544.52 | 547.44 | 757.44 | +| 50 standard questions | 49/50 | 49/50 | 44/50 | 419.10 | 363.76 | 549.74 | + +Every native row retains its previous oracle verdict. Standard-suite mean cost +falls 13.2%; the natural mean grows slightly because compact answers retain +uncertainty warnings. Both native means remain below the comparator on these +panels. The known `Find` omission in the natural Cobra question and +`route_endpoint` omission in the standard Axum impact question remain failures. +Tokens are the existing harness's `ceil(stdout UTF-8 bytes / 4)` estimate over +its complete workflow. Comparator installation warnings are recorded separately +on stderr. These source-reviewed text-recall panels do not independently measure +precision or establish population-wide superiority. MCP text is compact but its +unchanged full structured content still contributes to clients consuming both. + +A separately registered cache panel executes the first callers question in each +standard-suite repository as raw JSON: three alternating trials per binary, +each with a fresh task-owned cache directory and six command invocations. Warm +cost is the median of invocations 2–6, summarized across the three trials. +All 180 raw JSON outputs are byte-identical across both binaries and repetitions +for each repository. Source excerpts and truncated execution responses remain +uncached. + +| Repository | Before cold median ms | Before warm median ms | After cold median ms | After warm median ms | Cache eligible | +| --- | ---: | ---: | ---: | ---: | --- | +| Cobra | 85 | 85 | 73 | 34 | Yes | +| Flask | 348 | 220 | 386 | 34 | Yes | +| Gson | 488 | 292 | 438 | 259 | No: partial graph | +| Zod | 967 | 994 | 897 | 34 | Yes | +| Axum | 401 | 284 | 648 | 30 | Yes | + +The initial panel stopped when Gson's frozen partial publication made its small +response truncated. The recorded amendment kept all questions, inputs, limits, +trials and repetitions, and recorded eligibility instead of requiring every +input to be complete. Gson remains in the table and its cache has no response +entry. Cold Flask and Axum observations regress; cache reuse is not a claim that +every command or first invocation gets faster. Ordinary OS caches and unrelated +host load remain uncontrolled, including an 8.135-second cold Zod observation. +The earlier debug replay had timeouts and is excluded from optimized timing and +final verdicts. + +[`compact_query_output_review.json`](benchmarks/agent_query/compact_query_output_review.json) +contains source and artifact pins, suite and binary hashes, the registration and +amendment, every oracle observation and every cold/warm timing. These focused +query measurements do not replace release-wide performance qualification. diff --git a/SECURITY.md b/SECURITY.md index da93cedc..d1ffaeb5 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -137,3 +137,22 @@ of source-backed prose per document/rationale resource, so exported artifacts include those excerpts as well as existing names and provenance. Apply the same repository disclosure policy to these graph artifacts. Corpus size/deadline failures remain explicit and do not publish a partially built semantic index. + +## Local query and saved-output caches + +Completed-output budgets can save source excerpts, report content and diagnostics +under the working directory's `.compass/cache/output`. Treat these records and +query-response cache databases as repository-derived sensitive data. They are +local only and must not be committed or attached to public issues. New saved +output directories use owner-only permissions on Unix; existing directory access +policy remains the owner's responsibility. Redirected directories and record +symlinks are rejected. Publication uses the existing atomic create primitive. +Readers enforce schema, digest, regular-file, byte-size and UTF-8 offset checks; +retention scans are bounded. A content digest detects corruption and does not +authenticate the original output or graph semantics. + +The response cache has 32 entries, a 1 MiB payload limit and a 64 MiB SQLite page +ceiling. Lookup requires the verified graph identity and Program IR/profile/ +request/mode identity, validates payload length, checksum and typed schema, and +checks deadlines. Source excerpts and partial responses are excluded. Cache +errors use native execution. Neither cache adds network or credential access. diff --git a/benchmarks/agent_query/compact_query_output_review.json b/benchmarks/agent_query/compact_query_output_review.json new file mode 100644 index 00000000..5bde88e2 --- /dev/null +++ b/benchmarks/agent_query/compact_query_output_review.json @@ -0,0 +1,4141 @@ +{ + "schema": "compass.compact-query-output-review/1", + "date": "2026-10-02", + "sourceCommit": "e1cf854f4a117e180f47142b3b80a614d6871854", + "sourceTree": "7af951e51b6ad981c0c585bc134011682d6eac27", + "baselineCommit": "9a60e3f2", + "baselineBinarySha256": "882e365990a0a0ccbfc074bed81428d7fc0d6559a568af0afe13d22244960000", + "candidateBinarySha256": "83f4188d673d8154cf4e53c26861e5bae3454ee02fb9de26d1b3686344e459df", + "toolchain": "Rust 1.97.1, normal workspace release profile, --locked", + "host": { + "platform": "macOS-26.5.2-arm64-arm-64bit-Mach-O", + "machine": "arm64", + "cpu": "Apple M2 Max", + "logicalCpus": 12 + }, + "comparisonVersion": "Graphify 0.9.67", + "tokenMetric": "ceil(stdout UTF-8 bytes / 4), entire existing harness workflow", + "inputs": [ + { + "repository": "cobra", + "url": "https://github.com/spf13/cobra.git", + "commit": "adbc8813901bba65827259daa8e22ff94ec1f30e", + "language": "Go", + "compassGraphSha256": "d4d6661a54e5df3213bfbda22d1640b2b2a26c5be456ce9886a9173372ca1ac5", + "graphifyGraphSha256": "218aa21e02fad4506206c34acfc6d9ff58a2a0052615d92e4377c74bf977a937" + }, + { + "repository": "flask", + "url": "https://github.com/pallets/flask.git", + "commit": "d73fa1cdcbd8b1465c151db8924ba58b1dd14e35", + "language": "Python", + "compassGraphSha256": "3fd3a613588d44429f21e6594c10a0037d779cce6536a2d0f62e0436ad7abc34", + "graphifyGraphSha256": "ca35435af286c5c565fc66f55de569c2e3235f75c902182290ee15603ff103cc" + }, + { + "repository": "gson", + "url": "https://github.com/google/gson.git", + "commit": "15ca7360379cf3c1502b59981569050489f2d73e", + "language": "Java", + "compassGraphSha256": "cf0fcb28f9e23832342c536d70cae118dfa09722a02224d48de21300d768d1d5", + "graphifyGraphSha256": "92471ab8f5bb5d34aedddd2263fa24a43ae4ed31b34d06c8d433fe0f0dada447" + }, + { + "repository": "zod", + "url": "https://github.com/colinhacks/zod.git", + "commit": "d2b135cfb7a3582b9eb515756b9166bcb9521f4a", + "language": "TypeScript", + "compassGraphSha256": "cf9a05945cf3fe77fa408835bff714e4df195a15d6683e388fd2893f7b45f9e6", + "graphifyGraphSha256": "c0f6d021616326da6b7daf5105b3b996c0bb4ba17c0bf54a062f2e692147b01a" + }, + { + "repository": "axum", + "url": "https://github.com/tokio-rs/axum.git", + "commit": "af1345b53a259b0990be1ff853f9b56c05040ef7", + "language": "Rust", + "compassGraphSha256": "71b436edc18e63e314799c97564a8a4faf8cf54632d5c0fc186816a33fdec8d6", + "graphifyGraphSha256": "6ed791a771190e98ba2905e08480a9fa6ed733f46d56212ae3a4c90018d79b52" + } + ], + "suites": [ + { + "file": "suite_natural.toml", + "sha256": "e5de76d8583a861beb48d64100f663c5028b6a5430cf2da576c6a318447c671a" + }, + { + "file": "suite_v2.toml", + "sha256": "3780a30fabad4b699e071297befb83a00a448d8b0d8a1c136f6758d990af62d1" + } + ], + "panels": [ + { + "name": "suite_natural.toml", + "questions": 25, + "beforePassed": 24, + "afterPassed": 24, + "comparisonPassed": 17, + "allNativeVerdictsUnchanged": true, + "beforeMeanTokens": 544.52, + "afterMeanTokens": 547.44, + "comparisonMeanTokens": 757.44, + "observations": { + "before": [ + { + "repository": "cobra", + "question": "cobra-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 430, + "totalTokens": 430, + "followUps": 0, + "stdoutBytes": 1719 + }, + { + "repository": "cobra", + "question": "cobra-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 786, + "totalTokens": 786, + "followUps": 0, + "stdoutBytes": 3144 + }, + { + "repository": "cobra", + "question": "cobra-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 795, + "totalTokens": 795, + "followUps": 0, + "stdoutBytes": 3177 + }, + { + "repository": "cobra", + "question": "cobra-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 117, + "totalTokens": 117, + "followUps": 0, + "stdoutBytes": 466 + }, + { + "repository": "cobra", + "question": "cobra-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": false, + "failures": [ + "missing 'Find'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 790, + "totalTokens": 790, + "followUps": 0, + "stdoutBytes": 3157 + }, + { + "repository": "flask", + "question": "flask-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 330, + "totalTokens": 330, + "followUps": 0, + "stdoutBytes": 1319 + }, + { + "repository": "flask", + "question": "flask-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 376, + "totalTokens": 376, + "followUps": 0, + "stdoutBytes": 1501 + }, + { + "repository": "flask", + "question": "flask-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 406, + "totalTokens": 406, + "followUps": 0, + "stdoutBytes": 1622 + }, + { + "repository": "flask", + "question": "flask-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 168, + "totalTokens": 168, + "followUps": 0, + "stdoutBytes": 670 + }, + { + "repository": "flask", + "question": "flask-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 784, + "totalTokens": 784, + "followUps": 0, + "stdoutBytes": 3133 + }, + { + "repository": "gson", + "question": "gson-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 746, + "totalTokens": 746, + "followUps": 0, + "stdoutBytes": 2982 + }, + { + "repository": "gson", + "question": "gson-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 469, + "totalTokens": 469, + "followUps": 0, + "stdoutBytes": 1874 + }, + { + "repository": "gson", + "question": "gson-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 780, + "totalTokens": 780, + "followUps": 0, + "stdoutBytes": 3119 + }, + { + "repository": "gson", + "question": "gson-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 258, + "totalTokens": 258, + "followUps": 0, + "stdoutBytes": 1031 + }, + { + "repository": "gson", + "question": "gson-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 769, + "totalTokens": 769, + "followUps": 0, + "stdoutBytes": 3074 + }, + { + "repository": "zod", + "question": "zod-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 312, + "totalTokens": 312, + "followUps": 0, + "stdoutBytes": 1246 + }, + { + "repository": "zod", + "question": "zod-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 793, + "totalTokens": 793, + "followUps": 0, + "stdoutBytes": 3170 + }, + { + "repository": "zod", + "question": "zod-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 769, + "totalTokens": 769, + "followUps": 0, + "stdoutBytes": 3075 + }, + { + "repository": "zod", + "question": "zod-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 779, + "totalTokens": 779, + "followUps": 0, + "stdoutBytes": 3115 + }, + { + "repository": "zod", + "question": "zod-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 780, + "totalTokens": 780, + "followUps": 0, + "stdoutBytes": 3119 + }, + { + "repository": "axum", + "question": "axum-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 265, + "totalTokens": 265, + "followUps": 0, + "stdoutBytes": 1060 + }, + { + "repository": "axum", + "question": "axum-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 154, + "totalTokens": 154, + "followUps": 0, + "stdoutBytes": 614 + }, + { + "repository": "axum", + "question": "axum-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 782, + "totalTokens": 782, + "followUps": 0, + "stdoutBytes": 3125 + }, + { + "repository": "axum", + "question": "axum-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 181, + "totalTokens": 181, + "followUps": 0, + "stdoutBytes": 722 + }, + { + "repository": "axum", + "question": "axum-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 794, + "totalTokens": 794, + "followUps": 0, + "stdoutBytes": 3175 + } + ], + "after": [ + { + "repository": "cobra", + "question": "cobra-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 438, + "totalTokens": 438, + "followUps": 0, + "stdoutBytes": 1751 + }, + { + "repository": "cobra", + "question": "cobra-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 794, + "totalTokens": 794, + "followUps": 0, + "stdoutBytes": 3176 + }, + { + "repository": "cobra", + "question": "cobra-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 791, + "totalTokens": 791, + "followUps": 0, + "stdoutBytes": 3164 + }, + { + "repository": "cobra", + "question": "cobra-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 125, + "totalTokens": 125, + "followUps": 0, + "stdoutBytes": 498 + }, + { + "repository": "cobra", + "question": "cobra-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": false, + "failures": [ + "missing 'Find'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 786, + "totalTokens": 786, + "followUps": 0, + "stdoutBytes": 3141 + }, + { + "repository": "flask", + "question": "flask-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 327, + "totalTokens": 327, + "followUps": 0, + "stdoutBytes": 1306 + }, + { + "repository": "flask", + "question": "flask-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 384, + "totalTokens": 384, + "followUps": 0, + "stdoutBytes": 1533 + }, + { + "repository": "flask", + "question": "flask-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 403, + "totalTokens": 403, + "followUps": 0, + "stdoutBytes": 1609 + }, + { + "repository": "flask", + "question": "flask-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 176, + "totalTokens": 176, + "followUps": 0, + "stdoutBytes": 702 + }, + { + "repository": "flask", + "question": "flask-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 780, + "totalTokens": 780, + "followUps": 0, + "stdoutBytes": 3117 + }, + { + "repository": "gson", + "question": "gson-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 743, + "totalTokens": 743, + "followUps": 0, + "stdoutBytes": 2969 + }, + { + "repository": "gson", + "question": "gson-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 472, + "totalTokens": 472, + "followUps": 0, + "stdoutBytes": 1885 + }, + { + "repository": "gson", + "question": "gson-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 777, + "totalTokens": 777, + "followUps": 0, + "stdoutBytes": 3106 + }, + { + "repository": "gson", + "question": "gson-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 261, + "totalTokens": 261, + "followUps": 0, + "stdoutBytes": 1042 + }, + { + "repository": "gson", + "question": "gson-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 799, + "totalTokens": 799, + "followUps": 0, + "stdoutBytes": 3193 + }, + { + "repository": "zod", + "question": "zod-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 309, + "totalTokens": 309, + "followUps": 0, + "stdoutBytes": 1233 + }, + { + "repository": "zod", + "question": "zod-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 796, + "totalTokens": 796, + "followUps": 0, + "stdoutBytes": 3181 + }, + { + "repository": "zod", + "question": "zod-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 766, + "totalTokens": 766, + "followUps": 0, + "stdoutBytes": 3062 + }, + { + "repository": "zod", + "question": "zod-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 772, + "totalTokens": 772, + "followUps": 0, + "stdoutBytes": 3087 + }, + { + "repository": "zod", + "question": "zod-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 800, + "totalTokens": 800, + "followUps": 0, + "stdoutBytes": 3198 + }, + { + "repository": "axum", + "question": "axum-natural-callers", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 265, + "totalTokens": 265, + "followUps": 0, + "stdoutBytes": 1059 + }, + { + "repository": "axum", + "question": "axum-natural-callees", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 162, + "totalTokens": 162, + "followUps": 0, + "stdoutBytes": 646 + }, + { + "repository": "axum", + "question": "axum-natural-impact", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 781, + "totalTokens": 781, + "followUps": 0, + "stdoutBytes": 3124 + }, + { + "repository": "axum", + "question": "axum-natural-path", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 189, + "totalTokens": 189, + "followUps": 0, + "stdoutBytes": 754 + }, + { + "repository": "axum", + "question": "axum-natural-broad", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 790, + "totalTokens": 790, + "followUps": 0, + "stdoutBytes": 3159 + } + ], + "comparison": [ + { + "repository": "cobra", + "question": "cobra-natural-callers", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 751, + "totalTokens": 751, + "followUps": 0, + "stdoutBytes": 3004 + }, + { + "repository": "cobra", + "question": "cobra-natural-callees", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'Find'", + "missing 'Traverse'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 756, + "totalTokens": 756, + "followUps": 0, + "stdoutBytes": 3021 + }, + { + "repository": "cobra", + "question": "cobra-natural-impact", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'Traverse'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 755, + "totalTokens": 755, + "followUps": 0, + "stdoutBytes": 3017 + }, + { + "repository": "cobra", + "question": "cobra-natural-path", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 754, + "totalTokens": 754, + "followUps": 0, + "stdoutBytes": 3014 + }, + { + "repository": "cobra", + "question": "cobra-natural-broad", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'Find'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 765, + "totalTokens": 765, + "followUps": 0, + "stdoutBytes": 3059 + }, + { + "repository": "flask", + "question": "flask-natural-callers", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'wsgi_app'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 759, + "totalTokens": 759, + "followUps": 0, + "stdoutBytes": 3035 + }, + { + "repository": "flask", + "question": "flask-natural-callees", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'preprocess_request'", + "missing 'finalize_request'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 759, + "totalTokens": 759, + "followUps": 0, + "stdoutBytes": 3035 + }, + { + "repository": "flask", + "question": "flask-natural-impact", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'wsgi_app'", + "missing '__call__'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 776, + "totalTokens": 776, + "followUps": 0, + "stdoutBytes": 3102 + }, + { + "repository": "flask", + "question": "flask-natural-path", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 786, + "totalTokens": 786, + "followUps": 0, + "stdoutBytes": 3144 + }, + { + "repository": "flask", + "question": "flask-natural-broad", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 756, + "totalTokens": 756, + "followUps": 0, + "stdoutBytes": 3022 + }, + { + "repository": "gson", + "question": "gson-natural-callers", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 783, + "totalTokens": 783, + "followUps": 0, + "stdoutBytes": 3132 + }, + { + "repository": "gson", + "question": "gson-natural-callees", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'setHtmlSafe'", + "missing 'setSerializeNulls'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 783, + "totalTokens": 783, + "followUps": 0, + "stdoutBytes": 3132 + }, + { + "repository": "gson", + "question": "gson-natural-impact", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 787, + "totalTokens": 787, + "followUps": 0, + "stdoutBytes": 3147 + }, + { + "repository": "gson", + "question": "gson-natural-path", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 766, + "totalTokens": 766, + "followUps": 0, + "stdoutBytes": 3061 + }, + { + "repository": "gson", + "question": "gson-natural-broad", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 756, + "totalTokens": 756, + "followUps": 0, + "stdoutBytes": 3024 + }, + { + "repository": "zod", + "question": "zod-natural-callers", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 736, + "totalTokens": 736, + "followUps": 0, + "stdoutBytes": 2941 + }, + { + "repository": "zod", + "question": "zod-natural-callees", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 740, + "totalTokens": 740, + "followUps": 0, + "stdoutBytes": 2957 + }, + { + "repository": "zod", + "question": "zod-natural-impact", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 736, + "totalTokens": 736, + "followUps": 0, + "stdoutBytes": 2941 + }, + { + "repository": "zod", + "question": "zod-natural-path", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 743, + "totalTokens": 743, + "followUps": 0, + "stdoutBytes": 2971 + }, + { + "repository": "zod", + "question": "zod-natural-broad", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 758, + "totalTokens": 758, + "followUps": 0, + "stdoutBytes": 3031 + }, + { + "repository": "axum", + "question": "axum-natural-callers", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 741, + "totalTokens": 741, + "followUps": 0, + "stdoutBytes": 2963 + }, + { + "repository": "axum", + "question": "axum-natural-callees", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 741, + "totalTokens": 741, + "followUps": 0, + "stdoutBytes": 2963 + }, + { + "repository": "axum", + "question": "axum-natural-impact", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 764, + "totalTokens": 764, + "followUps": 0, + "stdoutBytes": 3056 + }, + { + "repository": "axum", + "question": "axum-natural-path", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 745, + "totalTokens": 745, + "followUps": 0, + "stdoutBytes": 2977 + }, + { + "repository": "axum", + "question": "axum-natural-broad", + "kind": "broad", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'MethodRouter'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 740, + "totalTokens": 740, + "followUps": 0, + "stdoutBytes": 2960 + } + ] + } + }, + { + "name": "suite_v2.toml", + "questions": 50, + "beforePassed": 49, + "afterPassed": 49, + "comparisonPassed": 44, + "allNativeVerdictsUnchanged": true, + "beforeMeanTokens": 419.1, + "afterMeanTokens": 363.76, + "comparisonMeanTokens": 549.74, + "observations": { + "before": [ + { + "repository": "cobra", + "question": "cobra2-explain-parseflags", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 400, + "totalTokens": 400, + "followUps": 0, + "stdoutBytes": 1599 + }, + { + "repository": "cobra", + "question": "cobra2-source-find", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 415, + "totalTokens": 415, + "followUps": 0, + "stdoutBytes": 1657 + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 417, + "totalTokens": 417, + "followUps": 0, + "stdoutBytes": 1666 + }, + { + "repository": "cobra", + "question": "cobra2-callees-executec", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 267, + "totalTokens": 267, + "followUps": 0, + "stdoutBytes": 1068 + }, + { + "repository": "cobra", + "question": "cobra2-impact-parseflags", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 480, + "totalTokens": 480, + "followUps": 0, + "stdoutBytes": 1917 + }, + { + "repository": "cobra", + "question": "cobra2-path-find-legacyargs", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 36, + "totalTokens": 36, + "followUps": 0, + "stdoutBytes": 141 + }, + { + "repository": "cobra", + "question": "cobra2-file-path-md-docs", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 57, + "totalTokens": 57, + "followUps": 0, + "stdoutBytes": 226 + }, + { + "repository": "cobra", + "question": "cobra2-ambiguous-command", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 296, + "totalTokens": 296, + "followUps": 0, + "stdoutBytes": 1183 + }, + { + "repository": "cobra", + "question": "cobra2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 84, + "totalTokens": 84, + "followUps": 0, + "stdoutBytes": 335 + }, + { + "repository": "cobra", + "question": "cobra2-broad-resolve-run", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 602, + "totalTokens": 1191, + "followUps": 1, + "stdoutBytes": 4759 + }, + { + "repository": "flask", + "question": "flask2-explain-fulldispatch", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 538, + "totalTokens": 538, + "followUps": 0, + "stdoutBytes": 2151 + }, + { + "repository": "flask", + "question": "flask2-source-wsgiapp", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 797, + "totalTokens": 797, + "followUps": 0, + "stdoutBytes": 3187 + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 209, + "totalTokens": 209, + "followUps": 0, + "stdoutBytes": 833 + }, + { + "repository": "flask", + "question": "flask2-callees-fulldispatch", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 335, + "totalTokens": 335, + "followUps": 0, + "stdoutBytes": 1337 + }, + { + "repository": "flask", + "question": "flask2-impact-fulldispatch", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 446, + "totalTokens": 446, + "followUps": 0, + "stdoutBytes": 1784 + }, + { + "repository": "flask", + "question": "flask2-path-fulldispatch-finalize", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 48, + "totalTokens": 48, + "followUps": 0, + "stdoutBytes": 189 + }, + { + "repository": "flask", + "question": "flask2-file-path-views-app", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 42, + "totalTokens": 42, + "followUps": 0, + "stdoutBytes": 168 + }, + { + "repository": "flask", + "question": "flask2-ambiguous-urlfor", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 356, + "totalTokens": 356, + "followUps": 0, + "stdoutBytes": 1422 + }, + { + "repository": "flask", + "question": "flask2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 86, + "totalTokens": 86, + "followUps": 0, + "stdoutBytes": 341 + }, + { + "repository": "flask", + "question": "flask2-broad-url-rule", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 596, + "totalTokens": 596, + "followUps": 0, + "stdoutBytes": 2382 + }, + { + "repository": "gson", + "question": "gson2-explain-newjsonwriter", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 336, + "totalTokens": 336, + "followUps": 0, + "stdoutBytes": 1343 + }, + { + "repository": "gson", + "question": "gson2-source-newjsonwriter", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 336, + "totalTokens": 336, + "followUps": 0, + "stdoutBytes": 1343 + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 576, + "totalTokens": 576, + "followUps": 0, + "stdoutBytes": 2304 + }, + { + "repository": "gson", + "question": "gson2-callees-newjsonwriter", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 471, + "totalTokens": 471, + "followUps": 0, + "stdoutBytes": 1883 + }, + { + "repository": "gson", + "question": "gson2-impact-newjsonwriter", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 863, + "totalTokens": 863, + "followUps": 0, + "stdoutBytes": 3449 + }, + { + "repository": "gson", + "question": "gson2-path-newjsonwriter-jsonwriter", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 41, + "totalTokens": 41, + "followUps": 0, + "stdoutBytes": 162 + }, + { + "repository": "gson", + "question": "gson2-file-path-typeadapter-jsonwriter", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 51, + "totalTokens": 51, + "followUps": 0, + "stdoutBytes": 204 + }, + { + "repository": "gson", + "question": "gson2-ambiguous-tojson", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 417, + "totalTokens": 417, + "followUps": 0, + "stdoutBytes": 1667 + }, + { + "repository": "gson", + "question": "gson2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 111, + "totalTokens": 111, + "followUps": 0, + "stdoutBytes": 441 + }, + { + "repository": "gson", + "question": "gson2-broad-deserialize", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 600, + "totalTokens": 1178, + "followUps": 1, + "stdoutBytes": 4710 + }, + { + "repository": "zod", + "question": "zod2-explain-detectversion", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 383, + "totalTokens": 383, + "followUps": 0, + "stdoutBytes": 1532 + }, + { + "repository": "zod", + "question": "zod2-source-convertschema", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 1470, + "totalTokens": 1470, + "followUps": 0, + "stdoutBytes": 5880 + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 204, + "totalTokens": 204, + "followUps": 0, + "stdoutBytes": 815 + }, + { + "repository": "zod", + "question": "zod2-callees-convertschema", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 457, + "totalTokens": 457, + "followUps": 0, + "stdoutBytes": 1828 + }, + { + "repository": "zod", + "question": "zod2-impact-detectversion", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 1309, + "totalTokens": 1309, + "followUps": 0, + "stdoutBytes": 5236 + }, + { + "repository": "zod", + "question": "zod2-path-detectversion-convertschema", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 53, + "totalTokens": 53, + "followUps": 0, + "stdoutBytes": 209 + }, + { + "repository": "zod", + "question": "zod2-file-path-json-schema", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 61, + "totalTokens": 61, + "followUps": 0, + "stdoutBytes": 242 + }, + { + "repository": "zod", + "question": "zod2-ambiguous-safeparse", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 376, + "totalTokens": 376, + "followUps": 0, + "stdoutBytes": 1503 + }, + { + "repository": "zod", + "question": "zod2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 111, + "totalTokens": 111, + "followUps": 0, + "stdoutBytes": 441 + }, + { + "repository": "zod", + "question": "zod2-broad-json-schema", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 587, + "totalTokens": 587, + "followUps": 0, + "stdoutBytes": 2347 + }, + { + "repository": "axum", + "question": "axum2-explain-methodfilter", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 209, + "totalTokens": 209, + "followUps": 0, + "stdoutBytes": 835 + }, + { + "repository": "axum", + "question": "axum2-source-validate-path", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 288, + "totalTokens": 288, + "followUps": 0, + "stdoutBytes": 1149 + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 201, + "totalTokens": 201, + "followUps": 0, + "stdoutBytes": 803 + }, + { + "repository": "axum", + "question": "axum2-callees-validate-path", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 136, + "totalTokens": 136, + "followUps": 0, + "stdoutBytes": 544 + }, + { + "repository": "axum", + "question": "axum2-impact-validate-path", + "kind": "impact", + "tool": "compass", + "passed": false, + "failures": [ + "missing 'route_endpoint'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 663, + "totalTokens": 2566, + "followUps": 3, + "stdoutBytes": 10255 + }, + { + "repository": "axum", + "question": "axum2-path-validate-v07", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 44, + "totalTokens": 44, + "followUps": 0, + "stdoutBytes": 173 + }, + { + "repository": "axum", + "question": "axum2-file-path-routing", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 45, + "totalTokens": 45, + "followUps": 0, + "stdoutBytes": 180 + }, + { + "repository": "axum", + "question": "axum2-ambiguous-with-state", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 314, + "totalTokens": 314, + "followUps": 0, + "stdoutBytes": 1255 + }, + { + "repository": "axum", + "question": "axum2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 85, + "totalTokens": 85, + "followUps": 0, + "stdoutBytes": 337 + }, + { + "repository": "axum", + "question": "axum2-broad-routing", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 580, + "totalTokens": 580, + "followUps": 0, + "stdoutBytes": 2320 + } + ], + "after": [ + { + "repository": "cobra", + "question": "cobra2-explain-parseflags", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 400, + "totalTokens": 400, + "followUps": 0, + "stdoutBytes": 1599 + }, + { + "repository": "cobra", + "question": "cobra2-source-find", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 415, + "totalTokens": 415, + "followUps": 0, + "stdoutBytes": 1657 + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 283, + "totalTokens": 283, + "followUps": 0, + "stdoutBytes": 1129 + }, + { + "repository": "cobra", + "question": "cobra2-callees-executec", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 524, + "totalTokens": 524, + "followUps": 0, + "stdoutBytes": 2096 + }, + { + "repository": "cobra", + "question": "cobra2-impact-parseflags", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 467, + "totalTokens": 467, + "followUps": 0, + "stdoutBytes": 1865 + }, + { + "repository": "cobra", + "question": "cobra2-path-find-legacyargs", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 36, + "totalTokens": 36, + "followUps": 0, + "stdoutBytes": 141 + }, + { + "repository": "cobra", + "question": "cobra2-file-path-md-docs", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 57, + "totalTokens": 57, + "followUps": 0, + "stdoutBytes": 226 + }, + { + "repository": "cobra", + "question": "cobra2-ambiguous-command", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 277, + "totalTokens": 277, + "followUps": 0, + "stdoutBytes": 1107 + }, + { + "repository": "cobra", + "question": "cobra2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 23, + "totalTokens": 23, + "followUps": 0, + "stdoutBytes": 90 + }, + { + "repository": "cobra", + "question": "cobra2-broad-resolve-run", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 598, + "totalTokens": 1193, + "followUps": 1, + "stdoutBytes": 4767 + }, + { + "repository": "flask", + "question": "flask2-explain-fulldispatch", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 538, + "totalTokens": 538, + "followUps": 0, + "stdoutBytes": 2151 + }, + { + "repository": "flask", + "question": "flask2-source-wsgiapp", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 797, + "totalTokens": 797, + "followUps": 0, + "stdoutBytes": 3187 + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 120, + "totalTokens": 120, + "followUps": 0, + "stdoutBytes": 477 + }, + { + "repository": "flask", + "question": "flask2-callees-fulldispatch", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 213, + "totalTokens": 213, + "followUps": 0, + "stdoutBytes": 849 + }, + { + "repository": "flask", + "question": "flask2-impact-fulldispatch", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 212, + "totalTokens": 212, + "followUps": 0, + "stdoutBytes": 846 + }, + { + "repository": "flask", + "question": "flask2-path-fulldispatch-finalize", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 48, + "totalTokens": 48, + "followUps": 0, + "stdoutBytes": 189 + }, + { + "repository": "flask", + "question": "flask2-file-path-views-app", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 42, + "totalTokens": 42, + "followUps": 0, + "stdoutBytes": 168 + }, + { + "repository": "flask", + "question": "flask2-ambiguous-urlfor", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 341, + "totalTokens": 341, + "followUps": 0, + "stdoutBytes": 1363 + }, + { + "repository": "flask", + "question": "flask2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 23, + "totalTokens": 23, + "followUps": 0, + "stdoutBytes": 90 + }, + { + "repository": "flask", + "question": "flask2-broad-url-rule", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 592, + "totalTokens": 592, + "followUps": 0, + "stdoutBytes": 2366 + }, + { + "repository": "gson", + "question": "gson2-explain-newjsonwriter", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 336, + "totalTokens": 336, + "followUps": 0, + "stdoutBytes": 1343 + }, + { + "repository": "gson", + "question": "gson2-source-newjsonwriter", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 336, + "totalTokens": 336, + "followUps": 0, + "stdoutBytes": 1343 + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 324, + "totalTokens": 324, + "followUps": 0, + "stdoutBytes": 1294 + }, + { + "repository": "gson", + "question": "gson2-callees-newjsonwriter", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 291, + "totalTokens": 291, + "followUps": 0, + "stdoutBytes": 1164 + }, + { + "repository": "gson", + "question": "gson2-impact-newjsonwriter", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 634, + "totalTokens": 634, + "followUps": 0, + "stdoutBytes": 2533 + }, + { + "repository": "gson", + "question": "gson2-path-newjsonwriter-jsonwriter", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 41, + "totalTokens": 41, + "followUps": 0, + "stdoutBytes": 162 + }, + { + "repository": "gson", + "question": "gson2-file-path-typeadapter-jsonwriter", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 51, + "totalTokens": 51, + "followUps": 0, + "stdoutBytes": 204 + }, + { + "repository": "gson", + "question": "gson2-ambiguous-tojson", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 397, + "totalTokens": 397, + "followUps": 0, + "stdoutBytes": 1587 + }, + { + "repository": "gson", + "question": "gson2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 46, + "totalTokens": 46, + "followUps": 0, + "stdoutBytes": 184 + }, + { + "repository": "gson", + "question": "gson2-broad-deserialize", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 587, + "totalTokens": 1166, + "followUps": 1, + "stdoutBytes": 4664 + }, + { + "repository": "zod", + "question": "zod2-explain-detectversion", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 383, + "totalTokens": 383, + "followUps": 0, + "stdoutBytes": 1532 + }, + { + "repository": "zod", + "question": "zod2-source-convertschema", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 1470, + "totalTokens": 1470, + "followUps": 0, + "stdoutBytes": 5880 + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 142, + "totalTokens": 142, + "followUps": 0, + "stdoutBytes": 567 + }, + { + "repository": "zod", + "question": "zod2-callees-convertschema", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 368, + "totalTokens": 368, + "followUps": 0, + "stdoutBytes": 1470 + }, + { + "repository": "zod", + "question": "zod2-impact-detectversion", + "kind": "impact", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 276, + "totalTokens": 276, + "followUps": 0, + "stdoutBytes": 1101 + }, + { + "repository": "zod", + "question": "zod2-path-detectversion-convertschema", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 53, + "totalTokens": 53, + "followUps": 0, + "stdoutBytes": 209 + }, + { + "repository": "zod", + "question": "zod2-file-path-json-schema", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 61, + "totalTokens": 61, + "followUps": 0, + "stdoutBytes": 242 + }, + { + "repository": "zod", + "question": "zod2-ambiguous-safeparse", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 354, + "totalTokens": 354, + "followUps": 0, + "stdoutBytes": 1413 + }, + { + "repository": "zod", + "question": "zod2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 46, + "totalTokens": 46, + "followUps": 0, + "stdoutBytes": 184 + }, + { + "repository": "zod", + "question": "zod2-broad-json-schema", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 571, + "totalTokens": 571, + "followUps": 0, + "stdoutBytes": 2283 + }, + { + "repository": "axum", + "question": "axum2-explain-methodfilter", + "kind": "explain", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 209, + "totalTokens": 209, + "followUps": 0, + "stdoutBytes": 835 + }, + { + "repository": "axum", + "question": "axum2-source-validate-path", + "kind": "explain_source", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 288, + "totalTokens": 288, + "followUps": 0, + "stdoutBytes": 1149 + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "kind": "callers", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 118, + "totalTokens": 118, + "followUps": 0, + "stdoutBytes": 469 + }, + { + "repository": "axum", + "question": "axum2-callees-validate-path", + "kind": "callees", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 100, + "totalTokens": 100, + "followUps": 0, + "stdoutBytes": 400 + }, + { + "repository": "axum", + "question": "axum2-impact-validate-path", + "kind": "impact", + "tool": "compass", + "passed": false, + "failures": [ + "missing 'route_endpoint'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 657, + "totalTokens": 2542, + "followUps": 3, + "stdoutBytes": 10159 + }, + { + "repository": "axum", + "question": "axum2-path-validate-v07", + "kind": "path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 44, + "totalTokens": 44, + "followUps": 0, + "stdoutBytes": 173 + }, + { + "repository": "axum", + "question": "axum2-file-path-routing", + "kind": "file_path", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 45, + "totalTokens": 45, + "followUps": 0, + "stdoutBytes": 180 + }, + { + "repository": "axum", + "question": "axum2-ambiguous-with-state", + "kind": "ambiguity", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 296, + "totalTokens": 296, + "followUps": 0, + "stdoutBytes": 1184 + }, + { + "repository": "axum", + "question": "axum2-negative", + "kind": "negative", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 23, + "totalTokens": 23, + "followUps": 0, + "stdoutBytes": 90 + }, + { + "repository": "axum", + "question": "axum2-broad-routing", + "kind": "broad", + "tool": "compass", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 576, + "totalTokens": 576, + "followUps": 0, + "stdoutBytes": 2304 + } + ], + "comparison": [ + { + "repository": "cobra", + "question": "cobra2-explain-parseflags", + "kind": "explain", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 134, + "totalTokens": 134, + "followUps": 0, + "stdoutBytes": 535 + }, + { + "repository": "cobra", + "question": "cobra2-source-find", + "kind": "explain_source", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'func (c *Command) Find(args []string)'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 135, + "totalTokens": 135, + "followUps": 0, + "stdoutBytes": 537 + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "kind": "callers", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 71, + "totalTokens": 71, + "followUps": 0, + "stdoutBytes": 284 + }, + { + "repository": "cobra", + "question": "cobra2-callees-executec", + "kind": "callees", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 291, + "totalTokens": 291, + "followUps": 0, + "stdoutBytes": 1163 + }, + { + "repository": "cobra", + "question": "cobra2-impact-parseflags", + "kind": "impact", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 100, + "totalTokens": 100, + "followUps": 0, + "stdoutBytes": 400 + }, + { + "repository": "cobra", + "question": "cobra2-path-find-legacyargs", + "kind": "path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 18, + "totalTokens": 18, + "followUps": 0, + "stdoutBytes": 69 + }, + { + "repository": "cobra", + "question": "cobra2-file-path-md-docs", + "kind": "file_path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 20, + "totalTokens": 20, + "followUps": 0, + "stdoutBytes": 78 + }, + { + "repository": "cobra", + "question": "cobra2-ambiguous-command", + "kind": "ambiguity", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 1, + "timedOut": false, + "firstPageTokens": 188, + "totalTokens": 188, + "followUps": 0, + "stdoutBytes": 751 + }, + { + "repository": "cobra", + "question": "cobra2-negative", + "kind": "negative", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 13, + "totalTokens": 13, + "followUps": 0, + "stdoutBytes": 52 + }, + { + "repository": "cobra", + "question": "cobra2-broad-resolve-run", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 613, + "totalTokens": 2574, + "followUps": 1, + "stdoutBytes": 10291 + }, + { + "repository": "flask", + "question": "flask2-explain-fulldispatch", + "kind": "explain", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 210, + "totalTokens": 210, + "followUps": 0, + "stdoutBytes": 838 + }, + { + "repository": "flask", + "question": "flask2-source-wsgiapp", + "kind": "explain_source", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'def wsgi_app'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 192, + "totalTokens": 192, + "followUps": 0, + "stdoutBytes": 765 + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "kind": "callers", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 65, + "totalTokens": 65, + "followUps": 0, + "stdoutBytes": 258 + }, + { + "repository": "flask", + "question": "flask2-callees-fulldispatch", + "kind": "callees", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 210, + "totalTokens": 210, + "followUps": 0, + "stdoutBytes": 838 + }, + { + "repository": "flask", + "question": "flask2-impact-fulldispatch", + "kind": "impact", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 76, + "totalTokens": 76, + "followUps": 0, + "stdoutBytes": 303 + }, + { + "repository": "flask", + "question": "flask2-path-fulldispatch-finalize", + "kind": "path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 24, + "totalTokens": 24, + "followUps": 0, + "stdoutBytes": 94 + }, + { + "repository": "flask", + "question": "flask2-file-path-views-app", + "kind": "file_path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 32, + "totalTokens": 32, + "followUps": 0, + "stdoutBytes": 125 + }, + { + "repository": "flask", + "question": "flask2-ambiguous-urlfor", + "kind": "ambiguity", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 1, + "timedOut": false, + "firstPageTokens": 67, + "totalTokens": 67, + "followUps": 0, + "stdoutBytes": 266 + }, + { + "repository": "flask", + "question": "flask2-negative", + "kind": "negative", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 14, + "totalTokens": 14, + "followUps": 0, + "stdoutBytes": 55 + }, + { + "repository": "flask", + "question": "flask2-broad-url-rule", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 612, + "totalTokens": 612, + "followUps": 0, + "stdoutBytes": 2448 + }, + { + "repository": "gson", + "question": "gson2-explain-newjsonwriter", + "kind": "explain", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 296, + "totalTokens": 296, + "followUps": 0, + "stdoutBytes": 1184 + }, + { + "repository": "gson", + "question": "gson2-source-newjsonwriter", + "kind": "explain_source", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'public JsonWriter newJsonWriter(Writer writer)'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 296, + "totalTokens": 296, + "followUps": 0, + "stdoutBytes": 1184 + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "kind": "callers", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 138, + "totalTokens": 138, + "followUps": 0, + "stdoutBytes": 552 + }, + { + "repository": "gson", + "question": "gson2-callees-newjsonwriter", + "kind": "callees", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 296, + "totalTokens": 296, + "followUps": 0, + "stdoutBytes": 1184 + }, + { + "repository": "gson", + "question": "gson2-impact-newjsonwriter", + "kind": "impact", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 15371, + "totalTokens": 15371, + "followUps": 0, + "stdoutBytes": 61482 + }, + { + "repository": "gson", + "question": "gson2-path-newjsonwriter-jsonwriter", + "kind": "path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 21, + "totalTokens": 21, + "followUps": 0, + "stdoutBytes": 82 + }, + { + "repository": "gson", + "question": "gson2-file-path-typeadapter-jsonwriter", + "kind": "file_path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 31, + "totalTokens": 31, + "followUps": 0, + "stdoutBytes": 121 + }, + { + "repository": "gson", + "question": "gson2-ambiguous-tojson", + "kind": "ambiguity", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 1, + "timedOut": false, + "firstPageTokens": 211, + "totalTokens": 211, + "followUps": 0, + "stdoutBytes": 842 + }, + { + "repository": "gson", + "question": "gson2-negative", + "kind": "negative", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 13, + "totalTokens": 13, + "followUps": 0, + "stdoutBytes": 52 + }, + { + "repository": "gson", + "question": "gson2-broad-deserialize", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 611, + "totalTokens": 611, + "followUps": 0, + "stdoutBytes": 2444 + }, + { + "repository": "zod", + "question": "zod2-explain-detectversion", + "kind": "explain", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 107, + "totalTokens": 107, + "followUps": 0, + "stdoutBytes": 425 + }, + { + "repository": "zod", + "question": "zod2-source-convertschema", + "kind": "explain_source", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'function convertSchema(schema:'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 201, + "totalTokens": 201, + "followUps": 0, + "stdoutBytes": 803 + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "kind": "callers", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 71, + "totalTokens": 71, + "followUps": 0, + "stdoutBytes": 284 + }, + { + "repository": "zod", + "question": "zod2-callees-convertschema", + "kind": "callees", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 201, + "totalTokens": 201, + "followUps": 0, + "stdoutBytes": 803 + }, + { + "repository": "zod", + "question": "zod2-impact-detectversion", + "kind": "impact", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 116, + "totalTokens": 116, + "followUps": 0, + "stdoutBytes": 462 + }, + { + "repository": "zod", + "question": "zod2-path-detectversion-convertschema", + "kind": "path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 33, + "totalTokens": 33, + "followUps": 0, + "stdoutBytes": 130 + }, + { + "repository": "zod", + "question": "zod2-file-path-json-schema", + "kind": "file_path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 24, + "totalTokens": 24, + "followUps": 0, + "stdoutBytes": 95 + }, + { + "repository": "zod", + "question": "zod2-ambiguous-safeparse", + "kind": "ambiguity", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 1, + "timedOut": false, + "firstPageTokens": 176, + "totalTokens": 176, + "followUps": 0, + "stdoutBytes": 702 + }, + { + "repository": "zod", + "question": "zod2-negative", + "kind": "negative", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 13, + "totalTokens": 13, + "followUps": 0, + "stdoutBytes": 52 + }, + { + "repository": "zod", + "question": "zod2-broad-json-schema", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 602, + "totalTokens": 602, + "followUps": 0, + "stdoutBytes": 2407 + }, + { + "repository": "axum", + "question": "axum2-explain-methodfilter", + "kind": "explain", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 431, + "totalTokens": 431, + "followUps": 0, + "stdoutBytes": 1721 + }, + { + "repository": "axum", + "question": "axum2-source-validate-path", + "kind": "explain_source", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'fn validate_path(v7_checks: bool'" + ], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 137, + "totalTokens": 137, + "followUps": 0, + "stdoutBytes": 547 + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "kind": "callers", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 79, + "totalTokens": 79, + "followUps": 0, + "stdoutBytes": 314 + }, + { + "repository": "axum", + "question": "axum2-callees-validate-path", + "kind": "callees", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 137, + "totalTokens": 137, + "followUps": 0, + "stdoutBytes": 547 + }, + { + "repository": "axum", + "question": "axum2-impact-validate-path", + "kind": "impact", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 133, + "totalTokens": 133, + "followUps": 0, + "stdoutBytes": 532 + }, + { + "repository": "axum", + "question": "axum2-path-validate-v07", + "kind": "path", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 22, + "totalTokens": 22, + "followUps": 0, + "stdoutBytes": 86 + }, + { + "repository": "axum", + "question": "axum2-file-path-routing", + "kind": "file_path", + "tool": "graphify", + "passed": false, + "failures": [ + "missing 'mod.rs'", + "missing 'path_router.rs'", + "command exited 1" + ], + "firstPagePass": false, + "exitCode": 1, + "timedOut": false, + "firstPageTokens": 0, + "totalTokens": 0, + "followUps": 0, + "stdoutBytes": 0 + }, + { + "repository": "axum", + "question": "axum2-ambiguous-with-state", + "kind": "ambiguity", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 1, + "timedOut": false, + "firstPageTokens": 119, + "totalTokens": 119, + "followUps": 0, + "stdoutBytes": 475 + }, + { + "repository": "axum", + "question": "axum2-negative", + "kind": "negative", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": true, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 14, + "totalTokens": 14, + "followUps": 0, + "stdoutBytes": 53 + }, + { + "repository": "axum", + "question": "axum2-broad-routing", + "kind": "broad", + "tool": "graphify", + "passed": true, + "failures": [], + "firstPagePass": false, + "exitCode": 0, + "timedOut": false, + "firstPageTokens": 596, + "totalTokens": 2541, + "followUps": 1, + "stdoutBytes": 10162 + } + ] + } + } + ], + "cacheRegistration": { + "cases": "First callers question in each of the five suite_v2 repositories", + "trials": 3, + "runsPerTrial": 6, + "warmMetric": "median of runs 2 through 6", + "freshCache": "new task-owned directory per binary/repository/trial", + "format": "raw JSON; require byte equality across all runs and binaries", + "timeoutSeconds": 60 + }, + "cacheAmendment": { + "reason": "Initial panel stopped at Gson: frozen graph reports incomplete publication, causing response.truncated despite only 8 nodes and 6 edges.", + "change": "Record eligibility instead of requiring every frozen graph to be complete. Keep all five questions, binaries, inputs, request limits, trials and repetitions unchanged.", + "priorAttempt": "cache-panel.log and cache-panel/ retain initial attempt captures", + "expectation": "Partial responses must remain uncached; require identical raw JSON for all cases." + }, + "cacheSummary": [ + { + "repository": "cobra", + "partial": false, + "beforeColdMedianMs": 85, + "beforeWarmMedianMs": 85, + "afterColdMedianMs": 73, + "afterWarmMedianMs": 34, + "rawJsonEqualAcrossBothBinariesAndAllRuns": true + }, + { + "repository": "flask", + "partial": false, + "beforeColdMedianMs": 348, + "beforeWarmMedianMs": 220, + "afterColdMedianMs": 386, + "afterWarmMedianMs": 34, + "rawJsonEqualAcrossBothBinariesAndAllRuns": true + }, + { + "repository": "gson", + "partial": true, + "beforeColdMedianMs": 488, + "beforeWarmMedianMs": 292, + "afterColdMedianMs": 438, + "afterWarmMedianMs": 259, + "rawJsonEqualAcrossBothBinariesAndAllRuns": true + }, + { + "repository": "zod", + "partial": false, + "beforeColdMedianMs": 967, + "beforeWarmMedianMs": 994, + "afterColdMedianMs": 897, + "afterWarmMedianMs": 34, + "rawJsonEqualAcrossBothBinariesAndAllRuns": true + }, + { + "repository": "axum", + "partial": false, + "beforeColdMedianMs": 401, + "beforeWarmMedianMs": 284, + "afterColdMedianMs": 648, + "afterWarmMedianMs": 30, + "rawJsonEqualAcrossBothBinariesAndAllRuns": true + } + ], + "cacheObservations": [ + { + "repository": "cobra", + "question": "cobra2-callers-find", + "binary": "before", + "trial": 0, + "wallMs": [ + 186, + 62, + 81, + 155, + 202, + 130 + ], + "coldMs": 186, + "warmMedianMs": 130, + "stdoutSha256": "bc4cfc163439359f79fd7686f6423b9b98241f9ef7abb001469db868b79e2947", + "responseTruncated": false + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "binary": "after", + "trial": 0, + "wallMs": [ + 406, + 71, + 36, + 32, + 37, + 39 + ], + "coldMs": 406, + "warmMedianMs": 37, + "stdoutSha256": "bc4cfc163439359f79fd7686f6423b9b98241f9ef7abb001469db868b79e2947", + "responseTruncated": false + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "binary": "after", + "trial": 1, + "wallMs": [ + 73, + 32, + 34, + 42, + 75, + 28 + ], + "coldMs": 73, + "warmMedianMs": 34, + "stdoutSha256": "bc4cfc163439359f79fd7686f6423b9b98241f9ef7abb001469db868b79e2947", + "responseTruncated": false + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "binary": "before", + "trial": 1, + "wallMs": [ + 85, + 57, + 79, + 89, + 85, + 139 + ], + "coldMs": 85, + "warmMedianMs": 85, + "stdoutSha256": "bc4cfc163439359f79fd7686f6423b9b98241f9ef7abb001469db868b79e2947", + "responseTruncated": false + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "binary": "before", + "trial": 2, + "wallMs": [ + 65, + 55, + 59, + 70, + 62, + 78 + ], + "coldMs": 65, + "warmMedianMs": 62, + "stdoutSha256": "bc4cfc163439359f79fd7686f6423b9b98241f9ef7abb001469db868b79e2947", + "responseTruncated": false + }, + { + "repository": "cobra", + "question": "cobra2-callers-find", + "binary": "after", + "trial": 2, + "wallMs": [ + 64, + 26, + 29, + 31, + 25, + 31 + ], + "coldMs": 64, + "warmMedianMs": 29, + "stdoutSha256": "bc4cfc163439359f79fd7686f6423b9b98241f9ef7abb001469db868b79e2947", + "responseTruncated": false + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "binary": "before", + "trial": 0, + "wallMs": [ + 244, + 159, + 168, + 127, + 135, + 154 + ], + "coldMs": 244, + "warmMedianMs": 154, + "stdoutSha256": "18c8f50ac82dae853fed9a5594bb8d8901e3516e5b414db9f0993735cc2c7915", + "responseTruncated": false + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "binary": "after", + "trial": 0, + "wallMs": [ + 168, + 37, + 31, + 28, + 32, + 40 + ], + "coldMs": 168, + "warmMedianMs": 32, + "stdoutSha256": "18c8f50ac82dae853fed9a5594bb8d8901e3516e5b414db9f0993735cc2c7915", + "responseTruncated": false + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "binary": "after", + "trial": 1, + "wallMs": [ + 386, + 32, + 32, + 52, + 54, + 37 + ], + "coldMs": 386, + "warmMedianMs": 37, + "stdoutSha256": "18c8f50ac82dae853fed9a5594bb8d8901e3516e5b414db9f0993735cc2c7915", + "responseTruncated": false + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "binary": "before", + "trial": 1, + "wallMs": [ + 348, + 423, + 234, + 232, + 195, + 311 + ], + "coldMs": 348, + "warmMedianMs": 234, + "stdoutSha256": "18c8f50ac82dae853fed9a5594bb8d8901e3516e5b414db9f0993735cc2c7915", + "responseTruncated": false + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "binary": "before", + "trial": 2, + "wallMs": [ + 414, + 283, + 172, + 222, + 134, + 220 + ], + "coldMs": 414, + "warmMedianMs": 220, + "stdoutSha256": "18c8f50ac82dae853fed9a5594bb8d8901e3516e5b414db9f0993735cc2c7915", + "responseTruncated": false + }, + { + "repository": "flask", + "question": "flask2-callers-fulldispatch", + "binary": "after", + "trial": 2, + "wallMs": [ + 544, + 34, + 36, + 31, + 56, + 29 + ], + "coldMs": 544, + "warmMedianMs": 34, + "stdoutSha256": "18c8f50ac82dae853fed9a5594bb8d8901e3516e5b414db9f0993735cc2c7915", + "responseTruncated": false + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "binary": "before", + "trial": 0, + "wallMs": [ + 807, + 330, + 292, + 264, + 230, + 343 + ], + "coldMs": 807, + "warmMedianMs": 292, + "stdoutSha256": "f3ea086632ef9da6e9933d44e90939ebe77c81dd83b6d253e01d29c6a5d98975", + "responseTruncated": true + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "binary": "after", + "trial": 0, + "wallMs": [ + 438, + 265, + 259, + 226, + 272, + 168 + ], + "coldMs": 438, + "warmMedianMs": 259, + "stdoutSha256": "f3ea086632ef9da6e9933d44e90939ebe77c81dd83b6d253e01d29c6a5d98975", + "responseTruncated": true + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "binary": "after", + "trial": 1, + "wallMs": [ + 335, + 209, + 153, + 144, + 140, + 192 + ], + "coldMs": 335, + "warmMedianMs": 153, + "stdoutSha256": "f3ea086632ef9da6e9933d44e90939ebe77c81dd83b6d253e01d29c6a5d98975", + "responseTruncated": true + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "binary": "before", + "trial": 1, + "wallMs": [ + 170, + 186, + 169, + 308, + 191, + 168 + ], + "coldMs": 170, + "warmMedianMs": 186, + "stdoutSha256": "f3ea086632ef9da6e9933d44e90939ebe77c81dd83b6d253e01d29c6a5d98975", + "responseTruncated": true + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "binary": "before", + "trial": 2, + "wallMs": [ + 488, + 370, + 410, + 379, + 248, + 422 + ], + "coldMs": 488, + "warmMedianMs": 379, + "stdoutSha256": "f3ea086632ef9da6e9933d44e90939ebe77c81dd83b6d253e01d29c6a5d98975", + "responseTruncated": true + }, + { + "repository": "gson", + "question": "gson2-callers-newjsonwriter", + "binary": "after", + "trial": 2, + "wallMs": [ + 521, + 462, + 302, + 199, + 311, + 478 + ], + "coldMs": 521, + "warmMedianMs": 311, + "stdoutSha256": "f3ea086632ef9da6e9933d44e90939ebe77c81dd83b6d253e01d29c6a5d98975", + "responseTruncated": true + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "binary": "before", + "trial": 0, + "wallMs": [ + 4487, + 512, + 1324, + 919, + 1913, + 994 + ], + "coldMs": 4487, + "warmMedianMs": 994, + "stdoutSha256": "7b7a0fd4daa82310d97e43d791db53d870c9030da42e3d40e398dffef623de0e", + "responseTruncated": false + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "binary": "after", + "trial": 0, + "wallMs": [ + 897, + 44, + 31, + 38, + 21, + 34 + ], + "coldMs": 897, + "warmMedianMs": 34, + "stdoutSha256": "7b7a0fd4daa82310d97e43d791db53d870c9030da42e3d40e398dffef623de0e", + "responseTruncated": false + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "binary": "after", + "trial": 1, + "wallMs": [ + 588, + 32, + 30, + 32, + 20, + 31 + ], + "coldMs": 588, + "warmMedianMs": 31, + "stdoutSha256": "7b7a0fd4daa82310d97e43d791db53d870c9030da42e3d40e398dffef623de0e", + "responseTruncated": false + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "binary": "before", + "trial": 1, + "wallMs": [ + 463, + 512, + 485, + 446, + 537, + 468 + ], + "coldMs": 463, + "warmMedianMs": 485, + "stdoutSha256": "7b7a0fd4daa82310d97e43d791db53d870c9030da42e3d40e398dffef623de0e", + "responseTruncated": false + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "binary": "before", + "trial": 2, + "wallMs": [ + 967, + 700, + 2278, + 2332, + 2548, + 1081 + ], + "coldMs": 967, + "warmMedianMs": 2278, + "stdoutSha256": "7b7a0fd4daa82310d97e43d791db53d870c9030da42e3d40e398dffef623de0e", + "responseTruncated": false + }, + { + "repository": "zod", + "question": "zod2-callers-detectversion", + "binary": "after", + "trial": 2, + "wallMs": [ + 8135, + 44, + 89, + 91, + 28, + 83 + ], + "coldMs": 8135, + "warmMedianMs": 83, + "stdoutSha256": "7b7a0fd4daa82310d97e43d791db53d870c9030da42e3d40e398dffef623de0e", + "responseTruncated": false + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "binary": "before", + "trial": 0, + "wallMs": [ + 560, + 970, + 628, + 455, + 564, + 287 + ], + "coldMs": 560, + "warmMedianMs": 564, + "stdoutSha256": "03dc5f663161d32ed1b04109fed7124bb2741e54aca99d5130b9c97dd5161d25", + "responseTruncated": false + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "binary": "after", + "trial": 0, + "wallMs": [ + 1390, + 794, + 82, + 266, + 122, + 36 + ], + "coldMs": 1390, + "warmMedianMs": 122, + "stdoutSha256": "03dc5f663161d32ed1b04109fed7124bb2741e54aca99d5130b9c97dd5161d25", + "responseTruncated": false + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "binary": "after", + "trial": 1, + "wallMs": [ + 648, + 33, + 27, + 28, + 30, + 39 + ], + "coldMs": 648, + "warmMedianMs": 30, + "stdoutSha256": "03dc5f663161d32ed1b04109fed7124bb2741e54aca99d5130b9c97dd5161d25", + "responseTruncated": false + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "binary": "before", + "trial": 1, + "wallMs": [ + 306, + 267, + 232, + 341, + 287, + 284 + ], + "coldMs": 306, + "warmMedianMs": 284, + "stdoutSha256": "03dc5f663161d32ed1b04109fed7124bb2741e54aca99d5130b9c97dd5161d25", + "responseTruncated": false + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "binary": "before", + "trial": 2, + "wallMs": [ + 401, + 541, + 115, + 128, + 197, + 117 + ], + "coldMs": 401, + "warmMedianMs": 128, + "stdoutSha256": "03dc5f663161d32ed1b04109fed7124bb2741e54aca99d5130b9c97dd5161d25", + "responseTruncated": false + }, + { + "repository": "axum", + "question": "axum2-callers-validate-path", + "binary": "after", + "trial": 2, + "wallMs": [ + 172, + 34, + 19, + 18, + 30, + 29 + ], + "coldMs": 172, + "warmMedianMs": 29, + "stdoutSha256": "03dc5f663161d32ed1b04109fed7124bb2741e54aca99d5130b9c97dd5161d25", + "responseTruncated": false + } + ], + "limits": [ + "Five public source trees; text recall proxy does not independently measure precision.", + "Ordinary OS caches and unrelated host load are uncontrolled.", + "Archived competitor graphs and exact installed 0.9.67 comparator; no new extraction or dependency.", + "CLI stdout costs exclude comparator installation warnings recorded on stderr.", + "The debug replay had timeouts and was not used for optimized timing or final verdicts.", + "Partial Gson responses remained uncached; the panel does not claim acceleration for partial answers.", + "MCP compact text does not reduce the unchanged full structured wire contract.", + "Two existing native failures remain: cobra-natural-broad lacks Find; axum2-impact-validate-path lacks route_endpoint." + ] +} diff --git a/crates/compass-cli/assets/compass-skill/references/operations.md b/crates/compass-cli/assets/compass-skill/references/operations.md index a80ecb69..8aebf8c7 100644 --- a/crates/compass-cli/assets/compass-skill/references/operations.md +++ b/crates/compass-cli/assets/compass-skill/references/operations.md @@ -53,3 +53,14 @@ equivalent. Write merged output to a new file first. For any unfamiliar recovery option, run `compass --help`. Prefer a normal `compass update` or `compass extract` when the build can be reproduced cleanly. + +## Bounded answers + +Use `--budget N` to cap completed command output at N approximate tokens +(UTF-8 bytes divided by four). Typed query text is compact by default; add +`--verbose` or `--evidence` for audit detail. Preserve match and coverage caveats +when summarizing an answer. Native pages continue with `--cursor`; other +commands print `compass output ID --offset N --budget N` to read the saved +output without repeating the command. Saved output is worktree-local and +bounded to sixteen records. Streaming `watch`, `serve`, and JSONL sessions reject +this presentation control; noninteractive `init --yes` supports it. diff --git a/crates/compass-cli/src/bin/compass.rs b/crates/compass-cli/src/bin/compass.rs index 64b6547f..d7e0225c 100644 --- a/crates/compass-cli/src/bin/compass.rs +++ b/crates/compass-cli/src/bin/compass.rs @@ -54,6 +54,31 @@ fn main() -> ExitCode { std::env::var_os("NO_COLOR").as_deref(), std::env::var_os("TERM").as_deref(), ); + let budgeted = arguments.iter().any(|arg| { + arg.to_str() + .is_some_and(|value| value == "--budget" || value.starts_with("--budget=")) + }); + if budgeted + && (events + || arguments.iter().any(|arg| arg == "--repl") + || arguments + .first() + .and_then(|arg| arg.to_str()) + .is_some_and(|command| matches!(command, "watch" | "serve"))) + { + eprintln!( + "error: --budget requires completed output; streaming events, watch, serve and REPL cannot be paged" + ); + return ExitCode::from(2); + } + if budgeted && arguments.first().and_then(|arg| arg.to_str()) != Some("init") { + let outcome = compass_cli::run(compass_cli::Frontend::Compass, arguments); + return ExitCode::from(compass_cli::write_outcome( + &outcome, + &mut io::stdout(), + &mut io::stderr(), + )); + } if let Some(outcome) = compass_cli::compass_help_request(&arguments, style) { return ExitCode::from(compass_cli::write_outcome( &outcome, diff --git a/crates/compass-cli/src/code_query_commands.rs b/crates/compass-cli/src/code_query_commands.rs index 602de154..c7b877a4 100644 --- a/crates/compass-cli/src/code_query_commands.rs +++ b/crates/compass-cli/src/code_query_commands.rs @@ -9,7 +9,7 @@ use compass_model::query_contract::{ use compass_output::{ AgentOperandRole, AgentQueryContext, AgentTextPageOptions, DEFAULT_AGENT_TEXT_PAGE_TOKENS, build_code_query_brief, build_code_query_view, decode_agent_text_page_cursor, - render_code_query_text_page, + render_code_query_evidence_page, render_compact_code_query_text_page, }; use compass_query::{ EngineSelection, ExactSearchFilter, NaturalQueryIntent, NaturalQueryRequest, QueryError, @@ -32,13 +32,13 @@ pub(crate) fn command(operation: &str, args: &[String]) -> Outcome { && query_args.iter().any(|arg| { matches!( arg.as_str(), - "--cursor" | "--text-budget" | "--evidence" | "--result-envelope" + "--cursor" | "--text-budget" | "--evidence" | "--verbose" | "--result-envelope" ) || arg.starts_with("--cursor=") || arg.starts_with("--text-budget=") }) { return Outcome::failure( - "error: --cursor, --text-budget, --evidence, and --result-envelope are text-only and require --format text".to_owned(), + "error: --cursor, --text-budget, --verbose, --evidence, and --result-envelope are text-only and require --format text".to_owned(), ); } let text_budget = match number(&query_args, "--text-budget", DEFAULT_AGENT_TEXT_PAGE_TOKENS) { @@ -66,7 +66,7 @@ pub(crate) fn command(operation: &str, args: &[String]) -> Outcome { let result = if format == SharedOutputFormat::Text { execute_paged(operation, &query_args, deadline) } else { - execute(operation, &query_args, 1, deadline) + execute(operation, &query_args, 1, deadline, &mut None) }; match result { Ok(execution) => { @@ -88,7 +88,15 @@ pub(crate) fn command(operation: &str, args: &[String]) -> Outcome { Err(error) => Outcome::failure(format!("error: {error}")), } } else if format == SharedOutputFormat::Text { - match render_code_query_text_page( + let renderer = if query_args + .iter() + .any(|arg| matches!(arg.as_str(), "--evidence" | "--verbose")) + { + render_code_query_evidence_page + } else { + render_compact_code_query_text_page + }; + match renderer( &execution.response, execution.context, AgentTextPageOptions { @@ -132,8 +140,9 @@ fn execute_paged( deadline: Instant, ) -> Result { let mut scale = 1_u32; + let mut engine = None; loop { - let execution = execute(operation, args, scale, deadline)?; + let execution = execute(operation, args, scale, deadline, &mut engine)?; if !execution.response.truncated || scale >= MAX_PAGE_WIDENING_SCALE || args.iter().any(|arg| arg == "--exact") @@ -149,6 +158,7 @@ fn execute( args: &[String], page_scale: u32, deadline: Instant, + reused_engine: &mut Option, ) -> Result { let calls_only = args.iter().any(|arg| arg == "--calls-only"); if args.iter().any(|arg| arg.starts_with("--calls-only=")) { @@ -235,7 +245,9 @@ fn execute( .map(PathBuf::from) .map(resolve_snapshot_artifact) .transpose()?; - let engine = if let Some(revision) = revision { + let engine = if let Some(engine) = reused_engine.take() { + engine + } else if let Some(revision) = revision { let (realization, document) = super::history_commands::load_typed_graph_at(revision)?; let current = std::env::current_dir().map_err(|error| error.to_string())?; let history_cache = current @@ -412,6 +424,7 @@ fn execute( for (role, value) in operands { context = context.with_operand(role, value); } + *reused_engine = Some(engine); Ok(QueryExecution { response, context }) } diff --git a/crates/compass-cli/src/help.rs b/crates/compass-cli/src/help.rs index 8408b164..2dd16560 100644 --- a/crates/compass-cli/src/help.rs +++ b/crates/compass-cli/src/help.rs @@ -107,6 +107,7 @@ const GROUPS: &[Group] = &[ "explain", "affected", "benchmark", + "output", ], }, Group { @@ -148,6 +149,12 @@ const GROUPS: &[Group] = &[ ]; const PAGES: &[Page] = &[ + page!( + "output", + "Read the rest of an immutable saved command output", + ["compass output [--offset N] [--budget N]"], + "Arguments:\n Saved answer ID printed by a budgeted command\n\nOptions:\n --offset Continue at the printed UTF-8 byte offset\n --budget Approximate token budget [default: 2000; range: 32-65536]\n\nSaved answers live in .compass/cache/output and retain at most 16 records of 4 MiB each. Reading them never repeats the original command. Digest or schema mismatches fail explicitly." + ), page!( "agent-graph", "Manage grounded agent-authored graph overlays", @@ -176,7 +183,7 @@ const PAGES: &[Page] = &[ "ask", "Route a natural-language question to a typed code-graph query", ["compass ask [OPTIONS]"], - "Arguments:\n Natural-language code-graph question\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --at Use an immutable trusted revision graph\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-depth Traversal radius\n --max-nodes Node bound\n --max-edges Edge bound\n --max-paths Path bound\n --max-candidates Candidate bound [default: 64]\n --semantic-search Optional offline corpus-LSA fallback [default: off]\n --include-heuristic Include heuristic evidence\n --format Output format [default: text]\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass ask \"who calls PaymentService.charge?\"\n compass ask \"what does CheckoutController.create call?\" --format json\n compass ask \"path from CheckoutController.create to PaymentGateway.charge\"\n compass ask \"who calls PaymentService.charge?\" --at HEAD~2 --format json\n\nNotes:\n High-confidence callers, callees, impact, and path questions route to the matching typed operation. Contradictory or low-confidence input falls back to bounded symbol search. --at is mutually exclusive with --graph, --program, --cache, and --engine. The response uses compass.query/1." + "Arguments:\n Natural-language code-graph question\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --at Use an immutable trusted revision graph\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-depth Traversal radius\n --max-nodes Node bound\n --max-edges Edge bound\n --max-paths Path bound\n --max-candidates Candidate bound [default: 64]\n --semantic-search Optional offline corpus-LSA fallback [default: off]\n --include-heuristic Include heuristic evidence\n --format Output format [default: text]\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass ask \"who calls PaymentService.charge?\"\n compass ask \"what does CheckoutController.create call?\" --format json\n compass ask \"path from CheckoutController.create to PaymentGateway.charge\"\n compass ask \"who calls PaymentService.charge?\" --at HEAD~2 --format json\n\nNotes:\n High-confidence callers, callees, impact, and path questions route to the matching typed operation. Contradictory or low-confidence input falls back to bounded symbol search. --at is mutually exclusive with --graph, --program, --cache, and --engine. The response uses compass.query/1." ), page!( "call-graph", @@ -320,37 +327,37 @@ const PAGES: &[Page] = &[ "search", "Search typed code symbols by name", ["compass search [OPTIONS]"], - "Arguments:\n Symbol name or qualified name\n\nOptions:\n --exact Exact ID or normalized name only; no lexical fallback\n --file Exact stored source path (requires --exact)\n --line Declaration start line (requires --exact and --file)\n --kind Stored node kind, e.g. function (requires --exact)\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-candidates Candidate bound [default: 64]\n --semantic-search Optional offline corpus-LSA fallback [default: off]\n --format Output format [default: text]\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass search PaymentService\n compass search checkout --format json\n compass search PaymentService --text-budget 800\n\nNotes:\n Exact mode retains all matches. Candidate bounds apply before filters; truncation never proves uniqueness or absence. Exact text results do not automatically widen bounds.\n Search uses the versioned compass.query/1 response contract in all formats. Text output is paged: the footer carries a cursor that continues the same deterministic result at the same page budget." + "Arguments:\n Symbol name or qualified name\n\nOptions:\n --exact Exact ID or normalized name only; no lexical fallback\n --file Exact stored source path (requires --exact)\n --line Declaration start line (requires --exact and --file)\n --kind Stored node kind, e.g. function (requires --exact)\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-candidates Candidate bound [default: 64]\n --semantic-search Optional offline corpus-LSA fallback [default: off]\n --format Output format [default: text]\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass search PaymentService\n compass search checkout --format json\n compass search PaymentService --text-budget 800\n\nNotes:\n Exact mode retains all matches. Candidate bounds apply before filters; truncation never proves uniqueness or absence. Exact text results do not automatically widen bounds.\n Search uses the versioned compass.query/1 response contract in all formats. Text output is paged: the footer carries a cursor that continues the same deterministic result at the same page budget." ), page!( "callers", "List direct incoming usages of a typed symbol", ["compass callers [OPTIONS]"], - "Arguments:\n Symbol ID, name, or qualified name\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-nodes Node bound\n --max-edges Edge bound\n --include-heuristic Include heuristic evidence (default is exact-first)\n --format Output format [default: text]\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass callers PaymentService.charge\n compass callers sym:checkout --format json\n compass callers PaymentService.charge --text-budget 800\n\nNotes:\n Incoming usages include calls, routes, references, imports, exports, and aliases; each result retains its exact relationship kind. Text output is paged: the footer carries a cursor that continues the same deterministic result at the same page budget." + "Arguments:\n Symbol ID, name, or qualified name\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-nodes Node bound\n --max-edges Edge bound\n --include-heuristic Include heuristic evidence (default is exact-first)\n --format Output format [default: text]\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass callers PaymentService.charge\n compass callers sym:checkout --format json\n compass callers PaymentService.charge --text-budget 800\n\nNotes:\n Incoming usages include calls, routes, references, imports, exports, and aliases; each result retains its exact relationship kind. Text output is paged: the footer carries a cursor that continues the same deterministic result at the same page budget." ), page!( "callees", "List direct callees of a typed symbol", ["compass callees [OPTIONS]"], - "Arguments:\n Symbol ID, name, or qualified name\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-nodes Node bound\n --max-edges Edge bound\n --include-heuristic Include heuristic evidence (default is exact-first)\n --format Output format [default: text]\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass callees CheckoutController.create\n compass callees sym:checkout --format json" + "Arguments:\n Symbol ID, name, or qualified name\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-nodes Node bound\n --max-edges Edge bound\n --include-heuristic Include heuristic evidence (default is exact-first)\n --format Output format [default: text]\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass callees CheckoutController.create\n compass callees sym:checkout --format json" ), page!( "impact", "Compute the bounded transitive impact of a symbol", ["compass impact [OPTIONS]"], - "Arguments:\n Changed symbol ID, name, or qualified name\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-depth Traversal radius\n --max-nodes Node bound\n --max-edges Edge bound\n --relation Follow stored relationship type; repeatable\n --include-heuristic Traverse heuristic evidence\n --format Output format [default: text]\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass impact PaymentGateway --max-depth 3\n compass impact sym:gateway --include-heuristic --format json" + "Arguments:\n Changed symbol ID, name, or qualified name\n\nOptions:\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-depth Traversal radius\n --max-nodes Node bound\n --max-edges Edge bound\n --relation Follow stored relationship type; repeatable\n --include-heuristic Traverse heuristic evidence\n --format Output format [default: text]\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass impact PaymentGateway --max-depth 3\n compass impact sym:gateway --include-heuristic --format json" ), page!( "explore", "Return related source grouped by file with connecting paths", ["compass explore [SYMBOL...] [OPTIONS]"], - "Arguments:\n Symbol IDs, names, or qualified names\n\nOptions:\n --root Repository root used to read source\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-paths Path bound\n --max-source-bytes Source-byte bound\n --max-response-bytes Serialized response bound\n --include-heuristic Include heuristic evidence (default is exact-first)\n --format Output format [default: text]\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass explore CheckoutController PaymentGateway --root .\n compass explore sym:a sym:b --root . --format json" + "Arguments:\n Symbol IDs, names, or qualified names\n\nOptions:\n --root Repository root used to read source\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-paths Path bound\n --max-source-bytes Source-byte bound\n --max-response-bytes Serialized response bound\n --include-heuristic Include heuristic evidence (default is exact-first)\n --format Output format [default: text]\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n\nExamples:\n compass explore CheckoutController PaymentGateway --root .\n compass explore sym:a sym:b --root . --format json" ), page!( "node", "Show a bounded evidence trail between two symbols", ["compass node [OPTIONS]"], - "Arguments:\n Trail origin symbol\n Trail destination symbol\n\nOptions:\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-depth Traversal radius\n --max-paths Path bound\n --calls-only Follow only directed calls edges\n --include-heuristic Include heuristic evidence\n --format Output format [default: text]\n\nExamples:\n compass node route:/checkout CheckoutController.create\n compass node sym:a sym:b --include-heuristic --format json\n compass node Caller Target --calls-only" + "Arguments:\n Trail origin symbol\n Trail destination symbol\n\nOptions:\n --budget Bound completed output, including continuation\n --verbose, --evidence Show full status and evidence\n --text-budget Approximate tokens per text page [default: 2000]\n --cursor Continue the same text result (text only)\n --timeout-ms Query deadline in milliseconds [default: 60000; max: 600000]\n --brief Emit compass.query.agent-view.brief/1 (agent-json only)\n --graph Typed graph [default: compass-out/graph.json]\n --program Optional Program IR enrichment\n --cache Query-index cache directory\n --engine Graph storage engine [default: default]\n --max-depth Traversal radius\n --max-paths Path bound\n --calls-only Follow only directed calls edges\n --include-heuristic Include heuristic evidence\n --format Output format [default: text]\n\nExamples:\n compass node route:/checkout CheckoutController.create\n compass node sym:a sym:b --include-heuristic --format json\n compass node Caller Target --calls-only" ), page!( "query", @@ -1068,7 +1075,7 @@ fn render_root(style: HelpStyle) -> String { let _ = output.pop(); } output.push_str( - "\n\nOptions:\n -h, --help Show help\n -V, --version Show version\n\nRun `compass help ` for detailed help.", + "\n\nOptions:\n -h, --help Show help\n -V, --version Show version\n --budget N Bound completed output (UTF-8 bytes / 4); continue with the printed command\n\nRun `compass help ` for detailed help.", ); finish(output, style) } @@ -1334,7 +1341,7 @@ mod tests { #[test] fn catalog_has_unique_complete_public_roots() { let roots = root_commands(); - assert_eq!(roots.len(), 54); + assert_eq!(roots.len(), 55); for root in roots { let matches = PAGES.iter().filter(|page| page.path == root).count(); assert_eq!(matches, 1, "{root}"); diff --git a/crates/compass-cli/src/init_commands.rs b/crates/compass-cli/src/init_commands.rs index 6139ae66..a92eef9a 100644 --- a/crates/compass-cli/src/init_commands.rs +++ b/crates/compass-cli/src/init_commands.rs @@ -36,6 +36,53 @@ pub fn run_init( stderr: &mut impl Write, input_is_terminal: bool, ) -> u8 { + let mut strings = arguments + .iter() + .map(|value| value.to_string_lossy().into_owned()) + .collect::>(); + let controls = match crate::output_budget::take(&mut strings) { + Ok(controls) => controls, + Err(error) => { + let _ = writeln!(stderr, "error: {error}"); + return 2; + } + }; + if controls.budget.is_some() { + if !strings.iter().any(|arg| arg == "--yes") { + let _ = writeln!( + stderr, + "error: init --budget requires --yes; interactive prompts cannot be saved for later reading" + ); + return 2; + } + let args = strings.into_iter().map(OsString::from).collect::>(); + let mut captured_out = crate::output_budget::Capture::default(); + let mut captured_err = crate::output_budget::Capture::default(); + let code = run_init( + &args, + input, + &mut captured_out, + &mut captured_err, + input_is_terminal, + ); + let outcome = if captured_out.exceeded || captured_err.exceeded { + Outcome::failure("error: completed init output exceeds 4 MiB".to_owned()) + } else { + Outcome { + code, + stdout: String::from_utf8_lossy(&captured_out.bytes).into_owned(), + stderr: String::from_utf8_lossy(&captured_err.bytes).into_owned(), + stdout_trailing_newline: false, + stderr_trailing_newline: false, + html_output: None, + } + }; + let mut outcome = crate::output_budget::finish(outcome, &controls); + // Budgeted page strings do not already contain their final newline. + outcome.stdout_trailing_newline = !outcome.stdout.ends_with('\n'); + outcome.stderr_trailing_newline = !outcome.stderr.ends_with('\n'); + return write_outcome(&outcome, stdout, stderr); + } run_init_with_builder( arguments, input, diff --git a/crates/compass-cli/src/lib.rs b/crates/compass-cli/src/lib.rs index 306d22a4..a8c83bf2 100644 --- a/crates/compass-cli/src/lib.rs +++ b/crates/compass-cli/src/lib.rs @@ -18,6 +18,7 @@ mod install_commands; mod integration_commands; mod label_commands; mod models_commands; +mod output_budget; mod program_commands; mod provider_commands; mod prs_commands; @@ -78,9 +79,10 @@ use compass_output::{ graph_artifact_identity, graph_community_view_model_document, graph_search_index, graph_view_model_bundle_document_with_hierarchy, graph_view_model_document, node_filenames, project_architecture, project_architecture_or_summary, render_agent_query_continuation_header, - render_agent_query_header_lines, render_agent_query_text, render_orientation_json, - validate_orientation_graph_identity, write_callflow_html, write_canvas, write_cypher, - write_graphml, write_svg, write_tree_html, write_workbench_html_with_source_navigation, + render_agent_query_header_lines, render_agent_query_text, render_compact_query_header_lines, + render_orientation_json, validate_orientation_graph_identity, write_callflow_html, + write_canvas, write_cypher, write_graphml, write_svg, write_tree_html, + write_workbench_html_with_source_navigation, }; use compass_prs::{ProcessRunner, SystemRunner}; use compass_query::{ @@ -425,6 +427,57 @@ fn write_output( #[must_use] pub fn run(frontend: Frontend, arguments: impl IntoIterator) -> Outcome { + let mut args = arguments + .into_iter() + .map(|argument| argument.to_string_lossy().into_owned()) + .collect::>(); + let controls = match output_budget::take(&mut args) { + Ok(controls) => controls, + Err(error) => return Outcome::failure(format!("error: {error}")), + }; + if args.first().is_some_and(|command| command == "output") { + return output_budget::command(&args[1..], controls.budget); + } + if let Some(budget) = controls.budget { + let command = args.first().map(String::as_str).unwrap_or_default(); + let native = matches!( + command, + "ask" | "search" | "callers" | "callees" | "impact" | "explore" | "node" + ) || (command == "query" + && !args.iter().any(|arg| { + matches!( + arg.split('=').next().unwrap_or_default(), + "--traverse" | "--page" | "--cql" | "--repl" | "--stdin" | "--file" + ) + })); + if !controls.machine + && (command == "explain" + || (command == "query" + && !native + && !args.iter().any(|arg| { + matches!( + arg.split('=').next().unwrap_or_default(), + "--cql" | "--repl" | "--stdin" | "--file" + ) + }))) + { + args.extend(["--budget".to_owned(), budget.to_string()]); + } + let machine = controls.machine; + if native + && !machine + && !args + .iter() + .any(|arg| arg == "--text-budget" || arg.starts_with("--text-budget=")) + { + args.extend(["--text-budget".to_owned(), budget.to_string()]); + } + } + let outcome = run_unbudgeted(frontend, args.into_iter().map(OsString::from)); + output_budget::finish(outcome, &controls) +} + +fn run_unbudgeted(frontend: Frontend, arguments: impl IntoIterator) -> Outcome { let mut args = arguments .into_iter() .map(|argument| argument.to_string_lossy().into_owned()) @@ -459,6 +512,7 @@ pub fn run(frontend: Frontend, arguments: impl IntoIterator) -> command.clone() }; let outcome = match command.as_str() { + "output" => output_budget::command(&args, None), "history" => history_commands::command(frontend, &args), "agent-graph" => agent_graph_commands::command(&args), "call-graph" => call_graph_commands::command(frontend, &args), @@ -5773,7 +5827,7 @@ pub(crate) fn command_natural_query(frontend: Frontend, args: &[String]) -> Outc discovery_requested = true; index += 1; } - "--evidence" => { + "--evidence" | "--verbose" => { if !seen_discovery_options.insert("--evidence".to_owned()) { return Outcome::failure("error: --evidence must not be repeated".to_owned()); } @@ -6112,7 +6166,9 @@ fn command_discovery_query( Ok(view) => view, Err(error) => return Outcome::failure(format!("error: {error}")), }; - let header = if cursor.is_some() { + let header = if !include_evidence { + render_compact_query_header_lines(&view) + } else if cursor.is_some() { render_agent_query_continuation_header(&view) } else { render_agent_query_header_lines(&view) @@ -7287,7 +7343,7 @@ fn touch_selected_query_stamp(selection: &GraphSelection) { fn query_help(frontend: Frontend) -> String { let prefix = frontend_name(frontend); let help = format!( - "Usage: {prefix} query \"\" [--direction auto|incoming|outgoing|both] [--scope KIND:VALUE] [--context VALUE] [--dfs] [--evidence] [--format text|agent-json|json] [--graph PATH|--at REV]\n\nNatural discovery options (default for a typed graph):\n --direction Direction: auto, incoming, outgoing, or both [default: auto]\n --scope Repeatable OR scope; KIND is community, source, package, or node\n --context Repeatable strict relationship-context filter\n --dfs Use depth-first expansion [default: breadth-first]\n --include-heuristic Include heuristic evidence [default: excluded]\n --evidence Include full provenance and typed detail in text\n --format Discovery output [default: text]\n --text-budget Approximate tokens in one text page [default: 800]\n --cursor Continue the same immutable semantic result (text only)\n --max-depth Traversal depth [default: 2; hard maximum: 8]\n --max-seeds Ranked seed count [default: 3; hard maximum: 3]\n --max-candidates Ranked candidate count [default/hard maximum: 256]\n --max-nodes Returned node count [default/hard maximum: 500]\n --max-edges Returned edge count [default/hard maximum: 1000]\n --max-expanded-relationships Examined relationships [default/hard maximum: 10000]\n --max-response-bytes Serialized response bytes [default/hard maximum: 8388608]\n --timeout-ms Discovery deadline in milliseconds [default/hard maximum: 30000]\n\nLegacy traversal options:\n --traverse Force legacy relevance traversal\n --budget Approximate tokens per page [default: 2000]\n --page Result page, starting at 1 [default: 1]\n\nGraph selection:\n --graph Read a graph JSON file\n --at Resolve REV once to an immutable typed realization; conflicts with --graph\n\nCompassQL options:\n --cql Use CompassQL mode\n --timeout-ms CompassQL execution timeout\n --max-expanded-relationships CompassQL relationship expansion limit\n Run `{prefix} help query` for all CompassQL controls and examples.\n\nDiscovery limits must be positive; values above a hard maximum are rejected rather than clamped. JSON rejects text-only pagination/evidence controls. Legacy --traverse, --budget, and --page cannot be mixed with discovery controls." + "Usage: {prefix} query \"\" [--direction auto|incoming|outgoing|both] [--scope KIND:VALUE] [--context VALUE] [--dfs] [--evidence] [--format text|agent-json|json] [--graph PATH|--at REV]\n\nNatural discovery options (default for a typed graph):\n --direction Direction: auto, incoming, outgoing, or both [default: auto]\n --scope Repeatable OR scope; KIND is community, source, package, or node\n --context Repeatable strict relationship-context filter\n --dfs Use depth-first expansion [default: breadth-first]\n --include-heuristic Include heuristic evidence [default: excluded]\n --evidence Include full provenance and typed detail in text\n --format Discovery output [default: text]\n --text-budget Approximate tokens in one text page [default: 800]\n --cursor Continue the same immutable semantic result (text only)\n --max-depth Traversal depth [default: 2; hard maximum: 8]\n --max-seeds Ranked seed count [default: 3; hard maximum: 3]\n --max-candidates Ranked candidate count [default/hard maximum: 256]\n --max-nodes Returned node count [default/hard maximum: 500]\n --max-edges Returned edge count [default/hard maximum: 1000]\n --max-expanded-relationships Examined relationships [default/hard maximum: 10000]\n --max-response-bytes Serialized response bytes [default/hard maximum: 8388608]\n --timeout-ms Discovery deadline in milliseconds [default/hard maximum: 30000]\n\nLegacy traversal options:\n --traverse Force legacy relevance traversal\n --budget Bound completed output [default text page: 2000]\n --page Result page, starting at 1 [default: 1]\n\nGraph selection:\n --graph Read a graph JSON file\n --at Resolve REV once to an immutable typed realization; conflicts with --graph\n\nCompassQL options:\n --cql Use CompassQL mode\n --timeout-ms CompassQL execution timeout\n --max-expanded-relationships CompassQL relationship expansion limit\n Run `{prefix} help query` for all CompassQL controls and examples.\n\nDiscovery limits must be positive; values above a hard maximum are rejected rather than clamped. JSON rejects text-only pagination/evidence controls. Legacy --traverse and --page cannot be mixed with discovery controls. --budget applies to all finite commands." ); let help = help .replace( diff --git a/crates/compass-cli/src/output_budget.rs b/crates/compass-cli/src/output_budget.rs new file mode 100644 index 00000000..fdec73ff --- /dev/null +++ b/crates/compass-cli/src/output_budget.rs @@ -0,0 +1,380 @@ +//! A common budget for completed outputs, including commands without a native pager. +use crate::Outcome; +use compass_files::{read_bytes_bounded, write_json_atomic_new}; +use compass_output::render_budgeted_text; +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use std::path::{Path, PathBuf}; + +const MAX_OUTPUT_BYTES: u64 = 4 * 1024 * 1024; +const MAX_SAVED_OUTPUTS: usize = 16; +const SCHEMA: &str = "compass.saved-output/1"; + +#[derive(Default)] +pub(crate) struct Controls { + pub budget: Option, + pub machine: bool, +} + +pub(crate) fn take(args: &mut Vec) -> Result { + let mut controls = Controls { + machine: machine_requested(args), + ..Controls::default() + }; + let mut index = 0; + while index < args.len() { + let (value, count) = if args[index] == "--budget" { + ( + args.get(index + 1) + .ok_or("--budget requires a positive integer")? + .as_str(), + 2, + ) + } else if let Some(value) = args[index].strip_prefix("--budget=") { + (value, 1) + } else { + index += 1; + continue; + }; + if controls.budget.is_some() { + return Err("--budget may be supplied only once".to_owned()); + } + let budget = value + .parse::() + .map_err(|_| "--budget requires a positive integer")?; + if !(32..=65_536).contains(&budget) { + return Err("--budget must be between 32 and 65536".to_owned()); + } + controls.budget = Some(budget); + args.drain(index..index + count); + } + Ok(controls) +} + +pub(crate) fn machine_requested(args: &[String]) -> bool { + args.iter().enumerate().any(|(index, arg)| { + let value = arg.strip_prefix("--format=").or_else(|| { + (arg == "--format") + .then(|| args.get(index + 1).map(String::as_str)) + .flatten() + }); + value.is_some_and(|value| matches!(value, "json" | "agent-json" | "jsonl")) + || matches!(arg.as_str(), "--json" | "--events=jsonl") + }) +} + +fn bounded_error(message: &str, budget: usize) -> Outcome { + let limit = budget.saturating_mul(4).saturating_sub(1); + let text = format!("error: {message}"); + if text.len() <= limit { + return Outcome::failure(text); + } + Outcome::failure("error: output unavailable; rerun with --budget 256".to_owned()) +} + +#[derive(Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +struct SavedOutput { + schema: String, + code: u8, + text: String, +} + +fn directory() -> Result { + std::env::current_dir() + .and_then(std::fs::canonicalize) + .map(|path| path.join(".compass/cache/output")) + .map_err(|error| error.to_string()) +} +fn identity(bytes: &[u8]) -> String { + format!("{:x}", Sha256::digest(bytes)) +} +fn is_identity(value: &str) -> bool { + value.len() == 64 && value.bytes().all(|byte| byte.is_ascii_hexdigit()) +} + +fn save(directory: &Path, bytes: &[u8]) -> Result { + // Saved output may contain source excerpts. Never follow a redirected + // cache directory or a redirected record while publishing it. + for ancestor in directory.ancestors().take(3) { + if ancestor.is_symlink() { + return Err("saved-output directory is a symlink".to_owned()); + } + } + #[cfg(unix)] + { + use std::os::unix::fs::DirBuilderExt; + std::fs::DirBuilder::new() + .recursive(true) + .mode(0o700) + .create(directory) + .map_err(|error| error.to_string())?; + } + #[cfg(not(unix))] + std::fs::create_dir_all(directory).map_err(|error| error.to_string())?; + let id = identity(bytes); + let path = directory.join(format!("{id}.json")); + if path.is_symlink() { + return Err("saved-output record is a symlink".to_owned()); + } + if path.exists() { + let existing = + read_bytes_bounded(&path, MAX_OUTPUT_BYTES).map_err(|error| error.to_string())?; + let record: SavedOutput = + serde_json::from_slice(&existing).map_err(|error| error.to_string())?; + if serde_json::to_vec(&record).map_err(|error| error.to_string())? != bytes { + return Err("saved-output digest mismatch".to_owned()); + } + return Ok(id); + } + let mut entries = Vec::new(); + for (scanned, entry) in std::fs::read_dir(directory) + .map_err(|error| error.to_string())? + .take(129) + .enumerate() + { + let entry = entry.map_err(|error| error.to_string())?; + if scanned >= 128 { + return Err("saved-output directory exceeds its scan bound".to_owned()); + } + let name = entry.file_name(); + let Some(name) = name.to_str() else { + continue; + }; + if name.strip_suffix(".json").is_some_and(is_identity) { + let metadata = entry.metadata().map_err(|error| error.to_string())?; + entries.push((metadata.modified().ok(), entry.path())); + } + } + entries.sort(); + let remove = entries + .len() + .saturating_add(1) + .saturating_sub(MAX_SAVED_OUTPUTS); + for (_, path) in entries.into_iter().take(remove) { + std::fs::remove_file(path).map_err(|error| error.to_string())?; + } + let record: SavedOutput = serde_json::from_slice(bytes).map_err(|error| error.to_string())?; + write_json_atomic_new(&path, &record, false).map_err(|error| error.to_string())?; + Ok(id) +} + +pub(crate) fn finish(mut outcome: Outcome, controls: &Controls) -> Outcome { + let Some(budget) = controls.budget else { + return outcome; + }; + let bytes = outcome.stdout.len() + + outcome.stderr.len() + + usize::from(!outcome.stdout.is_empty() && outcome.stdout_trailing_newline) + + usize::from(!outcome.stderr.is_empty() && outcome.stderr_trailing_newline); + if bytes <= budget.saturating_mul(4) { + return outcome; + } + if bytes as u64 > MAX_OUTPUT_BYTES { + return bounded_error( + "output exceeds 4 MiB; narrow the query or write it to a file", + budget, + ); + } + let text = if outcome.stdout.is_empty() { + outcome.stderr.clone() + } else if outcome.stderr.is_empty() { + outcome.stdout.clone() + } else { + format!("{}\nSTDERR:\n{}", outcome.stdout, outcome.stderr) + }; + let record = SavedOutput { + schema: SCHEMA.to_owned(), + code: outcome.code, + text, + }; + let result = (|| { + let mut capture = Capture::default(); + serde_json::to_writer(&mut capture, &record).map_err(|_| "saved output exceeds 4 MiB")?; + let bytes = capture.bytes; + if bytes.len() as u64 > MAX_OUTPUT_BYTES { + return Err("output exceeds the 4 MiB saved-output bound".to_owned()); + } + let id = save(&directory()?, &bytes)?; + if controls.machine || serde_json::from_str::(&outcome.stdout).is_ok() { + return Ok((format!("error: budget exceeded; compass output {id}"), true)); + } + page(&record, &id, 0, budget).map(|text| (text, false)) + })(); + match result { + Ok((text, machine)) => { + if machine { + return bounded_error(text.strip_prefix("error: ").unwrap_or(&text), budget); + } + if outcome.code == 0 { + outcome.stdout = text; + outcome.stderr.clear(); + } else { + outcome.stderr = text; + outcome.stdout.clear(); + } + outcome + } + Err(error) => bounded_error(&error, budget), + } +} + +fn page(record: &SavedOutput, id: &str, start: usize, budget: usize) -> Result { + render_budgeted_text(&record.text, start, budget, |end| { + format!("More: compass output {id} --offset {end} --budget {budget}") + }) + .map(|page| page.text) + .map_err(|error| error.to_string()) +} + +pub(crate) fn command(args: &[String], budget: Option) -> Outcome { + let result = (|| { + let id = args + .first() + .filter(|value| is_identity(value)) + .ok_or("output requires the saved answer ID")?; + let offset = match &args[1..] { + [] => 0, + [flag, value] if flag == "--offset" => value + .parse::() + .map_err(|_| "--offset requires a byte offset")?, + _ => return Err("usage: compass output ID [--offset N] [--budget N]".to_owned()), + }; + let root = directory()?; + for ancestor in root.ancestors().take(3) { + if ancestor.is_symlink() { + return Err("saved-output directory is a symlink".to_owned()); + } + } + let path = root.join(format!("{id}.json")); + if path.is_symlink() { + return Err("saved-output record is a symlink".to_owned()); + } + let bytes = read_bytes_bounded(&path, MAX_OUTPUT_BYTES).map_err(|_| { + "saved output missing or unreadable; rerun the original command".to_owned() + })?; + let record: SavedOutput = + serde_json::from_slice(&bytes).map_err(|error| error.to_string())?; + if record.schema != SCHEMA { + return Err("unsupported saved-output schema".to_owned()); + } + let canonical = serde_json::to_vec(&record).map_err(|error| error.to_string())?; + if identity(&canonical) != *id { + return Err("saved-output digest mismatch".to_owned()); + } + let text = page(&record, id, offset, budget.unwrap_or(2_000))?; + Ok(Outcome { + code: record.code, + stdout: text, + stderr: String::new(), + stdout_trailing_newline: true, + stderr_trailing_newline: true, + html_output: None, + }) + })(); + result.unwrap_or_else(|error| bounded_error(&error, budget.unwrap_or(2_000))) +} + +/// Capture a completed initializer without allowing its output buffer to grow. +#[derive(Default)] +pub(crate) struct Capture { + pub bytes: Vec, + pub exceeded: bool, +} +impl std::io::Write for Capture { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + if self.bytes.len().saturating_add(bytes.len()) > MAX_OUTPUT_BYTES as usize { + self.exceeded = true; + return Err(std::io::Error::other("completed output exceeds 4 MiB")); + } + self.bytes.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn saved_records_are_immutable_bounded_and_detect_corruption() + -> Result<(), Box> { + let root = tempfile::tempdir()?; + let record = SavedOutput { + schema: SCHEMA.to_owned(), + code: 0, + text: "é東京🙂".repeat(200), + }; + let bytes = serde_json::to_vec(&record)?; + let id = save(root.path(), &bytes)?; + assert_eq!(save(root.path(), &bytes)?, id); + let mut start = 0; + while start < record.text.len() { + let rendered = render_budgeted_text(&record.text, start, 32, |end| { + format!("More: compass output {id} --offset {end} --budget 32") + })?; + assert!(rendered.text.len() < 128); + assert!(rendered.end > start); + start = rendered.end; + } + std::fs::write(root.path().join(format!("{id}.json")), b"{}")?; + assert!(save(root.path(), &bytes).is_err()); + assert!(!is_identity("../outside")); + Ok(()) + } + + #[test] + fn retention_and_machine_budget_controls_are_bounded() -> Result<(), Box> + { + let root = tempfile::tempdir()?; + for index in 0..MAX_SAVED_OUTPUTS + 3 { + save( + root.path(), + &serde_json::to_vec(&SavedOutput { + schema: SCHEMA.to_owned(), + code: 0, + text: index.to_string(), + })?, + )?; + } + assert_eq!(std::fs::read_dir(root.path())?.count(), MAX_SAVED_OUTPUTS); + let mut args = ["query", "json", "--format", "jsonl", "--budget=32"] + .map(str::to_owned) + .to_vec(); + let controls = take(&mut args)?; + assert!(controls.machine); + assert_eq!(controls.budget, Some(32)); + assert!(!machine_requested(&[ + "search".to_owned(), + "json".to_owned() + ])); + let error = bounded_error(&"x".repeat(1000), 32); + assert!(error.stderr.len() < 128); + Ok(()) + } + + #[cfg(unix)] + #[test] + fn redirected_cache_directories_and_records_are_rejected() + -> Result<(), Box> { + let root = tempfile::tempdir()?; + let outside = tempfile::tempdir()?; + let redirected = root.path().join("redirected"); + std::os::unix::fs::symlink(outside.path(), &redirected)?; + let bytes = serde_json::to_vec(&SavedOutput { + schema: SCHEMA.to_owned(), + code: 0, + text: "source".to_owned(), + })?; + assert!(save(&redirected, &bytes).is_err()); + std::os::unix::fs::symlink( + outside.path().join("missing"), + root.path().join(format!("{}.json", identity(&bytes))), + )?; + assert!(save(root.path(), &bytes).is_err()); + Ok(()) + } +} diff --git a/crates/compass-cli/tests/code_query_cli.rs b/crates/compass-cli/tests/code_query_cli.rs index 6eddc8e3..f3e7ab4d 100644 --- a/crates/compass-cli/tests/code_query_cli.rs +++ b/crates/compass-cli/tests/code_query_cli.rs @@ -960,10 +960,6 @@ fn natural_discovery_rejects_invalid_duplicate_and_mixed_public_controls() vec!["--direction", "both", "--traverse"], "legacy traversal controls cannot be combined with discovery controls", ), - ( - vec!["--scope", "node:n:target", "--budget", "1000"], - "legacy traversal controls cannot be combined with discovery controls", - ), ( vec!["--format", "json", "--page", "2"], "legacy traversal controls cannot be combined with discovery controls", @@ -1723,6 +1719,30 @@ fn configured_typescript_path_alias_agrees_across_search_callers_and_path() fn typed_text_paging_continues_the_same_result_with_a_cursor() -> Result<(), Box> { let directory = tempfile::tempdir()?; let graph = support::write_typed_graph(directory.path())?; + // A compact one-hop answer now fits on one page. Use a real multi-entry + // answer to exercise continuation rather than spending budget on headers. + let mut document = GraphDocument::load(&graph)?; + let template = document.nodes[0].clone(); + let edge_template = document.links[0].clone(); + for index in 0..10 { + let mut caller = template.clone(); + caller.id = format!("n:additional-caller-{index}"); + caller.name = format!("AdditionalCaller{index}"); + caller.qualified_name = format!("Fixture.AdditionalCaller{index}"); + let mut edge = edge_template.clone(); + edge.source = caller.id.clone(); + edge.id = compass_model::identity::edge_id( + &edge.source, + edge.kind, + &edge.target, + edge.relationship_site.as_ref(), + None, + ); + edge.key = edge.id.clone(); + document.nodes.push(caller); + document.links.push(edge); + } + std::fs::write(&graph, serde_json::to_vec_pretty(&document)?)?; let graph_arg = graph.as_os_str().to_owned(); let first = run( Frontend::Compass, @@ -1762,7 +1782,10 @@ fn typed_text_paging_continues_the_same_result_with_a_cursor() -> Result<(), Box ], ); assert_eq!(second.code, 0, "{}", second.stderr); - assert!(second.stdout.contains("range=2-"), "{}", second.stdout); + assert!(second.stdout.contains("range="), "{}", second.stdout); + assert_ne!(first.stdout, second.stdout); + assert!(first.stdout.len() + 1 <= 480); + assert!(second.stdout.len() + 1 <= 480); let rejected = run( Frontend::Compass, @@ -2244,8 +2267,8 @@ fn natural_query_and_explain_accept_agent_controlled_budgets_and_pages() ], ); assert_eq!(first.code, 0, "{command}: {}", first.stderr); - assert!(first.stdout.contains("Pagination: page=1/")); - assert!(first.stdout.contains("next=2")); + assert!(first.stdout.len() + 1 <= 240); + assert!(first.stdout.contains("compass output")); let second = run( Frontend::Compass, @@ -2261,7 +2284,7 @@ fn natural_query_and_explain_accept_agent_controlled_budgets_and_pages() ], ); assert_eq!(second.code, 0, "{command}: {}", second.stderr); - assert!(second.stdout.contains("Pagination: page=2/")); + assert!(second.stdout.len() + 1 <= 240); assert_ne!(first.stdout, second.stdout); } @@ -2867,3 +2890,146 @@ fn concept_search_discloses_synonyms_and_keeps_exact_mode_strict() -> Result<(), assert!(rejected.stderr.contains("cannot be combined with --exact")); Ok(()) } + +#[test] +fn budget_is_shared_by_typed_discovery_and_report_commands() -> Result<(), Box> { + let directory = tempfile::tempdir()?; + let graph = support::write_typed_graph(directory.path())?; + for arguments in [ + vec!["callers", "Target"], + vec!["query", "who calls Target?"], + vec!["query", "Target", "--scope", "node:n:target"], + vec!["explain", "Target"], + vec!["path", "Caller", "Target"], + vec!["architecture"], + vec!["search", "AbsentQuantumSymbol"], + ] { + let output = support::compass_command() + .args(&arguments) + .args(["--graph", graph.to_str().ok_or("path")?, "--budget", "256"]) + .current_dir(directory.path()) + .output()?; + assert!( + output.status.success(), + "{arguments:?}: {}", + String::from_utf8_lossy(&output.stderr) + ); + assert!( + output.stdout.len() + output.stderr.len() <= 1024, + "{arguments:?}" + ); + } + let output = support::compass_command() + .args(["--help", "--budget", "64"]) + .current_dir(directory.path()) + .output()?; + assert!(output.status.success()); + assert!(output.stdout.len() <= 256); + let text = String::from_utf8(output.stdout)?; + let command = text + .lines() + .last() + .and_then(|line| line.strip_prefix("More: compass ")) + .ok_or("continuation")?; + let read = support::compass_command() + .args(command.split_whitespace()) + .current_dir(directory.path()) + .output()?; + assert!( + read.status.success(), + "{}", + String::from_utf8_lossy(&read.stderr) + ); + assert!(read.stdout.len() <= 256); + assert_ne!(String::from_utf8(read.stdout)?, text); + Ok(()) +} + +#[test] +fn compact_defaults_and_explicit_evidence_share_the_same_native_answer() +-> Result<(), Box> { + let directory = tempfile::tempdir()?; + let graph = support::write_typed_graph(directory.path())?; + let base = [ + OsString::from("callers"), + OsString::from("Target"), + OsString::from("--graph"), + graph.into_os_string(), + ]; + let compact = run(Frontend::Compass, base.clone()); + assert_eq!(compact.code, 0, "{}", compact.stderr); + assert!(compact.stdout.len() < 400, "{}", compact.stdout); + assert!( + compact + .stdout + .contains("Fixture.Caller --calls--> Fixture.Target") + ); + assert!(!compact.stdout.contains("graphIdentity")); + for flag in ["--verbose", "--evidence"] { + let mut args = base.to_vec(); + args.push(flag.into()); + let full = run(Frontend::Compass, args); + assert_eq!(full.code, 0, "{}", full.stderr); + assert!( + full.stdout + .contains("Fixture.Caller --calls--> Fixture.Target") + ); + assert!(full.stdout.contains("graphIdentity")); + assert!(full.stdout.contains("Node evidence:")); + } + Ok(()) +} + +#[test] +fn budgeted_machine_answers_fail_intact_and_saved_records_are_readable() +-> Result<(), Box> { + let root = tempfile::tempdir()?; + let graph = support::write_typed_graph(root.path())?; + let result = support::compass_command() + .args([ + "callers", + "Target", + "--format", + "json", + "--budget", + "32", + "--graph", + graph.to_str().ok_or("path")?, + ]) + .current_dir(root.path()) + .output()?; + assert!(!result.status.success()); + assert!(result.stdout.is_empty()); + assert!(result.stderr.len() <= 128); + let error = String::from_utf8(result.stderr)?; + let id = error + .split("compass output ") + .nth(1) + .ok_or("saved ID")? + .trim(); + let read = support::compass_command() + .args(["output", id, "--budget", "65536"]) + .current_dir(root.path()) + .output()?; + assert!(read.status.success()); + let response: Value = serde_json::from_slice(&read.stdout)?; + assert_eq!(response["operation"], "callers"); + std::fs::write( + root.path().join(format!(".compass/cache/output/{id}.json")), + b"{}", + )?; + let corrupt = support::compass_command() + .args(["output", id, "--budget", "32"]) + .current_dir(root.path()) + .output()?; + assert!(!corrupt.status.success()); + assert!(corrupt.stderr.len() <= 128); + let interactive = support::compass_command() + .args(["init", "--budget", "32"]) + .current_dir(root.path()) + .output()?; + assert!(!interactive.status.success()); + assert!(interactive.stderr.len() <= 128); + assert!(!root.path().join(".compass/project.json").exists()); + Ok(()) +} diff --git a/crates/compass-cli/tests/coverage_paths.rs b/crates/compass-cli/tests/coverage_paths.rs index d35fe2d9..f2bd2aea 100644 --- a/crates/compass-cli/tests/coverage_paths.rs +++ b/crates/compass-cli/tests/coverage_paths.rs @@ -640,6 +640,7 @@ fn completed_read_query_diagnostic_merge_tree_and_export_commands_run_end_to_end vec![ "query".to_owned(), "attention".to_owned(), + "--traverse".to_owned(), "--budget=100".to_owned(), format!("--graph={graph}"), ], @@ -814,6 +815,7 @@ fn split_value_read_export_and_cluster_forms_complete_against_a_real_graph() vec![ "query".to_owned(), "attention".to_owned(), + "--traverse".to_owned(), "--budget".to_owned(), "80".to_owned(), "--graph".to_owned(), diff --git a/crates/compass-cli/tests/init_cli.rs b/crates/compass-cli/tests/init_cli.rs index 12baae25..3509186d 100644 --- a/crates/compass-cli/tests/init_cli.rs +++ b/crates/compass-cli/tests/init_cli.rs @@ -400,3 +400,53 @@ fn jsonl_init_reports_each_indexed_file_against_the_total() -> Result<(), Box Result<(), Box> { + let root = tempfile::tempdir()?; + fs::write( + root.path().join("lib.rs"), + "pub fn native_budget_fixture() {}\n", + )?; + let output = Command::new(env!("CARGO_BIN_EXE_compass")) + .args([ + "init", + ".", + "--yes", + "--inference-level", + "low", + "--budget", + "64", + ]) + .current_dir(root.path()) + .env_remove("COMPASS_OUT") + .output()?; + assert!( + output.status.success(), + "{}", + String::from_utf8_lossy(&output.stderr) + ); + assert!(output.stdout.len() + output.stderr.len() <= 256); + assert!(root.path().join(".compass/config.toml").is_file()); + let graph = BuildGuard::resolve_artifact(&root.path().join("compass-out"), "graph.json")?; + let before = fs::read(&graph)?; + let text = String::from_utf8(output.stdout)?; + let continuation = text + .lines() + .last() + .and_then(|line| line.strip_prefix("More: compass ")) + .ok_or("saved continuation")?; + let read = Command::new(env!("CARGO_BIN_EXE_compass")) + .args(continuation.split_whitespace()) + .current_dir(root.path()) + .output()?; + assert!( + read.status.success(), + "{}", + String::from_utf8_lossy(&read.stderr) + ); + assert!(read.stdout.len() <= 256); + assert_eq!(fs::read(&graph)?, before); + Ok(()) +} diff --git a/crates/compass-mcp/src/lib.rs b/crates/compass-mcp/src/lib.rs index d21f3d43..d8593ba1 100644 --- a/crates/compass-mcp/src/lib.rs +++ b/crates/compass-mcp/src/lib.rs @@ -35,7 +35,7 @@ use compass_model::{Graph, GraphArtifact, GraphDocument, GraphError, NodeIndex}; use compass_output::{ AgentOperandRole, AgentOperation, AgentOrientation, AgentQueryContext, ORIENTATION_JSON_MAX_BYTES, build_code_query_view, build_discovery_query_view, - render_agent_query_text, render_agent_report_markdown, render_orientation_json, + render_agent_report_markdown, render_compact_agent_query_text, render_orientation_json, validate_orientation_graph_identity, }; use compass_prs::{ @@ -1076,7 +1076,7 @@ fn invoke_typed_tool( let context = typed_agent_query_context(name, arguments, &response, &engine); let view = build_code_query_view(&response, context) .map_err(|error| InvocationError::Internal(error.to_string()))?; - let text = render_agent_query_text(&view) + let text = render_compact_agent_query_text(&view) .map_err(|error| InvocationError::Internal(error.to_string()))?; let semantic_result_digest = compass_query::code_query_response_digest(&response) .map_err(|error| InvocationError::Internal(error.to_string()))?; @@ -1192,7 +1192,7 @@ fn invoke_discovery_tool( .with_operand(AgentOperandRole::Query, question.to_owned()); let view = build_discovery_query_view(&response, context) .map_err(|error| InvocationError::Internal(error.to_string()))?; - let text = render_agent_query_text(&view) + let text = render_compact_agent_query_text(&view) .map_err(|error| InvocationError::Internal(error.to_string()))?; if let Some(question) = arguments.get("question").and_then(Value::as_str) { log_discovery_mcp_query(question, graph_path, &response, started.elapsed()); diff --git a/crates/compass-mcp/tests/code_query_tools.rs b/crates/compass-mcp/tests/code_query_tools.rs index d90320bc..7db566ac 100644 --- a/crates/compass-mcp/tests/code_query_tools.rs +++ b/crates/compass-mcp/tests/code_query_tools.rs @@ -665,7 +665,12 @@ async fn mcp_code_queries_publish_structured_content_and_protocol_errors() assert_eq!(structured["result"]["schema"], "compass.query/1"); let agent_view: compass_output::AgentQueryView = serde_json::from_value(structured["agentView"].clone())?; - assert_eq!(text, compass_output::render_agent_query_text(&agent_view)?); + assert_eq!( + text, + compass_output::render_compact_agent_query_text(&agent_view)? + ); + assert!(text.len() < 400, "{text}"); + assert!(!text.contains("NEXT ACTIONS")); assert!( client .call_tool(CallToolRequestParams::new("search_symbols")) diff --git a/crates/compass-output/src/agent_query.rs b/crates/compass-output/src/agent_query.rs index babcb62d..26521de0 100644 --- a/crates/compass-output/src/agent_query.rs +++ b/crates/compass-output/src/agent_query.rs @@ -1,6 +1,6 @@ //! Deterministic, bounded projections for coding-agent query consumers. -use std::collections::{BTreeMap, HashMap, HashSet}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::fmt::Write as _; use compass_model::code_graph::NodeRole; @@ -1022,6 +1022,130 @@ fn validate_endpoint(endpoint: &AgentEndpoint) -> Result<(), OutputError> { Ok(()) } +/// Confidence and coverage remain visible even when audit detail is hidden. +pub fn render_compact_query_header_lines( + view: &AgentQueryView, +) -> Result, OutputError> { + view.validate()?; + let mut lines = vec![format!( + "RESULT {} · match={} · evidence={} · coverage={}{}", + result_state_name(view.status.result_state), + match_state_name(view.status.match_state), + evidence_state_name(view.status.evidence_state), + coverage_state_name(view.status.coverage), + if view.status.source_execution == AgentExecution::Partial { + "; partial" + } else { + "" + }, + )]; + lines.push("ANSWER".to_owned()); + if view.status.result_state != AgentResultState::Answered + || view.status.match_state != AgentMatch::Exact + || matches!( + view.request.operation, + AgentOperation::Callees | AgentOperation::Impact + ) + { + if view.status.result_state == AgentResultState::NoMatch && view.primary_results.is_empty() + { + lines.push("No match found.".to_owned()); + } else { + lines.push(escape_scalar(&view.answer.headline)); + } + } + let mut seen = BTreeSet::new(); + for caveat in &view.caveats { + // These states are already explicit above. Other caveats can change + // the meaning of a result and must survive the compact projection. + if matches!( + caveat.code.as_str(), + "coverage_unknown" | "evidence_hidden" | "projection_truncation" + ) { + continue; + } + if caveat.code == "ambiguous_match" && caveat.node_id.is_none() { + continue; + } + if caveat.code == "no_match" + && view.status.result_state == AgentResultState::NoMatch + && view.request.operation != AgentOperation::NodeTrail + { + continue; + } + let line = format!( + "{}: {}", + escape_scalar(&caveat.code), + escape_scalar(&caveat_statement(caveat)) + ); + if seen.insert(line.clone()) { + lines.push(line); + } + } + Ok(lines) +} + +fn compact_entries( + entries: &mut Vec<(TextPageSection, String)>, + response: &CodeQueryResponse, + view: &AgentQueryView, +) { + let has_paths = !response.paths.is_empty(); + let has_edges = !response.edges.is_empty(); + let mut primary_count = 0; + entries.retain(|(section, _)| match section { + TextPageSection::PrimaryResults => { + primary_count += 1; + view.request.operation == AgentOperation::Search + || view.request.operation == AgentOperation::Impact + || view.request.operation == AgentOperation::NodeTrail + || view.request.operation == AgentOperation::Explore + || !has_edges + || primary_count == 1 + || view.status.result_state == AgentResultState::NeedsResolution + } + TextPageSection::Paths => view.request.operation == AgentOperation::NodeTrail, + TextPageSection::Relationships => { + view.request.operation != AgentOperation::Impact + && !(view.request.operation == AgentOperation::NodeTrail && has_paths) + } + TextPageSection::Source => true, + TextPageSection::Evidence => false, + }); +} + +fn compact_page_body( + header: &[String], + entries: &[(TextPageSection, String)], + start: usize, + end: usize, + next_cursor: Option<&str>, + truncated: bool, +) -> String { + let mut lines = header.to_vec(); + lines.extend(entries[start..end].iter().map(|(_, text)| text.clone())); + if let Some(cursor) = next_cursor { + lines.push(format!( + "Pagination: range={}-{} of {}; continue with --cursor; next={cursor}", + start + 1, + end, + entries.len() + )); + } + if next_cursor.is_none() && start > 0 { + lines.push(format!( + "Pagination: range={}-{} of {} next=none", + start + 1, + end, + entries.len() + )); + } + if truncated { + lines.push("Query bound reached; repeat with larger --max-nodes/--max-edges.".to_owned()); + } + lines.join("\n") +} + pub fn render_agent_query_header_lines(view: &AgentQueryView) -> Result, OutputError> { view.validate()?; let mut lines = render_result_lines(view); @@ -1105,6 +1229,58 @@ pub fn render_agent_query_text(view: &AgentQueryView) -> Result Result { + let mut lines = render_compact_query_header_lines(view)?; + let duplicates = duplicated_entity_labels(&view.primary_results); + for (index, entity) in view.primary_results.iter().enumerate() { + if index == 0 + || view.relationships.is_empty() + || matches!( + view.request.operation, + AgentOperation::Search + | AgentOperation::Impact + | AgentOperation::NodeTrail + | AgentOperation::Explore + ) + { + lines.push(render_entity( + entity, + entity_needs_identity(entity, view.status.match_state, &duplicates), + )); + } + } + lines.extend(view.paths.iter().map(render_path)); + if view.request.operation != AgentOperation::NodeTrail || view.paths.is_empty() { + lines.extend(view.relationships.iter().map(render_relationship)); + } + if view.projection_truncated { + lines.push(format!( + "{} records omitted from text; full records: structuredContent.result.", + view.omissions.total + )); + } + let mut text = lines.join("\n"); + if matches!( + view.status.result_state, + AgentResultState::NoMatch + | AgentResultState::NeedsResolution + | AgentResultState::Candidates + ) { + text = crate::render_budgeted_text(&text, 0, 199, |_| { + "More: structuredContent.result and agentView.".to_owned() + })? + .text; + } + if text.len() > AGENT_VIEW_TEXT_MAX_BYTES { + return Err(OutputError::AgentQueryTextBudgetExceeded { + rendered_bytes: text.len(), + limit: AGENT_VIEW_TEXT_MAX_BYTES, + }); + } + Ok(text) +} + /// The compact agent projection. /// /// The brief view keeps the reviewed answer semantics of @@ -1255,7 +1431,6 @@ pub const AGENT_TEXT_PAGE_VERSION: &str = "compass.query.agent-text-page/1"; /// Default page budget, in approximate tokens, for paged agent text output. pub const DEFAULT_AGENT_TEXT_PAGE_TOKENS: usize = 2_000; -const AGENT_TEXT_PAGE_FOOTER_RESERVE_CHARS: usize = 384; const AGENT_TEXT_PAGE_MAX_CURSOR_BYTES: usize = 4_096; /// Continuation state for one paged agent text response. @@ -1388,6 +1563,7 @@ enum TextPageSection { Paths, Relationships, Source, + Evidence, } impl TextPageSection { @@ -1397,6 +1573,7 @@ impl TextPageSection { Self::Paths => "PATHS", Self::Relationships => "RELATIONSHIPS", Self::Source => "SOURCE", + Self::Evidence => "EVIDENCE", } } @@ -1413,6 +1590,7 @@ impl TextPageSection { Self::Relationships => AGENT_VIEW_MAX_RELATIONSHIPS, // Source blocks are already bounded per anchor and by the budget. Self::Source => usize::MAX, + Self::Evidence => AGENT_VIEW_MAX_RELATIONSHIPS, } } } @@ -1435,13 +1613,41 @@ pub fn decode_agent_text_page_cursor(value: &str) -> Result, +) -> Result { + render_code_query_page(response, context, options, false, false) +} + +/// Answer-sized text projection; audit projections and raw records are unchanged. +pub fn render_compact_code_query_text_page( + response: &CodeQueryResponse, + context: AgentQueryContext, + options: AgentTextPageOptions<'_>, +) -> Result { + render_code_query_page(response, context, options, true, false) +} + +/// Full status and per-record provenance, explicitly requested by the caller. +pub fn render_code_query_evidence_page( + response: &CodeQueryResponse, + context: AgentQueryContext, + options: AgentTextPageOptions<'_>, +) -> Result { + render_code_query_page(response, context, options, false, true) +} + +fn render_code_query_page( + response: &CodeQueryResponse, + context: AgentQueryContext, + options: AgentTextPageOptions<'_>, + compact: bool, + evidence: bool, ) -> Result { if options.token_budget == 0 { return Err(OutputError::InvalidAgentTextPage( @@ -1450,12 +1656,34 @@ pub fn render_code_query_text_page( } let view = build_code_query_view(response, context)?; view.validate()?; - let entries = text_page_entries( + let mut entries = text_page_entries( response, view.request.operation, &view.request.operands, view.status.match_state, ); + if compact { + compact_entries(&mut entries, response, &view); + } else if evidence { + for node in &response.nodes { + entries.push(( + TextPageSection::Evidence, + format!("Node evidence: {}", serde_json::to_string(node)?), + )); + } + for edge in &response.edges { + entries.push(( + TextPageSection::Evidence, + format!("Edge evidence: {}", serde_json::to_string(edge)?), + )); + } + for diagnostic in &response.diagnostics { + entries.push(( + TextPageSection::Evidence, + format!("Diagnostic: {}", serde_json::to_string(diagnostic)?), + )); + } + } let (page, start) = match options.cursor { None => (1_u32, 0_usize), Some(cursor) => { @@ -1491,11 +1719,33 @@ pub fn render_code_query_text_page( (envelope.page, envelope.prefix_count) } }; - let max_chars = options.token_budget.saturating_mul(4); - let header = render_agent_query_header_lines(&view)?; + let budget = if compact + && matches!( + view.status.result_state, + AgentResultState::NoMatch + | AgentResultState::NeedsResolution + | AgentResultState::Candidates + ) { + options.token_budget.min(199) + } else { + options.token_budget + }; + let max_chars = budget.saturating_mul(4).saturating_sub(1); + let mut header = if compact { + render_compact_query_header_lines(&view)? + } else { + render_agent_query_header_lines(&view)? + }; + if evidence { + header.push(format!("Status: {}", serde_json::to_string(&view.status)?)); + header.push(format!( + "Identity: {}", + serde_json::to_string(&view.identity)? + )); + } let header_chars = rendered_chars(&header); let mut end = start; - let mut used = header_chars.saturating_add(AGENT_TEXT_PAGE_FOOTER_RESERVE_CHARS); + let mut used = header_chars; let mut rendered_per_section: BTreeMap = BTreeMap::new(); while end < entries.len() { let (section, text) = &entries[end]; @@ -1537,7 +1787,19 @@ pub fn render_code_query_text_page( page, next_cursor.as_deref(), ); - if text.chars().count() <= max_chars || end <= start.saturating_add(1) { + let text = if compact { + compact_page_body( + &header, + &entries, + start, + end, + next_cursor.as_deref(), + response.truncated, + ) + } else { + text + }; + if text.len() <= max_chars { return Ok(AgentTextPage { text, next_cursor, @@ -1547,6 +1809,12 @@ pub fn render_code_query_text_page( entry_total: entries.len(), }); } + if end <= start.saturating_add(1) { + let required = text.len().saturating_add(4) / 4; + return Err(OutputError::InvalidAgentTextPage(format!( + "one answer entry requires --text-budget {required}; increase the budget" + ))); + } end = end.saturating_sub(1); } } @@ -1584,18 +1852,8 @@ fn render_agent_text_page_body( entries.len(), next_cursor.unwrap_or("none") )); - if view.omissions.total > 0 || response.truncated { - lines.push(if response.truncated { - format!( - "Bound: {} record(s) beyond this page; next= continues, --max-nodes/--max-edges widens the query bound.", - view.omissions.total - ) - } else { - format!( - "Bound: {} record(s) beyond this page; next= continues.", - view.omissions.total - ) - }); + if response.truncated { + lines.push("Query bound reached; repeat with larger --max-nodes/--max-edges.".to_owned()); } lines.join("\n") } @@ -2828,7 +3086,7 @@ fn answer_for_code( } AgentResultState::Candidates => format!( "Found {} candidate matches for \"{requested}\".", - primary_results.len() + response.results.len().max(primary_results.len()) ), _ if match_state == AgentMatch::Exact => { format!("Found an exact match for \"{requested}\".") diff --git a/crates/compass-output/src/lib.rs b/crates/compass-output/src/lib.rs index 4e80afc5..4c857ec0 100644 --- a/crates/compass-output/src/lib.rs +++ b/crates/compass-output/src/lib.rs @@ -19,11 +19,14 @@ mod palette; mod report; mod review; mod svg; +mod text_budget; mod tree; mod viewer_model; mod wiki; mod workbench; +pub use text_budget::{BudgetedText, render_budgeted_text}; + pub use agent_query::{ AGENT_BRIEF_VIEW_SCHEMA, AGENT_QUERY_VIEW_SCHEMA, AGENT_TEXT_PAGE_VERSION, AGENT_VIEW_MAX_BYTES, AGENT_VIEW_MAX_CAVEATS, AGENT_VIEW_MAX_NEXT_ACTIONS, @@ -39,7 +42,9 @@ pub use agent_query::{ DEFAULT_AGENT_TEXT_PAGE_TOKENS, build_code_query_brief, build_code_query_view, build_discovery_query_view, decode_agent_text_page_cursor, render_agent_query_continuation_header, render_agent_query_header_lines, - render_agent_query_text, render_code_query_text_page, + render_agent_query_text, render_code_query_evidence_page, render_code_query_text_page, + render_compact_agent_query_text, render_compact_code_query_text_page, + render_compact_query_header_lines, }; pub use architecture_projection::{ ARCHITECTURE_OVERLAY_SCHEMA, ARCHITECTURE_SUMMARY_KIND_COUNT_POLICY, diff --git a/crates/compass-output/src/text_budget.rs b/crates/compass-output/src/text_budget.rs new file mode 100644 index 00000000..bce1b487 --- /dev/null +++ b/crates/compass-output/src/text_budget.rs @@ -0,0 +1,79 @@ +//! UTF-8 byte budgets for completed human-readable command output. + +use crate::OutputError; + +#[derive(Debug, Eq, PartialEq)] +pub struct BudgetedText { + pub text: String, + pub end: usize, +} + +/// Keep whole UTF-8 characters, including the continuation instruction and +/// the newline written by the CLI. Offsets are bytes in the immutable answer. +pub fn render_budgeted_text( + text: &str, + start: usize, + tokens: usize, + continuation: impl Fn(usize) -> String, +) -> Result { + if start > text.len() || !text.is_char_boundary(start) { + return Err(OutputError::InvalidAgentTextPage( + "invalid output offset".to_owned(), + )); + } + let limit = tokens.saturating_mul(4).saturating_sub(1); + if text.len().saturating_sub(start) <= limit { + return Ok(BudgetedText { + text: text[start..].to_owned(), + end: text.len(), + }); + } + let mut end = start.saturating_add(limit).min(text.len()); + loop { + while end > start && !text.is_char_boundary(end) { + end -= 1; + } + let footer = continuation(end); + if end + .saturating_sub(start) + .saturating_add(footer.len()) + .saturating_add(1) + <= limit + { + return Ok(BudgetedText { + text: format!("{}\n{footer}", &text[start..end]), + end, + }); + } + if end == start { + return Err(OutputError::InvalidAgentTextPage( + "budget too small for continuation; use --budget 256".to_owned(), + )); + } + end -= 1; + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn unicode_pages_respect_bytes_and_reconstruct_the_answer() + -> Result<(), Box> { + let input = "é東京🙂\n".repeat(100); + let mut restored = String::new(); + let mut start = 0; + while start < input.len() { + let page = + render_budgeted_text(&input, start, 64, |end| format!("More: --offset {end}"))?; + assert!(page.text.len() + 1 <= 256); + assert!(page.end > start); + restored.push_str(&input[start..page.end]); + start = page.end; + } + assert_eq!(restored, input); + assert!(render_budgeted_text("é", 1, 64, |_| String::new()).is_err()); + Ok(()) + } +} diff --git a/crates/compass-output/tests/agent_query.rs b/crates/compass-output/tests/agent_query.rs index 05842975..2cc34870 100644 --- a/crates/compass-output/tests/agent_query.rs +++ b/crates/compass-output/tests/agent_query.rs @@ -1120,3 +1120,143 @@ fn unresolved_call_inventory_never_becomes_a_zero_call_claim() -> Result<(), Box assert!(view.answer.headline.contains("coverage is unavailable")); Ok(()) } + +#[test] +fn compact_one_hop_and_no_match_answers_cost_only_the_facts() -> Result<(), Box> { + let caller_anchor = anchor("caller.rs", 1); + let target_anchor = anchor("target.rs", 2); + let mut response = response(CodeQueryOperation::Callers); + response.nodes = vec![ + node("a", "Caller", &caller_anchor), + node("b", "Target", &target_anchor), + ]; + response.edges.push(QueryEdge { + id: "edge".to_owned(), + source: "a".to_owned(), + target: "b".to_owned(), + kind: EdgeKind::Calls, + relationship_site: Some(caller_anchor.clone()), + details: None, + evidence: vec![evidence(&caller_anchor)], + }); + let context = context(AgentOperation::Callers) + .with_operand(compass_output::AgentOperandRole::Symbol, "Target"); + let options = AgentTextPageOptions { + token_budget: 128, + cursor: None, + }; + let compact = + compass_output::render_compact_code_query_text_page(&response, context.clone(), options)?; + assert!(compact.text.len() + 1 <= 400, "{}", compact.text); + assert!( + compact.text.contains("Caller --calls--> Fixture.Target"), + "{}", + compact.text + ); + assert!(compact.text.contains("target.rs")); + assert!(compact.text.contains("caller.rs")); + assert!(compact.text.contains("coverage=unknown")); + assert!(compact.next_cursor.is_none()); + response.nodes.clear(); + response.edges.clear(); + response.diagnostics.push(QueryDiagnostic { + code: QueryDiagnosticCode::NoMatch, + message: "No matching symbol".to_owned(), + node_id: None, + path: None, + }); + let empty = compass_output::render_compact_code_query_text_page(&response, context, options)?; + assert!(empty.text.len() + 1 < 800); + assert!(empty.text.contains("No match")); + assert!(empty.next_cursor.is_none()); + Ok(()) +} + +#[test] +fn compact_unicode_pages_never_overflow_the_budget_or_skip_entries() -> Result<(), Box> { + let mut response = response(CodeQueryOperation::Search); + for index in 0..30 { + let id = format!("n:{index:02}"); + response.nodes.push(node( + &id, + &format!("東京🙂{index}"), + &anchor("src/東京.rs", index + 1), + )); + response.results.push(SearchHit { + node_id: id, + score: 1.0, + matched_fields: vec!["name".to_owned()], + }); + } + let context = context(AgentOperation::Search) + .with_operand(compass_output::AgentOperandRole::Query, "東京"); + let mut cursor = None; + let mut count = 0; + for _ in 0..30 { + let page = compass_output::render_compact_code_query_text_page( + &response, + context.clone(), + AgentTextPageOptions { + token_budget: 256, + cursor: cursor.as_deref(), + }, + )?; + assert!(page.text.len() + 1 <= 1024); + count += page.entry_end - page.entry_start; + cursor = page.next_cursor; + if cursor.is_none() { + break; + } + } + assert_eq!(count, 30); + assert!(cursor.is_none()); + assert!( + compass_output::render_compact_code_query_text_page( + &response, + context, + AgentTextPageOptions { + token_budget: 1, + cursor: None + } + ) + .is_err() + ); + Ok(()) +} + +#[test] +fn compact_exploration_retains_an_isolated_requested_symbol() -> Result<(), Box> { + let location = anchor("map.rs", 1); + let mut response = response(CodeQueryOperation::Explore); + response.nodes = vec![ + node("a", "Caller", &location), + node("b", "Target", &location), + node("c", "Isolated", &location), + ]; + response.edges.push(QueryEdge { + id: "edge".to_owned(), + source: "a".to_owned(), + target: "b".to_owned(), + kind: EdgeKind::Calls, + relationship_site: Some(location.clone()), + details: None, + evidence: vec![evidence(&location)], + }); + let context = context(AgentOperation::Explore) + .with_operand(compass_output::AgentOperandRole::Symbol, "Caller") + .with_operand(compass_output::AgentOperandRole::Symbol, "Isolated"); + let page = compass_output::render_compact_code_query_text_page( + &response, + context.clone(), + AgentTextPageOptions { + token_budget: 2000, + cursor: None, + }, + )?; + assert!(page.text.contains("Fixture.Isolated"), "{}", page.text); + assert!(page.text.contains("Caller --calls--> Fixture.Target")); + let view = build_code_query_view(&response, context)?; + let text = compass_output::render_compact_agent_query_text(&view)?; + assert!(text.contains("Fixture.Isolated"), "{text}"); + Ok(()) +} diff --git a/crates/compass-query/src/code_query.rs b/crates/compass-query/src/code_query.rs index c7569815..79afbd33 100644 --- a/crates/compass-query/src/code_query.rs +++ b/crates/compass-query/src/code_query.rs @@ -359,6 +359,7 @@ pub struct CodeQueryEngine { pub(crate) deadline: Option, pub(crate) semantic_search: bool, pub(crate) semantic_index: Mutex>, + pub(crate) response_cache: crate::response_cache::ResponseCache, } #[derive(Clone, Debug, Eq, PartialEq)] @@ -1650,7 +1651,12 @@ impl CodeQueryEngine { pub fn search(&self, request: SearchRequest) -> Result { self.check_deadline()?; - self.search_instrumented(request, &mut QueryInstrumentation::default()) + self.cached_response( + "search", + &request, + || self.search_instrumented(request.clone(), &mut QueryInstrumentation::default()), + |response| !response.truncated, + ) } /// Return every bounded exact ID/name candidate satisfying explicit filters. @@ -2855,12 +2861,34 @@ impl CodeQueryEngine { pub fn callers(&self, request: CallRequest) -> Result { self.check_deadline()?; - self.call_neighbors_instrumented(request, true, &mut QueryInstrumentation::default()) + self.cached_response( + "callers", + &request, + || { + self.call_neighbors_instrumented( + request.clone(), + true, + &mut QueryInstrumentation::default(), + ) + }, + |response| !response.truncated, + ) } pub fn callees(&self, request: CallRequest) -> Result { self.check_deadline()?; - self.call_neighbors_instrumented(request, false, &mut QueryInstrumentation::default()) + self.cached_response( + "callees", + &request, + || { + self.call_neighbors_instrumented( + request.clone(), + false, + &mut QueryInstrumentation::default(), + ) + }, + |response| !response.truncated, + ) } pub(crate) fn call_neighbors_instrumented( @@ -3095,7 +3123,12 @@ impl CodeQueryEngine { pub fn impact(&self, request: ImpactRequest) -> Result { self.check_deadline()?; - self.impact_instrumented(request, &mut QueryInstrumentation::default()) + self.cached_response( + "impact", + &request, + || self.impact_instrumented(request.clone(), &mut QueryInstrumentation::default()), + |response| !response.truncated, + ) } /// Restrict dependent traversal to an explicit relationship family. Direct @@ -3115,7 +3148,18 @@ impl CodeQueryEngine { let mut kinds = relations.to_vec(); kinds.sort_by_key(|kind| kind.as_str()); kinds.dedup(); - self.impact_instrumented_with_kinds(request, &kinds, &mut QueryInstrumentation::default()) + self.cached_response( + "impact-relations", + &(&request, &kinds), + || { + self.impact_instrumented_with_kinds( + request.clone(), + &kinds, + &mut QueryInstrumentation::default(), + ) + }, + |response| !response.truncated, + ) } /// Resolve inbound dependency relationships for the compatibility @@ -3401,7 +3445,12 @@ impl CodeQueryEngine { pub fn node_trail(&self, request: NodeTrailRequest) -> Result { self.check_deadline()?; - self.node_trail_instrumented(request, &mut QueryInstrumentation::default()) + self.cached_response( + "node_trail", + &request, + || self.node_trail_instrumented(request.clone(), &mut QueryInstrumentation::default()), + |response| !response.truncated, + ) } pub(crate) fn node_trail_instrumented( diff --git a/crates/compass-query/src/discovery.rs b/crates/compass-query/src/discovery.rs index 23759cb0..382fa6bb 100644 --- a/crates/compass-query/src/discovery.rs +++ b/crates/compass-query/src/discovery.rs @@ -380,7 +380,12 @@ impl CodeQueryEngine { &self, request: DiscoveryQueryRequest, ) -> Result { - self.discover_with_cancellation(request, None) + self.cached_response( + "discovery", + &request, + || self.discover_with_cancellation(request.clone(), None), + |response| !response.truncated, + ) } /// Execute discovery while observing an optional caller-owned cancellation flag. @@ -2959,6 +2964,7 @@ mod tests { deadline: None, semantic_search: false, semantic_index: std::sync::Mutex::new(None), + response_cache: crate::response_cache::ResponseCache::disabled(), } } diff --git a/crates/compass-query/src/discovery_text.rs b/crates/compass-query/src/discovery_text.rs index 8fd80573..cbdbad8f 100644 --- a/crates/compass-query/src/discovery_text.rs +++ b/crates/compass-query/src/discovery_text.rs @@ -15,7 +15,7 @@ use crate::text_cursor::{ pub const DISCOVERY_TEXT_PAGE_VERSION: &str = "compass.query.discovery-text-page/3"; pub const DEFAULT_DISCOVERY_TEXT_TOKEN_BUDGET: usize = 800; const MAX_CURSOR_BYTES: usize = 4_096; -const MIN_TEXT_BUDGET: usize = 256; +const MIN_TEXT_BUDGET: usize = 32; const MAX_TEXT_BUDGET: usize = 65_536; const MAX_RENDERED_SCALAR_CHARS: usize = 512; const MAX_RENDERED_LIST_CHARS: usize = 2_048; @@ -262,11 +262,8 @@ fn render_discovery_text_page_internal( || default_fixed_lines(response, options.include_evidence, &semantic_result_digest), ToOwned::to_owned, ); - let max_chars = options.token_budget.saturating_mul(4); - let fixed_chars = fixed - .iter() - .map(|line| line.chars().count() + 1) - .sum::(); + let max_chars = options.token_budget.saturating_mul(4).saturating_sub(1); + let fixed_chars = fixed.iter().map(|line| line.len() + 1).sum::(); let mut end = start; let mut entries_chars = 0_usize; let mut truncated_entry: Option = None; @@ -283,11 +280,8 @@ fn render_discovery_text_page_internal( candidate_cursor.as_deref(), options.include_evidence, ); - let candidate_entry_chars = entry.text.chars().count().saturating_add(1); - let footer_chars = footer - .iter() - .map(|line| line.chars().count() + 1) - .sum::(); + let candidate_entry_chars = entry.text.len().saturating_add(1); + let footer_chars = footer.iter().map(|line| line.len() + 1).sum::(); if fixed_chars .saturating_add(entries_chars) .saturating_add(candidate_entry_chars) @@ -330,12 +324,8 @@ fn render_discovery_text_page_internal( options.include_evidence, ); if entries.is_empty() - && fixed_chars.saturating_add( - page_footer - .iter() - .map(|line| line.chars().count() + 1) - .sum::(), - ) > max_chars + && fixed_chars.saturating_add(page_footer.iter().map(|line| line.len() + 1).sum::()) + > max_chars { return Err(DiscoveryTextPageError::PageMetadataTooLarge); } @@ -364,9 +354,13 @@ fn truncated_entry_text(text: &str, remaining_chars: usize) -> Option { let budget = remaining_chars.saturating_sub(1); let marker = [ENTRY_TRUNCATION_MARKER, SHORT_ENTRY_TRUNCATION_MARKER] .into_iter() - .find(|marker| budget > marker.chars().count() + 1)?; - let keep = budget - marker.chars().count(); - let mut rendered = text.chars().take(keep).collect::(); + .find(|marker| budget > marker.len() + 1)?; + let keep = budget - marker.len(); + let mut end = keep.min(text.len()); + while end > 0 && !text.is_char_boundary(end) { + end -= 1; + } + let mut rendered = text[..end].to_owned(); rendered.push_str(marker); Some(rendered) } @@ -1216,7 +1210,7 @@ mod tests { DiscoveryTextPageOptions { // The minimum budget must fit visibly capped compact // diagnostics rather than failing on oversized source text. - token_budget: MIN_TEXT_BUDGET, + token_budget: 256, cursor: None, request_digest: &"a".repeat(64), graph_identity: "generation-1", @@ -1228,7 +1222,7 @@ mod tests { page.text.contains('…'), "oversized scalar must be visibly capped" ); - assert!(page.text.chars().count() <= MIN_TEXT_BUDGET * 4); + assert!(page.text.chars().count() <= 256 * 4); assert!(page.entry_end > page.entry_start); // Compact diagnostics fit on one page after their scalar cap; the // evidence renderer still exercises oversized-entry pagination below. @@ -1249,7 +1243,7 @@ mod tests { DiscoveryTextPageOptions { // Two entries cannot share one page at this budget, so the // first page always ends with a continuation cursor. - token_budget: MIN_TEXT_BUDGET, + token_budget: 256, cursor: None, request_digest: &"a".repeat(64), graph_identity: "generation-1", diff --git a/crates/compass-query/src/index.rs b/crates/compass-query/src/index.rs index 56a2159e..e993958d 100644 --- a/crates/compass-query/src/index.rs +++ b/crates/compass-query/src/index.rs @@ -266,10 +266,10 @@ pub fn open_with_engine( // a store reference is present: a corrupt or mismatched sidecar must fail // closed instead of silently querying a different realization. if selection == EngineSelection::Default && has_published_store(graph_path) { - return open_from_local_store(graph_path, program_path); + return open_from_local_store(graph_path, program_path, cache_root); } if selection == EngineSelection::Store { - return open_from_local_store(graph_path, program_path); + return open_from_local_store(graph_path, program_path, cache_root); } let graph_engine = open_graph_engine(graph_path, selection)?; open_from_graph_engine(graph_path, program_path, cache_root, graph_engine) @@ -374,6 +374,8 @@ fn open_from_graph_engine( diagnostic.message ) }); + let response_cache = + crate::response_cache::ResponseCache::open(cache_root, &graph_identity, program.as_ref()); Ok(CodeQueryEngine { backend: CodeGraphBackend::Materialized { graph: Box::new(graph), @@ -393,12 +395,14 @@ fn open_from_graph_engine( deadline: None, semantic_search: false, semantic_index: std::sync::Mutex::new(None), + response_cache, }) } fn open_from_local_store( graph_path: &Path, program_path: Option<&Path>, + cache_root: &Path, ) -> Result { let snapshot = open_local_store_snapshot(graph_path)?; let graph_identity = snapshot.graph_identity.clone(); @@ -406,6 +410,8 @@ fn open_from_local_store( let partial_graph_message = snapshot.partial_graph_message.clone(); let (program, _) = load_program(program_path)?; let index_path = snapshot.store_path.clone(); + let response_cache = + crate::response_cache::ResponseCache::open(cache_root, &graph_identity, program.as_ref()); Ok(CodeQueryEngine { backend: CodeGraphBackend::Store(Box::new(snapshot)), program, @@ -421,6 +427,7 @@ fn open_from_local_store( deadline: None, semantic_search: false, semantic_index: std::sync::Mutex::new(None), + response_cache, }) } diff --git a/crates/compass-query/src/intent.rs b/crates/compass-query/src/intent.rs index 6f4a18e7..181d44d0 100644 --- a/crates/compass-query/src/intent.rs +++ b/crates/compass-query/src/intent.rs @@ -15,7 +15,7 @@ const MAX_NATURAL_QUERY_BYTES: usize = 4_096; const AUTO_ROUTE_CONFIDENCE: u8 = 90; pub(crate) type NaturalSelections = Vec<(String, String)>; -#[derive(Clone, Debug, Eq, PartialEq)] +#[derive(Clone, Debug, Eq, PartialEq, Serialize)] pub struct NaturalQueryRequest { pub question: String, pub include_heuristic: bool, @@ -81,8 +81,15 @@ impl CodeQueryEngine { &self, request: NaturalQueryRequest, ) -> Result { - self.execute_natural_query(request) - .map(|(response, _, _)| response) + self.cached_response( + "natural", + &request, + || { + self.execute_natural_query(request.clone()) + .map(|(response, _, _)| response) + }, + |response| !response.truncated && response.files.is_empty(), + ) } pub fn query_natural_profiled( diff --git a/crates/compass-query/src/lib.rs b/crates/compass-query/src/lib.rs index c11a5f28..246f86f1 100644 --- a/crates/compass-query/src/lib.rs +++ b/crates/compass-query/src/lib.rs @@ -21,6 +21,7 @@ mod program_join; mod ranking; mod recall; mod relevance; +mod response_cache; mod score; mod semantic_search; mod source; diff --git a/crates/compass-query/src/response_cache.rs b/crates/compass-query/src/response_cache.rs new file mode 100644 index 00000000..98bcd574 --- /dev/null +++ b/crates/compass-query/src/response_cache.rs @@ -0,0 +1,372 @@ +//! Disposable, bounded response cache. Every engine pins a verified graph; +//! request keys also include Program IR, query profiles and semantic mode. +use crate::{CodeQueryEngine, QueryError}; +use rusqlite::{Connection, OptionalExtension, params}; +use serde::{Serialize, de::DeserializeOwned}; +use sha2::{Digest, Sha256}; +use std::path::Path; +use std::sync::Mutex; + +const MAX_PAYLOAD_BYTES: usize = 1024 * 1024; +const MAX_ENTRIES: usize = 32; + +pub(crate) struct ResponseCache { + connection: Mutex>, + identity: String, +} + +impl ResponseCache { + #[cfg(test)] + pub(crate) fn disabled() -> Self { + Self { + connection: Mutex::new(None), + identity: String::new(), + } + } + + pub(crate) fn open( + root: &Path, + graph: &str, + program: Option<&compass_ir::ProgramBundle>, + ) -> Self { + let mut digest = Sha256::new(); + digest.update(graph.as_bytes()); + digest.update(env!("CARGO_PKG_VERSION").as_bytes()); + digest.update(crate::QUERY_PLANNER_PROFILE_V2.as_bytes()); + digest.update(crate::QUERY_RANKER_PROFILE_V1.as_bytes()); + if let Some(program) = program { + // Hash through a writer, without constructing another IR-sized buffer. + if serde_json::to_writer(DigestWriter(&mut digest), program).is_err() { + return Self { + connection: Mutex::new(None), + identity: String::new(), + }; + } + } + let identity = format!("{:x}", digest.finalize()); + let connection = (|| { + std::fs::create_dir_all(root).ok()?; + let path = root.join("responses-v1.sqlite3"); + if std::fs::metadata(&path) + .is_ok_and(|metadata| !metadata.is_file() || metadata.len() > 64 * 1024 * 1024) + || path.is_symlink() + { + return None; + } + let connection = Connection::open(path).ok()?; + connection + .busy_timeout(std::time::Duration::from_millis(50)) + .ok()?; + let page_size = connection + .query_row("PRAGMA page_size", [], |row| row.get::<_, u64>(0)) + .ok()?; + if !(512..=65536).contains(&page_size) { + return None; + } + let max_pages = 64 * 1024 * 1024 / page_size; + connection + .pragma_update(None, "max_page_count", max_pages) + .ok()?; + connection + .execute_batch( + "PRAGMA journal_mode=DELETE; + CREATE TABLE IF NOT EXISTS responses_v1 ( + key TEXT PRIMARY KEY, payload BLOB NOT NULL, checksum TEXT NOT NULL, + accessed INTEGER NOT NULL);", + ) + .ok()?; + Some(connection) + })(); + Self { + connection: Mutex::new(connection), + identity, + } + } + fn key(&self, operation: &str, semantic: bool, request: &T) -> Option { + let mut digest = Sha256::new(); + digest.update(self.identity.as_bytes()); + digest.update(operation.as_bytes()); + digest.update([u8::from(semantic)]); + serde_json::to_writer(DigestWriter(&mut digest), request).ok()?; + Some(format!("{:x}", digest.finalize())) + } + fn get(&self, key: &str) -> Option { + let mut state = self + .connection + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let connection = state.as_mut()?; + let (bytes, checksum): (Vec, String) = connection + .query_row( + "SELECT payload, checksum FROM responses_v1 WHERE key=?1 AND length(payload)<=?2", + params![key, MAX_PAYLOAD_BYTES], + |row| Ok((row.get(0)?, row.get(1)?)), + ) + .optional() + .ok()??; + if checksum != payload_digest(key, &bytes) { + return None; + } + let result = serde_json::from_slice(&bytes).ok()?; + let _ = connection.execute("UPDATE responses_v1 SET accessed=(SELECT COALESCE(MAX(accessed),0)+1 FROM responses_v1) WHERE key=?1", [key]); + Some(result) + } + fn put(&self, key: &str, response: &R) { + let mut buffer = CacheBuffer(Vec::new()); + if serde_json::to_writer(&mut buffer, response).is_err() { + return; + } + let bytes = buffer.0; + let checksum = payload_digest(key, &bytes); + let mut state = self + .connection + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let Some(connection) = state.as_mut() else { + return; + }; + let Ok(transaction) = connection.transaction() else { + return; + }; + if transaction.execute("INSERT OR REPLACE INTO responses_v1 VALUES(?1,?2,?3,(SELECT COALESCE(MAX(accessed),0)+1 FROM responses_v1))", params![key, bytes, checksum]).is_err() { return; } + if transaction.execute("DELETE FROM responses_v1 WHERE key IN (SELECT key FROM responses_v1 ORDER BY accessed DESC,key LIMIT -1 OFFSET ?1)", [MAX_ENTRIES]).is_err() { return; } + let _ = transaction.commit(); + } +} +fn payload_digest(key: &str, bytes: &[u8]) -> String { + let mut digest = Sha256::new(); + digest.update(key.as_bytes()); + digest.update(bytes); + format!("{:x}", digest.finalize()) +} +struct CacheBuffer(Vec); +impl std::io::Write for CacheBuffer { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + if self.0.len().saturating_add(bytes.len()) > MAX_PAYLOAD_BYTES { + return Err(std::io::Error::other("cache payload bound")); + } + self.0.extend_from_slice(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } +} +struct DigestWriter<'a>(&'a mut Sha256); +impl std::io::Write for DigestWriter<'_> { + fn write(&mut self, bytes: &[u8]) -> std::io::Result { + self.0.update(bytes); + Ok(bytes.len()) + } + fn flush(&mut self) -> std::io::Result<()> { + Ok(()) + } +} +impl CodeQueryEngine { + pub(crate) fn cached_response( + &self, + operation: &str, + request: &T, + execute: impl FnOnce() -> Result, + cacheable: impl Fn(&R) -> bool, + ) -> Result { + self.check_deadline()?; + let key = self + .response_cache + .key(operation, self.semantic_search, request); + if let Some(key) = &key + && let Some(response) = self.response_cache.get::(key) + { + self.check_deadline()?; + if response.valid_cache_record() { + return Ok(response); + } + } + let response = execute()?; + self.check_deadline()?; + if cacheable(&response) + && response.valid_cache_record() + && let Some(key) = key + { + self.response_cache.put(&key, &response); + } + self.check_deadline()?; + Ok(response) + } +} + +pub(crate) trait CacheRecord { + fn valid_cache_record(&self) -> bool; +} +impl CacheRecord for compass_model::query_contract::CodeQueryResponse { + fn valid_cache_record(&self) -> bool { + self.schema == compass_model::query_contract::CODE_QUERY_SCHEMA_V1 + && !self.truncated + && self.files.is_empty() + } +} +impl CacheRecord for compass_model::query_contract::DiscoveryQueryResponse { + fn valid_cache_record(&self) -> bool { + self.schema == compass_model::query_contract::DISCOVERY_QUERY_SCHEMA_V1 && !self.truncated + } +} + +#[cfg(test)] +mod tests { + use super::*; + use compass_model::query_contract::{CodeQueryLimits, CodeQueryOperation, CodeQueryResponse}; + + #[test] + fn reopened_cache_is_bound_to_request_graph_and_mode_and_checks_corruption() + -> Result<(), Box> { + let root = tempfile::tempdir()?; + let cache = ResponseCache::open(root.path(), "graph-a", None); + let key = cache.key("callers", false, &"target").ok_or("key")?; + let response = + CodeQueryResponse::empty(CodeQueryOperation::Callers, CodeQueryLimits::default()); + cache.put(&key, &response); + drop(cache); + let cache = ResponseCache::open(root.path(), "graph-a", None); + assert_eq!(cache.get::(&key), Some(response)); + for changed in [ + cache.key("callees", false, &"target"), + cache.key("callers", false, &"other"), + cache.key("callers", true, &"target"), + ] { + assert!( + cache + .get::(&changed.ok_or("key")?) + .is_none() + ); + } + let other = ResponseCache::open(root.path(), "graph-b", None); + assert!( + other + .get::(&other.key("callers", false, &"target").ok_or("key")?) + .is_none() + ); + let program = compass_ir::ProgramBundle::default(); + let different_ir = ResponseCache::open(root.path(), "graph-a", Some(&program)); + assert!( + different_ir + .get::( + &different_ir.key("callers", false, &"target").ok_or("key")? + ) + .is_none() + ); + let connection = Connection::open(root.path().join("responses-v1.sqlite3"))?; + connection.execute("UPDATE responses_v1 SET payload='{}' WHERE key=?1", [&key])?; + assert!(cache.get::(&key).is_none()); + Ok(()) + } + + #[test] + fn bounded_cache_evicts_old_entries_and_skips_oversized_payloads() + -> Result<(), Box> { + let root = tempfile::tempdir()?; + let cache = ResponseCache::open(root.path(), "graph", None); + for index in 0..MAX_ENTRIES + 5 { + cache.put(&index.to_string(), &index); + } + assert!(cache.get::("0").is_none()); + assert_eq!(cache.get::("36"), Some(36)); + cache.put("oversized", &"x".repeat(MAX_PAYLOAD_BYTES + 1)); + assert!(cache.get::("oversized").is_none()); + let connection = Connection::open(root.path().join("responses-v1.sqlite3"))?; + assert_eq!( + connection.query_row("SELECT COUNT(*) FROM responses_v1", [], |row| row + .get::<_, usize>(0))?, + MAX_ENTRIES + ); + Ok(()) + } + #[test] + fn engine_hits_skip_execution_and_partial_responses_are_not_cached() + -> Result<(), Box> { + use compass_model::code_graph::{BuildMetadata, GraphDocument}; + let root = tempfile::tempdir()?; + let path = root.path().join("graph.json"); + let graph = GraphDocument::empty_v1(BuildMetadata { + builder_version: "test".to_owned(), + schema_fingerprint: "sha256:test".to_owned(), + source_tree_digest: "sha256:test".to_owned(), + configuration_digest: "sha256:test".to_owned(), + generation_id: "sha256:test".to_owned(), + source_commit: None, + }); + std::fs::write(&path, serde_json::to_vec(&graph)?)?; + let engine = crate::open(&path, None, &root.path().join("cache"))?; + let executions = std::cell::Cell::new(0); + let limits = CodeQueryLimits::default(); + for _ in 0..2 { + engine.cached_response( + "test", + &limits, + || { + executions.set(executions.get() + 1); + Ok(CodeQueryResponse::empty( + CodeQueryOperation::Callers, + limits.clone(), + )) + }, + |_| true, + )?; + } + assert_eq!(executions.get(), 1); + for _ in 0..2 { + engine.cached_response( + "partial", + &limits, + || { + executions.set(executions.get() + 1); + let mut response = + CodeQueryResponse::empty(CodeQueryOperation::Callers, limits.clone()); + response.truncated = true; + Ok(response) + }, + |_| true, + )?; + } + assert_eq!(executions.get(), 3); + for _ in 0..2 { + engine.cached_response( + "source", + &limits, + || { + executions.set(executions.get() + 1); + let mut response = + CodeQueryResponse::empty(CodeQueryOperation::Callers, limits.clone()); + response + .files + .push(compass_model::query_contract::QueryFile { + path: "src/lib.rs".to_owned(), + content_digest: "sha256:test".to_owned(), + source: Some("code".to_owned()), + truncated: false, + }); + Ok(response) + }, + |_| true, + )?; + } + assert_eq!(executions.get(), 5); + let different_limits = CodeQueryLimits { + max_nodes: 1, + ..limits + }; + engine.cached_response( + "test", + &different_limits, + || { + executions.set(executions.get() + 1); + Ok(CodeQueryResponse::empty( + CodeQueryOperation::Callers, + different_limits.clone(), + )) + }, + |_| true, + )?; + assert_eq!(executions.get(), 6); + Ok(()) + } +} diff --git a/docs/implementation/compact-query-output.md b/docs/implementation/compact-query-output.md new file mode 100644 index 00000000..858cedc8 --- /dev/null +++ b/docs/implementation/compact-query-output.md @@ -0,0 +1,43 @@ +# Compact query output qualification + +This change makes human-readable answers smaller while preserving typed query +records, source anchors, direction, multiplicity and uncertainty. Native search +and traversal remain offline and do not depend on a comparison tool. + +| Requirement | Implementation | Verification | +| --- | --- | --- | +| Compact defaults with full audit access | `compass-output` shares compact status and source-located ledgers; CLI `--verbose`/`--evidence` adds status and record evidence | Renderer and CLI answer-equivalence tests; MCP structured-content test | +| Answer-sized one-hop cost | Suppress duplicate entity inventories and empty sections | One-hop regression requires fewer than 100 estimated tokens with both source locations | +| Short non-answers | Cap compact no-match/candidate text at 199 approximate tokens | Empty/ambiguous renderer and CLI fixtures; MCP points to full structured records | +| Strict budget with continuation | Four UTF-8 bytes per approximate token, including newline and continuation; whole typed entries or explicit required budget | Unicode reconstruction and page-budget tests; finite-command interface panel | +| Read remaining output without repeating actions | Immutable `compass.saved-output/1` records and additive `compass output` reader | Digest/corruption, retention, redirected-path, machine-output and init publication tests | +| Faster repeated native queries | Complete-response cache bound to verified graph, Program IR, profiles, complete request and semantic mode | Cache-hit execution counter, reopen/invalidation/corruption tests and registered optimized cold/warm panel | +| Preserve correctness while reducing average cost | Replay unchanged source-reviewed questions over frozen public-repository graphs | Fixed 25-question natural suite and 50-question standard suite; per-row verdict comparison | + +The cache excludes truncated executions and source excerpts. Profiling and +caller-supplied cancellation paths execute natively. Cache failures fall back to +native execution. Storage has entry, payload, page and scan limits; neither +cache rewrites graphs or historical realizations. + +Finite budgets reject streaming events, watch, serve and REPL before starting. +Interactive init requires `--yes`. Oversized JSON/JSONL fails without emitting a +partial machine record and points to the completed saved output. Saved records +are work-directory-local and can be evicted; this is a disposable continuation +facility, not archival storage. + +Token measurements concern CLI stdout and use the existing harness's UTF-8 +byte estimate. MCP text is compact, while full structured content remains +available and still contributes to clients that consume both. The five public +source trees and their source-reviewed text oracles are a focused recall panel; +they do not independently establish precision or population-wide accuracy. + +Exact-only lookup and source-bearing exploration remain native indexed queries; +the response cache does not promise acceleration for those paths. + +The [recorded optimized replay](../../benchmarks/agent_query/compact_query_output_review.json) +keeps every native oracle verdict: 49/50 standard questions at 363.76 mean +estimated tokens versus the comparator's 549.74, and 24/25 natural questions at +547.44 versus 757.44. The [performance reference](../../PERFORMANCE.md#compact-query-text-and-complete-response-caching) +records both remaining failures, slightly larger natural answers, cold-run +regressions and the partial Gson graph that intentionally remains uncached. +All repeated raw JSON answers match across both optimized binaries. diff --git a/docs/reference/commands.md b/docs/reference/commands.md index ba527b19..fbd9925f 100644 --- a/docs/reference/commands.md +++ b/docs/reference/commands.md @@ -329,7 +329,7 @@ relationship context, and DFS compose within that contract. `--result-envelope` requires `--format json` and opt-in wraps the unchanged discovery result in `compass.query.discovery-result/1` with a query-owned `semanticResultDigest`. Without this flag, the existing JSON shape remains unchanged. `--traverse`, -`--budget`, or `--page` explicitly select legacy relevance traversal and cannot +or `--page` explicitly select legacy relevance traversal and cannot be mixed with discovery controls. CompassQL routing is unchanged. The default focused neighborhood is 64 nodes and 128 edges. Use `--max-nodes 500 --max-edges 1000` when a query intentionally needs the full @@ -346,15 +346,15 @@ semantic digest. `--evidence` selects the full audit projection. Exact-looking operands that do not resolve emit `NO EXACT MATCH`; bounded fuzzy and lexical candidates can still follow as suggestions but are not represented as exact. -`--text-budget` bounds the discovery text projection and defaults to 8,000 +`--text-budget` bounds the discovery text projection and defaults to 800 approximate tokens. Its opaque cursor binds the contract version, normalized request/options, selected graph generation and digest, semantic-response digest, evidence tier, and next stable section/item. Fetch the next page with `--cursor TOKEN` and otherwise unchanged semantic inputs. The presentation-only `--text-budget` may change between pages. Pages contain whole deterministic entries; changed inputs fail instead of silently continuing a -different result. JSON rejects text pagination controls. Legacy `--budget` and -numeric `--page` retain their existing meaning only with legacy traversal. +different result. JSON rejects text pagination controls. `--budget N` bounds completed output +on all finite commands; use `--traverse` or numeric `--page` for legacy traversal. Query seeds prefer source-backed declarations over unresolved external-symbol placeholders with the same callable label. Source-less placeholder nodes retain @@ -523,7 +523,7 @@ it prints. `compass.query/1` response and is the right choice when an audit consumer needs every evidence record. The natural `query` command accepts the same `agent-json` format for discovery; its text header is answer-first while the -existing discovery entry ledger and v2 cursor semantics are unchanged. +existing discovery entry ledger and v3 cursor semantics are unchanged. Text output is paged. `--text-budget ` sets the approximate token budget of one page (default 2000) and the closing `Pagination:` line carries a @@ -602,6 +602,46 @@ Such an edge proves a module-level dependency, not a direct symbol call; impact paths include the containment hop instead of silently jumping from the selected symbol to the importer. +### Compact answers and completed-output budgets + +Typed query text defaults to a compact answer: one confidence/coverage line, +source locations and relationships. Empty sections and duplicate inventories are +omitted. No-match and unresolved candidate pages are capped below 200 approximate +tokens. `--verbose` or `--evidence` adds status, identities and per-record +provenance to the paged text ledger. Raw JSON and Agent View JSON keep their typed +contracts. MCP human-readable content is compact; its full `structuredContent` +continues to carry raw records and the Agent View. + +Every finite command accepts `--budget N` (32–65536). The ceiling is `4*N` UTF-8 +bytes across stdout and stderr, including the final newline and continuation. +This is the repository's approximate token metric, not a model-token guarantee. +Typed query pages preserve whole entries; a budget too small for one entry fails +with the required larger budget. Keep the same evidence mode when following +`--cursor`. + +Commands without a native cursor save an immutable completed answer locally and +print its continuation: + +```text +compass output ID --offset BYTE --budget N +``` + +Run that exact command from the same working directory. Reading saved output +never repeats the original action. Records under `.compass/cache/output` are +bounded to 4 MiB each, with retention of 16 answers; rerun the original command +if its answer has been evicted. Combined stdout/stderr uses an explicit `STDERR:` +separator. Machine output that exceeds a budget fails without emitting partial +JSON/JSONL and points to the saved complete output. Use a sufficiently large +budget to read it as one record, or omit the budget on the original command. +Streaming JSONL events, watch, REPL and MCP serve reject completed-output budgets +before starting. Interactive `init` requires `--yes` with a budget. + +Complete, excerpt-free native search, natural query, discovery, usage, impact +and node-path responses use a disposable local response cache. Graph identity, +Program IR, query profiles, request limits and semantic mode bind each entry. +The graph is verified before cache lookup. Partial answers and source excerpts +are recomputed, and cache failures fall back to native execution. + ### `architecture` ```text @@ -1396,3 +1436,6 @@ the exact command boundary your automation uses. **Next step:** run `compass --help` for the command you will automate, then pin its input, structured output, and exit expectations in an integration test. + +Exact-only lookup and source-bearing exploration remain native indexed queries; +the response cache does not promise acceleration for those paths. diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 1daa8caa..0346b118 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -366,10 +366,10 @@ community, or subsystem. Use repeatable `--scope KIND:VALUE` for an explicit OR scope over `community`, `source`, `package`, or `node`. `--text-budget` controls approximate rendered tokens per discovery page -(default 8,000). The default projection is concise; `--evidence` includes full +(default 800). The default projection is concise; `--evidence` includes full node/edge provenance and semantic digests. Follow the opaque `next` cursor with the same semantic query and evidence tier; the presentation-only text budget may -change. `--traverse`, `--budget`, and +change. `--budget N` bounds completed output; `--traverse` and `--page` explicitly select the bounded legacy compatibility renderer. The default semantic neighborhood contains at most 64 nodes and 128 edges. `--max-nodes` and `--max-edges` may raise those bounds to the hard ceilings of @@ -504,3 +504,8 @@ job metadata even if Compass's own history fingerprint already includes it. **Next step:** replace implicit defaults in one automation workflow with explicit non-secret options and record the selected profile/version. + +Completed-output budgets apply to finite commands via `--budget N` (32–65536), +using four UTF-8 bytes per approximate token, including continuations. See +[compact answers and budgets](commands.md#compact-answers-and-completed-output-budgets) +for saved output, machine-output handling and streaming exceptions.