[getappz]

Closes agentflare item #381 (NOT GitHub PR #381).

Found dogfooding mcp__flare__docs after the rename landed in #342/#344.

The problem

action="list" serialized every live document with its content, and limit was optional with no default. Per-item rustdoc/npm indexing (#337/#343) means one crate contributes hundreds of documents, so a routine cache is five figures. Measured against my own store:

	payload	approx tokens
before	6,332,854 bytes	~1,583,000
after (default page)	24,278 bytes	~6,069

261x. Both numbers come from running each code path's exact SQL against the live 10,857-document cache, not a fixture.

The bounded call was broken too: limit: 15 returned 71,073 characters and tripped the harness's max-tokens guard, because the cap bounded the number of bodies, not the bytes.

The change

// before
[ { "id": "...", "path": "...", "content": "<the entire page>", ... }, ...×10857 ]

// after
{ "total": 10857, "offset": 0, "returned": 100, "docs": [ { "id": "...", "path": "...",
  "title": "...", "doc_type": "...", "source": "...", "version": 3,
  "created_at": ..., "updated_at": ..., "content_bytes": 5732 }, ... ] }

Three decisions worth flagging for review:

total travels with the page. A truncated listing that cannot say what it withheld reads exactly like a complete one — that is precisely how a caller concludes a cache is smaller than it is.

offset is new, and was not in the filed item. I added it because a cap is only defensible if bounding a response does not put a document out of reach. Without it the fix would trade one silent wrongness for another.

The projection is pushed into SQL, not applied to a fetched Vec. Truncating in Rust still makes SQLite read and allocate every row's content first, which is the cost being removed. content_bytes casts to BLOB before measuring because SQLite's LENGTH() counts characters on TEXT, and a caller sizing a response needs bytes — there is a test pinning that (日本 is 6, not 2).

doc_list/DocsStore::list are deliberately untouched. The CLI writes to a terminal a human can pipe through jq/head; changing its output shape would break scripts to solve a problem it does not have. doc_list_summaries/doc_count are additive on the shared store, so existing consumers are unaffected.

Tests

The existing list_returns_everything_by_default_but_honours_a_capped_limit encoded the bug as the contract, so it is replaced rather than extended. Four tests take its place:

• default cap applies and total reports the full set
• an oversized limit clamps rather than errors
• offset reaches documents past the first page (paths zero-padded so ORDER BY path matches numeric order — otherwise the assertion would not say what it appears to say)
• the size property itself: a 100 KB body must not make the listing 100 KB bigger, asserted as raw.len() < 1_000 rather than a brittle byte count

Plus three at the store layer: paging without bodies, deleted/other-project exclusion, and the bytes-not-characters case.

Verification

• cargo test --workspace — 834 bin tests + every crate suite, 0 failures
• cargo fmt --all — clean
• cargo clippy — 0 warnings on agentflare-store, flare-docs, and the agentflare bin

Summary by CodeRabbit

• New Features

  • Added paginated document listings with total counts, offsets, and returned-item metadata.
  • Listings now provide compact document summaries, including content size, without returning full document bodies.
  • Added document count reporting and support for navigating multiple pages.
  • Added default and maximum limits to keep listing responses bounded.

• Documentation

  • Clarified pagination behavior and limits in the request schema.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 884de2f7-7fe1-4651-b7f9-80c63da24aa6

📥 Commits

Reviewing files that changed from the base of the PR and between 3d33b6a and 15c18c2.

📒 Files selected for processing (5)

• crates/agentflare-store/src/documents.rs
• crates/flare-docs/src/lib.rs
• crates/flare-docs/src/store.rs
• src/mcp_server/flare_docs.rs
• src/mcp_server/types.rs

---

📝 Walkthrough

Walkthrough

The PR adds lightweight document summaries and count APIs, threads bounded pagination through DocsStore, and changes the MCP list action to return paginated metadata and summaries without document bodies.

Changes

Document listing pagination

Layer / File(s)	Summary
Document summary storage APIs crates/agentflare-store/src/documents.rs	Adds DocumentSummary, SQL-side summary pagination, live-document counting, byte-length calculation, and coverage for filtering, pagination, and UTF-8 content sizes.
DocsStore pagination layer crates/flare-docs/src/store.rs, crates/flare-docs/src/lib.rs	Adds default and maximum listing limits, summary listing and counting methods, and public re-exports for the new limits.
MCP paginated list contract src/mcp_server/types.rs, src/mcp_server/flare_docs.rs	Adds list offsets and updates the MCP list response to include total, offset, returned, and body-free document summaries, with pagination tests.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MCPClient
  participant flare_docs_impl
  participant DocsStore
  participant Store
  MCPClient->>flare_docs_impl: submit list request
  flare_docs_impl->>DocsStore: count()
  DocsStore->>Store: count live documents
  Store-->>DocsStore: total
  flare_docs_impl->>DocsStore: list_summaries(limit, offset)
  DocsStore->>Store: fetch bounded summary page
  Store-->>DocsStore: DocumentSummary rows
  DocsStore-->>flare_docs_impl: summaries
  flare_docs_impl-->>MCPClient: paginated JSON response

Loading

Possibly related PRs

• getappz/agentflare#316: Earlier flare_docs listing implementation that this paginated summary flow extends.
• getappz/agentflare#344: Related changes to limit handling in the same MCP list action.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	Clearly summarizes the main change: the list command no longer returns full cached documents.
Description check	✅ Passed	Covers the problem, the change, tests, and verification, but does not use the requested template sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch task/381

---

_{Comment @coderabbitai help to get the list of available commands.}