[getappz]

Summary

Adds flare-docs, a new crate that fetches third-party package documentation on demand (Rust crates via docs.rs's official JSON API for Phase 1), exposed via a new flare_docs MCP tool (action=search|get|list|refresh) and an agentflare docs <search|get|list|refresh> CLI subcommand. Reuses agentflare-store's existing document/blob storage (SQLite + FTS5) in its own dedicated global DB file (~/.agentflare/flare-docs.db, project_id="global"), separate from the shared ~/.agentflare/store.db.

• Fully synchronous crate — HTTP fetch via ureq (this repo's house-standard client, already used in 6+ other crates), no tokio/async-trait inside flare-docs itself
• flare_docs MCP tool offloads the network fetch via tokio::task::spawn_blocking so a slow/hung docs.rs response can't freeze the (single-threaded) MCP server
• PROJECT_ID="global" sentinel enforced everywhere through a single DocsStore facade — no path bypasses it or touches the shared store

Design note: the fetch client went through two pivots before landing on ureq — wreq (browser-fingerprint-aware) was dropped after it dragged in BoringSSL/NASM/CMake native-build fragility on Windows; reqwest worked but wasn't house-standard. Also fixed along the way: the docs.rs rustdoc-JSON resolver initially assumed "root" is always a JSON string — the real payload uses a JSON number — caught via a live network smoke test and fixed with end-to-end verification against real serde docs.

Test plan

• cargo test --workspace — 744 passed, 0 failed, 1 ignored
• cargo test -p flare-docs — full crate suite green
• Live end-to-end: agentflare docs get serde --version latest returns real serde documentation
• Live MCP concurrency smoke test confirming a slow flare_docs get no longer blocks concurrent flare_docs list calls
• Every task + fix individually reviewed (spec compliance + code quality), plus a final whole-branch review — its one Important finding (blocking fetch on the single-threaded runtime) fixed and re-approved

Known follow-ups (logged, not blocking): 404s map to internal_error instead of invalid_params; limit is unbounded; zstd decompression has no output-size cap; CLI get's help text implies cache-awareness it doesn't have yet (TTL/cache-check deferred); a dead status-recheck in the fetch client; no committed regression test for the spawn_blocking concurrency property (only an ad-hoc manual script).

Summary by CodeRabbit

• New Features
  • Added agentflare docs CLI commands for search, list, get, and refresh with pretty-printed JSON.
  • Introduced an on-demand MCP docs tool (flare_docs) backed by a local, cacheable store (including in-memory mode).
  • Added docs ingestion for docs.rs rustdoc-JSON and storage for reuse.
• Bug Fixes
  • Fixed rustdoc-JSON root-id parsing to handle numeric and string forms.
  • Prevented MCP server stalls by moving network fetches off the server thread and improving concurrency behavior.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The pull request adds a flare-docs crate for synchronous docs.rs rustdoc JSON fetching, parsing, and storage, then exposes search, retrieval, listing, and refresh operations through CLI commands and an asynchronous MCP tool.

Changes

Flare docs integration

Layer / File(s)	Summary
Fetcher and document store foundation Cargo.toml, crates/flare-docs/Cargo.toml, crates/flare-docs/src/{fetch,store,lib}.rs	The workspace adds flare-docs, providing synchronous ureq fetching, Zstd decompression, scoped document persistence, and public API exports.
Rustdoc ingestion pipeline crates/flare-docs/src/rustdoc.rs, .superpowers/sdd/progress.md	Rustdoc JSON is fetched, decompressed, parsed with string or numeric root IDs, converted into document metadata, blob-stored, and persisted.
CLI documentation commands src/cli/mod.rs, src/cli/docs.rs	The CLI adds docs search, get, list, and refresh commands with JSON output and default version and limit handling.
MCP documentation tool src/mcp_server.rs, src/mcp_server/types.rs, src/mcp_server/flare_docs.rs	The MCP server adds lazy docs-store management and a flare_docs tool supporting search, list, get, and refresh actions, with blocking network work moved to Tokio’s blocking pool.
Estimated code review effort: 4 (Complex)	~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MCPClient
  participant AgentflareMcp
  participant UreqFetcher
  participant DocsStore
  MCPClient->>AgentflareMcp: Submit flare_docs action
  AgentflareMcp->>UreqFetcher: Fetch docs.rs rustdoc JSON in spawn_blocking
  UreqFetcher-->>AgentflareMcp: Return compressed response
  AgentflareMcp->>DocsStore: Parse and persist document
  DocsStore-->>AgentflareMcp: Return document result
  AgentflareMcp-->>MCPClient: Return JSON response

Loading

Possibly related PRs

• getappz/agentflare#60: Modifies the CLI command dispatch that this change extends with the Docs subcommand.
• getappz/agentflare#157: Modifies MCP tool registration in the same server routing area.
• getappz/agentflare#253: Introduces the MCP request-type and handler structure used by the new FlareDocsRequest tool.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly captures the main change: adding the flare-docs crate, MCP tool, and CLI docs subcommand.
Description check	✅ Passed	The description covers the summary and test plan well, with only the Notes for reviewers section left implicit rather than explicitly templated.
Docstring Coverage	✅ Passed	Docstring coverage is 88.24% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch task/325

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 6

🧹 Nitpick comments (1)

src/mcp_server/flare_docs.rs (1)

78-97: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Unbounded blocking fetch: a stalled docs.rs response can hang this tool call for up to 5 minutes.

UreqFetcher is configured with a 300s read timeout (per crates/flare-docs/src/fetch.rs), and fetch_and_store_via_spawn_blocking awaits the spawn_blocking task with no additional deadline. A slow/stalled docs.rs response leaves the calling agent/session blocked on this MCP tool call for up to five minutes with no partial feedback.

Consider wrapping the spawn_blocking await in tokio::time::timeout with a shorter, tool-appropriate bound (e.g. 30-60s), returning a clear timeout error instead of silently hanging.

⏱️ Sketch of a bounded fetch

-        let fetched = tokio::task::spawn_blocking(move || {
-            let fetcher = UreqFetcher::new();
-            fetcher.fetch(&url)
-        })
-        .await
-        .map_err(|e| ErrorData::internal_error(format!("fetch task panicked: {e}"), None))?
-        .map_err(|e| ErrorData::internal_error(e.to_string(), None))?;
+        let fetched = tokio::time::timeout(
+            std::time::Duration::from_secs(60),
+            tokio::task::spawn_blocking(move || {
+                let fetcher = UreqFetcher::new();
+                fetcher.fetch(&url)
+            }),
+        )
+        .await
+        .map_err(|_| ErrorData::internal_error("docs.rs fetch timed out".to_string(), None))?
+        .map_err(|e| ErrorData::internal_error(format!("fetch task panicked: {e}"), None))?
+        .map_err(|e| ErrorData::internal_error(e.to_string(), None))?;

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/mcp_server/flare_docs.rs` around lines 78 - 97, Update
fetch_and_store_via_spawn_blocking to wrap the spawn_blocking task await in
tokio::time::timeout using a shorter tool-appropriate deadline, such as 30–60
seconds. Preserve the existing panic and fetch-error mappings, and return a
clear internal timeout error when the deadline expires.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.superpowers/sdd/progress.md:
- Line 144: Update the documentation text in the progress note so the Rust
attribute #[tool] is wrapped in inline-code backticks, preventing the Markdown
heading rule from interpreting it as a heading.

In `@crates/flare-docs/src/fetch.rs`:
- Around line 22-23: Replace the unbounded response buffering and
decompress_zstd implementation with a shared bounded reader helper: reject
declared Content-Length values above the compressed-size limit, cap chunked
reads, and stream zstd output while enforcing the decompressed-size limit.
Preserve FetchError mapping and add tests covering both compressed and
decompressed limit violations.

In `@crates/flare-docs/src/rustdoc.rs`:
- Around line 42-48: Update the docs extraction flow in the function returning
`Ok(docs)` to distinguish valid `docs: null` from malformed responses: require
`index[root_key]` and its string `docs` field to be present, and return an error
when either is missing or non-string. Preserve `Some` for valid strings and
`None` only when the existing docs value is explicitly null, preventing
`store_fetched` from caching a placeholder for invalid responses.

In `@src/cli/docs.rs`:
- Around line 10-31: Update the documentation comment for the DocsCmd::Get
variant to describe its actual behavior through the shared fetch_and_store path,
distinguishing it from Refresh only if the implementation does so. Do not change
command behavior or the Refresh documentation.

In `@src/mcp_server/flare_docs.rs`:
- Line 1: Separate the package-based get and refresh flows in the flare_docs
handler and the corresponding CLI docs logic. Make get read the cached docs
entry from DocsStore at docsrs/{package}/{version} without fetching, while
refresh continues through the network fetch-and-store path; preserve the
existing behavior for other request modes.
- Around line 50-63: Update the `refresh` branch in the request handler so it
does not simply call `fetch_and_store_via_spawn_blocking` with the same behavior
as `get`; add or pass a force-refresh option through the relevant helper path to
bypass cached package data, while preserving the existing `get` behavior and
validation.

---

Nitpick comments:
In `@src/mcp_server/flare_docs.rs`:
- Around line 78-97: Update fetch_and_store_via_spawn_blocking to wrap the
spawn_blocking task await in tokio::time::timeout using a shorter
tool-appropriate deadline, such as 30–60 seconds. Preserve the existing panic
and fetch-error mappings, and return a clear internal timeout error when the
deadline expires.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f6ebd991-c7af-473f-a43e-a8ca054d2e18

📥 Commits

Reviewing files that changed from the base of the PR and between 45a6b86 and 6447c4c.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (12)

• .superpowers/sdd/progress.md
• Cargo.toml
• crates/flare-docs/Cargo.toml
• crates/flare-docs/src/fetch.rs
• crates/flare-docs/src/lib.rs
• crates/flare-docs/src/rustdoc.rs
• crates/flare-docs/src/store.rs
• src/cli/docs.rs
• src/cli/mod.rs
• src/mcp_server.rs
• src/mcp_server/flare_docs.rs
• src/mcp_server/types.rs

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

.superpowers/sdd/progress.md (1)

1-7: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Remove or clearly separate the unrelated historical section.

The header says the previous stale plan content was reset, but lines 203-448 still contain the unrelated gateway-search plan history. This makes the progress document ambiguous and can mislead future readers or automation. Remove that section or place it under an explicitly labeled archive.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.superpowers/sdd/progress.md around lines 1 - 7, Remove the unrelated
gateway-search plan history from the progress document, specifically the stale
section spanning the historical content after the current plan header. If
retaining it, move it under a clearly labeled archive section so the active
2026-07-23-flare-docs-implementation-plan content remains unambiguous.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In @.superpowers/sdd/progress.md:
- Around line 1-7: Remove the unrelated gateway-search plan history from the
progress document, specifically the stale section spanning the historical
content after the current plan header. If retaining it, move it under a clearly
labeled archive section so the active 2026-07-23-flare-docs-implementation-plan
content remains unambiguous.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4175b3eb-31f3-44e2-80b0-b05ffaffa592

📥 Commits

Reviewing files that changed from the base of the PR and between 6447c4c and 8bf4244.

📒 Files selected for processing (8)

• .superpowers/sdd/progress.md
• crates/flare-docs/src/fetch.rs
• crates/flare-docs/src/lib.rs
• crates/flare-docs/src/rustdoc.rs
• crates/flare-docs/src/store.rs
• src/mcp_server.rs
• src/mcp_server/flare_docs.rs
• src/mcp_server/types.rs

🚧 Files skipped from review as they are similar to previous changes (7)

• crates/flare-docs/src/lib.rs
• src/mcp_server/types.rs
• src/mcp_server/flare_docs.rs
• crates/flare-docs/src/store.rs
• src/mcp_server.rs
• crates/flare-docs/src/rustdoc.rs
• crates/flare-docs/src/fetch.rs