Skip to content

feat(server): make research.fetch actually fetch (timeouts, size caps, extraction) #102

Description

@flyingrobots

Re-scoped 2026-08-16 after verifying this issue against the code. The original text is preserved at the bottom.

The quotas are real and tested. The fetcher does not fetch.

server/routes/research.js:38:

// 3. Perform "fetch" (stubbed for now as in memory path)
const snapshot = { url: input.url, title: 'Snapshot', content: '...', created_at: ... };

There is no HTTP request anywhere in the file — no timeout, no size cap, no title/author/excerpt/canonical-URL extraction. Every snapshot ever written to research_cache has the literal title Snapshot and content ....

Shipped

  • research_usage_increment with quota_exceeded → 429 (db/rpc.sql:1035-1058, server/routes/research.js:21-28,54-55), tested with a real first-succeeds/second-429 sequence.
  • Cache dedup by url_hash (db/schema.sql:185-190), tested.

Remaining

  1. An actual fetch with a timeout and a response size cap.
  2. Snapshot extraction: title, authors, excerpt, canonical URL — the fields docs/specs/ResearchTools.md specifies and that the web preview card (fix(web): snippet previews read snapshot fields the server never returns #101) already tries to read.
  3. pgTAP for the cache constraints and quota hooks — zero pgTAP files mention research.

⚠️ Note the quota is also unreachable in practice: its limit comes from rooms.config, which room_create never persists (#200).


Original issue text

See docs/specs/ResearchTools.md. Snapshot fetcher with timeouts and size caps; enforce quotas per room/round.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions