Skip to content

Add Session Explorer API - #552

Open
jamby77 wants to merge 9 commits into
session-explorer/fixturesfrom
session-explorer/api
Open

jamby77 wants to merge 9 commits into
session-explorer/fixturesfrom
session-explorer/api

Conversation

@jamby77

@jamby77 jamby77 commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Second of three stacked PRs for the Session Explorer. This PR adds namespace storage, a bounded scanner and reader for session keys, and the /session-explorer API, with oversize flags gated to Pro.

Stack: fixtures → api → web

Changes

  • Namespaces (name, key pattern, preset, optional custom schema, oversize threshold) are stored per connection in memory, SQLite and Postgres, with a shared contract test.
  • Scanner: SCAN across all cluster primaries with a 10,000-key cap. Keys are grouped into sessions with size, TTL and last access, and partial results are reported when nodes fail.
  • Reader: per-key and total byte caps (2 MB / 32 MB), slot-routed cluster reads, and pipelined batches. Keys that vanish or fail are counted, never fatal.
  • API:
    • CRUD for namespaces;
    • preview of one sampled key;
    • session list;
    • session detail, including a complete flag with the reasons it is false;
    • streamed JSON export;
    • GET .../oversize, which requires SESSION_OVERSIZE_FLAGS.
  • Privacy: session contents are never logged or persisted. 404 messages are generic.

Checklist

  • Unit / integration tests added
  • Docs added / updated
  • Roborev review passed — run roborev review --branch or /roborev-review-branch in Claude Code (internal)
  • Competitive analysis done / discussed (internal)
  • Blog post about it discussed (internal)

🤖 Generated with Claude Code


Note

Medium Risk
New live-read paths against customer Valkey data (including export) with cluster fan-out; mitigated by caps and no persistence, but responses can contain sensitive session contents.

Overview
Adds a Session Explorer API so users can define per-connection namespaces (key pattern, parsing preset, oversize threshold) and inspect live Valkey session data without persisting key contents.

Namespace config is stored via new StoragePort methods and session_explorer_namespaces tables in SQLite/Postgres (plus memory adapter), with shared contract tests. Live operations use SCAN (not KEYS) across cluster primaries (primary-nodes, cluster-key-slot routing), a 10k-key scan cap, and a bounded reader (2 MB/key, 32 MB total) that pipelines reads and reports truncation, partial cluster failures, and unmeasured memory.

/session-explorer endpoints cover namespace CRUD, unsaved preview, session list/detail (with a complete flag), streamed JSON export, and Pro-gated oversize session IDs (SESSION_OVERSIZE_FLAGS). Shared types gain oversize/preview/export shapes and richer session metadata (unmeasuredKeys, completeness fields). Tests assert session payloads are not logged or written to app storage.

Reviewed by Cursor Bugbot for commit 253dea0. Bugbot is set up for automated code reviews on this repo. Configure here.

jamby77 and others added 6 commits October 16, 2026 10:12
- Add namespace CRUD to StoragePort and memory, sqlite, postgres adapters
- Add session_explorer_namespaces table indexed by connection
- Add validateNamespaceInput rejecting bare wildcard patterns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Share one storage contract across memory, sqlite and postgres specs
- Add live-Postgres spec with BIGINT threshold and JSONB schema cases
- Reject cross-connection saves in tests for every adapter
- Require safe integer thresholds and whitespace-free trimmed patterns

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Fan out SCAN MATCH/COUNT 1000 over cluster primaries, reusing parent
  auth/TLS and closing every node client it opens
- Stop at the key cap, pipeline TYPE/PTTL/MEMORY USAGE/OBJECT IDLETIME
  per batch and aggregate per session; flag partial on failed nodes
- Read session keys from their owning primary with a per-value byte cap,
  windowed list/zset reads, bounded HSCAN and raw JSON.GET strings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Pipeline first reads per batch after a batched locate, keeping only
  windowed continuations serial
- Add a 32 MB total byte budget; readSessionKeys and keysForSession now
  report truncation and failed nodes, plus missing keys for reads
- Route cluster reads by key slot via cluster-key-slot; MOVED and
  per-key read errors omit the key instead of failing the request
- Size JSON with JSON.DEBUG MEMORY when MEMORY USAGE is unavailable
- Read strings with GETRANGE 0 cap-1 and flag truncation explicitly

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Namespace CRUD, preview, session list, detail and streamed export
- Detail reports completeness: unparsed keys, truncation, failed nodes
  and missing keys

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: db4ebb6b-86a6-4b7d-9239-19b0df52713f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 765c423. Configure here.

Comment thread apps/api/src/session-explorer/session-explorer.service.ts
Comment thread apps/api/src/session-explorer/session-explorer.service.ts
Comment thread apps/api/src/session-explorer/session-scanner.ts Outdated

@jamby77 jamby77 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the scanner, reader, primary routing, service, controller and export stream, licence gating (LicenseGuard is global and SESSION_OVERSIZE_FLAGS is in the tier map), and the memory/sqlite/postgres namespace adapters, which behave the same. I skipped issues already raised in review comments and everything #553 fixes. Two defects remain: session lookup gives a 404 once a namespace is past the scan cap, and large hashes are read without a byte bound.

Comment thread apps/api/src/session-explorer/session-scanner.ts Outdated
Comment thread apps/api/src/session-explorer/session-reader.ts
jamby77 and others added 3 commits October 20, 2026 12:46
- Keep failed MEMORY USAGE reads as unknown instead of zero
- Return unmeasured sessions, partial and failedNodes from oversize

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant