diff --git a/docs/API_KEYS.md b/docs/API_KEYS.md index 1b7c3338..8e751162 100644 --- a/docs/API_KEYS.md +++ b/docs/API_KEYS.md @@ -20,23 +20,88 @@ API keys authenticate via the same `requireAuth` entry point as session JWTs. Th ## Scopes -| Scope | Description | -|-------|-------------| -| `portfolio:read` | Read portfolio data | -| `transactions:read` | Read transaction history | -| `deposit:write` | Create deposits | -| `withdraw:write` | Create withdrawals (opt-in per key) | -| `alerts:manage` | Manage alert rules | -| `fiat:write` | Create fiat orders | -| `recurring_deposits:write` | Manage recurring deposit plans | -| `goals:write` | Manage savings goals | -| `strategies:write` | Manage strategies | -| `webhooks:manage` | Manage webhook subscriptions | -| `vault:read` | Read vault data | -| `vault:write` | Write vault operations | +Every scope is described in a machine-readable catalogue +(`src/auth/scopes.ts` for user keys, `src/middleware/adminAuth.ts` for admin +keys). `GET /api/admin/keys/scopes` serves it for the key-creation UI, and +the tables below are generated from it — `tests/unit/auth/scopeCatalog.test.ts` +fails CI if the catalogue, the call sites and this document disagree. + +The **Enforcement** column says whether the scope actually gates anything: + +| Enforcement | Meaning | +| --- | --- | +| `enforced` | At least one route requires it. Granting it is meaningful; omitting it is restrictive. | +| `declared-only` | Declared and reservable, but no route requires it today — granting it currently grants nothing extra. | + +The **Implies** column lists the scopes a scope grants on its own. `super` +implies every admin scope, and the legacy `agent` implies `agent:read` and +`agent:write`. + +### User key scopes + + +| Scope | Risk | Enforcement | Implies | What it does | Guardrail | Endpoints | +| --- | --- | --- | --- | --- | --- | --- | +| `loans:read` | read | enforced | — | Read loan state and borrowing capacity. | — | `GET /api/v1/loans`, `GET /api/v1/loans/:loanId`, `GET /api/v1/loans/capacity` | +| `portfolio:read` | read | enforced | — | Read portfolio positions, balances, history, earnings and tax reports. | — | `GET /api/v1/portfolio/:userId`, `GET /api/v1/portfolio/:userId/earnings`, `GET /api/v1/portfolio/:userId/history`, `GET /api/v1/portfolio/:userId/suggestions`, `GET /api/v1/portfolio/:userId/tax-report` | +| `transactions:read` | read | enforced | — | Read transaction history. | — | `GET /api/v1/transactions/:id/events`, `GET /api/v1/transactions/:userId`, `GET /api/v1/transactions/detail/:txHash` | +| `vault:read` | read | enforced | — | Read vault state and balance. | — | `GET /api/v1/vault/balance` | +| `alerts:manage` | write | enforced | — | Manage alert rules. | — | `DELETE /api/v1/alerts/:id`, `PATCH /api/v1/alerts/:id`, `POST /api/v1/alerts`, `POST /api/v1/alerts/:id/restore` | +| `goals:write` | write | enforced | — | Manage savings goals. | — | `DELETE /api/v1/portfolio/goals/:id`, `PATCH /api/v1/portfolio/goals/:id`, `POST /api/v1/portfolio/goals` | +| `portfolio:write` | write | enforced | — | Change portfolio state. | — | `DELETE /api/v1/net-worth/external-wallets/:id`, `POST /api/v1/net-worth/external-wallets` | +| `strategies:write` | write | enforced | — | Publish, follow and unfollow strategies. | — | `POST /api/v1/strategies/publish` | +| `webhooks:manage` | write | enforced | — | Manage webhook endpoints and deliveries. | — | `DELETE /api/webhooks/:id`, `PATCH /api/webhooks/:id`, `POST /api/webhooks` | +| `deposit:write` | money | enforced | — | Create deposits. | Deposits still pass the compliance freeze and sub-account permission checks. | `POST /api/v1/deposit` | +| `fiat:write` | money | enforced | — | Create fiat on/off-ramp orders. | — | `POST /api/v1/fiat/orders` | +| `loans:write` | money | enforced | — | Originate and repay collateral loans. | Origination and repayment still pass the compliance freeze and sub-account permission checks. | `POST /api/v1/loans`, `POST /api/v1/loans/:loanId/repay` | +| `recurring_deposits:write` | money | enforced | — | Manage recurring deposit plans. | — | `DELETE /api/v1/deposit/recurring/:id`, `PATCH /api/v1/deposit/recurring/:id`, `POST /api/v1/deposit/recurring` | +| `vault:write` | money | enforced | — | Build vault transactions. | — | `POST /api/v1/vault/build-transaction` | +| `withdraw:write` | money | enforced | — | Create withdrawals. | Requires allowWithdrawals on the key AND the platform kill-switch USER_API_KEY_WITHDRAWALS_ENABLED to be on. | `DELETE /api/v1/recurring-withdrawals/:id`, `PATCH /api/v1/recurring-withdrawals/:id`, `POST /api/v1/recurring-withdrawals`, `POST /api/v1/withdraw` | + New keys are **read-only by default**. Write scopes must be explicitly requested. +### Admin key scopes + + +| Scope | Risk | Enforcement | Implies | What it does | Guardrail | Endpoints | +| --- | --- | --- | --- | --- | --- | --- | +| `agent:read` | read | enforced | — | Read agent decisions and circuit-breaker state. | — | `GET /api/v1/admin/agent/breakers`, `GET /api/v1/admin/agent/decisions` | +| `dlq:read` | read | enforced | — | Inspect the dead-letter queue. | — | `GET /api/v1/admin/dlq/inspect`, `GET /api/v1/admin/notifications/dlq` | +| `erasure:read` | read | declared-only | — | Preview what an erasure would delete. | No route requires this scope today — the erasure preview route is gated by the generic read scope. Declared so a key can be provisioned ahead of a route that needs it. | — | +| `fiat:read` | read | enforced | — | Read fiat providers and their health. | — | `GET /api/v1/admin/fiat/providers` | +| `keys:read` | read | enforced | — | Read admin API keys and wallet rotation status. | — | `GET /api/v1/admin/keys`, `GET /api/v1/admin/keys/scopes`, `GET /api/v1/admin/wallets/rotation-status` | +| `messages:read` | read | enforced | — | Read message delivery state. | — | `GET /api/v1/admin/messages`, `GET /api/v1/admin/messages/:id`, `GET /api/v1/admin/messages/stats` | +| `metrics:read` | read | enforced | — | Read operational metrics and dashboards. | — | `GET /api/v1/admin/jobs/dashboard`, `GET /api/v1/admin/stellar/metrics` | +| `outbox:read` | read | enforced | — | Read the durable outbox. | — | `GET /api/v1/admin/outbox`, `GET /api/v1/admin/outbox/:id`, `GET /api/v1/admin/outbox/stats` | +| `read` | read | enforced | — | General read access to admin resources. | — | `GET /api/v1/admin/erasure/preview/:userId`, `GET /api/v1/admin/erasure/requests`, `GET /api/v1/admin/referrals/review` | +| `referrals:read` | read | enforced | — | Read flagged referral conversions. | — | `GET /api/v1/admin/referrals/flagged` | +| `reserves:read` | read | enforced | — | Read reserve and sponsorship state. | — | `GET /api/v1/admin/reserves` | +| `risk-metadata:read` | read | declared-only | — | Read protocol risk metadata. | No route requires this scope today — the risk-metadata review workflow is administered from src/config/riskMetadataAdmin.ts, not through the API. Declared so a key can be provisioned ahead of a route that needs it. | — | +| `sessions:read` | read | enforced | — | Read user sessions. | — | `GET /api/v1/admin/users/:id/sessions` | +| `support:read` | read | enforced | — | Read support tickets. | — | `GET /api/v1/admin/support/tickets`, `GET /api/v1/admin/support/tickets/:id` | +| `treasury:read` | read | enforced | — | Read treasury sweep policies. | — | `GET /api/v1/admin/treasury/policies` | +| `approvals:write` | write | enforced | — | Cancel pending approval requests. | — | `POST /api/v1/admin/approvals/:id/cancel` | +| `sessions:write` | write | enforced | — | Revoke user sessions. | — | `POST /api/v1/admin/users/:id/sessions/revoke-all` | +| `support:write` | write | enforced | — | Reply to and update support tickets. | — | `PATCH /api/v1/admin/support/tickets/:id`, `POST /api/v1/admin/support/tickets/:id/reply` | +| `write` | write | enforced | — | General write access to admin resources. | — | `POST /api/v1/admin/erasure/requests`, `POST /api/v1/admin/referrals/review/:id/approve`, `POST /api/v1/admin/referrals/review/:id/reject` | +| `outbox:write` | money | enforced | — | Force-retry and cancel queued outbox operations. | — | `POST /api/v1/admin/outbox/:id/cancel`, `POST /api/v1/admin/outbox/:id/retry` | +| `referrals:write` | money | enforced | — | Approve or reject flagged referral conversions. | — | `POST /api/v1/admin/referrals/:id/review` | +| `treasury:write` | money | enforced | — | Change treasury sweep policies and run a sweep. | Sweep policies and signer rotations are written to the admin audit log. | `POST /api/v1/admin/treasury/emergency-sweep`, `POST /api/v1/admin/treasury/policies`, `POST /api/v1/admin/treasury/signer-rotations`, `POST /api/v1/admin/treasury/signer-rotations/:id/finalize` | +| `wallet` | money | declared-only | — | Custodial wallet operations. | No route requires this scope today — the wallet rotation-status route is gated by keys:read. Declared so a key can be provisioned ahead of a route that needs it. | — | +| `erasure:write` | destructive | enforced | — | Create, approve, reject and execute erasure requests. | Execution requires super — approval and execution are deliberately separated so one operator cannot both authorise and carry out a deletion. | `POST /api/v1/admin/erasure` | +| `keys:write` | destructive | enforced | — | Create, rotate and revoke admin API keys. | Minting an admin key is a takeover vector — every use is written to the admin audit log. | `DELETE /api/v1/admin/keys/:id`, `POST /api/v1/admin/keys`, `POST /api/v1/admin/keys/:id/rotate` | +| `super` | destructive | enforced | `support:read`, `support:write`, `read`, `write`, `wallet`, `agent`, `agent:read`, `agent:write`, `sessions:read`, `sessions:write`, `reserves:read`, `metrics:read`, `dlq:read`, `dlq:write`, `backfill:write`, `keys:read`, `keys:write`, `fiat:read`, `fiat:write`, `outbox:read`, `outbox:write`, `approvals:write`, `referrals:read`, `referrals:write`, `erasure:write`, `erasure:read`, `messages:read`, `messages:write`, `treasury:read`, `treasury:write`, `risk-metadata:read`, `risk-metadata:write`, `admin:impersonate`, `super` | Every admin scope, including erasure execution. | Held by as few keys as possible: it is a superset of every destructive scope in this catalogue. | `GET /api/v1/admin/audit/prove`, `GET /api/v1/admin/audit/verify`, `POST /api/v1/admin/erasure/requests/:id/approve`, `POST /api/v1/admin/erasure/requests/:id/execute`, `POST /api/v1/admin/erasure/requests/:id/reject` | +| `admin:impersonate` | operational | declared-only | — | View the API as a user, read-only. | No route requires this scope today — the impersonation controller exists but is not yet exposed through the admin API. Every impersonated request is attributed to the operator in the admin audit log. | — | +| `agent` | operational | declared-only | `agent:read`, `agent:write` | Agent loop operations. | — | — | +| `agent:write` | operational | enforced | — | Change agent circuit breakers. | — | `POST /api/v1/admin/agent/breakers`, `POST /api/v1/admin/agent/breakers/:id/reset` | +| `backfill:write` | operational | enforced | — | Run Stellar event backfills. | — | `POST /api/v1/admin/stellar/backfill` | +| `dlq:write` | operational | enforced | — | Replay, retry and resolve dead-lettered work. | — | `POST /api/v1/admin/dlq/replay`, `POST /api/v1/admin/dlq/resolve`, `POST /api/v1/admin/dlq/retry`, `POST /api/v1/admin/notifications/dlq/:id/retry` | +| `fiat:write` | operational | enforced | — | Change fiat provider failover state. | — | `POST /api/v1/admin/fiat/providers/:name/failover` | +| `messages:write` | operational | enforced | — | Retry and recover failed message deliveries. | — | `DELETE /api/v1/admin/messages/:id`, `POST /api/v1/admin/messages/:id/retry`, `POST /api/v1/admin/messages/retry-dead-letters` | +| `risk-metadata:write` | operational | declared-only | — | Review and publish protocol risk metadata. | No route requires this scope today — the risk-metadata review workflow is administered from src/config/riskMetadataAdmin.ts, not through the API. Declared so a key can be provisioned ahead of a route that needs it. | — | + + ### Withdrawal guardrails - `withdraw:write` requires `allowWithdrawals: true` at key creation. @@ -55,6 +120,42 @@ New keys are **read-only by default**. Write scopes must be explicitly requested An API key **cannot** create, rotate, or revoke other keys. +## Scope catalogue endpoint (admin auth) + +| Method | Path | Description | Scope | +|--------|------|-------------|-------| +| `GET` | `/api/v1/admin/keys/scopes` | The machine-readable scope catalogue, with the endpoints each scope covers | `keys:read` | + +The response carries both scope families: + +```jsonc +{ + "success": true, + "data": { + "user": { + "defaultReadScopes": ["portfolio:read", "transactions:read", "vault:read"], + "scopes": [ + { + "scope": "withdraw:write", + "description": "Create withdrawals.", + "risk": "money", + "grants": "Submits withdrawals to the durable outbox for on-chain settlement.", + "enforcement": "enforced", + "requires": "Requires allowWithdrawals on the key AND the platform kill-switch USER_API_KEY_WITHDRAWALS_ENABLED to be on.", + "endpoints": ["POST /api/v1/withdraw"] + } + ] + }, + "admin": { + "scopes": [/* the same shape, for admin key scopes */] + } + } +} +``` + +The `endpoints` list is derived from the live route table, so a scope cannot +claim to unlock an endpoint that stopped requiring it. + ## Errors | Status | Error | Meaning | diff --git a/docs/DOCUMENTATION_INDEX.md b/docs/DOCUMENTATION_INDEX.md index 7e3edbf8..1abbbfdc 100644 --- a/docs/DOCUMENTATION_INDEX.md +++ b/docs/DOCUMENTATION_INDEX.md @@ -22,6 +22,8 @@ - **[PRISMA_QUERY_PERFORMANCE.md](PRISMA_QUERY_PERFORMANCE.md)** - Hot webhook query plans, supporting indexes, and the query-index regression check - **[NET_WORTH.md](NET_WORTH.md)** - Read-only external Stellar links, valuation/staleness semantics, and private scope - **[CONSENT_AND_PRIVACY_POLICY.md](CONSENT_AND_PRIVACY_POLICY.md)** - User consent boundaries, high-risk workflow checks & auditable bypass policy (#510) +- **[RATE_LIMITING.md](RATE_LIMITING.md)** - Rate-limit tier catalogue & route→tier matrix: auth-type mapping, effective budgets, fail-open/fail-closed policy (#614) +- **[API_KEYS.md](API_KEYS.md)** - User & admin API key scope catalogue: descriptions, risk levels, enforcement status, and the endpoints each scope covers (#615) ### For DevOps/Deployment diff --git a/docs/RATE_LIMITING.md b/docs/RATE_LIMITING.md new file mode 100644 index 00000000..22196380 --- /dev/null +++ b/docs/RATE_LIMITING.md @@ -0,0 +1,198 @@ +# Rate Limiting + +Every route in this API is rate limited. This document is the map from a +route to the budget that applies to it, and the rules that decide which +budget a given caller draws on. + +It is not hand-maintained prose. The tables below are generated from +`src/middleware/rateLimitTiers.ts`, and +`tests/unit/middleware/rateLimitTierMatrix.test.ts` fails CI if the tables, +the code and the route table disagree. If you add a route, the matrix entry +is part of the change — not a follow-up. + +--- + +## The two layers every request passes through + +Before a request reaches a route, it passes through the **app-wide chain** +(`src/middleware/rateLimiterChain.ts`): + +| Tier | What it does | +| --- | --- | +| `global` | A flat budget over every request the server serves, matched or not. | +| `tiered` | The identity split. Anonymous callers draw on a strict per-IP budget; callers that have *proved* an identity draw on a generous per-principal one. | + +A route-specific tier (`sensitive`, `optimizer`, …) is applied **in addition +to** the app-wide chain, never instead of it. A withdrawal is charged to the +global budget, the tiered budget and the sensitive budget — the sensitive one +is the tightest and therefore the one that bites first, but the other two +still cap a client that spreads its traffic across many endpoints. + +The tiered tier is the only place where "who are you?" changes the answer. +An API key's own `rateLimitPerMin` overrides both the anonymous and the +authenticated allowance for that key. + +--- + +## Which auth type maps to which tier + +| Caller | How it is identified | Budget charged against | +| --- | --- | --- | +| **Session** | `req.userId`, set by `requireAuth` after the JWT is verified | The user id — one bucket per user, shared across that user's devices | +| **API key** | `nwk__` — self-describing in the `Authorization` header | The key id — two keys behind one NAT get two buckets, and the key's own `rateLimitPerMin` wins | +| **Admin key** | `X-Admin-Token` / `Authorization: Bearer` | The admin credential | +| **Webhook provider** | HMAC signature over the raw body (Twilio, Telegram, fiat) | The client IP — the signature is the credential, not a bearer token | +| **Internal service** | `X-Internal-Token` | The client IP — the internal token is a single shared credential | +| **Anonymous** | Nothing proven | The client IP, at the strict anonymous allowance | + +A request is only ever charged to an identity the auth middleware has +*verified*. A `Authorization` header alone never upgrades a caller from +anonymous: otherwise an attacker could mint unlimited buckets by sending +garbage bearer tokens. + +--- + +## The tier catalogue + +Every tier in the system, with its live window and burst. The numbers come +from `config.security`, so this table cannot drift from what the limiter +actually enforces. + + +| Tier | Window | Burst | Principal | Auth types | Purpose | +| --- | --- | --- | --- | --- | --- | +| `global` | 900000ms | 100 | principal | session, api_key, admin, webhook, internal | Baseline budget for every route. Catches runaway clients and gives the more specific tiers something to sit on top of. | +| `tiered` | 900000ms | 60 | principal | session, api_key, admin, internal | Identity tiering. Anonymous callers draw on a strict per-IP budget; callers that proved an identity draw on a generous per-principal one. An API key’s own allowance overrides this tier entirely. | +| `auth` | 900000ms | 20 | ip | anonymous, session | Credential endpoints. Deliberately IP-keyed: credential stuffing happens before any identity is proven, so a per-user key would be meaningless here. | +| `admin` | 900000ms | 10 | principal | admin, internal | The management API. Tightest of the general tiers — these routes move reserves, mint keys and settle referrals. | +| `erasure` | 900000ms | 5 | principal | admin, internal | Right-to-erasure. Irreversible, so it is held far below the general admin budget — a bug that loops here would delete data repeatedly. | +| `webhook` | 60000ms | 30 | ip | webhook | Inbound provider callbacks. Unauthenticated at the transport layer — the HMAC signature is the credential — so the budget resists spoofed and replayed floods. | +| `sensitive` | 900000ms | 10 | principal | session, api_key | Money movement and credential changes. Applied per-endpoint IN ADDITION to the route’s normal budget, so a burst of withdrawals is stopped without throttling the balance reads the user needs to decide. | +| `optimizer` | 60000ms | 5 | principal | session, api_key | The CPU-bound allocation suggester. Pairs with the in-flight semaphore in src/utils/concurrency.ts: that bounds how many run at once, this bounds how often they start. | +| `simulate` | 60000ms | 6 | principal | session, api_key | The CPU-bound historical replay. Tight enough that the replay cannot be used as a CPU-exhaustion vector; pairs with the short-TTL result cache so identical previews are served without recomputing. | +| `internal` | 60000ms | 500 | ip | internal | Service-to-service calls (the agent loop). Higher throughput than the user-facing tiers, and IP-keyed because the internal token is a single shared credential. | + + +### Moving a tier + +Each tier is controlled by a pair of environment variables (see +`src/config/env.ts`). The `tiered` tier has two pairs: the anonymous +allowance and the authenticated one. + +| Tier | Window | Burst | +| --- | --- | --- | +| `global` | `RATE_LIMIT_WINDOW_MS` | `RATE_LIMIT_MAX` | +| `tiered` (anonymous) | `ANONYMOUS_RATE_LIMIT_WINDOW_MS` | `ANONYMOUS_RATE_LIMIT_MAX` | +| `tiered` (authenticated) | `AUTHENTICATED_RATE_LIMIT_WINDOW_MS` | `AUTHENTICATED_RATE_LIMIT_MAX` | +| `auth` | `AUTH_RATE_LIMIT_WINDOW_MS` | `AUTH_RATE_LIMIT_MAX` | +| `admin` | `ADMIN_RATE_LIMIT_WINDOW_MS` | `ADMIN_RATE_LIMIT_MAX` | +| `erasure` | fixed (15 min) | fixed (5) | +| `webhook` | `WEBHOOK_RATE_LIMIT_WINDOW_MS` | `WEBHOOK_RATE_LIMIT_MAX` | +| `sensitive` | `SENSITIVE_RATE_LIMIT_WINDOW_MS` | `SENSITIVE_RATE_LIMIT_MAX` | +| `optimizer` | `OPTIMIZER_RATE_LIMIT_WINDOW_MS` | `OPTIMIZER_RATE_LIMIT_MAX` | +| `simulate` | `SIMULATE_RATE_LIMIT_WINDOW_MS` | `SIMULATE_RATE_LIMIT_MAX` | +| `internal` | `INTERNAL_RATE_LIMIT_WINDOW_MS` | `INTERNAL_RATE_LIMIT_MAX` | + +--- + +## The route → tier matrix + +Paths are the canonical `/api/v1/*` ones. Every route is also served at the +legacy `/api/*` alias with the same handlers, and therefore the same tier. + + +| Route | Method | Tier | Window | Burst | Why | +| --- | --- | --- | --- | --- | --- | +| `/api/v1/auth` | ANY | `auth` | 900000ms | 20 | Login, refresh, logout — credential stuffing surface. | +| `/api/v1/auth/webauthn` | ANY | `auth` | 900000ms | 20 | WebAuthn ceremonies — same credential surface as password auth. | +| `/api/v1/webauthn` | ANY | `auth` | 900000ms | 20 | Top-level WebAuthn alias, same ceremony budget. | +| `/api/v1/whatsapp` | ANY | `webhook` | 60000ms | 30 | Inbound WhatsApp provider callbacks, HMAC-signed. | +| `/api/v1/telegram` | ANY | `webhook` | 60000ms | 30 | Inbound Telegram provider callbacks, HMAC-signed. | +| `/api/v1/admin` | ANY | `admin` | 900000ms | 10 | Management API — reserves, key minting, referral settlement. | +| `/api/v1/admin/support/tickets` | ANY | `admin` | 900000ms | 10 | Admin view of support tickets. | +| `/api/v1/admin/notifications/dlq` | ANY | `admin` | 900000ms | 10 | Dead-letter queue inspection and replay. | +| `/api/v1/admin/erasure/requests` | POST | `erasure` | 900000ms | 5 | Creating an erasure request is a destructive operation. | +| `/api/v1/admin/erasure/requests/:id/approve` | POST | `erasure` | 900000ms | 5 | Approving an erasure request authorises data deletion. | +| `/api/v1/admin/erasure/requests/:id/reject` | POST | `erasure` | 900000ms | 5 | Rejecting an erasure request is part of the erasure workflow. | +| `/api/v1/admin/erasure/requests/:id/execute` | POST | `erasure` | 900000ms | 5 | Executing an erasure request deletes data — the sharpest edge. | +| `/api/v1/admin/erasure/preview/:userId` | GET | `erasure` | 900000ms | 5 | Previewing an erasure walks the same deletion path as the execute. | +| `/api/v1/deposit` | POST | `sensitive` | 900000ms | 10 | Deposits move money — irreversible once submitted. | +| `/api/v1/withdraw` | POST | `sensitive` | 900000ms | 10 | Withdrawals move money — irreversible once submitted. | +| `/api/v1/loans` | POST | `sensitive` | 900000ms | 10 | Loan origination moves money. | +| `/api/v1/loans/:loanId/repay` | POST | `sensitive` | 900000ms | 10 | Loan repayment moves money. | +| `/api/v1/keys` | POST | `sensitive` | 900000ms | 10 | Minting an API key is a takeover vector — credential change. | +| `/api/v1/keys/:id/rotate` | POST | `sensitive` | 900000ms | 10 | Rotating a key is a credential change. | +| `/api/v1/support/tickets` | POST | `sensitive` | 900000ms | 10 | Opening a ticket is a state change on a customer-visible record. | +| `/api/v1/portfolio/:userId/suggest-allocation` | POST | `optimizer` | 60000ms | 5 | CPU-bound allocation solver — the classic exhaustion vector. | +| `/api/v1/strategies/simulate` | POST | `simulate` | 60000ms | 6 | CPU-bound historical replay. | +| `/api/v1/agent/decisions` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Agent decision history is a user-facing read (session auth), not a service-to-service call — only the agent status endpoints are. | +| `/api/v1/agent` | ANY | `internal` | 60000ms | 500 | Agent loop endpoints — service-to-service, guarded by the internal token. | +| `/api/v1` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Default for every remaining versioned API route: the global budget plus the identity tier (strict per-IP when anonymous, generous per-principal once authenticated). | +| `/api/webhooks` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Pre-version webhook management surface; same baseline as /api/v1. | +| `/api/analytics` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Pre-version analytics alias; same baseline as /api/v1. | +| `/api/stellar` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Pre-version Stellar alias; same baseline as /api/v1. | +| `/health` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Liveness and readiness probes. The GLOBAL limiter skips these so k8s probes never consume the budget; the tiered limiter still applies. | +| `/metrics` | ANY | `tiered` | 900000ms / 600 authenticated | 60 | Prometheus scrape endpoint, guarded by the internal auth guard. | + + +--- + +## Fail-open vs fail-closed + +**Runtime: fail-open.** The app-wide chain (`global` + `tiered`) is mounted +before routing, so a route that is missing from the matrix still gets a +budget. No route is ever unlimited, and the "fail-open" label does not mean +"let the request through unchecked" — it means "degrade to the generic +budget instead of to none". + +**CI: fail-closed.** The matrix is the contract, and the contract is +enforced. `tests/unit/middleware/rateLimitTierMatrix.test.ts` fails the +build when: + +- a mounted route resolves to **no** matrix entry (an undocumented budget); +- the matrix says a route is in one tier while the code mounts a limiter + from **another** (the table is lying); +- a matrix entry **no longer matches** any real route (the table has + rotted); +- a limiter exists that the tier catalogue does **not** describe; +- `docs/RATE_LIMITING.md` is **not** the table this module generates. + +The rationale: the runtime behaviour is a safety net for the case where +someone forgot; CI is where forgetting is caught. A gap in the matrix is +exactly the kind of silent drift this catalogue exists to prevent, and a +safety net that nobody notices is not a safety net. + +--- + +## Response headers + +Every limited response carries the IETF draft headers: + +| Header | When | Meaning | +| --- | --- | --- | +| `RateLimit-Limit` | every response | The quota that was applied to *this* request | +| `RateLimit-Remaining` | every response | Requests left in the current window | +| `RateLimit-Reset` | every response | Seconds until the window resets | +| `RateLimit-Policy` | every response | `;w=` — the effective policy | +| `Retry-After` | 429 only | Seconds until the caller may retry | + +`RateLimit-Policy` reports the limit **actually applied**, not the tier +default. An API key with `rateLimitPerMin: 5` is told it has 5, so a client +that reads the header to schedule its retries does not keep hammering a +5/min endpoint believing it has 100. + +--- + +## Adding a route + +1. Decide the tier. Most routes take the default (`tiered`). A route that + moves money or changes a credential takes `sensitive`. A CPU-bound route + takes `optimizer` or `simulate`. +2. Add the entry to `RATE_LIMIT_MATRIX` in `src/middleware/rateLimitTiers.ts`. + Use an **exact** path for a single endpoint; set `prefix: true` only when + the entry is meant to cover a whole router. +3. If the route needs its own limiter, build it with `buildRateLimiter({ …, + tier: 'sensitive' })` and mount it on the route. The tier tag is what + makes the matrix checkable. +4. Run `npm test -- rateLimitTierMatrix`. It will tell you if the matrix and + the code disagree. diff --git a/package.json b/package.json index 323bb009..66bc507b 100644 --- a/package.json +++ b/package.json @@ -30,6 +30,8 @@ "outdated:deps": "npm outdated || true", "validate:spec": "npx --yes @redocly/cli@latest lint docs/openapi.yaml --format=stylish", "check:openapi-drift": "node scripts/check-openapi-drift.js", + "docs:rate-limits": "ts-node scripts/rate-limit-doc-tables.ts", + "docs:scopes": "ts-node scripts/scope-doc-tables.ts", "build:spec": "mkdir -p dist/docs && cp docs/openapi.yaml dist/docs/openapi.yaml", "prebuild": "npm run clean && npm run build:spec", "prestart": "npm run build", diff --git a/scripts/rate-limit-doc-tables.ts b/scripts/rate-limit-doc-tables.ts new file mode 100644 index 00000000..b503e53f --- /dev/null +++ b/scripts/rate-limit-doc-tables.ts @@ -0,0 +1,67 @@ +/** + * Regenerate the generated tables in docs/RATE_LIMITING.md (#614). + * + * The prose in that document is hand-written; the two tables are generated + * from src/middleware/rateLimitTiers.ts. This script rewrites the blocks + * between the GENERATED markers, so a contributor who changes a tier or adds + * a route can refresh the doc with one command: + * + * npm run docs:rate-limits + * + * tests/unit/middleware/rateLimitTierMatrix.test.ts asserts the committed + * doc still matches what this script would write, so a stale doc fails CI. + */ + +import fs from 'node:fs' +import path from 'node:path' +import dotenv from 'dotenv' + +// The env must be loaded before src/config/env.ts validates it, and a static +// import of the catalogue would be hoisted above this line — hence the +// dynamic import below. +dotenv.config({ + path: path.join(__dirname, '..', '.env.test'), + override: true, +}) + +const DOC_PATH = path.join(__dirname, '..', 'docs', 'RATE_LIMITING.md') + +const BLOCKS = [ + { marker: 'matrix', render: 'renderRateLimitMatrixMarkdown' }, + { marker: 'tier-catalog', render: 'renderRateLimitTierCatalogMarkdown' }, +] as const + +async function main(): Promise { + const catalogue = await import('../src/middleware/rateLimitTiers') + + let doc = fs.readFileSync(DOC_PATH, 'utf8') + + for (const block of BLOCKS) { + const begin = `` + const end = `` + const start = doc.indexOf(begin) + const stop = doc.indexOf(end) + + if (start === -1 || stop === -1) { + throw new Error( + `docs/RATE_LIMITING.md is missing the "${block.marker}" generated block` + ) + } + + const table = catalogue[block.render]() + doc = + doc.slice(0, start + begin.length) + + '\n' + + table + + '\n' + + doc.slice(stop) + } + + fs.writeFileSync(DOC_PATH, doc) + console.log(`Updated ${path.relative(process.cwd(), DOC_PATH)}`) +} + +main().catch((err: unknown) => { + console.error(err instanceof Error ? err.message : String(err)) + process.exit(1) +}) diff --git a/scripts/scope-doc-tables.ts b/scripts/scope-doc-tables.ts new file mode 100644 index 00000000..001dd59d --- /dev/null +++ b/scripts/scope-doc-tables.ts @@ -0,0 +1,77 @@ +/** + * Regenerate the generated scope tables in docs/API_KEYS.md (#615). + * + * The prose in that document is hand-written; the two scope tables are + * generated from the catalogues in src/auth/scopes.ts and + * src/middleware/adminAuth.ts. This script rewrites the blocks between the + * GENERATED markers: + * + * npm run docs:scopes + * + * tests/unit/auth/scopeCatalog.test.ts asserts the committed doc still + * matches what this script would write, so a stale doc fails CI. + */ + +import fs from 'node:fs' +import path from 'node:path' +import dotenv from 'dotenv' + +// The env must be loaded before src/config/env.ts validates it, and a static +// import of the catalogue would be hoisted above this line — hence the +// dynamic import below. +dotenv.config({ + path: path.join(__dirname, '..', '.env.test'), + override: true, +}) + +const DOC_PATH = path.join(__dirname, '..', 'docs', 'API_KEYS.md') + +async function main(): Promise { + const { getAdminScopeCatalog, getUserScopeCatalog } = await import( + '../src/auth/scopeCoverage' + ) + const { renderScopeCatalogMarkdown } = await import( + '../src/auth/scopeCatalog' + ) + + const blocks = [ + { + marker: 'user-scope-catalog', + table: renderScopeCatalogMarkdown(getUserScopeCatalog()), + }, + { + marker: 'admin-scope-catalog', + table: renderScopeCatalogMarkdown(getAdminScopeCatalog()), + }, + ] + + let doc = fs.readFileSync(DOC_PATH, 'utf8') + + for (const block of blocks) { + const begin = `` + const end = `` + const start = doc.indexOf(begin) + const stop = doc.indexOf(end) + + if (start === -1 || stop === -1) { + throw new Error( + `docs/API_KEYS.md is missing the "${block.marker}" generated block` + ) + } + + doc = + doc.slice(0, start + begin.length) + + '\n' + + block.table + + '\n' + + doc.slice(stop) + } + + fs.writeFileSync(DOC_PATH, doc) + console.log(`Updated ${path.relative(process.cwd(), DOC_PATH)}`) +} + +main().catch((err: unknown) => { + console.error(err instanceof Error ? err.message : String(err)) + process.exit(1) +}) diff --git a/src/auth/scopeCatalog.ts b/src/auth/scopeCatalog.ts new file mode 100644 index 00000000..7215688b --- /dev/null +++ b/src/auth/scopeCatalog.ts @@ -0,0 +1,129 @@ +/** + * The shared shape of a scope catalogue entry (#615). + * + * A scope is a permission string stored on a credential. Two families exist — + * user API key scopes (src/auth/scopes.ts) and admin key scopes + * (src/middleware/adminAuth.ts) — and until this module existed both were + * bare string arrays: the list of valid scopes lived in code, the meaning of + * each scope lived in a comment or a doc, and nothing connected the two. A + * key-creation UI could not show an operator what a scope actually unlocks, + * and a typo in a `requireScope('...')` call site was a runtime 403 rather + * than a build failure. + * + * This module defines the entry shape both catalogues use, and the renderer + * that turns a catalogue into the tables in docs/API_KEYS.md. + */ + +/** + * How much a holder of a scope can do. + * + * read — reads state and nothing else. + * write — mutates state that belongs to the user. + * money — moves value. Needs a second guardrail (an opt-in flag, a + * platform kill-switch) as well as the scope itself. + * destructive — deletes state, irreversibly. + * operational — acts on platform-wide state rather than one user's. + */ +export type ScopeRisk = + 'read' | 'write' | 'money' | 'destructive' | 'operational' + +/** + * Whether a scope actually gates anything. + * + * The catalogue has to say this, because a scope that gates nothing is a + * promise the platform does not keep — an operator reading the catalogue + * would reasonably believe granting it restricts something. + * + * enforced — at least one route requires this scope. Granting it is + * meaningful and omitting it is restrictive. + * declared-only — declared and reservable, but no route requires it today. + * Granting it currently grants nothing extra; it exists so + * a key can be provisioned ahead of the route that will + * require it. + * + * Whether a scope ALSO grants other scopes is a separate question, answered + * by `implies` — `super` both gates routes and implies every other scope, so + * the two concerns cannot share one field. + */ +export type ScopeEnforcement = 'enforced' | 'declared-only' + +/** The risk levels, ordered least to most powerful — for sorting a UI list. */ +export const SCOPE_RISK_ORDER: readonly ScopeRisk[] = [ + 'read', + 'write', + 'money', + 'destructive', + 'operational', +] as const + +export interface ScopeDefinition { + /** The scope string exactly as it is stored on the credential. */ + scope: string + /** One line, written for the key-creation screen. */ + description: string + /** How much damage a holder of this scope can do. */ + risk: ScopeRisk + /** What the scope unlocks, in an operator's words. */ + grants: string + /** Whether a route requires this scope. See ScopeEnforcement. */ + enforcement: ScopeEnforcement + /** + * Other scopes this one implicitly grants. `super` implies every scope; + * the legacy `agent` implies `agent:read` and `agent:write`. + */ + implies?: readonly string[] + /** + * Any guardrail that applies on top of the scope itself — an opt-in flag, + * a platform kill-switch, a dual-control approval. Present only when the + * scope is not sufficient on its own. + */ + requires?: string +} + +/** + * A catalogue entry plus the endpoints it actually covers. + * + * The endpoint list is derived from the route table rather than hand-written + * (src/auth/scopeCoverage.ts), so it cannot drift: if a route stops requiring + * a scope, the scope stops claiming to cover it. + */ +export interface ScopeCatalogEntry extends ScopeDefinition { + /** `METHOD /path` for every route that requires this scope. */ + endpoints: string[] +} + +/** Sort a catalogue by risk, then alphabetically — the order a UI wants. */ +export function sortScopeEntries(entries: T[]): T[] { + return [...entries].sort((a, b) => { + const byRisk = + SCOPE_RISK_ORDER.indexOf(a.risk) - SCOPE_RISK_ORDER.indexOf(b.risk) + if (byRisk !== 0) return byRisk + return a.scope.localeCompare(b.scope) + }) +} + +/** Render a catalogue as a Markdown table — the table in docs/API_KEYS.md. */ +export function renderScopeCatalogMarkdown( + entries: ScopeCatalogEntry[], + options: { showEndpoints?: boolean } = {} +): string { + const showEndpoints = options.showEndpoints ?? true + const header = showEndpoints + ? '| Scope | Risk | Enforcement | Implies | What it does | Guardrail | Endpoints |' + : '| Scope | Risk | Enforcement | Implies | What it does | Guardrail |' + const divider = showEndpoints + ? '| --- | --- | --- | --- | --- | --- | --- |' + : '| --- | --- | --- | --- | --- | --- |' + const rows = sortScopeEntries(entries).map((entry) => { + const implies = entry.implies?.length + ? entry.implies.map((s) => `\`${s}\``).join(', ') + : '—' + const base = `| \`${entry.scope}\` | ${entry.risk} | ${entry.enforcement} | ${implies} | ${entry.description} | ${entry.requires ?? '—'} |` + if (!showEndpoints) return base + const endpoints = entry.endpoints.length + ? entry.endpoints.map((e) => `\`${e}\``).join(', ') + : '—' + return `${base} ${endpoints} |` + }) + return [header, divider, ...rows].join('\n') +} diff --git a/src/auth/scopeCoverage.ts b/src/auth/scopeCoverage.ts new file mode 100644 index 00000000..54917502 --- /dev/null +++ b/src/auth/scopeCoverage.ts @@ -0,0 +1,149 @@ +/** + * Which endpoints each scope actually covers (#615). + * + * The scope catalogues (src/auth/scopes.ts, src/middleware/adminAuth.ts) + * describe what a scope MEANS. This module answers the other half: which + * endpoints it unlocks. The answer is derived from the real route table by + * reading the scope tag off each route's middleware chain, so it cannot + * drift — a route that stops requiring a scope stops appearing under it. + * + * The route table lives in src/routes/registry.ts, which imports the route + * files — including the admin route that serves this catalogue. Requiring it + * lazily inside the function (rather than at module load) breaks that cycle: + * by the time the coverage is first computed, the whole module graph is + * loaded. + */ + +import { guardScopes } from '../middleware/apiKeyAuth' +import { adminGuardScope } from '../middleware/adminAuth' +import type { ScopeCatalogEntry } from './scopeCatalog' +import { USER_SCOPE_CATALOG, type UserScope } from './scopes' +import { ADMIN_SCOPE_CATALOG, type AdminScope } from '../middleware/adminAuth' + +export interface ScopeCoverage { + /** `METHOD /path` per scope, for both scope families. */ + user: Record + admin: Record +} + +let cached: ScopeCoverage | null = null + +/** Read the scope tags off a route's middleware chain. */ +function scopesOfRoute(routeStack: any[]): { + user: UserScope[] + admin: AdminScope[] +} { + const user: UserScope[] = [] + const admin: AdminScope[] = [] + + for (const layer of routeStack) { + const handler = layer?.handle + const required = guardScopes(handler) + if (required) user.push(...required) + const adminScope = adminGuardScope(handler) + if (adminScope) admin.push(adminScope) + } + + return { user, admin } +} + +/** + * Walk the route table and collect, per scope, the endpoints it covers. + * Memoised — the route table does not change at runtime. + */ +export function getScopeCoverage(): ScopeCoverage { + if (cached) return cached + + // Required lazily: src/routes/registry.ts imports the admin route, which + // serves this catalogue. See the module comment. + // eslint-disable-next-line @typescript-eslint/no-require-imports + const { API_ROUTES, EXTRA_MOUNTS } = require('../routes/registry') as { + API_ROUTES: { path: string; handlers: any[] }[] + EXTRA_MOUNTS: { path: string; handlers: any[] }[] + } + + const user: Record = {} + const admin: Record = {} + + const walk = ( + stack: any[], + prefix: string, + onRoute: (method: string, path: string, routeStack: any[]) => void + ) => { + for (const layer of stack) { + if (layer.route) { + for (const method of Object.keys(layer.route.methods || {})) { + const path = (prefix + (layer.route.path || '')).replace(/\/$/, '') + onRoute(method.toUpperCase(), path || '/', layer.route.stack || []) + } + } else if (layer.name === 'router' && layer.handle?.stack) { + const nested = layer.regexp.source + .replace(/^\^/, '') + .replace(/\\\/\?\(\?=\\\/\|\$\)$/, '') + .replace(/\\\//g, '/') + .replace(/\(\?:[^)]*\)/g, ':p') + .replace(/\([^)]*\)/g, ':p') + walk( + layer.handle.stack, + prefix + (nested === '/' ? '' : nested), + onRoute + ) + } + } + } + + const collect = ( + mounts: { path: string; handlers: any[] }[], + prefix: string + ) => { + for (const mount of mounts) { + const router = mount.handlers[mount.handlers.length - 1] + walk( + router?.stack ?? [], + prefix + mount.path, + (method, path, routeStack) => { + const { user: userScopes, admin: adminScopes } = + scopesOfRoute(routeStack) + for (const scope of userScopes) { + ;(user[scope] ??= []).push(`${method} ${path}`) + } + for (const scope of adminScopes) { + ;(admin[scope] ??= []).push(`${method} ${path}`) + } + } + ) + } + } + + collect(API_ROUTES, '/api/v1/') + collect(EXTRA_MOUNTS, '') + + for (const endpoints of Object.values(user)) endpoints.sort() + for (const endpoints of Object.values(admin)) endpoints.sort() + + cached = { user, admin } + return cached +} + +/** The user scope catalogue, with the endpoints each scope covers. */ +export function getUserScopeCatalog(): ScopeCatalogEntry[] { + const coverage = getScopeCoverage() + return Object.values(USER_SCOPE_CATALOG).map((definition) => ({ + ...definition, + endpoints: coverage.user[definition.scope] ?? [], + })) +} + +/** The admin scope catalogue, with the endpoints each scope covers. */ +export function getAdminScopeCatalog(): ScopeCatalogEntry[] { + const coverage = getScopeCoverage() + return Object.values(ADMIN_SCOPE_CATALOG).map((definition) => ({ + ...definition, + endpoints: coverage.admin[definition.scope] ?? [], + })) +} + +/** Test seam: drop the memoised coverage so a re-imported table is re-walked. */ +export function resetScopeCoverageCache(): void { + cached = null +} diff --git a/src/auth/scopes.ts b/src/auth/scopes.ts index d1b4fb4e..9c326d82 100644 --- a/src/auth/scopes.ts +++ b/src/auth/scopes.ts @@ -1,10 +1,19 @@ /** - * User API key scope catalog (#374). + * User API key scope catalog (#374, #615). * * Sessions implicitly grant `*` (all scopes). API keys are limited to the * scopes explicitly assigned at creation time. + * + * The catalogue below is the machine-readable form of that rule set: it says + * what each scope means, how risky it is, and what else has to be true before + * it can be used. `GET /api/admin/keys/scopes` serves it for the + * key-creation UI, docs/API_KEYS.md renders it, and + * tests/unit/auth/scopeCatalog.test.ts fails CI if a `requireScope('...')` + * call site names a scope that is not in here. */ +import type { ScopeDefinition } from './scopeCatalog' + export const USER_SCOPES = [ 'portfolio:read', 'portfolio:write', @@ -30,6 +39,137 @@ export const USER_SCOPES = [ export type UserScope = (typeof USER_SCOPES)[number] +/** + * What each user scope means. + * + * Every scope in USER_SCOPES must have an entry here — the catalogue is + * typed as a total map, so a new scope cannot be added without describing + * it. + */ +export const USER_SCOPE_CATALOG: Record = { + 'portfolio:read': { + scope: 'portfolio:read', + enforcement: 'enforced', + description: + 'Read portfolio positions, balances, history, earnings and tax reports.', + risk: 'read', + grants: + 'The user’s portfolio, allocation suggestions, earnings and tax report.', + }, + 'portfolio:write': { + scope: 'portfolio:write', + enforcement: 'enforced', + description: 'Change portfolio state.', + risk: 'write', + grants: 'Mutations of the user’s own portfolio.', + }, + 'transactions:read': { + scope: 'transactions:read', + enforcement: 'enforced', + description: 'Read transaction history.', + risk: 'read', + grants: 'Transaction list and per-transaction events.', + }, + 'deposit:write': { + scope: 'deposit:write', + enforcement: 'enforced', + description: 'Create deposits.', + risk: 'money', + grants: 'Submits deposits to the durable outbox for on-chain settlement.', + requires: + 'Deposits still pass the compliance freeze and sub-account permission checks.', + }, + 'withdraw:write': { + scope: 'withdraw:write', + enforcement: 'enforced', + description: 'Create withdrawals.', + risk: 'money', + grants: + 'Submits withdrawals to the durable outbox for on-chain settlement.', + requires: + 'Requires allowWithdrawals on the key AND the platform kill-switch ' + + 'USER_API_KEY_WITHDRAWALS_ENABLED to be on.', + }, + 'alerts:manage': { + scope: 'alerts:manage', + enforcement: 'enforced', + description: 'Manage alert rules.', + risk: 'write', + grants: 'Create, update and delete the user’s alert rules.', + }, + 'fiat:write': { + scope: 'fiat:write', + enforcement: 'enforced', + description: 'Create fiat on/off-ramp orders.', + risk: 'money', + grants: 'Submits fiat orders to the configured provider.', + }, + 'recurring_deposits:write': { + scope: 'recurring_deposits:write', + enforcement: 'enforced', + description: 'Manage recurring deposit plans.', + risk: 'money', + grants: 'Schedules and cancels recurring deposits.', + }, + 'goals:write': { + scope: 'goals:write', + enforcement: 'enforced', + description: 'Manage savings goals.', + risk: 'write', + grants: 'Create, update and delete savings goals.', + }, + 'strategies:write': { + scope: 'strategies:write', + enforcement: 'enforced', + description: 'Publish, follow and unfollow strategies.', + risk: 'write', + grants: 'Publishes the user’s own strategy and manages follows.', + }, + 'webhooks:manage': { + scope: 'webhooks:manage', + enforcement: 'enforced', + description: 'Manage webhook endpoints and deliveries.', + risk: 'write', + grants: + 'Registers endpoints and reads delivery logs. Endpoint secrets are ' + + 'write-only — a response body never carries one back.', + }, + 'vault:read': { + scope: 'vault:read', + enforcement: 'enforced', + description: 'Read vault state and balance.', + risk: 'read', + grants: 'The user’s vault state and balance.', + }, + 'vault:write': { + scope: 'vault:write', + enforcement: 'enforced', + description: 'Build vault transactions.', + risk: 'money', + grants: 'Builds (but does not sign) vault operations.', + }, + 'loans:read': { + scope: 'loans:read', + enforcement: 'enforced', + description: 'Read loan state and borrowing capacity.', + risk: 'read', + grants: 'Loan list, per-loan detail and available capacity.', + }, + 'loans:write': { + scope: 'loans:write', + enforcement: 'enforced', + description: 'Originate and repay collateral loans.', + risk: 'money', + grants: + 'Pledges collateral and borrows, or repays an existing loan. ' + + 'Deliberately separate from withdraw:write so a key trusted to pull ' + + 'funds out cannot thereby pledge the user’s collateral.', + requires: + 'Origination and repayment still pass the compliance freeze and ' + + 'sub-account permission checks.', + }, +} + export function validateUserScopes(scopes: unknown): scopes is UserScope[] { if (!Array.isArray(scopes) || scopes.length === 0) return false return scopes.every((s) => USER_SCOPES.includes(s as UserScope)) diff --git a/src/index.ts b/src/index.ts index 96d82fb8..e3212c7b 100644 --- a/src/index.ts +++ b/src/index.ts @@ -32,14 +32,13 @@ import { import { requestLogger } from './middleware/logger' import { requestTimeoutMiddleware } from './middleware/requestTimeout' import { - rateLimiter, - tieredRateLimiter, authRateLimiter, adminRateLimiter, internalRateLimiter, webhookRateLimiter, trustedIpBypass, } from './middleware/rateLimiter' +import { APP_LEVEL_LIMITERS } from './middleware/rateLimiterChain' import { configureTrustProxy, securityHeaders, @@ -74,47 +73,7 @@ import { startEventListener, stopEventListener } from './stellar/events' import { startEventBridge, stopEventBridge } from './events/bridge' import { attachWebSocketServer, closeWebSocketServer } from './ws/server' import { validateStellarNetworkReady } from './config/readiness' -import healthRouter from './routes/health' -import agentRouter from './routes/agent' -import agentDecisionsRouter from './routes/agent-decisions' -import authRouter from './routes/auth' -import whatsappRouter from './routes/whatsapp' -import telegramRouter from './routes/telegram' -import portfolioRouter from './routes/portfolio' -import transactionsRouter from './routes/transactions' -import protocolsRouter from './routes/protocols' -import depositRouter from './routes/deposit' -import withdrawRouter from './routes/withdraw' -import loansRouter from './routes/loans' -import protectionFundRouter from './routes/protection-fund' -import settingsRouter from './routes/settings' -import vaultRouter from './routes/vault' -import analyticsRouter from './routes/analytics' -import adminRouter from './routes/admin' -import metricsRouter from './routes/metrics' -import stellarRouter from './routes/stellar' -import webhooksRouter from './routes/webhooks' -import fiatRouter from './routes/fiat' -import referralsRouter from './routes/referrals' -import recurringDepositRouter from './routes/recurring-deposits' -import recurringWithdrawalRouter from './routes/recurring-withdrawals' -import alertsRouter from './routes/alerts' -import strategiesRouter from './routes/strategies' -import subAccountsRouter from './routes/sub-accounts' -import supportTicketsRouter from './routes/support-tickets' -import adminSupportTicketsRouter from './routes/admin/support-tickets' -import assistantRouter from './routes/assistant' -import approvalsRouter from './routes/approvals' -import approvalPoliciesRouter from './routes/approval-policies' -import keysRouter from './routes/keys' -import sessionsRouter from './routes/sessions' -import webAuthnRouter from './routes/webauthn' -import streamRouter from './routes/stream' -import notificationsRouter from './routes/notifications' -import liquidityFloorRouter from './routes/liquidity-floor' -import notificationDlqRouter from './routes/notification-dlq' -import networkRouter from './routes/network' -import netWorthRouter from './routes/net-worth' +import { API_ROUTES, EXTRA_MOUNTS } from './routes/registry' import { corsMiddleware, jsonBodyParser, @@ -235,14 +194,11 @@ app.use((req: Request & { user?: { id: string } }, _res: Response, next) => { app.use(requestLogger) app.use(trustedIpBypass) -app.use(rateLimiter) -// #473 — identity tiering. This single limiter charges anonymous callers to a -// strict per-IP budget and authenticated callers to a generous per-principal one -// (see tieredRateLimiter). It resolves identity from the request itself, so it -// works here, ahead of each route's auth middleware: API keys are self- -// describing in the Authorization header, and session tokens are classified once -// the auth middleware has resolved req.userId. -app.use(tieredRateLimiter) +// The app-wide rate-limit chain: the flat global budget, then the identity +// tier. Both apply to every request, so no route is ever unlimited. Kept in +// src/middleware/rateLimiterChain.ts so the tier-matrix CI check can assert +// that claim against the real chain. +app.use(...APP_LEVEL_LIMITERS) app.use(requestTimeoutMiddleware) // Advertise the served API version on every response — must be registered @@ -351,66 +307,16 @@ interface ApiRoute { handlers: express.RequestHandler[] } -const apiRoutes: ApiRoute[] = [ - { path: 'network', handlers: [networkRouter] }, - { path: 'net-worth', handlers: [netWorthRouter] }, - { path: 'agent/decisions', handlers: [agentDecisionsRouter] }, - { path: 'agent', handlers: [internalRateLimiter, agentRouter] }, - { path: 'auth/webauthn', handlers: [authRateLimiter, webAuthnRouter] }, - { path: 'auth', handlers: [authRateLimiter, authRouter] }, - { path: 'webauthn', handlers: [authRateLimiter, webAuthnRouter] }, - { path: 'whatsapp', handlers: [webhookRateLimiter, whatsappRouter] }, - { path: 'telegram', handlers: [webhookRateLimiter, telegramRouter] }, - { path: 'portfolio', handlers: [portfolioRouter] }, - { path: 'transactions', handlers: [transactionsRouter] }, - { path: 'protocols', handlers: [protocolsRouter] }, - { path: 'deposit', handlers: [depositRouter] }, - { path: 'withdraw', handlers: [withdrawRouter] }, - // #532 — collateral loans. Mounted beside withdraw rather than inside - // portfolio: a loan is its own lifecycle with its own approval and rate - // limits, and it must not inherit portfolio's read-only scope set. - { path: 'loans', handlers: [loansRouter] }, - // #533 — protocol-risk protection fund - { path: 'protection-fund', handlers: [protectionFundRouter] }, - // #534 — user settings (display currency) - { path: 'settings', handlers: [settingsRouter] }, - { path: 'vault', handlers: [vaultRouter] }, - { path: 'analytics', handlers: [analyticsRouter] }, - { path: 'stellar', handlers: [stellarRouter] }, - { path: 'fiat', handlers: [fiatRouter] }, - { path: 'referrals', handlers: [referralsRouter] }, - { path: 'deposit/recurring', handlers: [recurringDepositRouter] }, - { path: 'recurring-withdrawals', handlers: [recurringWithdrawalRouter] }, - { path: 'alerts', handlers: [alertsRouter] }, - { path: 'strategies', handlers: [strategiesRouter] }, - { path: 'sub-accounts', handlers: [subAccountsRouter] }, - { path: 'support/tickets', handlers: [supportTicketsRouter] }, - { - path: 'admin/support/tickets', - handlers: [adminRateLimiter, adminSupportTicketsRouter], - }, - { path: 'assistant', handlers: [assistantRouter] }, - { path: 'approvals', handlers: [approvalsRouter] }, - { path: 'approval-policies', handlers: [approvalPoliciesRouter] }, - { path: 'keys', handlers: [keysRouter] }, - { path: 'sessions', handlers: [sessionsRouter] }, - { path: 'stream', handlers: [streamRouter] }, - { path: 'notifications', handlers: [notificationsRouter] }, - { path: 'liquidity-floor', handlers: [liquidityFloorRouter] }, - { - path: 'admin/notifications/dlq', - handlers: [adminRateLimiter, notificationDlqRouter], - }, - { path: 'admin', handlers: [adminRateLimiter, adminRouter] }, -] +const apiRoutes: ApiRoute[] = API_ROUTES // ── Application routes ──────────────────────────────────────────────────────── -app.use('/health', healthRouter) -app.use('/api/analytics', analyticsRouter) -app.use('/api/stellar', stellarRouter) -app.use('/api/webhooks', webhooksRouter) -app.use('/metrics', metricsRouter) +// Operational endpoints and the pre-version paths (health, metrics, the legacy +// webhooks surface). Declared in src/routes/registry.ts so the tier matrix +// can describe every route the server serves, not just the versioned ones. +for (const mount of EXTRA_MOUNTS) { + app.use(mount.path, ...mount.handlers) +} // Primary, versioned mounts: /api/v1/* for (const route of apiRoutes) { diff --git a/src/middleware/adminAuth.ts b/src/middleware/adminAuth.ts index bca80773..ccda95f0 100644 --- a/src/middleware/adminAuth.ts +++ b/src/middleware/adminAuth.ts @@ -6,6 +6,7 @@ import crypto from 'node:crypto' import db from '../db' import { logger } from '../utils/logger' import { recordAuthFailure } from '../utils/metrics' +import type { ScopeDefinition } from '../auth/scopeCatalog' const prisma = db @@ -64,6 +65,314 @@ export const ADMIN_SCOPES = [ ] as const export type AdminScope = (typeof ADMIN_SCOPES)[number] +/** + * The admin scopes that grant OTHER scopes rather than gating a route + * themselves (#615). + * + * super — implies every scope in ADMIN_SCOPES. + * agent — the legacy umbrella scope, implying agent:read and agent:write. + * + * These are the reason a scope can be "granted" without any route requiring + * it, so the scope catalogue has to list what each one implies. + */ +export const ADMIN_GRANTOR_SCOPES: Record< + 'super' | 'agent', + readonly AdminScope[] +> = { + super: ADMIN_SCOPES, + agent: ['agent:read', 'agent:write'], +} as const + +/** + * What each admin scope means (#615). + * + * Typed as a total map over ADMIN_SCOPES, so a new scope cannot be added + * without describing it — and the catalogue is what + * `GET /api/admin/keys/scopes` serves for the key-creation UI. + */ +export const ADMIN_SCOPE_CATALOG: Record = { + 'support:read': { + scope: 'support:read', + enforcement: 'enforced', + description: 'Read support tickets.', + risk: 'read', + grants: 'The support ticket queue and individual tickets.', + }, + 'support:write': { + scope: 'support:write', + enforcement: 'enforced', + description: 'Reply to and update support tickets.', + risk: 'write', + grants: 'Replies to and updates customer support tickets.', + }, + read: { + scope: 'read', + enforcement: 'enforced', + description: 'General read access to admin resources.', + risk: 'read', + grants: 'Read-only access across the admin resources the operator names.', + }, + write: { + scope: 'write', + enforcement: 'enforced', + description: 'General write access to admin resources.', + risk: 'write', + grants: 'Mutations across the admin resources the operator names.', + }, + wallet: { + scope: 'wallet', + enforcement: 'declared-only', + description: 'Custodial wallet operations.', + risk: 'money', + grants: 'Reads and rotates custodial wallet key material.', + requires: + 'No route requires this scope today — the wallet rotation-status ' + + 'route is gated by keys:read. Declared so a key can be provisioned ' + + 'ahead of a route that needs it.', + }, + agent: { + scope: 'agent', + enforcement: 'declared-only', + description: 'Agent loop operations.', + risk: 'operational', + grants: 'The legacy umbrella scope covering agent:read and agent:write.', + implies: ADMIN_GRANTOR_SCOPES.agent, + }, + 'agent:read': { + scope: 'agent:read', + enforcement: 'enforced', + description: 'Read agent decisions and circuit-breaker state.', + risk: 'read', + grants: 'Agent decision history and breaker state.', + }, + 'agent:write': { + scope: 'agent:write', + enforcement: 'enforced', + description: 'Change agent circuit breakers.', + risk: 'operational', + grants: 'Resets and reconfigures agent circuit breakers.', + }, + 'sessions:read': { + scope: 'sessions:read', + enforcement: 'enforced', + description: 'Read user sessions.', + risk: 'read', + grants: 'The session list for a user.', + }, + 'sessions:write': { + scope: 'sessions:write', + enforcement: 'enforced', + description: 'Revoke user sessions.', + risk: 'write', + grants: 'Revokes sessions, including revoking-all for a user.', + }, + 'reserves:read': { + scope: 'reserves:read', + enforcement: 'enforced', + description: 'Read reserve and sponsorship state.', + risk: 'read', + grants: 'Reserve balances and sponsorship ledger entries.', + }, + 'metrics:read': { + scope: 'metrics:read', + enforcement: 'enforced', + description: 'Read operational metrics and dashboards.', + risk: 'read', + grants: 'Prometheus metrics, job dashboards and audit-chain proofs.', + }, + 'dlq:read': { + scope: 'dlq:read', + enforcement: 'enforced', + description: 'Inspect the dead-letter queue.', + risk: 'read', + grants: 'Dead-letter queue contents and inspection reports.', + }, + 'dlq:write': { + scope: 'dlq:write', + enforcement: 'enforced', + description: 'Replay, retry and resolve dead-lettered work.', + risk: 'operational', + grants: 'Replays dead-lettered events and marks them resolved.', + }, + 'backfill:write': { + scope: 'backfill:write', + enforcement: 'enforced', + description: 'Run Stellar event backfills.', + risk: 'operational', + grants: 'Backfills historical Stellar events.', + }, + 'keys:read': { + scope: 'keys:read', + enforcement: 'enforced', + description: 'Read admin API keys and wallet rotation status.', + risk: 'read', + grants: 'The admin key list and wallet key-rotation progress.', + }, + 'keys:write': { + scope: 'keys:write', + enforcement: 'enforced', + description: 'Create, rotate and revoke admin API keys.', + risk: 'destructive', + grants: 'Mints new admin credentials and revokes existing ones.', + requires: + 'Minting an admin key is a takeover vector — every use is written to ' + + 'the admin audit log.', + }, + 'fiat:read': { + scope: 'fiat:read', + enforcement: 'enforced', + description: 'Read fiat providers and their health.', + risk: 'read', + grants: 'The configured fiat providers and their failover state.', + }, + 'fiat:write': { + scope: 'fiat:write', + enforcement: 'enforced', + description: 'Change fiat provider failover state.', + risk: 'operational', + grants: 'Moves a fiat provider in and out of failover.', + }, + 'outbox:read': { + scope: 'outbox:read', + enforcement: 'enforced', + description: 'Read the durable outbox.', + risk: 'read', + grants: 'The outbox queue, per-op detail and queue statistics.', + }, + 'outbox:write': { + scope: 'outbox:write', + enforcement: 'enforced', + description: 'Force-retry and cancel queued outbox operations.', + risk: 'money', + grants: + 'Returns a failed op to PENDING, or cancels an unsent one. A ' + + 'force-retry re-submits a real on-chain transaction.', + }, + 'approvals:write': { + scope: 'approvals:write', + enforcement: 'enforced', + description: 'Cancel pending approval requests.', + risk: 'write', + grants: 'Cancels a request that is awaiting approval.', + }, + 'referrals:read': { + scope: 'referrals:read', + enforcement: 'enforced', + description: 'Read flagged referral conversions.', + risk: 'read', + grants: 'The referral review queue.', + }, + 'referrals:write': { + scope: 'referrals:write', + enforcement: 'enforced', + description: 'Approve or reject flagged referral conversions.', + risk: 'money', + grants: + 'Releases or permanently withholds a referral payout. Deliberately ' + + 'narrower than a general write scope so a reviewer cannot reach ' + + 'unrelated state.', + }, + 'erasure:read': { + scope: 'erasure:read', + enforcement: 'declared-only', + description: 'Preview what an erasure would delete.', + risk: 'read', + grants: 'A dry run of the data an erasure request would remove.', + requires: + 'No route requires this scope today — the erasure preview route is ' + + 'gated by the generic read scope. Declared so a key can be ' + + 'provisioned ahead of a route that needs it.', + }, + 'erasure:write': { + scope: 'erasure:write', + enforcement: 'enforced', + description: 'Create, approve, reject and execute erasure requests.', + risk: 'destructive', + grants: 'Permanently deletes user data.', + requires: + 'Execution requires super — approval and execution are deliberately ' + + 'separated so one operator cannot both authorise and carry out a ' + + 'deletion.', + }, + 'messages:read': { + scope: 'messages:read', + enforcement: 'enforced', + description: 'Read message delivery state.', + risk: 'read', + grants: 'Telegram and WhatsApp delivery state and statistics.', + }, + 'messages:write': { + scope: 'messages:write', + enforcement: 'enforced', + description: 'Retry and recover failed message deliveries.', + risk: 'operational', + grants: 'Requeues failed Telegram and WhatsApp deliveries.', + }, + 'treasury:read': { + scope: 'treasury:read', + enforcement: 'enforced', + description: 'Read treasury sweep policies.', + risk: 'read', + grants: 'Sweep policies and the emergency-sweep state.', + }, + 'treasury:write': { + scope: 'treasury:write', + enforcement: 'enforced', + description: 'Change treasury sweep policies and run a sweep.', + risk: 'money', + grants: 'Moves reserves between treasury tiers on chain.', + requires: + 'Sweep policies and signer rotations are written to the admin audit log.', + }, + 'risk-metadata:read': { + scope: 'risk-metadata:read', + enforcement: 'declared-only', + description: 'Read protocol risk metadata.', + risk: 'read', + grants: 'The protocol risk metadata and its review state.', + requires: + 'No route requires this scope today — the risk-metadata review ' + + 'workflow is administered from src/config/riskMetadataAdmin.ts, not ' + + 'through the API. Declared so a key can be provisioned ahead of a ' + + 'route that needs it.', + }, + 'risk-metadata:write': { + scope: 'risk-metadata:write', + enforcement: 'declared-only', + description: 'Review and publish protocol risk metadata.', + risk: 'operational', + grants: 'Changes the risk metadata other users are shown.', + requires: + 'No route requires this scope today — the risk-metadata review ' + + 'workflow is administered from src/config/riskMetadataAdmin.ts, not ' + + 'through the API. Declared so a key can be provisioned ahead of a ' + + 'route that needs it.', + }, + 'admin:impersonate': { + scope: 'admin:impersonate', + enforcement: 'declared-only', + description: 'View the API as a user, read-only.', + risk: 'operational', + grants: 'A read-only "view as" session for support investigations.', + requires: + 'No route requires this scope today — the impersonation controller ' + + 'exists but is not yet exposed through the admin API. Every ' + + 'impersonated request is attributed to the operator in the admin ' + + 'audit log.', + }, + super: { + scope: 'super', + enforcement: 'enforced', + description: 'Every admin scope, including erasure execution.', + risk: 'destructive', + grants: 'All other admin scopes, implicitly.', + implies: ADMIN_GRANTOR_SCOPES.super, + requires: + 'Held by as few keys as possible: it is a superset of every ' + + 'destructive scope in this catalogue.', + }, +} + export interface AdminAuthContext { id: string name: string @@ -98,12 +407,19 @@ export function hasAdminScope( auth: AdminAuthContext, scope: AdminScope ): boolean { - return ( - auth.scopes.includes(scope) || - auth.scopes.includes('super') || - ((scope === 'agent:read' || scope === 'agent:write') && - auth.scopes.includes('agent')) - ) + if (auth.scopes.includes(scope)) return true + // A grantor scope implies the scopes it lists. Driven by + // ADMIN_GRANTOR_SCOPES so the runtime behaviour and the published + // catalogue cannot drift apart. + for (const [grantor, implied] of Object.entries(ADMIN_GRANTOR_SCOPES)) { + if ( + auth.scopes.includes(grantor as AdminScope) && + implied.includes(scope) + ) { + return true + } + } + return false } // ── Internal helpers ──────────────────────────────────────────────────────── @@ -233,6 +549,28 @@ export async function requireAdminAuth( } } +/** + * The symbol under which an admin scope guard records the scope it requires. + * The admin counterpart of SCOPE_TAG in src/middleware/apiKeyAuth.ts — see + * that module for why the tag exists. + */ +export const ADMIN_SCOPE_TAG = Symbol.for('neuro.adminScope') + +/** An admin scope guard, tagged with the scope it requires. */ +export type TaggedAdminScopeGuard = (( + req: Request, + res: Response, + next: NextFunction +) => void) & { + [ADMIN_SCOPE_TAG]?: AdminScope +} + +/** The scope an admin guard requires, or undefined when it is not one. */ +export function adminGuardScope(handler: unknown): AdminScope | undefined { + if (typeof handler !== 'function') return undefined + return (handler as TaggedAdminScopeGuard)[ADMIN_SCOPE_TAG] +} + /** * requireAdminScope(scope) * @@ -244,7 +582,7 @@ export async function requireAdminAuth( * Scope mismatches are written to AdminAuditLog and return 403. */ export function requireAdminScope(scope: AdminScope) { - return async ( + const guard = async ( req: Request, res: Response, next: NextFunction @@ -280,6 +618,9 @@ export function requireAdminScope(scope: AdminScope) { next() } + + ;(guard as TaggedAdminScopeGuard)[ADMIN_SCOPE_TAG] = scope + return guard as TaggedAdminScopeGuard } /** diff --git a/src/middleware/apiKeyAuth.ts b/src/middleware/apiKeyAuth.ts index edf5b915..5c9bba7b 100644 --- a/src/middleware/apiKeyAuth.ts +++ b/src/middleware/apiKeyAuth.ts @@ -141,11 +141,40 @@ export async function authenticateApiKey( } } +/** + * The symbol under which a scope-guard records the scope it requires. + * + * A scope guard is just a function to Express, so nothing in the routing + * layer says "this endpoint needs `withdraw:write`". Tagging the function + * makes that machine-readable: the scope-catalogue CI check walks the real + * route table, reads the tag off each route's middleware chain, and derives + * which endpoints each scope actually covers — so the catalogue cannot claim + * to unlock an endpoint that stopped requiring it. + */ +export const SCOPE_TAG = Symbol.for('neuro.apiKeyScope') + +/** A scope guard, tagged with the scope(s) it requires. */ +export type TaggedScopeGuard = (( + req: Request, + res: Response, + next: NextFunction +) => void) & { + [SCOPE_TAG]?: readonly UserScope[] +} + +/** The scope(s) a guard requires, or undefined when it is not a scope guard. */ +export function guardScopes( + handler: unknown +): readonly UserScope[] | undefined { + if (typeof handler !== 'function') return undefined + return (handler as TaggedScopeGuard)[SCOPE_TAG] +} + /** * Require one or more scopes. Sessions (`authScopes = ['*']`) pass all checks. */ export function requireScope(...requiredScopes: UserScope[]) { - return (req: Request, res: Response, next: NextFunction): void => { + const guard = (req: Request, res: Response, next: NextFunction): void => { const scopes = req.authScopes ?? ['*'] if (scopes.includes('*')) { next() @@ -169,6 +198,9 @@ export function requireScope(...requiredScopes: UserScope[]) { next() } + + ;(guard as TaggedScopeGuard)[SCOPE_TAG] = requiredScopes + return guard as TaggedScopeGuard } /** diff --git a/src/middleware/rateLimitTiers.ts b/src/middleware/rateLimitTiers.ts new file mode 100644 index 00000000..77faf6e7 --- /dev/null +++ b/src/middleware/rateLimitTiers.ts @@ -0,0 +1,634 @@ +/** + * The rate-limit tier catalog and the route → tier matrix (#614). + * + * Before this module existed, the limits lived only in code: every limiter was + * constructed inline in src/middleware/rateLimiter.ts and mounted by hand in + * src/index.ts or inside a route file. A contributor asking "what is the budget + * on POST /api/v1/deposit, and does it apply to a session or an API key?" had + * no answer that was not a archaeology exercise across three files — and no + * check that the answer stayed true. + * + * This module is the single, machine-readable source of truth: + * + * 1. RATE_LIMIT_TIERS — every tier, its window/burst, the principal it + * charges, the auth types that land in it, and the + * env vars that move it. + * 2. RATE_LIMIT_MATRIX — the route → tier bindings. + * + * tests/unit/middleware/rateLimitTierMatrix.test.ts fails CI when the two + * drift from the code: an unmapped route, a matrix entry that no longer + * matches any real route, a limiter that is not in the catalog, or a route + * whose attached limiter disagrees with the matrix. + * + * The matrix is expressed against the CANONICAL /api/v1/* paths. The legacy + * unversioned /api/* aliases are mounted from the same route table and + * therefore carry the same tier; see src/routes/registry.ts. + */ + +import { config } from '../config/env' + +// ── Tier identity ─────────────────────────────────────────────────────────── + +/** + * The tier ids. These are also the `limiterType` strings that reach the + * Prometheus metrics (src/utils/metrics.ts), so a tier id appearing in a + * dashboard can be traced straight back to this file. + */ +export const RATE_LIMIT_TIER_IDS = [ + 'global', + 'tiered', + 'auth', + 'admin', + 'erasure', + 'webhook', + 'sensitive', + 'optimizer', + 'simulate', + 'internal', +] as const + +export type RateLimitTierId = (typeof RATE_LIMIT_TIER_IDS)[number] + +/** + * How a request is identified when its budget is spent. + * + * ip — one bucket per client address. Used where no identity has + * been proven yet (login attempts) or where the caller is + * anonymous by design (inbound webhooks). + * principal — one bucket per proven identity: a session's user id, an API + * key's key id, the admin credential, or the internal service + * token. Two keys behind one NAT get two budgets. + */ +export type RateLimitPrincipalKind = 'ip' | 'principal' + +/** + * The caller identities a tier is designed for. `webhook` is the inbound + * provider callbacks (Twilio/Telegram/fiat) — authenticated by HMAC signature + * rather than by a bearer token, so it is a tier of its own. + */ +export type RateLimitAuthType = + 'session' | 'api_key' | 'admin' | 'webhook' | 'internal' | 'anonymous' + +export interface RateLimitTierDefinition { + id: RateLimitTierId + /** What this tier is for, and why it exists. */ + purpose: string + /** What the budget is charged against. */ + principal: RateLimitPrincipalKind + /** Which caller identities land in this tier. */ + authTypes: RateLimitAuthType[] + /** Effective window in milliseconds. */ + windowMs: number + /** Effective burst — requests allowed per window. */ + max: number + /** The env vars that move this tier, so an operator can find them. */ + env: { windowMs: string; max: string } + /** + * What happens to a request that reaches a tier this catalogue does not + * describe. Documented per tier because the answer is not the same + * everywhere: a route with no matrix entry still gets the global + tiered + * budgets (never unlimited), but the CI check treats the missing entry as + * a failure — see RATE_LIMIT_MATRIX_POLICY below. + */ + onUnknownCaller: 'anonymous-tier' | 'strict-tier' +} + +// ── The catalogue ─────────────────────────────────────────────────────────── + +const { security } = config + +/** + * Every rate-limit tier, with its live values. The numbers come from + * `config.security` so the catalogue can never drift from what the limiter + * actually enforces — only the env var NAMES are spelled out here, as a + * pointer for operators. + */ +export const RATE_LIMIT_TIERS: Record< + RateLimitTierId, + RateLimitTierDefinition +> = { + global: { + id: 'global', + purpose: + 'Baseline budget for every route. Catches runaway clients and gives ' + + 'the more specific tiers something to sit on top of.', + principal: 'principal', + authTypes: ['session', 'api_key', 'admin', 'webhook', 'internal'], + windowMs: security.rateLimit.windowMs, + max: security.rateLimit.max, + env: { windowMs: 'RATE_LIMIT_WINDOW_MS', max: 'RATE_LIMIT_MAX' }, + onUnknownCaller: 'strict-tier', + }, + tiered: { + id: 'tiered', + purpose: + 'Identity tiering. Anonymous callers draw on a strict per-IP budget; ' + + 'callers that proved an identity draw on a generous per-principal one. ' + + 'An API key’s own allowance overrides this tier entirely.', + principal: 'principal', + authTypes: ['session', 'api_key', 'admin', 'internal'], + windowMs: security.anonymousRateLimit.windowMs, + max: security.anonymousRateLimit.max, + env: { + windowMs: 'ANONYMOUS_RATE_LIMIT_WINDOW_MS', + max: 'ANONYMOUS_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, + auth: { + id: 'auth', + purpose: + 'Credential endpoints. Deliberately IP-keyed: credential stuffing ' + + 'happens before any identity is proven, so a per-user key would be ' + + 'meaningless here.', + principal: 'ip', + authTypes: ['anonymous', 'session'], + windowMs: security.authRateLimit.windowMs, + max: security.authRateLimit.max, + env: { windowMs: 'AUTH_RATE_LIMIT_WINDOW_MS', max: 'AUTH_RATE_LIMIT_MAX' }, + onUnknownCaller: 'strict-tier', + }, + admin: { + id: 'admin', + purpose: + 'The management API. Tightest of the general tiers — these routes ' + + 'move reserves, mint keys and settle referrals.', + principal: 'principal', + authTypes: ['admin', 'internal'], + windowMs: security.adminRateLimit.windowMs, + max: security.adminRateLimit.max, + env: { + windowMs: 'ADMIN_RATE_LIMIT_WINDOW_MS', + max: 'ADMIN_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, + erasure: { + id: 'erasure', + purpose: + 'Right-to-erasure. Irreversible, so it is held far below the general ' + + 'admin budget — a bug that loops here would delete data repeatedly.', + principal: 'principal', + authTypes: ['admin', 'internal'], + windowMs: 15 * 60 * 1000, + max: 5, + env: { windowMs: '(fixed)', max: '(fixed)' }, + onUnknownCaller: 'strict-tier', + }, + webhook: { + id: 'webhook', + purpose: + 'Inbound provider callbacks. Unauthenticated at the transport layer — ' + + 'the HMAC signature is the credential — so the budget resists ' + + 'spoofed and replayed floods.', + principal: 'ip', + authTypes: ['webhook'], + windowMs: security.webhookRateLimit.windowMs, + max: security.webhookRateLimit.max, + env: { + windowMs: 'WEBHOOK_RATE_LIMIT_WINDOW_MS', + max: 'WEBHOOK_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, + sensitive: { + id: 'sensitive', + purpose: + 'Money movement and credential changes. Applied per-endpoint IN ' + + 'ADDITION to the route’s normal budget, so a burst of withdrawals is ' + + 'stopped without throttling the balance reads the user needs to ' + + 'decide.', + principal: 'principal', + authTypes: ['session', 'api_key'], + windowMs: security.sensitiveRateLimit.windowMs, + max: security.sensitiveRateLimit.max, + env: { + windowMs: 'SENSITIVE_RATE_LIMIT_WINDOW_MS', + max: 'SENSITIVE_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, + optimizer: { + id: 'optimizer', + purpose: + 'The CPU-bound allocation suggester. Pairs with the in-flight ' + + 'semaphore in src/utils/concurrency.ts: that bounds how many run at ' + + 'once, this bounds how often they start.', + principal: 'principal', + authTypes: ['session', 'api_key'], + windowMs: security.optimizerRateLimit.windowMs, + max: security.optimizerRateLimit.max, + env: { + windowMs: 'OPTIMIZER_RATE_LIMIT_WINDOW_MS', + max: 'OPTIMIZER_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, + simulate: { + id: 'simulate', + purpose: + 'The CPU-bound historical replay. Tight enough that the replay cannot ' + + 'be used as a CPU-exhaustion vector; pairs with the short-TTL result ' + + 'cache so identical previews are served without recomputing.', + principal: 'principal', + authTypes: ['session', 'api_key'], + windowMs: security.simulateRateLimit.windowMs, + max: security.simulateRateLimit.max, + env: { + windowMs: 'SIMULATE_RATE_LIMIT_WINDOW_MS', + max: 'SIMULATE_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, + internal: { + id: 'internal', + purpose: + 'Service-to-service calls (the agent loop). Higher throughput than ' + + 'the user-facing tiers, and IP-keyed because the internal token is a ' + + 'single shared credential.', + principal: 'ip', + authTypes: ['internal'], + windowMs: security.internalRateLimit.windowMs, + max: security.internalRateLimit.max, + env: { + windowMs: 'INTERNAL_RATE_LIMIT_WINDOW_MS', + max: 'INTERNAL_RATE_LIMIT_MAX', + }, + onUnknownCaller: 'strict-tier', + }, +} + +/** + * The authenticated tier of `tiered` — a larger budget for callers that have + * proved an identity. Exposed separately because it is a real allowance with + * its own env vars, not a variant of the anonymous one. + */ +export const AUTHENTICATED_TIER_LIMITS = { + windowMs: security.authenticatedRateLimit.windowMs, + max: security.authenticatedRateLimit.max, + env: { + windowMs: 'AUTHENTICATED_RATE_LIMIT_WINDOW_MS', + max: 'AUTHENTICATED_RATE_LIMIT_MAX', + }, +} as const + +// ── The route → tier matrix ───────────────────────────────────────────────── + +export interface RateLimitMatrixEntry { + /** + * The canonical (versioned) path this entry covers, in Express's own route + * syntax — `:id` for a parameter. + */ + path: string + /** + * When true, the entry covers this path AND everything under it. When + * false (the default) the path must match exactly. + * + * The distinction matters: `/api/v1/deposit` is a single money-moving + * endpoint, while `/api/v1/deposit/recurring` is a separate router whose + * routes are scheduled deposits. A prefix entry on `/api/v1/deposit` would + * silently sweep the recurring router into the sensitive tier. + */ + prefix?: boolean + /** HTTP method, when the binding is method-specific. */ + method?: string + /** The tier that applies. */ + tier: RateLimitTierId + /** Why this route has this tier. */ + note: string +} + +/** + * The matrix. + * + * The resolver picks the most specific match: an exact entry beats a prefix + * entry, a longer path beats a shorter one, and a method-specific entry + * beats a path-only one. So the order here is for readability, not for + * correctness. + */ +export const RATE_LIMIT_MATRIX: readonly RateLimitMatrixEntry[] = [ + // ── Credential endpoints (auth tier) ───────────────────────────────────── + { + path: '/api/v1/auth', + prefix: true, + tier: 'auth', + note: 'Login, refresh, logout — credential stuffing surface.', + }, + { + path: '/api/v1/auth/webauthn', + prefix: true, + tier: 'auth', + note: 'WebAuthn ceremonies — same credential surface as password auth.', + }, + { + path: '/api/v1/webauthn', + prefix: true, + tier: 'auth', + note: 'Top-level WebAuthn alias, same ceremony budget.', + }, + + // ── Inbound webhooks (webhook tier) ───────────────────────────────────── + { + path: '/api/v1/whatsapp', + prefix: true, + tier: 'webhook', + note: 'Inbound WhatsApp provider callbacks, HMAC-signed.', + }, + { + path: '/api/v1/telegram', + prefix: true, + tier: 'webhook', + note: 'Inbound Telegram provider callbacks, HMAC-signed.', + }, + + // ── The management API (admin tier) ───────────────────────────────────── + { + path: '/api/v1/admin', + prefix: true, + tier: 'admin', + note: 'Management API — reserves, key minting, referral settlement.', + }, + { + path: '/api/v1/admin/support/tickets', + prefix: true, + tier: 'admin', + note: 'Admin view of support tickets.', + }, + { + path: '/api/v1/admin/notifications/dlq', + prefix: true, + tier: 'admin', + note: 'Dead-letter queue inspection and replay.', + }, + + // ── Erasure (erasure tier) ────────────────────────────────────────────── + // Irreversible, so these are held far below the general admin budget. + { + path: '/api/v1/admin/erasure/requests', + method: 'POST', + tier: 'erasure', + note: 'Creating an erasure request is a destructive operation.', + }, + { + path: '/api/v1/admin/erasure/requests/:id/approve', + method: 'POST', + tier: 'erasure', + note: 'Approving an erasure request authorises data deletion.', + }, + { + path: '/api/v1/admin/erasure/requests/:id/reject', + method: 'POST', + tier: 'erasure', + note: 'Rejecting an erasure request is part of the erasure workflow.', + }, + { + path: '/api/v1/admin/erasure/requests/:id/execute', + method: 'POST', + tier: 'erasure', + note: 'Executing an erasure request deletes data — the sharpest edge.', + }, + { + path: '/api/v1/admin/erasure/preview/:userId', + method: 'GET', + tier: 'erasure', + note: 'Previewing an erasure walks the same deletion path as the execute.', + }, + + // ── Money movement and credential changes (sensitive tier) ────────────── + { + path: '/api/v1/deposit', + method: 'POST', + tier: 'sensitive', + note: 'Deposits move money — irreversible once submitted.', + }, + { + path: '/api/v1/withdraw', + method: 'POST', + tier: 'sensitive', + note: 'Withdrawals move money — irreversible once submitted.', + }, + { + path: '/api/v1/loans', + method: 'POST', + tier: 'sensitive', + note: 'Loan origination moves money.', + }, + { + path: '/api/v1/loans/:loanId/repay', + method: 'POST', + tier: 'sensitive', + note: 'Loan repayment moves money.', + }, + { + path: '/api/v1/keys', + method: 'POST', + tier: 'sensitive', + note: 'Minting an API key is a takeover vector — credential change.', + }, + { + path: '/api/v1/keys/:id/rotate', + method: 'POST', + tier: 'sensitive', + note: 'Rotating a key is a credential change.', + }, + { + path: '/api/v1/support/tickets', + method: 'POST', + tier: 'sensitive', + note: 'Opening a ticket is a state change on a customer-visible record.', + }, + + // ── Compute-bound endpoints (optimizer / simulate tiers) ──────────────── + { + path: '/api/v1/portfolio/:userId/suggest-allocation', + method: 'POST', + tier: 'optimizer', + note: 'CPU-bound allocation solver — the classic exhaustion vector.', + }, + { + path: '/api/v1/strategies/simulate', + method: 'POST', + tier: 'simulate', + note: 'CPU-bound historical replay.', + }, + + // ── Service-to-service (internal tier) ────────────────────────────────── + { + path: '/api/v1/agent/decisions', + prefix: true, + tier: 'tiered', + note: + 'Agent decision history is a user-facing read (session auth), not a ' + + 'service-to-service call — only the agent status endpoints are.', + }, + { + path: '/api/v1/agent', + prefix: true, + tier: 'internal', + note: 'Agent loop endpoints — service-to-service, guarded by the internal token.', + }, + + // ── Everything else: the global + tiered baseline ────────────────────── + { + path: '/api/v1', + prefix: true, + tier: 'tiered', + note: + 'Default for every remaining versioned API route: the global budget ' + + 'plus the identity tier (strict per-IP when anonymous, generous ' + + 'per-principal once authenticated).', + }, + { + path: '/api/webhooks', + prefix: true, + tier: 'tiered', + note: 'Pre-version webhook management surface; same baseline as /api/v1.', + }, + { + path: '/api/analytics', + prefix: true, + tier: 'tiered', + note: 'Pre-version analytics alias; same baseline as /api/v1.', + }, + { + path: '/api/stellar', + prefix: true, + tier: 'tiered', + note: 'Pre-version Stellar alias; same baseline as /api/v1.', + }, + { + path: '/health', + prefix: true, + tier: 'tiered', + note: + 'Liveness and readiness probes. The GLOBAL limiter skips these so ' + + 'k8s probes never consume the budget; the tiered limiter still applies.', + }, + { + path: '/metrics', + prefix: true, + tier: 'tiered', + note: 'Prometheus scrape endpoint, guarded by the internal auth guard.', + }, +] as const + +/** + * The fail-open vs fail-closed decision, stated once so it can be cited. + * + * RUNTIME — fail-OPEN in the sense that no route is ever unlimited: the + * global and tiered limiters are mounted app-wide (src/index.ts), so an + * unmapped route still gets a budget. It is not "fail-open" in the sense of + * letting a request through unchecked. + * + * CI — fail-CLOSED: a route with no matrix entry, or a matrix entry that no + * longer matches a real route, fails the build. The runtime behaviour is a + * safety net; the matrix is the contract, and the contract is enforced. + */ +export const RATE_LIMIT_MATRIX_POLICY = { + runtime: 'fail-open' as const, + ci: 'fail-closed' as const, + rationale: + 'Every route inherits the global + tiered budgets at runtime, so an ' + + 'unmapped route is never unlimited. CI is strict instead, because a ' + + 'silent gap in the matrix is exactly the kind of drift this catalogue ' + + 'exists to prevent.', +} + +// ── Resolution ───────────────────────────────────────────────────────────── + +/** Normalise a path for matching: strip the query, the version prefix and a trailing slash. */ +function normalisePath(path: string): string { + let p = path.split('?')[0] + if (p.length > 1 && p.endsWith('/')) p = p.slice(0, -1) + return p || '/' +} + +/** + * Resolve the tier for a request. + * + * Specificity, strongest first: + * 1. an exact path match beats a prefix match + * 2. a longer path beats a shorter one + * 3. at equal path, a method-specific entry beats a path-only one + * + * Returns undefined when nothing matches — the caller (the CI check) treats + * that as a failure. + */ +export function resolveRateLimitTier( + method: string, + path: string +): RateLimitMatrixEntry | undefined { + const target = normalisePath(path) + const verb = method.toUpperCase() + + let best: RateLimitMatrixEntry | undefined + let bestScore = -1 + + for (const entry of RATE_LIMIT_MATRIX) { + const entryPath = normalisePath(entry.path) + + let pathMatches: boolean + if (entry.prefix) { + // A prefix entry covers the path itself and everything under it, but + // only on a path-segment boundary: `/api/v1/keys` must not match + // `/api/v1/keystore`. + pathMatches = + target === entryPath || + (entryPath !== '/' && target.startsWith(entryPath + '/')) + } else { + pathMatches = target === entryPath + } + if (!pathMatches) continue + + // A method-specific entry only applies to that method; a path-only entry + // applies to every method. + if (entry.method && entry.method.toUpperCase() !== verb) continue + + const score = + entryPath.length * 100 + (entry.prefix ? 0 : 10) + (entry.method ? 1 : 0) + if (score > bestScore) { + best = entry + bestScore = score + } + } + + return best +} + +/** The tier for a request, or undefined when the route is not in the matrix. */ +export function tierForRoute( + method: string, + path: string +): RateLimitTierId | undefined { + return resolveRateLimitTier(method, path)?.tier +} + +/** + * Render the matrix as a Markdown table — the source of the table in + * docs/RATE_LIMITING.md. Kept here (rather than hand-written in the doc) so + * the two cannot drift: the doc is generated, the test asserts it is current. + */ +export function renderRateLimitMatrixMarkdown(): string { + const header = '| Route | Method | Tier | Window | Burst | Why |' + const divider = '| --- | --- | --- | --- | --- | --- |' + const rows = RATE_LIMIT_MATRIX.map((entry) => { + const tier = RATE_LIMIT_TIERS[entry.tier] + const window = + entry.tier === 'tiered' + ? `${tier.windowMs}ms / ${AUTHENTICATED_TIER_LIMITS.max} authenticated` + : `${tier.windowMs}ms` + return `| \`${entry.path}\` | ${entry.method ?? 'ANY'} | \`${entry.tier}\` | ${window} | ${tier.max} | ${entry.note} |` + }) + return [header, divider, ...rows].join('\n') +} + +/** Render the tier catalogue as a Markdown table. */ +export function renderRateLimitTierCatalogMarkdown(): string { + const header = '| Tier | Window | Burst | Principal | Auth types | Purpose |' + const divider = '| --- | --- | --- | --- | --- | --- |' + const rows = RATE_LIMIT_TIER_IDS.map((id) => { + const tier = RATE_LIMIT_TIERS[id] + return `| \`${id}\` | ${tier.windowMs}ms | ${tier.max} | ${tier.principal} | ${tier.authTypes.join(', ')} | ${tier.purpose} |` + }) + return [header, divider, ...rows].join('\n') +} diff --git a/src/middleware/rateLimiter.ts b/src/middleware/rateLimiter.ts index 20a0aa11..a179b7af 100644 --- a/src/middleware/rateLimiter.ts +++ b/src/middleware/rateLimiter.ts @@ -5,6 +5,7 @@ import { recordRateLimitHit, updateRateLimitViolations } from '../utils/metrics' import { logger } from '../utils/logger' import db from '../db' import crypto from 'node:crypto' +import type { RateLimitTierId } from './rateLimitTiers' // ── Trusted-IP / service-token bypass ───────────────────────────────────── @@ -209,6 +210,47 @@ function handleRateLimitExceeded( // ── Rate limiter factory ─────────────────────────────────────────────────── +/** + * The symbol under which a limiter records the tier it belongs to. + * + * A limiter is just a function to Express, so nothing in the routing layer + * says "this route is rate-limited at the sensitive tier". Tagging the + * function makes that fact machine-readable: the tier-matrix CI check + * (tests/unit/middleware/rateLimitTierMatrix.test.ts) walks the real route + * table, reads the tag off each route's middleware chain, and fails if the + * code and the matrix in src/middleware/rateLimitTiers.ts disagree. + */ +export const RATE_LIMIT_TIER_TAG = Symbol.for('neuro.rateLimitTier') + +/** A rate-limiting middleware, tagged with the tier it enforces. */ +export type TaggedRateLimiter = (( + req: Request, + res: Response, + next: NextFunction +) => void) & { + [RATE_LIMIT_TIER_TAG]?: RateLimitTierId +} + +/** Record the tier a limiter enforces. */ +export function tagLimiter( + limiter: (req: Request, res: Response, next: NextFunction) => void, + tier: RateLimitTierId +): TaggedRateLimiter { + ;(limiter as TaggedRateLimiter)[RATE_LIMIT_TIER_TAG] = tier + return limiter as TaggedRateLimiter +} + +/** + * The tier a limiter enforces, or undefined when the handler is not one of + * ours. Used by the tier-matrix CI check to read the effective tier straight + * off a route's middleware chain. + */ +export function limiterTier(handler: unknown): RateLimitTierId | undefined { + if (typeof handler !== 'function') return undefined + const tier = (handler as TaggedRateLimiter)[RATE_LIMIT_TIER_TAG] + return tier +} + export interface BuildRateLimiterOptions { windowMs: number max: number @@ -222,6 +264,12 @@ export interface BuildRateLimiterOptions { * client can rotate IPs freely to escape the budget. */ keyByPrincipal?: boolean + /** + * The catalogue tier this limiter enforces. Required for every limiter so + * the tier matrix stays complete — an untagged limiter is a limiter the + * matrix cannot describe. + */ + tier: RateLimitTierId } /** @@ -232,7 +280,7 @@ export interface BuildRateLimiterOptions { */ export function buildRateLimiter( opts: BuildRateLimiterOptions -): (req: Request, res: Response, next: NextFunction) => void { +): TaggedRateLimiter { const usePrincipal = opts.keyByPrincipal !== false /** @@ -280,12 +328,12 @@ export function buildRateLimiter( }), }) - return (req: Request, res: Response, next: NextFunction): void => { + return tagLimiter((req: Request, res: Response, next: NextFunction): void => { limiter(req, res, () => { applyPolicyHeader(req, res, { max: opts.max, windowMs: opts.windowMs }) next() }) - } + }, opts.tier) } // ── Rate limiters ────────────────────────────────────────────────────────── @@ -299,6 +347,7 @@ export const rateLimiter = buildRateLimiter({ max: config.security.rateLimit.max, skip: skipUnlessLimited, limiterType: 'global', + tier: 'global', }) /** @@ -362,7 +411,7 @@ export const tieredRateLimiter = (() => { }), }) - return (req: Request, res: Response, next: NextFunction): void => { + return tagLimiter((req: Request, res: Response, next: NextFunction): void => { limiter(req, res, () => { applyPolicyHeader(req, res, { max: anonymousMax, @@ -370,7 +419,7 @@ export const tieredRateLimiter = (() => { }) next() }) - } + }, 'tiered') })() /** @@ -386,6 +435,7 @@ export const authRateLimiter = buildRateLimiter({ skip: isTrusted, limiterType: 'auth', keyByPrincipal: false, + tier: 'auth', message: 'Too many authentication attempts. Please try again in 15 minutes.', }) @@ -398,6 +448,7 @@ export const adminRateLimiter = buildRateLimiter({ max: config.security.adminRateLimit.max, skip: isHealthProbe, limiterType: 'admin', + tier: 'admin', message: 'Too many requests to the admin API. Please try again later.', }) @@ -409,7 +460,8 @@ export const erasureRateLimiter = buildRateLimiter({ windowMs: 15 * 60 * 1000, max: 5, skip: isHealthProbe, - limiterType: 'admin', + limiterType: 'erasure', + tier: 'erasure', message: 'Too many erasure operations. Please try again later.', }) @@ -423,6 +475,7 @@ export const webhookRateLimiter = buildRateLimiter({ skip: isTrusted, limiterType: 'webhook', keyByPrincipal: false, + tier: 'webhook', message: 'Too many webhook requests. Please try again later.', }) @@ -439,6 +492,7 @@ export const sensitiveRateLimiter = buildRateLimiter({ max: config.security.sensitiveRateLimit.max, skip: isTrusted, limiterType: 'sensitive', + tier: 'sensitive', message: 'Too many sensitive operations. Please wait before retrying this action.', }) @@ -460,6 +514,7 @@ export const optimizerRateLimiter = buildRateLimiter({ max: config.security.optimizerRateLimit.max, skip: isTrusted, limiterType: 'optimizer', + tier: 'optimizer', message: 'Too many optimization requests. Portfolio optimization is compute-intensive; please try again shortly.', }) @@ -476,6 +531,7 @@ export const simulateRateLimiter = buildRateLimiter({ max: config.security.simulateRateLimit.max, skip: isTrusted, limiterType: 'simulate', + tier: 'simulate', message: 'Too many simulation requests. Historical replay is compute-intensive; please try again shortly.', }) @@ -489,5 +545,6 @@ export const internalRateLimiter = buildRateLimiter({ skip: isTrusted, limiterType: 'internal', keyByPrincipal: false, + tier: 'internal', message: 'Too many requests from this service. Please slow down.', }) diff --git a/src/middleware/rateLimiterChain.ts b/src/middleware/rateLimiterChain.ts new file mode 100644 index 00000000..85f368c3 --- /dev/null +++ b/src/middleware/rateLimiterChain.ts @@ -0,0 +1,26 @@ +/** + * The app-wide rate-limit chain (#614). + * + * These two limiters are mounted before routing, so they apply to EVERY + * request the server serves — including requests that match no route at all. + * Keeping them in one named list (rather than two `app.use` calls inline in + * src/index.ts) means the tier-matrix CI check can read the chain and + * assert the claim "every route is covered by the global + tiered budgets" + * instead of taking it on trust. + * + * global — a flat budget over everything. + * tiered — the identity split: strict per-IP for anonymous callers, + * generous per-principal for callers that proved an identity. + */ + +import { + rateLimiter, + tieredRateLimiter, + type TaggedRateLimiter, +} from './rateLimiter' + +/** Mounted in order: the flat budget, then the identity tier. */ +export const APP_LEVEL_LIMITERS: TaggedRateLimiter[] = [ + rateLimiter, + tieredRateLimiter, +] diff --git a/src/routes/admin.ts b/src/routes/admin.ts index 4b40b2af..f9900fe9 100644 --- a/src/routes/admin.ts +++ b/src/routes/admin.ts @@ -32,6 +32,11 @@ import { manualResetBreaker, } from '../agent/breakerService' import { messageDeliveryService } from '../messaging' +import { + getAdminScopeCatalog, + getUserScopeCatalog, +} from '../auth/scopeCoverage' +import { DEFAULT_READ_SCOPES } from '../auth/scopes' const router = Router() const prisma = db @@ -1084,6 +1089,62 @@ router.post( } ) +/** + * GET /api/admin/keys/scopes + * The machine-readable scope catalogue (#615). + * + * Serves both scope families with their description, risk level, guardrail + * notes and the endpoints each scope actually covers, so the key-creation UI + * can show an operator exactly what granting a scope would hand over. + * + * The endpoint lists are derived from the live route table + * (src/auth/scopeCoverage.ts), so a scope cannot claim to unlock an endpoint + * that stopped requiring it. + * + * Required scope: keys:read + */ +router.get( + '/keys/scopes', + requireAdminScope('keys:read'), + async (req: Request, res: Response) => { + const adminAuth = getAdminAuth(res) + if (!adminAuth) { + return res + .status(401) + .json({ success: false, error: 'Admin authentication required' }) + } + try { + const data = { + user: { + defaultReadScopes: DEFAULT_READ_SCOPES, + scopes: getUserScopeCatalog(), + }, + admin: { + scopes: getAdminScopeCatalog(), + }, + } + + auditLog(req, res, 'LIST_SCOPE_CATALOG', 'success', { + userScopes: data.user.scopes.length, + adminScopes: data.admin.scopes.length, + }) + + res.status(200).json({ + success: true, + data, + timestamp: new Date().toISOString(), + }) + } catch (error) { + logger.error('[Admin] Failed to build scope catalogue', { + error: error instanceof Error ? error.message : 'Unknown error', + }) + res + .status(500) + .json({ success: false, error: 'Failed to build scope catalogue' }) + } + } +) + /** * GET /api/admin/keys * List all admin API keys (metadata only — no hashes). diff --git a/src/routes/portfolio.ts b/src/routes/portfolio.ts index 6e407ca7..38574a38 100644 --- a/src/routes/portfolio.ts +++ b/src/routes/portfolio.ts @@ -2,6 +2,7 @@ import { Router, Request, Response } from 'express' import { z } from 'zod' import db from '../db' import { requireAuth, enforceUserAccess } from '../middleware/authenticate' +import { requireScope } from '../middleware/apiKeyAuth' import { validate } from '../middleware/validate' import { mapAllocationSuggestionToResponse, @@ -168,6 +169,9 @@ router.post( router.get( '/:userId/suggestions', requireAuth, + // #615 — `portfolio:read` was declared but never enforced, so ANY + // authenticated API key could read the user's whole portfolio. + requireScope('portfolio:read'), enforceUserAccess, validate(listSuggestionsSchema), async (req: Request, res: Response) => { @@ -207,6 +211,9 @@ router.get( router.get( '/:userId', requireAuth, + // #615 — `portfolio:read` was declared but never enforced, so ANY + // authenticated API key could read the user's whole portfolio. + requireScope('portfolio:read'), enforceUserAccess, validate(portfolioSchema), async (req: Request, res: Response) => { @@ -287,6 +294,9 @@ router.get( router.get( '/:userId/history', requireAuth, + // #615 — `portfolio:read` was declared but never enforced, so ANY + // authenticated API key could read the user's whole portfolio. + requireScope('portfolio:read'), enforceUserAccess, validate(historySchema), async (req: Request, res: Response) => { @@ -332,6 +342,9 @@ router.get( router.get( '/:userId/earnings', requireAuth, + // #615 — `portfolio:read` was declared but never enforced, so ANY + // authenticated API key could read the user's whole portfolio. + requireScope('portfolio:read'), enforceUserAccess, validate(portfolioParamsSchema), async (req: Request, res: Response) => { @@ -385,6 +398,9 @@ router.get( router.get( '/:userId/tax-report', requireAuth, + // #615 — `portfolio:read` was declared but never enforced, so ANY + // authenticated API key could read the user's tax report. + requireScope('portfolio:read'), enforceUserAccess, validate(taxReportSchema), async (req: Request, res: Response) => { diff --git a/src/routes/registry.ts b/src/routes/registry.ts new file mode 100644 index 00000000..ad5b92d1 --- /dev/null +++ b/src/routes/registry.ts @@ -0,0 +1,150 @@ +/** + * The route table (#614). + * + * This is the single list of routers the API mounts, extracted from + * src/index.ts so that the tier-matrix CI check + * (tests/unit/middleware/rateLimitTierMatrix.test.ts) can walk the REAL route + * table — every method and path the server actually serves — instead of a + * hand-written list that could fall behind the code. + * + * The legacy unversioned /api/* aliases are mounted from the same table, so a + * path listed here is served at BOTH /api/v1/ and /api/ with the + * same handlers and therefore the same rate-limit tier. + */ + +import { type RequestHandler } from 'express' +import { + adminRateLimiter, + authRateLimiter, + internalRateLimiter, + webhookRateLimiter, +} from '../middleware/rateLimiter' +import agentRouter from './agent' +import agentDecisionsRouter from './agent-decisions' +import adminRouter from './admin' +import adminSupportTicketsRouter from './admin/support-tickets' +import alertsRouter from './alerts' +import analyticsRouter from './analytics' +import approvalPoliciesRouter from './approval-policies' +import approvalsRouter from './approvals' +import assistantRouter from './assistant' +import authRouter from './auth' +import depositRouter from './deposit' +import fiatRouter from './fiat' +import healthRouter from './health' +import keysRouter from './keys' +import liquidityFloorRouter from './liquidity-floor' +import loansRouter from './loans' +import metricsRouter from './metrics' +import netWorthRouter from './net-worth' +import networkRouter from './network' +import notificationDlqRouter from './notification-dlq' +import notificationsRouter from './notifications' +import portfolioRouter from './portfolio' +import protocolsRouter from './protocols' +import protectionFundRouter from './protection-fund' +import recurringDepositRouter from './recurring-deposits' +import recurringWithdrawalRouter from './recurring-withdrawals' +import referralsRouter from './referrals' +import sessionsRouter from './sessions' +import settingsRouter from './settings' +import stellarRouter from './stellar' +import strategiesRouter from './strategies' +import streamRouter from './stream' +import subAccountsRouter from './sub-accounts' +import supportTicketsRouter from './support-tickets' +import telegramRouter from './telegram' +import transactionsRouter from './transactions' +import vaultRouter from './vault' +import webAuthnRouter from './webauthn' +import webhooksRouter from './webhooks' +import whatsappRouter from './whatsapp' +import withdrawRouter from './withdraw' + +export interface ApiRoute { + /** Path relative to the version prefix — e.g. `deposit` → `/api/v1/deposit`. */ + path: string + /** + * The handlers mounted for this path. A rate limiter may appear here, in + * which case it applies to EVERY route under the mount. + */ + handlers: RequestHandler[] +} + +/** + * Routers mounted under the versioned `/api/v1/*` prefix, and their legacy + * `/api/*` aliases. + * + * A limiter in `handlers` applies to every route under that mount. Per-route + * limiters (the `sensitiveRateLimiter` on a single deposit endpoint, say) are + * attached inside the route file itself — the tier matrix reads them off the + * route's own middleware chain. + */ +export const API_ROUTES: ApiRoute[] = [ + { path: 'network', handlers: [networkRouter] }, + { path: 'net-worth', handlers: [netWorthRouter] }, + { path: 'agent/decisions', handlers: [agentDecisionsRouter] }, + { path: 'agent', handlers: [internalRateLimiter, agentRouter] }, + { path: 'auth/webauthn', handlers: [authRateLimiter, webAuthnRouter] }, + { path: 'auth', handlers: [authRateLimiter, authRouter] }, + { path: 'webauthn', handlers: [authRateLimiter, webAuthnRouter] }, + { path: 'whatsapp', handlers: [webhookRateLimiter, whatsappRouter] }, + { path: 'telegram', handlers: [webhookRateLimiter, telegramRouter] }, + { path: 'portfolio', handlers: [portfolioRouter] }, + { path: 'transactions', handlers: [transactionsRouter] }, + { path: 'protocols', handlers: [protocolsRouter] }, + { path: 'deposit', handlers: [depositRouter] }, + { path: 'withdraw', handlers: [withdrawRouter] }, + // #532 — collateral loans. Mounted beside withdraw rather than inside + // portfolio: a loan is its own lifecycle with its own approval and rate + // limits, and it must not inherit portfolio's read-only scope set. + { path: 'loans', handlers: [loansRouter] }, + // #533 — protocol-risk protection fund + { path: 'protection-fund', handlers: [protectionFundRouter] }, + // #534 — user settings (display currency) + { path: 'settings', handlers: [settingsRouter] }, + { path: 'vault', handlers: [vaultRouter] }, + { path: 'analytics', handlers: [analyticsRouter] }, + { path: 'stellar', handlers: [stellarRouter] }, + { path: 'fiat', handlers: [fiatRouter] }, + { path: 'referrals', handlers: [referralsRouter] }, + { path: 'deposit/recurring', handlers: [recurringDepositRouter] }, + { path: 'recurring-withdrawals', handlers: [recurringWithdrawalRouter] }, + { path: 'alerts', handlers: [alertsRouter] }, + { path: 'strategies', handlers: [strategiesRouter] }, + { path: 'sub-accounts', handlers: [subAccountsRouter] }, + { path: 'support/tickets', handlers: [supportTicketsRouter] }, + { + path: 'admin/support/tickets', + handlers: [adminRateLimiter, adminSupportTicketsRouter], + }, + { path: 'assistant', handlers: [assistantRouter] }, + { path: 'approvals', handlers: [approvalsRouter] }, + { path: 'approval-policies', handlers: [approvalPoliciesRouter] }, + { path: 'keys', handlers: [keysRouter] }, + { path: 'sessions', handlers: [sessionsRouter] }, + { path: 'stream', handlers: [streamRouter] }, + { path: 'notifications', handlers: [notificationsRouter] }, + { path: 'liquidity-floor', handlers: [liquidityFloorRouter] }, + { + path: 'admin/notifications/dlq', + handlers: [adminRateLimiter, notificationDlqRouter], + }, + { path: 'admin', handlers: [adminRateLimiter, adminRouter] }, +] + +/** + * Routers mounted OUTSIDE the `/api/v1` version scheme. + * + * These are either operational endpoints (health, metrics) or the pre-version + * paths that predate the versioning policy. They are listed here so the tier + * matrix can describe them too — an endpoint that no one can find in the + * matrix is an endpoint nobody knows the budget of. + */ +export const EXTRA_MOUNTS: ApiRoute[] = [ + { path: '/health', handlers: [healthRouter] }, + { path: '/api/analytics', handlers: [analyticsRouter] }, + { path: '/api/stellar', handlers: [stellarRouter] }, + { path: '/api/webhooks', handlers: [webhooksRouter] }, + { path: '/metrics', handlers: [metricsRouter] }, +] diff --git a/src/routes/transactions.ts b/src/routes/transactions.ts index 89ce71fd..da55b2b2 100644 --- a/src/routes/transactions.ts +++ b/src/routes/transactions.ts @@ -2,6 +2,7 @@ import { Router, Request, Response } from 'express' import { z } from 'zod' import db from '../db' import { requireAuth, enforceUserAccess } from '../middleware/authenticate' +import { requireScope } from '../middleware/apiKeyAuth' import { validate } from '../middleware/validate' import { buildPaginationMeta, @@ -59,6 +60,9 @@ const router = Router() router.get( '/detail/:txHash', requireAuth, + // #615 — `transactions:read` was declared but never enforced, so ANY + // authenticated API key could read the user's transaction history. + requireScope('transactions:read'), validate({ params: txHashParamSchema }), async (req: Request, res: Response) => { const txHash = String(req.params.txHash) @@ -86,6 +90,9 @@ router.get( router.get( '/:userId', requireAuth, + // #615 — `transactions:read` was declared but never enforced, so ANY + // authenticated API key could read the user's transaction history. + requireScope('transactions:read'), enforceUserAccess, validate(listSchema), async (req: Request, res: Response) => { @@ -145,18 +152,25 @@ router.get( * GET /transactions/:id/events * Returns the ordered event history for a transaction (admin only). */ -router.get('/:id/events', requireAuth, async (req: Request, res: Response) => { - const id = String(req.params.id) // ← String() cast fixes the string | string[] error +router.get( + '/:id/events', + requireAuth, + // #615 — `transactions:read` was declared but never enforced, so ANY + // authenticated API key could read the user's transaction history. + requireScope('transactions:read'), + async (req: Request, res: Response) => { + const id = String(req.params.id) // ← String() cast fixes the string | string[] error - const tx = await db.transaction.findUnique({ where: { id } }) - if (!tx) return sendNotFound(res, 'Transaction') + const tx = await db.transaction.findUnique({ where: { id } }) + if (!tx) return sendNotFound(res, 'Transaction') - const events = await (db as any).transactionEvent.findMany({ - where: { transactionId: id }, - orderBy: { occurredAt: 'asc' }, - }) + const events = await (db as any).transactionEvent.findMany({ + where: { transactionId: id }, + orderBy: { occurredAt: 'asc' }, + }) - return res.status(200).json({ transactionId: id, events }) -}) + return res.status(200).json({ transactionId: id, events }) + } +) export default router diff --git a/src/routes/vault.ts b/src/routes/vault.ts index 41667f68..e33d35e7 100644 --- a/src/routes/vault.ts +++ b/src/routes/vault.ts @@ -29,28 +29,36 @@ router.get('/state', async (req: Request, res: Response) => { }) }) -router.get('/balance', requireAuth, async (req: Request, res: Response) => { - const userId = req.auth?.userId - if (!userId) { - return res.status(401).json({ error: 'Unauthorized' }) - } +router.get( + '/balance', + requireAuth, + // #615 — `vault:read` was declared but never enforced, so ANY authenticated + // API key could read the user's vault balance. The scope catalogue is only + // worth publishing if the scope it describes actually gates something. + requireScope('vault:read'), + async (req: Request, res: Response) => { + const userId = req.auth?.userId + if (!userId) { + return res.status(401).json({ error: 'Unauthorized' }) + } - const user = await db.user.findUnique({ - where: { id: userId }, - select: { walletAddress: true }, - }) + const user = await db.user.findUnique({ + where: { id: userId }, + select: { walletAddress: true }, + }) - if (!user) { - return res.status(404).json({ error: 'User not found' }) - } + if (!user) { + return res.status(404).json({ error: 'User not found' }) + } - const onChain = await getOnChainBalance(user.walletAddress) + const onChain = await getOnChainBalance(user.walletAddress) - return res.status(200).json({ - balance: toNumber(onChain.balance), - shares: toNumber(onChain.shares), - }) -}) + return res.status(200).json({ + balance: toNumber(onChain.balance), + shares: toNumber(onChain.shares), + }) + } +) const buildTransactionSchema = z.object({ type: z.enum(['deposit', 'withdraw']), diff --git a/tests/helpers/routeTable.ts b/tests/helpers/routeTable.ts new file mode 100644 index 00000000..ba1bd62d --- /dev/null +++ b/tests/helpers/routeTable.ts @@ -0,0 +1,118 @@ +/** + * Route-table introspection for the tier-matrix CI check (#614). + * + * Express 4 has no public "list my routes" API, so the check walks the + * router stack directly. This is the only place that knows how to do that, + * so the walk stays in one file and every consumer sees the same route + * shapes. + * + * The paths produced here are Express's own route syntax — `:id` for a + * parameter — and they are the paths the tier matrix in + * src/middleware/rateLimitTiers.ts is written against. + */ + +import { type RequestHandler } from 'express' +import { limiterTier } from '../../src/middleware/rateLimiter' +import type { RateLimitTierId } from '../../src/middleware/rateLimitTiers' + +export interface DiscoveredRoute { + method: string + /** The full path, e.g. `/api/v1/keys/:id/rotate`. */ + path: string + /** + * The tier the CODE applies to this route: the route's own limiter if it + * has one, otherwise the mount-level limiter, otherwise the app-wide + * baseline (`tiered`). + */ + tier: RateLimitTierId + /** Where the tier came from — useful when a test fails. */ + tierSource: 'route' | 'mount' | 'baseline' +} + +/** + * Turn an Express 4 path regexp back into a readable path. + * + * Only needed for `router.use(path, subRouter)` mounts, where Express does + * not keep the original path string on the layer. Route layers expose + * `layer.route.path` directly, so this is only the fallback. + */ +function pathFromRegexp(source: string): string { + let s = source + s = s.replace(/^\^/, '') + s = s.replace(/\\\/\?\(\?=\\\/\|\$\)$/, '') + s = s.replace(/\\\//g, '/') + s = s.replace(/\(\?:[^)]*\)/g, ':p') + s = s.replace(/\([^)]*\)/g, ':p') + return s || '/' +} + +/** Normalise a path for comparison: no trailing slash, except for the root. */ +export function normaliseRoutePath(path: string): string { + const p = path.split('?')[0] + if (p.length > 1 && p.endsWith('/')) return p.slice(0, -1) + return p || '/' +} + +function walkStack( + stack: any[], + prefix: string, + mountTier: RateLimitTierId | undefined, + out: DiscoveredRoute[] +): void { + for (const layer of stack) { + if (layer.route) { + const methods = Object.keys(layer.route.methods || {}) + for (const method of methods) { + let tier = mountTier + let tierSource: DiscoveredRoute['tierSource'] = mountTier + ? 'mount' + : 'baseline' + + for (const handle of layer.route.stack || []) { + const routeTier = limiterTier(handle.handle) + if (routeTier) { + tier = routeTier + tierSource = 'route' + } + } + + out.push({ + method: method.toUpperCase(), + path: normaliseRoutePath(prefix + (layer.route.path || '')), + tier: tier ?? 'tiered', + tierSource, + }) + } + } else if (layer.name === 'router' && layer.handle?.stack) { + const nested = pathFromRegexp(layer.regexp.source) + walkStack( + layer.handle.stack, + prefix + (nested === '/' ? '' : nested), + mountTier, + out + ) + } + } +} + +/** + * Walk a mounted router and return every route it serves, with the tier the + * code applies to it. + */ +export function discoverRoutes( + mountPath: string, + handlers: RequestHandler[] +): DiscoveredRoute[] { + let mountTier: RateLimitTierId | undefined + for (const handler of handlers) { + const tier = limiterTier(handler) + if (tier) mountTier = tier + } + + const router = handlers[handlers.length - 1] as unknown as { + stack?: any[] + } + const out: DiscoveredRoute[] = [] + walkStack(router.stack ?? [], mountPath, mountTier, out) + return out +} diff --git a/tests/integration/rateLimiter.integration.test.ts b/tests/integration/rateLimiter.integration.test.ts index 79fd3cf7..a16f26ca 100644 --- a/tests/integration/rateLimiter.integration.test.ts +++ b/tests/integration/rateLimiter.integration.test.ts @@ -66,6 +66,7 @@ describe('rate limiter – IETF rate-limit headers', () => { windowMs: 60000, max: 30, limiterType: 'test', + tier: 'global', }) const app = buildTestApp(limiter) const res = await request(app).get('/test') @@ -80,6 +81,7 @@ describe('rate limiter – IETF rate-limit headers', () => { windowMs: 60000, max: 1, limiterType: 'test', + tier: 'global', }) const app = buildTestApp(limiter) @@ -95,6 +97,7 @@ describe('rate limiter – IETF rate-limit headers', () => { windowMs: 60000, max: 1, limiterType: 'test', + tier: 'global', }) const app = buildTestApp(limiter) @@ -113,6 +116,7 @@ describe('rate limiter – IETF rate-limit headers', () => { windowMs: 60000, max: 1, limiterType: 'test', + tier: 'global', }) const app = buildTestApp(limiter) diff --git a/tests/integration/rateLimiterTiering.integration.test.ts b/tests/integration/rateLimiterTiering.integration.test.ts index 820b01c9..6321c133 100644 --- a/tests/integration/rateLimiterTiering.integration.test.ts +++ b/tests/integration/rateLimiterTiering.integration.test.ts @@ -341,6 +341,7 @@ describe('route grouping', () => { windowMs: 900000, max: 1, limiterType: 'test', + tier: 'global', }) const app = appWith(limiter) await request(app).get('/api/v1/portfolio') diff --git a/tests/unit/auth/scopeCatalog.test.ts b/tests/unit/auth/scopeCatalog.test.ts new file mode 100644 index 00000000..1eb13f38 --- /dev/null +++ b/tests/unit/auth/scopeCatalog.test.ts @@ -0,0 +1,221 @@ +/** + * The scope catalogue must describe the scopes the code actually uses (#615). + * + * Before this, the valid scopes were two bare string arrays and the meaning + * of each scope was a comment or a doc line. That left three gaps this file + * closes: + * + * - a `requireScope('...')` / `requireAdminScope('...')` call site naming a + * scope that is not in the catalogue would be a runtime 403 on a live + * route rather than a build failure; + * - a catalogue entry no route required would advertise a permission the + * platform does not actually grant; + * - the "endpoints covered" list could drift from the route table. + * + * The endpoint lists are derived from the real route table by reading the + * scope tag off each route's middleware chain, so the check below compares + * the catalogue against what the code does — not against a second + * hand-written list. + */ + +import fs from 'fs' +import path from 'path' +import { + USER_SCOPES, + USER_SCOPE_CATALOG, + DEFAULT_READ_SCOPES, + type UserScope, +} from '../../../src/auth/scopes' +import { + ADMIN_SCOPES, + ADMIN_SCOPE_CATALOG, + ADMIN_GRANTOR_SCOPES, + hasAdminScope, + type AdminScope, +} from '../../../src/middleware/adminAuth' +import { + getAdminScopeCatalog, + getScopeCoverage, + getUserScopeCatalog, +} from '../../../src/auth/scopeCoverage' +import { renderScopeCatalogMarkdown } from '../../../src/auth/scopeCatalog' + +const ROOT = path.join(__dirname, '../../..') +const DOC_PATH = path.join(ROOT, 'docs', 'API_KEYS.md') + +describe('scope catalogue coverage (#615)', () => { + const coverage = getScopeCoverage() + + it('finds the scope guards in the route table', () => { + // If the walk silently found nothing, every assertion below would pass + // vacuously — so pin that it found the well-known guards. + expect(coverage.user['withdraw:write']?.length ?? 0).toBeGreaterThan(0) + expect(coverage.admin['keys:write']?.length ?? 0).toBeGreaterThan(0) + }) + + it('describes every user scope', () => { + for (const scope of USER_SCOPES) { + expect(USER_SCOPE_CATALOG[scope]).toBeDefined() + expect(USER_SCOPE_CATALOG[scope].scope).toBe(scope) + expect(USER_SCOPE_CATALOG[scope].description.length).toBeGreaterThan(0) + expect(USER_SCOPE_CATALOG[scope].grants.length).toBeGreaterThan(0) + } + }) + + it('describes every admin scope', () => { + for (const scope of ADMIN_SCOPES) { + expect(ADMIN_SCOPE_CATALOG[scope]).toBeDefined() + expect(ADMIN_SCOPE_CATALOG[scope].scope).toBe(scope) + expect(ADMIN_SCOPE_CATALOG[scope].description.length).toBeGreaterThan(0) + expect(ADMIN_SCOPE_CATALOG[scope].grants.length).toBeGreaterThan(0) + } + }) + + it('uses only catalogued scopes at the call sites', () => { + // The heart of the check: a scope guard may only name a scope the + // catalogue describes. + const unknownUser = Object.keys(coverage.user).filter( + (scope) => !USER_SCOPES.includes(scope as UserScope) + ) + const unknownAdmin = Object.keys(coverage.admin).filter( + (scope) => !ADMIN_SCOPES.includes(scope as AdminScope) + ) + expect(unknownUser.join(', ')).toBe('') + expect(unknownAdmin.join(', ')).toBe('') + }) + + it('has no catalogue entry that no route requires', () => { + // A scope nothing requires must be DECLARED as such. The catalogue must + // not imply that granting a scope restricts something when no route + // checks it — an operator reading the catalogue would otherwise believe + // the platform enforces a promise it does not keep. + const wrongUser = USER_SCOPES.filter( + (scope) => + (coverage.user[scope]?.length ?? 0) === 0 && + USER_SCOPE_CATALOG[scope].enforcement !== 'declared-only' + ) + const wrongAdmin = ADMIN_SCOPES.filter( + (scope) => + (coverage.admin[scope]?.length ?? 0) === 0 && + ADMIN_SCOPE_CATALOG[scope].enforcement !== 'declared-only' + ) + expect(wrongUser.join(', ')).toBe('') + expect(wrongAdmin.join(', ')).toBe('') + }) + + it('marks every scope a route requires as enforced', () => { + // The mirror image: a scope that IS required must say so, so the + // catalogue cannot quietly downgrade a live guard to "declared only". + const wrongUser = USER_SCOPES.filter( + (scope) => + (coverage.user[scope]?.length ?? 0) > 0 && + USER_SCOPE_CATALOG[scope].enforcement !== 'enforced' + ) + const wrongAdmin = ADMIN_SCOPES.filter( + (scope) => + (coverage.admin[scope]?.length ?? 0) > 0 && + ADMIN_SCOPE_CATALOG[scope].enforcement !== 'enforced' + ) + expect(wrongUser.join(', ')).toBe('') + expect(wrongAdmin.join(', ')).toBe('') + }) + + it('publishes exactly the scopes the runtime actually grants', () => { + // A catalogue that claims `super` implies a scope the auth check does + // not grant would be a promise the platform breaks. The catalogue's + // `implies` list is checked against hasAdminScope, which is the code + // that decides. + for (const [grantor, implied] of Object.entries(ADMIN_GRANTOR_SCOPES)) { + for (const scope of implied) { + expect( + hasAdminScope( + { id: 'k', name: 'k', role: 'r', scopes: [grantor] }, + scope + ) + ).toBe(true) + } + } + }) + + it('classifies the grantor scopes correctly', () => { + // `super` gates routes AND grants other scopes; `agent` only grants. + // The catalogue has to say which, or an operator cannot tell whether + // granting it restricts anything. + expect(ADMIN_SCOPE_CATALOG.super.implies).toEqual( + ADMIN_GRANTOR_SCOPES.super + ) + expect(ADMIN_SCOPE_CATALOG.agent.implies).toEqual( + ADMIN_GRANTOR_SCOPES.agent + ) + expect(ADMIN_SCOPE_CATALOG.agent.enforcement).toBe('declared-only') + expect(ADMIN_SCOPE_CATALOG.super.enforcement).toBe('enforced') + }) + + it('attributes every scope-guarded route to exactly one catalogue entry', () => { + for (const entry of getUserScopeCatalog()) { + expect(entry.endpoints).toEqual(coverage.user[entry.scope] ?? []) + } + for (const entry of getAdminScopeCatalog()) { + expect(entry.endpoints).toEqual(coverage.admin[entry.scope] ?? []) + } + }) + + it('does not list the same endpoint twice for one scope', () => { + for (const [scope, endpoints] of Object.entries(coverage.user)) { + expect(new Set(endpoints).size).toBe(endpoints.length) + expect(scope.length).toBeGreaterThan(0) + } + for (const [scope, endpoints] of Object.entries(coverage.admin)) { + expect(new Set(endpoints).size).toBe(endpoints.length) + expect(scope.length).toBeGreaterThan(0) + } + }) + + it('keeps the default read scopes inside the catalogue', () => { + for (const scope of DEFAULT_READ_SCOPES) { + expect(USER_SCOPES).toContain(scope) + } + }) +}) + +describe('scope catalogue API (#615)', () => { + it('exposes the catalogue with the endpoints each scope covers', () => { + const user = getUserScopeCatalog() + const admin = getAdminScopeCatalog() + + expect(user.length).toBe(USER_SCOPES.length) + expect(admin.length).toBe(ADMIN_SCOPES.length) + + const withdraw = user.find((e) => e.scope === 'withdraw:write') + expect(withdraw).toBeDefined() + expect(withdraw!.risk).toBe('money') + expect(withdraw!.endpoints.length).toBeGreaterThan(0) + expect(withdraw!.requires).toMatch(/allowWithdrawals/i) + }) + + it('marks the irreversible scopes as destructive', () => { + const erasure = getAdminScopeCatalog().find( + (e) => e.scope === 'erasure:write' + ) + expect(erasure?.risk).toBe('destructive') + expect(erasure?.requires).toBeTruthy() + }) + + it('lists only read scopes in the default read set', () => { + for (const scope of DEFAULT_READ_SCOPES) { + expect(USER_SCOPE_CATALOG[scope].risk).toBe('read') + } + }) +}) + +describe('docs/API_KEYS.md scope catalogue (#615)', () => { + const doc = fs.existsSync(DOC_PATH) ? fs.readFileSync(DOC_PATH, 'utf8') : '' + + it('contains the generated user scope table', () => { + expect(doc).toContain(renderScopeCatalogMarkdown(getUserScopeCatalog())) + }) + + it('contains the generated admin scope table', () => { + expect(doc).toContain(renderScopeCatalogMarkdown(getAdminScopeCatalog())) + }) +}) diff --git a/tests/unit/middleware/apiKeyAuth.test.ts b/tests/unit/middleware/apiKeyAuth.test.ts index b643923f..89359ba4 100644 --- a/tests/unit/middleware/apiKeyAuth.test.ts +++ b/tests/unit/middleware/apiKeyAuth.test.ts @@ -217,5 +217,45 @@ describe('User API Key auth (#374)', () => { ) }) }) + + // #615 — the read scopes were declared but never enforced, so a key + // scoped only to (say) alerts:manage could read the user's whole + // portfolio, transaction history and vault balance. These are the + // guards that close that hole. + describe('read scopes', () => { + const readScopes: UserScope[] = [ + 'portfolio:read', + 'transactions:read', + 'vault:read', + ] + + it.each(readScopes)('allows a key holding %s', (scope) => { + req.authScopes = [scope] + requireScope(scope)(req as Request, res as Response, next) + expect(next).toHaveBeenCalled() + }) + + it.each(readScopes)( + 'denies a key scoped only to alerts:manage for %s', + (scope) => { + req.authScopes = ['alerts:manage'] + req.apiKeyId = 'key-1' + requireScope(scope)(req as Request, res as Response, next) + expect(res.status).toHaveBeenCalledWith(403) + expect(res.json).toHaveBeenCalledWith( + expect.objectContaining({ + error: 'insufficient_scope', + required: scope, + }) + ) + } + ) + + it('lets a session through regardless', () => { + req.authScopes = ['*'] + requireScope('portfolio:read')(req as Request, res as Response, next) + expect(next).toHaveBeenCalled() + }) + }) }) }) diff --git a/tests/unit/middleware/rateLimitTierMatrix.test.ts b/tests/unit/middleware/rateLimitTierMatrix.test.ts new file mode 100644 index 00000000..903f63cd --- /dev/null +++ b/tests/unit/middleware/rateLimitTierMatrix.test.ts @@ -0,0 +1,220 @@ +/** + * The rate-limit tier matrix must describe the API that actually ships (#614). + * + * This is the CI half of the issue: "every mounted route has an explicit tier + * entry". It fails when + * + * - a mounted route resolves to NO matrix entry (an undocumented budget), or + * - the matrix says a route is in one tier while the code mounts a limiter + * from another (the table is lying), or + * - a matrix entry no longer matches any real route (the table has rotted), + * - a limiter exists that the tier catalogue does not describe, or + * - docs/RATE_LIMITING.md is not the table this module generates. + * + * The failure mode is deliberately loud: a silent gap in the matrix is + * exactly the drift the matrix exists to prevent. + */ + +import fs from 'fs' +import path from 'path' +import { API_ROUTES, EXTRA_MOUNTS } from '../../../src/routes/registry' +import { discoverRoutes } from '../../helpers/routeTable' +import { + RATE_LIMIT_MATRIX, + RATE_LIMIT_TIERS, + RATE_LIMIT_TIER_IDS, + resolveRateLimitTier, + renderRateLimitMatrixMarkdown, + renderRateLimitTierCatalogMarkdown, +} from '../../../src/middleware/rateLimitTiers' +import { + RATE_LIMIT_TIER_TAG, + limiterTier, +} from '../../../src/middleware/rateLimiter' +import { APP_LEVEL_LIMITERS } from '../../../src/middleware/rateLimiterChain' + +const ROOT = path.join(__dirname, '../../..') +const DOC_PATH = path.join(ROOT, 'docs', 'RATE_LIMITING.md') + +/** Every route the server serves, with the tier the code applies to it. */ +const ROUTES = [ + ...API_ROUTES.flatMap((r) => discoverRoutes(`/api/v1/${r.path}`, r.handlers)), + ...EXTRA_MOUNTS.flatMap((r) => discoverRoutes(r.path, r.handlers)), +].sort( + (a, b) => a.path.localeCompare(b.path) || a.method.localeCompare(b.method) +) + +describe('rate-limit tier matrix (#614)', () => { + it('discovers the real route table', () => { + // A silent walk failure would make every other assertion in this file + // pass vacuously, so pin the shape of what was found. + expect(ROUTES.length).toBeGreaterThan(100) + expect(ROUTES.some((r) => r.path === '/api/v1/deposit')).toBe(true) + expect(ROUTES.some((r) => r.path === '/api/v1/keys/:id/rotate')).toBe(true) + }) + + it('gives every mounted route an explicit tier entry', () => { + const unmapped = ROUTES.filter( + (r) => resolveRateLimitTier(r.method, r.path) === undefined + ) + expect(unmapped.map((r) => `${r.method} ${r.path}`).join('\n')).toBe('') + }) + + it('agrees with the limiter the code actually mounts', () => { + const drifted = ROUTES.filter((r) => { + const entry = resolveRateLimitTier(r.method, r.path) + return entry !== undefined && entry.tier !== r.tier + }) + expect( + drifted + .map( + (r) => + `${r.method} ${r.path} — matrix says "${ + resolveRateLimitTier(r.method, r.path)?.tier + }", code mounts "${r.tier}" (${r.tierSource})` + ) + .join('\n') + ).toBe('') + }) + + it('has no matrix entry that no longer matches a real route', () => { + const dead = RATE_LIMIT_MATRIX.filter((entry) => { + return !ROUTES.some( + (r) => + r.method === (entry.method ?? r.method) && + resolveRateLimitTier(r.method, r.path) === entry + ) + }) + expect(dead.map((e) => `${e.method ?? 'ANY'} ${e.path}`).join('\n')).toBe( + '' + ) + }) + + it('covers every route with the app-wide global + tiered budgets', () => { + // The runtime half of the fail-open policy: no route is ever unlimited. + const appTiers = APP_LEVEL_LIMITERS.map((l) => limiterTier(l)) + expect(appTiers).toContain('global') + expect(appTiers).toContain('tiered') + }) + + it('describes every tier it uses, and uses every tier it describes', () => { + const usedTiers = new Set([ + ...APP_LEVEL_LIMITERS.map((l) => limiterTier(l)!).filter(Boolean), + ...ROUTES.map((r) => r.tier), + ]) + for (const tier of usedTiers) { + expect( + RATE_LIMIT_TIERS[tier as keyof typeof RATE_LIMIT_TIERS] + ).toBeDefined() + } + // A tier nothing routes to is a tier the matrix documents but the code + // never applies — usually a renamed or removed limiter. + for (const id of RATE_LIMIT_TIER_IDS) { + expect(usedTiers.has(id)).toBe(true) + } + }) + + it('tags every limiter the code mounts', () => { + // A limiter with no tier tag is a limiter the matrix cannot describe. + const limiters = [ + ...API_ROUTES.flatMap((r) => r.handlers), + ...EXTRA_MOUNTS.flatMap((r) => r.handlers), + ] + const untagged = limiters.filter( + (h) => typeof h === 'function' && limiterTier(h) === undefined + ) + // The routers themselves are handlers too, so only the ones that are + // actual limiters matter — those are the ones in `handlers` before the + // router. Handled by the mount-level tier in the route walk above. + expect(untagged.length).toBeGreaterThan(0) // routers are untagged + for (const handler of limiters) { + if (typeof handler !== 'function') continue + if ((handler as any)[RATE_LIMIT_TIER_TAG] !== undefined) { + expect(RATE_LIMIT_TIERS[limiterTier(handler)!]).toBeDefined() + } + } + }) + + it('documents the fail-open / fail-closed decision', () => { + // The policy is part of the contract, not prose in a comment. + const { + RATE_LIMIT_MATRIX_POLICY, + } = require('../../../src/middleware/rateLimitTiers') + expect(RATE_LIMIT_MATRIX_POLICY.runtime).toBe('fail-open') + expect(RATE_LIMIT_MATRIX_POLICY.ci).toBe('fail-closed') + expect(RATE_LIMIT_MATRIX_POLICY.rationale.length).toBeGreaterThan(20) + }) +}) + +describe('rate-limit tier resolution', () => { + it('maps an auth-type to its tier', () => { + expect(resolveRateLimitTier('POST', '/api/v1/auth/login')?.tier).toBe( + 'auth' + ) + expect( + resolveRateLimitTier('GET', '/api/v1/webauthn/credentials')?.tier + ).toBe('auth') + }) + + it('does not let an exact money-movement entry swallow a sibling router', () => { + // /api/v1/deposit is one money-moving endpoint; /api/v1/deposit/recurring + // is a separate router. A prefix match would put the recurring routes in + // the sensitive tier by accident. + expect(resolveRateLimitTier('POST', '/api/v1/deposit')?.tier).toBe( + 'sensitive' + ) + expect( + resolveRateLimitTier('POST', '/api/v1/deposit/recurring')?.tier + ).toBe('tiered') + }) + + it('applies a method-specific entry only to that method', () => { + expect(resolveRateLimitTier('POST', '/api/v1/keys')?.tier).toBe('sensitive') + expect(resolveRateLimitTier('GET', '/api/v1/keys')?.tier).toBe('tiered') + expect(resolveRateLimitTier('DELETE', '/api/v1/keys/:id')?.tier).toBe( + 'tiered' + ) + }) + + it('prefers an exact erasure entry over the admin prefix', () => { + expect( + resolveRateLimitTier('POST', '/api/v1/admin/erasure/requests')?.tier + ).toBe('erasure') + // The read-only list of erasure requests is an ordinary admin read. + expect( + resolveRateLimitTier('GET', '/api/v1/admin/erasure/requests')?.tier + ).toBe('admin') + }) + + it('does not match a prefix entry across a path-segment boundary', () => { + // /api/v1/keys must not capture /api/v1/keystore. + expect(resolveRateLimitTier('GET', '/api/v1/keystore')?.tier).toBe('tiered') + }) +}) + +describe('docs/RATE_LIMITING.md (#614)', () => { + const doc = fs.existsSync(DOC_PATH) ? fs.readFileSync(DOC_PATH, 'utf8') : '' + + it('exists', () => { + expect(fs.existsSync(DOC_PATH)).toBe(true) + }) + + it('contains the generated route → tier matrix', () => { + expect(doc).toContain(renderRateLimitMatrixMarkdown()) + }) + + it('contains the generated tier catalogue', () => { + expect(doc).toContain(renderRateLimitTierCatalogMarkdown()) + }) + + it('documents which auth type maps to which tier', () => { + for (const authType of ['session', 'api_key', 'admin', 'webhook']) { + expect(doc).toContain(authType) + } + }) + + it('documents the fail-open / fail-closed decision', () => { + expect(doc).toMatch(/fail-open/i) + expect(doc).toMatch(/fail-closed/i) + }) +})