Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 64 additions & 7 deletions backend/SEP0001_GENERATOR_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ curl https://api.example.com/.well-known/stellar.toml?merchant_id=550e8400-e29b-
NETWORK_PASSPHRASE = "Test SDF Network ; September 2015"
TRANSFER_SERVER = "https://api.example.com/api"
DOCUMENTATION = "https://api.example.com/api-docs"
ACCOUNTS = ["GBUQWP3BOUZX34ULNQG23RQ6F4YUSXHTQSXUSMIQSTBE2BRUY4DQAT2B"]
ACCOUNTS = ["GDVEU3DD4KOFECV66VIHWEZOYX4ZKR3WV27L464SIIPOU2IUI3JCZA57"]

[ORG]
name = "Acme Payment Services"
Expand All @@ -56,7 +56,7 @@ The stellar.toml is generated from:
- `API_BASE_URL`: Base URL of the API (default: "http://localhost:4000")
- `FEDERATION_SERVER_URL`: Optional federation server
- `DOCS_URL`: Documentation URL (default: `{API_BASE_URL}/api-docs`)
- `SIGNING_KEY`: Optional signing key for transactions
- `SIGNING_KEY`: Optional **public** signing key (`G...`). A secret seed (`S...`) is never published.

2. **Merchant Settings** (from database):
- `business_name`: Organization name
Expand All @@ -79,7 +79,7 @@ API_BASE_URL="https://api.example.com"
TRANSFER_SERVER_URL="https://api.example.com/api"
FEDERATION_SERVER_URL="https://federation.example.com"
DOCS_URL="https://docs.example.com"
SIGNING_KEY="GBUQWP3BOUZX34ULNQG23RQ6F4YUSXHTQSXUSMIQSTBE2BRUY4DQAT2B"
SIGNING_KEY="GDVEU3DD4KOFECV66VIHWEZOYX4ZKR3WV27L464SIIPOU2IUI3JCZA57"
```

## Merchant Configuration
Expand All @@ -92,7 +92,7 @@ Merchants can configure their stellar.toml information through:
- `notification_email`: Support email

2. **Stellar Account** (set during registration):
- `recipient`: Your Stellar account address (e.g., `GBUQWP3BOUZX34ULNQG23RQ6F4YUSXHTQSXUSMIQSTBE2BRUY4DQAT2B`)
- `recipient`: Your Stellar account address (e.g., `GDVEU3DD4KOFECV66VIHWEZOYX4ZKR3WV27L464SIIPOU2IUI3JCZA57`)

3. **Branding Configuration** (via merchant settings):
- `branding_config.homepage`: Your website URL
Expand Down Expand Up @@ -129,13 +129,13 @@ curl -X POST https://api.example.com/api/merchant-branding \
- Example: `"https://federation.example.com"`

- **ACCOUNTS**: List of Stellar accounts operated by the merchant
- Example: `["GBUQWP3BOUZX34ULNQG23RQ6F4YUSXHTQSXUSMIQSTBE2BRUY4DQAT2B"]`
- Example: `["GDVEU3DD4KOFECV66VIHWEZOYX4ZKR3WV27L464SIIPOU2IUI3JCZA57"]`

- **DOCUMENTATION**: URL to API documentation
- Example: `"https://api.example.com/api-docs"`

- **SIGNING_KEY**: Public key for transaction signing
- Example: `"GBUQWP3BOUZX34ULNQG23RQ6F4YUSXHTQSXUSMIQSTBE2BRUY4DQAT2B"`
- Example: `"GDVEU3DD4KOFECV66VIHWEZOYX4ZKR3WV27L464SIIPOU2IUI3JCZA57"`

### Organization Section

Expand All @@ -151,10 +151,64 @@ The `[ORG]` section contains merchant information:

The stellar.toml is validated to ensure:

1. **Required fields present**: NETWORK_PASSPHRASE and TRANSFER_SERVER
1. **Required fields present**: NETWORK_PASSPHRASE and TRANSFER_SERVER, each at the start of a line
2. **Valid TOML syntax**: Proper formatting and escaping
3. **Valid URLs**: Properly formatted URLs
4. **Valid Stellar addresses**: Properly formatted account addresses
5. **Size**: the document is at most 100 KB (the SEP-0001 limit)

### Payload sanitization and strict validation (issue #1457)

Every value comes from the merchant record or server environment, and the
generator treats all of it as untrusted.

| Input | Rule | If invalid |
|---|---|---|
| `merchant_id` query param | Single value, must be a UUID | `400`. The database is not queried. |
| `business_name` | Bidi override characters (U+202A–202E, U+2066–2069, U+200E/F) stripped, trimmed, max 256 characters | Falls back to `Stellar Payment Merchant` when empty or not a string |
| All string values | TOML-escaped, including `\r`, `\t`, and every other control character (`\uXXXX`) | A value can never close its quotes and inject keys or `[TABLES]` |
| `recipient` → `ACCOUNTS` | Valid Stellar public key, StrKey checksum included | `ACCOUNTS` omitted |
| `SIGNING_KEY` | Valid Stellar **public** key | Omitted. A secret seed is never published and is logged as an error (without the value). |
| `email`, `notification_email` | Email syntax, max 254 characters | Field omitted |
| `branding_config.homepage` / `logo_url` | `https` URL, no credentials, no whitespace, max 2048 characters | Field omitted |
| `TRANSFER_SERVER_URL` / `API_BASE_URL` | `https` URL. `http` is allowed only for `localhost`/`127.0.0.1`. No credentials. | `500 Failed to generate valid stellar.toml`. Details are logged, never returned. |
| `FEDERATION_SERVER_URL`, `DOCS_URL` | Same URL rules | Field omitted |
| `STELLAR_NETWORK_PASSPHRASE` | Printable, max 128 characters | `500`, as above |

Omitted or altered fields are logged (`SEP-0001: fields omitted or altered by
sanitization`) and counted in `sep0001_rejected_fields_total{field,reason}`.

Responses also send `Access-Control-Allow-Origin: *`, which SEP-0001 requires
so browser wallets can read the file, and `X-Content-Type-Options: nosniff`.

## Monitoring (issue #1458)

Metrics (merged into `GET /metrics`):

| Metric | Labels | Meaning |
|---|---|---|
| `sep0001_requests_total` | `outcome` = `success` / `invalid_request` / `not_found` / `lookup_error` / `generation_error` | Requests by outcome |
| `sep0001_generation_duration_seconds` | `outcome` | Lookup + generation latency |
| `sep0001_document_size_bytes` | — | Served document size |
| `sep0001_rejected_fields_total` | `field`, `reason` | Sanitization rejections |
| `sep0001_health_state` | — | `0` healthy, `1` degraded, `2` unhealthy |
| `sep0001_error_ratio` | — | Rolling-window server-error ratio |
| `sep0001_last_success_timestamp_seconds` | — | Last successful response |

All label values come from fixed sets. Merchant ids never become labels.

Health: `GET /health/sep0001` returns the rolling-window snapshot (5 minutes by
default). It responds `503` only when unhealthy. `/health` also reports
`services.sep0001_generator` for information.

| Status | Condition (with at least `SEP0001_HEALTH_MIN_SAMPLES`, default 10) |
|---|---|
| `unhealthy` | server-error ratio (`lookup_error` + `generation_error`) ≥ `SEP0001_UNHEALTHY_ERROR_RATIO` (default 0.25) |
| `degraded` | server-error ratio ≥ `SEP0001_DEGRADED_ERROR_RATIO` (default 0.05), or a secret seed was refused as `SIGNING_KEY` in the window |

`400` and `404` responses are client errors and do not affect health. The
window is set by `SEP0001_HEALTH_WINDOW_MS`. Alerting rules are in
`docs/alerts/sep0001-generator.rules.yml`.

## Caching

Expand Down Expand Up @@ -267,6 +321,9 @@ console.log('Accounts:', toml.ACCOUNTS);
1. **Public Information**: stellar.toml is publicly accessible
- Don't include sensitive information
- Only include information you want to share publicly
- `SIGNING_KEY` must be the public key. If a secret seed is configured, the
generator refuses to publish it and `Sep0001SecretSigningKeyRefused` fires.
Treat that seed as exposed in config and rotate it.

2. **HTTPS Required**: Always use HTTPS for stellar.toml
- Wallets will reject HTTP endpoints
Expand Down
60 changes: 60 additions & 0 deletions backend/docs/IDEMPOTENCY_KEY_MANAGER.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# Idempotency Key Manager

`src/lib/idempotency.js` makes `POST` requests with an `Idempotency-Key`
header safe to retry. The first 2xx response is cached in Redis for 24 hours
under `idempotency:<merchant>:<key>` together with a SHA-256 hash of the
request body. A later request with the same key and the same body replays the
cached response (`201`). The same key with a different body gets a `400`.

## Retry with exponential backoff (issue #1464)

Every Redis call the manager makes goes through `withIdempotencyRetry()`
(`src/lib/idempotency-retry.js`):

| Operation label | Call | On final failure |
|---|---|---|
| `get` | middleware cache read | Request proceeds without idempotency (fail-open, unchanged) |
| `set` | caching the response | Logged. Runs in the background and never delays the response. |
| `lookup` | `lookupIdempotentResponse()` (payment session handler) | Error propagates to the caller, unchanged |

```
delay(attempt) = min(baseDelayMs * 2^attempt, maxDelayMs) * (1 ± jitterRatio)
```

Retries stop at `maxRetries`, or when the next delay would push total elapsed
time past `totalBudgetMs`, so a Redis outage adds bounded latency.

Only transient errors are retried: socket closed or reset, connect timeouts,
`ECONNRESET`/`ETIMEDOUT`/..., and the server replies `LOADING`, `BUSY`,
`TRYAGAIN`, `READONLY`, `MASTERDOWN` and `CLUSTERDOWN`. A closed client
(`redis.js` disables reconnects), auth/permission/`WRONGTYPE` errors and
corrupt cache JSON fail immediately.

| Env var | Default | Bounds |
|---|---|---|
| `IDEMPOTENCY_REDIS_MAX_RETRIES` | 2 | 0–5 |
| `IDEMPOTENCY_REDIS_RETRY_BASE_MS` | 25 | 1–1000 |
| `IDEMPOTENCY_REDIS_RETRY_MAX_MS` | 250 | base–5000 |
| `IDEMPOTENCY_REDIS_RETRY_BUDGET_MS` | 500 | 0–2000 |

Jitter is fixed at ±20%.

## Metrics

Exposed on `GET /metrics`, from their own registry:

| Metric | Labels |
|---|---|
| `idempotency_redis_operations_total` | `operation` (`get`/`set`/`lookup`), `outcome` (`success`/`recovered`/`failed`/`non_retryable`) |
| `idempotency_redis_retries_total` | `operation` |
| `idempotency_redis_retry_delay_seconds` | `operation` |

A rising `outcome="failed"` rate for `get` means idempotency is currently
failing open, so duplicate payments are possible. Alert on it together with
Redis availability.

## Tests

```bash
npx vitest run src/lib/idempotency-retry.test.js src/lib/idempotency.test.js
```
29 changes: 29 additions & 0 deletions backend/docs/WEBSOCKET_RELAY_TESTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# WebSocket Relay Service: integration and stress tests

Issue #1456.

| Suite | File | What it proves |
|---|---|---|
| Integration | `src/lib/websocket-relay.integration.test.js` | The real security, query, recovery and metrics modules wired as a relay worker: handshake (origin allow-list, HS256-pinned JWT, `alg: none`/expired/wrong-secret rejection), frame size limit, unknown/prototype-polluting field stripping, parameterised batch insert, FIFO dequeue, delivery through the circuit breaker, `markEventComplete`/`markEventFailed`, dead-lettering, HALF_OPEN recovery and DLQ replay, DB failure wrapping, pagination clamps, reconnect backoff, and audit log integrity |
| Stress | `src/lib/websocket-relay.stress.test.js` | 50 concurrent consumers drain 2,000 events with every event claimed exactly once; interleaved async latency never double-claims; a 1,000-row single-statement insert; 10,000 frames sanitized; oversized-frame floods; byte-accurate size checks for multi-byte text; 1,000 concurrent fast-fails on an open circuit; 10,000 mixed breaker outcomes; a 10,000-entry DLQ in FIFO order; 200 concurrent reconnect loops staying under the delay cap |

Only Postgres is faked. `tests/helpers/fake-relay-db.js` interprets the exact
SQL emitted by `websocket-relay-queries.js`. Each statement applies
atomically, mirroring `FOR UPDATE SKIP LOCKED` row locking, so the
exactly-once assertions test the claiming logic rather than the fake.

Timing assertions in the stress suite are deliberately loose. They catch
pathological regressions (quadratic or blocking behaviour), not small slowdowns.

## Finding fixed

`auditRelayEvent()` spread caller metadata **after** the fixed fields, so
metadata containing `audit`, `ts` or `event` (for example values echoed from
a client) could overwrite them and forge an audit line. Metadata is now
spread first.

## Running

```bash
npx vitest run src/lib/websocket-relay
```
63 changes: 63 additions & 0 deletions backend/docs/alerts/sep0001-generator.rules.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# Prometheus alerting rules for the SEP-0001 stellar.toml generator (issue #1458).
#
# Load with: rule_files: ["docs/alerts/sep0001-generator.rules.yml"]
# Validate: promtool check rules docs/alerts/sep0001-generator.rules.yml
#
# Metric reference: SEP0001_GENERATOR_GUIDE.md ("Monitoring")

groups:
- name: sep0001-generator
rules:
- alert: Sep0001GeneratorUnhealthy
expr: max(sep0001_health_state) >= 2
for: 5m
labels:
severity: critical
annotations:
summary: stellar.toml generation is failing
description: >
The rolling-window server error ratio is above the unhealthy
threshold. Wallets cannot discover TRANSFER_SERVER or verify
merchant accounts. See GET /health/sep0001 and logs for
"SEP-0001: failed to fetch merchant" / "invalid server configuration".

- alert: Sep0001GeneratorDegraded
expr: max(sep0001_health_state) == 1
for: 15m
labels:
severity: warning
annotations:
summary: stellar.toml generator degraded for 15 minutes
description: Elevated error ratio or a refused secret SIGNING_KEY. See GET /health/sep0001.

- alert: Sep0001SecretSigningKeyRefused
expr: sum(increase(sep0001_rejected_fields_total{field="SIGNING_KEY",reason="secret_seed"}[15m])) > 0
for: 0m
labels:
severity: critical
annotations:
summary: SIGNING_KEY is configured with a secret seed
description: >
The generator refused to publish it, but the seed is present in the
environment. Rotate the key and set SIGNING_KEY to the public key.

- alert: Sep0001InvalidConfiguration
expr: sum(increase(sep0001_requests_total{outcome="generation_error"}[10m])) > 0
for: 10m
labels:
severity: warning
annotations:
summary: stellar.toml cannot be generated
description: >
Usually TRANSFER_SERVER_URL / API_BASE_URL is not https or
STELLAR_NETWORK_PASSPHRASE is invalid. Check logs for
"SEP-0001: invalid server configuration".

- alert: Sep0001MerchantDataRejected
expr: sum by (field, reason) (increase(sep0001_rejected_fields_total{field!="SIGNING_KEY"}[1h])) > 20
for: 0m
labels:
severity: info
annotations:
summary: "Many stellar.toml {{ $labels.field }} values rejected ({{ $labels.reason }})"
description: Merchant records contain values that are being omitted from stellar.toml.
26 changes: 26 additions & 0 deletions backend/src/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ import { versionDeprecationMiddleware } from "./lib/version-deprecation.js";
import oracleRouter from "./routes/oracle.js";
import { getPaymentSessionValidatorHealth } from "./lib/payment-session-validator.js";
import { getExchangeRateOracleHealth } from "./lib/exchange-rate-oracle-telemetry.js";
import { getSep0001Health } from "./lib/sep0001-telemetry.js";
import { configureExchangeRateCoordination } from "./services/exchangeRateService.js";

export async function createApp({ redisClient }) {
Expand Down Expand Up @@ -262,6 +263,7 @@ export async function createApp({ redisClient }) {
// Informational only — does not affect `ok` / the status code.
payment_session_validator: getPaymentSessionValidatorHealth().status,
exchange_rate_oracle_cache: getExchangeRateOracleHealth().status,
sep0001_generator: getSep0001Health().status,
},
});
});
Expand Down Expand Up @@ -314,6 +316,30 @@ export async function createApp({ redisClient }) {
res.status(health.status === "unhealthy" ? 503 : 200).json(health);
});

/**
* @swagger
* /health/sep0001:
* get:
* summary: SEP-0001 stellar.toml generator health telemetry
* description: >
* Rolling-window request counts, server error ratio and derived status
* for GET /.well-known/stellar.toml (issue #1458). Returns 503 only
* when the generator is unhealthy. A degraded status (elevated error
* ratio, or a secret seed refused as SIGNING_KEY) still returns 200.
* The body contains no merchant ids or key material.
* tags: [Health]
* security: []
* responses:
* 200:
* description: Generator healthy or degraded
* 503:
* description: Generator unhealthy
*/
app.get("/health/sep0001", (_req, res) => {
const health = getSep0001Health();
res.status(health.status === "unhealthy" ? 503 : 200).json(health);
});

const verifyPaymentRateLimit = createVerifyPaymentRateLimit({
store: redisAvailable ? createRedisRateLimitStore({ client: redisClient }) : undefined,
});
Expand Down
Loading