Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 12 additions & 21 deletions .env.local.example
Original file line number Diff line number Diff line change
Expand Up @@ -209,24 +209,15 @@ NEXT_PUBLIC_CLASSIC_LIQ_ISSUER=
# NEXT_PUBLIC_FEATURE_PHASE_124=1
# FEATURE_PHASE_124=1

# Spikes 620/621/630/631. All off by default. Rebuild after public flag changes.
NEXT_PUBLIC_FEATURE_PHASE_174=0
FEATURE_PHASE_174=0
NEXT_PUBLIC_FEATURE_PHASE_175=0
FEATURE_PHASE_175=0
NEXT_PUBLIC_FEATURE_PHASE_176=0
FEATURE_PHASE_176=0
NEXT_PUBLIC_FEATURE_PHASE_177=0
FEATURE_PHASE_177=0
# Verified native app association identifiers; empty arrays are served until configured.
PHASE_ANDROID_PACKAGE=
PHASE_ANDROID_SHA256_CERTS=
PHASE_IOS_APP_ID=
# KYC adapter. Mock is accepted only on testnet. Never put these keys in NEXT_PUBLIC_*.
PHASE_KYC_MODE=off
PHASE_KYC_THRESHOLD_STROOPS=10000000000
PHASE_KYC_HASH_KEY=
PHASE_KYC_WEBHOOK_SECRET=
PHASE_SANCTIONS_URL=
PHASE_SANCTIONS_API_KEY=
PHASE_KYC_MOCK_BLOCKED_WALLETS=
# ─── Spikes 603–606 (phase-170..173) — default off ────────────────────────
# phase-170 log pipeline (https only). Secrets/seeds are redacted before shipping.
# LOG_SINK_URL=https://logs.example.com/ingest
# LOG_SINK_TOKEN=
# LOG_SAMPLE_RATE_INFO=0.1
# phase-172 enters dual-write only when this is set
# PHASE_DATABASE_URL=
# phase-173 production identity fingerprints (public/hashed only) for preview checks
# PHASE_PROD_ISSUER=G...
# PHASE_PROD_CRON_SECRET_SHA256=
# PHASE_PROD_CONTRACT_IDS=C...,C...
# PHASE_PREVIEW_CRON_ENABLED=1
9 changes: 4 additions & 5 deletions PROJECT_ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,11 +212,10 @@ is a fixed-size digest of the canonical payload, keeping it under wallet
| `phase-167` | `NEXT_PUBLIC_FEATURE_PHASE_167` / `FEATURE_PHASE_167` | WalletConnect v2 mobile session resilience (`lib/walletconnect-resilience.ts`): configurable relay URL, multi-chain support, session persistence, chain-switch guard, and mobile deep-link return. Spike 623 (issue #486) | off | Unset var, restart — WalletConnect banner and reconnect actions disabled, kit falls back to default single-session behavior |
| `phase-168` | `NEXT_PUBLIC_FEATURE_PHASE_168` / `FEATURE_PHASE_168` | xBull wallet full support and parity (`lib/xbull-adapter.ts`): Freighter-difference matrix, xBull module detection, sign result normalization, trustline error translation, and badge surfacing. Spike 624 (issue #487) | off | Unset var, restart — xBull transactions fall through generic kit handlers, badge hidden |
| `phase-169` | `NEXT_PUBLIC_FEATURE_PHASE_169` / `FEATURE_PHASE_169` | Ledger blind-signing to clear-signing review for Soroban (`lib/ledger-soroban-decoder.ts`): invokeHostFunction pre-image decode, verification code, on-device review modal, Gate #13 preemptive validation, and hardware error humanizer. Spike 625 (issue #488) | off | Unset var, restart — Ledger transactions fall back to default blind-signing flow without preflight review modal |
| `phase-174` | `NEXT_PUBLIC_FEATURE_PHASE_174` / `FEATURE_PHASE_174` | Mobile Forge/Chamber viewport, controls sheet, focus and touch layout (#483) | off | Unset both vars and restart; no stored layout state |
| `phase-175` | `NEXT_PUBLIC_FEATURE_PHASE_175` / `FEATURE_PHASE_175` | Validated inbound offer/quest/collection links with wallet-connect intent retention (#484) | off | Unset both vars and restart; session intent inert; association endpoints return 404 |
| `phase-176` | `NEXT_PUBLIC_FEATURE_PHASE_176` / `FEATURE_PHASE_176` | Amount-aware KYC and fresh sanctions adapter before credits (#493); additionally requires PHASE_KYC_MODE | off | Unset both vars and restart; additive credential tables remain inert |
| `phase-177` | `NEXT_PUBLIC_FEATURE_PHASE_177` / `FEATURE_PHASE_177` | Durable transaction evidence and signed CSV export (#494) | off | Unset both vars and restart; additive evidence remains inert; export returns 404 |

| `phase-170` | `NEXT_PUBLIC_FEATURE_PHASE_170` / `FEATURE_PHASE_170` | Structured log pipeline (`lib/log-pipeline.ts`): deep secret redaction, deterministic per-correlation-id info sampling (warn/error always kept), closed index-label set (`level`, `event`, `route`, `method`, `status_class`), and a bounded non-blocking HTTPS batcher (`LOG_SINK_URL`, https only). `api-observability` routes every line through it. Spike 606 (issue #470) — partial slice: no `waitUntil` flush on freeze, no counter export, ad-hoc `console.*` callers not migrated | off | Unset var, restart — `emitLog` is the identity and console lines are written exactly as before; in-memory buffer dropped |
| `phase-171` | `NEXT_PUBLIC_FEATURE_PHASE_171` / `FEATURE_PHASE_171` | Backup and DR snapshots (`lib/backup-snapshot.ts`): lock-consistent sidecar set + `VACUUM INTO` SQLite image, sha256/row-count manifest, `integrity_check`/`foreign_key_check` verify, tmp+rename restore with a restore marker that makes `/api/cron/faucet-health` return 503, and an AES-256-GCM sealed secrets envelope. `scripts/backup-drill.ts` runs snapshot→verify→restore→parity. Spike 605 (issue #469) — partial slice: no off-box shipping or scheduled snapshots | off | Unset var, restart — entry points throw `FLAG_DISABLED`, restore marker ignored; snapshots on disk are inert |
| `phase-172` | `NEXT_PUBLIC_FEATURE_PHASE_172` / `FEATURE_PHASE_172` | SQLite→Postgres rehearsal (`lib/db-parity.ts`): dialect helpers (`?`→`$n`, type map, explicit upsert), typed per-table parity (value, `type_coercion`, checksum), per-test schema names, and a `DualWriter` (primary first, bounded/timeout-guarded secondary, drift log, `primary-only` rollback). Needs `PHASE_DATABASE_URL` to enter `dual` mode. Spike 604 (issue #468) — partial slice: no Postgres driver, async CAS re-plumbing or pool storm test | off | Unset var, restart — `resolveDualWriteMode` returns `primary-only`; SQLite stays authoritative, nothing in production paths calls the module |
| `phase-173` | `NEXT_PUBLIC_FEATURE_PHASE_173` / `FEATURE_PHASE_173` | Vercel preview isolation (`lib/preview-isolation.ts`): per-preview data root (`<root>/previews/<pr-N or br-slug-hash>`), `validatePhaseEnv` fails a preview that shares the production issuer, cron secret, contract IDs, or runs mainnet (declared via `PHASE_PROD_ISSUER`, `PHASE_PROD_CRON_SECRET_SHA256`, `PHASE_PROD_CONTRACT_IDS`), preview cron skipped unless `PHASE_PREVIEW_CRON_ENABLED=1`, and `x-vercel-cron` no longer trusted in previews. Spike 603 (issue #467) — partial slice: no per-PR contract deploy, preview CI job, or branch-DB seeding | off | Unset var, restart — data root, cron auth and env validation revert; per-preview directories stay inert |



Expand Down
19 changes: 18 additions & 1 deletion app/api/cron/faucet-health/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ import { Keypair, StrKey } from "@stellar/stellar-sdk"
import { fetchDistributorBalance, distributorNeedsTopup } from "@/lib/distributor-topup"
import { sendWebhookAlert, WebhookAlertType } from "@/lib/webhook-alerts"
import { executeDistributorRefill } from "@/lib/distributor-refill"
import { isRestoreInProgress } from "@/lib/backup-snapshot"
import { isCronBlockedInPreview, previewTrustsVercelCronHeader } from "@/lib/preview-isolation"
import { incSecurityCounter } from "@/lib/security-counters"
import { getDistributorHealthStatus, recordHealthCheck } from "@/lib/distributor-health-store"

/**
Expand Down Expand Up @@ -44,7 +47,7 @@ function validateCronAuth(req: NextRequest): boolean {

// Vercel Cron sends this header
const vercelCronHeader = req.headers.get("x-vercel-cron")
if (vercelCronHeader) {
if (vercelCronHeader && previewTrustsVercelCronHeader()) {
return true
}

Expand Down Expand Up @@ -84,6 +87,20 @@ export async function GET(req: NextRequest) {
)
}

// phase-173: previews do not run scheduled refills against the shared issuer.
if (isCronBlockedInPreview()) {
incSecurityCounter("preview_cron_blocked")
return NextResponse.json({ ok: true, skipped: "preview_cron_disabled" })
}

// phase-171: a refill mid-restore would act on half-restored distributor state.
if (await isRestoreInProgress()) {
return NextResponse.json(
{ error: "Restore in progress" },
{ status: 503, headers: { "Retry-After": "60" } },
)
}

// Only run health checks in transfer mode
if (!faucetUsesDistributorTransfer()) {
return NextResponse.json({
Expand Down
48 changes: 41 additions & 7 deletions docs/TECHNICAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,6 +419,11 @@ NEXT_PUBLIC_FEATURE_PHASE_158=1
NEXT_PUBLIC_FEATURE_PHASE_159=1
NEXT_PUBLIC_FEATURE_PHASE_160=1
NEXT_PUBLIC_FEATURE_PHASE_161=1
# Spikes 603/604/605/606 (issues #467/#468/#469/#470)
NEXT_PUBLIC_FEATURE_PHASE_170=1
NEXT_PUBLIC_FEATURE_PHASE_171=1
NEXT_PUBLIC_FEATURE_PHASE_172=1
NEXT_PUBLIC_FEATURE_PHASE_173=1
# Server-only aliases also accepted: FEATURE_PHASE_104, etc.
```

Expand Down Expand Up @@ -695,17 +700,46 @@ Flag: `phase-169` (`NEXT_PUBLIC_FEATURE_PHASE_169` / `FEATURE_PHASE_169`). Defau
- Audit counters: `ledger_clear_signing_reviewed`, `ledger_clear_signing_approved`, `ledger_clear_signing_rejected`, `ledger_blind_signing_blocked`, `ledger_gate13_preempted` in `lib/security-counters.ts`.
- Routes: `POST /api/wallet/ledger` for clear-signing lifecycle telemetry.

## 20. Log pipeline: redaction, sampling and external sink (spike 606)

Flag: `phase-170` (`NEXT_PUBLIC_FEATURE_PHASE_170` / `FEATURE_PHASE_170`). Default off. See `PROJECT_ARCHITECTURE.md` §10 and `docs/spikes/606-log-pipeline-findings.md`.

- `lib/log-pipeline.ts`: `redactLogValue` (masks `S...` seeds and secret-named keys such as `rawPayload`, `errorResult`, `authorization`; cycle/depth safe), `shouldSampleLog` (warn/error always kept, info sampled by FNV hash of the correlation id so a request is kept or dropped whole), `toPipelineRecord` (only `level`, `event`, `route`, `method`, `status_class` are labels; wallets, CIDs, token ids, hashes stay unindexed in `fields`), and `LogPipeline` (bounded buffer, drop-oldest, single-flight flush, abort timeout, at-most-once delivery).
- `lib/api-observability.ts`: every `log()` goes through `emitLog`, so the console line is redacted too when the flag is on.
- Env: `LOG_SINK_URL` (https only, validated in `serverEnvSchema`), `LOG_SINK_TOKEN`, `LOG_SAMPLE_RATE_INFO` (0..1, default 0.1), `LOG_SINK_BATCH_SIZE`, `LOG_SINK_MAX_BUFFER`, `LOG_SINK_FLUSH_MS`, `LOG_SINK_TIMEOUT_MS`.
- Counters (`lib/security-counters.ts`): `log_pipeline_enqueued`, `_sampled_out`, `_dropped`, `_redacted`, `_flush_ok`, `_flush_failed`.
- Not covered: `waitUntil`-style flush on serverless freeze, counter export, migrating ad-hoc `console.*` callers.

## 21. Backup and disaster recovery (spike 605)

## Spikes 620/621/630/631 — mobile, inbound links, screening and CSV
Flag: `phase-171` (`NEXT_PUBLIC_FEATURE_PHASE_171` / `FEATURE_PHASE_171`). Default off. See `docs/spikes/605-backup-dr-findings.md`.

Flags `phase-174` through `phase-177` default off. Set literal public flags at build time for browser features; server-only flags cannot enable browser UI. Rollback requires unsetting both public/server flags and rebuilding/restarting. New SQLite tables are additive and stay inert when their features are off; do not delete reporting evidence during rollback. No contract deployment, token migration, issuer rotation or gateway change is part of these spikes.
- `lib/backup-snapshot.ts`: `createSnapshot` locks every sidecar path (sorted, via `withFileLock`) and takes the SQLite image with `VACUUM INTO`, so the set and the database share one logical instant and no WAL copy is involved; a corrupt sidecar fails the snapshot. `verifySnapshot` re-hashes, parses JSON, runs `integrity_check` and `foreign_key_check`, and compares per-table row counts. `restoreSnapshot` verifies first, removes stale `-wal`/`-shm`, writes tmp+rename, and holds `.restore-in-progress` (stale after 10 min).
- `app/api/cron/faucet-health/route.ts`: returns `503` + `Retry-After: 60` while the marker is fresh, so a refill cannot run against half-restored distributor state.
- Secrets: never written to a snapshot. `sealSecrets`/`openSecrets` use AES-256-GCM with a scrypt key (passphrase ≥ 16 chars); failures never echo plaintext; `diffSecretsMap` reports drift by name only.
- `lib/server-data-paths.ts`: `serverDataRoot`, `serverDataFileKeys`, `serverDataFileName` exported so backups cannot drift from the file table.
- Drill: `npm run backup:drill` (snapshot → verify → restore to scratch → SQLite parity).
- Restore replaces the database file: close the connection (`resetDbForTests`) and run offline for the live root.
- Counters: `backup_snapshot_created|verified|corrupt`, `backup_restore_completed|blocked`.

- Mobile layout: one responsive breakpoint at 768px; `useMobileViewport` publishes the actual viewport height and cleans up listeners. Radix sheets provide focus management and Escape handling; custom Forge/Chamber overlays trap phone focus and restore the opener. Pointer capture keeps drawing inside the canvas without stealing page scroll elsewhere.
- Link capture: `phase://quest/ID`, `stellar://phase/quest/ID`, and same-origin `/link/quest/ID` share a parser. Offer paths contain listing and offer IDs; collection IDs are positive safe integers. Arbitrary callbacks, other origins, duplicate parameters and invalid G addresses are rejected. Capture survives wallet connection in sessionStorage without initiating a claim or signing. Target actions revalidate server-side. Browser fallback `/link` accepts pasted custom URIs; SEP-7 `pay`/`tx` requests are deliberately rejected rather than reinterpreted as Soroban intents.
- Compliance: `PHASE_KYC_MODE=off|mock|live`, threshold in exact stroops. Each protected request screens against the configured HTTPS adapter (3s timeout, no redirect/cache). A result older than one hour or already expired fails closed. Webhook signing format is HMAC-SHA256(timestamp + '.' + raw UTF-8 body), with millisecond timestamps and 5-minute skew. Providers must bind subjects to wallets before invoking this operator adapter. `expectedVersion` starts at zero; stale callbacks return 409/current_version. 2FA and identity verification remain separate.
- Reporting: confirmed faucet/bootstrap receipts and unverified marketplace agreements enter `tax_events` with stable event keys. No historical basis/FX reconstruction is invented. Export requires a SEP-53 signature binding wallet, network, UTC range, policy, expiry and nonce. Private no-store responses carry a safe attachment filename and X-Tax-Snapshot. Each CSV record carries a chained recomputation hash. Pages use `(occurred_at, seq)` ordering at a frozen maximum sequence; same-time records cannot fall between pages. Wash/unverified records remain visibly flagged and excluded from proceeds/gains/income.
## 22. SQLite to Postgres migration rehearsal (spike 604)

API request/response details and outstanding live proof requirements are in [mobile integration API](api/mobile-integration.md) and [spike evidence](spikes/620-621-630-631-integration.md).
Flag: `phase-172` (`NEXT_PUBLIC_FEATURE_PHASE_172` / `FEATURE_PHASE_172`). Default off. See `docs/spikes/604-postgres-rehearsal-findings.md`.

- `lib/db-parity.ts`: `toPositionalParams`, `sqliteTypeToPostgres` (integer flags stay `BIGINT` so parity compares like with like), `buildPostgresCreateTable`, `buildPostgresUpsert` (explicit conflict target replaces `INSERT OR REPLACE`), `perTestSchemaName`, `compareSources`/`compareTable` (value diffs, `type_coercion`, order-independent checksum), `DualWriter`.
- Dual-write ordering: primary (SQLite) first and authoritative; secondary is bounded (`maxSecondaryInflight`, default 5) and timed out (1.5 s). Failures, timeouts and shed writes land in `driftLog()` and never throw. `setMode("primary-only")` is the rollback.
- `PHASE_DATABASE_URL` + flag → `dual`; otherwise `primary-only`.
- Check two databases with `npm run db:parity -- <left> <right>`.
- Counters: `db_parity_checked`, `db_parity_mismatch`, `db_dual_write_ok|drift|shed`.
- Not covered: a `pg` driver, async re-plumbing of the `MAX_CAS_ATTEMPTS` loops, advisory locks replacing `fileQueues`, pool storm testing.

## 23. Preview environment isolation (spike 603)

Flag: `phase-173` (`NEXT_PUBLIC_FEATURE_PHASE_173` / `FEATURE_PHASE_173`). Default off. See `docs/spikes/603-preview-isolation-findings.md`.

- `lib/preview-isolation.ts`: `resolvePreviewId` (`pr-<n>` from `VERCEL_GIT_PULL_REQUEST_ID`, else `br-<slug>-<hash>`; always one safe path segment), `checkPreviewIsolation`, `isCronBlockedInPreview`, `previewTrustsVercelCronHeader`. The module has no `node:*` imports because `lib/env-validation.ts` reaches client bundles through `phase-protocol.ts`.
- `lib/server-data-paths.ts`: in a preview the root becomes `<root>/previews/<id>`.
- `lib/env-validation.ts`: `validatePhaseEnv` reports `isolation_violation` for a preview that resolves to the production issuer (public key or derived from `CLASSIC_LIQ_ISSUER_SECRET`), reuses the production cron secret, uses a production contract ID, or sets `STELLAR_NETWORK=mainnet`. Production identity is declared with `PHASE_PROD_ISSUER`, `PHASE_PROD_CRON_SECRET_SHA256`, `PHASE_PROD_CONTRACT_IDS`; messages never contain secret values.
- `app/api/cron/faucet-health/route.ts`: preview runs return `{ skipped: "preview_cron_disabled" }` unless `PHASE_PREVIEW_CRON_ENABLED=1`; the unauthenticated-header shortcut is disabled in previews.
- Caveat: Vercel `/tmp` is per-instance, so namespacing prevents cross-preview bleed, not loss between instances.
- Counters: `preview_isolation_violation`, `preview_cron_blocked`.
Loading