Skip to content

feat(backend): async campaign exports, deposit watcher, fraud detection and API docs - #1397

Merged
joelpeace48-cell merged 1 commit into
FinesseStudioLab:mainfrom
leojay-net:feat/backend-export-worker-horizon-watcher-fraud-openapi
Sep 26, 2026
Merged

joelpeace48-cell merged 1 commit into
FinesseStudioLab:mainfrom
leojay-net:feat/backend-export-worker-horizon-watcher-fraud-openapi

Conversation

@leojay-net

@leojay-net leojay-net commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Backend work for the growth engine and analytics area, one feature per issue. Details are in
docs/BACKEND_ANALYTICS_AND_GROWTH.md.

Campaign export analytics worker

  • POST /api/v1/campaigns/:id/exports (format csv/json, optional from/to) creates an export_jobs row, queues it and returns 202 with a jobId and statusUrl, so heavy exports no longer run inside the HTTP request.
  • GET /campaigns/:id/exports/:jobId reports queued → running → completed | failed (with downloadUrl when done); GET /campaigns/:id/exports lists recent jobs.
  • The job runs in jobs/campaignExportWorker.js on the existing durable job queue (retries with backoff, dead-lettering) and uploads through the configured storage adapter. Redelivery of a finished job is a no-op.
  • The participant query and CSV serializer moved to services/campaignExportData.js and are shared with the existing synchronous GET /campaigns/:id/export endpoint, which behaves as before.
  • Note: the issue mentions BullMQ. The backend already ships a SQLite-backed durable queue with retries and a dead-letter store, so I used that instead of adding a new Redis-based dependency; the worker only depends on queue.enqueue, so it can be moved to BullMQ later without changing the API.

Stellar Horizon account balance watcher

  • services/horizonDepositWatcher.js streams payments (Horizon SSE) for each address in OPERATOR_DEPOSIT_ACCOUNTS and records incoming deposits of the reward asset (REWARD_TOKEN_CODE / REWARD_TOKEN_ISSUER, default XLM). It also handles create_account and path payments, and ignores outgoing payments and transfers between watched accounts.
  • Deposits are idempotent (unique Horizon paging token), stream cursors are persisted so a restart resumes instead of missing deposits, and dropped streams reconnect with exponential backoff.
  • Master-key endpoints: GET /api/v1/operator/deposits, GET /operator/deposits/:txHash (verify a deposit by transaction hash) and GET /operator/watcher/status. The watcher only starts when accounts are configured and jobs are enabled.

OpenAPI documentation generator and Swagger UI

  • Swagger UI at GET /docs/api and the generated document at GET /docs/api/openapi.json.
  • The document is built from openapi.yaml (always wins), component schemas derived from the Zod request schemas in schemas.js (so documented bodies match runtime validation), and path items for the router-mounted endpoints added here. It is validated with the OpenAPI parser and openapi.yaml itself is unchanged, so the generated SDK is unaffected. The existing /docs UI keeps working.

Automated fraud detection for anomalous IP clusters

  • Each referral signup records the network prefix of the request IP (/24 IPv4, /64 IPv6 — only the prefix is stored). When distinct accounts from one prefix reach FRAUD_SUBNET_THRESHOLD within FRAUD_WINDOW_HOURS, or accounts share an address on FRAUD_PROXY_EXIT_LIST (IPs or IPv4 CIDRs), a flag is raised.
  • Detection is advisory and never blocks or fails a signup. A dismissed flag reopens if the cluster grows.
  • Master-key review API: GET /api/v1/admin/fraud/flags and PATCH /admin/fraud/flags/:id.

Also adds migrations 041–043, the new env vars to backend/.env.example, and a changelog entry.

Testing

node --test src/services/fraudDetection.test.js src/services/horizonDepositWatcher.test.js \
  src/services/openapiGenerator.test.js src/jobs/campaignExportWorker.test.js \
  src/routes/campaignExportJobs.test.js src/routes/backendGrowthFeatures.test.js   # 50 tests
node --test src/integration/openapi-contract.test.js     # 8 passing
node --test src/tests/api-snapshot.test.js               # 20 passing
npx eslint <changed files>                               # clean
npx prettier --check <changed files>                     # clean

The new tests cover the subnet/proxy rules and windowing, deposit parsing/idempotency/cursor
resume/reconnect, the export worker (success, retry, failure, redelivery) and its HTTP flow, and
the generated docs endpoints. routes/campaignExport.test.js has 3 failing tests that are
identical with and without this change.

Closes #1260
Closes #1261
Closes #1259
Closes #1258

…er and API docs

- Campaign export worker: POST /campaigns/:id/exports queues a CSV/JSON export on the
  durable job queue and returns 202; poll /campaigns/:id/exports/:jobId for status and the
  download URL. Export queries are shared with the synchronous endpoint.
- Horizon deposit watcher: streams payments for OPERATOR_DEPOSIT_ACCOUNTS, records incoming
  reward-token deposits idempotently, persists cursors and reconnects with backoff.
- Fraud detection: flags clusters of distinct accounts signing up from one IP subnet or
  proxy exit node, with an admin review API.
- API docs: OpenAPI document generated from openapi.yaml plus the Zod request schemas,
  served with Swagger UI at /docs/api.
@drips-wave

drips-wave Bot commented Sep 26, 2026

Copy link
Copy Markdown

@leojay-net Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

handleRecord(record, account);
} catch (err) {
metrics.errors++;
logger.error?.({ err, account }, 'depositWatcher:handle_error');
onerror: (/** @type {unknown} */ err) => {
metrics.errors++;
state.connected = false;
logger.warn?.({ err, account }, 'depositWatcher:stream_error');
});
} catch (err) {
metrics.errors++;
logger.error?.({ err, account }, 'depositWatcher:connect_failed');
@joelpeace48-cell
joelpeace48-cell merged commit d5284bd into FinesseStudioLab:main Sep 26, 2026
9 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants