Skip to content

feat(backend): readiness probes, IPFS pinning, GitHub task verification and live leaderboard - #1398

Merged
joelpeace48-cell merged 2 commits into
FinesseStudioLab:mainfrom
zerroai:feat/backend-ipfs-pin-github-webhook-probes-leaderboard-ws
Sep 26, 2026
Merged

joelpeace48-cell merged 2 commits into
FinesseStudioLab:mainfrom
zerroai:feat/backend-ipfs-pin-github-webhook-probes-leaderboard-ws

Conversation

@zerroai

@zerroai zerroai commented Sep 26, 2026

Copy link
Copy Markdown
Contributor

Summary

Four backend features, one per issue. Details, endpoints and env vars are in
docs/BACKEND_PROBES_IPFS_GITHUB_LEADERBOARD.md.

Health check and readiness endpoints with RPC status

  • /readyz (and /health/ready) now runs the dependency checks in parallel, each under a timeout (READINESS_TIMEOUT_MS, default 2s): the database (SELECT 1), Redis (PING, only when configured) and the Soroban RPC. The response lists each check's status, latency and error.
  • Required checks failing (or shutdown in progress) returns 503. The database and Redis are required. The RPC is reported but only required with READINESS_REQUIRE_RPC=true; otherwise a failure yields "status": "degraded" with 200, so an RPC blip does not take every pod out of rotation.
  • /healthz now returns uptime and the RPC pool state (cached, no network call); /livez is unchanged. A hanging dependency cannot hang the probe.

Automated IPFS metadata pin service

  • services/ipfsPinService.js pins a campaign's image, rules JSON and badge JSON, then a metadata document linking them by ipfs:// CID, through Pinata (PINATA_JWT). Pins are content-addressed and de-duplicated per campaign (ipfs_pins), and transient provider errors (network, 429, 5xx) are retried with backoff.
  • POST /campaigns/:id/ipfs-pins and GET /campaigns/:id/ipfs-pins (API key). New campaigns are also pinned in the background through the durable job queue when the service is configured.
  • Image download is restricted to public https hosts (no private/internal addresses, no redirects), image/* content and 5 MiB, so campaign data cannot be used to make the server fetch internal URLs. Web3.Storage is not implemented; the provider interface is small (pinJson, pinFile) if you want to add it.

GitHub webhook listener for bounty task verification

  • POST /api/v1/webhooks/github verifies X-Hub-Signature-256 in constant time over the raw body (mounted before express.json()), then records a task only when GitHub reports a pull request merged or an issue closed as completed (not_planned and unmerged closes are ignored). Deliveries are idempotent (github_task_events). ping is answered; other events return 202.
  • Master-key endpoints GET /admin/github-tasks and GET /admin/github-tasks/verify?githubLogin=&repo=&number=&kind= let reward logic confirm a task before crediting. This PR records and exposes verification; it does not itself credit on-chain rewards.

Real-time WebSocket gateway for live leaderboard updates

  • Clients subscribe with { "type": "subscribe", "channel": "leaderboard", "campaignId": "42" }, receive a leaderboard_snapshot, then leaderboard_update messages (top 10 plus per-entry changes: new / up / down / same) whenever awarded points change the ranking. Bursts are coalesced (LEADERBOARD_DEBOUNCE_MS), nothing is sent when the ranking did not change, and wallets are masked.
  • Points are awarded today via referral credits, which is the per-campaign leaderboard the API already exposes, so that is the trigger.
  • While testing I found the existing WebSocket server could not start on the installed ws 8 (new WebSocket.Server is undefined with the ESM default import). It now uses the named WebSocketServer export, which the gateway needs.

Also adds migrations 044–045, the new env vars to backend/.env.example, and a changelog entry.

Testing

node --test src/middleware/probes.test.js src/services/githubWebhook.test.js \
  src/services/ipfsPinService.test.js src/websocket/leaderboardGateway.test.js   # 36 tests
node --test src/integration/openapi-contract.test.js    # 8 passing
node --test src/tests/api-snapshot.test.js              # 20 passing
npx eslint <changed files>                              # no errors
npx prettier --check <changed files>                    # clean

The tests cover the probe semantics (required vs. optional, timeouts, shutdown) and the wired /readyz, signature verification and every event type for the webhook (through the real app with a raw body), the pin service with a fake provider and fetch (dedupe, retry, SSRF and size limits, Pinata request shape), and the leaderboard diff/gateway plus an end-to-end run over a real WebSocket connection.

I did not get a full local run of the backend suite: several existing test files hang here on network calls and open handles (for example websocket/live-updates.test.js), before and independent of this change.

Closes #1255
Closes #1256
Closes #1251
Closes #1257

zerroai and others added 2 commits September 26, 2026 11:32
…on and live leaderboard

- /readyz verifies the database, Redis and Soroban RPC with per-check timeouts (RPC is
  degraded-but-ready unless READINESS_REQUIRE_RPC=true); /healthz reports uptime and RPC state.
- IPFS pin service (Pinata) for campaign image, rules, badge and metadata, with de-duplication,
  retries, an SSRF-safe image fetch, POST/GET /campaigns/:id/ipfs-pins and background pinning.
- GitHub webhook listener with HMAC verification that records merged PRs and completed issues,
  plus a master-key lookup for reward verification.
- Live leaderboard gateway on the WebSocket server (snapshot on subscribe, rank-change updates,
  coalesced and masked); fix the ws 8 server constructor so the gateway can start.
@drips-wave

drips-wave Bot commented Sep 26, 2026

Copy link
Copy Markdown

@zerroai Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@joelpeace48-cell
joelpeace48-cell merged commit ea99db6 into FinesseStudioLab:main Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment