(feat):welfie referral fork

WELFIE_FORK_IMPLEMENTATION.md

@@ -0,0 +1,113 @@
+# Welfie referral fork — RefRef implementation notes
+
+This fork adds the two capabilities the plan identified as missing from upstream
+`refrefhq/refref` — **the referee form** and **the outbound webhook** — plus the
+supporting schema, hand-off token flow, qualified-signup reconciliation, and
+abuse controls. All changes are **additive** (no upstream table or route was
+modified in a breaking way), so the fork stays easy to rebase.
+
+Branch: `feat/welfie-referral-fork`.
+
+## What the repo actually had (Phase 0 findings)
+
+Several plan "open questions" were resolved by reading the code, not the docs:
+
+- **`participant.external_id` already exists** (unique per product). The
+  Welfie-user-id → participant mapping needs no new column.
+- **A qualifying-event ingestion path already exists**: `POST /v1/track/signup`
+  (API-key auth) creates a `referral` + `event` and runs the reward engine. This
+  is the endpoint welfie-backend calls on Welfie signup. (Fallback of "write the
+  event directly to the DB" is unnecessary.)
+- **`apps/refer` was a pure redirect service** (`/r/:code` → 307 to the
+  product's own landing page). There was no RefRef-hosted form. It is now the
+  home of the public invite page, which is the natural fit (it is already the
+  public, rate-limited, no-auth service).
+- **No** webhook dispatcher, lead/consent storage, or hand-off tokens existed.
+- The referrer-side share UI (widget) and JWT init already exist
+  (`/v1/widget/init`), so Phase 2 largely stands.
+
+## Schema (`packages/coredb`, migration `0003`)
+
+New, additive tables (+ entity-id prefixes in `@refref/id`):
+
+- `referral_lead` — a referee captured by the form **before** they have an
+  account (a new table, because `referral.external_id` is `NOT NULL` = the
+  referee's Welfie id, which doesn't exist yet at form time). Stores status,
+  `email_verified`, versioned consent, attribution (`utm`, `ip`, `user_agent`),
+  and `qualified_referral_id` / `matched_by` once reconciled.
+- `handoff_token` — single-use, short-lived; stores only the SHA-256 **hash**.
+- `webhook_endpoint` — per-product URL + `secret` + `secondary_secret` (rotation
+  overlap) + subscribed `event_types`.
+- `webhook_delivery` — outbound log **and** dead-letter queue; `event_id` is the
+  idempotency key sent to the receiver, `dedupe_key` guards against
+  double-enqueue. The worker polls this table.
+
+Apply with `pnpm --filter @refref/coredb db:migrate` (or `db:push` in dev).
+
+## Webhook dispatcher (`packages/webhooks`, new)
+
+- **Signing**: HMAC-SHA256 over `${timestamp}.${rawBody}` → `X-Refref-Signature:
+  sha256=<hex>` plus `X-Refref-Timestamp`. Binding the timestamp into the signed
+  string makes replays detectable; the receiver also rejects skew > 120s.
+- **`verifySignature`** accepts an **array of secrets** — the reference the
+  receiver mirrors, and what enables rotation with overlap.
+- **`enqueueWebhook`** writes one delivery row per subscribed endpoint. Never
+  calls out inline (a slow downstream can't become a form timeout).
+- **Worker** (`runWebhookWorker` / `runWebhookWorkerOnce`): claims due rows
+  atomically, POSTs with a timeout, on failure applies exponential backoff with
+  jitter, and dead-letters after `max_attempts`. `replayDelivery` re-queues a
+  dead-lettered row.
+- Runs inside the API when `WEBHOOK_WORKER_ENABLED=true`, or standalone via
+  `apps/api` `pnpm --filter @refref/api worker` (`src/worker.ts`).
+
+## Referee form + hand-off (`apps/refer`)
+
+- `GET  {basePath}/invite/:code` — resolves the code server-side and renders a
+  personalised, dependency-free HTML page. Fields are limited to **first name +
+  email** by policy, with a versioned consent checkbox, a honeypot, and optional
+  Turnstile.
+- `POST {basePath}/invite/:code` — validates, runs abuse checks, writes the
+  lead, enqueues `referral.created`, issues a single-use hand-off token, and
+  `303`-redirects to `WELFIE_SIGNUP_URL?lt=<token>`.
+- `basePath` (`REFER_BASE_PATH`, e.g. `/refer`) matches the reverse-proxy mount.
+  `trustProxy` is on so client IP is correct behind the proxy.
+
+## Qualified + token exchange (`apps/api`)
+
+- `POST /v1/handoff/exchange` (API-key auth, **internal only** — allowlist to the
+  private network at the proxy): single-use, constant-time hash claim; returns
+  the referee prefill + referrer attribution.
+- `POST /v1/track/signup` extended: after recording the event it calls
+  `reconcileSignup`, which matches a pending lead by (0) explicit `leadId`, (1)
+  hand-off token, or (2) **email within the attribution window** — the last one
+  covers "submitted on mobile, signed up days later on desktop". On a match the
+  lead flips to `qualified` and `referral.qualified` is enqueued. Reward status
+  is reported as `pending_approval` (manual approval in v1). Failures here never
+  fail the signup call.
+
+## Abuse controls
+
+Honeypot (silent drop), disposable-domain rejection, self-referral (referrer's
+own email or shared **custom** domain — shared free providers like gmail are
+allowed), tighter per-route rate limits, and optional Turnstile. Per the plan,
+per-IP/per-code rate limits + WAF also belong at the proxy.
+
+## Tests
+
+- `packages/webhooks` — signing round-trip, tamper, skew, rotation overlap,
+  backoff (8).
+- `apps/refer` — abuse helpers (9) + existing redirect/health (19).
+- `apps/api` — existing suite unaffected (53).
+- **End-to-end** against a real Postgres: `apps/refer/scripts/itest.mts`
+  (`pnpm --filter @refref/refer itest`) — 38 assertions covering page render,
+  form submit, lead creation, `referral.created` enqueue + **retry-then-deliver**
+  with signature verification, idempotent double-submit, single-use hand-off,
+  email-reconciliation qualify + `referral.qualified`, and every abuse path.
+
+## Still owned by welfie-backend (not in this fork)
+
+Receiving the webhooks (verify HMAC + timestamp before parsing, upsert on
+`event_id`), the HubSpot writes from a separate queue, double-opt-in email
+verification before promoting a lead to a HubSpot contact, retention/deletion,
+and the reward-approval UI. `verifySignature` in `@refref/webhooks` is the
+reference to mirror on the receiver.

apps/api/.env.example

@@ -10,3 +10,15 @@ DATABASE_URL="postgresql://postgres:postgres@localhost:5432/refref"
 # Referral Configuration
 # With portless: http://refref-refer.localhost:1355 | Without: http://localhost:3002
 REFERRAL_HOST_URL="http://refref-refer.localhost:1355"
+
+# --- Welfie referral fork additions ---
+
+# Outbound webhook dispatcher. Set to "true" to run the worker inside the API
+# process; leave unset and run `pnpm --filter @refref/api worker` as its own
+# process in production.
+WEBHOOK_WORKER_ENABLED="false"
+# Poll interval for the webhook worker (ms).
+WEBHOOK_WORKER_INTERVAL_MS=5000
+
+# Email-reconciliation attribution window for late/cross-device signups (days).
+ATTRIBUTION_WINDOW_DAYS=30

apps/api/package.json

@@ -7,6 +7,7 @@
     "dev": "portless refref-api -- infisical run --env=dev -- tsx watch src/index.ts || portless refref-api -- tsx watch src/index.ts",
     "build": "tsc",
     "start": "node dist/index.js",
+    "worker": "node dist/worker.js",
     "test": "infisical run --env=dev -- vitest run || vitest run",
     "test:watch": "infisical run --env=dev -- vitest || vitest",
     "test:ui": "infisical run --env=dev -- vitest --ui || vitest --ui",
@@ -24,6 +25,7 @@
     "@refref/id": "workspace:*",
     "@refref/types": "workspace:*",
     "@refref/utils": "workspace:*",
+    "@refref/webhooks": "workspace:*",
     "@tsndr/cloudflare-worker-jwt": "^3.2.0",
     "dotenv": "^17.2.3",
     "drizzle-orm": "^0.44.7",

apps/api/src/app.ts

@@ -9,6 +9,8 @@ import jwtAuthPlugin from "./plugins/jwt-auth.js";
 import widgetInitRoutes from "./routes/v1/widget/init.js";
 import trackRoutes from "./routes/v1/track.js";
 import programsRoutes from "./routes/v1/programs.js";
+import handoffRoutes from "./routes/v1/handoff.js";
+import { runWebhookWorker } from "@refref/webhooks";
 
 export async function buildApp(): Promise<FastifyInstance> {
   // Validate required environment variables
@@ -66,9 +68,26 @@ export async function buildApp(): Promise<FastifyInstance> {
 
       // Programs routes
       await fastify.register(programsRoutes, { prefix: "/programs" });
+
+      // Internal hand-off token exchange (API-key auth; keep off public ingress)
+      await fastify.register(handoffRoutes, { prefix: "/handoff" });
     },
     { prefix: "/v1" },
   );
 
+  // Optionally run the outbound webhook dispatcher inside the API process.
+  // In production you may prefer the standalone worker (see src/worker.ts).
+  if (process.env.WEBHOOK_WORKER_ENABLED === "true") {
+    const intervalMs = Number(process.env.WEBHOOK_WORKER_INTERVAL_MS) || 5000;
+    const stop = runWebhookWorker(db, intervalMs, {
+      logger: {
+        info: (o, m) => app.log.info(o, m),
+        error: (o, m) => app.log.error(o, m),
+      },
+    });
+    app.addHook("onClose", async () => stop());
+    app.log.info(`Webhook worker started (interval ${intervalMs}ms)`);
+  }
+
   return app;
 }

apps/api/src/routes/v1/handoff.ts

@@ -0,0 +1,74 @@
+import { FastifyInstance, FastifyRequest, FastifyReply } from "fastify";
+import { z } from "zod";
+import { schema } from "@refref/coredb";
+import { eq } from "drizzle-orm";
+import { consumeHandoffToken } from "../../services/referral-lead.js";
+
+const { participant } = schema;
+
+const exchangeSchema = z.object({
+  token: z.string().min(1),
+});
+
+/**
+ * Internal-only hand-off token exchange. welfie-backend calls this server-side
+ * to turn the single-use `lt` token from the redirect into the referee's
+ * prefill data. It is guarded by the same API-key auth as the track endpoints
+ * and MUST NOT be exposed to public ingress (allowlist it to the private
+ * network at the proxy). Single-use: a second exchange of the same token 404s.
+ */
+export default async function handoffRoutes(fastify: FastifyInstance) {
+  fastify.post(
+    "/exchange",
+    { preHandler: [fastify.authenticateApiKey] },
+    async (request: FastifyRequest, reply: FastifyReply) => {
+      const parsed = exchangeSchema.safeParse(request.body);
+      if (!parsed.success) {
+        return reply
+          .code(400)
+          .send({ success: false, message: "A token is required." });
+      }
+
+      const lead = await consumeHandoffToken(request.db, parsed.data.token);
+      if (!lead) {
+        // Unknown, expired, or already consumed — do not distinguish (no oracle).
+        return reply
+          .code(404)
+          .send({ success: false, message: "Invalid or expired token." });
+      }
+
+      // Fetch the referrer for attribution prefill.
+      const [referrer] = await request.db
+        .select({
+          id: participant.id,
+          externalId: participant.externalId,
+          name: participant.name,
+          email: participant.email,
+        })
+        .from(participant)
+        .where(eq(participant.id, lead.referrerParticipantId))
+        .limit(1);
+
+      return reply.send({
+        success: true,
+        lead: {
+          id: lead.id,
+          email: lead.email,
+          firstName: lead.firstName,
+          code: lead.code,
+          programId: lead.programId,
+          productId: lead.productId,
+          consentVersion: lead.consentVersion,
+          consentAt: lead.consentAt,
+        },
+        referrer: referrer
+          ? {
+              participantId: referrer.id,
+              externalId: referrer.externalId,
+              name: referrer.name,
+            }
+          : null,
+      });
+    },
+  );
+}

apps/api/src/routes/v1/track/signup.ts

@@ -1,11 +1,46 @@
 import { FastifyInstance, FastifyRequest, FastifyReply } from "fastify";
 import { z } from "zod";
 import { schema } from "@refref/coredb";
-const { participant, referral, refcode } = schema;
+const { participant, referral, refcode, rewardRule } = schema;
 import { eq, and } from "drizzle-orm";
-import { type EventMetadataV1Type } from "@refref/types";
+import {
+  type EventMetadataV1Type,
+  type RewardRuleConfigV1Type,
+  type ReferralQualifiedPayload,
+} from "@refref/types";
 import { createEvent } from "../../../services/events.js";
 import { normalizeCode } from "@refref/utils";
+import { enqueueWebhook } from "@refref/webhooks";
+import { reconcileSignup } from "../../../services/referral-lead.js";
+
+/**
+ * Derive the referrer reward summary for the qualified webhook. Reward release
+ * is gated on manual approval in v1, so status is always reported as
+ * "pending_approval" regardless of the rule's disbursal type.
+ */
+async function deriveRewardSummary(
+  db: FastifyRequest["db"],
+  programId: string,
+): Promise<ReferralQualifiedPayload["reward"]> {
+  const rules = await db
+    .select()
+    .from(rewardRule)
+    .where(
+      and(eq(rewardRule.programId, programId), eq(rewardRule.isActive, true)),
+    );
+  const referrerRule = rules.find(
+    (r) => (r.config as RewardRuleConfigV1Type)?.participantType === "referrer",
+  );
+  if (!referrerRule) return null;
+  const c = referrerRule.config as RewardRuleConfigV1Type;
+  return {
+    id: null,
+    type: c.reward.type,
+    amount: Number(c.reward.amount) || 0,
+    currency: c.reward.currency || "AUD",
+    status: "pending_approval",
+  };
+}
 
 // Signup event request schema (no eventType discriminator needed)
 const signupRequestSchema = z.object({
@@ -17,6 +52,9 @@ const signupRequestSchema = z.object({
     refcode: z.string().optional(),
     email: z.string().email().optional(),
     name: z.string().optional(),
+    // Optional deterministic hand-off matching (from the referee form flow).
+    handoffToken: z.string().optional(),
+    leadId: z.string().optional(),
   }),
 });
 
@@ -119,6 +157,59 @@ export default async function signupTrackRoutes(fastify: FastifyInstance) {
           metadata: result.metadata,
         });
 
+        // Reconcile any pending referee lead into "qualified" and emit the
+        // referral.qualified webhook. Best-effort: a hiccup here must never fail
+        // the signup tracking itself (the event is already recorded).
+        try {
+          const qualified = await reconcileSignup(request.db, {
+            productId: body.productId,
+            email: body.payload.email,
+            leadId: body.payload.leadId,
+            handoffToken: body.payload.handoffToken,
+            referralId: result.referralId,
+            refereeExternalId: body.payload.userId,
+            windowDays: Number(process.env.ATTRIBUTION_WINDOW_DAYS) || 30,
+          });
+
+          if (qualified) {
+            const reward = await deriveRewardSummary(
+              request.db,
+              qualified.programId,
+            );
+            const qualifiedPayload: Omit<
+              ReferralQualifiedPayload,
+              "event_id"
+            > = {
+              event: "referral.qualified",
+              occurred_at: new Date().toISOString(),
+              program_id: qualified.programId,
+              referral: {
+                id: result.referralId ?? qualified.leadId,
+                code: qualified.code,
+                status: "qualified",
+              },
+              referee: {
+                email: qualified.email,
+                external_id: body.payload.userId,
+              },
+              reward,
+              matched_by: qualified.matchedBy,
+            };
+
+            await enqueueWebhook(request.db, {
+              productId: body.productId,
+              eventType: "referral.qualified",
+              payload: qualifiedPayload as unknown as Record<string, unknown>,
+              dedupeKey: `qualified:${qualified.leadId}`,
+            });
+          }
+        } catch (err) {
+          request.log.error(
+            { err },
+            "referral qualification reconcile failed",
+          );
+        }
+
         return reply.send({
           success: true,
           message: "Signup tracked successfully.",