From 37ce7a8821005b50c6f71c90e421554ee9d4796a Mon Sep 17 00:00:00 2001 From: Michael Shimeles Date: Sat, 19 Sep 2026 17:33:13 -0400 Subject: [PATCH 1/2] feat(infra): deploy the managed cloud to Cloudflare with Alchemy Site: SvelteKit on Workers via Alchemy (replaces the Vercel adapter, analytics and deploy path); prod stage owns the custom domain, Web Analytics, R2 bucket. Control plane and gateway: Cloudflare Containers behind one edge Worker (api., gateway., preview wildcard). The Worker signs each caller's address for the control plane the way the gateway does; the gateway trusts CF-Connecting-IP from the edge. Hosts: keep their WireGuard address as identity but are reached through their own Cloudflare Tunnel at
. behind a Cloudflare Access service token (ADR 0005). Transport is selected by NEHEMIAH_HOST_TRANSPORT / NEHEMIAH_GATEWAY_HOST_TRANSPORT; overlay stays the default. nehemiahd can present an Access token toward the control plane. Dockerfiles for both services, a deploy workflow (prod on main, pr- site previews), runbook and ADR. --- .dockerignore | 14 + .github/workflows/deploy-cloudflare.yml | 129 + .gitignore | 1 + .prettierignore | 1 + README.md | 8 +- apps/nehemiah/.env.example | 9 + apps/nehemiah/Dockerfile | 38 + apps/nehemiah/src/clients/host-transport.ts | 112 + .../src/clients/nehemiahd-templates.ts | 32 +- apps/nehemiah/src/clients/nehemiahd.ts | 32 +- apps/nehemiah/src/config.ts | 65 +- apps/nehemiah/src/main.ts | 4 +- apps/nehemiah/test/host-transport.test.ts | 88 + apps/web/.env.example | 25 +- apps/web/.gitignore | 3 + apps/web/alchemy.run.ts | 362 + apps/web/edge/worker.ts | 126 + apps/web/package.json | 15 +- apps/web/src/hooks.client.ts | 4 - apps/web/tsconfig.alchemy.json | 12 + apps/web/tsconfig.edge.json | 12 + apps/web/vite.config.ts | 7 +- docs/architecture.md | 5 +- docs/cloudflare.md | 143 + ...-cloudflare-containers-and-host-tunnels.md | 52 + gateway/.dockerignore | 5 + gateway/Dockerfile | 16 + gateway/config.go | 67 +- gateway/host_transport.go | 61 + gateway/host_transport_test.go | 174 + gateway/proxy.go | 16 + gateway/router.go | 10 +- infra/cloudflare/cloudflared.service | 43 + infra/latitude/deploy-web.sh | 29 +- nehemiahd/config.go | 216 +- nehemiahd/config_test.go | 33 + nehemiahd/controlplane.go | 4 + package-lock.json | 6946 +++++++++-------- package.json | 6 + turbo.json | 2 +- 40 files changed, 5738 insertions(+), 3189 deletions(-) create mode 100644 .dockerignore create mode 100644 .github/workflows/deploy-cloudflare.yml create mode 100644 apps/nehemiah/Dockerfile create mode 100644 apps/nehemiah/src/clients/host-transport.ts create mode 100644 apps/nehemiah/test/host-transport.test.ts create mode 100644 apps/web/alchemy.run.ts create mode 100644 apps/web/edge/worker.ts delete mode 100644 apps/web/src/hooks.client.ts create mode 100644 apps/web/tsconfig.alchemy.json create mode 100644 apps/web/tsconfig.edge.json create mode 100644 docs/cloudflare.md create mode 100644 docs/nehemiah/adr/0005-cloudflare-containers-and-host-tunnels.md create mode 100644 gateway/.dockerignore create mode 100644 gateway/Dockerfile create mode 100644 gateway/host_transport.go create mode 100644 gateway/host_transport_test.go create mode 100644 infra/cloudflare/cloudflared.service diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..7911f22 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,14 @@ +# Build context for apps/nehemiah/Dockerfile (repository root). Allow only +# what the npm workspace install and the control-plane build need. +* +!package.json +!package-lock.json +!.npmrc +!apps/nehemiah +!apps/web/package.json +!packages +**/node_modules +**/dist +**/.turbo +**/test +**/*.test.ts diff --git a/.github/workflows/deploy-cloudflare.yml b/.github/workflows/deploy-cloudflare.yml new file mode 100644 index 0000000..70bc738 --- /dev/null +++ b/.github/workflows/deploy-cloudflare.yml @@ -0,0 +1,129 @@ +name: Deploy Cloudflare + +# Everything that runs on Cloudflare, via Alchemy (apps/web/alchemy.run.ts): +# main -> stage `prod` (site, control plane and gateway containers, edge +# Worker, Access, R2); pull requests from this repository -> stage `pr-` +# (site only, workers.dev), destroyed when the pull request closes. +# See docs/cloudflare.md. + +on: + push: + branches: [main] + paths: + - "apps/web/**" + - "apps/nehemiah/**" + - "gateway/**" + - "packages/typescript-config/**" + - "package.json" + - "package-lock.json" + - ".dockerignore" + - ".github/workflows/deploy-cloudflare.yml" + pull_request: + types: [opened, reopened, synchronize, closed] + paths: + - "apps/web/**" + - "packages/typescript-config/**" + - "package.json" + - "package-lock.json" + - ".github/workflows/deploy-cloudflare.yml" + +permissions: + contents: read + +concurrency: + group: deploy-cloudflare-${{ github.ref }} + cancel-in-progress: false + +env: + STAGE: ${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.number) || 'prod' }} + +jobs: + deploy: + name: alchemy deploy + # Forks cannot read the deploy secrets, so previews are limited to branches + # in this repository. + if: >- + github.event.action != 'closed' && + (github.event_name != 'pull_request' || + github.event.pull_request.head.repo.full_name == github.repository) + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: 24 + cache: npm + - run: npm ci + # The control-plane image never migrates; apply migrations with the + # migration role before the new containers roll out. + - name: Apply control-plane migrations + if: env.STAGE == 'prod' + run: npm run build -w @nehemiah/nehemiah && npm run migrate -w @nehemiah/nehemiah + env: + NODE_ENV: production + DATABASE_URL: ${{ secrets.DATABASE_URL }} + MIGRATION_DATABASE_URL: ${{ secrets.MIGRATION_DATABASE_URL }} + - name: Deploy stage ${{ env.STAGE }} + run: npm run deploy -w web -- --stage "$STAGE" --yes + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} + # Site settings, forwarded to the site Worker only when non-empty. + PUBLIC_NEHEMIAH_URL: ${{ vars.PUBLIC_NEHEMIAH_URL }} + PRIVATE_NEHEMIAH_URL: ${{ vars.PRIVATE_NEHEMIAH_URL }} + PUBLIC_CLERK_PUBLISHABLE_KEY: ${{ vars.PUBLIC_CLERK_PUBLISHABLE_KEY }} + PUBLIC_CLERK_FRONTEND_API: ${{ vars.PUBLIC_CLERK_FRONTEND_API }} + STATUS_CONTROL_PLANE_URL: ${{ vars.STATUS_CONTROL_PLANE_URL }} + STATUS_GATEWAY_URL: ${{ vars.STATUS_GATEWAY_URL }} + PUBLIC_SUPPORT_URL: ${{ vars.PUBLIC_SUPPORT_URL }} + PUBLIC_SUPPORT_EMAIL: ${{ vars.PUBLIC_SUPPORT_EMAIL }} + # Edge and container settings (prod stage only; defaults in apps/web/.env.example). + SITE_DOMAIN: ${{ vars.SITE_DOMAIN }} + API_HOSTNAME: ${{ vars.API_HOSTNAME }} + GATEWAY_HOSTNAME: ${{ vars.GATEWAY_HOSTNAME }} + PREVIEW_BASE_DOMAIN: ${{ vars.PREVIEW_BASE_DOMAIN }} + PREVIEW_ZONE_NAME: ${{ vars.PREVIEW_ZONE_NAME }} + HOST_TUNNEL_DOMAIN: ${{ vars.HOST_TUNNEL_DOMAIN }} + FLEET_ACCESS_TOKEN_ID: ${{ vars.FLEET_ACCESS_TOKEN_ID }} + R2_BUCKET: ${{ vars.R2_BUCKET }} + NEHEMIAH_SERVICE_VERSION: ${{ vars.NEHEMIAH_SERVICE_VERSION }} + NEHEMIAH_OTEL_ENDPOINT: ${{ vars.NEHEMIAH_OTEL_ENDPOINT }} + NEHEMIAH_DEFAULT_REGION: ${{ vars.NEHEMIAH_DEFAULT_REGION }} + NEHEMIAH_HOST_CIDRS: ${{ vars.NEHEMIAH_HOST_CIDRS }} + CLERK_ISSUER: ${{ vars.CLERK_ISSUER }} + CLERK_AUDIENCE: ${{ vars.CLERK_AUDIENCE }} + # Container secrets (prod stage only). + DATABASE_URL: ${{ secrets.DATABASE_URL }} + NEHEMIAH_HOST_CREDENTIAL_KEY: ${{ secrets.NEHEMIAH_HOST_CREDENTIAL_KEY }} + NEHEMIAH_GATEWAY_TOKEN: ${{ secrets.NEHEMIAH_GATEWAY_TOKEN }} + NEHEMIAH_GATEWAY_SECRET: ${{ secrets.NEHEMIAH_GATEWAY_SECRET }} + NEHEMIAH_DEVICE_CODE_PEPPER: ${{ secrets.NEHEMIAH_DEVICE_CODE_PEPPER }} + NEHEMIAH_OTEL_AUTHORIZATION: ${{ secrets.NEHEMIAH_OTEL_AUTHORIZATION }} + GATEWAY_OTEL_AUTHORIZATION: ${{ secrets.GATEWAY_OTEL_AUTHORIZATION }} + STRIPE_WEBHOOK_SECRET: ${{ secrets.STRIPE_WEBHOOK_SECRET }} + + cleanup: + name: alchemy destroy (closed preview) + if: >- + github.event_name == 'pull_request' && + github.event.action == 'closed' && + github.event.pull_request.head.repo.full_name == github.repository + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: 24 + cache: npm + - run: npm ci + - name: Refuse to destroy prod + run: | + if [ "$STAGE" = "prod" ]; then + echo "refusing to destroy the prod stage from a cleanup job" >&2 + exit 1 + fi + - name: Destroy stage ${{ env.STAGE }} + run: npm run destroy -w web -- --stage "$STAGE" --yes + env: + CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }} + CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }} diff --git a/.gitignore b/.gitignore index 1a959c2..c43b7a6 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,7 @@ node_modules # Build output (per-workspace outputs are also ignored in each app) .output .vercel +.alchemy .netlify .wrangler .svelte-kit diff --git a/.prettierignore b/.prettierignore index 4cc7104..54b63ff 100644 --- a/.prettierignore +++ b/.prettierignore @@ -4,6 +4,7 @@ build .svelte-kit .turbo .vercel +.alchemy coverage package-lock.json *.min.js diff --git a/README.md b/README.md index 02b5341..8910c3b 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,11 @@ npm run dev -w web Full REST + WebSocket API in the [docs](https://boringcomputers.com/docs). +The site itself deploys to Cloudflare Workers with [Alchemy](https://alchemy.run) +(`apps/web/alchemy.run.ts`), together with the edge in front of the managed +cloud: custom domain, origin tunnel, DNS, and R2. See +[`docs/cloudflare.md`](docs/cloudflare.md). + **From any AI** — an MCP server ([`nehemiah-mcp`](packages/mcp)) lets Claude Desktop, Cursor, and other agents spin up and drive your computers as a tool: @@ -111,11 +116,12 @@ hosts are provisioned from a signed release via the managed runbook A [Turborepo](https://turbo.build/repo) monorepo (npm workspaces): ``` -apps/web/ the site — SvelteKit +apps/web/ the site — SvelteKit; alchemy.run.ts deploys it + the Cloudflare edge nehemiahd/ the host daemon — Go, runs the microVMs packages/sdk/ nehemiah-sdk — Effect-native TypeScript client packages/mcp/ nehemiah-mcp — MCP server infra/latitude/ managed-host provisioning (provision/teardown), image builds, networking +infra/cloudflare/ cloudflared unit for the origin VM behind the tunnel ``` ```sh diff --git a/apps/nehemiah/.env.example b/apps/nehemiah/.env.example index 20e4538..2694a55 100644 --- a/apps/nehemiah/.env.example +++ b/apps/nehemiah/.env.example @@ -5,6 +5,15 @@ DATABASE_URL=postgres://postgres:postgres@127.0.0.1:5432/nehemiah NEHEMIAH_HOST_CREDENTIAL_KEY=MDEyMzQ1Njc4OWFiY2RlZjAxMjM0NTY3ODlhYmNkZWY= NEHEMIAH_HOST_CIDRS=127.0.0.0/8,10.0.0.0/8,fd00::/8 NEHEMIAH_HOST_PORT=8080 +# How host internal APIs are reached (docs/nehemiah/adr/0005). `overlay` +# (default) dials the WireGuard address directly. `cloudflare-tunnel` keeps +# that address as the host identity but dials the host's own Cloudflare Tunnel +# at https://
. with the Access +# service token; 10.64.0.7 becomes 10-64-0-7.hosts.example.com. +# NEHEMIAH_HOST_TRANSPORT=cloudflare-tunnel +# NEHEMIAH_HOST_TUNNEL_DOMAIN=hosts.example.com +# NEHEMIAH_HOST_ACCESS_CLIENT_ID=replace-with-access-service-token-client-id +# NEHEMIAH_HOST_ACCESS_CLIENT_SECRET=replace-with-access-service-token-client-secret NEHEMIAH_GATEWAY_TOKEN=replace-me NEHEMIAH_GATEWAY_SECRET=replace-me-too # Generate 32 random bytes, base64 encode them, and keep this pepper distinct. diff --git a/apps/nehemiah/Dockerfile b/apps/nehemiah/Dockerfile new file mode 100644 index 0000000..f5dcc0c --- /dev/null +++ b/apps/nehemiah/Dockerfile @@ -0,0 +1,38 @@ +# syntax=docker/dockerfile:1 +# +# Control plane (apps/nehemiah) image for Cloudflare Containers. The build +# context is the repository root so the npm workspace lockfile resolves; see +# docs/cloudflare.md. The runtime is Debian-based because the lockfile pins the +# glibc build of @node-rs/argon2. +# +# Migrations are not run here: the deploy workflow applies them with the +# migration role before the new image rolls out. + +FROM node:24.19.0-slim@sha256:a9f5f7c91a432850b2a8a7797adf5eadb6c733ceed61167806cee7ea7fbc29df AS build +WORKDIR /src +COPY package.json package-lock.json .npmrc ./ +COPY apps/nehemiah/package.json apps/nehemiah/ +COPY apps/web/package.json apps/web/ +COPY packages ./packages +RUN npm ci --workspace @nehemiah/nehemiah --no-audit --no-fund +COPY apps/nehemiah ./apps/nehemiah +RUN npm run build --workspace @nehemiah/nehemiah + +FROM node:24.19.0-slim@sha256:a9f5f7c91a432850b2a8a7797adf5eadb6c733ceed61167806cee7ea7fbc29df AS runtime-deps +WORKDIR /src +COPY package.json package-lock.json .npmrc ./ +COPY apps/nehemiah/package.json apps/nehemiah/ +COPY apps/web/package.json apps/web/ +COPY packages ./packages +RUN npm ci --workspace @nehemiah/nehemiah --omit=dev --no-audit --no-fund \ + && npm cache clean --force + +FROM node:24.19.0-slim@sha256:a9f5f7c91a432850b2a8a7797adf5eadb6c733ceed61167806cee7ea7fbc29df +ENV NODE_ENV=production +WORKDIR /srv/nehemiah +COPY --from=runtime-deps /src ./ +COPY --from=build /src/apps/nehemiah/dist ./apps/nehemiah/dist +COPY apps/nehemiah/src/db/migrations ./apps/nehemiah/src/db/migrations +USER node +EXPOSE 8081 +CMD ["node", "apps/nehemiah/dist/main.js"] diff --git a/apps/nehemiah/src/clients/host-transport.ts b/apps/nehemiah/src/clients/host-transport.ts new file mode 100644 index 0000000..4b9cac9 --- /dev/null +++ b/apps/nehemiah/src/clients/host-transport.ts @@ -0,0 +1,112 @@ +import { isIP } from 'node:net'; +import { Redacted } from 'effect'; + +/** + * How the control plane reaches a host's internal API. + * + * - `overlay` dials the host's stored WireGuard address directly (ADR 0002). + * - `cloudflare-tunnel` keeps that address as the host's identity but reaches + * it through the host's own Cloudflare Tunnel, published under a hostname + * derived from the address and guarded by a Cloudflare Access service token + * (ADR 0005). Nothing about how hosts are stored or validated changes. + */ +export type HostTransport = + | { readonly kind: 'overlay'; readonly port: number } + | { + readonly kind: 'cloudflare-tunnel'; + /** DNS suffix hosts publish their tunnels under, e.g. `hosts.example.com`. */ + readonly domain: string; + readonly accessClientId: string; + readonly accessClientSecret: Redacted.Redacted; + }; + +export const overlayTransport = (port = 8080): HostTransport => ({ kind: 'overlay', port }); + +export class HostTransportError extends Error {} + +const dnsName = /^[a-z0-9](?:[a-z0-9.-]{0,251}[a-z0-9])?$/; +/** Access service-token credentials are opaque; require printable, unpadded ASCII. */ +const accessCredential = /^[\x21-\x7e]{8,512}$/; + +export const validateHostTransport = (transport: HostTransport): HostTransport => { + if (transport.kind === 'overlay') { + if (!Number.isSafeInteger(transport.port) || transport.port < 1 || transport.port > 65_535) { + throw new HostTransportError('host port must be between 1 and 65535'); + } + return transport; + } + if ( + !dnsName.test(transport.domain) || + !transport.domain.includes('.') || + transport.domain.includes('..') + ) { + throw new HostTransportError( + 'host tunnel domain must be a DNS suffix such as hosts.example.com' + ); + } + if ( + !accessCredential.test(transport.accessClientId) || + !accessCredential.test(Redacted.value(transport.accessClientSecret)) + ) { + throw new HostTransportError('host tunnel transport requires an Access service token'); + } + return transport; +}; + +/** Strip the IPv4-mapped prefix so both address families map like the gateway does. */ +const unmap = (address: string): string => { + const mapped = address.toLowerCase().match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); + return mapped ? mapped[1] : address.toLowerCase(); +}; + +/** Expand an IPv6 address to eight zero-padded four-digit groups. */ +const expandIPv6 = (address: string): string[] => { + let value = address; + const embedded = value.match(/^(.*:)(\d+)\.(\d+)\.(\d+)\.(\d+)$/); + if (embedded) { + const [, head, a, b, c, d] = embedded; + const octets = [a, b, c, d].map(Number); + value = `${head}${(((octets[0] ?? 0) << 8) | (octets[1] ?? 0)).toString(16)}:${(((octets[2] ?? 0) << 8) | (octets[3] ?? 0)).toString(16)}`; + } + const [left = '', right] = value.split('::'); + const head = left ? left.split(':') : []; + const tail = right ? right.split(':') : []; + const groups = + right === undefined + ? head + : [...head, ...Array.from({ length: 8 - head.length - tail.length }, () => '0'), ...tail]; + return groups.map((group) => group.padStart(4, '0')); +}; + +/** + * The DNS label a host's tunnel is published under: dotted IPv4 with dashes + * (`10.64.0.7` → `10-64-0-7`), or the fully expanded IPv6 groups joined with + * dashes. Identical to the gateway's `hostTunnelLabel` and to what operators + * configure in cloudflared on the host, so the three never disagree. + */ +export const hostTunnelLabel = (address: string): string => { + const value = unmap(address); + const version = isIP(value); + if (version === 4) return value.replaceAll('.', '-'); + if (version === 6) return expandIPv6(value).join('-'); + throw new HostTransportError('invalid host address'); +}; + +/** Origin (no trailing slash) for a host's internal API under the transport. */ +export const hostBaseUrl = (address: string, transport: HostTransport): string => { + const version = isIP(address); + if (version === 0) throw new HostTransportError('invalid host address'); + if (transport.kind === 'overlay') { + return `http://${version === 6 ? `[${address}]` : address}:${transport.port}`; + } + return `https://${hostTunnelLabel(address)}.${transport.domain}`; +}; + +/** Extra request headers the transport needs; empty for the overlay. */ +export const hostTransportHeaders = (transport: HostTransport): ReadonlyArray<[string, string]> => + transport.kind === 'cloudflare-tunnel' + ? [ + ['cf-access-client-id', transport.accessClientId], + ['cf-access-client-secret', Redacted.value(transport.accessClientSecret)] + ] + : []; diff --git a/apps/nehemiah/src/clients/nehemiahd-templates.ts b/apps/nehemiah/src/clients/nehemiahd-templates.ts index ecf70f2..d465b5c 100644 --- a/apps/nehemiah/src/clients/nehemiahd-templates.ts +++ b/apps/nehemiah/src/clients/nehemiahd-templates.ts @@ -1,4 +1,3 @@ -import { isIP } from 'node:net'; import { HostRequestError } from './nehemiahd.js'; import type { HostCredentialResolver } from '../domain/host-credentials.js'; import { @@ -8,6 +7,14 @@ import { } from '../domain/templates.js'; import type { TemplateActivator } from '../jobs/replicate-template.js'; import { injectTraceHeaders } from '../telemetry.js'; +import { + hostBaseUrl, + hostTransportHeaders, + HostTransportError, + overlayTransport, + validateHostTransport, + type HostTransport +} from './host-transport.js'; const exportIdPattern = /^te_[0-9a-f]{32}$/; const hostTemplatePattern = /^t-[0-9a-f]{29}$/; @@ -128,20 +135,22 @@ export const templateObjectOrigins = (input: { export class NehemiahdTemplateClient implements TemplatePublisher, TemplateActivator { readonly #origins: ReadonlySet; + readonly #transport: HostTransport; + constructor( private readonly credentials: HostCredentialResolver, allowedObjectOrigins: ReadonlyArray, private readonly fetcher: typeof fetch = fetch, private readonly requestTimeoutMs = 15 * 60 * 1_000, - private readonly hostPort = 8080 + transport: number | HostTransport = 8080 ) { this.#origins = normalizedOrigins(allowedObjectOrigins); if (!Number.isSafeInteger(requestTimeoutMs) || requestTimeoutMs < 1_000) { throw new Error('template host timeout must be at least one second'); } - if (!Number.isSafeInteger(hostPort) || hostPort < 1 || hostPort > 65_535) { - throw new Error('host port must be between 1 and 65535'); - } + this.#transport = validateHostTransport( + typeof transport === 'number' ? overlayTransport(transport) : transport + ); } async export(input: Parameters[0]) { @@ -243,10 +252,16 @@ export class NehemiahdTemplateClient implements TemplatePublisher, TemplateActiv } async #request(address: string, path: string, init: RequestInit): Promise { - const version = isIP(address); - if (version === 0) throw new HostRequestError(undefined, 'invalid host address', false); + let base: string; + try { + base = hostBaseUrl(address, this.#transport); + } catch (error) { + if (error instanceof HostTransportError) { + throw new HostRequestError(undefined, error.message, false); + } + throw error; + } const credential = await this.credentials.resolve(address); - const base = `http://${version === 6 ? `[${address}]` : address}:${this.hostPort}`; const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), this.requestTimeoutMs); try { @@ -254,6 +269,7 @@ export class NehemiahdTemplateClient implements TemplatePublisher, TemplateActiv headers.set('accept', 'application/json'); headers.set('content-type', 'application/json'); headers.set('authorization', `Bearer ${credential}`); + for (const [name, value] of hostTransportHeaders(this.#transport)) headers.set(name, value); injectTraceHeaders(headers); const response = await this.fetcher(`${base}${path}`, { ...init, diff --git a/apps/nehemiah/src/clients/nehemiahd.ts b/apps/nehemiah/src/clients/nehemiahd.ts index 11b4a99..9929cd4 100644 --- a/apps/nehemiah/src/clients/nehemiahd.ts +++ b/apps/nehemiah/src/clients/nehemiahd.ts @@ -1,7 +1,14 @@ -import { isIP } from 'node:net'; import type { HostCredentialResolver } from '../domain/host-credentials.js'; import { EgressPolicy, type NetworkPolicyDeclaration } from '../domain/network-policy.js'; import { injectTraceHeaders } from '../telemetry.js'; +import { + hostBaseUrl, + hostTransportHeaders, + HostTransportError, + overlayTransport, + validateHostTransport, + type HostTransport +} from './host-transport.js'; export interface HostMachine { readonly id: string; @@ -112,15 +119,17 @@ export class HostForkContractError extends HostRequestError { } export class NehemiahdClient implements HostClient { + readonly #transport: HostTransport; + constructor( private readonly credentials: string | HostCredentialResolver, private readonly fetcher: typeof fetch = fetch, private readonly requestTimeoutMs = 30_000, - private readonly hostPort = 8080 + transport: number | HostTransport = 8080 ) { - if (!Number.isSafeInteger(hostPort) || hostPort < 1 || hostPort > 65_535) { - throw new Error('host port must be between 1 and 65535'); - } + this.#transport = validateHostTransport( + typeof transport === 'number' ? overlayTransport(transport) : transport + ); } async create(address: string, request: CreateOnHostRequest): Promise { @@ -399,11 +408,15 @@ export class NehemiahdClient implements HostClient { init: RequestInit = {}, timeoutMs = this.requestTimeoutMs ): Promise { - const version = isIP(address); - if (version === 0) { - throw new HostRequestError(undefined, 'invalid host address', false); + let base: string; + try { + base = hostBaseUrl(address, this.#transport); + } catch (error) { + if (error instanceof HostTransportError) { + throw new HostRequestError(undefined, error.message, false); + } + throw error; } - const base = `http://${version === 6 ? `[${address}]` : address}:${this.hostPort}`; const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), timeoutMs); try { @@ -415,6 +428,7 @@ export class NehemiahdClient implements HostClient { headers.set('accept', 'application/json'); headers.set('content-type', 'application/json'); headers.set('authorization', `Bearer ${internalToken}`); + for (const [name, value] of hostTransportHeaders(this.#transport)) headers.set(name, value); injectTraceHeaders(headers); const response = await this.fetcher(`${base}${path}`, { ...init, diff --git a/apps/nehemiah/src/config.ts b/apps/nehemiah/src/config.ts index 69c2fc8..1b17c82 100644 --- a/apps/nehemiah/src/config.ts +++ b/apps/nehemiah/src/config.ts @@ -1,6 +1,11 @@ import { isIP } from 'node:net'; import { Data, Effect, Redacted } from 'effect'; import type { ApiRateLimitConfig } from './auth/api-admission.js'; +import { + HostTransportError, + validateHostTransport, + type HostTransport +} from './clients/host-transport.js'; import { productionDatabaseUrlIssue } from './db/url.js'; export type Environment = 'development' | 'test' | 'production'; @@ -51,6 +56,8 @@ export interface ServiceConfig { readonly trustedSiteDomain?: string; readonly hostCidrs: ReadonlyArray; readonly hostPort: number; + /** How host internal APIs are reached; the overlay unless configured otherwise (ADR 0005). */ + readonly hostTransport: HostTransport; readonly clerkIssuer?: string; readonly clerkAudience?: string; readonly hostStaleAfterMs: number; @@ -77,6 +84,59 @@ const integer = (env: NodeJS.ProcessEnv, name: string, fallback: number) => { return value; }; +/** + * Host transport selection (ADR 0005). `overlay` is the default and keeps the + * WireGuard contract untouched. `cloudflare-tunnel` reaches each host through + * its own tunnel at `
.` with the + * Access service token in NEHEMIAH_HOST_ACCESS_CLIENT_ID / _SECRET. + */ +const loadHostTransport = (env: NodeJS.ProcessEnv, hostPort: number): HostTransport => { + const kind = env.NEHEMIAH_HOST_TRANSPORT?.trim() || 'overlay'; + const domain = env.NEHEMIAH_HOST_TUNNEL_DOMAIN?.trim().toLowerCase(); + const clientId = env.NEHEMIAH_HOST_ACCESS_CLIENT_ID?.trim(); + const clientSecret = env.NEHEMIAH_HOST_ACCESS_CLIENT_SECRET?.trim(); + if (kind !== 'overlay' && kind !== 'cloudflare-tunnel') { + throw new ConfigError({ + variable: 'NEHEMIAH_HOST_TRANSPORT', + message: 'must be overlay or cloudflare-tunnel' + }); + } + if (kind === 'overlay') { + if (domain || clientId || clientSecret) { + throw new ConfigError({ + variable: 'NEHEMIAH_HOST_TRANSPORT', + message: 'must be cloudflare-tunnel when tunnel settings are provided' + }); + } + return { kind, port: hostPort }; + } + if (!domain) { + throw new ConfigError({ + variable: 'NEHEMIAH_HOST_TUNNEL_DOMAIN', + message: 'must be the DNS suffix hosts publish their tunnels under' + }); + } + if (!clientId || !clientSecret) { + throw new ConfigError({ + variable: 'NEHEMIAH_HOST_ACCESS_CLIENT_ID', + message: 'and NEHEMIAH_HOST_ACCESS_CLIENT_SECRET must both carry the Access service token' + }); + } + try { + return validateHostTransport({ + kind, + domain, + accessClientId: clientId, + accessClientSecret: Redacted.make(clientSecret) + }); + } catch (error) { + if (error instanceof HostTransportError) { + throw new ConfigError({ variable: 'NEHEMIAH_HOST_TUNNEL_DOMAIN', message: error.message }); + } + throw error; + } +}; + const boolean = (env: NodeJS.ProcessEnv, name: string, fallback: boolean): boolean => { const raw = env[name]; if (raw === undefined || raw === '') return fallback; @@ -780,6 +840,8 @@ export const loadConfig = ( if (!hostCidrs.length) { throw new ConfigError({ variable: 'NEHEMIAH_HOST_CIDRS', message: 'must not be empty' }); } + const hostPort = integer(env, 'NEHEMIAH_HOST_PORT', 8080); + const hostTransport = loadHostTransport(env, hostPort); return { environment, host: env.NEHEMIAH_HOST ?? '0.0.0.0', @@ -794,7 +856,8 @@ export const loadConfig = ( previewBaseDomain, trustedSiteDomain, hostCidrs, - hostPort: integer(env, 'NEHEMIAH_HOST_PORT', 8080), + hostPort, + hostTransport, clerkIssuer: env.CLERK_ISSUER, clerkAudience: env.CLERK_AUDIENCE, hostStaleAfterMs: integer(env, 'NEHEMIAH_HOST_STALE_MS', 30_000), diff --git a/apps/nehemiah/src/main.ts b/apps/nehemiah/src/main.ts index c7de590..068e3f3 100644 --- a/apps/nehemiah/src/main.ts +++ b/apps/nehemiah/src/main.ts @@ -187,7 +187,7 @@ const start = async (): Promise => { new PostgresHostCredentialResolver(database, hostCredentialCipher), fetch, 30_000, - config.hostPort + config.hostTransport ); const templateTransferClient = config.templateTransfers && config.objectStorage @@ -200,7 +200,7 @@ const start = async (): Promise => { }), fetch, config.templateTransfers.hostTimeoutMs, - config.hostPort + config.hostTransport ) : undefined; const usage = new UsageLedger(database); diff --git a/apps/nehemiah/test/host-transport.test.ts b/apps/nehemiah/test/host-transport.test.ts new file mode 100644 index 0000000..140dd57 --- /dev/null +++ b/apps/nehemiah/test/host-transport.test.ts @@ -0,0 +1,88 @@ +import { Redacted } from 'effect'; +import { describe, expect, it } from 'vitest'; +import { + hostBaseUrl, + hostTransportHeaders, + hostTunnelLabel, + overlayTransport, + validateHostTransport, + type HostTransport +} from '../src/clients/host-transport.js'; +import { NehemiahdClient } from '../src/clients/nehemiahd.js'; + +const tunnel: HostTransport = { + kind: 'cloudflare-tunnel', + domain: 'hosts.example.test', + accessClientId: 'a1b2c3d4e5f6.access', + accessClientSecret: Redacted.make( + '0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef' + ) +}; + +describe('host transport', () => { + it('maps overlay addresses to stable tunnel labels', () => { + expect(hostTunnelLabel('10.64.0.7')).toBe('10-64-0-7'); + expect(hostTunnelLabel('::ffff:10.64.0.7')).toBe('10-64-0-7'); + expect(hostTunnelLabel('fd00:6e65:6865::7')).toBe('fd00-6e65-6865-0000-0000-0000-0000-0007'); + expect(hostTunnelLabel('FD00:6E65:6865:0:0:0:0:7')).toBe( + 'fd00-6e65-6865-0000-0000-0000-0000-0007' + ); + expect(hostTunnelLabel('::1')).toBe('0000-0000-0000-0000-0000-0000-0000-0001'); + expect(() => hostTunnelLabel('hosts.example.test')).toThrow('invalid host address'); + }); + + it('keeps the overlay transport byte-for-byte compatible', () => { + expect(hostBaseUrl('10.64.0.7', overlayTransport(8080))).toBe('http://10.64.0.7:8080'); + expect(hostBaseUrl('fd00::7', overlayTransport(8080))).toBe('http://[fd00::7]:8080'); + expect(hostTransportHeaders(overlayTransport(8080))).toEqual([]); + }); + + it('addresses hosts through their tunnel hostname with the Access service token', () => { + expect(hostBaseUrl('10.64.0.7', tunnel)).toBe('https://10-64-0-7.hosts.example.test'); + expect(hostTransportHeaders(tunnel)).toEqual([ + ['cf-access-client-id', 'a1b2c3d4e5f6.access'], + ['cf-access-client-secret', Redacted.value(tunnel.accessClientSecret)] + ]); + }); + + it('rejects malformed transports before any request is sent', () => { + expect(() => validateHostTransport(overlayTransport(0))).toThrow('host port'); + expect(() => validateHostTransport({ ...tunnel, domain: 'hosts' })).toThrow('DNS suffix'); + expect(() => validateHostTransport({ ...tunnel, domain: '.hosts.example.test' })).toThrow( + 'DNS suffix' + ); + expect(() => validateHostTransport({ ...tunnel, accessClientId: 'short' })).toThrow( + 'Access service token' + ); + expect(validateHostTransport(tunnel)).toBe(tunnel); + }); + + it('routes host client requests through the tunnel without changing host identity', async () => { + let requestUrl = ''; + let requestInit: RequestInit | undefined; + const client = new NehemiahdClient( + { resolve: async () => 'host-control-token-that-is-long-enough-000000' }, + async (input, init) => { + requestUrl = String(input); + requestInit = init; + return Response.json({ + id: 'local-1', + status: 'running', + ready: true, + lease_id: 'lease-1', + metadata: { public_machine_id: 'unused' } + }); + }, + 1_000, + tunnel + ); + await client.get('10.64.0.9', 'local-1'); + expect(requestUrl).toBe('https://10-64-0-9.hosts.example.test/internal/v1/machines/local-1'); + const headers = new Headers(requestInit?.headers); + expect(headers.get('cf-access-client-id')).toBe('a1b2c3d4e5f6.access'); + expect(headers.get('cf-access-client-secret')).toBe(Redacted.value(tunnel.accessClientSecret)); + expect(headers.get('authorization')).toBe( + 'Bearer host-control-token-that-is-long-enough-000000' + ); + }); +}); diff --git a/apps/web/.env.example b/apps/web/.env.example index 330d74f..380fe70 100644 --- a/apps/web/.env.example +++ b/apps/web/.env.example @@ -6,9 +6,11 @@ # NEHEMIAH_URL=http://localhost:8080 # NEHEMIAH_TOKEN= # -# --- Production (set on your Vercel/host project, not here) --- +# --- Production (Cloudflare Workers, deployed by `npm run deploy`) --- +# alchemy.run.ts forwards the settings below to the Worker when they are set +# in the deploying shell, this file, or CI (see docs/cloudflare.md). # The browser talks to nehemiahd directly (the TTY/VNC/agent are WebSockets and -# can't go through a serverless function), so expose your endpoint publicly: +# can't go through the Worker), so expose your endpoint publicly: # PUBLIC_NEHEMIAH_URL=https://your-nehemiahd.example.com # # Optional: auto-open an SSH tunnel to a remote/private nehemiahd when you run @@ -36,3 +38,22 @@ # Optional public support contacts. Unsafe values are omitted from the page. # PUBLIC_SUPPORT_URL=https://support.boringcomputers.com/help # PUBLIC_SUPPORT_EMAIL=support@boringcomputers.com + +# --- Cloudflare stack (deploy time only; never read by the app) --- +# Local deploys authenticate with `npx alchemy profile edit --add Cloudflare`. +# CI sets these two instead: +# CLOUDFLARE_ACCOUNT_ID= +# CLOUDFLARE_API_TOKEN= +# Stage: `prod` owns the domain, tunnel, DNS, analytics, and R2 bucket. Any +# other stage deploys the site alone on workers.dev. +# ALCHEMY_STAGE=prod +# SITE_DOMAIN=boringcomputers.com +# API_HOSTNAME=api.boringcomputers.com # control plane, via the tunnel +# GATEWAY_HOSTNAME=gateway.boringcomputers.com # public gateway, via the tunnel +# CONTROL_PLANE_ORIGIN=http://127.0.0.1:8081 # as reached from cloudflared +# GATEWAY_ORIGIN=http://127.0.0.1:8082 +# Machine previews need their own registrable domain (not a subdomain of the +# site); the wildcard is only created when this is set. +# PREVIEW_BASE_DOMAIN=example-previews.app +# PREVIEW_ZONE_NAME=example-previews.app # zone holding it (default: same) +# R2_BUCKET=nehemiah-artifacts diff --git a/apps/web/.gitignore b/apps/web/.gitignore index 41f1bdc..f9d2715 100644 --- a/apps/web/.gitignore +++ b/apps/web/.gitignore @@ -3,10 +3,13 @@ node_modules # Output .output .vercel +.alchemy .netlify .wrangler /.svelte-kit /build +# Alchemy's Cloudflare adapter writes the workerd bundle here during deploys. +/dist # OS .DS_Store diff --git a/apps/web/alchemy.run.ts b/apps/web/alchemy.run.ts new file mode 100644 index 0000000..ddbd068 --- /dev/null +++ b/apps/web/alchemy.run.ts @@ -0,0 +1,362 @@ +/** + * Cloudflare deployment of everything that can run on Cloudflare, declared + * with Alchemy (https://alchemy.run) and applied with + * `npm run deploy -- --stage ` from this directory. + * + * Stages: + * - `prod` (pushes to main): the site on its custom domain with a www redirect + * and Web Analytics; the control plane and the public gateway as Cloudflare + * Containers behind one edge Worker (`api.`, `gateway.`, and the preview + * wildcard); the Access applications and service token that guard host + * tunnels and internal routes; the R2 artifact bucket. + * - anything else (`pr-` previews, `live_` local runs): the site + * only, served from its workers.dev URL. Nothing in the zone is touched. + * + * Deliberately not here: the Latitude hosts (Firecracker needs bare metal). + * They keep their overlay address as identity and publish their internal API + * through their own Cloudflare Tunnel (docs/nehemiah/adr/0005). See + * docs/cloudflare.md for the runbook and .env.example for every setting. + */ +import * as Alchemy from 'alchemy'; +import * as Cloudflare from 'alchemy/Cloudflare'; +import * as Output from 'alchemy/Output'; +import * as Config from 'effect/Config'; +import * as Effect from 'effect/Effect'; +import * as Option from 'effect/Option'; +import * as Redacted from 'effect/Redacted'; + +/** + * Site settings forwarded to the site Worker when present. They surface + * through `$env/dynamic/*`; a missing value means the matching feature + * (dashboard, status probes, support page) is unconfigured. In `prod` the + * control-plane and gateway URLs default to the hostnames deployed here. + */ +const SITE_ENV = [ + 'PUBLIC_NEHEMIAH_URL', + 'PRIVATE_NEHEMIAH_URL', + 'PUBLIC_CLERK_PUBLISHABLE_KEY', + 'PUBLIC_CLERK_FRONTEND_API', + 'STATUS_CONTROL_PLANE_URL', + 'STATUS_GATEWAY_URL', + 'PUBLIC_SUPPORT_URL', + 'PUBLIC_SUPPORT_EMAIL' +] as const; + +/** Control-plane tuning forwarded verbatim when set (apps/nehemiah/.env.example). */ +const CONTROL_PLANE_OPTIONAL_ENV = [ + 'STRIPE_WEBHOOK_SECRET', + 'NEHEMIAH_DEFAULT_REGION', + 'NEHEMIAH_HOST_STALE_MS', + 'NEHEMIAH_API_RATE_LIMIT_ENABLED', + 'NEHEMIAH_API_RATE_LIMIT_FAIL_CLOSED', + 'NEHEMIAH_API_RATE_WINDOW_SECONDS', + 'NEHEMIAH_API_PREAUTH_IP_REQUESTS', + 'NEHEMIAH_API_PREAUTH_KEY_REQUESTS', + 'NEHEMIAH_API_PRINCIPAL_REQUESTS', + 'NEHEMIAH_API_PROJECT_REQUESTS', + 'NEHEMIAH_API_ORGANIZATION_REQUESTS', + 'NEHEMIAH_OTEL_EXPORT_INTERVAL_MS', + 'NEHEMIAH_OTEL_EXPORT_TIMEOUT_MS', + 'NEHEMIAH_OTEL_TRACE_SAMPLE_RATIO' +] as const; + +/** Gateway tuning forwarded verbatim when set (gateway/config.go). */ +const GATEWAY_OPTIONAL_ENV = [ + 'NEHEMIAH_OTEL_EXPORT_INTERVAL_MS', + 'NEHEMIAH_OTEL_EXPORT_TIMEOUT_MS', + 'NEHEMIAH_OTEL_TRACE_SAMPLE_RATIO', + 'NEHEMIAH_GATEWAY_STREAM_MAX_DURATION', + 'NEHEMIAH_GATEWAY_TENANT_CONNECTIONS', + 'NEHEMIAH_GATEWAY_TENANT_BYTES_PER_SECOND', + 'NEHEMIAH_GATEWAY_REST_REQUESTS_PER_WINDOW' +] as const; + +/** Build contexts are anchored here so the stack works from any cwd. */ +const repositoryRoot = `${import.meta.dirname}/../..`; + +/** An optional setting; blank values (an unset CI variable) count as unset. */ +const optional = (name: string) => + Config.String(name).pipe( + Config.option, + Config.map(Option.filter((value) => value.trim() !== '')) + ); + +const withDefault = (name: string, fallback: string) => + optional(name).pipe(Config.map(Option.getOrElse(() => fallback))); + +/** A required secret; a missing value fails the deploy with its name. */ +const secret = (name: string) => Config.Redacted(name); + +/** + * Container environment values: literals, redacted secrets, or outputs of + * sibling resources (Alchemy resolves those at apply time and keeps redacted + * outputs encrypted in state). + */ +type ContainerEnvValue = string | Redacted.Redacted; +type ContainerEnv = Record>; + +/** Collect the optional settings that are present, keeping them redacted in state. */ +const optionalEnv = (names: ReadonlyArray) => + Effect.gen(function* () { + const env: ContainerEnv = {}; + for (const name of names) { + const value = yield* optional(name); + if (Option.isSome(value)) env[name] = Redacted.make(value.value); + } + return env; + }); + +export default Alchemy.Stack( + 'boring-computers', + { providers: Cloudflare.providers(), state: Cloudflare.state() }, + Effect.gen(function* () { + const stage = yield* Alchemy.Stage; + const production = stage === 'prod'; + const siteDomain = yield* withDefault('SITE_DOMAIN', 'boringcomputers.com'); + const apiHostname = yield* withDefault('API_HOSTNAME', `api.${siteDomain}`); + const gatewayHostname = yield* withDefault('GATEWAY_HOSTNAME', `gateway.${siteDomain}`); + + // ---- The site ----------------------------------------------------------- + const siteDefaults: Record = production + ? { + PUBLIC_NEHEMIAH_URL: `https://${gatewayHostname}`, + PRIVATE_NEHEMIAH_URL: `https://${apiHostname}`, + STATUS_CONTROL_PLANE_URL: `https://${apiHostname}`, + STATUS_GATEWAY_URL: `https://${gatewayHostname}` + } + : {}; + const siteEnv: Record< + string, + Config.Config> | Redacted.Redacted + > = {}; + for (const name of SITE_ENV) { + if (Option.isSome(yield* optional(name))) siteEnv[name] = Config.Redacted(name); + else if (siteDefaults[name]) siteEnv[name] = Redacted.make(siteDefaults[name]); + } + + const site = yield* Cloudflare.Website.SvelteKit('Website', { + env: siteEnv, + // The zone must already exist in the account; Alchemy attaches the + // custom domain (DNS + certificate) and 301s www to the apex. + domain: production ? { name: siteDomain, redirects: [`www.${siteDomain}`] } : undefined + }); + + if (!production) return { url: site.url }; + + // ---- Zone-level pieces: only the production stage owns anything here. ---- + + // The zone predates this stack: adopt it for its id and never delete it. + const zone = yield* Cloudflare.Zone.Zone('Zone', { name: siteDomain }).pipe( + Alchemy.AdoptPolicy.adopt(true), + Alchemy.RemovalPolicy.retain() + ); + + // Privacy-first page analytics, injected at the edge (replaces + // @vercel/analytics, so the app ships no analytics code of its own). + yield* Cloudflare.Rum.Site('WebAnalytics', { zoneTag: zone.zoneId, autoInstall: true }); + + // Release/template artifacts. Retained on destroy: the control plane's + // S3-compatible client holds customer-adjacent data here. + const artifacts = yield* Cloudflare.R2.Bucket('Artifacts', { + name: yield* withDefault('R2_BUCKET', 'nehemiah-artifacts') + }).pipe(Alchemy.RemovalPolicy.retain()); + + // ---- The managed cloud: control plane + gateway as Containers ---------- + + // Machine previews must use a different registrable domain from the site + // (both services refuse anything else), and both refuse to start in + // production without one, so it is required here. + const previewBaseDomain = yield* Config.String('PREVIEW_BASE_DOMAIN').pipe( + Config.map((value) => value.trim().toLowerCase()) + ); + const previewZoneName = yield* withDefault('PREVIEW_ZONE_NAME', previewBaseDomain); + // Hosts publish their tunnels at
.. + const hostTunnelDomain = yield* withDefault('HOST_TUNNEL_DOMAIN', `hosts.${siteDomain}`); + const hostCidrs = yield* withDefault('NEHEMIAH_HOST_CIDRS', '10.64.0.0/16,fd00:6e65:6865::/64'); + // The managed-host release version; the control plane rejects daemons + // that do not report it. + const serviceVersion = yield* Config.String('NEHEMIAH_SERVICE_VERSION'); + // Telemetry region label for both services (the control plane reads it + // as NEHEMIAH_DEFAULT_REGION, the gateway as NEHEMIAH_REGION). + const region = yield* withDefault('NEHEMIAH_DEFAULT_REGION', 'ca-tor-1'); + const otelEndpoint = yield* Config.String('NEHEMIAH_OTEL_ENDPOINT'); + const otelAuthorization = yield* secret('NEHEMIAH_OTEL_AUTHORIZATION'); + const gatewayOtelAuthorization = Option.getOrElse( + yield* Config.Redacted('GATEWAY_OTEL_AUTHORIZATION').pipe(Config.option), + () => otelAuthorization + ); + const databaseUrl = yield* secret('DATABASE_URL'); + // Dashboard sign-in; the control plane refuses to start in production + // without both. + const clerkIssuer = yield* Config.String('CLERK_ISSUER'); + const clerkAudience = yield* Config.String('CLERK_AUDIENCE'); + const hostCredentialKey = yield* secret('NEHEMIAH_HOST_CREDENTIAL_KEY'); + const gatewayToken = yield* secret('NEHEMIAH_GATEWAY_TOKEN'); + const gatewaySecret = yield* secret('NEHEMIAH_GATEWAY_SECRET'); + const deviceCodePepper = yield* secret('NEHEMIAH_DEVICE_CODE_PEPPER'); + // The Access service token hosts present on internal routes. Minted by + // the operator (Zero Trust → Access → Service Auth) so its secret never + // enters this stack's state; only its id is needed for the policy. + const fleetAccessTokenId = Option.getOrUndefined(yield* optional('FLEET_ACCESS_TOKEN_ID')); + + // The service token the control plane and gateway present to Access, + // both toward host tunnels and toward the control plane's internal routes. + const services = yield* Cloudflare.Access.ServiceToken('ServicesToken', { + name: `boring-computers-${stage}-services`, + duration: '8760h' + }); + // Cloudflare reveals the secret only on create or rotate. A token adopted + // from the account has none in state, so fail the apply rather than start + // the containers with an empty credential. + const servicesSecret = services.clientSecret.pipe( + Output.map((value) => { + if (value === undefined) { + throw new Error( + 'The services Access token has no secret in state; bump clientSecretVersion to mint one.' + ); + } + return value; + }) + ); + const serviceTokenRule = { serviceToken: services.serviceTokenId }; + + yield* Cloudflare.Access.Application('HostsAccess', { + type: 'self_hosted', + name: `boring-computers-${stage}-hosts`, + domain: `*.${hostTunnelDomain}`, + policies: [ + { + name: 'control plane and gateway', + decision: 'non_identity', + include: [serviceTokenRule] + } + ] + }); + yield* Cloudflare.Access.Application('InternalRoutesAccess', { + type: 'self_hosted', + name: `boring-computers-${stage}-internal`, + domain: `${apiHostname}/internal`, + policies: [ + { + name: 'gateway and fleet service tokens', + decision: 'non_identity', + include: [ + serviceTokenRule, + ...(fleetAccessTokenId ? [{ serviceToken: fleetAccessTokenId }] : []) + ] + } + ] + }); + + const controlPlaneEnv: ContainerEnv = { + NODE_ENV: 'production', + NEHEMIAH_HOST: '0.0.0.0', + PORT: '8081', + DATABASE_URL: databaseUrl, + CLERK_ISSUER: clerkIssuer, + CLERK_AUDIENCE: clerkAudience, + NEHEMIAH_HOST_CREDENTIAL_KEY: hostCredentialKey, + NEHEMIAH_GATEWAY_TOKEN: gatewayToken, + NEHEMIAH_GATEWAY_SECRET: gatewaySecret, + NEHEMIAH_DEVICE_CODE_PEPPER: deviceCodePepper, + NEHEMIAH_GATEWAY_URL: `https://${gatewayHostname}`, + NEHEMIAH_TRUSTED_SITE_DOMAIN: siteDomain, + NEHEMIAH_PREVIEW_BASE_DOMAIN: previewBaseDomain, + NEHEMIAH_HOST_CIDRS: hostCidrs, + NEHEMIAH_HOST_PORT: '8080', + NEHEMIAH_HOST_TRANSPORT: 'cloudflare-tunnel', + NEHEMIAH_HOST_TUNNEL_DOMAIN: hostTunnelDomain, + NEHEMIAH_HOST_ACCESS_CLIENT_ID: services.clientId, + NEHEMIAH_HOST_ACCESS_CLIENT_SECRET: servicesSecret, + NEHEMIAH_OTEL_ENABLED: 'true', + NEHEMIAH_OTEL_ENDPOINT: otelEndpoint, + NEHEMIAH_OTEL_AUTHORIZATION: otelAuthorization, + NEHEMIAH_SERVICE_VERSION: serviceVersion, + NEHEMIAH_INSTANCE_ID: `control-plane-${stage}`, + NEHEMIAH_DEPLOYMENT_ENVIRONMENT: 'production', + ...(yield* optionalEnv(CONTROL_PLANE_OPTIONAL_ENV)) + }; + + const gatewayEnv: ContainerEnv = { + NEHEMIAH_ENV: 'production', + NEHEMIAH_GATEWAY_ADDR: '0.0.0.0:8082', + NEHEMIAH_CONTROL_PLANE_URL: `https://${apiHostname}`, + NEHEMIAH_GATEWAY_TOKEN: gatewayToken, + NEHEMIAH_GATEWAY_SECRET: gatewaySecret, + NEHEMIAH_PREVIEW_BASE_DOMAIN: previewBaseDomain, + NEHEMIAH_TRUSTED_SITE_DOMAIN: siteDomain, + NEHEMIAH_HOST_PORT: '8080', + NEHEMIAH_GATEWAY_ALLOWED_HOST_CIDRS: hostCidrs, + NEHEMIAH_GATEWAY_HOST_TRANSPORT: 'cloudflare-tunnel', + NEHEMIAH_GATEWAY_HOST_TUNNEL_DOMAIN: hostTunnelDomain, + NEHEMIAH_GATEWAY_HOST_ACCESS_CLIENT_ID: services.clientId, + NEHEMIAH_GATEWAY_HOST_ACCESS_CLIENT_SECRET: servicesSecret, + NEHEMIAH_GATEWAY_CONTROL_PLANE_ACCESS_CLIENT_ID: services.clientId, + NEHEMIAH_GATEWAY_CONTROL_PLANE_ACCESS_CLIENT_SECRET: servicesSecret, + // The edge Worker is the only path into the container, so every peer + // is the edge and CF-Connecting-IP is authoritative. + NEHEMIAH_GATEWAY_TRUSTED_EDGE_CIDRS: '0.0.0.0/0,::/0', + NEHEMIAH_REGION: region, + NEHEMIAH_OTEL_ENABLED: 'true', + NEHEMIAH_OTEL_ENDPOINT: otelEndpoint, + NEHEMIAH_OTEL_AUTHORIZATION: gatewayOtelAuthorization, + NEHEMIAH_SERVICE_VERSION: serviceVersion, + NEHEMIAH_INSTANCE_ID: `gateway-${stage}`, + NEHEMIAH_DEPLOYMENT_ENVIRONMENT: 'production', + ...(yield* optionalEnv(GATEWAY_OPTIONAL_ENV)) + }; + + // One Worker fronts both containers (edge/worker.ts). Custom domains + // carry the API and gateway; the preview wildcard is a zone route. + const edge = yield* Cloudflare.Worker('Edge', { + main: 'edge/worker.ts', + env: { + API_HOSTNAME: apiHostname, + // Lets the Worker sign each caller's address for the control plane + // the way the gateway does (edge/worker.ts). + NEHEMIAH_GATEWAY_TOKEN: gatewayToken, + CONTROL_PLANE: Cloudflare.Container('ControlPlane', { + context: repositoryRoot, + dockerfile: 'apps/nehemiah/Dockerfile', + className: 'ControlPlane', + instanceType: 'standard-2', + maxInstances: 2, + env: controlPlaneEnv + }), + GATEWAY: Cloudflare.Container('Gateway', { + context: `${repositoryRoot}/gateway`, + className: 'Gateway', + instanceType: 'standard-2', + maxInstances: 2, + env: gatewayEnv + }) + }, + domain: { name: apiHostname, aliases: [gatewayHostname] }, + routes: [{ pattern: `*.${previewBaseDomain}/*`, zoneName: previewZoneName }] + }); + + // A zone route only fires for proxied DNS names; point the wildcard at + // the documentation placeholder address so Cloudflare terminates it. + const previewZone = yield* Cloudflare.Zone.Zone('PreviewZone', { + name: previewZoneName + }).pipe(Alchemy.AdoptPolicy.adopt(true), Alchemy.RemovalPolicy.retain()); + yield* Cloudflare.DNS.Record('PreviewDns', { + zoneId: previewZone.zoneId, + name: `*.${previewBaseDomain}`, + type: 'A', + content: '192.0.2.1', + proxied: true, + comment: 'Nehemiah machine previews via the edge Worker (managed by alchemy)' + }); + + return { + url: site.url, + api: `https://${apiHostname}`, + gateway: `https://${gatewayHostname}`, + edgeWorker: edge.workerName, + hostTunnelDomain, + servicesAccessClientId: services.clientId, + bucket: artifacts.bucketName + }; + }) +); diff --git a/apps/web/edge/worker.ts b/apps/web/edge/worker.ts new file mode 100644 index 0000000..66038a2 --- /dev/null +++ b/apps/web/edge/worker.ts @@ -0,0 +1,126 @@ +/** + * Edge Worker in front of the Cloudflare Containers that run the managed + * cloud (apps/web/alchemy.run.ts, `prod` stage): + * + * - `api.` → the control plane (apps/nehemiah) + * - `gateway.` → the public gateway (gateway/) + * - `*.` → the public gateway (machine previews) + * + * Requests are forwarded byte-for-byte, WebSocket upgrades included. Cloudflare + * Access guards `api./internal/*` at the edge before this code runs, so + * host and gateway traffic to internal routes always carries a service token. + * + * Client identity. Inside a container the TCP peer is always this Worker's + * runtime, so the services cannot learn a caller's address from the socket. + * The gateway already trusts `CF-Connecting-IP` from a configured edge. The + * control plane trusts only an address signed with the gateway token + * (apps/nehemiah/src/http/client-identity.ts), so for `api.` this Worker + * signs `CF-Connecting-IP` the way the gateway does when it proxies. A request + * that already carries a valid, fresh signature (the gateway calling the + * control plane through this same edge) is passed through unchanged, so the + * end user's address, not the gateway container's egress, reaches admission. + * + * Each class is a Durable Object that owns one container instance. Both are + * addressed by a fixed name, so the beta runs exactly one control plane and + * one gateway; host heartbeats and open streams keep them awake. + */ +import { Container, getContainer } from '@cloudflare/containers'; + +export class ControlPlane extends Container { + defaultPort = 8081; + sleepAfter = '2h'; + enableInternet = true; +} + +export class Gateway extends Container { + defaultPort = 8082; + sleepAfter = '2h'; + enableInternet = true; +} + +interface Env { + readonly CONTROL_PLANE: DurableObjectNamespace; + readonly GATEWAY: DurableObjectNamespace; + readonly API_HOSTNAME: string; + /** The private gateway credential; also the key for signed client addresses. */ + readonly NEHEMIAH_GATEWAY_TOKEN: string; +} + +const clientAddressHeader = 'x-nehemiah-client-address'; +const clientTimestampHeader = 'x-nehemiah-client-timestamp'; +const clientSignatureHeader = 'x-nehemiah-client-signature'; +/** Matches the control plane's acceptance window for a signed timestamp. */ +const signatureMaxSkewSeconds = 30; + +const encoder = new TextEncoder(); + +const hex = (bytes: ArrayBuffer): string => + Array.from(new Uint8Array(bytes), (byte) => byte.toString(16).padStart(2, '0')).join(''); + +const signClientAddress = async ( + token: string, + address: string, + timestamp: string +): Promise => { + const key = await crypto.subtle.importKey( + 'raw', + encoder.encode(token), + { name: 'HMAC', hash: 'SHA-256' }, + false, + ['sign'] + ); + const payload = `nehemiah-client-address-v1\n${timestamp}\n${address}`; + return hex(await crypto.subtle.sign('HMAC', key, encoder.encode(payload))); +}; + +const constantTimeEqual = (a: string, b: string): boolean => { + if (a.length !== b.length) return false; + let difference = 0; + for (let index = 0; index < a.length; index += 1) { + difference |= a.charCodeAt(index) ^ b.charCodeAt(index); + } + return difference === 0; +}; + +/** True when the request already carries a fresh signature made with our token. */ +const carriesValidSignature = async (headers: Headers, token: string): Promise => { + const address = headers.get(clientAddressHeader)?.trim(); + const timestamp = headers.get(clientTimestampHeader)?.trim(); + const signature = headers.get(clientSignatureHeader)?.trim().toLowerCase(); + if (!address || !timestamp || !signature) return false; + if (!/^[0-9]{1,12}$/.test(timestamp) || !/^[a-f0-9]{64}$/.test(signature)) return false; + const skew = Math.abs(Math.floor(Date.now() / 1_000) - Number(timestamp)); + if (skew > signatureMaxSkewSeconds) return false; + return constantTimeEqual(await signClientAddress(token, address, timestamp), signature); +}; + +/** Forward to the control plane with the caller's address signed for admission. */ +const toControlPlane = async (request: Request, env: Env): Promise => { + const headers = new Headers(request.headers); + if (!(await carriesValidSignature(headers, env.NEHEMIAH_GATEWAY_TOKEN))) { + headers.delete(clientAddressHeader); + headers.delete(clientTimestampHeader); + headers.delete(clientSignatureHeader); + const address = request.headers.get('cf-connecting-ip')?.trim(); + if (address) { + const timestamp = Math.floor(Date.now() / 1_000).toString(); + headers.set(clientAddressHeader, address); + headers.set(clientTimestampHeader, timestamp); + headers.set( + clientSignatureHeader, + await signClientAddress(env.NEHEMIAH_GATEWAY_TOKEN, address, timestamp) + ); + } + } + return getContainer(env.CONTROL_PLANE, 'primary').fetch(new Request(request, { headers })); +}; + +export default { + async fetch(request: Request, env: Env): Promise { + const hostname = new URL(request.url).hostname.toLowerCase(); + if (hostname === env.API_HOSTNAME) return toControlPlane(request, env); + // The gateway reads the caller from CF-Connecting-IP itself + // (NEHEMIAH_GATEWAY_TRUSTED_EDGE_CIDRS covers this Worker's runtime). + return getContainer(env.GATEWAY, 'primary').fetch(request); + } +} satisfies ExportedHandler; diff --git a/apps/web/package.json b/apps/web/package.json index 2aaa31e..f736845 100644 --- a/apps/web/package.json +++ b/apps/web/package.json @@ -8,20 +8,26 @@ "build": "vite build", "preview": "vite preview", "prepare": "svelte-kit sync || echo ''", - "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json", + "check": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json && tsc --noEmit -p tsconfig.alchemy.json && tsc --noEmit -p tsconfig.edge.json", "check:watch": "svelte-kit sync && svelte-check --tsconfig ./tsconfig.json --watch", "lint": "prettier --check . && eslint .", "format": "prettier --write .", "test:unit": "vitest", "test": "npm run test:unit -- --run", - "test:e2e": "playwright install && playwright test" + "test:e2e": "playwright install && playwright test", + "deploy": "alchemy deploy", + "plan": "alchemy plan", + "destroy": "alchemy destroy" }, "devDependencies": { + "@alchemy.run/frontend-frameworks": "2.0.0-beta.79", + "@cloudflare/containers": "0.3.7", + "@cloudflare/workers-types": "^5.20260919.1", + "@effect/platform-node": "4.0.0-rc.116", "@nehemiah/eslint-config": "*", "@nehemiah/prettier-config": "*", "@nehemiah/typescript-config": "*", "@playwright/test": "^1.60.0", - "@sveltejs/adapter-vercel": "^6.3.3", "@sveltejs/kit": "^2.63.0", "@sveltejs/vite-plugin-svelte": "^7.1.2", "@tailwindcss/forms": "^0.5.11", @@ -29,6 +35,8 @@ "@tailwindcss/vite": "^4.3.0", "@types/node": "^24", "@vitest/browser-playwright": "^4.1.10", + "alchemy": "2.0.0-beta.79", + "effect": "4.0.0-rc.116", "eslint": "^10.4.1", "playwright": "^1.60.0", "prettier": "^3.8.3", @@ -42,7 +50,6 @@ }, "dependencies": { "@novnc/novnc": "^1.7.0", - "@vercel/analytics": "^2.0.1", "@xterm/addon-fit": "^0.10.0", "@xterm/xterm": "^5.5.0", "cookie": "0.7.2" diff --git a/apps/web/src/hooks.client.ts b/apps/web/src/hooks.client.ts deleted file mode 100644 index ffad00f..0000000 --- a/apps/web/src/hooks.client.ts +++ /dev/null @@ -1,4 +0,0 @@ -import { dev } from '$app/environment'; -import { injectAnalytics } from '@vercel/analytics/sveltekit'; - -injectAnalytics({ mode: dev ? 'development' : 'production' }); diff --git a/apps/web/tsconfig.alchemy.json b/apps/web/tsconfig.alchemy.json new file mode 100644 index 0000000..6e5d84e --- /dev/null +++ b/apps/web/tsconfig.alchemy.json @@ -0,0 +1,12 @@ +{ + "extends": "@nehemiah/typescript-config/base.json", + "compilerOptions": { + "module": "ESNext", + "moduleResolution": "bundler", + "target": "ES2023", + "lib": ["ES2023"], + "types": ["node"], + "noEmit": true + }, + "include": ["alchemy.run.ts"] +} diff --git a/apps/web/tsconfig.edge.json b/apps/web/tsconfig.edge.json new file mode 100644 index 0000000..b2f8826 --- /dev/null +++ b/apps/web/tsconfig.edge.json @@ -0,0 +1,12 @@ +{ + "extends": "@nehemiah/typescript-config/base.json", + "compilerOptions": { + "module": "ESNext", + "moduleResolution": "bundler", + "target": "ES2023", + "lib": ["ES2023"], + "types": ["@cloudflare/workers-types"], + "noEmit": true + }, + "include": ["edge/**/*.ts"] +} diff --git a/apps/web/vite.config.ts b/apps/web/vite.config.ts index 28fb020..e8e0d68 100644 --- a/apps/web/vite.config.ts +++ b/apps/web/vite.config.ts @@ -2,7 +2,6 @@ import tailwindcss from '@tailwindcss/vite'; import { defineConfig } from 'vitest/config'; import { loadEnv } from 'vite'; import { playwright } from '@vitest/browser-playwright'; -import adapter from '@sveltejs/adapter-vercel'; import { sveltekit } from '@sveltejs/kit/vite'; // nehemiahd host daemon. Reads NEHEMIAH_URL / NEHEMIAH_TOKEN from apps/web/.env @@ -29,8 +28,10 @@ export default defineConfig(({ mode }) => { // Force runes mode for the project, except for libraries. Can be removed in svelte 6. runes: ({ filename }) => filename.split(/[/\\]/).includes('node_modules') ? undefined : true - }, - adapter: adapter() + } + // No adapter here: `npm run deploy` (alchemy.run.ts) builds with + // Alchemy's Cloudflare Workers adapter. A plain `vite build` still + // type-checks and prerenders; it just has no platform to package for. }) ], server: { diff --git a/docs/architecture.md b/docs/architecture.md index bcce19c..0993c74 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -92,7 +92,8 @@ the local/self-hosted prototype contract remains useful as a development target. ## One-box prototype Everything below runs on a **single Latitude.sh `c3.small.x86` in MIA2** (Ubuntu -24.04, 6 cores / 32 GB, `/dev/kvm`). The SvelteKit hero site is on Vercel; the +24.04, 6 cores / 32 GB, `/dev/kvm`). The SvelteKit hero site runs on Cloudflare +Workers (deployed with Alchemy, see [`cloudflare.md`](cloudflare.md)); the browser talks to nehemiahd over an SSH-tunneled WebSocket. ``` @@ -103,7 +104,7 @@ browser talks to nehemiahd over an SSH-tunneled WebSocket. │ HTTPS │ │ REST host API + /tty WebSocket bridge │ │ ▼ │ │ registry (sync.Mutex) · TTL reaper · NEHEMIAH_MAX │ │ ┌──────────────┐ WS/HTTP│ └───┬───────────────┬───────────────┬─────────────┘ │ - │ Vercel hero │ (SSH │ │ stdin/stdout │ stdin/stdout │ stdin/stdout │ + │ Workers hero │ (SSH │ │ stdin/stdout │ stdin/stdout │ stdin/stdout │ │ site (Svelte)│ tunnel)│ │ (serial) │ (serial) │ (serial) │ │ ── WS ───────┼────────────────┼───────────────┼───────────────┼────────── │ └──────────────┘ │ ▼ ▼ ▼ │ diff --git a/docs/cloudflare.md b/docs/cloudflare.md new file mode 100644 index 0000000..a9df577 --- /dev/null +++ b/docs/cloudflare.md @@ -0,0 +1,143 @@ +# Cloudflare: the site and the edge, as code + +`apps/web/alchemy.run.ts` is an [Alchemy](https://alchemy.run) stack named +`boring-computers`. One deploy builds the SvelteKit site, uploads it as a +Cloudflare Worker, and reconciles the edge resources that sit in front of the +managed cloud. Everything below is applied from `apps/web`. + +## What the stack owns + +| Resource | Stage | Notes | +| ------------------------------------------ | ----- | --------------------------------------------------------------------------------------------- | +| Worker `Website` (SvelteKit) | all | Built by Alchemy's wrangler-free Cloudflare adapter; `nodejs_compat` is set automatically. | +| Custom domain + `www` 301 | prod | `SITE_DOMAIN` (default `boringcomputers.com`); DNS and certificate are managed by Cloudflare. | +| Web Analytics | prod | Edge-injected beacon on the zone. Replaces `@vercel/analytics`; the app ships no beacon. | +| Tunnel `boring-computers-prod-origin` | prod | Ingress: `api.` → control plane, `gateway.` and `*.` → gateway, catch-all 404. | +| DNS CNAMEs (`api`, `gateway`, `*.preview`) | prod | Proxied, pointing at `.cfargotunnel.com`. | +| R2 bucket | prod | `R2_BUCKET` (default `nehemiah-artifacts`). Retained on `destroy`. | + +The stack adopts the existing zone(s) by name for their ids and marks them +`retain`, so `alchemy destroy` never deletes a zone. Everything else in the +table is created by the stack. + +Not managed here, on purpose: + +- **Latitude hosts.** Firecracker needs bare metal with `/dev/kvm`; Cloudflare + Containers top out at 4 vCPU / 12 GiB and expose no KVM. Use + [`infra/latitude`](../infra/latitude/README.md). +- **The origin VM** running `apps/nehemiah` and `gateway/`. It sits on the + WireGuard overlay and only needs outbound connectivity once the tunnel is up. +- Neon, Clerk, Stripe, and WAF/rate-limit rules. The Alchemy providers exist + for the first three if they should move into the stack later. + +## One-time setup + +```sh +cd apps/web +npx alchemy profile edit --add Cloudflare # browser login or an API token; stored in ~/.alchemy +npm run plan -- --stage prod # shows the plan, changes nothing +npm run deploy -- --stage prod +``` + +The first deploy also creates the `alchemy-state-store` Worker in the account. +That store holds the stack state so laptops and CI see the same resources; +never fall back to local state for `prod`. + +Cutting over from Vercel: + +1. Delete the apex and `www` records that point at Vercel. Attaching a Worker + custom domain requires the hostnames to be free; Alchemy then creates them. +2. `npm run deploy -- --stage prod`, then confirm `https://boringcomputers.com` + and that `https://www.boringcomputers.com` redirects. +3. Set the CI secrets and variables below, then remove the Vercel project. + +## CI + +`.github/workflows/deploy-web.yml` deploys `prod` on pushes to `main` and a +`pr-` preview (site only, on `workers.dev`) for pull requests from this +repository, destroying the preview when the pull request closes. + +Repository secrets: + +- `CLOUDFLARE_ACCOUNT_ID` +- `CLOUDFLARE_API_TOKEN`: an account-scoped token. Start from Cloudflare's + "Edit Cloudflare Workers" template and add **Workers R2 Storage Write**, + **Cloudflare Tunnel Write**, **Secrets Store Write** (state store), plus the + zone permissions **DNS Write**, **Workers Routes Write**, and **SSL and + Certificates Write** for the site and preview zones. A failed deploy names + the missing permission in its 403. + +Repository variables (all optional; blank values are not forwarded): the +`PUBLIC_*`, `PRIVATE_NEHEMIAH_URL`, and `STATUS_*` site settings from +[`apps/web/.env.example`](../apps/web/.env.example), and the edge settings +`SITE_DOMAIN`, `API_HOSTNAME`, `GATEWAY_HOSTNAME`, `CONTROL_PLANE_ORIGIN`, +`GATEWAY_ORIGIN`, `PREVIEW_BASE_DOMAIN`, `PREVIEW_ZONE_NAME`, `R2_BUCKET`. + +## The origin VM behind the tunnel + +The tunnel connector runs next to the control plane and gateway: + +1. Take `tunnelId` from the deploy output. +2. On the VM, install `cloudflared`, mint the connector token with a logged-in + `cloudflared` (`cloudflared tunnel login`, then + `cloudflared tunnel token `), and install + [`infra/cloudflare/cloudflared.service`](../infra/cloudflare/cloudflared.service) + following the comments at the top of that file. +3. Point the ingress at the local listeners: `CONTROL_PLANE_ORIGIN` defaults to + `http://127.0.0.1:8081` (`PORT` in `apps/nehemiah`) and `GATEWAY_ORIGIN` to + `http://127.0.0.1:8082` (`NEHEMIAH_GATEWAY_ADDR`). Bind both to loopback. +4. Requests now arrive from `cloudflared` on loopback. Configure the gateway's + trusted-edge CIDR to that loopback address and its client-address header to + `CF-Connecting-IP` (see `gateway/config.go`) so rate limits key on the real + client, and keep the security checklist item "Cloudflare-to-origin + authentication is enforced" in mind: the tunnel is the only ingress, so + nothing else may listen on a public interface. + +WebSockets (TTY, VNC, agent streams) pass through the tunnel unchanged. + +## Machine previews + +The control plane and gateway both require `NEHEMIAH_PREVIEW_BASE_DOMAIN` to be +a **different registrable domain** from the site. Register one, add it as a +zone in the same account, set `PREVIEW_BASE_DOMAIN` (and `PREVIEW_ZONE_NAME` +if the zone is a parent of it), and the next `prod` deploy creates the proxied +`*.` CNAME and the matching tunnel ingress. A first-level wildcard on +its own zone is covered by Universal SSL; a deeper wildcard would need Advanced +Certificate Manager. + +## R2 credentials + +The bucket is created here, but the control plane talks S3. Mint an R2 S3 API +token in the dashboard (R2 → Manage API tokens) scoped to the bucket and set +the control plane's S3 settings from +[`apps/nehemiah/.env.example`](../apps/nehemiah/.env.example). Managed +template transfer and volumes stay disabled until the checks in +[`nehemiah/implementation-status.md`](nehemiah/implementation-status.md) pass. + +## Notes on the toolchain + +- Alchemy 2 is Effect 4 based, so `apps/web` carries `effect@4` (release + candidate) as a dev dependency. It is nested under `apps/web/node_modules` + and does not affect the Effect 3 used by the control plane, SDK, and CLI. +- `@alchemy.run/frontend-frameworks` declares an optional peer range of + SvelteKit 3 (pre-release) while the site runs SvelteKit 2. The mechanism it + relies on, passing kit options to the `sveltekit()` Vite plugin and reading + them back from the plugin API, exists since kit 2.62, and the build has been + verified against 2.70. The root `package.json` therefore overrides that + peer range to `^2.62.0`; drop the override when the site moves to kit 3. +- Alchemy replaces any adapter declared in `vite.config.ts` with its own, so + the site declares none. A plain `vite build` still succeeds; it only prints + that no adapter was specified. + +## Day to day + +```sh +cd apps/web +npm run deploy # your own stage (live_): site only, workers.dev +npm run destroy # tear that stage down +npm run plan -- --stage prod # diff prod without applying +npx alchemy dev # SvelteKit dev server with Worker-style platform.env +``` + +The existing `npm run dev` (Vite plus the optional SSH tunnel to a private +`nehemiahd`) is unchanged and remains the usual local loop. diff --git a/docs/nehemiah/adr/0005-cloudflare-containers-and-host-tunnels.md b/docs/nehemiah/adr/0005-cloudflare-containers-and-host-tunnels.md new file mode 100644 index 0000000..cb5e3ff --- /dev/null +++ b/docs/nehemiah/adr/0005-cloudflare-containers-and-host-tunnels.md @@ -0,0 +1,52 @@ +# ADR 0005: Control plane and gateway on Cloudflare Containers, hosts behind per-host tunnels + +- Status: accepted for the Cloudflare deployment; ADR 0002 remains the contract for a self-hosted control plane +- Date: 2026-09-19 +- Review trigger: a second region, a fleet large enough to need more than one gateway, or Cloudflare exposing inbound TCP/UDP to containers + +## Context + +The site already deploys to Cloudflare Workers with Alchemy (`apps/web/alchemy.run.ts`). The operator wants the rest of the managed cloud on Cloudflare too. `nehemiahd` cannot move: Firecracker needs bare metal with `/dev/kvm`, and Cloudflare Containers cap out at 4 vCPU / 12 GiB with ephemeral disk. The control plane (`apps/nehemiah`) and the public gateway (`gateway/`) can, but only if they can still reach hosts. + +ADR 0002 makes the gateway and control plane the hub of a WireGuard overlay. Cloudflare Containers cannot be that hub: Cloudflare documents that nothing but the fronting Worker can reach a container, so there is no inbound UDP endpoint for hosts to peer with, and outbound UDP is undocumented. + +## Decision + +Run the control plane and the gateway as Cloudflare Containers, each owned by a Durable Object behind one edge Worker, and reach hosts through a Cloudflare Tunnel that each host runs itself. + +- **Host identity does not change.** A host is still registered, validated, scheduled, and stored by its private overlay address (`hosts.address inet`). Only the transport to that address changes. +- **Deterministic tunnel hostnames.** A host's tunnel is published at `