Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
.env.development.local
.env.test.local
.env.production.local
.dev.vars
.dev.vars.*

npm-debug.log*
yarn-debug.log*
Expand Down Expand Up @@ -43,3 +45,6 @@ manifests
# Local Netlify folder
.netlify
.claude

# Wrangler local state
.wrangler
25 changes: 20 additions & 5 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ An overview of how the docs site is built. For writing guidelines see [`CONTRIBU
- [Overview](#overview)
- [Gotchas](#gotchas)
- [Cloudflare Worker](#cloudflare-worker)
- [Snowplow Assistant](#snowplow-assistant)
- [Custom plugins](#custom-plugins)
- [LLMs.txt and Markdown generation](#llmstxt-and-markdown-generation)
- [JSON-LD schema](#json-ld-schema)
Expand All @@ -20,7 +21,7 @@ An overview of how the docs site is built. For writing guidelines see [`CONTRIBU

## Overview

This is a Docusaurus project, deployed on Cloudflare Pages. The Docusaurus configuration is in `docusaurus.config.js`. Search is provided by [Algolia DocSearch](https://docsearch.algolia.com).
This is a Docusaurus project, deployed as a Cloudflare Worker with static assets. The Docusaurus configuration is in `docusaurus.config.js`. Search is provided by [Algolia DocSearch](https://docsearch.algolia.com).

Pages are MDX under the hood, but mostly carry `.md` extensions for Docusaurus legacy reasons.

Expand Down Expand Up @@ -54,13 +55,26 @@ Non-obvious things that could cause confusion if you don't know about them:

## Cloudflare Worker

`worker/index.js` runs on every request and does three things:
1. **Server-side Snowplow tracking**
2. **Forced redirects** via `findForcedRedirect(pathname)`, checked before asset fetch, returns 301 on match
3. **Fallback redirects** via `findFallbackRedirect(pathname)`, checked only after a 404
`worker/index.js` runs on every request and does four things:
1. **Assistant API proxy** for `/api/*` requests, handled first so they never fire a server-side page view (see [Snowplow Assistant](#snowplow-assistant))
2. **Server-side Snowplow tracking**
3. **Forced redirects** via `findForcedRedirect(pathname)`, checked before asset fetch, returns 301 on match
4. **Fallback redirects** via `findFallbackRedirect(pathname)`, checked only after a 404

Both redirect tiers are defined in `worker/redirects.js`. `move.sh` appends to it automatically.

## Snowplow Assistant

The "Ask AI" button in the navbar opens a chat drawer that answers questions from the documentation. It is the Snowplow Console's assistant (the `console-agent` service) running in a documentation-only mode: the agent only has the two documentation tools, and the current-version `llms.txt` index is loaded into its prompt up front so it can pick pages without an extra round trip.

**Request path.** The widget posts to the same-origin `POST /api/assistant/chat`. [`worker/assistant.js`](worker/assistant.js) checks the method and body size, applies a per-IP rate limit (the `ASSISTANT_RATE_LIMITER` binding in `wrangler.jsonc`, 10 requests per minute), then forwards the body to `${DOCS_ASSISTANT_AGENT_URL}/api/agent/docs/chat` with the `X-Docs-Assistant-Secret` header and streams the response back unchanged. The browser never talks to the agent directly and never sees the secret. JSON error bodies carry a `status` field so the widget can show a rate-limit countdown or a size message.

**Configuration.** `DOCS_ASSISTANT_AGENT_URL` is a plain var in `wrangler.jsonc`. `DOCS_ASSISTANT_SHARED_SECRET` is a Worker secret set in the Cloudflare dashboard (or `npx wrangler secret put DOCS_ASSISTANT_SHARED_SECRET`); it must match the agent's `DOCS_ASSISTANT_SHARED_SECRET`. Without both the Worker answers 503. For local development see [Run the assistant locally](CONTRIBUTING.md#run-the-assistant-locally).

**Frontend.** `src/components/Assistant/` holds the widget: `AssistantProvider` and `AssistantHost` are mounted in `src/theme/Root.js` (above the per-route layout, so an open conversation survives navigating to a linked page), `AskAiNavbarItem` is registered as the `custom-askAi` navbar item, and `AssistantDrawer` is lazy-loaded on first open so docs pages do not download the chat bundle. The chat is built on the Vercel AI SDK (`useChat` + `DefaultChatTransport`) and AI Elements components vendored into `src/components/ai-elements/` and ported to Tailwind 3; markdown answers render with `streamdown`. Internal links open in the same tab through Docusaurus routing. A single conversation is kept in `sessionStorage`.

**Analytics.** The widget emits `assistant_interaction` events (`open`, `close`, `message_submit`, `suggestion_click`) using the same schema and event specifications as the Console. The prompt form carries the `sp-assistant-form` class, which `snowplow.js` excludes from form tracking so questions are never sent as form payloads.

## Custom plugins

Live under `plugins/`.
Expand Down Expand Up @@ -105,6 +119,7 @@ The repo has multiple tracking implementations. Each tracking script manages its
| `reoTracking.js` | [Reo.dev](https://reo.dev) tracker | Loaded unconditionally | N/A | N/A |
| `src/qualified.js` | [Qualified](https://www.qualified.com) chat/conversion tracking | Loaded unconditionally | N/A | N/A |
| `worker/index.js` | Page view tracking for `.md` and `llms.txt` requests | Anonymous tracking | N/A | N/A |
| `src/components/Assistant/tracking.ts` | `assistant_interaction` events from the AI assistant drawer | Follows `snowplow.js` consent state | N/A | N/A |

## Styling and CSS

Expand Down
13 changes: 13 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,19 @@ Release notes live in `release-notes/<slug>/index.md`. Read [`release-notes/_REA

Notes carry a slug that becomes the published URL, so keep it short and don't change it after the note ships. Link out to the docs with absolute paths such as `/docs/signals/`; the build fails on a broken link, which is how a moved docs page gets caught.

## Run the assistant locally

The "Ask AI" drawer calls `/api/assistant/chat`, which only exists when the Cloudflare Worker is in front of the site. To try it locally you need the `console-agent` service running (`npm run dev` in that repo, port 3001) and the Worker:

1. Create `.dev.vars` in this repo (it is gitignored):
```
DOCS_ASSISTANT_AGENT_URL=http://localhost:3001
DOCS_ASSISTANT_SHARED_SECRET=<same value as the agent's DOCS_ASSISTANT_SHARED_SECRET>
```
2. Either build the site and serve it through the Worker with `yarn build && npx wrangler dev` (everything on `http://localhost:8787`), or keep the hot-reloading dev server and proxy only the assistant calls to the Worker: run `npx wrangler dev` in one terminal and `ASSISTANT_PROXY_TARGET=http://localhost:8787 yarn start` in another.

Without the Worker the drawer still opens, but sending a message shows "The assistant is unavailable right now".

## Submit changes

Before opening a PR, run `yarn build` locally. This runs the full production build, and catches errors, broken internal links, and broken anchors before they get to CI.
Expand Down
4 changes: 3 additions & 1 deletion WORKFLOWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ CI and deployment. For writing guidelines see [`CONTRIBUTING.md`](CONTRIBUTING.m

## Build and deploy

The site is deployed by **Cloudflare Pages** on push to `main`. Cloudflare runs the `yarn build:cf` script, which:
The site is deployed as a **Cloudflare Worker with static assets** (see `wrangler.jsonc`) on push to `main`, through Cloudflare's git integration. Cloudflare runs the `yarn build:cf` script, which:

1. Sets `NODE_OPTIONS='--max-old-space-size=4096'` (the build needs more than the default heap).
2. Runs `docusaurus build`.
3. Deletes `build/_redirects` — this is the file Docusaurus auto-generates from any installed redirect plugin. Removing it means nothing leaks into the deployed bundle. All redirects live in the [Cloudflare Worker](ARCHITECTURE.md#cloudflare-worker) instead.

The Worker needs one secret, `DOCS_ASSISTANT_SHARED_SECRET`, for the [Snowplow Assistant](ARCHITECTURE.md#snowplow-assistant) proxy. Set it once in the Cloudflare dashboard (Workers & Pages → documentation → Settings → Variables and Secrets) or with `npx wrangler secret put DOCS_ASSISTANT_SHARED_SECRET`. The agent URL is a plain var in `wrangler.jsonc`. Preview deployments share the same secret and var, so they talk to the same agent as production.

Other `package.json` scripts:

- `yarn start`: dev server. Does not run the broken-link check.
Expand Down
4 changes: 4 additions & 0 deletions docusaurus.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,7 @@ const config: Config = {
],
'./plugins/docusaurus-plugin-release-notes',
'./plugins/docusaurus-plugin-snowplow-schema',
'./plugins/docusaurus-plugin-assistant-dev-proxy',
[
'./plugins/docusaurus-plugin-llms-txt',
{
Expand Down Expand Up @@ -304,6 +305,9 @@ const config: Config = {
],

customFields: {
// Where the Snowplow Assistant widget sends chat requests. Same-origin by
// default (served by the Cloudflare Worker); override for local testing.
assistantApiUrl: process.env.ASSISTANT_API_URL ?? '/api/assistant/chat',
webpack: {
configure: (config) => {
// Add JSX runtime resolution
Expand Down
14 changes: 11 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
"format": "prettier --write ."
},
"dependencies": {
"@ai-sdk/react": "^4",
"@braintree/sanitize-url": "^6.0.1",
"@docusaurus/core": "^3.10.0",
"@docusaurus/faster": "^3.10.0",
Expand All @@ -36,6 +37,7 @@
"@mui/x-data-grid-premium": "6.20.4",
"@radix-ui/react-accordion": "^1.2.1",
"@radix-ui/react-checkbox": "^1.3.3",
"@radix-ui/react-collapsible": "^1",
"@radix-ui/react-dialog": "^1.1.6",
"@radix-ui/react-dropdown-menu": "^2.1.6",
"@radix-ui/react-label": "^2.1.7",
Expand All @@ -52,6 +54,8 @@
"@snowplow/browser-plugin-link-click-tracking": "^4.7.0",
"@snowplow/browser-plugin-media": "^4.7.0",
"@snowplow/browser-tracker": "^4.7.0",
"@streamdown/code": "^1",
"ai": "^7",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"d3-color": "^3.1.0",
Expand All @@ -78,13 +82,15 @@
"remark-gfm": "^4.0.1",
"remark-math": "3",
"semver": "^7.3.8",
"streamdown": "^2",
"tailwind-merge": "^3.0.2",
"ua-parser-js": "^0.7.33",
"unist-util-flatmap": "^1.0.0",
"url": "^0.11.0",
"use-stick-to-bottom": "^1",
"uuid": "^10.0.0",
"webpack": "^5.76.0",
"zod": "^3.23.8"
"zod": "^3.25.76"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "^3.10.0",
Expand All @@ -109,10 +115,12 @@
"rehype-remark": "^10.0.1",
"remark-stringify": "^11.0.0",
"style-loader": "3.3.3",
"tailwindcss": "3.3.0",
"tailwindcss": "^3.4.19",
"tailwindcss-animate": "^1.0.7",
"typescript": "^5.5.4",
"unified": "^11.0.5",
"unist-util-visit": "^5.1.0"
"unist-util-visit": "^5.1.0",
"wrangler": "^4.36.0"
},
"browserslist": {
"production": [
Expand Down
32 changes: 32 additions & 0 deletions plugins/docusaurus-plugin-assistant-dev-proxy/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/**
* Dev-only proxy for the Snowplow Assistant widget.
*
* In production the Cloudflare Worker serves `/api/assistant/*` on the same
* origin as the site. `docusaurus start` has no Worker, so when
* `ASSISTANT_PROXY_TARGET` is set (for example `http://localhost:8787` from
* `wrangler dev`) the dev server forwards `/api/assistant` requests there.
* Without the variable the plugin does nothing.
*/
module.exports = function assistantDevProxyPlugin() {
return {
name: 'docusaurus-plugin-assistant-dev-proxy',
configureWebpack() {
const target = process.env.ASSISTANT_PROXY_TARGET
if (!target) {
return {}
}
return {
devServer: {
proxy: [
{
context: ['/api/assistant'],
target,
changeOrigin: true,
secure: false,
},
],
},
}
},
}
}
10 changes: 9 additions & 1 deletion snowplow.js
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,15 @@ const setupBrowserTracker = () => {
}) // precise tracking for the unified log

enableButtonClickTracking()
enableFormTracking()
// The assistant's prompt form is excluded so user questions are never sent
// as form-tracking payloads.
enableFormTracking({
options: {
forms: {
filter: (form) => !form.classList.contains('sp-assistant-form'),
},
},
})
}

if (ExecutionEnvironment.canUseDOM) {
Expand Down
28 changes: 28 additions & 0 deletions src/components/Assistant/AskAiNavbarItem.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
import React from 'react'
import { SparklesIcon } from 'lucide-react'
import {
prefetchAssistantDrawer,
useOptionalAssistant,
} from './AssistantContext'

export default function AskAiNavbarItem() {
const assistant = useOptionalAssistant()
if (!assistant) return null

return (
<button
type="button"
className="sp-assistant-navbar-button"
aria-label="Ask the Snowplow AI assistant"
aria-haspopup="dialog"
aria-expanded={assistant.isOpen}
data-sp-button-label="assistant_open"
onClick={assistant.open}
onMouseEnter={prefetchAssistantDrawer}
onFocus={prefetchAssistantDrawer}
>
<SparklesIcon className="sp-assistant-navbar-icon" aria-hidden="true" />
<span className="sp-assistant-navbar-label">Ask AI</span>
</button>
)
}
Loading
Loading