add a welcome flow for new organizations

AGENTS.md

@@ -32,9 +32,9 @@ Personal Agent Proxy holds the integrations and personal information an AI agent
 
 - **Model**: an integration from the catalog (`@proxy/integrations`: Email over IMAP and SMTP, Granola's meeting notes, Notion's pages and Linear's issues over their MCP servers) has the actions an agent can be allowed with a connection of it (Read, Archive, Write drafts, Send) and collections, each a list of fields with the actions reading and writing it need, and the commands it offers beyond editing (Mark as read, Archive, Send), each needing one action. A command runs on a `record`, where its `where` picks out which (Archive only in the inbox, Send only on a draft), or on values typed in (`new`: sending a new email). A mailbox is one collection, its folder a field the agent side filters by. A `nested` collection is a tree, as Notion's pages are: a record can hold others of the collection (`hasChildren`), listed inside it with the list tool's `parent` and made inside it with the create tool's, and access is the same anywhere in it. Integrations join the catalog once they work end to end; Google Workspace comes later. A connection is a connected integration and its account, owned by the organization; an integration can be connected more than once, such as two mailboxes. Information is the built-in `info` connection (addresses, payment cards, notes), made with the organization, so access and the agent side treat it like any other. Nothing is copied from a provider: every request goes to it live. Fields marked `system` are set by the provider, never typed in.
 - **Access** is one list of actions per connection, and every action the catalog lists can be set: what Personal Agent Proxy can do with the provider is the limit. Two layers: the connection's default (a `ConnectionDefault` row per action that is on) and an agent's own setting, on or off, where it differs (`AgentGrant`). An agent gets its own setting, else the default as it is now; an action that `requires` another counts only with it (Archive needs Read; Send needs nothing, so an agent can send without reading the mailbox). A collection's `writes` name the action creating, editing and deleting need, and `editable` which records they apply to (drafts). An agent login is for one agent from `AGENT_PROVIDERS`; each organization can have one login per company. It has a generated username and a password that is shown once. Every request on the agent side is an audit entry, allowed or denied, with what a list searched for.
-- **Accounts**: a human signs in with Google or a magic link (`src/auth/` in the api). A session is a `Session` row and an httpOnly cookie; the web app's `/api/me` loader on the human side's route redirects to `/login?callbackUrl=…` without one. Every user gets one `Organization` at sign-up, created with the user. `ALLOWED_SIGNUP_EMAILS` limits who can sign up; the login page asks `/auth/methods` whether to offer Google. In development the login page has a Create dev user button; in development, or without `RESEND_KEY`, the api prints magic links instead of emailing them. Cookies are secure when `APP_URL` is HTTPS.
-- **Human side** behind `/login`: `/connections`, `/connections/new` (connects a mailbox with an app password, or Granola, Notion and Linear by signing in to them), `/connections/:id` (the defaults under Permissions, with how many agents differ per action, and a "Connection added successfully" when `?added` says it was just connected), `/info`, `/agents` (chooses an agent company that does not have a login yet), `/agents/:id` (the password shows once, after creating or resetting; access laid out like an editor's settings: one checkbox per action of each connection, the only control for it, each connection folded by default to a row saying Default or Changed; a setting of the agent's own is marked with a bar, a button that makes it the connection's default for every agent, and one that resets it, and a filter and Changed only narrow the list; `ConnectionAccess` serves both pages), `/activity` (filters in the URL, so the agent and connection pages link to their part of the log). Each page has a loader in `src/loaders.ts` and revalidates after a change. Layout follows the spanish repo: the shadcn sidebar as a rail, an `h-12` header, one `max-w-3xl` column, one-line `RowList` rows. Type has one size, `text-sm`, in rows, summaries and hints alike, except column headers (`RowHeader`), which are `text-xs`: the thing itself in the normal colour, what is said about it (column headers, times, summaries, descriptions) in `text-muted-foreground`, and `font-medium` only for page and section titles.
-- **MCP**: an agent that speaks MCP (Poke, Claude, ChatGPT) adds `<APP_URL>/mcp` as a custom connector and signs in; the person picks which agent login it works as on `/connect-agent`. The agent page shows the server's address and the MCP clients working as the login, each with a Disconnect; the activity log marks requests that came in over MCP (`AuditEntry.via`).
+- **Accounts**: a human signs in with Google or a magic link (`src/auth/` in the api). A session is a `Session` row and an httpOnly cookie; the web app's `/api/me` loader on the human side's route redirects to `/login?callbackUrl=…` without one. Every user gets one `Organization` at sign-up, created with the user. `ALLOWED_SIGNUP_EMAILS` limits who can sign up; the login page asks `/auth/methods` whether to offer Google. In development the login page has a Create dev user button; in development, or without `RESEND_KEY`, the api prints magic links instead of emailing them. Cookies are secure when `APP_URL` is HTTPS. A new organization first goes through the welcome flow (`/welcome`, outside the human side's layout; the human side redirects there until `Organization.onboardedAt` is set, which `POST /api/me/onboarded` does when the flow is finished or skipped): step one picks the agent from `AGENT_PROVIDERS` and does what its `connects` says, an `mcp` agent (Claude, Claude Code, Claude Desktop, ChatGPT, Poke) shown the server address and that agent's steps (Claude Code's a terminal command) while the page polls for the login its sign-in makes, and over http, as in development, only a `local` agent (Claude Code, Claude Desktop) can be picked, the rest greyed out with a pointer to self-hosting on a server with a domain, a `login` agent given a login at once, as a message to paste to it with the sign-in page, username and password; step two (`/welcome/connections`) is the catalog, a plus per integration with how many accounts are connected: a mailbox goes to `/welcome/connections/email`, the form on a page with Continue once signed in, and a sign-in started with `?returnTo=welcome` comes back to `/welcome/connections/:id`, the new connection's permissions (its defaults, saved as they are ticked) with Confirm back to the catalog. Organizations from before the flow were marked done by its migration.
+- **Human side** behind `/login`: `/connections`, `/connections/new` (the catalog, a plus per integration with how many accounts are connected: `/connections/new/email` connects a mailbox with an app password, with a Test connection that signs in and saves nothing, and Granola, Notion and Linear are connected by signing in to them; `ConnectEmailForm` and `IntegrationCatalog` serve the welcome flow too), `/connections/:id` (the defaults under Permissions, with how many agents differ per action, and a "Connection added successfully" when `?added` says it was just connected), `/info`, `/agents` (chooses an agent company that does not have a login yet), `/agents/:id` (the password shows once, after creating or resetting; access laid out like an editor's settings: one checkbox per action of each connection, the only control for it, each connection folded by default to a row saying Default or Changed; a setting of the agent's own is marked with a bar, a button that makes it the connection's default for every agent, and one that resets it, and a filter and Changed only narrow the list; `ConnectionAccess` serves both pages), `/activity` (filters in the URL, so the agent and connection pages link to their part of the log). Each page has a loader in `src/loaders.ts` and revalidates after a change. Layout follows the spanish repo: the shadcn sidebar as a rail, an `h-12` header, one `max-w-3xl` column, one-line `RowList` rows. Type has one size, `text-sm`, in rows, summaries and hints alike, except column headers (`RowHeader`), which are `text-xs`: the thing itself in the normal colour, what is said about it (column headers, times, summaries, descriptions) in `text-muted-foreground`, and `font-medium` only for page and section titles.
+- **MCP**: an agent that speaks MCP (Poke, Claude, Claude Code, Claude Desktop, ChatGPT) adds `<APP_URL>/mcp` as a custom connector and signs in; the person picks which agent login it works as on `/connect-agent`. The agent page shows the server's address and the MCP clients working as the login, each with a Disconnect; the activity log marks requests that came in over MCP (`AuditEntry.via`).
 - **Agent side** behind `/agent/login`, with its own cookie: `/agent` lists what the login can do with each connection and the lists it can open, `/agent/:connection/:collection` a collection's records (newest first, 50 to a page with an Older link, narrowed by its `filterField`, such as the folder, in the address as the list tool takes it, `?folder=Draft`, and searched where the collection has a `searchHint`, which says what the search takes; in a nested collection `?parent=` opens it inside a record, with the records above it as breadcrumbs, a record holding others opening into them, and New making one there), `…/new` and `…/:record` the generic `RecordForm` and `RecordFields`, with a button per command the agent may run on the record, and Send beside Save as draft on a new email (loaders in `src/agent-loaders.ts`). Every page runs a tool (`toolName` in `@proxy/integrations`) through the api. It is built for a model driving a browser: no sidebar, no icon-only buttons, nothing that only shows on hover. Lists it can neither read nor add to are left out, and one it can only add to (sending without Read) opens on its form; asking for one anyway shows the api's refusal, which it logs as denied.
 - **Testing email locally**: any IMAP server with TLS on 993 and SMTP on 465 or 587 works through Other. A self-signed one needs `NODE_TLS_REJECT_UNAUTHORIZED=0` on the api, never in production. The server's hostname needs a dot and can't be an IP address: `localhost4.localdomain4` reaches 127.0.0.1 on most Linux machines.
 

packages/api/src/connections/email/route.test.ts

@@ -115,3 +115,32 @@ describe('handleConnectEmailRoute', () => {
 		expect(createConnection).not.toHaveBeenCalled();
 	});
 });
+
+describe('handleTestEmailRoute', () => {
+	test('signs in to the mailbox and saves nothing', async () => {
+		const {handleTestEmailRoute} = await import('./route');
+		const result = await handleTestEmailRoute(makeRequest({provider: 'gmail', email: ' Alex@Gmail.com ', password: 'abcd efgh ijkl mnop'}));
+
+		expect(result.isOk()).toBe(true);
+		expect(checkMailbox).toHaveBeenCalledWith({imapHost: 'imap.gmail.com', smtpHost: 'smtp.gmail.com', smtpPort: 465, username: 'alex@gmail.com', password: 'abcdefghijklmnop'});
+		expect(createConnection).not.toHaveBeenCalled();
+		expect(encryptSecret).not.toHaveBeenCalled();
+	});
+
+	test("reports the mailbox's refusal", async () => {
+		checkMailbox.mockResolvedValue(Err(ApiErr.credentialsRejected()));
+
+		const {handleTestEmailRoute} = await import('./route');
+		const result = await handleTestEmailRoute(makeRequest({provider: 'gmail', email: 'alex@gmail.com', password: 'wrong'}));
+
+		expect(result.unwrapErr().kind).toBe('credentials_rejected');
+	});
+
+	test('another provider needs its servers', async () => {
+		const {handleTestEmailRoute} = await import('./route');
+		const result = await handleTestEmailRoute(makeRequest({provider: 'other', email: 'alex@example.com', password: 'secret'}));
+
+		expect(result.isErr()).toBe(true);
+		expect(checkMailbox).not.toHaveBeenCalled();
+	});
+});

packages/api/src/connections/email/route.ts

@@ -1,7 +1,7 @@
 import {Err, Ok, type Result} from 'ts-results-es';
 import {z} from 'zod';
 
-import {ApiErr, type ApiError, Do, parseSchema} from '@proxy/utils';
+import {ApiErr, type ApiError, Do, parseSchema, requirePresent} from '@proxy/utils';
 import {createConnection, listConnections} from '@proxy/db/connection';
 import {EMAIL_PROVIDERS, type EmailProvider, type MailServers} from '@proxy/integrations';
 
@@ -46,30 +46,49 @@ function passwordFor(provider: EmailProvider, password: string): string {
 	return password.replace(/\s/g, '');
 }
 
+// The credential a request describes: the provider's servers or the typed-in ones, and the
+// password as the server takes it.
+function credentialFor(body: unknown): Result<{email: string; credential: EmailCredential}, ApiError> {
+	return Do<{email: string; credential: EmailCredential}, ApiError>(($) => {
+		const parsed = $(parseSchema(connectEmailBodySchema, body));
+		const provider = $(
+			requirePresent(
+				EMAIL_PROVIDERS.find((candidate) => candidate.id === parsed.provider),
+				ApiErr.validationError('Unknown provider'),
+			),
+		);
+		const servers = $(serversFor(provider, parsed));
+		return {email: parsed.email, credential: {...servers, username: parsed.email, password: passwordFor(provider, parsed.password)}};
+	});
+}
+
+/** Signs in to the mailbox over IMAP and SMTP and saves nothing: the Test connection button. */
+export function handleTestEmailRoute(request: AuthenticatedRequest): Promise<Result<void, ApiError>> {
+	return Do(async ($) => {
+		const {credential} = $(credentialFor(request.body));
+		$(await checkMailbox(credential));
+	});
+}
+
 /**
  * Connects a mailbox with an app password. Signs in over IMAP and SMTP first, so a wrong password
  * or server is reported now rather than on an agent's first request. A new connection gives agents
  * nothing until a default or an agent's own setting says otherwise.
  */
 export function handleConnectEmailRoute(request: AuthenticatedRequest): Promise<Result<ConnectionResponse, ApiError>> {
 	return Do(async ($) => {
-		const body = $(parseSchema(connectEmailBodySchema, request.body));
-		const provider = EMAIL_PROVIDERS.find((candidate) => candidate.id === body.provider);
-		if (!provider) {
-			return $(Err(ApiErr.validationError('Unknown provider')));
-		}
+		const {email, credential} = $(credentialFor(request.body));
 
 		const orgId = $(await requireUserOrgId(request));
 		const existing = $(await listConnections(orgId));
-		if (existing.some((connection) => connection.integrationId === 'email' && connection.account === body.email)) {
+		if (existing.some((connection) => connection.integrationId === 'email' && connection.account === email)) {
 			return $(Err(ApiErr.conflict('This mailbox is already connected')));
 		}
 
-		const credential: EmailCredential = {...$(serversFor(provider, body)), username: body.email, password: passwordFor(provider, body.password)};
 		const encrypted = $(encryptSecret(JSON.stringify(credential)));
 		$(await checkMailbox(credential));
 
-		const row = $(await createConnection({orgId, integrationId: 'email', account: body.email, credential: encrypted}));
+		const row = $(await createConnection({orgId, integrationId: 'email', account: email, credential: encrypted}));
 		return toConnectionResponse(row);
 	});
 }

packages/api/src/connections/granola/route.test.ts

@@ -63,10 +63,10 @@ function makeRes() {
 const user = {userId: 'user-1', email: 'alex@example.com'};
 
 // The start route for a real state cookie and Granola sign-in address.
-async function start() {
+async function start(query: Record<string, string> = {}) {
 	const {handleStartGranolaRoute} = await import('./route');
 	const res = makeRes();
-	await handleStartGranolaRoute({user, query: {}, headers: {}} as never, res as never);
+	await handleStartGranolaRoute({user, query, headers: {}} as never, res as never);
 	const cookie = res.cookies.find((candidate) => candidate.name === 'proxy.granola-state');
 	const authorizeUrl = new URL(res.redirect.mock.calls[0]?.[0] as string);
 	return {res, cookie, authorizeUrl, state: authorizeUrl.searchParams.get('state') ?? ''};
@@ -151,4 +151,22 @@ describe('handleGranolaCallbackRoute', () => {
 		expect(res.redirect).toHaveBeenCalledWith('https://app.example.com/connections/new?error=granola');
 		expect(createConnection).not.toHaveBeenCalled();
 	});
+
+	test('a sign-in started from the welcome flow ends on its page for the connection, or back on its catalog', async () => {
+		const done = await start({returnTo: 'welcome'});
+		const finished = await callback({cookie: done.cookie?.value ?? '', query: {state: done.state, code: 'code-1'}});
+		expect(createConnection).toHaveBeenCalledTimes(1);
+		expect(finished.redirect).toHaveBeenCalledWith('https://app.example.com/welcome/connections/conn-1');
+
+		const turnedDown = await start({returnTo: 'welcome'});
+		const failed = await callback({cookie: turnedDown.cookie?.value ?? '', query: {state: turnedDown.state, error: 'access_denied'}});
+		expect(failed.redirect).toHaveBeenCalledWith('https://app.example.com/welcome/connections?error=granola');
+	});
+
+	test('an unknown returnTo is ignored', async () => {
+		const {cookie, state} = await start({returnTo: 'https://evil.example.com'});
+		const res = await callback({cookie: cookie?.value ?? '', query: {state, code: 'code-1'}});
+
+		expect(res.redirect).toHaveBeenCalledWith('https://app.example.com/connections/conn-1?added');
+	});
 });