From 4ff7bd9eed85094e66af236bd41d4decfe09bd08 Mon Sep 17 00:00:00 2001 From: Daniel Date: Fri, 25 Sep 2026 21:57:29 +0200 Subject: [PATCH] feat(analytics): four read-only MCP tools top_entries, unviewed_entries, entry_views and site_totals expose what the sync stored to EmDash's MCP server. Each is a private route with plugins:read, the permission the analytics pages use, and reads storage only: a sandboxed invocation has ten bridge calls, and the worst case per tool is seven. Every answer states its UTC window, whether stored history covers it, whether the content index is complete and when the last sync ran. The zod schemas live in src/tools/declare.ts and are referenced only from the plugin's `mcp` property, which the plugin build strips from the runtime, so zod stays a dev dependency and the bundle grows by the loaders only (81 KB to 86 KB). The handlers validate their input by hand, because the routes are also reachable over HTTP. The list tools read past rows kept for history (unpublished, deleted, moved) until the page is full, at least 25 rows per read, and repeat the last read up to the final row taken so the cursor stays exact. Cursors name their tool, window and collection; one from another query is refused instead of seeking by the wrong column. Verified on a create-emdash site with EmDash 0.39.1: sandboxed through @emdash-cms/sandbox-workerd, tools/list shows all four and every call passes the MCP server's output validation. Registered in plugins: [], EmDash lists none of them: its in-process loader reads tools from the runtime module, where the build removed them. The README says so. isPublished and oldestDay move to the store modules, shared by the per-entry page, the panel and the tools. --- .changeset/analytics-mcp-tools.md | 9 + packages/analytics/AGENTS.md | 15 + packages/analytics/README.md | 45 ++- packages/analytics/package.json | 3 +- packages/analytics/src/plugin.ts | 25 +- packages/analytics/src/store/access.ts | 7 + packages/analytics/src/store/rows.ts | 5 + packages/analytics/src/tools/declare.ts | 212 +++++++++++ packages/analytics/src/tools/load.ts | 451 ++++++++++++++++++++++++ packages/analytics/src/ui/content.ts | 11 +- packages/analytics/src/ui/panel.ts | 6 +- packages/analytics/tests/budget.test.ts | 87 +++++ packages/analytics/tests/tools.test.ts | 371 +++++++++++++++++++ pnpm-lock.yaml | 59 ++-- 14 files changed, 1257 insertions(+), 49 deletions(-) create mode 100644 .changeset/analytics-mcp-tools.md create mode 100644 packages/analytics/src/tools/declare.ts create mode 100644 packages/analytics/src/tools/load.ts create mode 100644 packages/analytics/tests/tools.test.ts diff --git a/.changeset/analytics-mcp-tools.md b/.changeset/analytics-mcp-tools.md new file mode 100644 index 0000000..8d65bdd --- /dev/null +++ b/.changeset/analytics-mcp-tools.md @@ -0,0 +1,9 @@ +--- +"@eisbachcode/emdash-plugin-analytics": minor +--- + +Adds four read-only MCP tools, so an AI agent connected to EmDash's MCP server can ask for the numbers: `analytics__top_entries`, `analytics__unviewed_entries`, `analytics__entry_views` and `analytics__site_totals`. They read what the plugin has stored, never call Cloudflare, and every answer says which UTC days it covers, where stored history starts and when the last sync ran. + +The tools stay off until an administrator turns on **Agent access** for the plugin under Plugins. A caller needs `plugins:read` (editor and above) and a token with the `mcp:tools:analytics` or `mcp:tools` scope. + +On EmDash 0.39 and 0.40 only sandboxed and registry installs get the tools. A plugin registered in `plugins: []` lists none; that is an EmDash limitation, and nothing else about the plugin changes there. diff --git a/packages/analytics/AGENTS.md b/packages/analytics/AGENTS.md index f32dcb7..36b8104 100644 --- a/packages/analytics/AGENTS.md +++ b/packages/analytics/AGENTS.md @@ -45,6 +45,21 @@ at the workspace root is enough. undefined there, so read `routeCtx.ui?.locale` and fall back to English; never return empty blocks when it is missing. The editor panel gets it in both modes, with `ui.entry`. +- **MCP schemas never reach the runtime.** `emdash-plugin build` strips + the `mcp` property of `src/plugin.ts` from the bundle and writes the zod + schemas into the manifest as JSON Schema. `src/tools/declare.ts` is + referenced only from there, so zod stays a dev dependency and out of the + bundle; a schema imported by a route handler would pull it back in. The + handlers in `src/tools/load.ts` validate by hand, because the routes are + also reachable over HTTP without the MCP server's validation. +- **An MCP output schema is strict.** Every object becomes + `additionalProperties: false` and the MCP server rejects an answer that + does not match, so a loader's result and its declared output have to + agree key for key. `tests/tools.test.ts` checks each answer against the + schema the build wrote. +- **MCP tools reach sandboxed installs only.** EmDash's in-process loader + reads tools from the runtime module, where the build removed them, so a + `plugins: []` install lists none. Test tools sandboxed. - **Block Kit keeps no state.** Anything a page needs to remember between interactions travels in an `action_id`, a button `value` or a table cursor (see `src/ui/content.ts`). diff --git a/packages/analytics/README.md b/packages/analytics/README.md index 5104d56..bff6444 100644 --- a/packages/analytics/README.md +++ b/packages/analytics/README.md @@ -6,10 +6,10 @@ Most analytics plugins either inject a script or show site-wide numbers. This one keeps a `path → entry` index, so views can be attributed to the entry that earned them rather than to a URL string. -> **Status: first release.** Cloudflare Web Analytics plus demo data: a -> dashboard widget, an Analytics page with a setup check, a per-entry page -> and an Analytics panel in the entry editor. Tested on EmDash 0.39 and 0.40. -> See "Not in this version". +> **Status: early releases.** Cloudflare Web Analytics plus demo data: a +> dashboard widget, an Analytics page with a setup check, a per-entry page, +> an Analytics panel in the entry editor and four read-only MCP tools. +> Tested on EmDash 0.39 and 0.40. See "Not in this version". The screenshots show demo data on a site made from EmDash's blog template. @@ -108,6 +108,40 @@ page views and the path they are counted at, and on a translated entry each language's numbers and their total. An entry that is not published at a public URL says so instead. +## MCP tools + +EmDash's MCP server can hand the same numbers to an AI agent. The plugin +declares four read-only tools: + +| Tool | Answers | +|---|---| +| `analytics__top_entries` | The most viewed published entries over 7 or 30 days, optionally for one collection | +| `analytics__unviewed_entries` | Published entries without a single view in 7 or 30 days | +| `analytics__entry_views` | One entry's 7- and 30-day views, found by id or by path, with its translations and their total | +| `analytics__site_totals` | Site visits and page views over 7, 30 or 90 days, against the period before | + +They read what the sync stored and never call Cloudflare. Every answer +names its window in UTC days, where stored history starts, whether the +content index is complete and when the last sync ran, so an agent can say +that the data is thin instead of presenting it as exact. + +To use them: + +1. Under **Plugins**, open Analytics and turn on **Agent access** after + reviewing the tools. +2. Call them as a user with `plugins:read` (editor and above), through a + token with the `mcp:tools:analytics` scope, or `mcp:tools` for every + plugin's tools. + +After an update that changes a tool, EmDash stops serving the tools until +they are approved again, but the switch stays on. Turn it off and on. + +**Sandboxed installs only, for now.** A plugin registered in `plugins: []` +gets none of its MCP tools on EmDash 0.39 and 0.40: the plugin build keeps +the tool declarations out of the runtime module, and EmDash's in-process +loader looks for them only there. Under `sandboxed: []` or installed from +the registry, the tools work. This is an EmDash limitation, not a setting. + ## What you need | | | @@ -263,6 +297,9 @@ The editor panel declares `content:edit_own`, and EmDash also checks that the user may edit that entry: **authors see their own entries' numbers, editors and admins every entry's**, contributors none. +The MCP tools declare `plugins:read` like the pages, so an agent acting for +an author gets a permission error from all of them. + Authors and contributors will still see the widget card in the dashboard and get a permission error inside it — EmDash renders every declared widget for every role and gates on dispatch. That is an upstream gap, not diff --git a/packages/analytics/package.json b/packages/analytics/package.json index c25afd2..cd5b2b7 100644 --- a/packages/analytics/package.json +++ b/packages/analytics/package.json @@ -64,7 +64,8 @@ "emdash": ">=0.39.0 <1.0.0", "tsdown": "0.20.3", "typescript": "^5.9.0", - "vitest": "^4.1.0" + "vitest": "^4.1.0", + "zod": "4.5.4" }, "publishConfig": { "access": "public" diff --git a/packages/analytics/src/plugin.ts b/packages/analytics/src/plugin.ts index 723898d..c73aec8 100644 --- a/packages/analytics/src/plugin.ts +++ b/packages/analytics/src/plugin.ts @@ -1,5 +1,5 @@ /** - * Hooks, the cron dispatcher and the single admin route. + * Hooks, the cron dispatcher, the admin routes and the MCP tools. * * Hooks and routes are declared here rather than in `emdash-plugin.jsonc` * because the authored manifest is strict and rejects both keys outright; @@ -50,6 +50,8 @@ import { CONTENT_PATH, loadContent, parseContentInput, renderContent } from "./u import { loadPanel, renderPanel } from "./ui/panel.js"; import { loadWidget, renderWidget, REFRESH_ACTION } from "./ui/widget.js"; import { DEFAULT_SYNC_INTERVAL } from "./settings.js"; +import { mcpTools } from "./tools/declare.js"; +import { entryViews, siteTotals, topEntries, TOOL_ROUTES, unviewedEntries } from "./tools/load.js"; /** * The cron hook's own timeout. The default is 5 000 ms, which a tick that @@ -196,7 +198,28 @@ const plugin: SandboxedPlugin = { return { blocks: renderPanel(await loadPanel(ctx, entryId, state, new Date(), ui?.locale)) }; }, }, + + // The MCP tools' routes read what the analytics pages show, so they + // need the same permission. + [TOOL_ROUTES.topEntries]: { + permission: "plugins:read", + handler: async (routeCtx, ctx) => await topEntries(ctx, routeCtx.input, new Date()), + }, + [TOOL_ROUTES.unviewedEntries]: { + permission: "plugins:read", + handler: async (routeCtx, ctx) => await unviewedEntries(ctx, routeCtx.input, new Date()), + }, + [TOOL_ROUTES.entryViews]: { + permission: "plugins:read", + handler: async (routeCtx, ctx) => await entryViews(ctx, routeCtx.input, new Date()), + }, + [TOOL_ROUTES.siteTotals]: { + permission: "plugins:read", + handler: async (routeCtx, ctx) => await siteTotals(ctx, routeCtx.input, new Date()), + }, }, + + mcp: { tools: mcpTools() }, }; /** diff --git a/packages/analytics/src/store/access.ts b/packages/analytics/src/store/access.ts index b9cb583..250bae4 100644 --- a/packages/analytics/src/store/access.ts +++ b/packages/analytics/src/store/access.ts @@ -16,6 +16,7 @@ import type { PluginContext } from "emdash/plugin"; +import type { Day } from "../sync/window.js"; import type { DailyRow, EntryRow, RollupRow } from "./rows.js"; export interface Page { @@ -112,3 +113,9 @@ export function rollupStore(ctx: PluginContext): Typed | null { export function entriesStore(ctx: PluginContext): Typed | null { return wrap(ctx.storage?.entries); } + +/** The oldest day a dated store holds, or null when it is empty or missing. */ +export async function oldestDay(store: Typed<{ date: Day }> | null): Promise { + const page = await store?.query({ orderBy: { date: "asc" }, limit: 1 }); + return page?.items[0]?.data.date ?? null; +} diff --git a/packages/analytics/src/store/rows.ts b/packages/analytics/src/store/rows.ts index 9ac2d00..f813f52 100644 --- a/packages/analytics/src/store/rows.ts +++ b/packages/analytics/src/store/rows.ts @@ -71,6 +71,11 @@ export interface EntryRow { updatedAt: string; } +/** Rows kept for history after an unpublish, a delete or a move are not on the site. */ +export function isPublished(row: EntryRow): boolean { + return row.status === "published"; +} + export const DAILY_ID_SEPARATOR = "|"; export function dailyId(date: Day, path: string): string { diff --git a/packages/analytics/src/tools/declare.ts b/packages/analytics/src/tools/declare.ts new file mode 100644 index 0000000..4474487 --- /dev/null +++ b/packages/analytics/src/tools/declare.ts @@ -0,0 +1,212 @@ +/** + * The MCP tool declarations. + * + * Only `src/plugin.ts` references `mcpTools()`, from its `mcp` property, + * which the plugin build strips from the runtime: these schemas end up in + * the manifest as JSON Schema, and neither zod nor this module ships in + * the bundle. Keep it that way. A schema used by a route handler would + * pull zod into every sandboxed invocation. + * + * Output schemas become strict JSON Schema (`additionalProperties: false`) + * that the MCP server checks every answer against, so each one has to + * match its loader's result type in `./load.ts` exactly, optional keys + * included. `tests/tools.test.ts` holds each loader's output against the + * schema the build wrote. + */ + +import type { SandboxedMcpTool } from "emdash/plugin"; +import { z } from "zod"; + +import { + DEFAULT_LIMIT, + MAX_CURSOR, + MAX_ENTRY_ID, + MAX_LIMIT, + MAX_PATH, + TOOL_ROUTES, +} from "./load.js"; + +export function mcpTools(): Record { + const day = z.string().describe("A UTC day, YYYY-MM-DD."); + const window = z.object({ + days: z.number().int(), + since: day, + until: day.describe("Today in UTC. Its numbers still move."), + partial: z + .boolean() + .describe("True when stored history starts after since, so the numbers cover fewer days than the window."), + }); + const pageHistorySince = day + .nullable() + .describe("The oldest day of per-page numbers in storage, or null before the first sync. Older days are pruned by the retention setting."); + const indexComplete = z + .boolean() + .describe( + "False while the plugin matches the site's pages to entries, on a new install or during a rebuild; entries may be missing or misattributed until then.", + ); + const lastSync = z.string().nullable().describe("When the plugin last synced from the provider, ISO 8601."); + + const entryDays = z + .union([z.literal(7), z.literal(30)]) + .optional() + .describe("Window in days, ending today (UTC). Default 30."); + const listInput = z.object({ + days: entryDays, + collection: z + .string() + .max(63) + .regex(/^[a-z][a-z0-9_]*$/) + .optional() + .describe("Only entries of this collection, by slug, such as posts."), + limit: z + .number() + .int() + .min(1) + .max(MAX_LIMIT) + .optional() + .describe(`Entries per page, 1 to ${MAX_LIMIT}. Default ${DEFAULT_LIMIT}.`), + cursor: z + .string() + .min(1) + .max(MAX_CURSOR) + .optional() + .describe("nextCursor from the previous page, sent with the same days and collection."), + }); + const listOutput = z.object({ + window, + collection: z.string().nullable(), + items: z.array( + z.object({ + entryId: z.string(), + title: z.string(), + collection: z.string(), + collectionLabel: z.string(), + locale: z.string(), + path: z.string(), + views: z.number().describe("Page views in the window."), + publishedAt: z.string().optional(), + }), + ), + nextCursor: z.string().optional().describe("Pass as cursor to get the next page. Absent on the last page."), + historySince: pageHistorySince, + indexComplete, + lastSync, + }); + + const views = { views7: z.number(), views30: z.number() }; + + return { + top_entries: { + description: + "The site's most viewed published entries over the last 7 or 30 days, from the analytics provider's page views matched to content. " + + "Use it to answer which posts or pages get the most traffic. " + + "The counts are from the plugin's last sync (lastSync) and include today, which is still counting. " + + "When window.partial is true, stored history is shorter than the window; when indexComplete is false, some entries are not matched yet: say so rather than presenting the list as complete. " + + "A page can hold fewer items than limit; follow nextCursor for more.", + route: TOOL_ROUTES.topEntries, + input: listInput, + output: listOutput, + destructive: false, + }, + unviewed_entries: { + description: + "Published entries with no page views in the last 7 or 30 days. " + + "Use it to find content nobody reads. " + + "Check publishedAt: an entry published inside the window had less time to be seen. " + + "The list is only as complete as the data: window.partial true means stored history is shorter than the window, indexComplete false means some entries are not matched yet. " + + "A page can hold fewer items than limit; follow nextCursor for more.", + route: TOOL_ROUTES.unviewedEntries, + input: listInput, + output: listOutput, + destructive: false, + }, + entry_views: { + description: + "Page views of one entry over the last 7 and 30 days, with its published translations and their combined total. " + + "Pass entryId, or path as the site serves it (a full URL works too). " + + "found is false when no page of the entry is in the plugin's index: it may be a draft, in a collection without a URL pattern, or not matched yet (indexComplete). " + + "The counts are from the plugin's last sync (lastSync) and include today, which is still counting.", + route: TOOL_ROUTES.entryViews, + input: z.object({ + entryId: z.string().min(1).max(MAX_ENTRY_ID).optional().describe("The entry's id."), + path: z + .string() + .min(1) + .max(MAX_PATH) + .optional() + .describe("The path or URL the entry is published at. Used when entryId is not given."), + }), + output: z.object({ + found: z.boolean(), + entry: z + .object({ + entryId: z.string(), + title: z.string(), + collection: z.string(), + collectionLabel: z.string(), + locale: z.string(), + path: z.string(), + status: z + .enum(["published", "unpublished", "deleted", "moved"]) + .describe("Not published: the numbers are from the path it was last published at."), + ...views, + publishedAt: z.string().optional(), + }) + .nullable(), + translations: z.array( + z.object({ + entryId: z.string(), + title: z.string(), + locale: z.string(), + path: z.string(), + ...views, + }), + ), + allLanguages: z + .object(views) + .nullable() + .describe("The published pages of the entry and its translations together."), + windows: z.object({ views7: window, views30: window }), + historySince: pageHistorySince, + indexComplete, + lastSync, + }), + destructive: false, + }, + site_totals: { + description: + "Visits and page views for the whole site over the last 7, 30 or 90 days, with the period of the same length before it for comparison. " + + "Use it for overall traffic and its trend. " + + "previous is null when stored history does not reach back far enough to compare. " + + "estimated is true when the provider sampled a day, so the numbers are extrapolated; provisional is true when today, still counting, is included.", + route: TOOL_ROUTES.siteTotals, + input: z.object({ + days: z + .union([z.literal(7), z.literal(30), z.literal(90)]) + .optional() + .describe("Window in days, ending today (UTC). Default 30."), + }), + output: z.object({ + window, + visits: z.number(), + pageviews: z.number(), + estimated: z.boolean(), + provisional: z.boolean(), + previous: z + .object({ + since: day, + until: day, + visits: z.number(), + pageviews: z.number(), + estimated: z.boolean(), + }) + .nullable(), + historySince: day + .nullable() + .describe("The oldest day of site totals in storage, or null before the first sync. Site totals are kept longer than per-page numbers."), + lastSync, + }), + destructive: false, + }, + }; +} diff --git a/packages/analytics/src/tools/load.ts b/packages/analytics/src/tools/load.ts new file mode 100644 index 0000000..0521852 --- /dev/null +++ b/packages/analytics/src/tools/load.ts @@ -0,0 +1,451 @@ +/** + * The MCP tools' routes: read-only answers from storage. + * + * Each call is one sandboxed route invocation with ten bridge calls, so + * nothing here asks the provider live. The numbers are the ones the sync + * stored, and every answer says which UTC days they cover, where stored + * history starts and when the last sync ran, so an agent can tell thin + * data from a full window. + * + * The input schemas in `./declare.ts` are build metadata: the build + * strips them from the runtime, and EmDash's MCP server validates a call + * against them before the route runs. The same route is reachable over + * HTTP without that validation, so each handler reads its input + * defensively, falls back to the default, and names the window it used. + */ + +import type { PluginContext } from "emdash/plugin"; + +import { normalizePath } from "../index/paths.js"; +import { dailyStore, entriesStore, oldestDay, rollupStore, BIND_LIMIT, type Typed } from "../store/access.js"; +import { isPublished, sumWindow, type EntryRow, type RollupRow } from "../store/rows.js"; +import { indexOutdated, readState, type SyncState } from "../sync/scheduler.js"; +import { addDays, daysBetween, utcDay, type Day } from "../sync/window.js"; +import { loadPanel } from "../ui/panel.js"; + +export const TOOL_ROUTES = { + topEntries: "mcp/top_entries", + entryViews: "mcp/entry_views", + unviewedEntries: "mcp/unviewed_entries", + siteTotals: "mcp/site_totals", +} as const; + +export const ENTRY_DAYS = [7, 30] as const; +export const TOTALS_DAYS = [7, 30, 90] as const; +export const DEFAULT_DAYS = 30; +export const DEFAULT_LIMIT = 20; +export const MAX_LIMIT = 50; +export const MAX_CURSOR = 2048; +export const MAX_ENTRY_ID = 128; +export const MAX_PATH = 2048; + +const COLLECTION_SLUG = /^[a-z][a-z0-9_]*$/; +const MAX_COLLECTION = 63; + +/** + * Entries pages a list tool reads to fill its page past rows that are not + * published, plus one exact re-read (see `collect`). + */ +const LIST_PAGES = 4; +/** Rows read per page while filling, however few items are still missing. */ +const SCAN_ROWS = 25; + +/** Rollup pages per call: 180 days, the longest range plus its comparison. */ +const ROLLUP_PAGES = 2; + +export interface ToolWindow { + days: number; + /** First UTC day of the window. */ + since: Day; + /** Last UTC day of the window: today, still counting. */ + until: Day; + /** True when stored history starts after `since`, so the numbers cover fewer days. */ + partial: boolean; +} + +export interface ToolEntry { + entryId: string; + title: string; + collection: string; + collectionLabel: string; + locale: string; + path: string; + views: number; + publishedAt?: string; +} + +export interface EntryListResult { + window: ToolWindow; + collection: string | null; + items: ToolEntry[]; + nextCursor?: string; + historySince: Day | null; + indexComplete: boolean; + lastSync: string | null; +} + +export interface EntryViewsResult { + found: boolean; + entry: { + entryId: string; + title: string; + collection: string; + collectionLabel: string; + locale: string; + path: string; + status: EntryRow["status"]; + views7: number; + views30: number; + publishedAt?: string; + } | null; + translations: Array<{ + entryId: string; + title: string; + locale: string; + path: string; + views7: number; + views30: number; + }>; + allLanguages: { views7: number; views30: number } | null; + windows: { views7: ToolWindow; views30: ToolWindow }; + historySince: Day | null; + indexComplete: boolean; + lastSync: string | null; +} + +export interface SiteTotalsResult { + window: ToolWindow; + visits: number; + pageviews: number; + /** True when the provider sampled at least one day in the window. */ + estimated: boolean; + /** True when the window holds today, whose numbers still move. */ + provisional: boolean; + previous: { since: Day; until: Day; visits: number; pageviews: number; estimated: boolean } | null; + historySince: Day | null; + lastSync: string | null; +} + +type ListKind = "top" | "unviewed"; + +interface ListInput { + kind: ListKind; + days: (typeof ENTRY_DAYS)[number]; + collection: string | null; + limit: number; + /** The storage cursor inside the caller's cursor. */ + cursor?: string; +} + +export function parseListInput(kind: ListKind, input: unknown): ListInput { + const record = asRecord(input); + const view: ListInput = { + kind, + days: pickDays(record.days, ENTRY_DAYS), + collection: slugOf(record.collection), + limit: limitOf(record.limit), + }; + const cursor = boundedString(record.cursor, MAX_CURSOR); + return cursor ? { ...view, cursor: storageCursor(view, cursor) } : view; +} + +/** + * A storage cursor seeks by the value of the column it was sorted on. One + * from another tool, window or collection would page through a different + * order without any sign of it, so the cursor names its query and a + * mismatch is refused. + */ +function queryTag(view: ListInput): string { + return `${view.kind}.${view.days}.${view.collection ?? ""}`; +} + +function ownCursor(view: ListInput, storage: string): string { + return `${queryTag(view)}~${storage}`; +} + +export function storageCursor(view: ListInput, cursor: string): string { + const cut = cursor.indexOf("~"); + if (cut < 0 || cursor.slice(0, cut) !== queryTag(view) || cut === cursor.length - 1) { + throw new Error( + "This cursor belongs to another query. Pass nextCursor with the days and collection it came with, or leave cursor out to start from the top.", + ); + } + return cursor.slice(cut + 1); +} + +/** + * Published entries by views over the window, most viewed first. + * Calls: state, the oldest stored day, up to `LIST_PAGES` entries pages + * and one re-read. + */ +export async function topEntries(ctx: PluginContext, input: unknown, now: Date): Promise { + const view = parseListInput("top", input); + const { state, historySince, entries } = await listContext(ctx); + const base = listBase(view, state, historySince, utcDay(now)); + if (!entries) return { ...base, items: [] }; + + const { rows, nextCursor } = await collect(entries, view, "desc", (row) => (isPublished(row) ? "take" : "skip")); + return { ...base, items: rows.map((row) => toolEntry(row, view.days, state)), ...(nextCursor && { nextCursor }) }; +} + +/** + * Published entries with no views over the window. + * + * Rows are read least viewed first, so the first row with views ends the + * list: everything after it has views too. + * Calls: state, the oldest stored day, up to `LIST_PAGES` entries pages + * and one re-read. + */ +export async function unviewedEntries(ctx: PluginContext, input: unknown, now: Date): Promise { + const view = parseListInput("unviewed", input); + const { state, historySince, entries } = await listContext(ctx); + const base = listBase(view, state, historySince, utcDay(now)); + if (!entries) return { ...base, items: [] }; + + const key = sortKey(view.days); + const { rows, nextCursor } = await collect(entries, view, "asc", (row) => { + if (row[key] > 0) return "stop"; + return isPublished(row) ? "take" : "skip"; + }); + return { ...base, items: rows.map((row) => toolEntry(row, view.days, state)), ...(nextCursor && { nextCursor }) }; +} + +/** + * Up to `view.limit` rows in view order, read past the ones `verdict` + * skips, and the caller's cursor for the rest. + * + * Rows kept for history after an unpublish, a delete or a move sort in + * among the published ones, so a single page can come back with few items + * or none. Each read therefore takes at least `SCAN_ROWS` rows. A storage + * cursor can only point past the end of a page, so when the page fills + * before a read is used up, the same read is repeated up to the last row + * taken: the next call then starts right after it. + */ +async function collect( + entries: Typed, + view: ListInput, + dir: "asc" | "desc", + verdict: (row: EntryRow) => "take" | "skip" | "stop", +): Promise<{ rows: EntryRow[]; nextCursor?: string }> { + const rows: EntryRow[] = []; + const next = (page: { hasMore: boolean; cursor?: string }) => + page.hasMore && page.cursor ? { rows, nextCursor: ownCursor(view, page.cursor) } : { rows }; + let cursor = view.cursor; + for (let i = 0; i < LIST_PAGES; i++) { + const spec = { + ...(view.collection ? { where: { collection: view.collection } } : {}), + orderBy: { [sortKey(view.days)]: dir }, + ...(cursor ? { cursor } : {}), + }; + const page = await entries.query({ ...spec, limit: Math.min(BIND_LIMIT, Math.max(view.limit - rows.length, SCAN_ROWS)) }); + for (const [index, { data }] of page.items.entries()) { + const decision = verdict(data); + if (decision === "stop") return { rows }; + if (decision === "skip") continue; + rows.push(data); + if (rows.length < view.limit) continue; + + const following = page.items[index + 1]; + if (!following) return next(page); + if (verdict(following.data) === "stop") return { rows }; + return next(await entries.query({ ...spec, limit: index + 1 })); + } + if (!page.hasMore || !page.cursor) return { rows }; + cursor = page.cursor; + } + return { rows, nextCursor: ownCursor(view, cursor!) }; +} + +/** + * One entry's views and its published translations', found by entry id + * or by the path the site serves it under. + * Calls: state, the oldest stored day, the path lookup, then the + * panel's two queries. + */ +export async function entryViews(ctx: PluginContext, input: unknown, now: Date): Promise { + const record = asRecord(input); + const [state, historySince] = await Promise.all([readState(ctx), oldestDay(dailyStore(ctx))]); + const today = utcDay(now); + const base = { + windows: { views7: windowOf(7, today, historySince), views30: windowOf(30, today, historySince) }, + historySince, + indexComplete: indexComplete(state), + lastSync: state.lastSync ?? null, + }; + const missing = { ...base, found: false, entry: null, translations: [], allLanguages: null }; + + let entryId = boundedString(record.entryId, MAX_ENTRY_ID); + const path = boundedString(record.path, MAX_PATH); + if (!entryId && path) { + const row = await entriesStore(ctx)?.get(normalizePath(path, ctx.site.trailingSlash)); + entryId = row?.entryId; + } + if (!entryId) return missing; + + const { current, members } = await loadPanel(ctx, entryId, state, now); + if (!current) return missing; + + const others = members + .filter((row) => row.entryId !== current.entryId) + .sort((a, b) => a.locale.localeCompare(b.locale)); + // An entry that is no longer published keeps its last path's numbers, + // but they are not part of what the site serves now. + const live = [current, ...others].filter(isPublished); + const translations = others.map((row) => ({ + entryId: row.entryId, + title: row.title, + locale: row.locale, + path: row.path, + views7: row.views7, + views30: row.views30, + })); + return { + ...base, + found: true, + entry: { + entryId: current.entryId, + title: current.title, + collection: current.collection, + collectionLabel: labelOf(current.collection, state), + locale: current.locale, + path: current.path, + status: current.status, + views7: current.views7, + views30: current.views30, + ...(current.publishedAt && { publishedAt: current.publishedAt }), + }, + translations, + allLanguages: { + views7: live.reduce((sum, row) => sum + row.views7, 0), + views30: live.reduce((sum, row) => sum + row.views30, 0), + }, + }; +} + +/** + * Site-wide visits and page views over the window and the equal period + * before it, from the stored daily rollups. + * Calls: state, the oldest stored day, up to two rollup pages. + */ +export async function siteTotals(ctx: PluginContext, input: unknown, now: Date): Promise { + const days = pickDays(asRecord(input).days, TOTALS_DAYS); + const rollup = rollupStore(ctx); + const [state, historySince] = await Promise.all([readState(ctx), oldestDay(rollup)]); + const today = utcDay(now); + const window = windowOf(days, today, historySince); + + const stored: RollupRow[] = []; + let cursor: string | undefined; + for (let i = 0; rollup && i < ROLLUP_PAGES; i++) { + const page = await rollup.query({ + where: { date: { gte: addDays(today, -(days * 2 - 1)) } }, + orderBy: { date: "asc" }, + limit: BIND_LIMIT, + ...(cursor ? { cursor } : {}), + }); + stored.push(...page.items.map((item) => item.data)); + if (!page.hasMore || !page.cursor) break; + cursor = page.cursor; + } + + const previousSince = addDays(today, -(days * 2 - 1)); + const previousUntil = addDays(today, -days); + const current = sumWindow(stored.filter((row) => within(row.date, window.since, today)), today); + // A comparison the store covers only in part would set a few days + // against a whole period. + const before = + historySince !== null && daysBetween(historySince, previousSince) >= 0 + ? sumWindow(stored.filter((row) => within(row.date, previousSince, previousUntil)), today) + : null; + + return { + window, + visits: current.visits, + pageviews: current.pageviews, + estimated: current.estimated, + provisional: current.provisional, + previous: before && { + since: previousSince, + until: previousUntil, + visits: before.visits, + pageviews: before.pageviews, + estimated: before.estimated, + }, + historySince, + lastSync: state.lastSync ?? null, + }; +} + +async function listContext(ctx: PluginContext) { + const [state, historySince] = await Promise.all([readState(ctx), oldestDay(dailyStore(ctx))]); + return { state, historySince, entries: entriesStore(ctx) }; +} + +function listBase(view: ListInput, state: SyncState, historySince: Day | null, today: Day) { + return { + window: windowOf(view.days, today, historySince), + collection: view.collection, + historySince, + indexComplete: indexComplete(state), + lastSync: state.lastSync ?? null, + }; +} + +/** A rebuild re-matches pages to entries, so the index is not complete while one runs. */ +function indexComplete(state: SyncState): boolean { + return Boolean(state.indexComplete) && !indexOutdated(state); +} + +function windowOf(days: number, today: Day, historySince: Day | null): ToolWindow { + const since = addDays(today, -(days - 1)); + return { days, since, until: today, partial: historySince === null || daysBetween(since, historySince) > 0 }; +} + +function toolEntry(row: EntryRow, days: ListInput["days"], state: SyncState): ToolEntry { + return { + entryId: row.entryId, + title: row.title, + collection: row.collection, + collectionLabel: labelOf(row.collection, state), + locale: row.locale, + path: row.path, + views: row[sortKey(days)], + ...(row.publishedAt && { publishedAt: row.publishedAt }), + }; +} + +function labelOf(collection: string, state: SyncState): string { + return state.collectionLabels?.[collection] ?? collection; +} + +function sortKey(days: ListInput["days"]): "views7" | "views30" { + return days === 7 ? "views7" : "views30"; +} + +function within(day: Day, since: Day, until: Day): boolean { + return daysBetween(since, day) >= 0 && daysBetween(day, until) >= 0; +} + +function asRecord(input: unknown): Record { + return typeof input === "object" && input !== null ? (input as Record) : {}; +} + +/** Numbers arrive as strings from a query string, so both are accepted. */ +function pickDays(value: unknown, allowed: readonly T[]): T { + const n = typeof value === "string" ? Number(value) : value; + return (allowed as readonly unknown[]).includes(n) ? (n as T) : (DEFAULT_DAYS as T); +} + +function limitOf(value: unknown): number { + const n = typeof value === "string" ? Number(value) : value; + if (typeof n !== "number" || !Number.isInteger(n) || n < 1) return DEFAULT_LIMIT; + return Math.min(n, MAX_LIMIT); +} + +function slugOf(value: unknown): string | null { + return typeof value === "string" && value.length <= MAX_COLLECTION && COLLECTION_SLUG.test(value) ? value : null; +} + +function boundedString(value: unknown, max: number): string | undefined { + return typeof value === "string" && value.length > 0 && value.length <= max ? value : undefined; +} diff --git a/packages/analytics/src/ui/content.ts b/packages/analytics/src/ui/content.ts index 7996c63..9868193 100644 --- a/packages/analytics/src/ui/content.ts +++ b/packages/analytics/src/ui/content.ts @@ -26,8 +26,8 @@ import type { PluginContext } from "emdash/plugin"; import { langOf, t, type Lang } from "../i18n.js"; -import { dailyStore, entriesStore, BIND_LIMIT, type Typed } from "../store/access.js"; -import type { EntryRow } from "../store/rows.js"; +import { dailyStore, entriesStore, oldestDay, BIND_LIMIT, type Typed } from "../store/access.js"; +import { isPublished, type EntryRow } from "../store/rows.js"; import { indexOutdated, type SyncState } from "../sync/scheduler.js"; import { addDays, daysBetween, utcDay, type Day } from "../sync/window.js"; import { actions, button, context, empty, link, table, type AnalyticsBlock } from "./blocks.js"; @@ -151,11 +151,11 @@ export async function loadContent( requested.collection && routable.some((c) => c.slug === requested.collection) ? requested.collection : null; const view = { ...requested, collection }; - const history = await dailyStore(ctx)?.query({ orderBy: { date: "asc" }, limit: 1 }); + const historySince = await oldestDay(dailyStore(ctx)); const base = { state, collections: routable, - historySince: history?.items[0]?.data.date, + ...(historySince && { historySince }), indexing: indexingOf(state, routable.map((c) => c.slug)), now, locale, @@ -382,9 +382,8 @@ function rowOf(row: EntryRow): ContentRow { }; } -/** Rows kept for history after an unpublish or delete are not on the site. */ function published(rows: EntryRow[]): EntryRow[] { - return rows.filter((row) => row.status === "published"); + return rows.filter(isPublished); } /** Every published translation of the given groups, by group. */ diff --git a/packages/analytics/src/ui/panel.ts b/packages/analytics/src/ui/panel.ts index 605ff7e..30b6018 100644 --- a/packages/analytics/src/ui/panel.ts +++ b/packages/analytics/src/ui/panel.ts @@ -15,7 +15,7 @@ import type { PluginContext } from "emdash/plugin"; import { langOf, t } from "../i18n.js"; import { entriesStore, BIND_LIMIT } from "../store/access.js"; -import type { EntryRow } from "../store/rows.js"; +import { isPublished, type EntryRow } from "../store/rows.js"; import type { SyncState } from "../sync/scheduler.js"; import { actions, context, empty, link, stats, table, type AnalyticsBlock } from "./blocks.js"; import { CONTENT_PATH } from "./content.js"; @@ -46,11 +46,11 @@ export async function loadPanel( if (!entries || !entryId) return { ...base, current: null, members: [] }; const mine = (await entries.query({ where: { entryId }, limit: BIND_LIMIT })).items.map((item) => item.data); - const current = mine.find((row) => row.status === "published") ?? latest(mine); + const current = mine.find(isPublished) ?? latest(mine); if (!current) return { ...base, current: null, members: [] }; const group = await entries.query({ where: { translationGroup: current.translationGroup }, limit: BIND_LIMIT }); - const members = group.items.map((item) => item.data).filter((row) => row.status === "published"); + const members = group.items.map((item) => item.data).filter(isPublished); return { ...base, current, members: members.length > 0 ? members : [current] }; } diff --git a/packages/analytics/tests/budget.test.ts b/packages/analytics/tests/budget.test.ts index 8855870..37e56c0 100644 --- a/packages/analytics/tests/budget.test.ts +++ b/packages/analytics/tests/budget.test.ts @@ -5,6 +5,7 @@ import { GRAPHQL_ENDPOINT, RUM_DATASET } from "../src/providers/cloudflare.js"; import { WAITING_KEY, type SyncState } from "../src/sync/scheduler.js"; import { addDays } from "../src/sync/window.js"; import { PAGE_REFRESH_ACTION, RANGE_ACTION, SETUP_ACTION } from "../src/ui/page.js"; +import { TOOL_ROUTES } from "../src/tools/load.js"; import { bridgeCalls, failNextCall } from "./bridge-calls.js"; import { daysBack, @@ -253,6 +254,92 @@ describe("admin requests", () => { }); }); +describe("MCP tools", () => { + /** + * Rows kept for history ahead of the published ones in both sort + * orders, so a list tool reads every page it may and fills its page + * part-way through the last one, which costs the exact re-read. + */ + async function withHistoryRows(runtime: PluginRuntimeTestHost, views: { kept: number; published: number }) { + const entry = (path: string, status: "moved" | "published", n: number) => + runtime.fixtures.plugin.storage("entries", path, { + path, + collection: "posts", + entryId: `id${path}`, + translationGroup: `id${path}`, + locale: "en", + title: path, + status, + views7: n, + views30: n, + updatedAt: NOW.toISOString(), + }); + for (const path of pathsOf(80, "/a-")) await entry(path, "moved", views.kept); + for (const path of pathsOf(20, "/p-")) await entry(path, "published", views.published); + await seedDaily(runtime, ["/p-00/"], daysBack(3)); + await setState(runtime, synced); + } + + it("top_entries, reading past a hundred rows kept for history", async () => { + host = await newHost(); + await withHistoryRows(host, { kept: 100, published: 50 }); + let result: { items: unknown[] } | undefined; + const calls = await bridgeCalls(async () => { + result = (await host!.transport.invokeRoute(TOOL_ROUTES.topEntries, { limit: 10 })) as { items: unknown[] }; + }); + expect(calls.length, calls.join(", ")).toBeLessThanOrEqual(LIMIT); + expect(calls.filter((c) => c === "storageQuery").length).toBe(6); + expect(result!.items).toHaveLength(10); + }); + + it("unviewed_entries, reading past a hundred rows kept for history", async () => { + host = await newHost(); + await withHistoryRows(host, { kept: 0, published: 0 }); + let result: { items: unknown[] } | undefined; + const calls = await bridgeCalls(async () => { + result = (await host!.transport.invokeRoute(TOOL_ROUTES.unviewedEntries, { limit: 10 })) as { items: unknown[] }; + }); + expect(calls.length, calls.join(", ")).toBeLessThanOrEqual(LIMIT); + expect(calls.filter((c) => c === "storageQuery").length).toBe(6); + expect(result!.items).toHaveLength(10); + }); + + it("entry_views by path, for an entry with translations", async () => { + host = await newHost(); + for (const locale of ["de", "en", "fr"]) { + const path = `/${locale}/a/`; + await host.fixtures.plugin.storage("entries", path, { + path, + collection: "posts", + entryId: `a-${locale}`, + translationGroup: "a-de", + locale, + title: path, + status: "published", + views7: 1, + views30: 2, + updatedAt: NOW.toISOString(), + }); + } + await setState(host, synced); + let result: unknown; + const calls = await bridgeCalls(async () => { + result = await host!.transport.invokeRoute(TOOL_ROUTES.entryViews, { path: "/en/a" }); + }); + expect(calls.length, calls.join(", ")).toBeLessThanOrEqual(LIMIT); + expect(result).toMatchObject({ found: true, allLanguages: { views30: 6 } }); + }); + + it("site_totals over 90 days with a full comparison period", async () => { + host = await newHost(); + await seedRollup(host, 180); + await setState(host, synced); + const calls = await bridgeCalls(() => host!.transport.invokeRoute(TOOL_ROUTES.siteTotals, { days: 90 })); + expect(calls.length, calls.join(", ")).toBeLessThanOrEqual(LIMIT); + expect(calls.filter((c) => c === "storageQuery").length).toBe(3); + }); +}); + describe("content hooks", () => { it("a save, which also schedules", async () => { host = await newHost(); diff --git a/packages/analytics/tests/tools.test.ts b/packages/analytics/tests/tools.test.ts new file mode 100644 index 0000000..7fe3618 --- /dev/null +++ b/packages/analytics/tests/tools.test.ts @@ -0,0 +1,371 @@ +import type { PluginRuntimeTestHost } from "@emdash-cms/plugin-test"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { z } from "zod"; + +import type { EntryRow } from "../src/store/rows.js"; +import { parseListInput, TOOL_ROUTES } from "../src/tools/load.js"; +import { addDays } from "../src/sync/window.js"; +import { daysBack, newHost, NOW, seedDaily, seedRollup, setState, synced, TODAY } from "./host.js"; + +let host: PluginRuntimeTestHost | undefined; + +afterEach(async () => { + await host?.dispose(); + host = undefined; + vi.unstubAllEnvs(); +}); + +interface ManifestTool { + name: string; + route: string; + permission: string; + destructive: boolean; + inputSchema: Record; + outputSchema?: Record; +} + +function toolsOf(runtime: PluginRuntimeTestHost): ManifestTool[] { + return (runtime.manifest as unknown as { mcp?: { tools: ManifestTool[] } }).mcp?.tools ?? []; +} + +/** + * Call a tool's route and hold the answer against the output schema the + * build wrote, converted the way EmDash converts it for a sandboxed + * install. The MCP server rejects an answer that does not match, so a + * loader that drifts from its declaration breaks the tool, not just a type. + */ +async function call(runtime: PluginRuntimeTestHost, route: string, input: Record = {}) { + const tool = toolsOf(runtime).find((t) => t.route === route); + if (!tool?.outputSchema) throw new Error(`no tool with an output schema calls ${route}`); + const result = await runtime.transport.invokeRoute(route, input); + const parsed = z.fromJSONSchema({ ...tool.outputSchema }).safeParse(result); + expect(parsed.error?.issues ?? []).toEqual([]); + return result as Record; +} + +const row = (path: string, over: Partial = {}): EntryRow => ({ + path, + collection: "posts", + entryId: `id${path}`, + translationGroup: `id${path}`, + locale: "en", + title: `Title ${path}`, + status: "published", + views7: 0, + views30: 0, + updatedAt: NOW.toISOString(), + ...over, +}); + +async function put(runtime: PluginRuntimeTestHost, ...rows: EntryRow[]) { + for (const data of rows) await runtime.fixtures.plugin.storage("entries", data.path, data); +} + +describe("the manifest's MCP tools", () => { + it("each call a private route with the permission the tool declares", async () => { + // EmDash skips a tool whose route is missing, public or has another + // permission, without an error anywhere. + host = await newHost(); + const manifest = host.manifest as unknown as { + routes?: Array; + }; + const routes = new Map( + (manifest.routes ?? []).map((r) => (typeof r === "string" ? [r, { name: r }] : [r.name, r])), + ); + const tools = toolsOf(host); + + expect(tools.length).toBeGreaterThan(0); + for (const tool of tools) { + const route = routes.get(tool.route) as { permission?: string; public?: boolean } | undefined; + expect(route, tool.name).toBeDefined(); + expect(route!.public, tool.name).not.toBe(true); + expect(route!.permission, tool.name).toBe(tool.permission); + } + }); +}); + +describe("top_entries", () => { + async function seeded() { + const runtime = await newHost(); + await put( + runtime, + row("/a/", { views7: 5, views30: 50 }), + row("/b/", { views7: 20, views30: 30 }), + row("/c/", { views7: 90, views30: 99, status: "unpublished" }), + row("/d/", { collection: "pages", views7: 0, views30: 10, publishedAt: "2026-09-01T08:00:00.000Z" }), + ); + await setState(runtime, { ...synced, collectionLabels: { posts: "Posts", pages: "Pages" } }); + return runtime; + } + + it("lists published entries by views in the window, most viewed first", async () => { + host = await seeded(); + + const thirty = await call(host, TOOL_ROUTES.topEntries, { days: 30 }); + expect(thirty.items.map((i: { path: string; views: number }) => [i.path, i.views])).toEqual([ + ["/a/", 50], + ["/b/", 30], + ["/d/", 10], + ]); + expect(thirty.window).toMatchObject({ days: 30, since: addDays(TODAY, -29), until: TODAY }); + expect(thirty.items[2]).toMatchObject({ collectionLabel: "Pages", publishedAt: "2026-09-01T08:00:00.000Z" }); + + const seven = await call(host, TOOL_ROUTES.topEntries, { days: 7 }); + expect(seven.items.map((i: { path: string }) => i.path)).toEqual(["/b/", "/a/", "/d/"]); + }); + + it("filters by collection and pages with the cursor", async () => { + host = await seeded(); + + const pages = await call(host, TOOL_ROUTES.topEntries, { collection: "pages" }); + expect(pages.items.map((i: { path: string }) => i.path)).toEqual(["/d/"]); + expect(pages.collection).toBe("pages"); + + const first = await call(host, TOOL_ROUTES.topEntries, { limit: 2 }); + expect(first.items.map((i: { path: string }) => i.path)).toEqual(["/a/", "/b/"]); + const second = await call(host, TOOL_ROUTES.topEntries, { limit: 2, cursor: first.nextCursor }); + expect(second.items.map((i: { path: string }) => i.path)).toEqual(["/d/"]); + expect(second.nextCursor).toBeUndefined(); + }); + + it("fills a page past entries that are no longer published", async () => { + // The most viewed row is an unpublished entry's; "top 1" must still + // name one. + host = await seeded(); + + const result = await call(host, TOOL_ROUTES.topEntries, { limit: 1 }); + + expect(result.items.map((i: { path: string }) => i.path)).toEqual(["/a/"]); + expect(result.nextCursor).toEqual(expect.any(String)); + }); + + it("pages through every published entry once, in order, past rows kept for history", async () => { + host = await newHost(); + const published: string[] = []; + for (let i = 0; i < 40; i++) { + const path = `/e-${String(i).padStart(2, "0")}/`; + const kept = i % 3 === 1; + await put(host, row(path, { views30: 100 - i, ...(kept && { status: "moved" as const }) })); + if (!kept) published.push(path); + } + await setState(host, synced); + + const seen: string[] = []; + let cursor: string | undefined; + for (let calls = 0; calls < 20; calls++) { + const page = await call(host, TOOL_ROUTES.topEntries, { limit: 4, ...(cursor && { cursor }) }); + seen.push(...page.items.map((i: { path: string }) => i.path)); + cursor = page.nextCursor; + if (!cursor) break; + } + + expect(seen).toEqual(published); + }); + + it("refuses a cursor from another window, collection or tool", async () => { + // A storage cursor seeks by the sorted column's value; used with + // another order it would skip or repeat entries silently. Checked at + // the parser: a route that throws trips the test host's hang detector. + host = await seeded(); + const { nextCursor: cursor } = await call(host, TOOL_ROUTES.topEntries, { days: 30, limit: 1 }); + + expect(parseListInput("top", { days: 30, cursor }).cursor).toEqual(expect.any(String)); + expect(() => parseListInput("top", { days: 7, cursor })).toThrow(/another query/); + expect(() => parseListInput("top", { days: 30, collection: "posts", cursor })).toThrow(/another query/); + expect(() => parseListInput("unviewed", { days: 30, cursor })).toThrow(/another query/); + expect(() => parseListInput("top", { days: 30, cursor: "garbage" })).toThrow(/another query/); + }); + + it("says when stored history is shorter than the window", async () => { + host = await seeded(); + await seedDaily(host, ["/a/"], daysBack(10)); + + const seven = await call(host, TOOL_ROUTES.topEntries, { days: 7 }); + const thirty = await call(host, TOOL_ROUTES.topEntries, { days: 30 }); + + expect(seven.historySince).toBe(addDays(TODAY, -9)); + expect(seven.window.partial).toBe(false); + expect(thirty.window.partial).toBe(true); + }); + + it("falls back to the defaults for input nothing validated", async () => { + // Over HTTP the route gets whatever the caller sent. + host = await seeded(); + + const result = await call(host, TOOL_ROUTES.topEntries, { days: 90, limit: 500, collection: "No Slug" }); + + expect(result.window.days).toBe(30); + expect(result.collection).toBeNull(); + expect(result.items).toHaveLength(3); + }); +}); + +describe("unviewed_entries", () => { + it("lists published entries without views and stops at the first with views", async () => { + host = await newHost(); + await put( + host, + row("/quiet-1/"), + row("/quiet-2/"), + row("/gone/", { status: "deleted" }), + row("/read/", { views7: 0, views30: 4 }), + ); + await setState(host, synced); + + const thirty = await call(host, TOOL_ROUTES.unviewedEntries, { days: 30 }); + expect(thirty.items.map((i: { path: string }) => i.path).sort()).toEqual(["/quiet-1/", "/quiet-2/"]); + expect(thirty.nextCursor).toBeUndefined(); + + const seven = await call(host, TOOL_ROUTES.unviewedEntries, { days: 7 }); + expect(seven.items.map((i: { path: string }) => i.path).sort()).toEqual(["/quiet-1/", "/quiet-2/", "/read/"]); + }); + + it("finds one entry behind many rows kept for history", async () => { + host = await newHost(); + for (let i = 0; i < 30; i++) await put(host, row(`/a-${String(i).padStart(2, "0")}/`, { status: "moved" })); + await put(host, row("/quiet/")); + await setState(host, synced); + + const result = await call(host, TOOL_ROUTES.unviewedEntries, { limit: 1 }); + + expect(result.items.map((i: { path: string }) => i.path)).toEqual(["/quiet/"]); + }); + + it("pages while every row read has no views", async () => { + host = await newHost(); + await put(host, row("/quiet-1/"), row("/quiet-2/"), row("/read/", { views30: 4 })); + await setState(host, synced); + + const first = await call(host, TOOL_ROUTES.unviewedEntries, { limit: 1 }); + expect(first.items).toHaveLength(1); + expect(first.nextCursor).toEqual(expect.any(String)); + + const second = await call(host, TOOL_ROUTES.unviewedEntries, { limit: 1, cursor: first.nextCursor }); + expect(second.items).toHaveLength(1); + expect(second.items[0].path).not.toBe(first.items[0].path); + // The row after it has views, so there is no page to offer. + expect(second.nextCursor).toBeUndefined(); + }); +}); + +describe("entry_views", () => { + async function translated() { + const runtime = await newHost(); + await put( + runtime, + row("/de/a/", { entryId: "a-de", translationGroup: "a-de", locale: "de", views7: 3, views30: 30 }), + row("/en/a/", { entryId: "a-en", translationGroup: "a-de", locale: "en", views7: 1, views30: 12 }), + row("/fr/a/", { entryId: "a-fr", translationGroup: "a-de", locale: "fr", status: "unpublished", views30: 7 }), + ); + await setState(runtime, synced); + return runtime; + } + + it("finds an entry by id, with its published translations and their total", async () => { + host = await translated(); + + const result = await call(host, TOOL_ROUTES.entryViews, { entryId: "a-en" }); + + expect(result.found).toBe(true); + expect(result.entry).toMatchObject({ entryId: "a-en", path: "/en/a/", views7: 1, views30: 12 }); + expect(result.translations.map((t: { entryId: string }) => t.entryId)).toEqual(["a-de"]); + expect(result.allLanguages).toEqual({ views7: 4, views30: 42 }); + expect(result.windows.views7).toMatchObject({ since: addDays(TODAY, -6), until: TODAY }); + }); + + it("finds an entry by the path or URL it is published at, in any spelling", async () => { + host = await translated(); + + for (const path of ["/de/a", "/de/a/", "https://example.test/de/a?ref=x"]) { + const result = await call(host, TOOL_ROUTES.entryViews, { path }); + expect(result.entry?.entryId, path).toBe("a-de"); + } + }); + + it("leaves an unpublished entry's old numbers out of the total", async () => { + host = await translated(); + + const result = await call(host, TOOL_ROUTES.entryViews, { entryId: "a-fr" }); + + expect(result.entry).toMatchObject({ entryId: "a-fr", status: "unpublished", views30: 7 }); + expect(result.allLanguages).toEqual({ views7: 4, views30: 42 }); + }); + + it("reports the index incomplete while a rebuild runs", async () => { + host = await translated(); + await setState(host, { ...synced, rebuild: true }); + + const result = await call(host, TOOL_ROUTES.entryViews, { entryId: "a-de" }); + + expect(result.indexComplete).toBe(false); + }); + + it("answers not found as data", async () => { + host = await translated(); + + for (const input of [{ path: "/nowhere/" }, { entryId: "missing" }, {}]) { + const result = await call(host, TOOL_ROUTES.entryViews, input); + expect(result, JSON.stringify(input)).toMatchObject({ found: false, entry: null, translations: [], allLanguages: null }); + } + }); +}); + +describe("site_totals", () => { + it("sums the window and the period before it", async () => { + host = await newHost(); + await seedRollup(host, 60); + await setState(host, synced); + + const result = await call(host, TOOL_ROUTES.siteTotals, { days: 30 }); + + expect(result).toMatchObject({ + window: { days: 30, since: addDays(TODAY, -29), until: TODAY, partial: false }, + pageviews: 300, + visits: 150, + estimated: false, + provisional: true, + previous: { since: addDays(TODAY, -59), until: addDays(TODAY, -30), pageviews: 300, visits: 150 }, + historySince: addDays(TODAY, -59), + }); + }); + + it("does not compare against a period the store covers only in part", async () => { + host = await newHost(); + await seedRollup(host, 60); + await setState(host, synced); + + const result = await call(host, TOOL_ROUTES.siteTotals, { days: 90 }); + + expect(result.previous).toBeNull(); + expect(result.window.partial).toBe(true); + expect(result.pageviews).toBe(600); + }); + + it("flags a window with a sampled day as estimated", async () => { + host = await newHost(); + await seedRollup(host, 7); + const date = addDays(TODAY, -3); + await host.fixtures.plugin.storage("rollup", date, { + date, + pageviews: 40, + visits: 20, + sampleInterval: 10, + fetchedAt: NOW.toISOString(), + }); + await setState(host, synced); + + const result = await call(host, TOOL_ROUTES.siteTotals, { days: 7 }); + + expect(result.estimated).toBe(true); + expect(result.pageviews).toBe(6 * 10 + 40); + }); + + it("answers an empty store with zeros and no history", async () => { + host = await newHost(); + + const result = await call(host, TOOL_ROUTES.siteTotals); + + expect(result).toMatchObject({ pageviews: 0, visits: 0, previous: null, historySince: null, lastSync: null }); + expect(result.window.partial).toBe(true); + }); +}); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 989fa5e..b37865a 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -22,22 +22,25 @@ importers: version: 0.39.0(@date-fns/tz@1.5.0)(@types/react@19.3.0)(date-fns@4.4.0)(react-dom@19.3.0(react@19.3.0))(react@19.3.0)(zod@4.5.4) '@emdash-cms/plugin-cli': specifier: 0.12.0 - version: 0.12.0(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) + version: 0.12.0(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) '@emdash-cms/plugin-test': specifier: 0.2.0 - version: 0.2.0(59949aeb10c26a9714a29cfa8eddca48) + version: 0.2.0(f863f6b7fe5271f41d62dece9033c2f3) emdash: specifier: '>=0.39.0 <1.0.0' version: 0.39.1(@astrojs/react@7.0.0(@types/node@26.6.2)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.3.0(react@19.3.0))(react@19.3.0)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.3(@tiptap/core@3.31.3(@tiptap/pm@3.31.3))(@tiptap/pm@3.31.3))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.5(@types/node@26.6.2)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.5)(react-dom@19.3.0(react@19.3.0))(react@19.3.0)(supports-color@10.2.2)(typescript@5.9.3) tsdown: specifier: 0.20.3 - version: 0.20.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)(typescript@5.9.3) + version: 0.20.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)(typescript@5.9.3) typescript: specifier: ^5.9.0 version: 5.9.3 vitest: specifier: ^4.1.0 version: 4.1.11(@types/node@26.6.2)(vite@8.3.0(@types/node@26.6.2)(esbuild@0.28.2)(yaml@2.9.1)) + zod: + specifier: 4.5.4 + version: 4.5.4 packages/shared: devDependencies: @@ -3999,9 +4002,6 @@ packages: zod@4.5.4: resolution: {integrity: sha512-sC95tT5iHHH9gtpj6A81kh+NEaRAUFN+qlUPDUbRfOMvNf5QCBqsb3WgvnpVtK5Y+4UfA6KqufotuTvMGiTlsA==} - zod@4.6.5: - resolution: {integrity: sha512-v5l/aFXZQeai4awLbOpSoHecE9UiMrnfx75tEXLjNonXVARxQ5mOeipTjROUchszUNCqnE+hqAMujRsRHsut2Q==} - zrender@6.1.0: resolution: {integrity: sha512-oEGMDB6pOP2S6OwRR4PdVv610zrjnA3Bh+JnSG12fYJlBKjtNAoEb5fSUoCOOINlH96I2fU38/A2UpRKs67xYQ==} @@ -4905,7 +4905,7 @@ snapshots: '@wordpress/block-serialization-default-parser': 5.56.0 parse5: 7.3.0 - '@emdash-cms/plugin-cli@0.12.0(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)': + '@emdash-cms/plugin-cli@0.12.0(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)': dependencies: '@atcute/client': 5.1.2(@atcute/lexicons@2.1.1)(typescript@6.0.3) '@atcute/identity-resolver': 2.0.2(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@atcute/lexicons@2.1.1)(typescript@6.0.3) @@ -4924,7 +4924,7 @@ snapshots: jsonc-parser: 3.3.1 modern-tar: 0.7.7 picocolors: 1.1.1 - tsdown: 0.20.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)(typescript@6.0.3) + tsdown: 0.20.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)(typescript@6.0.3) typescript: 6.0.3 zod: 4.5.4 transitivePeerDependencies: @@ -4944,12 +4944,12 @@ snapshots: - unplugin-unused - vue-tsc - '@emdash-cms/plugin-test@0.2.0(59949aeb10c26a9714a29cfa8eddca48)': + '@emdash-cms/plugin-test@0.2.0(f863f6b7fe5271f41d62dece9033c2f3)': dependencies: '@cloudflare/vitest-plugin': 1.2.5(@cloudflare/workers-types@5.20260924.1)(@types/node@26.6.2)(@vitest/runner@4.1.11)(@vitest/snapshot@4.1.11)(vitest@4.1.11(@types/node@26.6.2)(vite@8.3.0(@types/node@26.6.2)(esbuild@0.28.2)(yaml@2.9.1))) '@emdash-cms/blocks': 0.39.0(@date-fns/tz@1.5.0)(@types/react@19.3.0)(date-fns@4.4.0)(react-dom@19.3.0(react@19.3.0))(react@19.3.0)(zod@4.5.4) '@emdash-cms/cloudflare': 0.39.0(a7ee94a070d1e2ec5745e252646edfd0) - '@emdash-cms/plugin-cli': 0.12.0(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) + '@emdash-cms/plugin-cli': 0.12.0(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) '@emdash-cms/plugin-types': 0.4.0 emdash: 0.39.0(@astrojs/react@7.0.0(@types/node@26.6.2)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.3.0(react@19.3.0))(react@19.3.0)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@5.9.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.3(@tiptap/core@3.31.3(@tiptap/pm@3.31.3))(@tiptap/pm@3.31.3))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.5(@types/node@26.6.2)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.5)(react-dom@19.3.0(react@19.3.0))(react@19.3.0)(supports-color@10.2.2)(typescript@5.9.3) kysely: 0.29.6 @@ -5554,13 +5554,6 @@ snapshots: '@tybys/wasm-util': 0.10.4 optional: true - '@napi-rs/wasm-runtime@1.2.4(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)': - dependencies: - '@emnapi/core': 1.10.0 - '@emnapi/runtime': 1.11.3 - '@tybys/wasm-util': 0.10.4 - optional: true - '@napi-rs/wasm-runtime@1.2.4(@emnapi/core@1.11.1)(@emnapi/runtime@1.11.1)': dependencies: '@emnapi/core': 1.11.1 @@ -5773,9 +5766,9 @@ snapshots: '@napi-rs/wasm-runtime': 1.2.4(@emnapi/core@1.10.0)(@emnapi/runtime@1.10.0) optional: true - '@rolldown/binding-wasm32-wasi@1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)': + '@rolldown/binding-wasm32-wasi@1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)': dependencies: - '@napi-rs/wasm-runtime': 1.2.4(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) + '@napi-rs/wasm-runtime': 1.2.4(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) transitivePeerDependencies: - '@emnapi/core' - '@emnapi/runtime' @@ -6384,7 +6377,7 @@ snapshots: vitefu: 1.1.3(vite@8.3.0(@types/node@26.6.2)(esbuild@0.28.2)(yaml@2.9.1)) xxhash-wasm: 1.1.0 yargs-parser: 22.0.0 - zod: 4.6.5 + zod: 4.5.4 optionalDependencies: sharp: 0.35.4(@types/node@26.6.2) transitivePeerDependencies: @@ -7709,7 +7702,7 @@ snapshots: nlcst-to-string: 4.0.0 unist-util-visit: 5.1.0 - rolldown-plugin-dts@0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3))(typescript@5.9.3): + rolldown-plugin-dts@0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3))(typescript@5.9.3): dependencies: '@babel/generator': 8.0.0-rc.2 '@babel/helper-validator-identifier': 8.0.0-rc.2 @@ -7720,13 +7713,13 @@ snapshots: dts-resolver: 2.1.3 get-tsconfig: 4.14.3 obug: 2.2.1 - rolldown: 1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) + rolldown: 1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) optionalDependencies: typescript: 5.9.3 transitivePeerDependencies: - oxc-resolver - rolldown-plugin-dts@0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3))(typescript@6.0.3): + rolldown-plugin-dts@0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3))(typescript@6.0.3): dependencies: '@babel/generator': 8.0.0-rc.2 '@babel/helper-validator-identifier': 8.0.0-rc.2 @@ -7737,7 +7730,7 @@ snapshots: dts-resolver: 2.1.3 get-tsconfig: 4.14.3 obug: 2.2.1 - rolldown: 1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) + rolldown: 1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) optionalDependencies: typescript: 6.0.3 transitivePeerDependencies: @@ -7764,7 +7757,7 @@ snapshots: '@rolldown/binding-win32-arm64-msvc': 1.0.0-rc.17 '@rolldown/binding-win32-x64-msvc': 1.0.0-rc.17 - rolldown@1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3): + rolldown@1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3): dependencies: '@oxc-project/types': 0.112.0 '@rolldown/pluginutils': 1.0.0-rc.3 @@ -7779,7 +7772,7 @@ snapshots: '@rolldown/binding-linux-x64-gnu': 1.0.0-rc.3 '@rolldown/binding-linux-x64-musl': 1.0.0-rc.3 '@rolldown/binding-openharmony-arm64': 1.0.0-rc.3 - '@rolldown/binding-wasm32-wasi': 1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) + '@rolldown/binding-wasm32-wasi': 1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) '@rolldown/binding-win32-arm64-msvc': 1.0.0-rc.3 '@rolldown/binding-win32-x64-msvc': 1.0.0-rc.3 transitivePeerDependencies: @@ -8034,7 +8027,7 @@ snapshots: trough@2.2.0: {} - tsdown@0.20.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)(typescript@5.9.3): + tsdown@0.20.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)(typescript@5.9.3): dependencies: ansis: 4.4.0 cac: 6.7.14 @@ -8044,8 +8037,8 @@ snapshots: import-without-cache: 0.2.5 obug: 2.2.1 picomatch: 4.0.7 - rolldown: 1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) - rolldown-plugin-dts: 0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3))(typescript@5.9.3) + rolldown: 1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) + rolldown-plugin-dts: 0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3))(typescript@5.9.3) semver: 7.8.5 tinyexec: 1.3.1 tinyglobby: 0.2.17 @@ -8063,7 +8056,7 @@ snapshots: - synckit - vue-tsc - tsdown@0.20.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3)(typescript@6.0.3): + tsdown@0.20.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3)(typescript@6.0.3): dependencies: ansis: 4.4.0 cac: 6.7.14 @@ -8073,8 +8066,8 @@ snapshots: import-without-cache: 0.2.5 obug: 2.2.1 picomatch: 4.0.7 - rolldown: 1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3) - rolldown-plugin-dts: 0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.10.0)(@emnapi/runtime@1.11.3))(typescript@6.0.3) + rolldown: 1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3) + rolldown-plugin-dts: 0.22.5(rolldown@1.0.0-rc.3(@emnapi/core@1.11.3)(@emnapi/runtime@1.11.3))(typescript@6.0.3) semver: 7.8.5 tinyexec: 1.3.1 tinyglobby: 0.2.17 @@ -8352,8 +8345,6 @@ snapshots: zod@4.5.4: {} - zod@4.6.5: {} - zrender@6.1.0: dependencies: tslib: 2.3.0