Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/web/PRODUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ The console runs beside the administrator's own Core, with execution, files and

## Information Architecture

- **Monitor**: Overview (service status, running Sessions, sandbox slots, Sessions needing attention, 24-hour Session activity, a compact inventory of Core and up to four nodes, prioritizing offline and degraded nodes when the list is full, with a popover glance at each, the attention table, usage by project), Core metrics (the Core process's CPU and memory, execution slots and the Turn queue, connected daemons, the database and background jobs), Agent metrics (requests, errors, duration, tokens, models, tools, Agents and API keys for 1 h / 6 h / 24 h / 7 d), Sandbox metrics (node capacity and hosted Runtimes across projects; a node or a sandbox opens in a dialog with its figures and CPU and memory charts), Session log (every Session, read-only, with a failed Session's reason under its status, opening one Session's history, which jumps to its failed Turns; a self-hosted Session's page also has its environment's executor credentials). Agent metrics' By Agent table opens an Agent's page and, from its failed Turns, its Sessions in the Session log.
- **Monitor**: Overview (service status, running Sessions, sandbox slots, Sessions needing attention, 24-hour Session activity, a compact inventory of Core and up to four nodes, prioritizing offline and degraded nodes when the list is full, with a popover glance at each, the attention table, usage by project), Core metrics (the Core process's CPU and memory, execution slots and the Turn queue, connected daemons, the database and background jobs), Agent metrics (requests, errors, duration, tokens, models, tools, Agents and API keys for 1 h / 6 h / 24 h / 7 d), Sandbox metrics (node capacity and hosted Runtimes across projects; a node or a sandbox opens in a dialog with its figures and CPU and memory charts), Session log (every Session, read-only, with a failed Session's reason under its status, opening one Session's history, which jumps to its failed Turns; a self-hosted Session's page also has its environment's executor credentials). Session facts and Agent metrics link to an Agent's page only when that saved resource exists; inline snapshots and snapshots of deleted Agents remain readable without a broken resource link. Agent metrics' failed Turns open the Agent's Sessions in the Session log.
- **Resources**: Agents, Environment templates, Skills, Files, Vaults. Each list shows one project or all projects, with a Project column when all are shown and a Creator column naming the creating key. Detail pages show the resource's facts and offer Delete.
- **Platform**: Projects and keys (projects, their assets and usage, named keys, write history), Nodes (the node list, capacity, host figures, allocations and individual node operations). Add node asks for limits before issuing its one-time command; installers use Core's public URL and require supported node artifacts. Removal offers the host's uninstall command. System owns installation facts, the Domain and HTTPS secondary page, each harness's default model configuration, startup settings, and a link to the Sandbox configuration secondary page. That page owns setup, resource edits, rollout details and reset. Setup selects a backend, size and Runtime, then asks for a deliberate save; own-machine setup continues to Add node.
- A node whose provider is not ready names the reason as one Provider-neutral readiness class (provider unavailable, host unsupported, provider files or Runtime image missing, Runtime download failed, a host too small) and its fix in the help tip beside its status, wherever that status shows.
Expand Down
23 changes: 23 additions & 0 deletions apps/web/e2e/monitoring.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,29 @@ test("opens a Session's conversation from the Session log, read-only", async ({
await expect(page.getByRole("textbox")).toHaveCount(0);
});

test("links Session snapshots only to saved Agents that exist", async ({ page, request }) => {
await openConsole(page, request, "sessions");
await page.getByRole("row").filter({ hasText: "Churn forecaster" }).first().getByRole("button", { name: /^Open Session / }).click();
const agent = page.getByLabel("Session facts").getByRole("button", { name: /^Open Agent / });
await expect(agent).toBeVisible();
await agent.click();
await expect(page.getByLabel("Agent details")).toBeVisible();
});

test("keeps inline and deleted Agent snapshots readable without resource links", async ({ page, request }) => {
await page.route("**/core/v1/projects/*/agents?*", (route) => route.fulfill({ json: { object: "list", data: [], has_more: false, first_id: null, last_id: null } }));
await openConsole(page, request, "sessions");
await page.getByRole("row").filter({ hasText: "Churn forecaster" }).first().getByRole("button", { name: /^Open Session / }).click();
const facts = page.getByLabel("Session facts");
await expect(facts).toContainText("Churn forecaster");
await expect(facts.getByRole("button", { name: /^Open Agent / })).toHaveCount(0);
await page.getByRole("button", { name: "Agent metrics", exact: true }).click();
await expect(page.getByRole("heading", { name: "By Agent", exact: true })).toBeVisible();
await expect(page.getByRole("button", { name: /^Open Agent / })).toHaveCount(0);
// A snapshot ID still filters its Sessions even when it is not a saved resource.
await expect(page.getByRole("button", { name: /^\d+ failed Turns? — open .+'s Sessions$/ }).first()).toBeVisible();
});

test("keeps a failed Session's reason in sight in the Session log and on its page", async ({ page, request }) => {
const reasons = /Execution could not complete\. No specific cause was reported\./;
await page.setViewportSize({ width: 1280, height: 800 });
Expand Down
19 changes: 13 additions & 6 deletions apps/web/src/features/metrics/AgentMetricsPage.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ import { DashboardSkeleton, TableSkeleton } from "../../components/Skeleton";
import { useFailureToast } from "../../components/Toast";
import { useConsoleNavigation } from "../../lib/console-navigation";
import { formatClock, formatCompact, formatDuration, formatInteger, formatPercent, formatRelative, MISSING } from "../../lib/format";
import { ProjectFilter, ProjectName, useProjects, type ProjectFilterValue } from "../../lib/projects";
import { ProjectFilter, ProjectName, useProjectCollection, useProjects, type ProjectFilterValue } from "../../lib/projects";
import { collections } from "../../lib/queries";
import {
agentMetricsOutcome,
INLINE_AGENT_ID,
Expand Down Expand Up @@ -96,6 +97,8 @@ export function AgentMetricsPage() {
const { state: projectsState, byId } = useProjects();
const [filter, setFilter] = useState<ProjectFilterValue>("");
const [range, setRange] = useState<AgentMetricsRange>("24h");
const agents = useProjectCollection(collections.agents, filter);
const savedAgents = new Set(agents.items.filter((row) => !agents.failures.some((failure) => failure.project.id === row.project.id)).map((row) => `${row.project.id}:${row.value.id}`));
const projects = projectsState.projects;
const projectsReady = projectsState.status !== "loading" || projects.length > 0;
const { state, keyUsage, refreshing, refresh } = useAgentMetricsData(projects, projectsReady, filter, range);
Expand Down Expand Up @@ -123,7 +126,7 @@ export function AgentMetricsPage() {
) : null}
<ProjectFilter value={filter} onChange={setFilter} />
<SegmentedControl label={t("range.label")} value={range} options={RANGES.map((value) => ({ value, label: t(`range.${value}`) }))} onChange={setRange} />
<RefreshButton refreshing={refreshing} updatedAt={loaded ? formatClock(loaded.loadedAt, locale) : null} onClick={refresh} />
<RefreshButton refreshing={refreshing} updatedAt={loaded ? formatClock(loaded.loadedAt, locale) : null} onClick={() => { refresh(); agents.refresh(); }} />
</>}
/>
<PageBody>
Expand All @@ -136,6 +139,7 @@ export function AgentMetricsPage() {
) : (
<AgentMetricsContent
metrics={metrics}
savedAgents={savedAgents}
listFailures={loaded?.listFailures ?? []}
showProject={!filter}
projectOf={(id) => (id ? byId.get(id) : undefined)}
Expand Down Expand Up @@ -163,6 +167,7 @@ function coverageNote(loaded: Loaded, t: TFunction<"metrics">): string | null {

function AgentMetricsContent({
metrics,
savedAgents,
listFailures,
showProject,
projectOf,
Expand All @@ -171,6 +176,7 @@ function AgentMetricsContent({
failure,
}: {
metrics: AgentMetrics;
savedAgents: ReadonlySet<string>;
listFailures: ProjectReadFailure[];
showProject: boolean;
projectOf: (id: string | null) => AdminProject | undefined;
Expand Down Expand Up @@ -362,8 +368,9 @@ function AgentMetricsContent({
<tbody>
{metrics.byAgent.map((agent) => {
const label = agentLabel(agent.agentId, agent.label);
// A saved Agent opens its page, and its failures its Sessions in the Session log; an inline Agent has neither.
const saved = agent.projectId !== null && !isInlineAgent(agent.agentId) ? { project: agent.projectId, id: agent.agentId } : null;
// Snapshot IDs do not establish that a saved Agent resource exists.
const sessionFilter = agent.projectId !== null && !isInlineAgent(agent.agentId) ? { project: agent.projectId, id: agent.agentId } : null;
const saved = sessionFilter && savedAgents.has(`${agent.projectId}:${agent.agentId}`) ? sessionFilter : null;
// The figure counts failed Turns in the range; the link lists all the Agent's Sessions, and its name and tooltip say so.
const openSessions = t("agent.openSessions", { count: agent.failed, agent: label });
return (
Expand All @@ -377,8 +384,8 @@ function AgentMetricsContent({
<td className="numeric">{integer(agent.sessions)}</td>
<td className="numeric">{integer(agent.requests)}</td>
<td className={agent.failed ? "numeric numeric-danger" : "numeric"}>
{saved && agent.failed ? (
<button type="button" className="figure-link" aria-label={openSessions} title={openSessions} onClick={() => navigate("sessions", saved, "agent-sessions")}>{integer(agent.failed)}</button>
{sessionFilter && agent.failed ? (
<button type="button" className="figure-link" aria-label={openSessions} title={openSessions} onClick={() => navigate("sessions", sessionFilter, "agent-sessions")}>{integer(agent.failed)}</button>
) : integer(agent.failed)}
</td>
<td className="numeric">{formatPercent(agent.finished ? agent.failed / agent.finished : null, locale)}</td>
Expand Down
11 changes: 7 additions & 4 deletions apps/web/src/features/sessions/SessionPage.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import type { AgentSession } from "@oac/agents-client";
import { useQueryClient } from "@tanstack/react-query";
import { useQuery, useQueryClient } from "@tanstack/react-query";
import { ArrowLeft, Trash2 } from "lucide-react";
import { useEffect, useMemo, useRef, useState } from "react";
import { useTranslation } from "react-i18next";
Expand All @@ -11,7 +11,7 @@ import { CopyableId } from "../../components/list-ui";
import { useConsoleNavigation } from "../../lib/console-navigation";
import { formatClock, formatDateTime, formatInteger, MISSING } from "../../lib/format";
import { CreatorCell, ProjectName, useCreators, useProjects } from "../../lib/projects";
import { collections } from "../../lib/queries";
import { collectionQuery, collections } from "../../lib/queries";
import { forgetDeleted } from "../resources/detail-queries";
import { ExecutorCredentialsSection } from "./ExecutorCredentialsSection";
import { SessionDeleteDialog, type SessionDeleteTarget } from "./SessionDeleteDialog";
Expand Down Expand Up @@ -78,10 +78,13 @@ export function SessionPage() {
</button>
);
const session = history.history?.session ?? null;
const agents = useQuery({ ...collectionQuery(collections.agents, projectId ?? ""), enabled: Boolean(projectId && session) });
const savedAgent = !agents.isError && agents.data?.some((agent) => agent.id === session?.agent.id);
const loadedAt = history.history?.loadedAt ?? null;
const refresh = () => {
setRefreshToken((value) => value + 1);
history.refresh();
if (projectId && session) void agents.refetch();
if (projectId && sessionId) void queryClient.invalidateQueries({ queryKey: [...sessionKey(projectId, sessionId), "diagnostics"] });
if (projectId && sessionId) void queryClient.invalidateQueries({ queryKey: [...sessionKey(projectId, sessionId), "executor-credentials"] });
};
Expand Down Expand Up @@ -130,9 +133,9 @@ export function SessionPage() {
<div>
<dt>{t("detail.agent")}</dt>
<dd className="session-fact-stack">
<button type="button" className="session-link" onClick={() => navigate("agents", { project: projectId, id: session.agent.id })} aria-label={t("detail.openAgent", { name: session.agent.name || session.agent.id })}>
{savedAgent ? <button type="button" className="session-link" onClick={() => navigate("agents", { project: projectId, id: session.agent.id })} aria-label={t("detail.openAgent", { name: session.agent.name || session.agent.id })}>
{session.agent.name?.trim() || t("common.untitledAgent")}
</button>
</button> : <span>{session.agent.name?.trim() || t("common.untitledAgent")}</span>}
<CopyableId id={session.agent.id} compact />
</dd>
</div>
Expand Down
Loading