From 8021682762051a35a4148c519b76a1dfe25dc094 Mon Sep 17 00:00:00 2001 From: Snodgrass-Wilkerschnoz <56047210+Snodgrass-Wilkerschnoz@users.noreply.github.com> Date: Thu, 15 Oct 2026 08:52:25 -0600 Subject: [PATCH] Fix stale command and flag names in agent docs and skills `list-call-results` and `run-call` no longer exist on production or QC, and neither results command accepts `--since`/`--before`. Rename them to `list-results-by-call`, `run-monitor`, and `--from`/`--time`, document what the time-window flags mean, and note which API monitor commands are deprecated on platforms serving the v3 API. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 14 ++++++++------ .../apimetrics-config-as-code/SKILL.md | 2 +- .../apimetrics-failure-investigation/SKILL.md | 14 +++++++------- .../apimetrics-incident-triage/SKILL.md | 4 ++-- .../apimetrics-monitoring-estate-audit/SKILL.md | 4 ++-- .../apimetrics-project-bootstrap/SKILL.md | 10 +++++----- .../claude-skills/apimetrics-slo-review/SKILL.md | 4 ++-- .../apimetrics-weekly-health-review/SKILL.md | 10 +++++----- skills/embed/setup-api-monitor.md | 12 ++++++------ 9 files changed, 38 insertions(+), 36 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index e8b55cda..c99e6f9f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -9,7 +9,9 @@ Instructions for AI coding agents (OpenAI Codex, Cursor, Gemini CLI, and any too ## Core facts about this CLI -- **Commands are flat, generated from the platform's live OpenAPI description**: `list-calls`, `create-call`, `run-call` — not `calls create`. The set can change with the server spec, so treat `apimetrics --help` and `apimetrics --help` as authoritative and inspect a command's help before assuming an option or body field. +- **Commands are flat, generated from the platform's live OpenAPI description**: `list-calls`, `create-call`, `run-monitor` — not `calls create`. The set can change with the server spec, so treat `apimetrics --help` and `apimetrics --help` as authoritative and inspect a command's help before assuming an option or body field. +- **API monitor commands are moving to v3.** Platforms that serve the v3 API mark `create-call`, `list-calls`, `get-call`, `update-call`, and `delete-call` as deprecated (hidden from `--help`, still callable) in favour of `create-api-monitor`, `list-api-monitors`, `get-api-monitor`, `update-api-monitor`/`patch-api-monitor`, and `delete-api-monitor`. Production still serves only the older names, so check `apimetrics --help` before choosing. +- **Result time windows** use `--from ` (window start) and `--time ` (window end; returns results before it) on `list-results` and `list-results-by-call`. Pass full timestamps such as `2026-07-01T00:00:00Z`; a bare date returns nothing. - **Create/update commands read JSON from stdin** via heredoc. There is no `--body`, `--data`, or `-d`: ```bash apimetrics create-call <<'EOF' @@ -18,7 +20,7 @@ Instructions for AI coding agents (OpenAI Codex, Cursor, Gemini CLI, and any too ``` CLI Shorthand is also accepted (`apimetrics create-call meta.name: "Health", request.method: GET, request.url: https://example.com/health`). - **Output**: `-o json` for machine use; `-f` projects with a shorthand query; `-q` adds confirmed query params. The command spec is cached ~24h and refreshes automatically; `--rsh-no-cache` forces a refresh. -- **List envelopes are not uniform.** `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{ "meta": ..., "results": [...] }`; `list-schedules` returns `{ "data": [...] }`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{ "results": [...] }`. Inspect each command's own output before writing an `-f` path. +- **List envelopes are not uniform.** `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{ "meta": ..., "results": [...] }`; `list-schedules` returns `{ "data": [...] }`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{ "results": [...] }`. Inspect each command's own output before writing an `-f` path. - **`get-result` is a summary** (`result`, `http_code`, `response_time` ms, `location_id`, `test`, `created`; `result` ∈ `PASS`/`FAIL`/`WARN`/`ERROR`/`TIMEOUT`/`QUEUED`). Deeper data comes from `get-result-content`, `get-result-screenshot`, the `query-*-performance`/`query-*-dns-diagnostics` commands, and `conformance-results`. - **Percentiles are server-computed** by the `query-*-performance` commands (`measures`: `mean`/`p50`/`p90`/`p95`/`p99`; API `metrics`: `total`/`dns`/`connect`/`tls`/`ttfb`/`response`). Do not hand-average result summaries. - **API assertions** are set with `set-call-conditions ` (read with `get-call-conditions`), not in the `create-call` body. @@ -31,10 +33,10 @@ Each entry names when to reach for it and the key commands. Full step-by-step wo | Workflow | Use when | Key commands | |---|---|---| -| **project-bootstrap** | setting up monitoring / onboarding a service | `project select`, `create-call`/`create-browser-monitor`/`create-mcp-monitor`, `set-call-conditions`, `create-schedule`, `add-call-to-schedule`, `run-call`/`run-monitor` | -| **weekly-health-review** | reliability report for the last 7 complete days | `list-results --since --before`, `query-api-performance`, `list-calls` | -| **failure-investigation** | root-causing one failing result or a series | `get-result`, `list-call-results --since --before`, `get-result-content`, `query-api-monitor-dns-diagnostics`, `get-call-conditions` | -| **incident-triage** | blast radius / common cause across many monitors | `list-results --since --before`, `query-api-dns-diagnostics`, `query-api-performance` | +| **project-bootstrap** | setting up monitoring / onboarding a service | `project select`, `create-call`/`create-browser-monitor`/`create-mcp-monitor`, `set-call-conditions`, `create-schedule`, `add-call-to-schedule`, `run-monitor` | +| **weekly-health-review** | reliability report for the last 7 complete days | `list-results --from --time`, `query-api-performance`, `list-calls` | +| **failure-investigation** | root-causing one failing result or a series | `get-result`, `list-results-by-call --from --time`, `get-result-content`, `query-api-monitor-dns-diagnostics`, `get-call-conditions` | +| **incident-triage** | blast radius / common cause across many monitors | `list-results --from --time`, `query-api-dns-diagnostics`, `query-api-performance` | | **monitoring-estate-audit** | coverage & hygiene governance review | `list-calls`, `list-schedules`, `list-slos`, `list-auth-settings`, `list-calls-by-auth` | | **slo-review** | SLO design & attainment (no attainment endpoint — derive it) | `list-slos`, `get-slo`, `list-results`, `query-api-performance` | | **config-as-code** | reviewed bulk edits with rollback | `bulk init apimetrics:/monitors`, `bulk status`, `bulk diff`, `bulk push`, `bulk reset` | diff --git a/skills/claude-skills/apimetrics-config-as-code/SKILL.md b/skills/claude-skills/apimetrics-config-as-code/SKILL.md index 0a7a1c9a..32274d67 100644 --- a/skills/claude-skills/apimetrics-config-as-code/SKILL.md +++ b/skills/claude-skills/apimetrics-config-as-code/SKILL.md @@ -31,7 +31,7 @@ Use the CLI's built-in hidden `bulk` workflow for local JSON checkout and synchr EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. diff --git a/skills/claude-skills/apimetrics-failure-investigation/SKILL.md b/skills/claude-skills/apimetrics-failure-investigation/SKILL.md index 8080815d..a45b49e9 100644 --- a/skills/claude-skills/apimetrics-failure-investigation/SKILL.md +++ b/skills/claude-skills/apimetrics-failure-investigation/SKILL.md @@ -30,7 +30,7 @@ Find the failure boundary and the smallest evidence-backed remediation. Do not s EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. @@ -61,15 +61,15 @@ If given a result ID, retrieve the result summary: apimetrics get-result -o json ``` -**Know what this returns.** `get-result` (and each row of `list-call-results`/`list-results`) is a *summary*: `result` (`PASS`/`FAIL`/`WARN`/`ERROR`/`TIMEOUT`/`QUEUED`), `http_code`, `response_time` (ms), `location_id`, `test` (monitor ID), and `created`. It does **not** contain component timings, DNS/TLS detail, headers, body, or assertion breakdowns. Pull those from the dedicated commands in step 3 — do not claim a DNS or TLS cause from `get-result` alone. +**Know what this returns.** `get-result` (and each row of `list-results-by-call`/`list-results`) is a *summary*: `result` (`PASS`/`FAIL`/`WARN`/`ERROR`/`TIMEOUT`/`QUEUED`), `http_code`, `response_time` (ms), `location_id`, `test` (monitor ID), and `created`. It does **not** contain component timings, DNS/TLS detail, headers, body, or assertion breakdowns. Pull those from the dedicated commands in step 3 — do not claim a DNS or TLS cause from `get-result` alone. -For an API call series (the argument is the call/monitor ID; supports `--since`/`--before`/`--limit`): +For an API call series (the argument is the call/monitor ID; supports `--from`/`--time`/`--limit`): ```bash -apimetrics list-call-results --since 2026-07-01T00:00:00Z -o json +apimetrics list-results-by-call --from 2026-07-01T00:00:00Z -o json ``` -For a project-wide sweep across monitor types, use `list-results --since ... --before ...`. Discover browser/MCP result access from `apimetrics --help`. +For a project-wide sweep across monitor types, use `list-results --from ... --time ...`. Discover browser/MCP result access from `apimetrics --help`. Also read the monitor configuration (`get-call`, `read-browser-monitor`, `read-mcp-monitor`), conditions (`get-call-conditions `), schedule, auth-setting metadata, and related SLO where those commands exist. @@ -91,7 +91,7 @@ Compare failing and passing results across the dimensions below. Each names the | Dimension | Where it comes from | |---|---| -| status / HTTP code / total response time / location | `get-result`, `list-call-results`, `list-results` | +| status / HTTP code / total response time / location | `get-result`, `list-results-by-call`, `list-results` | | component timings (`dns`, `connect`, `tls`, `ttfb`, `response`) and percentiles | `query-api-monitor-performance` (per monitor) or `query-api-performance` (project), body `{from,to,metrics,measures,group_by}` | | DNS provider / resolved IPs / CNAME chain / NS / mean lookup time | `query-api-monitor-dns-diagnostics` / `query-api-dns-diagnostics`, body `{from,to,group_by,locations}` | | response body / headers / named content | `get-result-content ` (the second positional arg is the path into the captured content) | @@ -131,7 +131,7 @@ Run an on-demand check only when authorized and when it will not create harmful API: ```bash -apimetrics run-call +apimetrics run-monitor ``` Browser/MCP: diff --git a/skills/claude-skills/apimetrics-incident-triage/SKILL.md b/skills/claude-skills/apimetrics-incident-triage/SKILL.md index f8cc1e76..2ece4437 100644 --- a/skills/claude-skills/apimetrics-incident-triage/SKILL.md +++ b/skills/claude-skills/apimetrics-incident-triage/SKILL.md @@ -30,7 +30,7 @@ Use project-wide evidence to distinguish an isolated monitor failure from a shar EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. @@ -50,7 +50,7 @@ Set exact start/end timestamps and timezone. Default to a narrow window around t Collect the project-wide result stream for the window with server-side filtering, then segment: ```bash -apimetrics list-results --since --before -o json # {meta, results:[...]}, page via meta.next_cursor +apimetrics list-results --from --time -o json # {meta, results:[...]}, page via meta.next_cursor ``` Each row gives `result`, `http_code`, `response_time`, `location_id`, and `test` (monitor ID) — enough to segment by monitor, status, and location. For shared-cause hypotheses, use `query-api-dns-diagnostics` (DNS provider/IP/CNAME drift by location) and `query-api-performance` (latency shift by location/interval) over the incident window. Map monitor IDs back to names/hosts/auth with `list-calls`, `list-browser-monitors`, `list-mcp-monitors`, and `list-auth-settings`. Identify: diff --git a/skills/claude-skills/apimetrics-monitoring-estate-audit/SKILL.md b/skills/claude-skills/apimetrics-monitoring-estate-audit/SKILL.md index 5c721862..940dca69 100644 --- a/skills/claude-skills/apimetrics-monitoring-estate-audit/SKILL.md +++ b/skills/claude-skills/apimetrics-monitoring-estate-audit/SKILL.md @@ -30,7 +30,7 @@ Perform a read-only audit by default. The deliverable is a prioritized cleanup a EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. @@ -51,7 +51,7 @@ Run the available list/read commands (JSON output), noting each envelope shape: - schedules — `list-schedules` (**`{data:[...]}`**), plus `list-schedule-downtimes` for maintenance windows - SLOs — `list-slos` (bare `{results}`), detail via `get-slo ` - auth settings/tokens as metadata only — `list-auth-settings`, and `list-calls-by-auth ` to size the blast radius of each shared credential -- recent results — `list-results --since ` for working-vs-configured coverage +- recent results — `list-results --from ` for working-vs-configured coverage - which schedules a call belongs to — `list-schedules-for-call ` ### 2. Normalize the inventory diff --git a/skills/claude-skills/apimetrics-project-bootstrap/SKILL.md b/skills/claude-skills/apimetrics-project-bootstrap/SKILL.md index 7f21123d..be5b1ce1 100644 --- a/skills/claude-skills/apimetrics-project-bootstrap/SKILL.md +++ b/skills/claude-skills/apimetrics-project-bootstrap/SKILL.md @@ -31,7 +31,7 @@ Build a usable monitoring project from an empty or selected APImetrics project. EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. @@ -213,14 +213,14 @@ Frequency is in seconds. The body also accepts `locations` (agent IDs), `regions ### 7. Verify on demand -For API calls (`run-call` accepts an optional body of `location_id`, `run_delay` (0–86400s), and `context`; with no body it runs at the default location): +For API calls (`run-monitor` accepts an optional body of `location_id`, `run_delay` (0–86400s), and `context`; with no body it runs at the default location): ```bash -apimetrics run-call -apimetrics list-call-results -o json +apimetrics run-monitor +apimetrics list-results-by-call -o json ``` -`run-call` returns `{call_id, location_id, result_id}`. `list-call-results` takes the call ID and supports `--since`/`--before` (ISO-8601) and `--limit` (max 100). +`run-monitor` returns `{call_id, context, location_id, result_id, user}`. `list-results-by-call` takes the call ID and supports `--from` (window start) / `--time` (window end), as full ISO-8601 timestamps, and `--limit` (max 100). For browser or MCP monitors (both use `run-monitor`; there is no `run-mcp-monitor` command): diff --git a/skills/claude-skills/apimetrics-slo-review/SKILL.md b/skills/claude-skills/apimetrics-slo-review/SKILL.md index 8c40ec4a..3a201a28 100644 --- a/skills/claude-skills/apimetrics-slo-review/SKILL.md +++ b/skills/claude-skills/apimetrics-slo-review/SKILL.md @@ -30,7 +30,7 @@ Review both the SLO configuration and the evidence behind it. Do not recommend c EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. @@ -75,7 +75,7 @@ Flag missing ownership, ambiguous scope, unattached SLOs, conflicting windows, a Because the CLI exposes no attainment endpoint, reconstruct it from the monitors the SLO scopes (via `include_tags`/`exclude_tags` or `scope_id`): -- For availability/pass objectives, count `result` categories over the objective `period` using `list-results`/`list-call-results` with `--since`/`--before`. +- For availability/pass objectives, count `result` categories over the objective `period` using `list-results`/`list-results-by-call` with `--from`/`--time`. - For latency objectives (`measure: mean`, metrics like `total`), use `query-api-performance` / `query-api-monitor-performance` with matching `metrics`/`measures` and window; align the query `interval` to the objective `period`. Report: diff --git a/skills/claude-skills/apimetrics-weekly-health-review/SKILL.md b/skills/claude-skills/apimetrics-weekly-health-review/SKILL.md index 7b5c73fa..25ba58ed 100644 --- a/skills/claude-skills/apimetrics-weekly-health-review/SKILL.md +++ b/skills/claude-skills/apimetrics-weekly-health-review/SKILL.md @@ -30,7 +30,7 @@ Analyze the previous seven complete calendar days in the active project's timezo EOF ``` Do not invent `--body`, `--data`, or `-d`. -6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-call-results`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). +6. Use `-o json` for analysis. Use `-f` only after inspecting the response shape. The top-level response envelope includes status, headers, and `body`. List bodies are NOT uniform: `list-calls`, `list-results`, `list-results-by-call`, and `list-auth-settings` return `{"meta":..., "results":[...]}`; `list-schedules` returns `{"data":[...]}`; `list-slos`, `list-browser-monitors`, and `list-mcp-monitors` return a bare `{"results":[...]}` with no `meta`/pagination. Inspect each command's own output before writing an `-f` path (e.g. `-f body.results[0]` vs `-f body.data[0]`). 7. Use `-q key=value` only for query parameters confirmed by command help or observed request documentation. 8. Preserve evidence. Record the active project, commands run, IDs, time window, and the smallest response excerpts needed to support conclusions. 9. Never print, store, or paste credentials into the report. Prefer existing auth-setting IDs. Do not include bearer tokens, cookies, API keys, client secrets, or private certificate contents. @@ -61,12 +61,12 @@ Run `apimetrics --help` and inspect help for the available inventory, result, an ```bash apimetrics list-calls -o json # {meta, results:[...]} apimetrics list-schedules -o json # {data:[...]} (note: data, not results) -apimetrics list-results --since --before -o json # project-wide result summaries -apimetrics list-call-results --since --before -o json +apimetrics list-results --from --time -o json # project-wide result summaries +apimetrics list-results-by-call --from --time -o json apimetrics get-result -o json # single result summary ``` -`list-results` and `list-call-results` support **server-side** `--since`/`--before` (ISO-8601) plus `--limit` (max 100) and `--cursor` — prefer these over pulling everything and filtering by hand. `list-results.meta.more`/`next_cursor` drive pagination. +`list-results` and `list-results-by-call` support **server-side** `--from`/`--time` (ISO-8601) plus `--limit` (max 100) and `--cursor` — prefer these over pulling everything and filtering by hand. `list-results.meta.more`/`next_cursor` drive pagination. **Do not hand-compute percentiles from result summaries.** Result rows carry only `result`, `http_code`, `response_time` (ms), and `location_id` — not component timings. For latency distribution use the analytics commands, which compute the statistics server-side over your window: @@ -101,7 +101,7 @@ Flag unscheduled, disabled, orphaned, and never-run monitors separately; do not Two complementary sources, both with `-o json`: -- **Availability / pass-fail** — `list-results --since --before` (project-wide) or `list-call-results --since --before` (per monitor). Each row gives `result`, `http_code`, `response_time`, `location_id`, `test`, `created`. Page with `--cursor` until `meta.more` is false. Count by `result` category; do not disable pagination unless intentionally sampling. +- **Availability / pass-fail** — `list-results --from --time` (project-wide) or `list-results-by-call --from --time` (per monitor). Each row gives `result`, `http_code`, `response_time`, `location_id`, `test`, `created`. Page with `--cursor` until `meta.more` is false. Count by `result` category; do not disable pagination unless intentionally sampling. - **Latency distribution** — the `query-*-performance` commands from step 2, which return server-computed `mean`/`p50`/`p95`/`p99` per metric, optionally grouped by monitor, location, and interval. For conformance signal, `conformance-results-summary` is available but best-effort (it can return a 500 on projects with no conformance data — treat a failure as "no data", not a finding). diff --git a/skills/embed/setup-api-monitor.md b/skills/embed/setup-api-monitor.md index e5f930e4..37b4aa5d 100644 --- a/skills/embed/setup-api-monitor.md +++ b/skills/embed/setup-api-monitor.md @@ -94,9 +94,9 @@ EOF ### 3. Run the monitor on-demand -`run-call` takes the call ID as a positional argument: +`run-monitor` takes the call ID as a positional argument: ```bash -apimetrics run-call +apimetrics run-monitor ``` The response contains a `result_id`. Save it for the next step. @@ -106,16 +106,16 @@ The response contains a `result_id`. Save it for the next step. ### 4. Poll results to verify ```bash -apimetrics list-call-results +apimetrics list-results-by-call ``` -A successful result has `result.success: true` and an HTTP status code in the 2xx range. Poll until the result from step 3 appears (match by `result_id`). Use `-f` to narrow output: +A successful result has `meta.result_category: "PASS"` and an HTTP status code (`meta.http_code`) in the 2xx range. Poll until the result from step 3 appears (match `id` against the `result_id`). Use `-f` to narrow output: ```bash -apimetrics list-call-results -f body.results[0] +apimetrics list-results-by-call -f body.results[0] ``` -**Validation gate:** Confirm `result.success` is `true`. If `false`, inspect `result.failure_reason` and the response body for details. +**Validation gate:** Confirm `meta.result_category` is `PASS`. If not, inspect `meta.result` and fetch the response body with `get-result-content `. ## Hard rules