docs(observability): document the dashboards, alerts and local stack

[sourcehawk]

Towards #187

Summary

• New docs/observability.md: what ships (two templated dashboards, one per-operator and two shared cluster-wide rule files), the render pipeline (make dashboards|alerts, NAMESPACE_LABEL, ALERT_FORMAT, PROMETHEUSRULE_NAMESPACE/PROMETHEUSRULE_LABELS, output layout, installing rules and dashboards), matching the recorder's controller name to controller-runtime's, every alert with its threshold and for plus the tuning rationale drawn from the rule files, both dashboards (rows, browser variables and drill-down, the stale-series topk join, templated uids, cluster-scoped owners), the local stack with the scripted world as a scenario-to-alert table, and the three test targets. No screenshots.
• mkdocs.yml nav entry under Concepts, a card on docs/index.md, and a row in the README documentation table.
• docs/component.md Metrics section links to the new page and its example passes "webapp" to metrics.NewRecorder with a sentence on why it must equal the controller-runtime controller name; the same change in the pkg/metrics package and NewRecorder GoDoc.
• .ai/base.md lists the page, observability/, and a docs-table row; .github/copilot-instructions.md regenerated with make ai-instructions.
• make sync-plugin copies the page into plugin/skills/building-components/references/observability.md, and the skill's references list points at it.

Verification

• make fmt-md, make lint-md, make docs-build (strict), go build ./..., make all all pass.
• make sync-plugin and make ai-instructions leave the tree clean after the commits.
• Every make target, variable, alert name, metric name, PrometheusRule name and dashboard uid on the page was grepped against Makefile, observability/ and pkg/metrics; the controller-runtime default controller name and histogram bucket bounds were checked against the v0.24.1 module source.
• Two fresh-reader passes (an operator team installing the artifacts; a maintainer using the local stack and tests) with only the page text; the gaps they hit (the three meanings of "namespace", run from the repo root, passing the same variables to both targets, which bucket bounds a threshold may move to, the exact matcher edit for a custom condition, shared-rule ownership, which test file belongs to which rule file) were closed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JPihvXVfS997iGmGabTGsd

[Copilot]

🟢 Approval recommended

Changes are documentation-centric with consistent cross-linking and aligned GoDoc updates, and no correctness issues were found in the updated references and rendering description.

Pull request overview

Adds first-class observability documentation for Operator Component Framework, describing shipped Grafana dashboards and Prometheus alert rules, how to render/install them, and how to run the local Prometheus+Grafana dev stack—then wires the new page into the docs site, README, AI instruction references, and the Claude plugin skill references.

Changes:

• Added docs/observability.md covering artifacts, render pipeline variables, alert semantics/tuning, dashboard structure, and the local dev stack + tests.
• Updated component/metrics docs (and metrics.NewRecorder GoDoc) to require that the recorder controller name matches controller-runtime’s controller name for shipped dashboards/alerts to correlate series.
• Integrated the new page into MkDocs nav + docs index, README docs table, AI instruction reference lists, and plugin reference sync.

File summaries

File	Description
README.md	Adds Observability entry to the documentation table.
mkdocs.yml	Adds Observability page to site navigation under Concepts.
docs/index.md	Adds an Observability “card” linking to the new page.
docs/observability.md	New comprehensive documentation for dashboards, alerts, rendering, local stack, and testing.
docs/component.md	Updates metrics example/controller-name guidance and links to observability docs.
pkg/metrics/metrics.go	Updates package docs + NewRecorder GoDoc to clarify controller-name alignment requirement.
Makefile	Syncs docs/observability.md into plugin references via make sync-plugin.
plugin/skills/building-components/SKILL.md	Adds the new observability reference to the skill’s reference list.
plugin/skills/building-components/references/observability.md	Synced copy of docs/observability.md for the plugin.
plugin/skills/building-components/references/component.md	Syncs the updated metrics example/guidance into plugin reference docs.
.github/copilot-instructions.md	Regenerated instructions to include observability docs/paths and docs table row.
.ai/base.md	Updates the AI base reference lists and docs-update table to include observability.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟢 Approval recommended

The documentation and example/controller-name alignment are consistent with the existing Makefile targets, observability templates/tests, and E2E controller names.

Review details

• Files reviewed: 14/14 changed files
• Comments generated: 0 new
• Review effort level: Lite