Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -112,9 +112,7 @@ jobs:
# rules can each be individually correct and leave the state reported by
# nobody. kubeconform SKIPs the Grafana CRDs and no gate executes the
# PromQL, so an alert that can never fire looks exactly like one that has
# not fired yet. kubeconform SKIPs the Grafana CRDs and no gate
# executes the PromQL, so an alert that can never fire looks exactly like
# one that has not fired yet.
# not fired yet.
- name: Excluded alert states are somebody's job
run: ./scripts/check-alert-coverage.py

Expand All @@ -128,6 +126,17 @@ jobs:
- name: Every alert severity routes to a declared contact point
run: ./scripts/check-alert-severity-routes.py

# A burn-rate summary is the alert TITLE — the sentence that reaches a
# human first and often the only one they read. The figure in it is prose
# that nothing parses, so a rule can name a budget its own expression does
# not spend and pass every rule-level validation, because the expression is
# correct and only the sentence is wrong. This derives the figure from the
# rule's factor and window against the objective the dashboards measure, so
# there is no constant here to agree with the standard today and be
# compared to nothing tomorrow.
- name: Burn-rate summaries state the budget their expression spends
run: ./scripts/check-burn-rate-budgets.py

# ── Kyverno policy unit tests ────────────────────────────────────────
# `kyverno test` proves each policy rule PASSES a compliant resource and FAILS
# a violating one (policies/kyverno/tests). Without this, a policy could be
Expand Down
11 changes: 11 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,17 @@ aggregate, so a clean `task validate` is necessary but not sufficient:
`scripts/check-catalog-revision.py` (CI job `catalog-revision`)
- **Alert coverage** — `scripts/check-alert-coverage.py`, which runs inside the
`dashboards` job alongside the locally-available dashboard and Athena gates
- **Burn-rate budget claims** — `scripts/check-burn-rate-budgets.py`, in the same
`dashboards` job. A burn-rate summary is the alert title, and the budget figure
in it is prose nothing parses, so a rule can name a figure its own expression
does not spend while every rule-level validation passes. The figure is derived
rather than listed: burn factor times long window over the SLO window, with the
factor and window read from the rule's expression and the SLO window from the
dashboard panel measuring the same selector — out of the render, so it is a
panel a cluster receives rather than a file on disk. A rule whose selectors no
delivered panel measures is a finding, because a figure compared to nothing is
how the last wrong one survived. The figure has no independent existence:
there is no constant to correct and none in the summary that anything trusts
- **Image vulnerabilities** — `scripts/check-image-vulnerabilities.py` (CI job
`image-vulnerabilities`). The one with a target of its own,
`task validate:image-vulnerabilities`, kept out of the aggregate because it
Expand Down
2 changes: 1 addition & 1 deletion dashboards/base/alerting/agent-operator.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ spec:
service: eks-agent-platform
component: operator
annotations:
summary: operator reconcile latency budget burning (100% over 3d)
summary: operator reconcile latency budget burning slowest (10% in 3d)
description: >
The fraction of reconciles over 1s exceeds 1x the 99% latency objective
over both the 3d and 6h windows — the budget is being spent at exactly the
Expand Down
Loading