Scan the images the pins reference, and make a CRITICAL a decision - #239
Merged
Conversation
An adopter of this catalog inherits every chart pin in it without reading, and
the version a pin resolves to decides which image lands on their nodes. Nothing
in the pipeline asked what those images carry. trivy-operator reports the same
findings at runtime — on a cluster that already pulled the image, to an operator
who cannot move the pin without a pull request here — and `trivy config` scans
the rendered manifests, which is a different question entirely.
So the scan belongs at the pin. `scripts/check-image-vulnerabilities.py` derives
its population from `check-image-pins.inventory()`, the same rendered set that
gate already walks, and blocks on a CRITICAL with a published fix that
`image-advisories.yaml` does not acknowledge.
───────────────────────────── What blocks, and why ─────────────────────────────
Three qualifiers, and each is load-bearing. CRITICAL, because HIGH is counted and
printed instead: gating it would hold merges on the rate at which upstream chart
images accumulate advisories rather than on anything a commit changed. WITH A
PUBLISHED FIX, because a CRITICAL with none is not a decision anyone here can
take. NOT ACKNOWLEDGED, because the alternative to an advisory file is a scan
whose findings nobody has to answer.
That leaves the objection `check-image-pins.py` already records — a new advisory
can turn a pull request red for a reason the pull request did not cause. The
advisory file is the answer to it rather than a softer bar: the failure names the
image, the CVE and the fixed version, and clears by moving the pin or by
recording why the finding stands. Both are decisions with an author.
──────────────────────── The advisory file cannot rot shut ────────────────────────
`image-advisories.yaml` ships nine entries covering the fifty fixed CRITICALs the
catalog's fifty-five rendered images carry. Every one is checked against the scan
on every run, in four directions:
quay.io/argoproj/argo-events:v1.9.11: CVE-2026-33815 in github.com/jackc/pgx/v5
v5.7.5 is CRITICAL and fixed in 5.9.0, and no entry in image-advisories.yaml
names it. Move the chart pin to a version carrying the fix, or record why the
finding stands.
quay.io/cilium/cilium:v1.19.6@sha256:0df5b275…: CVE-2026-56854 in
golang.org/x/crypto is acknowledged, but the entry does not list
quay.io/cilium/cilium. An acknowledgement covers the images it names — a new
image acquiring a known CRITICAL is a decision, not an inheritance.
image-advisories.yaml: CVE-2026-33815 in github.com/jackc/pgx/v5 lists
quay.io/cilium/cilium, which no longer carries it. Drop that image from the
entry.
image-advisories.yaml: CVE-2026-33815 in github.com/jackc/pgx/v5 is
acknowledged but no scanned image carries it — the entry outlived its
reason. Delete it.
Entries name images without a tag, so a chart bump does not churn the file — but
a bump that FIXES a finding trips the third or fourth rule rather than leaving a
stale excuse behind. An entry with no reason, and an entry with no images, fail
on their own.
────────────────────── The canary, without which green is mute ──────────────────────
A scanner with no database returns a clean result for every image, and by exit
code that is indistinguishable from a healthy fleet. Every run therefore scans a
digest-pinned end-of-life image first and requires its known CRITICALs back:
canary OK: 3 fixed CRITICAL finding(s) from the pinned end-of-life image, so a
clean result below is a result rather than a silence — CVE-2019-14697 (musl),
CVE-2019-14697 (musl-utils), CVE-2021-36159 (apk-tools)
Repointed at a patched image, the run refuses rather than passing:
Cannot run: the canary returned no fixed CRITICAL findings.
mirror.gcr.io/library/alpine:3.22
The canary is pinned by digest to an image whose CRITICALs have published
fixes and cannot be patched away underneath it, so an empty result here is a
fact about the scanner.
An image that fails to scan exits 2 with the image named, because counting the
rest as the whole fleet is how a partial scan reads as a complete one.
──────────────────── Three gates that passed over nothing ────────────────────
The same defect class, found while grading and fixed here because it is the same
argument: a gate reports on the population it read, and nothing in an exit code
separates "this catalog holds no violation" from "this run held no catalog".
`scripts/check-hardcoded-org.py` guarded that `applicationsets/` exists and never
guarded what the glob returned. Against a directory holding nothing, under
`--blocking`:
Scanned 0 applied ApplicationSet(s) in applicationsets/ (opt-in/ excluded)
✓ no applied ApplicationSet hardcodes nanohype/eks-gitops in a repoURL
exit 0
It now exits 2 naming the count. `check-platform-crs.py` printed `walked` and
compared it to nothing; `check-policy-admission.py` printed its rendered count
and compared it only against a render failure. Both now carry floors, and both
reject a corpus-emptying edit:
FAIL walked 4 platform CR(s), below the floor of 5. The catalog's CRs were
not matched against the chart's schemas, which is not the same as their
being admissible.
FAIL 7 manifest(s) rendered, below the floor of 40. The policies were
evaluated against a fleet this catalog does not have, and 'no addon
flagged' is a statement about that fleet rather than this one.
`scripts/tests/test_corpus_floors.py` holds both bounds on all four floors
against the tree: above zero, because zero is the vacuous pass with a constant in
front of it; and below the real corpus, because a gate that is always red is a
gate people route around.
────────────────────────────────── Wiring ──────────────────────────────────
CI job `image-vulnerabilities`, blocking, in the `needs` of both `pr-summary` and
`merge-gate` — the merge gate refuses any workflow containing a job it does not
watch. It installs the same pinned trivy the `validate` job does, behind the same
guard against `setup-trivy` reading an empty version as "latest".
`task validate:image-vulnerabilities` runs it locally and is deliberately NOT in
`task validate`: it pulls every image the pinned charts render, which is minutes
rather than seconds.
25 unit tests over the verdict, plus 8 over the floors. The gate pulls images, so
`scripts/tests/controls.py` exempts it from a positive control for the reason
that list already records — and the exemption is asserted like every other.
Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
One or more checks failed — see the job logs. |
This was referenced Sep 2, 2026
Every image the fleet renders is in the scanned set, and every chart that renders
one contributes to it.
The population came from the render, which was right, but the extraction from
that render was a regex anchored on `image:` preceded only by whitespace. The
ordinary Kubernetes list-item form, `- image: <ref>` under `containers:`, never
matched it. Against this catalog the pattern yielded 55 images where the pod
specs hold 57, and the two it missed were the entire workload of two charts:
quay.io/argoproj/argo-rollouts:v1.9.1
quay.io/argoproj/kubectl-argo-rollouts:v1.9.1
ghcr.io/stakater/reloader:v1.4.20
velero/velero-plugin-for-aws:v1.14.2
A CRITICAL in any of them passed the vulnerability gate clean, and the gate said
"55 image(s) scanned across 26 chart(s)" while it did. A scanner that omits
images silently is worse than none, because the green result is evidence.
The walk is now structural: every container in every pod owner's podSpec —
initContainers, containers, ephemeralContainers — plus every `image:` key
anywhere, which is how a custom resource naming an image its operator runs stays
in the population. 59 images across 28 charts.
The pattern is kept as an independent floor under the walks rather than deleted.
A parser and a regex fail on different inputs, so an image only the text scan
finds is either a controller reading a string payload — declared in
TEXT_ONLY_IMAGES with the reason, asserted in both directions — or a structural
walk that stopped seeing a shape, which is reported:
gateway-helm: docker.io/envoyproxy/ratelimit:1e50889b appears in the rendered
text and in no pod template or `image:` key — either a structural walk
stopped seeing a shape, or a controller reads it out of a payload and it
belongs in TEXT_ONLY_IMAGES with the reason
MIN_IMAGES is gone. A total cannot see the shape that happened: two charts left
the inventory and the count stayed large enough to look healthy. The floor is now
per chart and derived — every chart the render covers contributes at least one
image, or is declared imageless with its reason and asserted both ways:
b rendered and contributed no image. Every chart shipping a workload
contributes at least one, so either the extraction stopped seeing a shape
this chart uses, or the chart ships only CRDs and belongs in
IMAGELESS_CHARTS with the reason.
prometheus-operator-crds is declared imageless but the fleet no longer
renders it — the entry outlived its chart.
ai-gateway-crds-helm is declared imageless and now contributes an image. It
ships a workload; delete the entry so its images are scanned like every other
chart's.
Three smaller repairs in the vulnerability gate. A `reason:` key written and left
empty parsed as None and raised on `.strip()`, reaching a traceback instead of
the sentence written for that case; a non-mapping entry under `advisories:` did
the same. The `covered` set was accumulated on the acknowledged path and deleted
unread, reading as a ledger the four rules consult and being none. And the header
named one direction of the gate's time-dependence where there are two: a database
that gains an advisory turns a passing tree red, and one that loses an advisory —
or a publisher re-pushing a version tag on a patched base — turns a passing tree
red the other way, through rules 3 and 4, where the clearing action is deleting
an entry.
MIN_RENDERED's upper bound is now held: EveryFloorIsBelowTheCorpusItGuards
asserts it below the largest render the catalog can produce, so an always-red
floor fails there rather than surviving.
Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
One or more checks failed — see the job logs. |
…plate declares
Every image-shaped reference the production render carries is classified: it is a
container the render declares, an image a named controller starts, or something
declared not to be a container. Nothing is silently outside the population.
A pod template is not the deployable surface. An operator is handed a reference —
in a flag, a ConfigMap value, a CR field — and creates the pod afterwards, so the
container never appears in what `helm template` prints. Eleven such images render
here and none was scanned:
docker.io/envoyproxy/ai-gateway-extproc quay.io/argoproj/argoexec
ghcr.io/aquasecurity/node-collector quay.io/jetstack/cert-manager-acmesolver
ghcr.io/kyverno/kyverno natsio/nats-server-config-reloader
natsio/prometheus-nats-exporter docker.io/envoyproxy/ratelimit
The population is 70 images across 28 charts, from 59. The fixed CRITICALs it
carries go from 52 to 86, and the newly visible ones include CVE-2026-22039 in
kyverno itself — CRITICAL, fixed in 1.15.3 and 1.16.3, on a reference the chart
carries as `:latest`. Two mutable tags surfaced with them, `ghcr.io/kyverno/kyverno`
and `natsio/prometheus-nats-exporter`, both now on ALLOWED_MUTABLE with what
clears them.
Whether a string is an image a controller starts is a fact about that
controller's behaviour, and this gate reads manifests. So it is declared rather
than inferred, and the declaration is what carries the assertion: every
image-shaped string must be in the render's own `image:` keys, on
CONTROLLER_IMAGES with the controller that starts it, or on NOT_A_CONTAINER if
pulling it runs nothing. Anything else is reported. `declaration_rot` closes the
other direction for both tables — the reverse assertion the old comment claimed
and did not have.
The grammar requires a path separator, which is what separates an image from the
addresses that fill a rendered config: `loki.monitoring.svc.cluster.local:3100`
and `127.0.0.1:8080` are not images, and a grammar admitting them would demand a
declaration for every endpoint in the fleet.
The pod-owner walk is deleted. POD_OWNERS, CONTAINER_LISTS, _podspec and
podspec_images were a strict subset of the `image:` key walk — every container's
image is an `image:` key — so removing them changed no result, which is why no
test failed when they went. Mechanism with no effect reads as mechanism.
Four smaller repairs. The MIN_CRS floor test matched any `/v1alpha1` apiVersion,
counting 44 documents — every ApplicationSet and Kyverno fixture in the tree —
against the 8 the gate walks, so it held for any floor up to 43 and could not see
the always-red case it was added to see; it now filters on the operator's API
groups. The MIN_RENDERED bound multiplied by four environments where the gate
renders two, leaving 66..127 passing while making the gate red on every run.
Rules 3 and 4 iterated the unfiltered advisory list and reached a traceback on a
non-mapping entry. And chart_coverage is keyed by chart NAME, which folds this
catalog's three opentelemetry-collector units into one — said in the code rather
than left to be discovered.
The header no longer claims every image the fleet renders. It scans the
production render; the other three environments are not asked, nothing asserts
they agree, and a component enabled only in development or on the hub is deployed
and unscanned. That is the narrower claim this gate holds.
Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
CI Results
One or more checks failed — see the job logs. |
An official image is its own whole reference. `nats:2.10.10` has no registry and no organisation, so a grammar requiring a path separator could not see one — and the argo-events event-bus controller declares exactly that shape in its version table, beside the two `natsio/*` sidecars it starts in the same StatefulSet. One StatefulSet had its helpers scanned and its main container silent, which the gate's own words rule out: an unclassified image is reported so the answer to is-this-deployed is never silence. The population is 80 images from 70, and the fixed CRITICALs it carries go from 86 to 134. The single-segment alternative needs two discriminators the prefixed one does not, because a rendered config is full of strings with that shape. The match may not begin part-way through a longer token, or `vault.example.com:8200` yields `com:8200`. The name must start with a letter and carry no dot, which excludes a timestamp, an address, an IPv6 fragment and a ratio. The tag must be `latest` or begin with a digit, which excludes the RBAC names — `kyverno:admission-controller`, `system:auth-delegator` — that are the shape's other occupant. What remains and is not a container is declared: the falco rulesfile artifacts, a digest tail, two service addresses. `nats-streaming:latest` came with them: a moving tag in the same version table, on a row nothing here selects and nothing here can pin. MIN_CRS drops from 5 to 2. `walked` counts documents whose kind the pinned operator chart ships a schema for, so it depends on what that chart resolves to where the gate runs — a reviewer measured three where this tree walks eight. A floor above the smallest resolution makes the gate red somewhere it should be green, which is what a floor above its corpus always is. Two narrowings are now written down rather than left to be discovered. The population is chart-sourced Applications: a kustomize-sourced one renders workloads through no chart and contributes nothing, and check-policy-admission.py names one by path in KUSTOMIZE_WORKLOADS whose images run on every full-tier cluster. And it is the production render; the other three environments are not asked. Both are boundaries this gate has, stated beside each other. Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
One or more checks failed — see the job logs. |
Two ways an image escaped the scan entirely, and neither reached a verdict. Both are the same shape: the code was narrower than the sentence printed next to it. ──────────── A digest-pinned reference was not in the population ──────────── `IMAGE_REF` — the pattern under the structural walk, which is how an image handed to a controller in a flag or a ConfigMap is found — had two alternatives and both required a tag. `<repo>@sha256:<hex>` matched neither. The whole reference yielded only its `sha256:<hex>` tail, a single-segment shape whose bare name is `sha256`, and a NOT_A_CONTAINER entry by that name passed over it. A NOT_A_CONTAINER entry excuses whatever has its bare name, so that one entry excused every digest-pinned reference in the fleet. The sentence that makes it a defect rather than a limit is the gate's own remediation, which tells an operator to pin to a digest. The gate recommended the one spelling its completeness floor could not see. A third alternative reads the digest form, with or without a tag alongside it, and is tried first so `<repo>:<tag>@sha256:<hex>` is captured whole rather than as its `repo:tag` head — the digest a container runs is what reaches the classifier. The 64-hex suffix is discriminator enough on its own, so the path separator and tag restrictions the tag alternatives need are not repeated. MUTABLE_REMEDIATION becomes a module constant and a test reads the reference forms out of it, so advice naming a spelling the pattern does not read fails rather than shipping. The `sha256` declaration went with the fix, and `declaration_rot` is what removed it: with the repository-prefixed form matched whole, the fleet renders no bare digest for the entry to describe, and an entry matching nothing fails. Should one appear, it is matched, therefore reported, and the repair is a declaration that states its own reason. ──────────── An unplaceable reference exited 0 under another heading ──────── `inventory` carried two facts in one list. A chart whose render could not be produced or parsed, and an image-shaped reference the classifier could not place — printed together under "N chart(s) could not be rendered and were NOT scanned". The second chart rendered perfectly well; the string was never added to `failures`, so the run printed success and exited 0. CI blocked only because the sibling gate read the same list as a refusal, and a gate whose correctness depends on another gate's reading of its output has not stated its own result. Two lists, because they are two repairs and two verdicts: * a reference that cannot be placed is a failure, exit 1, naming the two declarations that resolve it — the controller that starts it, or the reason pulling it runs nothing; * a chart that did not render leaves the fleet's image set unknown, so a run with no other finding exits 2 and says the result covers part of the fleet rather than the fleet. When there is also a real failure the verdict is that failure, with the unrendered charts printed above it as the bound on what was covered. check-image-vulnerabilities.py reads both and refuses on either, so the scan population is never quietly smaller than the fleet it reports on. ──────────────────────── What proves it ──────────────────────── Ten mutants, each reverting one behaviour, each killed and each naming a test: the digest alternative removed, moved last, its hex run unbounded, its single-segment form barred; the unplaceable reference returned to the unrendered list, or reaching no verdict; the unrendered chart reporting a clean fleet, or dropped from a failing run's scope line; a remediation naming a form the pattern cannot read; and the declaration that used to swallow the digest tail. On the real tree, both directions. A digest-only reference planted in a rendered pod annotation: this gate exits 1 naming it, and the same tree under the previous reader prints "all 80 rendered image(s) carry an immutable reference" and exits 0. A chart declared imageless whose values will not parse — chosen because any other chart is caught by the per-chart floor first: exit 2 here, exit 0 and the identical success line before. Both are probes in scripts/tests/reverify-gates.sh, floor 25 → 27. scripts/tests/test_image_extraction.py adds the digest forms, the strings that are digest-shaped and are not digests, and main()'s verdict over a planted inventory with a clean control on both ends. The coverage ratchet rises to 22% combined and 71% on this gate. Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
One or more checks failed — see the job logs. |
#241 landed the vacuity harness, GATE_ARGS, gatelib.read_yaml, and corpus floors on check-platform-crs, check-policy-admission, validate-dashboards and kyverno-test.sh. This branch had independently added floors to two of those same gates and a module by the same name, so most of the conflict is two answers to one question rather than two changes to one file. ──────────────────────────── Conflicts ──────────────────────────── scripts/check-platform-crs.py — both sides added a floor at the same point. This branch floored `walked`; main asserts completeness instead: candidates by API GROUP against reached by SCHEMA KIND, with the number floor on candidates alone. main's is strictly stronger and subsumes the other — `walked` is incremented exactly where `reached` gains an entry, so candidates >= MIN_CRS with candidates - reached empty forces walked >= MIN_CRS. Keeping both would leave a rule whose failure is unreachable, so this branch's floor and its duplicate MIN_CRS go. scripts/check-policy-admission.py, twice — the same floor at the same value, arrived at independently. main's returns the named gatelib.CANNOT_RUN rather than a literal 2 and records why the bound cannot be derived and why the canary makes the floor necessary. Taken whole. scripts/tests/test_corpus_floors.py (add/add) — two modules of the same name written against overlapping gates. Resolved as their union, not as a choice: main's structure and its record of the two circular derivations that were rejected, plus this branch's degenerate-render bound, its IMAGELESS_CHARTS rot check, and its per-chart image-floor assertion. The `TheFloorsGuardTheRightQuantity` table is main's, because it names the expressions the merged gates actually contain. scripts/tests/run.py — both sides list test_corpus_floors; kept once. CLAUDE.md — two different task targets added to one list; both exist. ──────────────── A digest with no repository is not a reference ──────────────── The image-vulnerability gate refuses on any reference the classifier cannot place, and five of the ten it named in CI are bare `sha256:<hex>` values with no repository before them. Those name no image: there is no registry and no repository in one, only the value. Excluded by SHAPE rather than by a declaration, because the two rules this repository already has cannot both hold over it. A NOT_A_CONTAINER entry excuses whatever carries its bare name, so an entry for `sha256` would excuse every reference whose only matched token is a digest — and declaration_rot deletes an entry the render does not support, which is every tree that happens not to carry a bare digest. One rule demands the entry and the other removes it; a shape that can never be a reference is not an exemption to keep re-reading. Nothing is lost. A digest belonging to a repository is matched with that repository, whole, by the alternative above it, and stays classified as a digest. Two mutants, both killed and named: the exclusion removed, and the exclusion written unanchored so it eats repository digests too. ──────────────── Verification on the merged tree ──────────────── ruff clean; mypy clean over 38 files; 171 tests across 10 modules; coverage ratchet 23.1% combined (floor 22 → 23); controls.py 17 controls each rejecting at exit 1 naming the mutated file; empty-corpus.py 29 probed gates, up from 28 because check-image-vulnerabilities.py joins the sweep; reverify-gates.sh 30/30; task validate exit 0; check-image-pins.py 80 images across 28 charts; check-named-things.py 191 references resolve; check-workflows.sh clean at MEDIUM and above. Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
One or more checks failed — see the job logs. |
Two floors conflicted, both raised from one base by both sides:
COMBINED_FLOOR and reverify-gates.sh's MIN_CHECKS. A ratchet is measured,
not chosen, so each is set from the merged tree rather than from either
side — 29 and 35, against readings of 30.0% and 38 checks.
──────────────── Five references from nowhere, placed ────────────────
The image-vulnerability gate refused on five image-shaped references that
reach no pod template, one per OCI-sourced chart, each equal to its own
chart's last path segment:
operator: ghcr.io/nanohype/eks-agent-platform/charts/operator:0.6.7
ai-gateway-crds-helm: docker.io/envoyproxy/ai-gateway-crds-helm:1.0.0
gateway-helm: docker.io/envoyproxy/gateway-helm:1.8.3
ai-gateway-helm: docker.io/envoyproxy/ai-gateway-helm:1.0.0
karpenter: public.ecr.aws/karpenter/karpenter:1.14.0
They are the charts' own OCI coordinates, and they arrive in helm's pull
report — two lines it prints when it fetches an artifact from a registry:
Pulled: public.ecr.aws/karpenter/karpenter:1.14.0
Digest: sha256:<hex>
Some helm builds write that to stdout, where it leads the manifest stream
this gate parses; some to stderr, where nothing sees it. The stream is a
property of the build, so the same tree was green where it rendered and red
where a different helm ran.
`inventory` now reads the report from BOTH streams and hands what helm
reported pulling to `extract_images`, which passes over a reference helm
fetched AS A CHART. That is an observation about the run, and it is the only
thing that can answer the question: an OCI registry serves charts and images
through one grammar, so `public.ecr.aws/karpenter/karpenter:1.14.0` is
indistinguishable from an image by shape — and it is the chart, while the
container beside it is `public.ecr.aws/karpenter/controller`, which the walk
already finds and classifies as a digest.
Compared whole, never on the bare name: a registry can serve a chart and an
image from one repository path, and passing over every reference sharing a
pulled chart's name would take the image with it.
──────────────── Why these are not a declaration ────────────────
NOT_A_CONTAINER is re-checked against the render by `declaration_rot`, which
removes an entry the render does not support. These references are in the
render only under the helm builds that print the report to stdout, so one
rule would require the entry and the other would delete it, and the tree
would be red either way. Nothing is declared, so nothing rots. The same
reasoning is why a bare `sha256:<hex>` is excluded by shape rather than
named.
──────────────── What proves it ────────────────
Reproduced on the real tree, both directions, with a shim that runs the real
helm and folds stderr into stdout on a cold OCI cache — the stream shape
those helm builds produce. Without the reader the gate exits 1 naming
exactly those five, each attributed to the chart that pulled it; with it,
exit 0 over the same 80 images. That shim is a probe in
`scripts/tests/reverify-gates.sh`, floor 34 → 35.
Twelve tests, and five mutants each killed and named: the report not read;
matched on the bare name so the image beside the chart goes with it; read
from stdout only, so the verdict depends on which helm ran; the `Digest:`
line taken as an artifact; and any line naming a reference counted. The
stdout-only mutant survived a first attempt, because a case with the report
on stderr and nothing in the render to place asserts the same either way —
the case now renders a chart carrying its own coordinate, where the two
stream shapes give different verdicts unless both are read.
Verification on the merged tree: ruff clean; mypy clean over 39 files; 272
tests across 11 modules; coverage ratchet 30.0% (floor 29); controls.py 17
controls; empty-corpus.py 29 probed gates; reverify-gates.sh 38/38; task
validate exit 0; check-named-things.py 195 references resolve;
check-workflows.sh clean at MEDIUM and above; check-renovate-coverage.py
exit 0.
Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
All checks passed. |
…gate
CodeQL reported two high py/redos in IMAGE_REF, both in the registry part:
[a-z0-9][a-z0-9._-]*(?:\.[a-z0-9._-]+)+
A starred class already carrying `.` and `-`, followed by a plus-quantified
group whose body is that same class. Every way of splitting a run of `-.`
between the two quantifiers is a distinct parse, and the engine tries all of
them before failing. Measured on the flagged expression: `0.` followed by
twenty `-.` repetitions — 43 characters — took two thirds of a second, and
every four further repetitions multiplied that by fifteen. Sixty-seven
characters is most of an hour.
The input is rendered content from charts this repository does not author
and this gate runs on every render, so the string arrives with a chart bump
rather than with an attacker. The symptom is a job that never returns rather
than one that fails, which is the same class as an unbounded read: it does
not need an attacker to be a defect.
CodeQL is advisory here rather than required, and this would have merged
green. That is a reason to read it, not to skip it.
──────────────── The registry grammar has one definition of a host ────────────────
Closed by the grammar rather than by tuning the two lines. A domain
COMPONENT carries no dot — dots are the separators — and neither starts nor
ends with a dash:
DOMAIN_COMPONENT = [a-z0-9]+(?:-+[a-z0-9]+)*
REGISTRY = (?:COMPONENT(?:\.COMPONENT)+(?::\d+)?/)?
`[a-z0-9]+` and `-+` are disjoint, so a run of dashes can only be a
separator inside one component and a dot can only end one. Every input has
exactly one parse or none, and the hostile string is linear: four thousand
characters in half a millisecond.
Written once and used by both alternatives that need it, because two
spellings of one grammar drift. The digest alternative keeps its place
first, so `<repo>:<tag>@sha256:<hex>` is still captured whole rather than as
its `repo:tag` head.
Two narrowings, both of which the grammar already required and neither of
which this fleet's registries use: no underscore in a host, and no dot
inside a component. Every reference the fleet renders parses unchanged.
──────────────── What proves it ────────────────
The regression cases assert COMPLETION, not output. A rewrite that
reintroduces the ambiguity still matches the same references and still
rejects the same non-references — the only thing it changes is how long it
takes to say so, so time is the property.
They run the match in a CHILD process with a timeout, because a matcher that
never returns cannot be caught by an in-process timing assertion: it never
reaches it. A hang is a failure with a message, in bounded time.
Four mutants, each killed and named. The first is the flagged expression
restored verbatim, and the two timing cases are what catch it. Also: a
repeated component allowed to begin with a dash, which is the ambiguity on
its own; a component allowed to start or end with a dash; and a dotless host
admitted as a registry — that last one is a decision with a consequence,
because `localhost:11211` in a Loki config reaches the walk as a
single-segment token, which is why NOT_A_CONTAINER carries an entry for it,
and admitting a dotless registry would leave that entry matching nothing.
Verification: ruff clean; mypy clean over 39 files; 279 tests across 11
modules; coverage ratchet 30.1% (floor 29); check-image-pins.py 80 images
across 28 charts; controls.py 17 controls; empty-corpus.py 29 probed gates;
reverify-gates.sh 38/38; task validate exit 0; check-named-things.py 195
references resolve; check-workflows.sh clean at MEDIUM and above.
Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
#238 landed controls.py's vacuity rules taking their lists as arguments, run.py's control-exempt coverage check, the per-file corpus assertion on discover(), and reverify-tests.sh in task validate. This branch reads the image population out of that same discover(), so the two meet there. ──────────────── scripts/check-image-pins.py ──────────────── Two hunks, both from #238 extracting the mutable-tag verdict out of main() into a testable `verdict(images, allowed)` and bringing a `bare_name` with it. The extraction is kept — it is what the tests #238 added call. The `bare_name` that came with it is dropped as a duplicate: this branch's strips the digest before looking for a tag, and #238's does not, so it would key `ghcr.io/x/y@sha256:<hex>` as `ghcr.io/x/y@sha256` and no exemption would match it. One reader, and it is the one that reads both forms. `verdict()` uses this branch's MUTABLE_REMEDIATION rather than the sentence it was extracted with, because a test reads the reference forms out of that constant and checks the pattern reads each one. A message retyped inside the function would drift from the pattern with nothing to notice. main() keeps this branch's shape — the per-chart floor, and the split between a chart that did not render and a reference that could not be placed — and calls verdict() for the mutable half. ──────────────── scripts/tests/run.py ──────────────── Four hunks. EXPECTED and PER_GATE_FLOORS are unions: both sides added entries, and both name check-image-pins.py, whose floor is measured on the merged tree that carries both sides' tests for it — 88 against a reading of 89%. COMBINED_FLOOR was raised from one base by both sides. A ratchet is measured, not chosen: 40, against 40.8%. MAX_UNCOVERED_GATES ratchets DOWNWARD, so the lower of the two holds: 11. This branch's paragraph stays with it, because what it records is still true — two of the files counted as covered are covered by IMPORT rather than by tests, since check-image-vulnerabilities.py loads check-image-pins.py by path and that loads render-addons.py in turn. ──────────────── One corpus, confirmed rather than assumed ──────────────── #238's per-file assertion requires every ApplicationSet pinning a chart to contribute a unit and every matrix element pinning one to appear among that appset's units. `inventory` walks exactly those units, and `chart_coverage` then requires each to have contributed an image. Passing over a reference helm reported pulling sits between the two. It cannot take a real contribution: a chart artifact is not an image the chart deploys and never entered the population. What it must not do is leave a chart looking covered while contributing nothing, and that is asserted rather than argued — a chart whose render carries only its own pulled coordinate contributes no image and the per-chart floor names it. Measured on the tree under the stream shape that engages the reader — helm's pull report on stdout, cold OCI cache: 32 units discovered, 28 contributing, no chart named by the inventory that discover() does not have, and chart_coverage empty. The reverify probe holds that shape durably. ──────────────── Verification on the merged tree ──────────────── ruff clean; mypy clean over 46 files; 476 tests across 18 modules; coverage ratchet 40.8% (floor 37 → 40), 13 per-gate floors held, 10 of 24 gates with no unit coverage against a ceiling of 11, and all 8 control-exempt gates carrying unit coverage; controls.py 17 controls; empty-corpus.py 29 probed gates; reverify-tests.sh 31/31; reverify-gates.sh 38/38; task validate exit 0; check-named-things.py 195 references resolve; check-workflows.sh clean at MEDIUM and above. Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
All checks passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Item 5: a CI-time scan of the images the pinned charts reference. Folds in the
three vacuous-gate fixes named as out of scope on the gate-coverage PR, because
they are the same argument — a gate that passes over nothing.
The gap
The catalog pinned 26 charts rendering 55 images and asked nothing about what
those images carry.
trivy configscans rendered manifests;check-image-pins.pyasks whether a reference is immutable;
trivy-operatorreports at runtime, to anoperator who cannot move a pin without a pull request here. Whoever moves the pin
owns what the image carries, so the scan belongs at the pin.
Measured on this commit, at
--severity CRITICAL,HIGH --ignore-unfixed:What blocks
A CRITICAL with a published fix that
image-advisories.yamldoes notacknowledge. Each qualifier is load-bearing, and the docstring says why. HIGH is
counted and printed — 1,125 of them — because gating it would hold merges on the
rate at which upstream chart images accumulate advisories rather than on anything
a commit changed.
A new advisory can turn a PR red for a reason the PR did not cause. That is the
objection
check-image-pins.pyalready records, and the advisory file is theanswer to it rather than a softer bar: the failure names the image, the CVE and
the fixed version, and clears by moving the pin or recording why the finding
stands.
The advisory file cannot rot shut
Nine entries, checked against the scan every run, in four directions. All four
proven by mutation against the real fleet:
Entries name images without a tag, so a chart bump does not churn the file — but
a bump that fixes a finding trips the third or fourth rule instead of leaving a
stale excuse behind.
The canary
A scanner with no database returns clean for every image, which by exit code is a
healthy fleet. Every run scans a digest-pinned end-of-life image first:
Repointed at a patched image, it refuses rather than passing:
Raising the image floor above the render refuses too, and an image that fails to
scan exits 2 with the image named — counting the rest as the whole fleet is how a
partial scan reads as a complete one.
Three gates that passed over nothing
check-hardcoded-org.pyguarded thatapplicationsets/exists and never guardedwhat the glob returned. Against a directory holding nothing, under
--blocking:Now exit 2 naming the count.
check-platform-crs.pyprintedwalkedandcompared it to nothing;
check-policy-admission.pycompared its rendered countonly against a render failure. Both now reject a corpus-emptying edit:
scripts/tests/test_corpus_floors.pyholds both bounds on all four floorsagainst the tree — above zero, and below the real corpus, because a gate that is
always red is a gate people route around.
Wiring
image-vulnerabilities, blocking, in theneedsof bothpr-summaryand
merge-gate(the merge gate refuses a workflow containing a job it doesnot watch). Same pinned trivy as the
validatejob, behind the same guardagainst
setup-trivyreading an empty version as "latest".task validate:image-vulnerabilitiesruns it locally; deliberately not intask validate, since it pulls all 55 images.controls.pyexempts it from a positive control for the reason that listalready records — and the exemption is asserted like every other.
Verification
All exit 0 on the branch head:
task validate— 24 gates (kyverno CLI at the CI pin v1.18.2)ruff check .;mypyover 36 filesCOVERAGE_REQUIRED=1 coverage run --rcfile=.coveragerc scripts/tests/run.py—128 tests across 9 modules, 18.9% combined, ceiling lowered 17 → 15
./scripts/tests/controls.py— 17 controls completed a proof./scripts/check-image-vulnerabilities.py— 55 images, 50 fixed CRITICAL allacknowledged, 1,125 fixed HIGH counted
yamllint,./scripts/check-workflows.sh,./scripts/check-named-things.pyOne finding routed onward
ghcr.io/nanohype/eks-agent-platform/eval-runner:0.1.0carries CVE-2026-59873(
tar, fixed 7.5.19). It is the one first-party image in the advisory file, sothe remedy is a rebuild in that repository and a new operator chart version here,
not a decision this catalog can take alone.
Note on the ceiling
Two files read as newly covered without gaining a test: the new gate loads
check-image-pins.pyby path so the two cannot disagree about the fleet, andthat loads
render-addons.pyin turn, so importing the first executes all threemodule bodies.
MAX_UNCOVERED_GATESis lowered anyway — a ratchet that declinesto move is one nobody can regress against — and the comment in
run.pysaysplainly that this is why the per-gate floors and the controls carry the real
claim.
Second round — the extraction blocker
The population came from the render, which was right; the extraction from it was
a pattern.
IMAGEwas anchored onimage:preceded only by whitespace, so theordinary list-item form
- image: <ref>undercontainers:never matched.Against this catalog the pattern yielded 55 images where the pod specs hold
57, and the four it missed were the entire workload of two charts:
A CRITICAL in any of them passed clean while the gate printed
55 image(s) scanned across 26 chart(s). Reproduced and then closed: 59 images across 28charts.
The walk is structural — every container in every pod owner podSpec
(initContainers, containers, ephemeralContainers), plus every
image:keyanywhere, which keeps a custom resource naming an image its operator runs in the
population. The pattern stays as an independent floor under the walks: a parser
and a regex fail on different inputs, so an image only the text scan finds is
either a controller reading a string payload — declared in
TEXT_ONLY_IMAGESwith a reason, asserted both ways — or a walk that stopped seeing a shape, which
is reported.
MIN_IMAGESis gone. A total could not see the shape that happened. Thefloor is now per chart and derived: every chart the render covers contributes at
least one image, or is declared imageless with its reason. That rule catches this
exact defect directly, and its exemption is asserted in both directions.
M16 replayed: dropping the two rollouts images from the advisory file now
fails with them named. Before this change they were outside the population
entirely.
The other five findings
MIN_IMAGEShad no upper bound; its docstring named a test that does not existEveryFloorIsBelowTheCorpusItGuardsasserted the wrong direction forMIN_RENDEREDreason:key present and empty crashedstr(... or ""); a non-mapping entry underadvisories:is reported toocoveredaccumulated and deleted unread15 new tests in
test_image_extraction.py, each planting a violation: thelist-item spelling, every pod-owner kind, the CronJob depth, a CR-declared image,
an undeclared text-only image, an unparseable render, the Helm
=sentinel, andall five directions of the per-chart floor.
Verification:
task validateexit 0;ruff;mypyover 37 files; 145 testsacross 10 modules;
controls.py17 controls; the CVE gate over 59 images.