Execute the digest rewrite the enforcing tier depends on - #243
Merged
Conversation
`mutateDigest: true` is the whole reason the enforcing tier enforces. Verification resolves a tag to a digest at admission; the kubelet resolves the same tag again at pull time. Pinning at admission is what stops anyone who can move a tag between those two moments from running an unverified image under a policy reporting success. In this repository that guarantee was a field value other files read, never a behaviour anything ran. Every Kyverno suite loads a BASE policy, where the field is false because Kyverno rejects true under an Audit failure action — so the only rendition carrying it was under no test at all. check-image-verification.py declines to assert the field and delegates the pairing; check-policy-validity.py asserts Enforce implies true and then applies a probe pod whose image is outside `ghcr.io/nanohype/*`, so the rule never matches and the mutation path never runs. ──────────────────────── What now runs ──────────────────────── scripts/check-digest-rewrite.sh renders each ENFORCING overlay and admits a pod through it — a real signed `ghcr.io/nanohype/*` release, referenced by TAG, because a digest-pinned fixture needs no rewrite and would satisfy every check against a policy that had stopped doing it. That fixture property is asserted rather than assumed. The overlays are enumerated from the tree, not named in the script. An overlay added later is one more tier depending on a step, and it has to be executed too. It reaches the registry for the manifest and Rekor for the transparency-log entry, which is the round trip admission makes — this exercises the path rather than a model of it. A verification that could not REACH those is told apart from a rewrite that did not happen: the first exits 2, because it observed nothing. ──────────── How the result is observed, and why by report ──────────── The Kyverno CLI offers no machine-readable channel carrying the object a verifyImages mutation admitted. Three were tried and each is recorded in the script. `kyverno test`'s `patchedResources` is compared against EVERY engine response for the rule, and a verifyImages rule produces two — one holding the resource as submitted, one holding it mutated — so one comparison always diffs whichever file is supplied, and supplying both produces a cross product with two failures. `kyverno apply -o` writes a file for mutate rules and leaves it empty for this one. The JSON policy report carries the verdict and the message and not the object. So the reading is the CLI's own detailed report. The expected patch supplied to it is the pod AS SUBMITTED, and that direction is what makes the reading non-circular: the mutated response differs from it, the report prints what the engine produced, and a digest in that output can only have come from the engine. An expectation carrying the digest would have the report echo the fixture back. That is text, and text is the weaker reading — but it fails in the safe direction. The assertion is that one reference token appears, so a report that stops printing it, wraps it, or renames the column makes the token absent and this gate red. It cannot go quiet. ──────────────────────── What it does not establish ──────────────────────── That a cluster admits what the CLI admits. The engine is the same code and the policy is the rendered one, and no environment reachable from this repository runs an API server. docs/threat-model.md now says that in those words, in place of a paragraph that had been mangled by an earlier edit into a duplicated fragment and an orphaned sentence. ci.yml carried a claim this work disproves — that `kyverno test` cannot reach Fulcio/Rekor and never verifies a signature. It does; the reason the suites never verified one is that they load base policies whose rule the fixtures do not match. Corrected rather than left standing beside a step that contradicts it. ──────────────────────── What proves it ──────────────────────── Both directions on the real tree. Unmutated: two enforcing overlays each admit the tagged pod carrying the digest verification resolved, exit 0. With the production overlay's patch flipped to `mutateDigest: false` — a change every YAML-reading gate still passes, because the field is simply gone rather than wrong — this exits 1 naming the overlay and what the tag then means. That is a probe in scripts/tests/reverify-gates.sh, floor 38 → 40, and the harness runs 43 checks. Registered in controls.py's NEEDS_NETWORK_SH rather than carrying a positive control: the round trip IS what it executes, and a control for it offline would be a control for a verification that did not happen. ──────────────────────── Verification ──────────────────────── ruff clean; mypy clean over 48 files; 493 tests across 19 modules; controls.py 18 controls; empty-corpus.py 31 probed gates; reverify-gates.sh 43/43; task validate exit 0; check-named-things.py 196 references resolve; yamllint clean; check-workflows.sh clean at MEDIUM and above. Co-authored-by: stxkxsbot <275011021+stxkxsbot@users.noreply.github.com>
CI Results
All checks passed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #221 (the execution half; see below for what #238/#241 already closed).
mutateDigest: trueis the whole reason the enforcing tier enforces, and in thisrepository it was a field value other files read rather than a behaviour anything
ran. Every Kyverno suite loads a BASE policy, where the field is false because
Kyverno rejects true under an
Auditfailure action — so the only renditioncarrying it was under no test at all.
scripts/check-digest-rewrite.shrenders each enforcing overlay and admits a realsigned
ghcr.io/nanohype/*release through it, referenced by TAG, reaching theregistry and Rekor — the round trip admission makes. The overlays are enumerated
from the tree, so an overlay added later is executed too.
Both directions, on the real tree. Unmutated: two enforcing overlays each admit
the tagged pod carrying the digest verification resolved, exit 0. With the
production overlay's patch flipped to
mutateDigest: false— a change everyYAML-reading gate still passes, because the field is simply gone rather than wrong —
it exits 1 naming the overlay. Kept as a probe in
reverify-gates.sh(floor 35 → 37,40/40).
How the result is observed is recorded in the script: the Kyverno CLI offers no
machine-readable channel carrying the object a
verifyImagesmutation admitted(
patchedResourcesis compared against every engine response and the rule producestwo;
apply -owrites nothing for it; the JSON report omits the object). The readingis the CLI's detailed report, with the expected patch supplied as the pod as
submitted so a digest in the output can only have come from the engine. It fails in
the safe direction — a report that stops printing the token makes this red.
What it does not establish: that a cluster admits what the CLI admits.
docs/threat-model.mdsays so in those words, replacing a paragraph an earlier edithad mangled into a duplicated fragment and an orphaned sentence.
ci.ymlcarried aclaim this disproves — that
kyverno testcannot reach Fulcio/Rekor — corrected.The issue's second half is already closed. It asks for coverage derived from
scripts/rather than a hand-writtenEXPECTED, and tests forcheck-policy-admission.pyandvalidate-dashboards.py. #238 landedcontrol_exempt_gates, theMAX_UNCOVERED_GATESceiling counted off the filesystem,and test modules for both.
Verification: ruff clean; mypy clean over 46 files; 476 tests / 18 modules;
controls.py17;empty-corpus.py30 probed gates;reverify-gates.sh40/40;task validateexit 0;check-named-things.py196 references; yamllint and zizmorclean.