Repository navigation
ci(k8s): test discovery against a KIND cluster instead of SANDBOX - #866
Merged
Merged
Conversation
The Discovery K8s Tests workflow depended on secrets.SANDBOX_KUBECONFIG and the shared SANDBOX cluster's ci-verify-basic namespace. Replace that with a disposable KIND cluster created per test job: - .test/k8s/basic/kind-up.sh creates/reuses a 'rwl-ci' KIND cluster, applies the ci-verify-basic fixture, and exports a kubeconfig pointing at the control-plane IP on the kind Docker network. - .test/k8s/basic/kind/ci-verify-basic.yaml mirrors the SANDBOX fixture (Namespace + podinfo Deployment/Service + Ingress) so the existing >=5 (basic) and >=12 (detailed) SLX minimums still hold. - _discovery_common.sh honours RWL_CONTAINER_NETWORK so the RunWhen Local container joins the kind network; CI sets it to 'kind'. - Test Taskfiles now use the 'kind-rwl-ci' context; README documents local use. - Remove the .test/k8s/upload harness, which required a SaaS upload service. Workflow also triggers on .test/k8s/** changes.
ci-test-2 (detailed LOD) produced 11 SLXs against KIND vs the SANDBOX baseline of 12. The missing SLX comes from the detailed-only k8s-fluxcd-helm-health generation rule, which matches helmreleases.helm.toolkit.fluxcd.io -- the SANDBOX ci-verify-basic namespace had a Flux HelmRelease that the first fixture pass omitted. Add a HelmRelease to the fixture plus a minimal local CRD (kind/crds/) so it is served and indexed without a full Flux install or network fetch. Basic LOD is unaffected (the rule is detailed-only), and ci-test-1 still expects 5.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The Discovery K8s Tests workflow depended on
secrets.SANDBOX_KUBECONFIGand the shared SANDBOX cluster'sci-verify-basicnamespace. This replaces that with a disposable KIND cluster created per test job.Changes
.github/workflows/test-k8s.yaml— dropsSANDBOX_KUBECONFIG; calls.test/k8s/basic/kind-up.sh rwl-ci, then runs the task withRWL_CONTAINER_NETWORK: kind. Adds.test/k8s/**to the PR path filter..test/k8s/basic/kind-up.sh(new) — creates/reuses arwl-ciKIND cluster (kindest/node:v1.31.0), applies the fixture, exportskubeconfig.secret. Auto-installskindon Linux; requires it on macOS..test/k8s/basic/kind/ci-verify-basic.yaml(new) — mirrors the SANDBOX fixture (runwhen/infra-flux-nonprod-sandbox/apps/ci-verify-basic): Namespace +podinfoDeployment + Service + Ingress. Discovery only reads those API objects, so SLX counts are driven by resource types..test/k8s/basic/Taskfile.yaml— context renamedsandbox-cluster-1→kind-rwl-ci..test/k8s/basic/_discovery_common.sh— honoursRWL_CONTAINER_NETWORKto attach the container to a Docker network..test/k8s/basic/README.md— documents local KIND usage..test/k8s/upload/— removed; it required a SaaS upload service we no longer operate.How the container reaches the cluster
KIND publishes its API on a random
127.0.0.1port that a bridge container can't reach.kind-up.shrewrites the exported kubeconfig to the control-plane container's IP on thekindDocker network (in the API server cert SANs via the kubeadm advertise address, so TLS verification still holds — no insecure flag), and the test container joins that same network. Air-gap--add-hostflags keep working because this is a bridge network, not--network host.Testing
ci-test-1(basic) andci-test-2(detailed) SLX assertions are minimums (>=5,>=12) carried over from the SANDBOX namespace; the fixture reproduces the same indexed resource types, so they should hold. A live CI run is the confirmation..github/workflows/test-aks.yamlstill usesSANDBOX_KUBECONFIGfor its real AKS coverage.