Skip to content

Commit b7afea2

Browse files
anandrkskdanandrkskd
authored
Mark openshift & xKS e2e tests (#1216)
* initial labeling tests for openshift Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * remove debugging Fit from tests Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * update openshift tests with label Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * update label to use notOnXKS instead Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * add make targets and update leftout tests Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * upload log artifact assisted-by: claude-code Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * update CI Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * skip tests Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * update fixture Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * update label, kind cluster with 3 nodes, add readme for labeling tests Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * update tests with todo for tests to update Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * fix make file target typo Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * fix HA tests Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * remove FIt Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * test failing because of service fqdn Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * skip 1-054_validate_argocd_agent_destination_mapping_test.go Should deploy principal and agent with destination-based mapping Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * fix lint error Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * updates from review comment Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> * label openshift gitops default instance test Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com> --------- Signed-off-by: Anand Kumar Singh <anandrkskd@gmail.com>
1 parent 72f2141 commit b7afea2

87 files changed

Lines changed: 433 additions & 329 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/kind-ci-automation.yaml‎

Lines changed: 37 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: Build, Deploy and Test on kind
1+
name: Build, Deploy and Test on KinD
22

33
on:
44
pull_request:
@@ -10,8 +10,12 @@ on:
1010
- "master"
1111
- "v*"
1212

13+
permissions:
14+
contents: read
15+
1316
env:
1417
IMG: gitops-operator:test
18+
LOG_DIR: /tmp/kind-ci-artifacts
1519

1620
jobs:
1721
ci-build:
@@ -26,10 +30,14 @@ jobs:
2630
with:
2731
go-version-file: 'go.mod'
2832

33+
- name: Prepare log directory
34+
run: mkdir -p "${{ env.LOG_DIR }}"
35+
2936
- name: Create kind cluster
3037
uses: helm/kind-action@ef37e7f390d99f746eb8b610417061a60e82a6cc # v1
3138
with:
3239
cluster_name: gitops-test
40+
config: test/kind-config.yaml
3341

3442
# TODO: check if porting is required for non-OCP clusters
3543
- name: Disable webhook and conversion for non-OCP cluster
@@ -44,7 +52,8 @@ jobs:
4452
4553
- name: Build manager image
4654
run: |
47-
make docker-build IMG=${{ env.IMG }}
55+
set -o pipefail
56+
make docker-build IMG=${{ env.IMG }} 2>&1 | tee "${{ env.LOG_DIR }}/docker-build.log"
4857
4958
- name: Load image into kind
5059
run: |
@@ -56,103 +65,36 @@ jobs:
5665
5766
- name: Deploy operator
5867
run: |
59-
echo "manifests: $(KUSTOMIZE) build config/default"
60-
make deploy IMG=${{ env.IMG }} | tee /tmp/deploy.log
68+
set -o pipefail
69+
make deploy IMG=${{ env.IMG }} 2>&1 | tee "${{ env.LOG_DIR }}/deploy.log"
6170
6271
- name: Verify Controller Manager deployment is available
6372
run: |
64-
kubectl get deployment -n openshift-gitops-operator
65-
kubectl describe deployment -n openshift-gitops-operator
66-
kubectl wait --for=condition=available --timeout=300s \
67-
deployment/openshift-gitops-operator-controller-manager \
68-
-n openshift-gitops-operator
69-
70-
- name: Create ArgoCD instance
73+
set -o pipefail
74+
{
75+
kubectl get deployment -n openshift-gitops-operator
76+
kubectl describe deployment -n openshift-gitops-operator
77+
kubectl wait --for=condition=available --timeout=300s \
78+
deployment/openshift-gitops-operator-controller-manager \
79+
-n openshift-gitops-operator
80+
} 2>&1 | tee "${{ env.LOG_DIR }}/verify-deployment.log"
81+
82+
- name: Run E2E tests Sequential on xKS
7183
run: |
72-
kubectl create ns test-argocd
73-
kubectl apply -f - <<'EOF'
74-
apiVersion: argoproj.io/v1beta1
75-
kind: ArgoCD
76-
metadata:
77-
name: argocd
78-
namespace: test-argocd
79-
EOF
84+
set -o pipefail
85+
NON_OLM=true make e2e-xks-tests-sequential-ginkgo 2>&1 | tee "${{ env.LOG_DIR }}/e2e-sequential.log"
8086
81-
- name: Wait for ArgoCD component pods to exist
87+
- name: Run E2E tests Parallel on xKS
8288
run: |
83-
EXPECTED_LABELS=("argocd-application-controller" "argocd-redis" "argocd-repo-server" "argocd-server")
84-
TIMEOUT=300
85-
INTERVAL=10
86-
ELAPSED=0
87-
88-
echo "Waiting for ArgoCD component pods to exist in test-argocd..."
89-
while true; do
90-
ALL_EXIST=true
91-
for label in "${EXPECTED_LABELS[@]}"; do
92-
if ! kubectl get pod -n test-argocd -l "app.kubernetes.io/name=${label}" --no-headers 2>/dev/null | grep -q .; then
93-
ALL_EXIST=false
94-
break
95-
fi
96-
done
97-
98-
if $ALL_EXIST; then
99-
echo "All ArgoCD component pods exist after ${ELAPSED}s."
100-
break
101-
fi
102-
103-
if [ $ELAPSED -ge $TIMEOUT ]; then
104-
echo "Timed out after ${TIMEOUT}s waiting for ArgoCD pods."
105-
kubectl get pods -n test-argocd
106-
kubectl get argocd -n test-argocd -o yaml
107-
exit 1
108-
fi
109-
110-
sleep $INTERVAL
111-
ELAPSED=$((ELAPSED + INTERVAL))
112-
done
89+
set -o pipefail
90+
NON_OLM=true make e2e-xks-tests-parallel-ginkgo 2>&1 | tee "${{ env.LOG_DIR }}/e2e-parallel.log"
11391
114-
- name: Verify ArgoCD components are ready
115-
run: |
116-
kubectl get pods -n test-argocd
117-
kubectl wait --for=condition=Ready -n test-argocd pod --timeout=300s \
118-
-l 'app.kubernetes.io/name in (argocd-application-controller,argocd-redis,argocd-repo-server,argocd-server)'
119-
echo "All ArgoCD components are ready."
120-
kubectl get pods -n test-argocd
121-
122-
- name: Collect operator debug info on failure
123-
if: failure()
124-
run: |
125-
echo "=== Deployment status ==="
126-
kubectl get deployment -n openshift-gitops-operator -o wide || true
127-
echo ""
128-
echo "=== Pod status ==="
129-
kubectl get pods -n openshift-gitops-operator -o wide || true
130-
echo ""
131-
echo "=== Pod descriptions ==="
132-
kubectl describe pods -n openshift-gitops-operator || true
133-
echo ""
134-
echo "=== Controller manager logs ==="
135-
kubectl logs deployment/openshift-gitops-operator-controller-manager \
136-
-n openshift-gitops-operator --all-containers=true --tail=200 || true
137-
echo ""
138-
echo "=== Events in operator namespace ==="
139-
kubectl get events -n openshift-gitops-operator --sort-by='.lastTimestamp' || true
140-
echo ""
141-
echo "=== CRD conversion config ==="
142-
kubectl get crd argocds.argoproj.io -o jsonpath='{.spec.conversion}' || true
143-
echo ""
144-
145-
- name: Collect ArgoCD debug info on failure
146-
if: failure()
147-
run: |
148-
echo "=== ArgoCD resources ==="
149-
kubectl get argocds -n test-argocd -o yaml 2>/dev/null || true
150-
echo ""
151-
echo "=== Pods in test-argocd ==="
152-
kubectl get pods -n test-argocd -o wide 2>/dev/null || true
153-
echo ""
154-
echo "=== Pod descriptions in test-argocd ==="
155-
kubectl describe pods -n test-argocd 2>/dev/null || true
156-
echo ""
157-
echo "=== Events in test-argocd ==="
158-
kubectl get events -n test-argocd --sort-by='.lastTimestamp' 2>/dev/null || true
92+
- name: Upload CI artifacts
93+
if: always()
94+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
95+
with:
96+
name: kind-ci-artifacts-${{ github.run_id }}
97+
path: ${{ env.LOG_DIR }}/
98+
if-no-files-found: ignore
99+
retention-days: 7
100+

‎Makefile‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,15 @@ endif
8181
SHELL = /usr/bin/env bash -o pipefail
8282
.SHELLFLAGS = -ec
8383

84+
# GINKGO_VERSION is the version of ginkgo to use.
85+
# Pick ginkgo version from go.mod file.
86+
# Update this command when ginkgo version is updated in go.mod file.
87+
# example: go list -m -f '{{.Version}}' github.com/onsi/ginkgo/v3
88+
GINKGO_VERSION := $(shell go list -m -f '{{.Version}}' github.com/onsi/ginkgo/v2)
89+
90+
# XKS_LABEL_FILTER is the label filter for XKS tests.
91+
XKS_LABEL_FILTER ?= "!openshift"
92+
8493
.PHONY: all
8594
all: build
8695

@@ -172,6 +181,16 @@ e2e-tests-sequential:
172181
e2e-tests-parallel:
173182
CI=prow make e2e-tests-parallel-ginkgo
174183

184+
.PHONY: e2e-xks-tests-sequential-ginkgo
185+
e2e-xks-tests-sequential-ginkgo: ginkgo ## Runs Ginkgo e2e sequential tests
186+
@echo "Running GitOps Operator sequential Ginkgo E2E tests..."
187+
$(GINKGO_CLI) -v --trace --label-filter=$(XKS_LABEL_FILTER) --timeout 240m -r ./test/openshift/e2e/ginkgo/sequential
188+
189+
.PHONY: e2e-xks-tests-parallel-ginkgo ## Runs Ginkgo e2e parallel tests, (Defaults to 5 runs at a time)
190+
e2e-xks-tests-parallel-ginkgo: ginkgo
191+
@echo "Running GitOps Operator parallel Ginkgo E2E tests..."
192+
$(GINKGO_CLI) -p -v -procs=5 --trace --label-filter=$(XKS_LABEL_FILTER) --timeout 60m -r ./test/openshift/e2e/ginkgo/parallel
193+
175194
##@ Build
176195

177196
.PHONY: build
@@ -247,7 +266,7 @@ kustomize: ## Download kustomize locally if necessary.
247266
GINKGO_CLI = $(shell pwd)/bin/ginkgo
248267
.PHONY: ginkgo
249268
ginkgo: ## Download ginkgo locally if necessary.
250-
$(call go-get-tool,$(GINKGO_CLI),github.com/onsi/ginkgo/v2/ginkgo@v2.29.0)
269+
$(call go-get-tool,$(GINKGO_CLI),github.com/onsi/ginkgo/v2/ginkgo@$(GINKGO_VERSION))
251270

252271

253272
# go-get-tool will 'go install' any package $2 and install it to $1.

‎test/e2e/argocd_route_test.go‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,15 +33,15 @@ var _ = Describe("Argo CD ConsoleLink controller", func() {
3333
route := &routev1.Route{}
3434
consoleLink := &console.ConsoleLink{}
3535

36-
It("Argocd route is present", func() {
36+
It("Argocd route is present", Label("openshift"), func() {
3737
checkIfPresent(types.NamespacedName{Name: argoCDRouteName, Namespace: argoCDNamespace}, route)
3838
})
3939

40-
It("ConsoleLink is created", func() {
40+
It("ConsoleLink is created", Label("openshift"), func() {
4141
checkIfPresent(types.NamespacedName{Name: consoleLinkName}, consoleLink)
4242
})
4343

44-
It("ConsoleLink and argocd route should match", func() {
44+
It("ConsoleLink and argocd route should match", Label("openshift"), func() {
4545
Eventually(func() error {
4646
err := k8sClient.Get(context.TODO(), types.NamespacedName{Name: consoleLinkName}, consoleLink)
4747
if err != nil {

‎test/kind-config.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
kind: Cluster
2+
apiVersion: kind.x-k8s.io/v1alpha4
3+
nodes:
4+
- role: control-plane
5+
kubeadmConfigPatches:
6+
- |
7+
kind: InitConfiguration
8+
nodeRegistration:
9+
taints: []
10+
- role: worker
11+
- role: worker

‎test/openshift/e2e/README.md‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,50 @@ SKIP_HA_TESTS=true LOCAL_RUN=true make e2e-tests-sequential-ginkgo
9494

9595

9696

97+
## Test Labels
98+
99+
Ginkgo [labels](https://onsi.github.io/ginkgo/#spec-labels) are used to categorize tests that are only supported on specific platforms or configurations. Labels allow the CI system to filter tests using `--label-filter` at runtime.
100+
101+
**Important**: Labels should only be applied to tests that are restricted to a specific platform or configuration. Do NOT add labels to generic tests that can run on any Kubernetes cluster (OpenShift, KinD, EKS, etc.). If a test works everywhere, it should remain unlabelled.
102+
103+
### Available labels
104+
105+
| Label | Meaning | When to use |
106+
|-------|---------|-------------|
107+
| `openshfit` | Test requires OpenShift-specific features (Routes, ConsoleLinks, OLM Subscriptions, SCCs, etc.) and cannot run on vanilla Kubernetes (KinD, EKS, GKE, AKS). | Add when the test depends on OpenShift APIs or resources not available on non-OCP clusters. |
108+
| `HA` | Test requires a cluster with at least 3 worker nodes for HA (High Availability) validation. | Add when the test creates HA Redis StatefulSets or validates pod anti-affinity across nodes. |
109+
110+
### How labels are used in CI
111+
112+
- **OpenShift CI**: Runs all tests (no label filter), so all labelled and unlabelled tests execute.
113+
- **xKS/KinD CI** (`make e2e-xks-tests-sequential-ginkgo` / `make e2e-xks-tests-parallel-ginkgo`): Uses `--label-filter="!openshfit"` to exclude OpenShift-only tests. The `XKS_LABEL_FILTER` env var can further customize filtering (e.g., `XKS_LABEL_FILTER="!openshfit && !HA"`).
114+
115+
### How to apply a label
116+
117+
Add the `Label(...)` decorator to `It`, `Context`, or `Describe` nodes:
118+
119+
```go
120+
// Single label — test requires OpenShift
121+
It("verifies ConsoleLink is created for ArgoCD route", Label("openshift"), func() {
122+
// ...
123+
})
124+
125+
// Label on Context — all specs inside inherit the label
126+
Context("1-120_repo_server_system_ca_trust", Label("openshift"), func() {
127+
It("ensures that missing Secret aborts startup", func() {
128+
// ...
129+
})
130+
})
131+
```
132+
133+
### Guidelines
134+
135+
- If your test uses OpenShift-specific resources (Routes, ConsoleLinks, SCCs, Subscriptions, CSVs, monitoring via prometheus-operator CRDs that are only present on OCP, etc.), add `Label("openshift")`.
136+
- If your test requires 3+ nodes for HA validation, add `Label("HA")`.
137+
- If your test works on any conformant Kubernetes cluster, do NOT add any label.
138+
- When in doubt, leave the test unlabelled — it will run everywhere, which provides broader coverage.
139+
140+
97141
## Test Code
98142

99143
gitops-operator E2E tests are defined within `test/openshift/e2e/ginkgo`.

0 commit comments

Comments
 (0)