Skip to content

Commit c92ef43

Browse files
lmicciniclaude
lmiccini
and
claude
committed
Add transport URL secret rotation on top of applied-input correlation
Builds on "Correlate Watcher readiness with applied input" (#434), which adds AppliedInputSecretHash to the sub-CRs and releases the application- credential consumer finalizer only once every sub-service has provably rolled out. This commit extends the same guarantee to the RabbitMQ transport URL and notification transport URL secrets, so infra-operator cannot revoke a transport credential still in use by running pods. Changes: 1. Add a openstack.org/watcher-transport-consumer finalizer to the current transport URL secret (and the notification transport URL secret when notifications are enabled) early in reconcile, via object.ManageSecretConsumerFinalizer. Status.TransportURLSecret / Status.NotificationsTransportURLSecret are set for first-time setup only; during rotation they are updated solely by FinalizeSecretRotation. 2. At the end of reconcile, release the old secret's finalizer with object.FinalizeSecretRotation, guarded by the existing allServicesReady correlation (Generation == ObservedGeneration && AppliedInputSecretHash matches && sub-CR ready). The transport and AC secrets share the same guard, so nothing is released until every child is rolled out. 3. When notifications are disabled, remove the finalizer from the tracked notification secret and clear the status. 4. Use the lib-common statefulset.IsReadyForInput helper in the sub-CR controllers instead of the local statefulSetReadyForInput, and pin lib-common to the fork commit carrying IsReadyForInput / FinalizeSecretRotation while the lib-common PR is in review. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 6c158b4 commit c92ef43

14 files changed

Lines changed: 223 additions & 82 deletions

‎api/bases/watcher.openstack.org_watchers.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -801,6 +801,13 @@ spec:
801801
type: string
802802
description: Map of hashes to track e.g. job status
803803
type: object
804+
notificationsTransportURLSecret:
805+
description: |-
806+
NotificationsTransportURLSecret - the notifications transport URL secret
807+
watcher is currently consuming and protecting with the
808+
openstack.org/watcher-transport-consumer finalizer. Tracked so the
809+
controller can remove its finalizer from the old secret on rotation.
810+
type: string
804811
observedGeneration:
805812
description: |-
806813
ObservedGeneration - the most recent generation observed for this
@@ -813,6 +820,13 @@ spec:
813820
description: ServiceID - The ID of the watcher service registered
814821
in keystone
815822
type: string
823+
transportURLSecret:
824+
description: |-
825+
TransportURLSecret - the transport URL secret watcher is currently
826+
consuming and protecting with the openstack.org/watcher-transport-consumer
827+
finalizer. Tracked so the controller can remove its finalizer from the
828+
old secret when the transport URL is rotated.
829+
type: string
816830
type: object
817831
type: object
818832
served: true

‎api/go.mod‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ module github.com/openstack-k8s-operators/watcher-operator/api
33
go 1.26.3
44

55
require (
6-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260811190835-054aa14112d8
7-
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260815052137-cfc51390472e
6+
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260820165931-18e54edd62e6
7+
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260824142420-113dc64ecfd4
88
k8s.io/api v0.33.13
99
k8s.io/apimachinery v0.33.13
1010
k8s.io/utils v0.0.0-20250820121507-0af2bda4dd1d

‎api/go.sum‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,10 @@ github.com/onsi/ginkgo/v2 v2.32.1 h1:6tlvcDm/3sE8lGJbZ4+d4mO3RLy24/tQWOFzVSQNIfw
7676
github.com/onsi/ginkgo/v2 v2.32.1/go.mod h1:+aXOY+vzZ5mu2iI2HpTZUPmM//oQfsNFX6gU9kNcA44=
7777
github.com/onsi/gomega v1.42.1 h1:iN1rCUX+44NZ1Dc97MPoeFYbFR0vh8zxoxMFwKdyZ6I=
7878
github.com/onsi/gomega v1.42.1/go.mod h1:REff/hsDsodHoKlWsP2mAPhu1+5/6hVYNf9rIEBpeSg=
79-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260811190835-054aa14112d8 h1:V+M1QXCXaYlrgMl863bjZWpCA0grSuePt2QjZ5QX6G0=
80-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260811190835-054aa14112d8/go.mod h1:nPqCNb6LtP2HIkAoOrM4Jr4BMdNnuY0puKx6v6fwOhI=
81-
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260815052137-cfc51390472e h1:+49VTYcsFatI4C5ZHn6UQvMwq26LWVPqR0vkGfgfFoY=
82-
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260815052137-cfc51390472e/go.mod h1:vDcTSyaI7R6tjtZOqonr0hzq8JMOfBhJCWZy0tVjQPw=
79+
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260820165931-18e54edd62e6 h1:tpEqoZd8Coet0tobEG19DPWgv8lfeqea8yhURk+A+e4=
80+
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260820165931-18e54edd62e6/go.mod h1:2nZqLpDUKJR2HzPJPuWGGiXS/nH4PPBwwTuNeKY7QT0=
81+
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260824142420-113dc64ecfd4 h1:zDxe4n0GPQ+U8VFZyyCoQed+9Xauz4G22J/X6B4gq20=
82+
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260824142420-113dc64ecfd4/go.mod h1:vDcTSyaI7R6tjtZOqonr0hzq8JMOfBhJCWZy0tVjQPw=
8383
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
8484
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
8585
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=

‎api/v1beta1/watcher_types.go‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,18 @@ type WatcherStatus struct {
6767
// finalizer. Tracked so the controller can remove its finalizer from the
6868
// old secret when the openstack-operator rotates the reference.
6969
ApplicationCredentialSecret string `json:"applicationCredentialSecret,omitempty"`
70+
71+
// TransportURLSecret - the transport URL secret watcher is currently
72+
// consuming and protecting with the openstack.org/watcher-transport-consumer
73+
// finalizer. Tracked so the controller can remove its finalizer from the
74+
// old secret when the transport URL is rotated.
75+
TransportURLSecret string `json:"transportURLSecret,omitempty"`
76+
77+
// NotificationsTransportURLSecret - the notifications transport URL secret
78+
// watcher is currently consuming and protecting with the
79+
// openstack.org/watcher-transport-consumer finalizer. Tracked so the
80+
// controller can remove its finalizer from the old secret on rotation.
81+
NotificationsTransportURLSecret string `json:"notificationsTransportURLSecret,omitempty"`
7082
}
7183

7284
// WatcherDBPurge defines the parameters for the Watcher database purging cron job

‎config/crd/bases/watcher.openstack.org_watchers.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -801,6 +801,13 @@ spec:
801801
type: string
802802
description: Map of hashes to track e.g. job status
803803
type: object
804+
notificationsTransportURLSecret:
805+
description: |-
806+
NotificationsTransportURLSecret - the notifications transport URL secret
807+
watcher is currently consuming and protecting with the
808+
openstack.org/watcher-transport-consumer finalizer. Tracked so the
809+
controller can remove its finalizer from the old secret on rotation.
810+
type: string
804811
observedGeneration:
805812
description: |-
806813
ObservedGeneration - the most recent generation observed for this
@@ -813,6 +820,13 @@ spec:
813820
description: ServiceID - The ID of the watcher service registered
814821
in keystone
815822
type: string
823+
transportURLSecret:
824+
description: |-
825+
TransportURLSecret - the transport URL secret watcher is currently
826+
consuming and protecting with the openstack.org/watcher-transport-consumer
827+
finalizer. Tracked so the controller can remove its finalizer from the
828+
old secret when the transport URL is rotated.
829+
type: string
816830
type: object
817831
type: object
818832
served: true

‎go.mod‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@ require (
88
github.com/onsi/ginkgo/v2 v2.32.1
99
github.com/onsi/gomega v1.42.1
1010
github.com/openshift/api v3.9.0+incompatible
11-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260811190835-054aa14112d8
12-
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260813151745-0d7ef88e4045
13-
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260815052137-cfc51390472e
14-
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20260815052137-cfc51390472e
15-
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260813083726-eee3e1c5aa5c
11+
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260820165931-18e54edd62e6
12+
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260822072640-7bd662ad0284
13+
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260824142420-113dc64ecfd4
14+
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20260824142420-113dc64ecfd4
15+
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260822071633-1e127e728e62
1616
github.com/openstack-k8s-operators/watcher-operator/api v0.0.0-00010101000000-000000000000
1717
go.uber.org/zap v1.28.0
1818
gopkg.in/yaml.v3 v3.0.1
@@ -59,8 +59,8 @@ require (
5959
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
6060
github.com/modern-go/reflect2 v1.0.3-0.20250322232337-35a7c28c31ee // indirect
6161
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
62-
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20260806140515-16161b7b6b53 // indirect
63-
github.com/openstack-k8s-operators/lib-common/modules/storage v0.6.1-0.20260806140515-16161b7b6b53 // indirect
62+
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20260824142420-113dc64ecfd4 // indirect
63+
github.com/openstack-k8s-operators/lib-common/modules/storage v0.6.1-0.20260824142420-113dc64ecfd4 // indirect
6464
github.com/pkg/errors v0.9.1 // indirect
6565
github.com/prometheus/client_golang v1.22.0 // indirect
6666
github.com/prometheus/client_model v0.6.2 // indirect

‎go.sum‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -116,20 +116,20 @@ github.com/onsi/gomega v1.42.1 h1:iN1rCUX+44NZ1Dc97MPoeFYbFR0vh8zxoxMFwKdyZ6I=
116116
github.com/onsi/gomega v1.42.1/go.mod h1:REff/hsDsodHoKlWsP2mAPhu1+5/6hVYNf9rIEBpeSg=
117117
github.com/openshift/api v0.0.0-20260710141509-36dec0bfafe4 h1:YVY7Mb0AdJBbNgC/Zz9R/ZwhixOmZNA8P+6ZPaDW/0M=
118118
github.com/openshift/api v0.0.0-20260710141509-36dec0bfafe4/go.mod h1:SPLf21TYPipzCO67BURkCfK6dcIIxx0oNRVWaOyRcXM=
119-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260811190835-054aa14112d8 h1:V+M1QXCXaYlrgMl863bjZWpCA0grSuePt2QjZ5QX6G0=
120-
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260811190835-054aa14112d8/go.mod h1:nPqCNb6LtP2HIkAoOrM4Jr4BMdNnuY0puKx6v6fwOhI=
121-
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260813151745-0d7ef88e4045 h1:WeWvFRHMGvLi3bYdXeOqEXt5RVAzvSyEMQgdAGlixPo=
122-
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260813151745-0d7ef88e4045/go.mod h1:eSJVec0eCTSudlsQ5+bNMHmn9aI1af2zNwMGStuzNOw=
123-
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260815052137-cfc51390472e h1:+49VTYcsFatI4C5ZHn6UQvMwq26LWVPqR0vkGfgfFoY=
124-
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260815052137-cfc51390472e/go.mod h1:vDcTSyaI7R6tjtZOqonr0hzq8JMOfBhJCWZy0tVjQPw=
125-
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20260806140515-16161b7b6b53 h1:0T92TYAzi40pa0G7N0/IukeA4caLLYMcKmpfEp3PKj4=
126-
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20260806140515-16161b7b6b53/go.mod h1:Q5wcK4PnlDJ55H+M38CFNLzL53OHYiwq81mR/gKtJXs=
127-
github.com/openstack-k8s-operators/lib-common/modules/storage v0.6.1-0.20260806140515-16161b7b6b53 h1:qgDk53Swas5I+wEkCK3V1yfNc/S8Xa/OeKIWAcAyejw=
128-
github.com/openstack-k8s-operators/lib-common/modules/storage v0.6.1-0.20260806140515-16161b7b6b53/go.mod h1:fJ9CCRKpo8mQODfv4UDJ3tjUTbyCznYslXQwCkDaGg0=
129-
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20260815052137-cfc51390472e h1:C/apf8vQcjqKGzGzRTjDAdQldLTQ61RX9sQ6uf/0Wv0=
130-
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20260815052137-cfc51390472e/go.mod h1:tT5uXteekJ3lRgFO5/C3i6Q5A7dj40SDy6XbOJhnRuw=
131-
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260813083726-eee3e1c5aa5c h1:N7KthVNlHdLJ6FZwpYsdsOdomm8eX6IXprWFqW3e7mc=
132-
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260813083726-eee3e1c5aa5c/go.mod h1:6T332Zk4/uC+ac27ZzQ7oslcePpk0PTcrFtoRohPxjg=
119+
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260820165931-18e54edd62e6 h1:tpEqoZd8Coet0tobEG19DPWgv8lfeqea8yhURk+A+e4=
120+
github.com/openstack-k8s-operators/infra-operator/apis v0.6.1-0.20260820165931-18e54edd62e6/go.mod h1:2nZqLpDUKJR2HzPJPuWGGiXS/nH4PPBwwTuNeKY7QT0=
121+
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260822072640-7bd662ad0284 h1:ERZkiJP+trC/2tiWZ7cVcQSFc3K16mhdVlV+nVy4Vhk=
122+
github.com/openstack-k8s-operators/keystone-operator/api v0.6.1-0.20260822072640-7bd662ad0284/go.mod h1:UCwCYAGhumXDqnViXuvwd6N+BC4fE5KTpbcJq+PFd40=
123+
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260824142420-113dc64ecfd4 h1:zDxe4n0GPQ+U8VFZyyCoQed+9Xauz4G22J/X6B4gq20=
124+
github.com/openstack-k8s-operators/lib-common/modules/common v0.6.1-0.20260824142420-113dc64ecfd4/go.mod h1:vDcTSyaI7R6tjtZOqonr0hzq8JMOfBhJCWZy0tVjQPw=
125+
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20260818072803-e18950de3098 h1:XF46E0eH0F91XaC6UIQqiuD5q+i2zPM8OWqJatRgap4=
126+
github.com/openstack-k8s-operators/lib-common/modules/openstack v0.6.1-0.20260818072803-e18950de3098/go.mod h1:Q5wcK4PnlDJ55H+M38CFNLzL53OHYiwq81mR/gKtJXs=
127+
github.com/openstack-k8s-operators/lib-common/modules/storage v0.6.1-0.20260818072803-e18950de3098 h1:1yHjBoRwSKTHVDDhgPdAPpBF9kSzw/ued/X+5KCkHyM=
128+
github.com/openstack-k8s-operators/lib-common/modules/storage v0.6.1-0.20260818072803-e18950de3098/go.mod h1:fJ9CCRKpo8mQODfv4UDJ3tjUTbyCznYslXQwCkDaGg0=
129+
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20260824142420-113dc64ecfd4 h1:n1ve1cxcG8ZAXCHb8SuNOA/+hXm+GYOl2Wp6oig6Mbo=
130+
github.com/openstack-k8s-operators/lib-common/modules/test v0.6.1-0.20260824142420-113dc64ecfd4/go.mod h1:tT5uXteekJ3lRgFO5/C3i6Q5A7dj40SDy6XbOJhnRuw=
131+
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260822071633-1e127e728e62 h1:kPodlXj+hrF6Q3m0VpEz9gBU8QcQXXuRkQbTcDwI00E=
132+
github.com/openstack-k8s-operators/mariadb-operator/api v0.6.1-0.20260822071633-1e127e728e62/go.mod h1:A1yXU4F8Qs+F29KcgaI3zBvPJjY3OUzlCt7QCY73wAI=
133133
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
134134
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
135135
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=

‎internal/controller/watcher_common.go‎

Lines changed: 0 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,6 @@ import (
88
"time"
99

1010
"github.com/go-logr/logr"
11-
appsv1 "k8s.io/api/apps/v1"
1211
corev1 "k8s.io/api/core/v1"
1312
k8s_errors "k8s.io/apimachinery/pkg/api/errors"
1413
"k8s.io/apimachinery/pkg/runtime"
@@ -187,37 +186,6 @@ func (r *ReconcilerBase) SetRequeueTimeout(timeout time.Duration) {
187186
r.RequeueTimeout = timeout
188187
}
189188

190-
// statefulSetReadyForInput verifies readiness against an uncached StatefulSet
191-
// and correlates it with the input hash in the pod template.
192-
func (r *ReconcilerBase) statefulSetReadyForInput(
193-
ctx context.Context,
194-
name types.NamespacedName,
195-
inputHash string,
196-
) (bool, error) {
197-
statefulSet := &appsv1.StatefulSet{}
198-
if err := r.APIReader.Get(ctx, name, statefulSet); err != nil {
199-
return false, err
200-
}
201-
202-
if statefulSet.Spec.Replicas == nil ||
203-
*statefulSet.Spec.Replicas != statefulSet.Status.ReadyReplicas ||
204-
*statefulSet.Spec.Replicas != statefulSet.Status.UpdatedReplicas ||
205-
statefulSet.Generation != statefulSet.Status.ObservedGeneration ||
206-
statefulSet.Status.CurrentRevision != statefulSet.Status.UpdateRevision {
207-
return false, nil
208-
}
209-
210-
for _, container := range statefulSet.Spec.Template.Spec.Containers {
211-
for _, envVar := range container.Env {
212-
if envVar.Name == "CONFIG_HASH" && envVar.Value == inputHash {
213-
return true, nil
214-
}
215-
}
216-
}
217-
218-
return false, nil
219-
}
220-
221189
// Reconcilers holds all the Reconciler objects of the watcher-operator to
222190
// allow generic management of them.
223191
type Reconcilers struct {

0 commit comments

Comments
 (0)