Skip to content

fix: prepare backup workers before quiesce and bound CSI captures - #772

Open
Baylem wants to merge 25 commits into
GameplanePanel:stack/pr-769from
Baylem:fix/audit-backup-budgets
Open

Baylem wants to merge 25 commits into
GameplanePanel:stack/pr-769from
Baylem:fix/audit-backup-budgets

Conversation

@Baylem

@Baylem Baylem commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Backup preparation could disable saving while a worker was still waiting for storage or repository access, and a stalled CSI capture could hold quiesce indefinitely. This prepares and verifies the real worker before save-off, then applies persisted deadlines to preparation, live copying and CSI capture.

  • Restic workers mount the source, initialize/check the repository and become Ready before quiesce. An owned projected ConfigMap authorizes only the exact Backup, worker Pod, source identity and expiry. Workers use resource defaults and prefer the source node, with an optional hard placement requirement. First binding waits for an initial source Pod to be created and scheduled within the preparation deadline; a replacement after binding still fails closed. They receive no Kubernetes API token.
  • Preparation, quiesce and CSI deadlines survive restarts and configuration changes. Copy failure/deletion revokes authorization and waits for actual worker drainage before save-on. Raw and already-stopped copies retain the full Job budget.
  • Aborted CSI snapshots remain invalid even if the driver later reports readiness. Cleanup verifies snapshot/content UIDs and respects Retain policy. Restore and new snapshot-backed PVCs reject incomplete, aborted or replaced captures.
  • Prepared workers record volume coverage before copy authorization and use the effective template under lifecycle ownership, including when a newer catalog revision is pending.

Includes controller, worker-script and snapshot regressions plus a dedicated two-node RWO preparation/restore CI drill. The Kind fixture does not claim qualification of attachable CSI drivers. Worker expiry assumes synchronized clocks; restoring saving after an operator outage still requires operator/agent recovery.

Dependencies included: #760 lifecycle exclusion, #766 stopped-server preparation, #767 capture metadata, #769 staged templates and its security/resource dependencies, #756 snapshot authorization, and #759 shared web build prerequisite. Review the commits for this issue group after those prerequisites. Shared restore cleanup retains ownership until failed startup drains, including force-deletion recovery. The preparation, placement and Restricted workload buckets remain registered in upstream change-based test selection.

Validation: production and regression-source compilation passed; required actionlint/offline zizmor and whitespace checks passed. Runtime unit, envtest and E2E execution remains in CI under the repository's compile-only local policy.

Closes #717.
Closes #719.

Issue coordination: #717 and #719 build on #718's exclusive ownership (#760) and #716's stopped-source path (#766).

The dependency stack now includes verified effective-template API redaction, clone password protection and diagnostics namespace isolation. Backup detail UI follows Pencil nodes zhLZN (achieved consistency and legacy volume coverage) and bHJy6 (recorded per-volume coverage), with saved JSON/PNG exports and screenshot coverage inherited from #769.

Final integrated API/operator builds and handler/controller regression-source compilation passed. Runtime and visual verification of this updated head remains pending CI.

Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions github-actions Bot added area: agent in-pod sidecar area: api REST/WebSocket gateway area: chart Helm chart and deployment area: e2e kind-based end-to-end suite area: modules game template OCI bundles area: operator controller-runtime operator and CRDs area: optional-components sentinel, capture-sidecar, mcp-server, tunnel, audit-syslog-bridge, telemetry-receiver area: shared netguard, gameaction, gameproto, svcutil and other shared packages area: web React dashboard type: fix Bug fix labels Oct 21, 2026
Baylem added 13 commits October 21, 2026 17:10
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
@Baylem
Baylem requested a review from ValgulNecron as a code owner October 24, 2026 15:07
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>
Signed-off-by: Baylem <zelathis@gmail.com>

Baylem commented Oct 31, 2026

Copy link
Copy Markdown
Member Author

Updated in 327163a with the legacy cleanup fix from #766 and stopped-capture artifact retry from #764, including their regressions. Also integrated upstream’s fixes for the operator lint error and the isolated frpc dependency flagged by the image scan; the compiled binary embeds go-jose 4.1.6. Source compilation and required workflow checks passed. Fresh CI is running for the combined head.

@ValgulNecron

Copy link
Copy Markdown
Collaborator

This pr is editing the UI without a pencil edit. please do the edit in pencil too

Signed-off-by: Baylem <zelathis@gmail.com>
@Baylem
Baylem changed the base branch from master to stack/pr-769 November 4, 2026 05:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: agent in-pod sidecar area: api REST/WebSocket gateway area: chart Helm chart and deployment area: e2e kind-based end-to-end suite area: modules game template OCI bundles area: operator controller-runtime operator and CRDs area: optional-components sentinel, capture-sidecar, mcp-server, tunnel, audit-syslog-bridge, telemetry-receiver area: shared netguard, gameaction, gameproto, svcutil and other shared packages area: web React dashboard type: fix Bug fix

Projects

None yet

2 participants