fix(operator): exclude conflicting backup, restore, wipe, sleep, and rollout operations #718

Backup ownership currently coordinates quiesced backups with other quiesced backups. It does not provide shared exclusion against restart, idle sleep, template rollout, or newly created restore/wipe work. Extend lifecycle ownership to cover the source data and workload throughout a copy, including backups with quiesce=false.

Evidence comes from static inspection of master 04e41075cafc4c1a99bcb3723b0a8745ee810957; the interleavings below are source-derived scenarios, not runtime reproductions.

Triggers and impact

During a long quiesced backup, idle sleep or a restart can stop the source game. Its final save may modify files while Restic reads them. A template update can replace the pod with a fresh process that never received the original quiesce request. The backup may fail or lose the application-consistency guarantee.

Restore already checks existing pods and Jobs using the PVC. However, an unquiesced Backup can create a new reader after that check because it acquires no backup guard and does not honor restore ownership. That copy can capture a partially restored or wiped filesystem. Scanning current users cannot prevent another reconciler from creating work after the scan.

Pinned evidence

• Unquiesced backups skip target acquisition; protected acquisition checks the backup guard.
• Idle busy state checks restart/wipe request tokens.
• Replica decisions honor wipe and restore guards, then stop/restart/idle state.
• Restore checks existing PVC users and writes a workload fence.
• Template reconciliation replaces pod containers.

Recommended implementation

Create one durable lifecycle ownership record bound to the GameServer UID, operation UID, and operation generation. Use an uncached read plus resourceVersion compare-and-swap for acquisition. Start with exclusive ownership for backup/restore/wipe to keep the compatibility rules reviewable; add compatible reader modes only with explicit invariants.

Require all Backup variants to claim ownership before creating a Job or VolumeSnapshot. Restore and wipe must acquire the same record. Make idle transitions, restart commands, and pod-template rollout defer while a protected copy owns the target. Fence the StatefulSet mutation path so a reconcile computed before acquisition cannot scale or roll a writer afterward.

Persist completion/abort state before releasing ownership. Recover orphan owners by UID, cancel and drain their workers, and finish any quiesce cleanup before admitting another operation. A force-delete must not turn into permission to ignore live workers. Keep per-server serialization so independent servers remain concurrent.

Acceptance criteria

• CI exercises backup versus restore, wipe, idle sleep, restart, and template rollout, with both quiesce settings.
• A new Backup created after restore's initial user scan cannot start until restore releases ownership.
• Stale reconciles, simultaneous claims, operator restarts, pod replacements, and deleted owners preserve exclusion.
• Failure/cancellation leaves the correct suspend intent and never releases a target with authorized writers still running.
• Existing duplicate-backup and restore/wipe serialization cases remain valid.

This extends #590, whose description separates its backup work from #585's restore/wipe guards. Preserve those protections and #454's cleanup retries.