Skip to content

feat(api): the control plane can record an archive a data plane reports - #192

Merged
NathaelB merged 2 commits into
mainfrom
chantier/v12-04-backup-outcomes
Sep 12, 2026
Merged

feat(api): the control plane can record an archive a data plane reports#192
NathaelB merged 2 commits into
mainfrom
chantier/v12-04-backup-outcomes

Conversation

@NathaelB

@NathaelB NathaelB commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Refs #182. Part of #177. Does not close #182: this is the receiving half only.

Merged on its own on purpose. The contents are complete and green; what is missing is a decision, not code, and holding four commits against a moving main to wait for it costs more than it saves.

What is in it

The control plane can now receive and record an archive.

  • RecordArchiveCommand / RecordArchiveFailureCommand, and a BackupService that applies the rules.
  • find_by_object_key on the repository, which is what makes recording idempotent.
  • POST /dataplanes/{id}/deployments/{id}/archive, wired through the application layer.
  • openapi.json and the console client regenerated.

What is not in it, and why #182 stays open

The sending half. Genesis does not watch IdentityInstanceBackup yet, and Herald has no report_archive. So nothing calls this endpoint: an archive taken in the cluster still does not appear in the control plane, which is what #182's acceptance asks for.

The design is settled and written down, so this is a continuation rather than a decision:

  • Genesis's IdentityInstanceStatusWatcher gains a sibling watching IdentityInstanceBackup, publishing an archive report to the broker the way DeploymentOutcomeReport already goes. Genesis holds AMQP credentials and Herald holds control plane credentials, and this keeps it that way.
  • Herald gains report_archive on ControlPlaneRepository and forwards to the endpoint in this PR.
  • A gap worth naming: a scheduled archive produces a CloudNativePG Backup directly, with no IdentityInstanceBackup beside it. Either the operator adopts those into Aether resources, or Genesis watches CNPG's kind as well. Without one of the two, only manually requested archives are ever reported, which is the smaller half of the feature.

I stopped here rather than half-writing the transport, because the gap above is a product decision rather than an implementation one. The endpoint is the contract the sending side will target, and it is testable and reviewable on its own.

The rules this lands

Recording is idempotent on the object key. One object, one archive. Reports arrive at least once by design, so without this a redelivery is counted twice by retention and offered twice as a restore. The second report returns the first archive unchanged rather than failing, so a redelivery is not an error anybody has to handle.

An attempt that produced no archive produces no row. It becomes an audit entry and nothing else. This is the same rule #190 built the table around, now enforced on the way in.

Three facts come from the control plane, not from the report:

  • the release, because a data plane reporting a version is reporting what it read off a pod, and the two disagree for the length of an upgrade;
  • the organisation, which is not a data plane's to state;
  • the archive location, built through ArchivePrefix, so a report cannot address another tenant's archive by claiming a path. There is a test feeding it ../../someone-else/base/data.tar.gz.

Only Herald may report, and only about deployments on its own data plane. The same two rules claim, ack, heartbeat and outcome already carry. A caller able to forge a report could record an archive that does not exist, and the platform would offer it as a restore.

Smaller decisions

  • The size travels as a string. It passes through a signed 32 bit integer somewhere in every Kubernetes toolchain, and a 3 GB archive read back as a negative number is worse than one nobody can sum. A value that is not a number of bytes is refused.
  • Every missing field is named in the error. A field guessed at here becomes an archive the platform offers as a restore and cannot read, discovered during an outage.
  • One endpoint for both outcomes, told apart by which fields arrived rather than by a flag somebody can set inconsistently.
  • record_archive runs in one transaction with the lookup. Two reports arriving together would otherwise both find nothing and both insert; the unique index catches it, but as a database error rather than as the no-op a redelivery is.

Verification

cargo clippy --workspace --all-targets    clean
cargo nextest run --workspace             988 passed, 1 skipped

Eight unit tests on the service cover the idempotency, the audit rule, both authorization rules, the zero byte refusal, the backwards timestamps and the traversal attempt. Three more on the request cover the parsing.

@NathaelB NathaelB added the enhancement New feature or request label Sep 12, 2026
@NathaelB NathaelB self-assigned this Sep 12, 2026
@NathaelB
NathaelB force-pushed the chantier/v12-03-operator-cnpg branch from 5e4ae47 to df980ef Compare September 12, 2026 03:56
@NathaelB
NathaelB force-pushed the chantier/v12-04-backup-outcomes branch from 574e9a6 to 7d97b67 Compare September 12, 2026 03:56
@NathaelB
NathaelB force-pushed the chantier/v12-03-operator-cnpg branch from df980ef to d038742 Compare September 12, 2026 10:14
@NathaelB
NathaelB force-pushed the chantier/v12-04-backup-outcomes branch from 7d97b67 to 480b8d5 Compare September 12, 2026 10:17
@NathaelB
NathaelB force-pushed the chantier/v12-03-operator-cnpg branch from d038742 to 52a386d Compare September 12, 2026 10:27
@NathaelB
NathaelB force-pushed the chantier/v12-04-backup-outcomes branch from 480b8d5 to 7ad68a7 Compare September 12, 2026 10:27
@NathaelB
NathaelB force-pushed the chantier/v12-03-operator-cnpg branch from 52a386d to 9d9136c Compare September 12, 2026 10:36
Base automatically changed from chantier/v12-03-operator-cnpg to main September 12, 2026 10:36
Reports arrive at least once by design, so recording is idempotent on the
object key: one object, one archive. Without it a redelivery is counted twice
by retention and offered twice as a restore, and the second report returns the
first archive unchanged rather than failing, so a redelivery is not an error
anybody has to handle.

An attempt that produced no archive produces no row. It becomes an audit entry
and nothing else, because there is no such thing as a backup that failed, only
an attempt that was made.

Three things come from what the control plane already knows rather than from
the report. The release, because a data plane reporting a version is reporting
what it read off a pod and the two disagree for the length of an upgrade. The
organisation, because that is not a data plane's to state. And the archive
location, built through the prefix, so a report cannot address another tenant's
archive by claiming a path.

Only herald may report, and only about deployments that run on its own data
plane. The same two rules claim, ack, heartbeat and outcome already carry: a
caller able to forge a report could record an archive that does not exist, and
the platform would offer it as a restore.
One endpoint for both outcomes of one attempt, told apart by which fields
arrived rather than by a flag somebody can set inconsistently.

Every missing field is named rather than defaulted. A field guessed at here
becomes an archive the platform offers as a restore and cannot actually read,
and that is discovered during an outage.

The size travels as a string. It passes through a signed 32 bit integer
somewhere in every Kubernetes toolchain, and a 3 GB archive read back as a
negative number is worse than one nobody can sum.

openapi.json and the console client are regenerated, not edited.
@NathaelB
NathaelB force-pushed the chantier/v12-04-backup-outcomes branch from 7ad68a7 to 1051d72 Compare September 12, 2026 12:12
@NathaelB
NathaelB marked this pull request as ready for review September 12, 2026 12:12
@NathaelB
NathaelB merged commit cbea508 into main Sep 12, 2026
18 checks passed
@NathaelB
NathaelB deleted the chantier/v12-04-backup-outcomes branch September 12, 2026 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

V4: backup outcomes reach the control plane

1 participant