Repository navigation
Stream backup encryption and authenticated restore (JVNAUTOSCI-2737) - #608
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The migration's existing encrypted database backup is 1.27 GB. Backup encryption and restore decryption previously loaded entire files into memory, exceeding the bounded resource budget needed to recover after the earlier machine crash.
Stream the existing Fernet format through private temporary files using Cryptography's AES-CBC, PKCS7 and HMAC primitives. Authenticate the whole ciphertext before decrypting, and publish completed output atomically. Invalid input and failed publication preserve any existing destination. Backup filenames, key format and retention timestamps remain compatible. This follows the Fernet format; the ordinary library's whole-message memory requirement explains the previous allocation.
Validation: 51 targeted backup, restore and encryption tests pass. Tests cover ordinary Fernet interoperability across AES/base64/chunk boundaries, wrong keys, tampering, malformed/truncated input, authenticated invalid padding, publication failure, destination preservation and no whole-file reads at the operator entrypoints. No new Ruff findings.
The actual 1,271,220,260-byte earlier backup decrypted successfully; all 94 ZIP members passed CRC checks. Streamed re-encryption and decryption reproduced its 953,415,126-byte plaintext SHA-256 exactly. The shared guard observed a peak of 0.234 GiB across the probe and active import workers; temporary plaintext was removed. This proves this file-codec path, not the still-pending fresh paired database/blob restore or overall cutover. Source captures and audit artefacts remain outside Git.
Merge decision: ready, subject to required CI. No change to task/project writer authority or running-system activation.