Repository navigation
Export a Cyd Bluesky archive people can actually be handed - #117
Merged
Merged
Conversation
#98 built the writer; this is the flow around it. Four things it adds. **It survives being killed.** Export staging now holds a checkpoint beside the files it describes: the moment the archive is of, the media inventory taken at that moment, and the digest of every file already read. Reusing an export id picks that up, so an export the operating system reclaims does not pay for the snapshot or the hashing pass twice — and, more to the point, comes back as the same export. Taking the snapshot again would have been easier and would have quietly moved the archive's point in time past whatever saving did in between (ADR 0006, ADR 0010). Translating and packaging are rebuilt from the staged snapshot each attempt, which is cheap and leaves no half-written archive; an export killed after packaging keeps the archive it finished and offers that again rather than building a second identical one. **It finishes around a file that moved.** Packaging is the last pass that can catch a file the inventory and the hashing both accepted, and it used to fail the whole export. A resumed export would then have reread the same stale digest and failed identically forever. It is now recorded as changed, demoted to unavailable, and the archive is rebuilt around it — an honest incomplete archive rather than a permanent failure (ADR 0019). **It says what the file is.** A Cyd Bluesky archive is plaintext, and the posts, chats and media in it are as sensitive as they ever were (ADR 0013). The flow asks before anything is written, not once the file is already in somebody's downloads, and hands the archive to the share sheet — hence expo-sharing, whose config plugin is deliberately *not* installed: that one builds an iOS share extension for receiving shares, which is not what this is. **It clears up after itself.** Staging is destroyed on delivery, on cancellation, and on a failure Cyd can name; anything else keeps the checkpoint, because the next launch can carry on from it. Declining the warning is not cancelling and keeps it too. An account keeps at most one staged export, so an abandoned run cannot leave a copy of the account database per attempt. Clearing staging is the export's own job on the way out of a cancellation, not the screen's: deleting the directory from under a pass that is still writing to it only means it gets written again. Hashing checkpoints in batches, because the checkpoint carries every digest so far and one write per file would rewrite the whole pass once per file. The export tests now assert the archive's whole entry set, so neither the snapshot nor the new checkpoint can drift into it; the account fixture moved to testUtils so the resume tests could share it. The entry point stays `__DEV__`, where #100 will find it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Export ended at a share sheet, and on Android that is `ACTION_SEND`: an intent that enumerates applications willing to *receive* content — cloud drives, mail, messengers — and never the device's own storage, because saving into a folder is `ACTION_OPEN_DOCUMENT_TREE` and a different intent entirely. So the one instruction the flow gives, to put the archive somewhere you trust like an encrypted drive, was the one thing an Android user could not do. Getting your own data out of Cyd required first giving it to a third party (ADR 0015), in plaintext (ADR 0013). Found on a device; the share sheet on a real export offered Drive and messaging apps and no local destination at all. Saving to the device is now its own delivery, on both platforms. It picks a folder with `Directory.pickDirectoryAsync` and copies into it with the native streaming `File.copy`, so an archive of preserved video costs what a thumbnail does — reading it into JavaScript to write it back out would not survive a real account. That needed the flow to stop delivering by itself. Packaging used to hand the archive straight to the share sheet and clear staging behind it; there is now a step where the finished archive exists and where it goes has not been decided. Staging is cleared when it has actually gone somewhere, which is what lets backing out of the folder picker return to the archive rather than destroy it: that cancellation is a real signal, and the staged copy is the only copy. A share sheet gives no such signal — expo-sharing discards the result code and resolves either way, and `Intent.createChooser` reports cancelled for most successful shares, so there is nothing to read. Sharing therefore still clears staging on the strength of the sheet having closed, unchanged from before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ADR 0019 said an export that found a preserved file changed between hashing and packaging had to demote that asset and rebuild the archive around it, because otherwise "a resumed export would reread the same stale digest and fail identically on every launch", leaving an account that could never export. That was never true. `asset-changed` is a BlueskyArchiveExportError, and every named error clears its own staging, so the next attempt already took a fresh snapshot, re-inventoried at the file's new size, and succeeded. The permanent failure the decision existed to prevent was not reachable in this code or in the code before it. What the retry loop really bought was the hashing pass, which is the most expensive thing an export does. That is worth something, but not its price: a rebuild of the interchange database and the whole archive for every digest that moved, where re-taking the inventory catches all of them at once. It was a loop, a checkpoint field, an error carrying an entry path, and a special case in the hashing pass, all for a case rarer than the one it made slower. So a file that moves while it is being packaged now fails the export and takes its staging with it, and the next attempt starts over. A file that moves before hashing reaches it is unchanged and still produces an archive: the point-in-time inventory catches it, and the asset is honestly unavailable. Both are now tested, which they were not as one path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things could put two export runs over a single staging directory, and nothing in the way of either. The first is a second tap. `confirm` read the generation without moving it on, so two taps landing before the screen caught up both passed the same check: two runs, one `exportId`, each taking its own snapshot and packaging into the other's files. The same shape was reachable from "Try again", and — since delivery became a state of its own — from "Save to device" and "Share". The second is slower to see. `cancel` moves the generation on and returns, but the run in flight only notices at its next boundary, and clearing its own staging is the last thing it does on the way out. Asking for another export inside that window found the dying run's checkpoint still on disk, so `claimBlueskyArchiveExport` handed back the very directory that was about to be deleted. What reached the person was a raw `unable to open database file` from part-way through translating, for having pressed a button twice. Nothing was lost — the account is never touched, and trying again worked — but nothing explained itself either. Both are one rule now: an export owns its staging directory for as long as it is running, and running outlasts the screen. `exclusively` will not start a second thing in there while the first is still going, and `start` waits for a departing run to finish clearing before it claims anything. Neither race is easy to hit by hand — the screen usually re-renders before a second tap lands — so both are held down at the hook's seam instead, where the interleaving can be pinned: a run parked mid-hash, cancelled, and asked to start again before it has let go. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An export of a real account showed one message for its whole run. "Pausing this account's work and copying its data…" stayed up for about seventy-five seconds and was then replaced by the finished archive. "Checking the preserved media…", "Writing the archive's database…" and "Packaging the archive…" were never drawn, no progress fraction ever appeared, and Cancel — which the card offers for the whole of hashing — did nothing whatever. A tap during translating was read fifty-six seconds later, by which point the export had finished and there was nothing left to cancel. Neither symptom is slowness in the ordinary sense. An export runs on the thread that draws the screen and delivers touches, and its long passes await work that has already finished: an already-resolved promise is drained inside the same tick, so React was told about progress it never got to commit, and the tap sat in the queue unread. `shouldCancel` was not being ignored — it was never asked a question anyone had had the chance to answer. Most of the wait was not the export thinking, either. `data.db` was written a row at a time with nothing around it, and outside a transaction SQLite commits after every statement and waits for the storage to say it is done. On this account that was most of a minute spent waiting on fsync. One transaction takes that write down to a few seconds and the export as a whole from about seventy-five seconds to about thirty. Foreign keys are still checked as each row lands, so the reference order the writer has always used is doing exactly what it always did; what is new is that a refused row takes the rest of the write with it, rather than leaving half an account in a file that is about to be packaged. The rest is giving way on a timer, which is the only thing that really hands the thread back, once a pass has held it for fifty milliseconds. Hashing and packaging both do it, and that is what makes their progress visible and their Cancel button real: cancelling mid-hash now clears its staging within a second, which is the first time that has been true on hardware. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #99.
#98 built the Cyd Bluesky archive v2 writer. This is the flow around it: resumable, cleaned up after itself, honest about what it produces, and clear about what the file is.
What changed
It survives being killed. Export staging holds a checkpoint beside the files it describes — the moment the archive is of, the media inventory taken at that moment, and the digest of every file already read. Reusing an export id picks that up, so an export the operating system reclaims comes back as the same export rather than a new one with a later point in time (ADR 0006, ADR 0010). Translating and packaging are rebuilt from the staged snapshot each attempt; an export killed after packaging keeps the archive it finished and offers that again rather than building a second identical one.
It fails an export whose file moved, rather than rebuilding around it. An earlier revision of this branch demoted such a file and rebuilt the archive around it, on the grounds that a resumed export would otherwise reread the same stale digest and fail identically forever. That turned out not to be true of this code:
asset-changedis a named error, every named error clears its own staging, and the next attempt therefore re-inventories at the file's new size and succeeds. The retry loop and its ADR were removed. A file that moves while it is being packaged now fails the export and takes its staging with it; a file that moves before hashing reaches it is caught by the point-in-time inventory and is honestly unavailable in a complete-enough archive.It says what the file is, and lets you keep it. A Cyd Bluesky archive is plaintext (ADR 0013), so the flow says so before anything is written. Delivery is then two things, not one: an Android share sheet is
ACTION_SENDand lists only applications that receive content, so "Save to device" is its own affordance on both platforms. Without it the one instruction this flow gives — put it somewhere you trust — is the one thing an Android user could not do.It starts one export however many times the button is pressed. Two taps landing before the screen caught up used to start two runs over a single staging directory, each snapshotting and packaging into the other's files. Separately,
cancelreturns before the run it cancelled has finished clearing up, and asking for another export inside that window handed the new one the dying run's directory — which was then deleted out from under it, surfacing a rawunable to open database file. An export now owns its staging directory for as long as it is running, and running outlasts the screen.The screen keeps up with it. An export runs on the thread that draws the screen, and its long passes await work that has already finished — so React was told about progress it never got to commit, and a tap on Cancel sat unread until the export was over. On a real account the card showed one message for seventy-five seconds and Cancel did nothing at all. Hashing and packaging now give way on a timer. Separately,
data.dbwas written a row at a time with no transaction around it, which outside one means SQLite committing and waiting on storage per statement; one transaction took the same export from about seventy-five seconds to about thirty.It clears up after itself. Staging goes on delivery, on cancellation, and on a failure Cyd can name. Anything else keeps the checkpoint, because the next attempt carries on from it — including declining the warning, which is not cancelling. An account keeps at most one staged export, and sweeps neither another account's staging nor anything it cannot read a checkpoint from.
Nothing here reaches for a Cyd account. Portability is not a premium feature (ADR 0015), and the card exporting successfully with no
CydAccountProviderabove it is what proves it.Acceptance criteria
Notes for review
expo-sharingis a new native dependency, so this needs a fresh dev build. Its config plugin is deliberately not inapp.config.ts: that plugin builds an iOS share extension for receiving shares, which is not what this uses.__DEV__. Per ADR 0004 the affordance is gated until Pass bidirectional Cyd Bluesky archive v2 conformance and enable the writer #100 proves conformance both ways; what Pass bidirectional Cyd Bluesky archive v2 conformance and enable the writer #100 changes is where this lives and what it is called, not what it does.share()clears staging on a sheet that cannot report success.expo-sharing's AndroidOnActivityResultdiscards the result code, andIntent.createChooserreturnsRESULT_CANCELEDeven on success, so dismissing the sheet deletes the archive. Left as-is deliberately; the alternative is keeping a copy of every account that ever opened one.CHECKPOINT_VERSIONwas not bumped when the checkpoint shape changed, because the removed field is only ever dropped and never reinterpreted. Verified on device: a checkpoint carrying the old field resumes in place and completes.testUtils/blueskyExportAccount.tsso the resume tests could share it.Testing
npm run test:archive-contractandnpm run test:archive-conformancegreen.PRAGMA integrity_checkok and no foreign-key violations; staging cleared on delivery, on cancel mid-hash, and on a named failure; two accounts not sweeping each other; three abandoned exports collapsing to the newest; junk staging swept; and the dev card confirmed absent from a real release build.(This was written by an LLM.)
🤖 Generated with Claude Code