Repository navigation
Gdrive time machine - #2
Merged
Merged
Conversation
Time Machine writes to a locally attached sparsebundle whose bands live on Google Drive behind an rclone mount with a 100 GB write-back cache. No network filesystem sits in the write path, so a dropped link stalls the drain instead of interrupting the backup. Bands are 64 MB rather than the 8 MB default. Drive allows roughly two operations per file per second and caps a drive at 400,000 files; for a 200 GB backup that is the difference between ~25,000 and ~3,200 files. The cost is that any change dirties a full 64 MB. Band size is fixed at image creation and cannot be changed later without starting over. Three rclone flags carry weight: --drive-use-trash=false keeps deleted bands from counting against quota, --drive-stop-on-upload-limit halts rather than spinning on 403 once the 750 GB daily cap is hit, and --drive-chunk-size matches the band size. The mount script excludes the cache directory from Time Machine; without that Time Machine would back up its own backup. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Band size drops from 64 MB to 32 MB. The original 64 MB was justified on the assumption that larger bands cost nothing; measurement says otherwise. Under the append pattern Time Machine actually produces, upload volume grows monotonically with band size — 768 MB at 64 MB bands versus 384 MB at 8 MB for the same 300 MB of real change. 32 MB is the smallest band at which the first upload stops being bound by Drive's per-file rate (6,400 files, ~0.9 h) and becomes bound by link bandwidth (~1.3 h). Below it the first backup drags; above it every increment pays more transfer for a benefit that is already gone. Under a scattered rewrite, by contrast, band size is irrelevant — nearly every band is dirtied whatever its size, so the whole image goes over the wire. That is the worst case, not the one this workload hits. poc-pullplug.sh pulls the backing store out from under the image mid-write, standing in for a dead rclone process or a crashed FUSE-T. Three rounds, no unrecoverable loss: the image passed fsck_apfs every time. Getting that answer took fixing two traps the test itself fell into, both now handled in the scripts: Stale devices. A forced detach can leave the image's device behind. Reattaching then yields a dead handle where fsck_apfs reports "failed to read container superblock" with an all-zero UUID — indistinguishable from a destroyed backup unless you check from a clean state. The first version of this test declared total loss three times running; the data was intact. cm_purge_stale_devices clears them before attaching. Orphaned mountpoints. An unclean detach leaves /Volumes/<name> behind, and it blocks reattachment with "no mountable file systems". The directory belongs to the user but sits in root-owned /Volumes, so rmdir is refused — a launchd agent running as the user cannot recover on its own and the backup stalls until someone intervenes. attach-image.sh now detects this and prints the exact command to run. Also adds install-rclone.sh, because the Homebrew rclone is built without mount support and refuses outright, and verify-image.sh, because sparsebundles carry no checksum and hdiutil verify cannot check them — fsck_apfs on the attached device is the right tool. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The buffer agent runs under KeepAlive, so any startup failure repeats every 30 seconds forever. mount-drive.sh now checks for FUSE before launching rclone — without it rclone exits immediately with "cgofuse: cannot find FUSE" and the log grows without bound. This project has already lost 3.3 GiB to a runaway log once, so the rclone log is also trimmed at startup past 100 MB. poc-launchd.sh substitutes a stub for rclone and exercises what the cloud layer does not affect: that the plists load, that the attach agent waits for the buffer instead of racing it, and that KeepAlive brings the buffer back after a kill -9. All three pass. It uses its own poc-* labels so it cannot disturb the production agents, and tears itself down. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
…f sight
Image creation and attachment now wait for rclone's upload queue to drain and
retry up to five times. FUSE-T mounts over NFS, and hdiutil against such a
volume is intermittently rejected with "RPC version wrong". The failure does
not depend on image size or content — one sweep failed at 100 GB and 400 GB
while succeeding at 600 GB, 1000 GB and 1500 GB — but on timing: five of five
attempts succeed with an idle queue, and it turns random while rclone is
uploading or deleting. Creating the production image proved the point, failing
once and succeeding on the retry.
The image must be created in place on the mounted Drive. Creating it locally
and moving it produces a bundle hdiutil will not open afterwards
("CBSDBackingStore::newProbe stat() failed") even though every file and band is
present and readable. Neither alternative FUSE-T backend helps: fskit does not
mount at all, smb mounts but hdiutil create fails with "Is a directory".
Two secret leaks closed in setup-remote.sh. Passing client_secret= on rclone's
command line exposed it in ps for the entire time the browser consent was
pending — minutes, not milliseconds; it now goes through
RCLONE_DRIVE_CLIENT_SECRET. And rclone prints the finished remote config,
refresh token included, to stdout on success; that output is now discarded,
since a refresh token grants lasting access to the whole Drive and has no
business in a terminal, a log, or a session transcript. Errors still surface —
they go to stderr.
Also fixes the FUSE preflight added in the previous commit, which used `ls` on
several candidate paths and so failed whenever ANY of them was missing rather
than all — it refused to start on a correctly installed FUSE-T. And status.sh
passed --no-auth to `rclone rc`, which is a server-side flag the client does
not accept.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Time Machine writes to the attached image at SSD speed while rclone uploads at link speed; the difference accumulates in the buffer. --vfs-cache-max-size is a soft limit — rclone only evicts what it has already uploaded, so when the whole cache is queued it keeps growing and can fill the disk. Measured at the start of the first backup: the buffer grew about 32 MB/s net, against 425 GB free and a source Time Machine estimates at 1.5 TB. buffer-guard.sh stops Time Machine above a threshold and restarts it once the queue has drained, trading backup speed for finishing at all. It also watches for Google Drive's daily upload ceiling: past it rclone exits by design, and the KeepAlive agent relaunching it achieves nothing until the quota renews, so the guard says so plainly instead of letting the system spin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The shell scripts under gdrive/ carried everything the Drive backing layer
knows, which made the app a spectator to its own job. The knowledge now lives
in CloudMachineCore, and the agent binary exposes it: install-rclone,
mount-drive, create-image, attach-image, verify-image, buffer-guard,
drive-status. launchd and the GUI can call the binary instead of a shell.
CMTooling exists because ProcessRunner.runRclone resolves `env rclone`, which
finds the Homebrew build — the one compiled without FUSE that refuses to mount
outright. Anything touching a mount must go through the managed binary instead.
Two bugs the port surfaced, both of the silent kind:
parseDevices filtered partitions with `!device.contains("s")`, and "disk"
contains an "s", so it discarded every device. Stale-device cleanup would have
been a no-op — and that cleanup is what stops a dead handle from being mistaken
for a destroyed backup. The test now pins it.
Free space came from volumeAvailableCapacityForImportantUsage, which counts
space macOS could reclaim from snapshots: it reported 1202 GB where df said
427 GB. The buffer guard exists to pause before the disk fills, so it has to
read what is free now, not what might be freed. It uses statfs, and a test
holds it to df's number.
The comments carry the measurements behind each choice — band size, retry on
FUSE-T's intermittent RPC failures, in-place image creation, the soft cache
limit — so the reasoning survives where the code is, not only in gdrive/README.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
launchd no longer points at scripts. Three agents run the bundled binary: gdrive-buffer (mount-drive, KeepAlive), gdrive-attach (attach-image, every 15 minutes to recover from a dropped mount) and buffer-guard. Verified live — the buffer comes up, the image attaches and the guard reports its thresholds, all from /Applications/CloudMachine.app. Two templates are removed rather than ported, because the architecture they served no longer exists. archive-watchdog copied completed backups from a local Time Machine volume to Drive; there is no local volume now, the backup goes straight to Drive. verify-watchdog is worse than obsolete — it runs `tmutil verifychecksums` over the whole backup, which against a Drive-backed volume means pulling every byte back down. Installing it would have been a trap. buffer-guard no longer exits when a backup finishes. Under KeepAlive that would mean an immediate relaunch, and with Time Machine idle a restart loop bounded only by ThrottleInterval. It goes idle instead and picks up the next backup. What stays here are the measurement harnesses — they measure hdiutil and FUSE-T behaviour, which no unit test reaches — plus the README, which keeps the numbers behind each decision. poc-common.sh replaces cm-env.sh so the harnesses no longer depend on the production scripts that are gone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Seven services went: archive and verify watchdogs, BackupVerifier, CloudArchiveService, LocalBackupService, NetworkShareService, CooldownGate. They served a design with a local Time Machine volume archived to Drive and a disk shared over SMB. Neither exists now — the backup writes straight through the buffer to Drive. The UI described that dead design too, so it is rewritten rather than patched. The dashboard answers one question on open — are my files safe — and shows detail only when the answer is no. The number it leads with is the upload queue, not the buffer size: until the queue returns to zero between backups, the data is still only on this Mac. Steps that genuinely need a terminal (OAuth, `tmutil setdestination`) are shown as copyable commands instead of buttons that would fail. The controller holds no backup logic. Everything it knows about Drive, the image and the buffer comes from CloudMachineCore, so the CLI and the GUI cannot drift apart. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The official installer leaves /Applications/fuse-t.app in place. That app hosts the FSKit extension — a backend this project does not use, since the mount goes over NFS — so it sits there doing nothing. Exactly two files are needed, and both link only against system libraries: libfuse-t.dylib, which rclone dlopens, and go-nfsv4, which actually holds the mount. install-fuse downloads the official package, extracts those two into ~/.cloudmachine/fuse and points /usr/local/lib/libfuse-t.dylib at our copy. The server path is passed through FUSE_NFSSRV_PATH so it can live anywhere; the library cannot, because rclone hardcodes absolute paths and dlopen on an absolute path ignores DYLD_*. A symlink covers that, and needs no root: /usr/local/lib belongs to the user and the admin group. Verified — the mount now runs from ~/.cloudmachine/fuse/go-nfsv4. The package is downloaded rather than committed. The binaries are 21 MB, and FUSE-T is not open source: the binary distribution is free for non-commercial use provided the copyright notice is kept, which is why LICENSE.rtf is copied next to them. Bundling with commercial software would need a licence from its authors. mount-drive restores the symlink before checking anything, because FUSE-T's own uninstaller wipes everything under its path — including our link. Tested by deleting the link: the agent recreated it and mounted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Detaching the image is itself a write: APFS flushes metadata into bands, and --vfs-write-back defers sending them by thirty seconds. Losing the buffer in that window does not cost "the last few changes" — it costs the volume's root directory. Observed live: 367 MiB of bands already on Drive and the reattached image completely empty, because three metadata bands were killed while still queued. detach() now waits out the write-back delay and then for the queue to go quiet, and says plainly when it could not: Odpiete, ale wysylka NIE zakonczyla sie w czasie - nie kasuj bufora. Proven on a fresh image, since the first attempt at this test had corrupted the old one and anything measured on it was meaningless: write 200 MB, detach with drain, SIGKILL rclone, wipe the local cache to zero, let the agents recover, read back from Drive. Byte-identical. hasFuse now counts our own copy. The FUSE-T uninstaller removes the symlink in /usr/local/lib along with the separate app, and without this the status claimed FUSE was missing while the mount was plainly working. checkReadiness repairs the link rather than waiting for the next launch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The dashboard carried a tail of the log, duplicating the Logs tab that exists for exactly that. It also meant reading the file on every refresh cycle so that nobody would look at it. The tab reads it itself when visible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Shutdown is where this architecture is fragile, not startup. All three agents carry RunAtLoad and the two that must never stay down carry KeepAlive, so after a reboot the buffer mounts, the image attaches and the guard resumes on their own; Time Machine finds the destination again because the volume UUID does not change. Powering off is the dangerous half. It kills rclone, and if bands are still queued the image can come back empty — the same failure that detach() now guards against. macOS gives an agent seconds to exit, which is not enough for a full queue. prepare-shutdown stops the backup first so nothing new is written, then detaches and waits for the queue to drain. It refuses to report success while data is still local, because the whole point is to know before pulling the plug rather than after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The guard halted a live backup after 109 GiB of the 750 GB daily allowance,
reporting the quota exhausted. It was not: the error was 403
userRateLimitExceeded, a transient throttle rclone retries by itself ("will
retry in 1m0s"). The trap is that rclone announces it as "Received upload limit
error", which no string match can tell apart from the real daily cap.
So detection now reads rclone's behaviour instead of its wording. The real cap
is fatal to rclone under --drive-stop-on-upload-limit, so the process exits and
the mount disappears; while the mount stands, rclone is coping and there is
nothing to pause. Only storageQuotaExceeded and teamDriveFileLimitExceeded
still count, and only within the last thirty minutes — without a time window a
single entry would linger in the log and hold the backup in a
pause-resume-pause cycle forever.
The parser is now pure so the case that caused this has a test.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Closing the dashboard window called stopAutoRefresh, which killed refreshing for the whole app — the controller is shared with the menu bar. Everything then kept showing whatever was true when the window closed. Reported as "the buffer is down and Drive is disconnected" while both were plainly working; the state was simply from minutes earlier. A frozen view and a broken system look identical, so the dashboard now shows when the state was last read. The user should not have to open a log to tell the difference. isConfigured asked the Homebrew rclone while everything else runs the managed binary. Both read the same config today, so the answers agree — but the status must describe what we actually use, not a second installation that may not always be there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The buffer size came from walking the cache directory — 6504 stat calls, every
ten seconds while the dashboard is open, on the same disk the backup writes to.
rclone publishes the number in vfs/stats as bytesUsed. It also publishes
outOfSpace, meaning it has nothing left to evict because nothing has been
uploaded yet; that is a harder fact than any threshold we could pick, and the
guard now pauses on it. The directory walk stays as a fallback for when the
control interface is unreachable.
Guard thresholds derive from the cache size instead of being written by hand.
150 GB was right only by coincidence for a 100 GB buffer: --vfs-cache-max-size
is a soft limit and the buffer sits at that limit in normal operation —
measured at exactly 100 GiB throughout the first upload — so a threshold at or
below it would pause the backup permanently. Changing one number without the
other would have produced a threshold that either never fires or never stops
firing. The relationship now has tests, which the extracted step() was supposed
to have from the start and did not.
verify() detached through `defer { Task { ... } }`, returning before the detach
happened. Callers attach again immediately, so the attach raced a pending
detach of the same device. It waits now.
detach() unmounts browsed backup snapshots first. Reading a backup — in Finder
or with plain ls under a path from tmutil listbackups — mounts its snapshot
read-only, and that mount holds the image device busy. hdiutil then refuses
with a message that never mentions what is blocking it. This blocked the
consistency check until it was traced by hand.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
The README still described the two-tier design: Time Machine to a real local disk, a background job archiving completed backups to Drive, and that disk optionally shared over SMB for a second Mac. None of it is true any more — the backup writes straight through the buffer to Drive, and the setup it documented would not work if followed. The new text leads with why the write path has no network filesystem in it, since that is the decision the whole design rests on, and carries the measured costs rather than estimates: 210 GiB full, ~370 MB per increment, ~9 GB a day against Google's 750 GB ceiling. The reasoning behind band size, the bundled rclone and FUSE-T, and the buffer guard is stated where someone deciding whether to trust this will look for it, including FUSE-T's non-commercial licence. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Running prepare-shutdown before every restart is the wrong deal to offer. The risk is not constant: it exists only while the buffer holds data that has not been uploaded. macOS gives agents seconds to exit, which is plenty for an empty queue and useless for a full one — and the queue returns to zero within minutes of each hourly backup, so most of the time a plain reboot is simply fine. drive-status now says which case you are in. An unreachable control interface counts as unsafe: with a question like this, silence cannot mean yes. install-launchd detaches the image before reloading the agents. Reloading them while the image is attached pulls the mount out from under it, and detaching is itself a write that still has to reach Drive. I made exactly this mistake three times in two days, including twice after writing the warning about it, so it is now the code's job rather than mine to remember. Adds a login agent that opens the app through `open` — launching the binary directly would put a menu-bar app outside the Aqua session with nothing to draw on. No KeepAlive: `open` returns immediately, so closing the window stays closed instead of being an invitation to relaunch. Backups do not depend on the app; it is the view, not the engine. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
Every command in the README and in conversation was written as `cloudmachine-agent ...`, but the binary lives inside the app bundle, so all of them end in "command not found". The user hit this on the one command that matters most — prepare-shutdown, right before a restart. install-launchd now symlinks it into /usr/local/bin, which needs no root since that directory belongs to the user. The README says where the binary is for anyone who has not run the installer yet. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWh4XwG2PhLEo4tBeTVumv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.