feat(sync): persist the local cache and sync in the background - #209
Open
stevenwcarter wants to merge 1 commit into
Open
feat(sync): persist the local cache and sync in the background#209stevenwcarter wants to merge 1 commit into
stevenwcarter wants to merge 1 commit into
Conversation
Every launch deleted the SQLite cache, then blocked the UI behind a centered "Loading data" overlay while it re-downloaded the entire account over four sequential API round-trips. The cache location was hardcoded, so users could not move it off a synced home directory, and `cargo test` deleted a real installation's database on every run. `auto_sync_interval_minutes` was parsed, defaulted and validated, but never read by any runtime code — no timer ever existed. Storage: - New `[storage] data_dir` config key. A leading `~` is expanded and relative paths resolve against the working directory; unset keeps the platform default. An unusable path fails at startup with the path in the error rather than silently falling back, which would send writes to the database the user was trying to move away from. - The cache is no longer deleted at startup, and the schema is created idempotently, so it survives across launches. - The backend row's UUID is derived from (type, name) instead of being generated per launch. An existing row's UUID is adopted as-is, so a database written by an earlier version keeps its cache instead of having every project and task orphaned under a dead backend_uuid. Duplicate rows left by older debug runs are collapsed before the new unique index is built. - The database file is created 0600 on Unix; it holds the API token. Sync: - Reconciles instead of rebuilding: rows are upserted on (backend_uuid, remote_id) and rows the remote no longer returns are deleted. The delete pass runs before parent re-linking, so completing a parent task cannot cascade its still-open subtasks out of the cache. Local UUIDs stay stable across syncs. - An empty fetch never wipes an entity table. - The four fetches run concurrently; per-resource error handling is unchanged, including sections degrading to empty rather than failing. Interface: - Startup paints cached data immediately and syncs in the background. - Sync state appears as a toast in the lower right of the task list — syncing, synced, or failed — instead of a modal that ate keystrokes. A failed sync now reports as failed rather than as success. - `auto_sync_interval_minutes` drives a real timer; 0 disables it, and `r` still forces a sync. Every terminal outcome records the attempt, so a failing backend is retried once per interval rather than ten times a second. - Task selection follows the selected task across a sync-driven reload, so data arriving out of band cannot move the cursor under the user's hands. Reloads the user asked for — a task operation, a sidebar change, a debug refresh — leave the cursor on the row it was on, so marking an overdue task due today does not drag the cursor along as the task moves out of the Overdue section. Each reload carries its own policy, rather than the list guessing from shared state. Behavior change: a locally completed or deleted task is removed from the cache at the next sync rather than lingering until the next launch, so it can be restored from the list for at most one auto-sync interval. Excluding those rows instead would make them permanent tombstones, since the remote never returns them again. README and the architecture, configuration and development docs described a system that never existed — an in-memory database, a five-minute staleness check, and a dependency list naming crates the project does not use. They now describe the code. Tests no longer touch any user path: storage is constructed against a tempfile directory throughout, and timing decisions are pure functions taking `now`, so nothing sleeps. Claude-Session: https://claude.ai/code/session_01QddAyPzAbdC5bRK7nccDv5
Owner
|
Hello @stevenwcarter and thanks for the PR ! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every launch deleted the SQLite cache, then blocked the UI behind a centered "Loading data" overlay while it re-downloaded the entire account over four sequential API round-trips. The cache location was hardcoded, so users could not move it off a synced home directory, and
cargo testdeleted a real installation's database on every run.auto_sync_interval_minuteswas parsed, defaulted and validated, but never read by any runtime code — no timer ever existed.Storage:
[storage] data_dirconfig key. A leading~is expanded and relative paths resolve against the working directory; unset keeps the platform default. An unusable path fails at startup with the path in the error rather than silently falling back, which would send writes to the database the user was trying to move away from.Sync:
Interface:
auto_sync_interval_minutesdrives a real timer; 0 disables it, andrstill forces a sync. Every terminal outcome records the attempt, so a failing backend is retried once per interval rather than ten times a second.Behavior change: a locally completed or deleted task is removed from the cache at the next sync rather than lingering until the next launch, so it can be restored from the list for at most one auto-sync interval. Excluding those rows instead would make them permanent tombstones, since the remote never returns them again.
README and the architecture, configuration and development docs described a system that never existed — an in-memory database, a five-minute staleness check, and a dependency list naming crates the project does not use. They now describe the code.
Tests no longer touch any user path: storage is constructed against a tempfile directory throughout, and timing decisions are pure functions taking
now, so nothing sleeps.Claude-Session: https://claude.ai/code/session_01QddAyPzAbdC5bRK7nccDv5
Changes
Checklist
cargo fmtpassescargo clippy -- -D warningspassescargo testpassesScreenshots / Logs (if UI/behavioral change)
Screenshots
Sync toasts now shown in the lower-right corner on refreshes instead of an input-blocking modal. (Note that if tasks are added in the current view for the user, their selection remains the same when the sync completed, even if new items are added to their current view/section)
Logs
[13:59:25.750] Loading 1 backend(s) from database
[13:59:25.756] ✅ Loaded backend: My Todoist (todoist)
[13:59:25.756] Loaded 1 backend instance(s)
[13:59:25.927] ✅ Added backend: My Todoist (todoist)
[13:59:25.928] AppComponent: Starting initial sync
[13:59:25.928] AppComponent: Initial sync scheduled
[13:59:25.928] 🔄 Starting sync process...
[13:59:26.029] Background: Received action StartSync
[13:59:26.029] Background: Received action InitialDataLoaded { projects: ... redacted ... }
[13:59:26.030] Background: Cleaned up 1 finished tasks
[13:59:26.030] Sync already in progress, ignoring
[13:59:26.030] InitialData: Loaded 61 projects, 51 labels, 5 sections, 20 tasks
[13:59:26.030] AppComponent: Set initial sidebar selection to Today
[13:59:26.030] AppComponent: Set initial sidebar selection after initial data load
[13:59:26.030] AppComponent: Scheduled data fetch for initial sidebar selection
[13:59:26.030] InitialData: Updated all component data after initial data load
[13:59:26.131] Background: Received action DataLoaded { projects: [Model { ... redacted ... } ] }
[13:59:26.131] Background: Cleaned up 1 finished tasks
[13:59:26.131] Data: Loaded 61 projects, 51 labels, 5 sections, 20 tasks
[13:59:26.131] Data: Updated all component data after data load
[13:59:27.099] ✅ Fetched 61 projects from backend
[13:59:27.099] ✅ Fetched 316 tasks from backend
[13:59:27.099] ✅ Fetched 51 labels from backend
[13:59:27.099] ✅ Fetched 5 sections from backend
[13:59:27.099] 💾 Storing data in local database...
[13:59:27.119] ✅ Stored projects in database
[13:59:27.121] ✅ Stored labels in database
[13:59:27.121] ✅ Stored sections in database
[13:59:27.167] ✅ Stored tasks in database
[13:59:27.247] Background: Received action SyncCompleted(Success)
[13:59:27.247] Background: Cleaned up 1 finished tasks
[13:59:27.247] Sync: Completed with status Success
[13:59:27.348] Background: Received action DataLoaded { projects: ... redacted ... }
[13:59:27.348] Background: Cleaned up 1 finished tasks
[13:59:27.348] Data: Loaded 61 projects, 51 labels, 5 sections, 20 tasks
[13:59:27.349] Data: Updated all component data after data load
[13:59:51.643] Global key: 'q' - quitting application