db backup|restore and the telesto restic job (#110)

BACKLOG.md

@@ -19,9 +19,9 @@ and vice versa.
       `secrets set|check|list|rotate|rm|rekey` ([#107](https://github.com/CryptoJones/FlatlineRoundtable/issues/107)) — shipped in PR #114
 - [x] Lane config in the store: `lanes`, `defaults`, `globals`, `lane_versions`,
       `db export|import`, `import-yaml` ([#108](https://github.com/CryptoJones/FlatlineRoundtable/issues/108)) — shipped in PR #123
-- [ ] Run from the store: `load_config` reads the DB, `fetch_keys` reads secrets,
-      YAML retired ([#109](https://github.com/CryptoJones/FlatlineRoundtable/issues/109)) — on feat/sqlite-store-run-from-store
-- [ ] `db backup|restore` and the telesto restic job ([#110](https://github.com/CryptoJones/FlatlineRoundtable/issues/110))
+- [x] Run from the store: `load_config` reads the DB, `fetch_keys` reads secrets,
+      YAML retired ([#109](https://github.com/CryptoJones/FlatlineRoundtable/issues/109)) — shipped in PR #124
+- [ ] `db backup|restore` and the telesto restic job ([#110](https://github.com/CryptoJones/FlatlineRoundtable/issues/110)) — on feat/sqlite-store-backup
 - [ ] Fleet rollout notes and the Python 3.11 floor ([#111](https://github.com/CryptoJones/FlatlineRoundtable/issues/111))
 
 ### Epic: run the roundtable from inside Orca — local web UI, no fork ([#115](https://github.com/CryptoJones/FlatlineRoundtable/issues/115)) — DONE 2026-10-03
@@ -183,6 +183,12 @@ stub.
       `ps auxww` during `db init` shows no key (it goes to `pass insert` on stdin).
 - [ ] `./install.sh` then `./install.sh --uninstall` round-trips, and refuses to
       delete anything that is not a symlink. *(human — touches `$HOME`, no test)*
+- [ ] **Restore drill:** `restic restore latest` from telesto → `db restore` into
+      a scratch `--config` → `db doctor` → `secrets check` → `--list`, and the
+      scratch store's `db export` equals the live one. *(human — needs telesto and
+      the real DB key; the commands are in the README.)* Last run 2026-10-06 on
+      makemake: all steps ok, 9/9 secrets present, export identical.
+      *Automated parts:* `TestBackup`.
 
 ## Done
 

README.md

@@ -126,6 +126,35 @@ copy. `--set` values are JSON when they parse and strings otherwise.
 [`examples/roster.example.json`](examples/roster.example.json) is the example
 YAML config after `import-yaml` and `db export`.
 
+**Backups (#110).** `roundtable db backup` writes an integrity-checked `0600`
+copy to `backups/` beside the store, with SQLite's online backup API, so it is
+consistent while a run has the store open. It keeps the newest 14 (`--keep N`).
+`roundtable db restore FILE` refuses a corrupt file, and a file from a newer
+schema unless `--force`; the store it replaces is kept as
+`roundtable.db.pre-restore-<stamp>`.
+
+[`scripts/backup-telesto.sh`](scripts/backup-telesto.sh) is a one-shot job: it
+takes a `db backup`, then pushes the backups and the gpg-encrypted DB key entry
+to a restic repository on telesto (`sftp:telesto:/NAS/backups/flatline-roundtable/<host>`,
+password in the `pass` entry `flatline-roundtable/restic-telesto`), prunes to
+14 daily, 8 weekly and 6 monthly snapshots, and runs `restic check` on Sundays.
+A store without its DB key cannot decrypt its secrets, which is why the key goes
+too; the key is in turn useless without your gpg private key.
+[`examples/net.thenetwerk.roundtable-backup.plist`](examples/net.thenetwerk.roundtable-backup.plist)
+runs it daily at 03:45 under launchd.
+
+The restore drill, run against a scratch path so the live store is untouched:
+
+```sh
+export RESTIC_REPOSITORY=sftp:telesto:/NAS/backups/flatline-roundtable/$(hostname -s)
+export RESTIC_PASSWORD_COMMAND="pass show flatline-roundtable/restic-telesto"
+restic restore latest --target /tmp/drill
+roundtable db restore /tmp/drill/.../backups/roundtable-<host>-<stamp>.db --config /tmp/drill/roundtable.db
+roundtable db doctor --config /tmp/drill/roundtable.db
+roundtable secrets check --config /tmp/drill/roundtable.db
+roundtable --config /tmp/drill/roundtable.db --list
+```
+
 ## Cost
 
 **`harness` decides cost, not `model`.**

examples/net.thenetwerk.roundtable-backup.plist

@@ -0,0 +1,38 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<!--
+  Daily roundtable store backup to telesto (#110). A timer that starts a
+  one-shot script, so nothing stays resident.
+
+  Edit the two paths below, then:
+    cp examples/net.thenetwerk.roundtable-backup.plist ~/Library/LaunchAgents/
+    launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/net.thenetwerk.roundtable-backup.plist
+-->
+<plist version="1.0">
+  <dict>
+    <key>Label</key>
+    <string>net.thenetwerk.roundtable-backup</string>
+    <key>ProgramArguments</key>
+    <array>
+      <string>/Users/YOU/source/repos/FlatlineRoundtable/scripts/backup-telesto.sh</string>
+    </array>
+    <key>EnvironmentVariables</key>
+    <dict>
+      <key>PATH</key>
+      <string>/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
+    </dict>
+    <key>StartCalendarInterval</key>
+    <dict>
+      <key>Hour</key>
+      <integer>3</integer>
+      <key>Minute</key>
+      <integer>45</integer>
+    </dict>
+    <key>RunAtLoad</key>
+    <false/>
+    <key>StandardOutPath</key>
+    <string>/Users/YOU/.local/state/flatline-roundtable/backup.log</string>
+    <key>StandardErrorPath</key>
+    <string>/Users/YOU/.local/state/flatline-roundtable/backup.log</string>
+  </dict>
+</plist>

roundtable

@@ -56,6 +56,7 @@ YAML is retired, and `import-yaml` moves an old config in once.
     roundtable lanes enable|disable|retire NAME | clone SRC NEW | rename OLD NEW
     roundtable defaults|globals show | set k=v ... | unset k ...
     roundtable db export [--out FILE] | import FILE [--merge|--replace]
+    roundtable db backup [--dir DIR] [--keep N] | restore FILE [--force]
     roundtable import-yaml PATH [--dry-run] [--pull-secrets]   one-time move off YAML
 
   * Round 1 is blind by construction — a lane cannot see the others because
@@ -891,7 +892,7 @@ def _referencing_lanes(conn: sqlite3.Connection, name: str) -> list[str]:
 # store admin commands
 # --------------------------------------------------------------------------- #
 ADMIN_VERBS = {
-    "db": ("init", "migrate", "doctor", "export", "import"),
+    "db": ("init", "migrate", "doctor", "export", "import", "backup", "restore"),
     "secrets": ("set", "check", "list", "rotate", "rm", "rekey"),
     "lanes": ("list", "show", "add", "edit", "enable", "disable", "retire", "clone",
               "rename", "history"),
@@ -1884,6 +1885,125 @@ def cmd_import_yaml(a) -> int:
     return 0
 
 
+# --------------------------------------------------------------------------- #
+# backup and restore (#110)
+#
+# A backup is a self-contained copy taken with SQLite's online backup API, so
+# it is consistent even while a run holds the store open in WAL mode -- a file
+# copy of roundtable.db alone would miss whatever still sits in -wal. The copy
+# is useless without the DB key, which lives in `pass`; scripts/backup-telesto.sh
+# ships both.
+# --------------------------------------------------------------------------- #
+def _sqlite_copy(src: sqlite3.Connection, dest: Path) -> None:
+    """Write `src` to a new, private `dest` as a rollback-journal database, so
+    the copy is one file with no -wal or -shm beside it."""
+    os.close(os.open(dest, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600))
+    dst = sqlite3.connect(dest, isolation_level=None)
+    try:
+        src.backup(dst)
+        dst.execute("PRAGMA journal_mode=DELETE")
+    finally:
+        dst.close()
+
+
+def _integrity(path: Path) -> str:
+    try:
+        c = sqlite3.connect(f"file:{path}?mode=ro", uri=True)
+        try:
+            return c.execute("PRAGMA integrity_check").fetchone()[0]
+        finally:
+            c.close()
+    except sqlite3.DatabaseError as e:
+        return str(e)
+
+
+def cmd_db_backup(a) -> int:
+    conn = open_db(a.config)
+    out_dir = a.dir or a.config.parent / "backups"
+    out_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
+    host = re.sub(r"[^A-Za-z0-9.-]", "_", socket.gethostname().split(".")[0])
+    stamp = dt.datetime.now(dt.timezone.utc).strftime("%Y%m%dT%H%M%SZ")
+    dest = out_dir / f"roundtable-{host}-{stamp}.db"
+    n = 1
+    while dest.exists():                      # two backups in one second
+        dest = out_dir / f"roundtable-{host}-{stamp}-{n}.db"
+        n += 1
+    try:
+        _sqlite_copy(conn, dest)
+    finally:
+        conn.close()
+    result = _integrity(dest)
+    if result != "ok":
+        dest.unlink()
+        die(f"backup copy failed integrity_check ({result}) — nothing kept")
+    # Newest first: by stamp, then by the -N a same-second backup gets. Plain
+    # name order would put "...Z-1.db" before "...Z.db", which is backwards.
+    pat = re.compile(rf"roundtable-{re.escape(host)}-(\d{{8}}T\d{{6}}Z)(?:-(\d+))?\.db")
+    mine = sorted((p for p in out_dir.iterdir() if pat.fullmatch(p.name)),
+                  key=lambda p: (pat.fullmatch(p.name)[1], int(pat.fullmatch(p.name)[2] or 0)),
+                  reverse=True)
+    pruned = mine[a.keep:] if a.keep > 0 else []
+    for old in pruned:
+        old.unlink()
+    print(f"backup {dest} (integrity ok, kept {min(len(mine), a.keep)}"
+          + (f", pruned {len(pruned)}" if pruned else "") + ")")
+    return 0
+
+
+def cmd_db_restore(a) -> int:
+    src_path = a.file
+    if not src_path.exists():
+        die(f"no backup at {src_path}")
+    result = _integrity(src_path)
+    if result != "ok":
+        die(f"{src_path} fails integrity_check ({result}) — refusing to restore it")
+    src = sqlite3.connect(f"file:{src_path}?mode=ro", uri=True)
+    try:
+        ver = schema_version(src)
+        kid = src.execute("SELECT value FROM meta WHERE key = 'key_id'").fetchone()
+    except sqlite3.DatabaseError:
+        kid = None
+    if kid is None:
+        src.close()
+        die(f"{src_path} is not a roundtable store (no key id in its meta table)")
+    # --force exists for this check only. A newer schema may hold tables this
+    # code does not know, which is a choice; a corrupt file never is.
+    if ver > SCHEMA_VERSION and not a.force:
+        src.close()
+        die(f"{src_path} is at schema {ver}, newer than this roundtable ({SCHEMA_VERSION}) — "
+            f"update roundtable, or pass --force")
+    target = a.config
+    kept = None
+    if target.exists():
+        live = open_db(target)
+        # Fold -wal into the main file so the copy kept aside is complete.
+        live.execute("PRAGMA wal_checkpoint(TRUNCATE)")
+        live.close()
+        stamp = dt.datetime.now(dt.timezone.utc).strftime("%Y%m%dT%H%M%SZ")
+        kept = target.with_name(f"{target.name}.pre-restore-{stamp}")
+        target.rename(kept)
+        for side in ("-wal", "-shm"):
+            Path(f"{target}{side}").unlink(missing_ok=True)
+    else:
+        target.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
+    try:
+        _sqlite_copy(src, target)
+    except BaseException:
+        if kept is not None and not target.exists():
+            kept.rename(target)
+        raise
+    finally:
+        src.close()
+    restored = open_db(target)
+    restored.execute("PRAGMA journal_mode=WAL")
+    restored.close()
+    print(f"restored {src_path} -> {target} (schema {ver}, key id {kid[0]})")
+    if kept is not None:
+        print(f"  the store it replaced is kept as {kept}")
+    print("  check it with:  roundtable db doctor && roundtable secrets check")
+    return 0
+
+
 def admin_main(argv: list[str]) -> int:
     common = argparse.ArgumentParser(add_help=False)
     common.add_argument("--config", type=Path, default=DEFAULT_DB,
@@ -1911,6 +2031,17 @@ def admin_main(argv: list[str]) -> int:
     mode.add_argument("--replace", action="store_true",
                       help="make the roster exactly the file; unlisted lanes are retired")
     p.set_defaults(fn=cmd_db_import)
+    p = dbv.add_parser("backup", parents=[common],
+                       help="integrity-checked 0600 copy, safe while the store is in use")
+    p.add_argument("--dir", type=Path, help="where to write (default: backups/ beside the store)")
+    p.add_argument("--keep", type=int, default=14, help="newest copies to keep (default 14)")
+    p.set_defaults(fn=cmd_db_backup)
+    p = dbv.add_parser("restore", parents=[common],
+                       help="replace the store with a backup; the old one is kept aside")
+    p.add_argument("file", type=Path)
+    p.add_argument("--force", action="store_true",
+                   help="restore a backup from a newer schema (never a corrupt one)")
+    p.set_defaults(fn=cmd_db_restore)
 
     ln = groups.add_parser("lanes", help="the lane roster in the store")
     lv = ln.add_subparsers(dest="verb", required=True)

scripts/backup-telesto.sh

@@ -0,0 +1,59 @@
+#!/usr/bin/env bash
+# One-shot backup of the roundtable store to telesto, with restic (#110).
+#
+# Runs to completion and exits: a launchd timer starts it (see
+# examples/net.thenetwerk.roundtable-backup.plist), which keeps the
+# "no long-lived processes" rule in AGENTS.md.
+#
+# What goes over, and why both halves:
+#   * the store's backups dir -- `roundtable db backup` writes an
+#     integrity-checked copy there first, so restic never reads a live WAL file;
+#   * the gpg-encrypted pass entries under flatline-roundtable/ -- the DB key.
+#     Without it a restored store's secrets cannot be decrypted. It is still
+#     useless without your gpg private key, which this does not touch.
+#
+# Override any of these from the environment:
+#   RESTIC_REPOSITORY        default sftp:telesto:/NAS/backups/flatline-roundtable/<host>
+#   RESTIC_PASSWORD_COMMAND  default pass show flatline-roundtable/restic-telesto
+#   KEEP                     local backup copies to keep (default 14)
+#   CHECK=1                  run `restic check` now (otherwise Sundays only)
+#
+# rsync is the fallback if restic is unavailable:
+#   rsync -a "$DATA/backups/" telesto:/NAS/backups/flatline-roundtable/<host>/
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+ROUNDTABLE="${SCRIPT_DIR}/../roundtable"
+HOST="$(hostname -s)"
+DATA="${XDG_DATA_HOME:-${HOME}/.local/share}/flatline-roundtable"
+PASS_DIR="${PASSWORD_STORE_DIR:-${HOME}/.password-store}/flatline-roundtable"
+
+export RESTIC_REPOSITORY="${RESTIC_REPOSITORY:-sftp:telesto:/NAS/backups/flatline-roundtable/${HOST}}"
+export RESTIC_PASSWORD_COMMAND="${RESTIC_PASSWORD_COMMAND:-pass show flatline-roundtable/restic-telesto}"
+
+log() { printf '%s %s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$*"; }
+die() { log "ERROR: $*" >&2; exit 1; }
+
+command -v restic >/dev/null || die "restic is not installed (brew install restic)"
+[[ -f "${PASS_DIR}/db-key.gpg" ]] || die "no DB key at ${PASS_DIR}/db-key.gpg — backing up a store without its key is pointless"
+
+log "snapshot the store"
+"${ROUNDTABLE}" db backup --keep "${KEEP:-14}"
+
+if ! restic cat config >/dev/null 2>&1; then
+    log "initialise ${RESTIC_REPOSITORY}"
+    restic init
+fi
+
+log "restic backup"
+restic backup --quiet --tag flatline-roundtable --host "${HOST}" "${DATA}/backups" "${PASS_DIR}"
+
+log "restic forget"
+restic forget --quiet --tag flatline-roundtable --host "${HOST}" \
+    --keep-daily 14 --keep-weekly 8 --keep-monthly 6 --prune
+
+if [[ "$(date +%u)" == 7 || "${CHECK:-}" == 1 ]]; then
+    log "restic check"
+    restic check
+fi
+log "done"