Collection-scale: cfg export + import --from, bulk --dry-run, adopt-all nudge#19
Merged
Merged
Conversation
…-dry-run, adopt-all nudge
Addresses the 0.2.0 follow-up feedback — the back-up-and-bulk-replace-a-collection story now
lives entirely inside cfgit:
- cfg export [record] [--out file]: read-only portable snapshot of live docs, each stamped with
its cfgit head seq/oid (engine.export_records). The restore-me-later artifact.
- cfg import --from <export.json> [--dry-run]: inverse of export — writes the docs back through
the drift-guarded bulk-commit path, so the restore itself is recorded in history. Accepts a
cfg-export artifact or a bare [{record,doc}] list.
- commit --bulk-from --dry-run: previews every record's delta (would_commit/noop/drift) without
writing (bulk_commit_preview). Fixes a footgun — --dry-run was silently ignored on --bulk-from
and the batch WROTE. Matters most for collection-scale replaces.
- doctor --status: nudges 'run cfg adopt --all to baseline' when a large share of tracked records
drifted, so restore has a clean baseline and edits stop needing a per-record adopt.
- MCP: cfg_export, cfg_import(from_export, dry_run), cfg_bulk_commit(dry_run).
Distinction: cfg tag + restore --tag remain the in-tool versioned rollback; export/import is the
portable-file backup whose write-back is audited.
Tests: test_export_import.py (11) — export shape/stamp/no-write, round-trip restore, import --from
dry-run + garbage rejection, bulk preview per-record + no-write + drift flag, nudge threshold.
Full live E2E vs Mongo replica set: import→export→bulk-dry-run→apply→restore-from-backup (history
shows the restore)→drift nudge. 133 pass / 1 skip, ruff clean.
…seline README + USAGE + SKILL: cfg export snapshot + cfg import --from restore (with the tag/restore vs export/import distinction), commit --bulk-from --dry-run (fixing the 'bulk is invisible' feedback), and the doctor adopt-all baseline nudge. Add cfg_export to the MCP tool lists.
Two robustness gaps from review:
- cfg export warns (soft, never a hard cap) when a snapshot exceeds ~2000 records OR ~50MB
serialized — a strong signal the config points at a data-plane collection (events/content/jobs)
cfgit is not built to version. Prints to stderr; export still completes; stdout stays clean.
- commit_many (the import --from / bulk write path) now catches KeyboardInterrupt — a mid-write
cancel returns a clean {state: partial, cancelled: true} with results + pending instead of dying
silently. KeyboardInterrupt is BaseException, so it previously escaped the Exception handler and
left the operator guessing about DB state. Each apply() is atomic, so rerun resumes.
Tests: +4 (size-warning absent/count/bytes, cancel→clean-partial). Live E2E: 2100-record export
warns on stderr with clean JSON stdout. 136 pass / 2 skip, ruff clean.
…ulk-commit preview) bulk_commit_preview returns state=dry_run, which shared the restore-phrased remedy 'this is what a restore would change' — misleading for a bulk commit preview. Made it neutral. Surfaced by the exhaustive live E2E (22/22).
AusafMo
added a commit
that referenced
this pull request
Jul 26, 2026
Minor bump for the collection-scale release (#19): cfg export + import --from round-trip, commit --bulk-from --dry-run, export size warning, cancellation-safe bulk writes, doctor adopt-all nudge. Additive features, nothing removed. Widen plugin cfgit pin to >=0.3.0,<0.4.0. Both build + twine check clean locally. Co-authored-by: AusafMo <AusafMo@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the post-0.2.0 feedback (
cfgit_suggestion_email.md) — the "back up + bulk-replace a whole collection" story now lives entirely inside cfgit instead of half in pymongo side-scripts. All three suggestions were verified against 0.2.0 before building.What
cfg export [record] [--out file]— read-only portable snapshot of live docs, each stamped with its cfgit head seq/oid (engine.export_records). The restore-me-later artifact the email's pymongojson.dumpwas standing in for.cfg import --from <export.json> [--dry-run]— inverse of export: writes docs back through the drift-guarded bulk-commit path, so the restore itself is recorded in history. Accepts a cfg-export artifact or a bare[{record, doc}]list.commit --bulk-from --dry-run— previews every record's delta (would_commit/noop/changed_outside_cfgit) without writing. Fixes a footgun:--dry-runwas silently ignored on--bulk-fromand the batch wrote. Matters most for collection-scale replaces.doctor --statusadopt-all nudge — when a large share of tracked records drifted, it suggestscfg adopt --allto baseline, so restore has a clean baseline and edits stop needing a per-record adopt.cfg_export,cfg_import(from_export, dry_run),cfg_bulk_commit(dry_run).Design note: two rollback paths, documented
cfg tag+cfg restore --tag/--as-of= in-tool versioned rollback (no file).cfg export/import --from= portable file backup; its write-back is audited in history.Testing
test_export_import.py(11): export shape/stamp/no-write, round-trip restore, import --from dry-run + garbage rejection, bulk-preview per-record + no-write + drift flag, nudge threshold.export --out→--bulk-from --dry-run(writes nothing, verified) → apply →import --from backup.jsonrestore (history shows the restore, live reverted) → forced 15/20 drift →doctor --statusnudgesadopt --all.Follows 0.2.0; suggest releasing as the next minor (0.3.0).