Skip to content

fix(folders): clean up blobs on recursive delete without destroying live citations (#177) - #214

Merged
PeterGuy326 merged 4 commits into
mainfrom
fix/177-folder-delete-source-guard
Sep 18, 2026
Merged

PeterGuy326 merged 4 commits into
mainfrom
fix/177-folder-delete-source-guard

Conversation

@waterbro-8

@waterbro-8 waterbro-8 commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

Refs #177

Requirement and scope

Successor to #210 for #177. Keeps the original blob-cleanup implementation and answers the remaining review request: recursive delete must not physically destroy an object still cited by an active/archived memory whose path is outside the folder.

files.storage_key remains per-row (users/<user_id>/<file_id>/<basename>), so unconditional key delete is still safe across file rows. The new hole was memories.source_file_id … ON DELETE SET NULL: a memory at /Work/task citing a file under /Photos would previously pass the path-only guard, the file row would vanish, the FK would null out, and this PR's blob cleanup would then make the cited bytes unrecoverable.

Changes

  • Recursive containsMemoriesTx also matches source_file_id against files in the deleted tree (folder_id or literal path prefix). Hit → existing ErrContainsMemories (forget first).
  • Integration regression TestRecursiveDeleteBlocksWhenMemoryCitesFileElsewhere.
  • Cleanup WARN distinguishes a per-object store error from the shared 30s budget running out.
  • docs/DEPLOYMENT.md and [Unreleased] changelog record the citation guard.

Original blob-cleanup work and contributor credit: @xiaocui-big via #210. This successor does not close #210 or #177 automatically.

Validation ledger

ID Criterion Command Environment Status
V1 Folder package hermetic tests gofmt + go test ./internal/folder This checkout; host Go is 1.22.12, module requires 1.25.0; toolchain download timed out NOT VERIFIED locally — CI must run
V2 Citation-guard integration MEM_TEST_DB=…_test go test ./internal/folder -run TestRecursiveDeleteBlocksWhenMemoryCitesFileElsewhere No disposable _test database in this environment NOT VERIFIED locally — CI PostgreSQL job
V3 Server regression make test-server Same toolchain gap NOT VERIFIED locally — CI Go job
V4 Patch whitespace git diff --check local PASS
V5 Worker / Web / MCP unchanged N/A N/A — no worker/web/mcp/API contract change beyond folder delete refusal

Risk / rollback

Recursive delete of a folder whose files are still cited by live memories now fails closed instead of deleting the blobs. Revert this branch to restore path-only guarding (and, if #210 is also reverted, orphan blobs on recursive delete).

No merge, release, or issue closure is claimed.

@waterbro-8

Copy link
Copy Markdown
Collaborator Author

Pushed a test-only follow-up (1db043b) for the CI failures:

  • create a resource-owner workspace before folder create (the lock was no rows in result set)
  • insert memories with idempotency_key_sha256 (the live column after 0012)
  • add source_file_id to the session-local memories shadow table so TestMemoryPathLifecycleIntegration still compiles against the recursive citation guard

PR body now also has Refs #177.

@waterbro-8

Copy link
Copy Markdown
Collaborator Author

Follow-up e7e5bba: the remaining Go failure was the same class of 42P08 as #218$3 was bound to both source_file_sha256 (text) and idempotency_key_sha256 (char(64)). Those are now separate parameters.

I cannot approve this PR: last push is mine. Needs another CODEOWNER after CI is green.

@waterbro-8

Copy link
Copy Markdown
Collaborator Author

Waiting on CI for e7e5bba (42P08 fixture). I cannot approve: last push is mine. Needs @PeterGuy326 or @Bindy-lbb after green.

@waterbro-8

Copy link
Copy Markdown
Collaborator Author

请评审合入(作者是我,不能自批)。

  • 递归删文件夹会清 blob,并在 source_file_id 仍被活记忆引用时拒绝删除。
  • 测试侧 42P08 已拆参数(e7e5bba)。

需要 @PeterGuy326@Bindy-lbb 作为非作者 CODEOWNER 批准后才能进 main。

@PeterGuy326 PeterGuy326 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independently reviewed current head e7e5bba against base 1b27aac. The folder regression suite passes on Go 1.25.9/darwin arm64 (go test ./internal/folder -count=1), git diff --check passes, and the required CI checks are green. No blocking finding remains; approving as a non-author CODEOWNER.

xiaocui-big and others added 4 commits September 18, 2026 10:59
Recursive folder delete removed DB rows but left every underlying object
in bucket storage. The code had an explicit TODO acknowledging this.

Changes:
- Add ObjectStore and logger options to folder.Service (WithStore, WithLogger)
- Collect storage keys before deleting files, then delete blobs after
  the transaction commits (best-effort, matching file.Delete's shape)
- Log failed blob deletions at WARN level so operators can see orphans
- Update memd wiring to pass store and logger to folder service
- Add integration tests verifying blob cleanup and backward compatibility
- Document object retention behavior in DEPLOYMENT.md
- Add CHANGELOG entry

The crash window (process killed between commit and blob delete) is
documented as a known gap. A reaper would need to record keys whose
delete was never attempted, since storage.Store has no listing.
Recursive blob cleanup would otherwise destroy objects still cited by
active/archived memories whose path is outside the folder, because
source_file_id is ON DELETE SET NULL.
Recursive-delete tests must lock a resource-owner workspace, insert
memories with idempotency_key_sha256, and keep source_file_id on the
session-local memories shadow table.
…columns

PostgreSQL 42P08: $3 was bound to source_file_sha256 (text) and
idempotency_key_sha256 (char(64)) at once.
@PeterGuy326
PeterGuy326 force-pushed the fix/177-folder-delete-source-guard branch from e7e5bba to 91166d3 Compare September 18, 2026 02:59
@PeterGuy326
PeterGuy326 merged commit 243697d into main Sep 18, 2026
21 checks passed
@PeterGuy326
PeterGuy326 deleted the fix/177-folder-delete-source-guard branch September 18, 2026 03:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants