Repository navigation
feat(drive): stop re-uploading permanently rejected files and ignore excluded watch events - #134
Merged
Merged
Conversation
This was referenced Sep 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
使用者遇到的問題
wspc-drive library 裡有一個 291 MB 的
heart-rate.jsonl,server 每次都以HTTP 413拒收。wspc drive watch沒有記住「這份內容已被拒收」,每 4–6 秒就把整個檔案重新讀一次、重傳一次,三天累積 444 次,頻寬與 CPU 一直被吃掉。更糟的是,Finder 開著資料夾會不斷改寫.DS_Store;即使.wspc-drive/ignore已經排除**/.DS_Store*,watch 仍會因為這些 event 排程新一輪 sync,每一輪都再重傳一次大檔。背景
tod_01M2SGPQ824RVZXH90RGH2JCEA(來源tod_01M2S8NHVDB66Y6971M4D1269V)docs/superpowers/specs/2026-09-18-drive-permanent-upload-rejection-design.md(docs(drive): specify permanent upload rejection handling [skip deploy] #133)runDriveSyncOnce()(每輪 scan → 決策 → 執行)、executeDrivePathAction()(單一 path 的 upload/download 等動作)、runDriveWatch()(把 fs event 轉成 sync 觸發)。state.json的scan_cache記錄每個本機檔案的mtime_ms、size_bytes、sha256,full scan 與 incremental scan 都會把它更新成本輪完整的本機 view;本 PR 用它當拒收紀錄的 fingerprint。問題出在哪裡
上傳失敗只進了該輪 summary,沒有進
state.json:uploadFile丟出DriveHttpError(413)後,外層 catch 只呼叫recordDrivePathError()。下一輪decideDriveAction()看到的 state 和上一輪一模一樣,仍判定upload_update,於是重讀、重傳。平常檔案上傳會成功,或失敗是 5xx 暫時性錯誤、本來就該重試,所以「永久失敗也會一直重試」這件事不會被注意到;只有 server 對同一份內容永遠說不的時候才會變成迴圈。watch 對 excluded path 的 fs event 照樣排程:
Exclude rules 只在 sync 內部生效。當初認為這類 sync 只是 no-op 所以沒關係,但只要有一個永久失敗的檔,每次 no-op 都變成一次整檔上傳。
修法
判定(
retry.ts):DriveHttpError、400 <= status < 500、不是 401/403/408/409/429、code !== "VERSION_CONFLICT"才算 Permanent Upload Rejection。本機讀檔錯誤、local file changed after scan不是DriveHttpError,自然排除。記錄(
path-executor.ts):只包住api.uploadFile()那一行,讓讀檔錯誤不可能被誤判:rememberUploadRejection()會確認scan_cache[path].sha256等於這次上傳的 digest 才記錄,確保 fingerprint 描述的就是剛被拒收的 bytes。欄位逐一挑出而非 spread,因為isDriveUploadRejection()採嚴格 key 白名單(比照scan_errors),多一個 key 下次讀state.json就會被拒。先回報 path error 再寫 state:寫入失敗時外層 catch 的recordDrivePathError()會因同 path 去重,使用者看到的仍是HTTP 413。略過與清除(
sync.ts):scan 與applyRenamesAsMoves()之後、計算 progresstotal之前掃過每筆紀錄:scan_cache裡的 fingerprint 與紀錄不同、或 path 不在scan_cache(刪除、改名、被排除)、或cli_version !== VERSION→ 刪紀錄,照一般流程處理。rejectedPaths,以op: "upload_rejected"回報同一筆path_errors(retryable: false),不呼叫executeDrivePathAction(),也不計入total。pendingPaths取代了原本迴圈裡的movedPaths.has(path)檢查,total與迴圈共用同一份清單,確保 progress 與實際處理一致。Watch(
watch.ts):啟動時載入 exclude rules;fs event 符合規則就不加入dirtyPaths、不排程。ignore的 event 先重新載入再排程 full sync。重新載入以 promise chain 串起來,避免兩次快速編輯時較舊的讀取結果晚到、蓋掉新規則;ignore無法解析時不過濾(sync 會照現行契約回報錯誤)。沿用exclude-rules.ts原本的emptyDriveExcludeRules(改為 export),沒有另寫一份。與 spec 的差異(已同步更新 spec)
spec 原本寫「上傳成功分支刪除紀錄」與「
removeExcludedState()一併移除紀錄」。實作上這兩處都走不到:會真的去上傳的 path,其紀錄必定已在上述清除步驟因 fingerprint/version 不同被刪;excluded path 不在scan_cache,同樣由該步驟清除。因此不加這兩段,並在b5514ec更新 spec 的 Implementation Decisions 文字。AC 5(排除後清除)與 AC 7(上傳成功後無紀錄)的測試照樣覆蓋這些行為。這個 PR 不修什麼
FILE_TOO_LARGE:tod_01M2S85BY0A38RM37TMNC4KTDY。已知副作用
retryable: false出現在每輪path_errors,但不再上傳;wspc-drive 的「1 file needs attention」維持顯示。state.json新增 optionalupload_rejections,schema_version仍為 1;舊版 CLI 讀到會忽略,回退只會恢復舊的重傳行為。測試
本機
npm run typecheck、npm test(66 files、786 tests)、npm run build通過。sync.test.tsuploads an unchanged rejected file only once across rounds and keeps reporting itAssertionError: expected 3 to be 1(三輪各上傳一次)keeps syncing other files around a rejected fileexpected [ 'big.jsonl', 'notes.txt', …(2) ] to deeply equal [ 'big.jsonl', 'notes.txt', …(1) ]state.test.tsround-trip 與rejects malformed upload rejection records;舊 state 由既有creates and reads empty state覆蓋promise resolved instead of rejecting)retry.test.tsstatus 表格;sync.test.ts502/429/408/401/403/VERSION_CONFLICT、本機讀檔錯誤(EACCES)、local file changed after scanisPermanentUploadRejection is not a functionwatch.test.tsdoes not schedule syncs for fs events on excluded paths and follows ignore editsexpected "vi.fn()" to be called 1 times, but got 2 timesretries once after the content changes/only the mtime changes、deleted/renamed/excluded、another CLI version、uploads successfullysha256比對無法經由真實 scanner 單獨觸發(mtime 與 size 相同時 scanner 會沿用 cache 的 hash),保留它是因為 spec 把 fingerprint 定義為這三個欄位。Review
src/generated/、空值用undefined、rejected_at走driveIsoTimestamp(clock),均符合。AC 4–7 屬「先實作後補測試+mutation 驗證」,如上表說明。drive_sync_once、drive_sync_progress、drive_watch_retry、path_errors)與 transient retry 行為未變。唯一差異是上述兩段走不到的清除邏輯被省略,已同步更新 spec。無額外未要求行為。未完成驗收
~/.hermes/profiles/pocketshark/library/.wspc-drive/debug.log:heart-rate.jsonl 每個 fingerprint 最多一次op: "process"上傳錯誤,.DS_Store的fs_event不再接sync_start。🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.