Skip to content

Fix unsupported filesystem identity handling and harden move recovery - #819

Open
therobbiedavis wants to merge 7 commits into
canaryfrom
815-harden-filesystem-identity-and-move-recovery
Open

Fix unsupported filesystem identity handling and harden move recovery#819
therobbiedavis wants to merge 7 commits into
canaryfrom
815-harden-filesystem-identity-and-move-recovery

Conversation

@therobbiedavis

@therobbiedavis therobbiedavis commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR is a follow-up to #717 and addresses filesystem safety issues found during final validation.

The most visible fix is for systems where the filesystem cannot provide the information Listenarr normally uses to safely identify a file or folder, including the older Linux kernel scenario reported in #815. Instead of allowing that condition to stop Listenarr, affected storage is treated as limited: reading and scanning can continue, while moves and deletions stay disabled when Listenarr cannot prove they are safe.

This also fixes #820 by making configured root case semantics authoritative throughout manual import instead of re-probing managed source, destination, and companion paths with Auto.

Fixes #815.
Fixes #820.

Changes

Added

  • Durable move execution protocol 2 with independently persisted source- and target-boundary physical-generation authorization.
  • Pinned path-only scan authority for filesystems that support safe reads but cannot provide durable object generations.
  • Pinned directory namespace-change tokens for authoritative scan reconciliation.
  • Regular-file validation at pinned file-handle boundaries on Windows, Linux, and macOS.
  • No-follow, handle-relative traversal beneath authorized move source and target boundaries.
  • Pinned folder metadata reads for unmatched scans.
  • Deterministic regression coverage for symlink/junction replacement races, named pipes, namespace mutations, generation replacement, protocol compatibility, read-only filesystems, unsupported Linux identity capabilities, Manual import to CIFS root ignores explicit case-sensitivity setting and fails in ManualImportDestinationTracker #820 configured semantics, consecutive moves, and root relocation after a move.

Changed

  • Linux durable object identity now requires strong generation evidence from a file handle or inode generation; birth time remains compatibility evidence only.
  • Unsupported durable filesystem identity becomes IdentityUnsupported scan-only capability where safe pinned reads are available.
  • Limited-storage scans use PinnedPathOnly authority and cannot perform destructive reconciliation or metadata writes.
  • Markerless move I/O resolves descendants relative to persisted generation-bound filesystem boundaries.
  • Source cleanup authority is persisted at enqueue time and reused exactly during worker execution and recovery.
  • Retry, startup reconciliation, and root relocation enforce the same protocol-2 source/target boundary requirements.
  • Finalized move completion re-verifies the target immediately before the completion/history/handoff transaction.
  • Unmatched scans avoid reopening public filesystem paths after durable discovery.
  • Root-folder capability reporting now distinguishes read, scan, and mutation support.
  • Root-folder UI scanning now uses canScanFilesystem instead of assuming scans require mutation support.

Fixed

  • Root-folder identity failure escapes as an unhandled exception instead of IdentityUnsupported, stopping the host on kernels < 4.11 #815 unsupported filesystem identity escaping as an unhandled exception on older Linux kernels.
  • Manual import to CIFS root ignores explicit case-sensitivity setting and fails in ManualImportDestinationTracker #820 manual import discarding configured root case semantics and re-probing managed paths with Auto.
  • Consecutive moves inside one managed root being rejected after the first move narrowed the tracked path-identity boundary to the audiobook directory.
  • Root-folder relocation rejecting safely tracked audiobooks whose path-identity boundary is narrower than the relocating root.
  • Generic manual-import empty-source cleanup running inside configured managed root folders.
  • Docker Desktop/9P libraries being unusable for scanning when durable mutation identity is unavailable.
  • Descendant symlink/junction replacement races that could redirect move publication or cleanup outside the authorized boundary.
  • Generic file deletion through intermediate ancestor replacement.
  • Markerless recovery accepting weaker namespace evidence than the original move.
  • Finalized target replacement between final verification and durable completion.
  • Protocol-1 move state being interpreted as current protocol state.
  • Root-relocation child moves missing complete source-boundary generation authority.
  • Retry paths disagreeing about source-boundary validity or recovery disposition.
  • Configured source roots being treated as unmanaged when current storage identity or case semantics were unavailable.
  • Authoritative scans reconciling absence after transient directory namespace mutation.
  • Scan authorization reopening a different root generation between identity verification and pinned traversal.
  • Limited scans reaching metadata-writing paths.
  • Unmatched scans reopening sidecars, cover art, and metadata through public paths after durable enumeration.
  • Audio named pipes and other special files reaching normal metadata or identity processing.
  • Move manifest timestamps being read from public paths instead of pinned file handles.

Removed

  • Birth-time-only Linux identity as destructive filesystem authority.
  • Worker-time rediscovery of source cleanup boundaries for current durable moves.
  • Weak target-boundary authorization parsing that did not bind persisted raw identity to its authorization digest.
  • Raw-path metadata enrichment from scan-only/limited filesystem scans.
  • Move-service namespace mutations that relied on absolute descendant paths instead of pinned boundary-relative traversal.

Testing

Current-head Windows, Linux, frontend, Docker, and adversarial-review validation is complete.

  • Local Windows backend: 2,737 passed, 0 failed, 119 skipped; build 0 warnings / 0 errors.
  • Current-head Windows CI: 2,788 passed, 0 failed, 77 skipped; build 0 warnings / 0 errors.
  • Current-head Ubuntu 24.04 backend: 2,779 passed, 0 failed, 89 skipped.
  • Frontend: build, lint, type-check, and 559/559 tests across 88 files passed.
  • CodeQL and all PR checks passed.

An image built directly from the exact pushed commit passed Docker Desktop acceptance against the same enrolled physical test storage:

  • Docker Desktop host bind: Limited / IdentityUnsupported, read and scan enabled, mutation disabled; unmatched scan completed.
  • Read-write named volume: Healthy, read/scan/mutation enabled; unmatched scan completed.
  • Read-only named volume: Limited / ReadOnlyFilesystem, read and scan enabled, mutation disabled; unmatched scan completed.
  • A real read-write move completed, an immediate second move from the newly moved directory completed, and the move back restored the original paths.
  • A healthy read-write source to unsupported host-bind target move was rejected before publication with destination_physical_identity_unavailable.
  • Normalized host-bind, read-write, and read-only library hashes were unchanged from their pre-test baselines.

Exact Docker image ID: sha256:40b05e9751ca174375be9ccd84e479dc8c48ed1962141509ad0d174be0bf62cc.

The unchanged current head completed two independently framed full-PR adversarial reviews with zero confirmed findings.

  • Reviewed head: 7b769fa937d091b5ed49677e0da796f0a1813f8b
  • Git tree: 5034249e006134c8cd1a60b6367f7c998795ae10
  • Source archive SHA-256: 627e112d8dc5779dfb509899d333e373c0ef89e4c1224cd9ab24e9aa06f94b38

Notes

This is follow-up hardening for filesystem and move work introduced in #717.

Older move jobs are not silently granted filesystem permissions they did not originally record. If Listenarr cannot prove an older job is safe to continue, it requires attention instead.

Storage that cannot provide the filesystem information required for crash-safe moves and deletions is not treated as broken. Reading and scanning remain available when they can be performed safely.

No temporary or marker files are introduced into library folders.

The #815 Synology/older-kernel scenario is covered by deterministic unsupported-identity tests and native Linux validation. An actual Synology host running kernel 4.4.302 was not available for direct testing.

@therobbiedavis
therobbiedavis requested review from a team and a lite review from Copilot August 14, 2026 02:38
@therobbiedavis therobbiedavis added the patch patch version bump - backward compatible bug fixes label Aug 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac211dba8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread listenarr.infrastructure/FileSystem/PinnedDirectoryCreation.NativeOperations.cs Outdated
Comment thread listenarr.infrastructure/Library/Scanning/ScanPathAuthorizationService.cs Outdated
Comment thread listenarr.api/Features/Library/LibraryMoveWorkflow.Paths.cs Fixed
Comment thread listenarr.application/Downloads/Import/DownloadImportService.cs Fixed
@therobbiedavis
therobbiedavis force-pushed the 815-harden-filesystem-identity-and-move-recovery branch from e58cceb to 24a0284 Compare August 14, 2026 10:32
@therobbiedavis
therobbiedavis force-pushed the 815-harden-filesystem-identity-and-move-recovery branch from 7e28453 to 0573c33 Compare August 14, 2026 11:47
Comment thread listenarr.api/Features/Library/LibraryMoveWorkflow.Physical.cs
Comment thread listenarr.infrastructure/FileSystem/FileSystemSemanticsResolver.cs Fixed
Comment thread listenarr.infrastructure/FileSystem/FileSystemSemanticsResolver.cs Fixed
Comment thread listenarr.infrastructure/FileSystem/FileSystemSemanticsResolver.cs Fixed
Comment thread tests/Features/Architecture/BackendArchitectureTests.cs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

patch patch version bump - backward compatible bug fixes

Projects

None yet

2 participants