Conversation
Queues affected Grimmory libraries on import, rename, delete and retag events and drains them via ProcessQueue after files land on disk, replacing the archived branch's in-handler sleep and all-libraries sync task. Book deletes now publish DeleteCompletedEvent after disk cleanup so queued notification work (Grimmory, Plex) drains at the right time.
Two connector toggles push Chaptarr's metadata (locking pushed fields) and cover to the matched Grimmory book on import, retag, and cover updates. A Grimmory Push toolbar button on book details and a bulk button in the book editor open a field-picker dialog driving the PushGrimmoryMetadata command; both disable without files on disk. A poller forwards edits made in Grimmory (audit log for metadata, cover stamps for covers, own-user echoes excluded) to any connection implementing the new IExternalLibraryEditTarget seam, keeping this branch mergeable without the ABS/CCS branches.
Grimmory rewrites <book>.metadata.json (and .cover.jpg) next to the book on every edit when sidecar write-on-update is enabled, so the forwarder now watches the root folders for those writes - the same mechanism the calibre forwarder uses on metadata.db - replacing the 2-minute audit-log poll and its admin requirement. Chaptarr's own pushes are filtered through a recent-push registry rather than by username, so edits made in Grimmory under the connection's account are forwarded too.
A push makes Grimmory rewrite the sidecar exactly once, so the suppression entry is spent on the first matching sidecar event; a person's edit made minutes later - previously discarded for the whole 10-minute window - forwards again.
Cover-only pushes leave no sidecar echo to consume, so recording them could swallow the next real edit made in Grimmory.
A push's echo and a person's edit made inside the same debounce window coalesced into one batch entry and were discarded together. Each filesystem event is now checked on arrival: the first event after a push consumes the entry, duplicates inside a short shadow are absorbed, and anything later is queued and forwarded.
The mount guard skipped cleanup whenever a scan found no media files, so a granular scan of a book folder whose only file was deleted externally (e.g. removed in Grimmory) never pruned the tracked rows and the deletion never reached connections. An empty subfolder scan now cleans up when the root folder itself provably has content; root-level empty scans keep the guard.
Grimmory never updates a locked field, even for the writer who locked it, and applies a request's values before its lock flags - so the first pushed value froze forever and corrected re-pushes silently no-oped. Each push now clears its target locks via toggle-field-locks first; the update itself re-locks them. Publish date also prefers the monitored edition's release date over the book's.
Reverts the unlock-before-write pass: a locked field in Grimmory now stays exactly as locked, including against Chaptarr's own re-pushes. Updating a locked value means unlocking it in Grimmory first. The monitored-edition release date preference stays.
Restores the archived quickstart section; the card drives the standard notification modal off the schema, so it picks up the current library dropdowns and push/forward toggles as-is.
Grimmory now implements IExternalLibraryEditTarget, so an edit made in one instance reaches every other configured instance alongside the ABS and content server targets; the forwarder already excludes the source. Applies descriptive fields and covers per each connection's push toggles, resolves the book by root-relative path in that connection's own libraries, respects its locks, and records the push so the target's own sidecar rewrite is absorbed instead of ping- ponging - Grimmory's no-change detection ends the chain once values converge.
Grimmory writes its sidecar while the update request is still in flight, so the filesystem event could reach the forwarder before the registry entry existed and get forwarded back out as a real edit.
Strip the reviewer-facing narration from the Grimmory connector, keeping only the comments that record an external constraint: Grimmory rewriting the sidecar during a metadata update, its async refresh, its handling of locked fields, and the AudioBookShelf rescan the forwarder debounces past. Drop IGrimmoryProxy.GetLibraryBooks, which no caller outside the proxy used, and inline the single-use edition lookup in GrimmoryPushService. File the two Grimmory push strings in their alphabetical place in en.json instead of the middle of the GoTo* run, and translate the push dialog's field labels the way QuickstartMatchingSection does rather than hardcoding English. Publish Date reuses the existing PublishedDate key.
Grimmory rewrites its sidecar in response to a push and the forwarder drops that event as an echo, so Audiobookshelf and the other connections never saw covers or metadata that were pushed from Chaptarr.
Those three were written without their lock flag, so Grimmory's own metadata refresh could overwrite them. The cover is uploaded before the lock is set because Grimmory rejects an upload outright once the cover is locked, and an already-locked cover is left alone for the same reason.
A file adopted by a disk scan raises BookImportedEvent with NewDownload false, which NotificationService dropped before any provider saw it. A book added through Grimmory and picked up by a scan therefore got no library refresh and no metadata or cover push until something else touched it. Add the NotifyOnLibraryImports opt-in to INotification and NotificationBase and let NotificationService deliver library imports to the providers that declare it, along with an OnLibraryFileAdded hook for providers that work per file. Grimmory declares it whenever it is already configured to push, so a scan pickup goes through the existing OnReleaseImport path: refresh the matching library, then push the selected fields once Grimmory's own scan has the book. The three shared notification files are byte-identical to the ones on feature/calibre-content-server-connector so the two branches still merge without conflict.
Add an Ignore Tags setting. Automatic metadata and cover pushes - after an import, a retag or a cover update - and edits forwarded from other connections now skip any Grimmory book that carries one of those tags, matched case-insensitively. Library refreshes are unaffected. The Grimmory Push dialog and Push Chaptarr Metadata to Grimmory queue their command manually, and those pushes still update tagged books. A skipped book is still mirrored to the other library edit targets, which apply their own ignore rules. The mirrored payload carries a new Manual flag on ExternalLibraryEditPayload, so a manual push reaches tagged items there too. Grimmory's book list already returns each book's tags, so no extra request is needed.
|
Bumping this with test data. You mentioned it needs a lot of testing, and I've had Grimmory running against my real library, so I went through both sides of the contract rather than just the Grimmory one. I found something that won't work. Multi-file audiobooks don't match, and the push skips them without saying anything. FindBookByPath compares full relative paths. For a folder-based audiobook Grimmory reports the folder rather than a file, and for that same audiobook Chaptarr reports one row per track, so the two sides hand you: Those can't be equal. FindGrimmoryBook walks every track of the book, matches none, returns null, and PushBook logs "not found in Grimmory library ... skipping". No metadata and no cover go across. What's actually in the two databases for the same 102-track book. Grimmory's book_file, one row for the whole folder: Chaptarr's BookFiles, one row per track: On my library that's 45 audiobook editions, 24 single-file and 21 multi-file (2 up to 102 tracks), against the matching 24 non-folder and 21 folder-based entries in Grimmory. The 21 multi-file ones fail, about half my audiobooks. Every ebook matches, because ebooks are one file per edition on both sides. I haven't run the connector, so read this as the two databases rather than watching the push fail. But the comparison can't succeed as written. The fix is small, and Grimmory already gives you the flag for it. Its BookFile DTO carries a folderBased boolean and GrimmoryBookFile doesn't deserialize it. Reading it lets you switch strategy for those rows and leave everything else alone. public class GrimmoryBookFile
{
[JsonProperty("fileName")]
public string FileName { get; set; }
[JsonProperty("fileSubPath")]
public string FileSubPath { get; set; }
+ // Grimmory sets this for multi-file audiobooks, where FileName is the folder itself.
+ [JsonProperty("folderBased")]
+ public bool FolderBased { get; set; }
+
public string RelativePath()
{
return FileSubPath.IsNotNullOrWhiteSpace() ? $"{FileSubPath}/{FileName}" : FileName;
}
} public GrimmoryBook FindBookByPath(GrimmorySettings settings, long libraryId, string relativePath, bool bypassCache = false)
{
var normalized = NormalizeRelativePath(relativePath);
if (normalized.IsNullOrWhiteSpace())
{
return null;
}
return GetLibraryBooks(settings, libraryId, bypassCache)
- .FirstOrDefault(b => b.AllFiles().Any(f => NormalizeRelativePath(f?.RelativePath()) == normalized));
+ .FirstOrDefault(b => b.AllFiles().Any(f => MatchesPath(f, normalized)));
}
+ private static bool MatchesPath(GrimmoryBookFile file, string normalizedPath)
+ {
+ var candidate = NormalizeRelativePath(file?.RelativePath());
+
+ if (candidate.IsNullOrWhiteSpace())
+ {
+ return false;
+ }
+
+ if (file.FolderBased)
+ {
+ // Chaptarr reports one row per track; the folder is the track's parent directory.
+ var separator = normalizedPath.LastIndexOf('/');
+
+ return separator > 0 && candidate == normalizedPath.Substring(0, separator);
+ }
+
+ return candidate == normalizedPath;
+ }I compiled this matching logic standalone against net10.0 and ran the current and proposed comparison GrimmoryLibraryChangeForwarder has the same problem in the other direction, in ResolveSidecarBookFile. It matches a sidecar named after the book against a Chaptarr file whose filename equals that name, and for a folder audiobook the tracks are named after the book plus a number, so the base names never match either. I don't know whether Grimmory writes that sidecar inside the folder book or beside it, so I'm not confident enough in the shape of the fix to hand you a diff for it. If it lands inside the folder, comparing the directory name is enough: return _mediaFileService.GetFilesWithBasePath(directory)
- .FirstOrDefault(f => f?.Path.IsNotNullOrWhiteSpace() == true &&
- Path.GetFileNameWithoutExtension(f.Path).Equals(baseName, StringComparison.OrdinalIgnoreCase));
+ .FirstOrDefault(f => f?.Path.IsNotNullOrWhiteSpace() == true &&
+ (Path.GetFileNameWithoutExtension(f.Path).Equals(baseName, StringComparison.OrdinalIgnoreCase) ||
+ Path.GetFileName(directory).Equals(baseName, StringComparison.OrdinalIgnoreCase)));Two things I did verify, so you don't have to take the rest on faith. The endpoints exist on v3.5.0. /api/v1/libraries returns 401 and /api/v1/auth/login returns 405 to a GET, so the POST login is right. I couldn't confirm the other methods without credentials, because Grimmory checks auth before routing and answers 401 for every method. Forward Edits needs a warning in its help text. It depends on Grimmory's sidecar "write on update" setting, and DISK_TYPE=NETWORK disables that setting while replacing the metadata persistence panel with a notice, so the feature fails with no visible cause. One more thing worth knowing while testing. Grimmory stores folder audiobook duration at exactly 2x for low-sample-rate stereo MP3s, which is 5 of my 21 folders. I'm reporting that to Grimmory separately. It doesn't affect the connector's logic, but don't use duration as a validation signal. I don't have a build, so all of the above is from reading the two databases and the two functions. Point me at a branch or an image and I'll run it against this library and report what it actually does. Environment: Chaptarr develop on Unraid (Postgres), Grimmory v3.5.0 on Unraid, both seeing /mnt/user/data/media. I drafted this with AI assistance (Hermes Agent, Nous Research), which also ran the comparison. I've checked the numbers and the reasoning myself. |
Awesome! Thanks for testing. I didn't test audiobooks too throughly in Grimmory. I use Audiobookshelf myself. I'll take a look and verify this week hopefully. Running a modified Chaptarr that combines all multi-file audiobooks to into one file so that might have affected contributed to some missed stuff. |
Grimmory tracks a multi-file audiobook as a single folder-based entry, while Chaptarr tracks one file per track, so the exact relative-path comparison never matched and every multi-file audiobook was skipped by pushes and forwarded edits. A folder-based entry now matches any Chaptarr file inside its folder. Grimmory writes such a book's sidecar beside the folder, named after it with the name trimmed at its last dot, so the sidecar resolver accepts tracks inside a matching folder as well as a file of the same base name. Forward Edits help text notes that Grimmory only writes sidecars with DISK_TYPE=LOCAL.
|
@SaxxyToo Verified your findings. Applied some fixes to address them. If you could retest with your audiobooks I'd appreciate it! This is the branch link: |
|
Retested on a real-library build — the matching fix works, every audiobook now resolves. One thing left: folder-based books now trip on a Grimmory-side 500 during cover upload, before metadata goes across. I built your branch (7272128, on top of v0.9.965) and ran it against a locked-down clone of my production database on a separate box, pushing to the same Grimmory v3.5.0 instance I reported against last time. The push was invoked exactly the way the UI invokes it (PushGrimmoryMetadata, manual, all fields). Matching — fixed. All 44 audiobooks with files resolved to their Grimmory counterparts, the 21 folder-based ones included — those are the ones that previously couldn't match and got skipped without a word. Zero "not found" skips this time. Spot pairs: This Inevitable Ruin 9890→224, The Butcher's Masquerade 9887→196, The Way of Kings 10342→185, Red Rising 10542→169. All 44 mappings land on the right books — 44 distinct Grimmory entries; the 45th audiobook there ("Iron Gold Part 1", dramatized) has no counterpart in my Chaptarr library. Single-file flow — clean. 23 of the pushes completed end to end; all verified on the Grimmory side — fields replaced and locked, covers uploaded. Three of those had no local cover file in Chaptarr, so the connector skipped the cover by design and pushed metadata only — correct behavior, and the only three where that happened. Folder-based flow — one blocker left, and it's server-side. All 21 matched and proceeded to the update stage, but every single one then failed on this call: Grimmory's log per book: it writes the storage cover and a cover.jpg into the audiobook folder, then throws: Deterministic: 21/21 folder-based books fail, 0/23 single-file books fail. Looks like the folder-based path — where Grimmory reports the directory as the book's file — trips a regular-file check after the writes. Because PushBook uploads the cover before it calls UpdateBookMetadata, the exception aborts that book entirely, so the metadata (the part audiobooks most need) doesn't go across either. I isolated it: pushing one of these same books with fields=["title"] (cover skipped) succeeds immediately — "Pushed 1 of 1", lock applied as designed. So the metadata write path is fine for folder books; only the cover call trips. Suggested fix: make the cover upload non-fatal per book (catch, warn, continue to the metadata write), or send metadata before the cover. Either way folder-based audiobooks get their metadata on this branch, and covers land once the endpoint is fixed — I'll raise the Grimmory-side bug on their tracker too, it isn't connector-specific. One small heads-up. The connection needs at least one trigger flag enabled: NotificationDefinition.Enable is derived from those flags, and with none set the connection drops out of GetAvailableProviders() — manual pushes then no-op with only a debug line (the command "completes" in about 10 ms). I hit this while configuring; switching on OnReleaseImport brought it up. Might be worth a validation message on save or a note in the setup docs. Forwarder. I didn't exercise the sidecar path at runtime (my test box has no media files for the watcher to see). By inspection, ResolveSidecarBookFile now handles a folder book's sidecar via IsInsideBookFolder, which covers the case I flagged — but it's inspection, not a live pass. Happy to retest that path if you want it verified. No migrations in the branch, and I saw no regressions anywhere else: ebooks spot-checked 3/3 (The Martian, Project Hail Mary, Dark Matter — cover + metadata both landed). Environment: branch at 7272128 built into a container; Postgres database cloned from production on a second Unraid box, with all indexers, download clients, import lists and notifications unplugged; Grimmory v3.5.0 with the same libraries as before. Push invoked as the UI does it — manual, all fields. I drafted this with AI assistance (Hermes Agent, Nous Research), which also built the test instance and verified the results against both databases. I've checked the numbers and the reasoning myself. |
Grimmory keeps a separate cover for books whose primary file is an audiobook, with its own lock, upload endpoint and media route, and its UI shows only that one for them. Covers for audiobooks were going to the book cover slot, which the UI never displays for them, and the upload failed outright for folder-based audiobooks because that endpoint's writer hashes the book path as a file. Cover upload, lock check, lock field and cover fetch now follow the matched Grimmory book's primary file type. A rejected cover upload no longer aborts the push: the metadata still goes across and the book is only counted as pushed when something landed.
A notification only counts as enabled when one of its event triggers is on, so a Grimmory connection configured purely for pushes or edit forwarding dropped out of the available providers and manual pushes did nothing. The three Grimmory toggles now enable the connection on their own.
Should be addressed by the most recent commits, if you'd like rebuild from the branch head and check the audiobook card in Grimmory's UI rather than the cover file on disk, since the visible cover is the thing that changed. The folder audiobook should now show both metadata and cover. |
|
Retested at the branch head ( I built Folder-based covers — fixed. Last time 21/21 folder books died on Metadata — landed for all 44. Every one shows Covers overall: 40 of 44 audiobook covers uploaded, with The 4 without a cover are all explained, and none is the connector:
Enable-by-toggles — verified. With every trigger flag off ( No migrations in the branch ( Environment: branch head I drafted this with AI assistance (Hermes Agent, Nous Research), which also built the test instance and verified the results against both databases. I've checked the numbers and the reasoning myself. |
2 participants
Description
Adds a Grimmory connection: a notification provider that refreshes Grimmory's libraries after
Chaptarr imports, renames, retags or deletes files, optionally pushes Chaptarr's metadata and
covers into Grimmory, and optionally forwards edits made in Grimmory back out to other
connections.
Library refreshes are queued at event time and drained in
ProcessQueue, per media type, so aburst of imports collapses into one refresh per library. Pushes are opt-in per connection
(Push Metadata / Push Covers) and lock the fields they write in Grimmory so its own metadata
refreshes don't overwrite them; a manual push dialog on Book Details and the book editor lets you
choose which fields to send.
Edit forwarding watches the root folders for Grimmory's sidecar writes (
.metadata.json/.cover.jpg) rather than polling, since Grimmory's database is remote. Chaptarr's own pushes alsorewrite the sidecar, so a small registry absorbs that echo and forwards only edits a person made.
Targets are discovered through
IExternalLibraryEditTarget, so no concrete provider is referencedand other connections can opt in independently.
Also includes a scan fix: an empty scan result for a subfolder of a root that provably still has
content is treated as a real delete rather than a dropped mount, so a book pruned from disk by an
external library app stops being tracked forever.
Fixes
Not a fix but resolves: #99
External Related PRs
grimmory-tools/grimmory#2567
This PR is not a blocker for this just the cover art won't get backed into the file by Grimmory. It's a small change. Confident it will get merged.
Database Migration
NO. Connection settings serialize into the existing
Notifications.SettingsJSON column. The newOnLibraryFileAdded/SupportsOnLibraryFileAddedmembers are interface-level only.How was this tested?
Docker on Ubuntu against Grimmory, plus a Calibre content server and AudioBookShelf
configured as sibling connections to receive forwarded edits. Edit forwarding requires Grimmory's
sidecar "write on update" setting to be on. Also see the External Related PRs
Verified:
Note: Delays between the systems are to be expected. Grimmory's library refresh is asynchronous,
and the edit forwarder debounces for 90s before sending, so several of these scenarios need a wait
rather than an immediate check.
Per-media-type library refresh
Import an ebook, then an audiobook, with both libraries selected on the connection.
Expected: each import refreshes only its own Grimmory library. Nothing is sent at event time —
refreshes queue and drain on
ProcessQueue, so a burst of imports collapses into one refresh perlibrary rather than one per file.
Book added in Grimmory is adopted by the scan
Add a book through Grimmory so the file lands in the shared root folder, then run a Chaptarr scan.
Expected: the file is adopted and mapped to the right book/edition if it matches one, otherwise it
shows on the unmapped files page for manual matching. Grimmory already holds the file, so no
refresh is needed for it to appear there.
Book deleted in Grimmory goes back to missing and redownloads
Delete a book in Grimmory so its file leaves the shared storage, then run a Chaptarr scan.
Expected: the file row is removed as MissingFromDisk and the book stays monitored — it drops back
to missing and is eligible for search again, rather than being unmonitored. An empty scan result
for a subfolder of a root that provably still has content is a real delete, not a dropped mount,
so only a root-level empty scan keeps the mount guard. Before this, the row lingered forever,
Chaptarr still believed it had the file, and the book was never re-searched.
Rename, retag, and delete trigger refresh
Rename an author's files, retag a book, delete a book with files, delete an author with files.
Expected: each queues a refresh for the right library. Deletes now drain the queue —
MediaFileDeletionServicepublishesDeleteCompletedEventon book deletes, which it previouslyonly did for author deletes, so a book delete no longer left the refresh sitting in the queue.
Field selection dialog
Open Grimmory Push on a book page.
Expected: ten selectable fields — Cover, Title, Author, Series, Description, Publisher, Published
Date, Language, Tags, Identifiers — each with the value that will be sent shown beside it. All are
ticked by default. Push is disabled if you untick everything. The /books bulk dialog is identical
minus the previews.
Push writes only what it has
Push all fields on a book with no description, publisher or genres in Chaptarr.
Expected: empty fields are skipped, not sent — whatever Grimmory already holds survives. Only the
fields with values appear in the request.
Pushed fields come back locked
Push title and description, then look at the book in Grimmory.
Expected: both land and both are locked there, so Grimmory's own metadata refreshes won't
overwrite them.
Re-push respects an existing lock
Change the description via an edition change in Chaptarr and pushing again without unlocking anything in Grimmory.
Expected: the field does not change — Grimmory skips locked fields even for the writer that locked
them. Unlock it in Grimmory, push again, and the new value lands. Locks stay authoritative.
Auto-push waits for Grimmory's scan
Enable Push Metadata and Push Covers, then import a new book.
Expected: the push is queued behind the refresh and retries until the book appears in Grimmory —
up to 90s, re-fetching the library list every 10s rather than trusting a cached one. Without the
wait the push lands before Grimmory has ingested the file and finds nothing.
Cover change in Chaptarr auto-pushes
Change a book's cover in Chaptarr with Push Covers on.
Expected: the new cover reaches Grimmory. Author-level cover events are deliberately ignored —
they fire during routine author refreshes and would otherwise fan out into a push for every book
of that author. Repeat pushes for the same book are suppressed for 5 minutes.
Bulk push over a selection
Select several books on /books, push.
Expected: one command carries all ids; each book pushes independently, and a failure on one does
not abort the rest.
Edit made in Grimmory is forwarded out
With Forward Grimmory Edits on, change a description or cover in Grimmory.
Expected: after the debounce, the edit reaches the other connections that accept library edits —
the Calibre content server and AudioBookShelf here. Identity fields stay Chaptarr's; only the
descriptive fields mirror. Nothing is written to Chaptarr's own database.
Chaptarr's own push is not echoed back
Push metadata to Grimmory with forwarding on.
Expected: Grimmory rewrites the sidecar in response to the push, and that write is recognised as
Chaptarr's own and dropped.
A real edit immediately after a push still forwards
Push to Grimmory, then edit the same book in Grimmory before the debounce window closes.
Expected: the push's echo is consumed once and the person's edit still comes out of the same
batch. This is the case that a per-batch decision would have collapsed — echo and genuine edit
coalescing and being discarded together.
Cover-only edit in Grimmory
Replace only the cover in Grimmory, touch no other field.
Expected: the cover reaches the sibling connections as image bytes, not as a URL, so a target that
cannot authenticate against Grimmory still gets it.
Sibling Grimmory connection receives the edit
Configure two Grimmory connections, edit a book in the first.
Expected: the second one receives the forwarded edit and converges; the source connection does not
receive its own edit back.
Screenshots (UI changes only)
Quick Start config option:
Connection config option:
Books ulk push 'Push Chaptarr metadata to Grimmory' button:
Books bulk push 'Push Chaptarr metadata to Grimmory' dialog after clicking:
Book details 'Grimmory Push' button:
Book details 'Push Chaptarr metadata to Grimmory' dialog with preview after clicking:
A note on AI: We know AI/agentic coding is everywhere and only getting
more popular. We won't insist that you disclose whether you used it or which
models you used, but in the same spirit, please don't take offense if your PR
is scrutinized and changes are requested.
Review time: The longer the PR and the more lines changed, the longer the
review will take. Small, focused PRs merge fastest. If yours is big, please be
patient.