Repository navigation
feat(watchsync): allow watch-sync batch sizes up to 1000 - #44
Conversation
Manifest validation now accepts max_batch_size from 1 to 1000 and exports the limit as manifest.MaxWatchSyncBatchSize. The proto comment and docs/compatibility.md state that the value bounds every ApplyEvents call and the ListRemoteState page size, and that hosts and catalog tooling on earlier SDKs reject values over 100. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Silo Kody — review completeReview finished. Check the inline comments for findings and verify each suggestion against the code and tests. Reviewing changes in Silo
Review settingsReview OptionsThe following review options are enabled or disabled:
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Merge Risk: ⚪ Minimal · up to
|
Problem
Related issue: N/A
A watch-sync plugin can declare at most 100 for
max_batch_size, so Silo sends it at most 100 plays perApplyEventscall. A large first sync then costs one round trip per 100 plays. MDBList takes far more per request: 1,000 real plays in one call took 2.3 seconds against its live API. At the MDBList plugin's pace of one request a second, a 13,000-play history needs about 390 requests (6–7 minutes) in batches of 100, and about 50 in batches of 1,000.Approach
Manifest validation accepts
max_batch_sizefrom 1 to 1000. The limit is exported asmanifest.MaxWatchSyncBatchSizeso a host can size its export pages from it.The field's proto comment and a new note in
docs/compatibility.mdsay what the value controls: everyApplyEventscall (watched history, watchlist, favorites and ratings) and the page size a host requests fromListRemoteState. A plugin that raises it has to handle batches and pages of that size in every flow it declares.Release impact: validation gets looser and one constant is added, so this is a minor release, v0.27.0.
Downstream, once v0.27.0 is tagged:
Validation
go test ./...: pass. A new test checks 0, 1, 100, 1000 and 1001.go vet ./...: clean.go buildof the four examples: pass.make proto: only the field comment inwatch_sync_provider.pb.gochanges, with gofmt's column realignment.gofmt -l .: lists onlypkg/pluginsdk/runtime/scan_source_test.go, whichmainalready lists.go.workon this branch, the real MDBList plugin manifest withmax_batch_size: 1000was checked against two validators. silo-server's install check (plugins.ValidateManifest) and the silo-plugins catalog tool (BuildPackageFromRelease) both accept it. On their current SDK pins (v0.26.0 and v0.21.0), both reject it withmax_batch_size must be between 1 and 100.Evidence
Evidence: https://evidence.siloserver.org/r/silo-plugin-sdk/pr-44/
Risks
Checklist
AI Disclosure
ApplyEventscall and theListRemoteStatepage size. The comments and docs were corrected.🤖 Generated with Claude Code
Note
Raise watch-sync
max_batch_sizelimit from 100 to 1000manifest.validateWatchSyncCapabilitywith the newmanifest.MaxWatchSyncBatchSizeconstant (1000), and includes the limit in the validation error messagesilo.plugin.v1.WatchSyncProviderDescriptormax_batch_sizeabove 100 now pass validation; hosts using the older 1–100 compatibility range must validate it separatelyMacroscope summarized 966f5ad.