Retry transient watcher run reports without new file events

[RowletCC]

A transient run-report API failure leaves an otherwise stable, single-file run unreported until another filesystem event arrives. The monitor removes the file from its pending queue, while the detector retains it and ignores duplicate callbacks.

This change schedules an in-process retry for transport failures, 429, and 5xx responses, then services due reports on the monitor's existing one-second tick even when no files are pending. It also retries failed run updates, honors Retry-After for rate limits, and leaves permanent 4xx errors for operator attention. The server upserts by instrument and run ID, so replaying an ambiguous POST does not create a second run.

The watcher package is bumped to 1.1.1 with the lockfile updated. Validation: 465 watcher unit tests passed; Ruff lint and format checks passed; git diff --check passed. Integration tests were not run because they require the shared package and server setup.

Closes #245.

[vercel]

@RowletCC is attempting to deploy a commit to the Arcadia Science Team on Vercel.

A member of the Team first needs to authorize it.

[wasimxyz]

Thanks, @RowletCC. This was a good catch. A run report that failed once could stay unreported until another file arrived or the watcher restarted, and your issue and fix described that accurately.

We've carried your change forward in #325. Your commit is the first commit there, with your authorship kept. On top of it we:

• Stop retrying after 16 retries (about 55 minutes), then record one run_report_retries_exhausted event.
• Retry only connection errors and 5xx responses, so the 429 and Retry-After code is gone. Data Hub never sends either to the watcher.
• Record only the first failure of a retry window, so a long outage doesn't fill the event queue.
• Count an unexpected error during a retry as a failed attempt.
• Move the tests next to the existing run detector tests, and bump the watcher to 1.2.2 with the docs, changelog entry, and regenerated catalogs this repo requires for watcher changes.

We're closing this one in favor of #325. Thanks again for the report and the fix.