Skip to content

P2: per-chunk retry on transient timeout in fetch path #36

Description

@sanity

Problem

Matrix user feedback (ofansifkapital, 2026-05-07): git clone freenet::3GEERif5ihbf/freenet-core repeatedly times out at individual chunk GETs after 180s.

```
==> 6 bundle(s), 10.9 MiB total (~60s per chunk under load; up to 8 in parallel)
[3/6] downloading 3 chunks (2.7 MiB)
chunk 1/3
chunk 2/3
error: fetch ChunkedPack 8ff7da358a0c8a9f1c746b8e9e16d0c177d84210cff2706eabca09b1b2841522
Caused by:
0: GET chunk 2
1: timed out waiting for GET response after 180s
```

A single chunk timing out fails the entire clone. The next attempt redoes the whole thing (or, with #22 Phase 1 sidecar cache, just the chunks that didn't make it the first time — but that requires a previous successful attempt). In a degraded gateway window, no single attempt completes because at least one chunk in the manifest hits 180s on every attempt.

Approach

Add a per-chunk retry loop in chunked.rs's fetch path:

  • N attempts (default 3) per chunk before bubbling failure.
  • Exponential backoff between attempts (e.g. 5s, 15s, 45s).
  • On retry, optionally drop and re-open the WS connection (current connection might be stuck in a bad state).
  • Make N configurable via FREENET_GIT_CHUNK_RETRIES.

Combined with the #22 sidecar cache, this means:

  1. First attempt: cache miss → network GET → 180s timeout → retry.
  2. Retry: same cache key → still miss → network GET (maybe succeeds this time).
  3. Second clone (after success): cache hit → instant.

Test plan

  • Inject a stub fetcher that fails once then succeeds; assert retry succeeds.
  • Inject a stub that always fails; assert it fails after N attempts with the right error.
  • Assert backoff timing is bounded (don't sleep N*N total, just N+sum-of-backoffs).

[AI-assisted - Claude]

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions