Durable Object reactivation stalls 3–28s before JS RPC handler (minimal repro + traces) #8010

Summary

On hosted Cloudflare Workers, a minimal SQLite-backed Durable Object intermittently waits 3–28 seconds before its JavaScript RPC handler begins. One captured call took 27,844 ms at the caller / 27,888 ms at the HTTP client; the target native invocation reported 27,830 ms wall time and 0 ms CPU. The handler does only a synchronous read of one never-written key and returns diagnostic fields.

Reproduction and exported traces: https://github.com/acchou/cloudflare-do-activation-repro

This is a hosted-platform report; I have not reproduced the delay in local workerd. I am filing here because the observed boundary is Durable Object RPC/activation, with related context in #7607. I do not know the internal cause.

Minimal reproduction

Pinned Worker source. The repo contains the measured Worker logic, a standalone idle/repeat runner, and sanitized synthetic evidence. No application service bindings, user data, external database, AI, or billing code are involved.

npm ci
npm run check
npm test
npx wrangler login
npm run setup
npm run deploy
npm run probe -- --objects-per-hint 6 --cycles 10 --idle-ms 25000
npm run cleanup

The path is HTTP Worker → Caller DO → Target DO. Target.ping() reads ctx.storage.kv.get('never-written') and returns entry time, instance UUID, isolate UUID, sequence and object ID. There are no application writes, timers, external requests, or initialization tasks in that handler. A separate location probe runs outside the measured ping.

The runner uses fresh object identities with auto, wnam, enam and weur hints, rotates callers, leaves objects idle, then pairs each first call with an immediate repeat. The failure is intermittent and identity-dependent; a fresh deployment is not guaranteed to hit a slow placement.

Expected / actual

Expected: after ordinary routing and activation overhead, this minimal handler should return promptly.

Actual: the caller and HTTP client occasionally wait seconds before handler entry. In the recorded study, all 18 calls over one second were first pings on their in-memory object instances. The 1,600 later pings peaked at 583 ms. An object can reactivate in an already-observed JS isolate, so this is not necessarily a full isolate cold start.

A fresh enam replacement placed in ORD subsequently stalled on five of six controlled reopens (3,726–16,925 ms), while its immediate repeats were all ≤83 ms. A newly created caller in DFW also hit the ORD delay. The original IAD object later recovered without changing identity. Renaming therefore did not reliably solve this case.

Trace evidence

Measured 2026-10-26 20:24:17.963–20:31:43.270 UTC, paid Workers account, compatibility date 2026-04-09, Wrangler 4.151.1. Worker codex-placement-446b6c, version 0c5183b4-c380-4983-bf82-15e3274012ba. The temporary Worker and its namespaces have since been deleted. Account ID can be supplied privately if required.

For the 27.844-second example:

• Caller dispatch: 20:27:02.142 UTC.
• Target handler entry: 20:27:29.957 UTC.
• Target object: f5547a44420b5abe411ba3c4e47654228210248ad98afc31ad68338d03a11c4b.
• Caller object: d38dbb929e143dc96ead34f507eb4dce8ef1e37182c9e051c2c7407f4a553fe5.
• Probe ID: efd651f2-6ae1-44bc-90e5-2b2d2c2ec08b.

The evidence directory contains all 2,016 synthetic probe records and four compact trace exports retaining native span relationships/timestamps. Native spans corroborated the self-reported locations of all 72 targets across 15 colos. The exports omit account identifiers, credentials, IP addresses and unrelated logs.

Interpretation / question

The independent HTTP and caller timings confirm a real stall, not just a long-lived completed trace span. Cross-host entry-time subtraction is supporting evidence; I rely primarily on those same-process durations and native spans. The data locates waits before application work but does not identify storage activation, scheduling, routing, or a physical host issue.

Unlike #7607's reported delays on running objects, this sample's long delays occur on object reactivation. Could Cloudflare inspect the saved object IDs/timestamps for activation or RPC dispatch delays, and advise whether this belongs to the same platform issue or a separate one?