Repository navigation
Conversation
The trip-plan cache lived only in the gateway Durable Object's memory, so every eviction made the next burst of plan requests spend the shared Golemio budget again (~8 s for 20 vehicles on a cold iOS launch). Plans now also go to a trip_plans table in the DO's own SQLite storage, behind the in-memory cache: a memory miss is promoted from there with its original fetch time, so the 6 h TTL and 24 h stale fallback hold across cold starts. Only found plans are stored; rows past 24 h or beyond the newest 4096 are pruned. Storage failures degrade to a miss. The vehicle snapshot carries no stops or shapes, so it can't warm this cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
🚀 Preview deployed: https://preview-52.tablo.run |
Owner
Author
|
Superseded by #56: trip plans are now built from PID's GTFS at deploy time and served from static assets, so persisting Golemio trip answers in the gateway DO is no longer needed. |
i11v
added a commit
that referenced
this pull request
Oct 17, 2026
## Why Every `GET /api/trips/:tripId` miss spent the one Golemio token's budget (20 calls / 8 s) through the singleton gateway Durable Object. A cold iOS launch took ~8 s to load ~20 vehicle plans, and that cost grows with every user, which matters for the public release. Trip plans are static timetable data, and Golemio's trip endpoint is just a view of PID's public GTFS feed. ## What - **Build:** `build:index` (same download, one unzip) now also writes every trip's plan to `apps/web/public/data/trips/v1/<route>.json`. There is one file per route prefix of the trip id: ~900 files, ~160 MB, about 6 s and 2.3 GB of RAM. Output is deterministic, so routes that didn't change upload nothing. - **Format:** `TripBucketV1`, `tripBucketKey` and `tripFromBucket` in `packages/contract/src/trip-bucket.ts`. The builder is in `apps/worker/src/gtfs/buckets.ts`; it applies the normalization `toTrip` did (time parsing, ASW ids, rounding). - **Serve:** the Worker reads the files through `ASSETS`. It keeps up to 3 parsed files and 512 plans per isolate and validates each plan against `Trip` before serving it. Trips never reach Golemio or the gateway DO. An id the deployed feed lacks returns 404 `TripNotFound`; an unreadable or malformed file returns 503 `UpstreamUnavailable`, which isn't cached, so the next request retries. - **Removed:** the Golemio trip-plan path (`fetchTrip`, the gateway's trip cache and `getTrip` RPC, `PidTripResponse`, `toTrip` and the Golemio trip fixture). Trips have one source of truth. The GTFS helpers moved to `gtfs/buckets.ts`, and the trip tests now build from GTFS rows. - **Refresh:** `deploy.yml` also runs daily at 03:30 UTC (after PID publishes the feed) and on manual dispatch. - `scripts/lib/csv.ts` now has a lazy `iterCsv` generator; `parseCsv` is unchanged. - **Unrelated fix:** `vitest.integration.config.ts` now points at `apps/*`. The move to `apps/` left it matching no files. - `docs/adr/0001-trip-plans-from-gtfs.md` explains why I used static assets over R2, D1, KV or a Worker cron, and `docs/DEPLOY.md` is updated. The HTTP contract is unchanged (`contract:check` passes, and `wire-shape.json` and the iOS fixtures didn't change). `/api/trips/:id/vehicle` and `/api/vehicles` still go to Golemio through the gateway. ## Verification - **Against production (2026-10-17):** all 366 live vehicles' trip ids were in the feed. Plans built from it matched production `/api/trips` byte for byte on all 19 trips I compared. The other requests in that sample got 503 because I had used up production's Golemio budget. - **Unit tests:** `bun run test` passes (140 tests). - Bucket tests on GTFS rows carry over every former `toTrip` expectation: order, normalization, shape rounding and dedupe, time fallbacks. - Plan-cache tests cover a hit loading its file once, misses, a failed load reported as unavailable and retried, and malformed data reported as unavailable. - API tests check that trips never reach the gateway, that a miss is a 404 and that an unreadable file is a 503. - **Integration:** `GOLEMIO_API_TOKEN=dummy bun run test:integration` passes (4 tests). The new test gets a plan from local workerd with a dummy Golemio token, so the plan can only have come from the assets. - `typecheck`, `lint`, `fmt:check`, `build:web` and `verify:pwa` pass. No trip files end up in the PWA precache. ## Notes - **#52:** closed. Persisting Golemio plans in the DO is no longer needed. - **Behaviour change:** an id missing from the deployed feed is now a 404, where Golemio would have been asked. Every live id I checked was in the feed, and the daily deploy keeps it current. - **Size:** deploy assets grow by ~160 MB, and each PR preview uploads them on its first deploy. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
GET /api/trips/{id}was cached only in the gateway DO's memory. After an eviction, every plan the iOS app asks for spends the shared Golemio limiter again (20 calls / 8 s), so a cold first launch takes ~8 s to load ~20 vehicle plans.What
makeOutcomeCachetakes an optionalpersisttier (load/save, both never fail). A memory miss checks it and promotes the entry with its originalfetchedAt, so TTL (6 h) and stale-if-error (24 h) carry over cold starts.TripStore(src/gateway/tripStore.ts): atrip_planstable in the gateway DO's SQLite storage (state.raw.storage.sql). It stores onlyokplans (a notFound is keyed by client input and cheap to forget), prunes rows older than 24 h and keeps at most the newest 4096, and decodes rows with the contractTripschema so a future shape change is treated as a miss. SQL errors are logged and handled as misses.Tests
bun run test(138 passed) andbun run typecheckboth pass. The new tests runTripStoreon Node'snode:sqliteand build a fresh gateway over the same database to stand in for an evicted DO. They check that:I checked that the cold-start tests fail when the store is swapped for
TripStore.layerNone. I haven't run this against a deployed DO yet.🤖 Generated with Claude Code