From 955ba1d9aae85d7f8b5adfb8f3bee1317cb605cc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 20:46:16 -0800 Subject: [PATCH 1/4] quest: agents collect mbx R2 numbers with the desktop's wrangler login Wrangler on the maintainer's desktop reads bucket sizes, and its OAuth token reads R2 request counts from Cloudflare's GraphQL API. The data wait moves from the maintainer to agents; the maintainer only picks the lifecycle. Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/mbx-r2-verify.md | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/quest/m0/ci/mbx-r2-verify.md b/quest/m0/ci/mbx-r2-verify.md index 9e1ec3bc00f..1d6bd1a5385 100644 --- a/quest/m0/ci/mbx-r2-verify.md +++ b/quest/m0/ci/mbx-r2-verify.md @@ -24,6 +24,11 @@ nightly.yml --branch main --event schedule`), never a dispatch on a branch. ## Plan +Decided 2026-11-07 (maintainer, /quest-unblock): agents collect the R2 +numbers with this desktop's wrangler login, and the maintainer only picks the +lifecycle. Baseline that day: 720,249 objects / 1.2 TB, with 18,006,968 +`GetObject` and 4,892,769 `PutObject` requests from 2026-10-16 to 2026-11-07. + Decided 2026-11-05 (maintainer, /quest-audit): ✅ the release pull request check moved to [m3](/quest/m3/mbx-r2-release-cache.md) (m2 before the milestone re-cut) behind the cut, since `release` gets the platform Rust cache only when the cut merges `main` into @@ -132,12 +137,16 @@ feature unification, so changed compiler inputs remain a separate cause. reuse, restore-plus-build comparison and rebuild-frequency evidence in the Goal. Current timings are not a matched-commit comparison, and macOS's inherited-prediction misses remain unexplained. -- Maintainer: from about 2026-11-26, collect R2 bucket size and request counts - for the first month and check that the six old oversized manifests expired. - R2's account metrics need the maintainer's Cloudflare login, which agents - lack. Then decide the lifecycle in `infra/mbx/lifecycle.json`: shorten, - keep or lengthen the 30 days. Recommended: keep 30 days unless storage cost - or the rebuild count argues otherwise. +- moq-dev/moq: no earlier than 2026-11-26, an agent on the maintainer's + desktop collects `mbx-moq-dev`'s size and request counts for the first + month and checks that the six old oversized manifests expired. Size comes + from `npx wrangler r2 bucket info mbx-moq-dev`. Request counts come from + Cloudflare's GraphQL API (`r2OperationsAdaptiveGroups`, filtered on + `bucketName` and grouped by `actionType`), authenticated with the + `oauth_token` in `~/.config/.wrangler/config/default.toml`. Then put the + lifecycle in `infra/mbx/lifecycle.json` to the maintainer, with the numbers: + shorten, keep or lengthen the 30 days. Recommended: keep 30 days unless + storage cost or the rebuild count argues otherwise. ## Related From 8b06b94927e2e7d76f1db13be8d46ecd2f00eff5 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 21:41:25 -0800 Subject: [PATCH 2/4] ci(mbx): expire Rust cache objects after 14 days The maintainer shrank mbx-moq-dev's expiry from 30 days to 14, and the rule is applied to the bucket. The mbx-r2-verify quest now only collects the numbers and counts rebuilds the shorter expiry causes. Co-Authored-By: Claude Opus 5.5 --- infra/mbx/lifecycle.json | 2 +- quest/m0/ci/mbx-r2-verify.md | 26 ++++++++++++++++---------- 2 files changed, 17 insertions(+), 11 deletions(-) diff --git a/infra/mbx/lifecycle.json b/infra/mbx/lifecycle.json index bd974fc6e88..8f0b0455622 100644 --- a/infra/mbx/lifecycle.json +++ b/infra/mbx/lifecycle.json @@ -4,7 +4,7 @@ "id": "expire", "enabled": true, "conditions": { "prefix": "" }, - "deleteObjectsTransition": { "condition": { "type": "Age", "maxAge": 2592000 } }, + "deleteObjectsTransition": { "condition": { "type": "Age", "maxAge": 1209600 } }, "abortMultipartUploadsTransition": { "condition": { "type": "Age", "maxAge": 86400 } } } ] diff --git a/quest/m0/ci/mbx-r2-verify.md b/quest/m0/ci/mbx-r2-verify.md index 1d6bd1a5385..15790eea2cd 100644 --- a/quest/m0/ci/mbx-r2-verify.md +++ b/quest/m0/ci/mbx-r2-verify.md @@ -10,14 +10,13 @@ that closes this: wrote. - Restore plus build time against the 150-210s Actions restore of the ~6 GB store it replaced, from the same commit before and after. -- Each object expires 30 days after upload, so a crate no Nightly builds in +- Each object expires 14 days after upload, so a crate no Nightly builds in that window stays cold for the next hosted job. Count how often a Nightly job pays that rebuild, including when `Cargo.lock` changed more than 8 times since the previous Nightly (see Plan). - Bucket size and request counts after a month with capped manifests - (mbx 1.28.0 and later, from about 2026-11-26), against the 30-day expiry + (mbx 1.28.0 and later, from about 2026-11-26), against the 14-day expiry in `infra/mbx/lifecycle.json`. - Shorten or lengthen it from that. Use scheduled Nightly runs on `main` only (`gh run list --workflow nightly.yml --branch main --event schedule`), never a dispatch on a branch. @@ -29,6 +28,12 @@ numbers with this desktop's wrangler login, and the maintainer only picks the lifecycle. Baseline that day: 720,249 objects / 1.2 TB, with 18,006,968 `GetObject` and 4,892,769 `PutObject` requests from 2026-10-16 to 2026-11-07. +Decided 2026-11-07 (maintainer, /quest-unblock): ✅ shrink the expiry to 14 +days, applied that day with `just infra mbx lifecycle`. At R2 list prices the +30-day window was headed for about 1.65 TB (about $25 a month), and 14 days halves +that. Not chosen: keeping 30 days until the 2026-11-26 numbers; +lengthening it. + Decided 2026-11-05 (maintainer, /quest-audit): ✅ the release pull request check moved to [m3](/quest/m3/mbx-r2-release-cache.md) (m2 before the milestone re-cut) behind the cut, since `release` gets the platform Rust cache only when the cut merges `main` into @@ -48,8 +53,8 @@ The first scheduled Nightly on mbx 1.28.0, [40506938673](https://github.com/moq-dev/moq/actions/runs/40506938673) (2026-10-27), settled the bump: every hosted job reports `configured read-write, effective read-write`, and macOS refused only a manifest already -oversized before it. Six such manifests stay unreadable until the 30-day -lifecycle deletes them, the last about 2026-11-26: `a90076a4...`, +oversized before it. Six such manifests stay unreadable until the +lifecycle deletes them, within a day of the 14-day rule applied 2026-11-07: `a90076a4...`, `5cb8fb79...`, `b0310ce4...`, `18d136a6...` (17235558 bytes, first refused in [40090959108](https://github.com/moq-dev/moq/actions/runs/40090959108)), `2ed9df47...` and `0d4f4e3f...`. A refusal for any other digest means the @@ -90,7 +95,8 @@ compilation is "not looked up". - macOS: on 2026-10-28 the build did inherit, prefetching 732 predictions, yet hit only 13, with 1565 compilations not looked up. The install order is not the cause. Find out why the inherited predictions mostly miss, for - example the lockfile changes or the 30-day expiry. + example the lockfile changes or the expiry (30 days until 2026-11-07, 14 + since). - Linux: Nightly's Linux jobs check out at depth 1, so mbx finds no earlier lockfile and reuses only a manifest written for the identical `Cargo.lock`. The doctest job hit 0 on 2026-10-24 and 2026-10-27, 115 on 2026-10-25, and @@ -143,10 +149,10 @@ feature unification, so changed compiler inputs remain a separate cause. from `npx wrangler r2 bucket info mbx-moq-dev`. Request counts come from Cloudflare's GraphQL API (`r2OperationsAdaptiveGroups`, filtered on `bucketName` and grouped by `actionType`), authenticated with the - `oauth_token` in `~/.config/.wrangler/config/default.toml`. Then put the - lifecycle in `infra/mbx/lifecycle.json` to the maintainer, with the numbers: - shorten, keep or lengthen the 30 days. Recommended: keep 30 days unless - storage cost or the rebuild count argues otherwise. + `oauth_token` in `~/.config/.wrangler/config/default.toml`. Record them + here beside the 2026-11-07 baseline, and count the Nightly jobs that + rebuilt a crate the 14-day expiry deleted. Bring the lifecycle back to the + maintainer only if that count argues for lengthening it. ## Related From 1d4ea919cec334c9cac24764037bb2272688075b Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 6 Nov 2026 23:02:03 -0800 Subject: [PATCH 3/4] docs(infra): mbx lifecycle expires objects after 14 days Co-Authored-By: Claude Opus 5.5 --- infra/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/infra/README.md b/infra/README.md index b5c9e6ea4eb..fa875979750 100644 --- a/infra/README.md +++ b/infra/README.md @@ -185,7 +185,7 @@ is done for `main`. just infra mbx create ``` - `lifecycle.json` deletes every object 30 days after upload and aborts + `lifecycle.json` deletes every object 14 days after upload and aborts multipart uploads left incomplete for a day. mbx writes each compilation once and never refreshes it, so a result still in use expires too; the next writer that misses it compiles and uploads it again. After editing From 1377a112d3ced9bc1be15c3e91f7cdefaf98fd86 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 7 Nov 2026 06:25:40 -0800 Subject: [PATCH 4/4] quest: the R2 verify plan counts every writer and splits requests at the lifecycle change Co-Authored-By: Claude Opus 5.5 --- quest/m0/ci/mbx-r2-verify.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/quest/m0/ci/mbx-r2-verify.md b/quest/m0/ci/mbx-r2-verify.md index 15790eea2cd..2daae41c551 100644 --- a/quest/m0/ci/mbx-r2-verify.md +++ b/quest/m0/ci/mbx-r2-verify.md @@ -10,7 +10,7 @@ that closes this: wrote. - Restore plus build time against the 150-210s Actions restore of the ~6 GB store it replaced, from the same commit before and after. -- Each object expires 14 days after upload, so a crate no Nightly builds in +- Each object expires 14 days after upload, so a crate no hosted job builds in that window stays cold for the next hosted job. Count how often a Nightly job pays that rebuild, including when `Cargo.lock` changed more than 8 times since the previous Nightly (see Plan). @@ -30,7 +30,7 @@ lifecycle. Baseline that day: 720,249 objects / 1.2 TB, with 18,006,968 Decided 2026-11-07 (maintainer, /quest-unblock): ✅ shrink the expiry to 14 days, applied that day with `just infra mbx lifecycle`. At R2 list prices the -30-day window was headed for about 1.65 TB (about $25 a month), and 14 days halves +30-day window was headed for about 1.65 TB (about $25 a month in storage alone), and 14 days halves that. Not chosen: keeping 30 days until the 2026-11-26 numbers; lengthening it. @@ -54,7 +54,7 @@ The first scheduled Nightly on mbx 1.28.0, (2026-10-27), settled the bump: every hosted job reports `configured read-write, effective read-write`, and macOS refused only a manifest already oversized before it. Six such manifests stay unreadable until the -lifecycle deletes them, within a day of the 14-day rule applied 2026-11-07: `a90076a4...`, +lifecycle deletes them, typically within a day of the 14-day rule applied 2026-11-07: `a90076a4...`, `5cb8fb79...`, `b0310ce4...`, `18d136a6...` (17235558 bytes, first refused in [40090959108](https://github.com/moq-dev/moq/actions/runs/40090959108)), `2ed9df47...` and `0d4f4e3f...`. A refusal for any other digest means the @@ -150,8 +150,9 @@ feature unification, so changed compiler inputs remain a separate cause. Cloudflare's GraphQL API (`r2OperationsAdaptiveGroups`, filtered on `bucketName` and grouped by `actionType`), authenticated with the `oauth_token` in `~/.config/.wrangler/config/default.toml`. Record them - here beside the 2026-11-07 baseline, and count the Nightly jobs that - rebuilt a crate the 14-day expiry deleted. Bring the lifecycle back to the + here beside the 2026-11-07 baseline, split at that date's lifecycle + change, and count the Nightly jobs that rebuilt a crate the 14-day expiry + deleted, listing as unknown any it cannot attribute. Bring the lifecycle back to the maintainer only if that count argues for lengthening it. ## Related