Repository navigation
fix(ci): stop three gates from reddening unrelated pull requests - #14518
Conversation
UX Review (Fable 5) — ✅ PASSUX-level review of UX-Verdict: PASS Nothing to check. [UX-REVIEWED] 60309b3 |
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Confirm the six new [DESIGN-REVIEWED] 60309b3 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Confirm the pidfd equivalence test green on a fleet runner that refused What this change shipsInventory (10 items) — 10 justifiedIntent: FIX — a contributor's PR shows only their own failures instead of three red lanes inherited from main.
Verified by grep: all 10 [FIRST-PRINCIPLES-REVIEWED] 60309b3 |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
7a20784 to
a2a184a
Compare
a2a184a to
01783f7
Compare
Three gates on main were failing on every open pull request at once, on three separate lanes, for reasons none of those branches contained. Measured at 18:00-18:25 UTC: Electron Shell Tests red on 9 of the 10 most recently updated open PRs, Frontend Tests (2) on 6 of 10, Backend Tests (3.12, 2) on 4 of 7 that carry the test at all. The one PR green on all three sits 75 commits behind main, which is to say it predates the breakage. They are three different shapes of one mistake -- a deterministic gate whose bill goes to a stranger -- so each is fixed at the shape, not just at the instance. bn numerals: a count ceiling with no diff-scoped companion. Two values carrying Bengali digits landed in separate PRs (app.metrics_card_old_data, pages.membersPage.create_name_unusable); the second crossed the ceiling of 8 and the next round reported a key neither author had touched. Both values now use Western digits per style/bn.md §2, which returns the count to 7, and the numerals rule gains the [changed-values] half that §5's register check in the same file already had: the count keeps guarding the inherited catalog while the values a branch writes are held at zero, so the failure names the key and its owner. docs/ci/i18n-gates.md already required exactly this pairing for the generated ledger; the rule now says an inline ceiling is bound by it too. Shipping that rule while leaving its own siblings unpaired would be a point patch against its stated cause, so every other companion-less ceiling under website/src/i18n/style/ is paired in the same change -- bnStyle's dari (30), hiStyle's purna viram (30), ruStyle's guillemets (20), deStyle's English compounds (12) and frStyle's accented capitals (11). The de and fr word-pair lists are hoisted to their describe so the ceiling and its companion cannot come to disagree about what a violation is; the rest reuse the changedXxValues() helper each file already carries. All 10 ceilings in that directory now have one. Port-owner registration pin: a restart-handoff identity check (successorOwnsPort) and an adopt-refusal arrived without being declared, leaving one probe call and seven sentinel literals unclaimed. Six sites are registered, all in IDENTITY_SITES, because each decides WHO holds the port rather than whether anything does. This pin scans the whole tree, so local-gate's diff relevance does not select it -- which is how the sites merged green; website/docs/testing.md now says so and names the two front-end gates with that property. pidfd ctypes fallback: the test asserted a HOST CAPABILITY instead of gating on one, so it failed wherever the capability was absent. It is restructured to need no capability gate at all rather than to skip on a boolean, because a boolean conflates "this host cannot" with "the fallback is broken": - the fallback assertion is an EQUIVALENCE -- with the wrappers deleted, _pidfd_open must answer exactly what os.pidfd_open answered for the same pid, both a descriptor or both the same errno. That holds on every Linux host, so nothing vacates: a container refusing pidfd fails both paths identically and the assertion still runs, a fallback that cannot resolve syscall raises a non-OSError, and a wrong syscall number answers a different errno; - where there is no wrapper to compare against -- the python-build-standalone interpreters the fallback exists for -- the oracle is a raw pidfd_open(2) whose syscall number this test file states. That is deliberately not a copy of the code: the module reads its own _SYS_PIDFD_OPEN, so comparing against a number stated here is what pins that constant, and accepting "some errno" instead would leave a wrong constant green on the one host class with no second path to catch it; - the end-to-end half, which genuinely needs the kernel to grant a descriptor, is its own test carrying this file's existing @Reaping marker, so the precondition is declared where the file already declares it. That the CI red was a host gap rather than a break is measurement, not inference: the failing shard read 22089 passed / 1030 skipped / 1 failed against a passing 22094 / 1026, so four capability-gated tests stood aside there and only the one that forgot to failed. Verified red-first: reverting the bn catalog values reddens the new [changed-values] assertion while the count ceiling stays silent; removing any one registered site reddens the pin naming that line; planting a compound or an unaccented capital reddens the new de and fr companions naming the key.
01783f7 to
60309b3
Compare
Problem / Motivation
Goal: a contributor opening a pull request sees only their own failures, instead of inheriting three red lanes from
mainthat name files they never touched.no linked issue: found by attributing three CI reds on an unrelated pull request
(#14327) rather than from a filed report, so there is no issue to close on merge.
Three gates on
mainare failing on every open pull request at once, on threeseparate lanes, for reasons none of those branches contain. Measured 18:00-18:25
UTC by reading the head check-runs of the most recently updated open PRs:
Electron Shell Testsgateway-supervisor.jsclaimed by no listFrontend Tests (2)bnStyle.test.ts: 9 Bengali-digit values against a ceiling of 8Backend Tests (3.12, 2)test_oneshot_supervisor.py,assert supervisor.can_reap()The one pull request green on all three sits 75 commits behind main — it
predates the breakage, which is the confirmation rather than a counterexample.
Why it matters
Every one of those authors now has to prove a red is not theirs before they can
read their own result, and the fastest honest way to do that — reproduce on a clean
base worktree — costs an hour. The bill for a merge lands on strangers, repeatedly,
until someone happens to fix it.
Backend Tests (3.12, 2)is worse than the othertwo because it is fleet-dependent: it passes on some runners, so a rerun can look
like a fix and the breakage survives.
These are not flakes. Each is deterministic given its input; what each gets wrong is
who it reports to.
Not a goal
Not clearing the inherited debt behind the numerals ceiling. Seven values in the
catalog still carry Bengali digits; they are the baselined inheritance and are not
this change's to rewrite. The ceiling stays at 8 rather than being tightened to 7,
because branches are in flight against it and a tightened ceiling would move the
bill rather than remove it — the diff-scoped half added here is what stops a new
violation from ever reaching the count.
Not closing the weakness the port-owner pin exists to record (security: POSIX port-owner classification trusts argv0, so a same-user process can present itself as ours #13826 / Bind the local mint identity to its connection by moving it onto the unix socket #13862).
The six sites are registered as IDENTITY sites because that is what they are; that
an identity verdict can rest on the holder's argv is unchanged and still tracked.
Not auditing the fleet. Why some runners refuse
pidfd_openis a host question;this change makes the test stand aside where the capability is absent, which is
what the rest of the file already does.
Not the
Bundle Size Gate, which is red on 6 of 12 other open pull requests andis therefore a fourth gate of the same character. It is a different KIND of
defect:
assets/t-*.js(the i18n runtime chunk) is 28 B past a 955 KB ceilingwhose own comment records it as "measured ~909 KB … (~5% headroom)", so main's
chunk has grown ~46 KB and spent that headroom. The sanctioned remedies are a
lazy-import boundary, a
codeSplittinggroup, or a re-measured ceiling justifiedby whatever grew — a product bundle decision, not a test-gate shape, and not one
to smuggle in under a PR about who a gate bills. Raising a budget to clear a red
is the exact move the repo warns against, so it gets its own change.
What changed (motivation → approach → change)
bnnumerals — a count ceiling with no diff-scoped companion.expect(bad.length).toBeLessThanOrEqual(8)tolerates inherited debt, which is fine,but it cannot say whose violation it is: a branch that adds one rides under the
ceiling and the round that finally reds belongs to whoever runs next. Two values
crossed it —
app.metrics_card_old_data(৯০) andpages.membersPage.create_name_unusable(৫০০), added in separate PRs. Both now useWestern digits per
style/bn.md§2, returning the count to 7, and the numerals rulegains the
[changed-values]half that §5's register check in the same file alreadyhad: the count keeps guarding the inheritance while the values a branch writes are
held at zero, so the failure names the key and its owner. Nothing is stored, so
two i18n branches still have no ledger line to conflict on.
The three sibling ceilings, paired in the same change. First Principles was
right that shipping the rule while leaving its own siblings unpaired is a point
patch against its stated cause — one of them in the very file this edits. So
bnStyle.test.ts's dari ceiling (30),hiStyle.test.ts's purna-viram ceiling (30)and
ruStyle.test.ts's guillemet ceiling (20) each gain the same[changed-values]companion over the same rule, using the
changedXxValues()helper each file alreadycarries. First Principles then counted the directory and found the claim overstated —
deStyle.test.ts's English-compound ceiling (12) andfrStyle.test.ts'saccented-capitals ceiling (11) were still unpaired — so those two are paired as
well, with their word-pair lists hoisted to the
describeso the ceiling and itscompanion cannot come to disagree about what a violation is. All 10 count ceilings
under
website/src/i18n/style/now carry one.Port-owner registration pin — new sites never declared. A restart-handoff
identity check (
successorOwnsPort) and an adopt-refusal arrived ingateway-supervisor.jswithout being registered, leaving one probe call and sevensentinel literals unclaimed. Six sites are added, all to
IDENTITY_SITES, becauseeach decides who holds the port rather than whether anything does — the
distinction the pin exists to keep. The pin was not weakened and no site was
reclassified.
pidfd ctypes fallback — a test that asserted a host capability.
assert supervisor.can_reap()is a claim about the runner, not about the code undertest, so it failed wherever the capability was absent instead of standing aside.
Two rounds of GPT 5.6 findings on this one site moved it from "skip on a boolean" to
a shape with no capability gate at all, which is what the lane was asking for and
is a better test than either earlier version:
_pidfd_openmust answer exactly whatos.pidfd_openanswered for the same pid —both a descriptor, or both the same
errno. That holds on EVERY Linux host, sonothing vacates: a container that refuses pidfd fails both paths identically and
the assertion still runs, a fallback that cannot resolve
syscallraises anon-
OSErrorand fails, and one passing a wrong syscall number answers a differenterrnofrom the wrapper and fails. On an interpreter with no wrapper to compareagainst — the python-build-standalone case the fallback exists for — it asserts the
call REACHED the kernel rather than having died inside ctypes.
SIGKILL— genuinely needs the kernel to grant a descriptor, so it is now its owntest carrying this file's existing
@reapingmarker. The precondition is declaredwhere the file already declares it instead of being inferred from a boolean that
conflates "this host cannot" with "the fallback is broken".
Docs, in the same commit as the code they cover.
testing-conventionsgains"A gate that reds on someone else's pull request": the three shapes, their fixes,
and the two diagnostic techniques used here — probe other PRs' heads before
believing a red is yours, and diff the passed/skipped counts between a passing and
a failing run of the same shard to tell a capability gap from a real break.
i18n-gatesalready required a diff-scoped companion foran upward-only ratchet; the rule now states that an inline
toBeLessThanOrEqualceiling is bound by it too, and why an inline one is strictly worse without it.
website/docs/testing.mdnames the two front-end gatesthat scan the whole tree, so a diff-scoped local run does not select them.
Three rounds on one file, and why each was fixed rather than escalated
GPT 5.6 raised a blocking finding on
test/test_oneshot_supervisor.pythree times,which is the third-recurrence threshold, so the retrospective came before the edit.
Each round names a DIFFERENT construct, and each construct was introduced by the
previous round's own fix:
can_reap()skip I added hid a fallback break on wrapper-less interpreters;wrapper-present host whose kernel refuses pidfd;
native is Nonebranch accepted any non-zero errno, so a wrong syscall number stayed green on the
one host class that has no second path to catch it.
That is a new site each round rather than one mechanism failing three times, and the
test got strictly stronger at every step — a skip, then a narrower skip, then no
skip at all. The remaining remedy is bounded exactly as the adjudication described
it: one independent raw
syscall(434)comparison inside the existing branch, no newmodule state and no cross-module contract. So the restructure trigger does not
apply, and the fix belongs here rather than in an escalation.
The oracle is deliberately NOT a copy of the code under test: the module reads its
own
_SYS_PIDFD_OPEN, so comparing against a syscall number the test file states iswhat pins that constant. Assuming it instead would leave a wrong constant green,
which is the finding.
Backwards compatibility
Compatible. No production code path changes: the diff is two catalog strings, two
test files, one test-registration list and three docs. The two catalog values are
user-visible Bengali strings whose digits change script (৯০ → 90, ৫০০ → 500), which
is what
style/bn.md§2 requires of them.Tests
npx vitest run src/i18n— 44 files pass withI18N_BASE_REF=origin/mainset, sothe diff-scoped halves actually evaluate rather than skipping.
node --test website/electron/test/port-owner-self-asserted-identity.test.js—7/7.
pytest test/test_oneshot_supervisor.py test/test_kiro_prerequisite.py— 214passed, 30 skipped. Both pidfd tests collect and stand aside on this Windows
host (
sys.platform != "linux"), which is why the equivalence argument below isstated as a construction rather than as a local measurement.
npm run build,npx tsc -p tsconfig.app.json,eslint,npm run jscpd(0 clones),
check_black_formatting,isort,flake8,mypy(1738 files),check_comment_history,check_subprocess_encoding,docs_lint— all clean.Red-first, each production hunk reverse-patched and the test observed red, then
restored:
[changed-values]assertion reds and namespages.membersPage.create_name_unusable— while the count ceiling stays silent at 8, which is the whole pointdevalue given "Slack Integration"terminalTab.rename: should be 'Slack-Integration'frvalue given "Etat"terminalTab.rename: has 'Etat' not 'État'Manual verification
Two things this Windows host cannot prove, stated plainly rather than implied:
sys.platform != "linux"), so its fix is notexercised locally. The diagnosis is measurement, not inference: the identical test
passes on PR fix: bound design tweak startup log reads #14285's and PR fix(kiro): reap helpers one-shot kiro-cli calls leave behind #14229's heads and fails on fix(dashboard): fence folder project bindings and refuse UNC project paths #13335's, feat(messaging): queue or steer into a busy bound session on every channel #13524's and
fix(windows): anchor six directory-link fences to pinned descriptors #14327's, and the failing shard's summary reads
22089 passed, 1030 skipped, 1 failedagainst a passing22094 passed, 1026 skipped— five fewer passes, fourmore skips, one failure. Those four extra skips ARE the
@reapingtests standingaside, which is what proves the runner lacked the capability and that the one
failure was the test that forgot to stand aside with them. After this change it
becomes the fifth skip on such a runner, and stays a failure on a runner where
os.pidfd_openworks and the fallback does not.Cannot find module 'electron'and friends, acrosspetOverlays,MacUpdater,build-config-schemaand
launchBlockingBundleParts, none of which this change touches. The oneelectron file it does touch passes 7/7.
Screenshots / video
N/A — no user-visible frontend surface changes. The two catalog edits change digit
script inside existing Bengali strings.
Pattern harvest
Rule candidate: agents-md / review-prompt — a count ceiling (
toBeLessThanOrEqual,a ratchet number, a baselined list length) must ship with a diff-scoped
zero-tolerance companion over the same defect. Without one it does not fail the
branch that introduces the violation; it fails whichever branch is unlucky enough to
cross the number, which makes it a gate that bills a stranger.
Rule candidate: review-prompt — when a change adds a call site of a shape some pin
scans the whole tree for, the local diff-relevance run will NOT select that pin. Ask
for the tree-scan pins explicitly. Two exist on the front end
(
port-owner-self-asserted-identity, the i18n style tests) and ~41 on the backend.Not generalizable: which runners in this fleet refuse
pidfd_openis a host fact,not a rule. What generalizes is the shape — assert the code, gate on the host — and
the cheap way to tell them apart from a CI summary alone (the passed/skipped count
delta), which is now written down.