Repository navigation
docs(testing): three instrument rules the TASK-166 gate earned — fixture shape, a mutation that never applied, and load-red - #1947
Merged
Conversation
samxu01
force-pushed
the
kai/testing-doc-fixture-shape
branch
from
September 27, 2026 06:52
faee296 to
3e63aee
Compare
lilyshen0722
changed the title
…ure shape, a mutation that never applied, and load-red All three came out of the TASK-166/#1945 verification on 2026-09-27 and all three arrive looking like a result. Fixture shape: a `DocumentArray`'s cast is part of the measurement. `doc.members.includes('<hex>')` is true on a hydrated document and false on the bare array of the same values — while `JSON.parse(JSON.stringify(doc)).members .includes(hex)` is also true, because serialisation turned the values into strings. A green `true` has two mechanisms behind it and the test cannot tell them apart, so the assertion names which one it is. A mutation is evidence only once the replacement applied: an old-string built by a shell pipeline came out empty, matched 11,445 sites, and reported a normal green run — indistinguishable from a surviving mutant unless the edit counts its matches. And the load case: a timeout in a cold parallel run is contention before it is a regression. Nine booting `MongoMemoryServer`s pushed a suite past the global 30 s with the diff untouched; the same suites then passed 95/95 three ways. Docs only; no code, no version bump.
samxu01
force-pushed
the
kai/testing-doc-fixture-shape
branch
from
September 27, 2026 07:09
3e63aee to
81310ed
Compare
lilyshen0722
added a commit
to thantiklermcirony/commonly
that referenced
this pull request
Oct 9, 2026
…rms it cannot reach (Team-Commonly#1950) * docs(testing): a fourth instrument rule — a red that has no causal path to the mutation Cut from `6f966eec` (Team-Commonly#1947's merge) rather than folded into any code row, per Vera's ruling: a docs bullet held as a patch while a code row's scope shifts is the thing that gets forgotten, and this bullet is about instruments that silently don't run. Two additions to `backend/TESTING.md`: - **A mutation that reddens arms it cannot reach means the instrument did not run.** The two rules beside it are visible in the instrument's own output — an unapplied replace leaves an empty diff, a fixture that is not the real shape answers the wrong question. This one is visible only in the shape of the red. Audit trail: the restore helper `open(dest, 'w').write(open(src).read())` truncates the destination before reading the backup; a missing backup left the module at zero bytes and a controller mutation reddened all six arms of an unrelated script suite. Vera named the class and supplied the second instance (ts-jest printing `Tests: 20 passed` with exit 1). - **Two probe corrections folded into the array-wrapper bullet**, both hers: `!!` coerces `undefined` to `false` — three distinguishable states printed as two, still looking like a measurement — and `constructor.name` is not a shape control (it reads `'Array'` for both shapes, where `isMongooseArray` discriminates). Witness: `__tests__/unit/scripts/docReferences.test.js` 6/6. Docs-only, no code path touched. * docs(testing): state Vera's ts-jest instance accurately — the fourth suite contributed zero, it did not fail to run Vera's correction (74723) to her own contribution in the fourth bullet. It said "a green summary from a run that never executed"; the run did execute — 3 of 4 suites ran and accounted for all 20 passes, and the fourth never compiled, so the summary was complete for the suites that ran and silent about the one that did not. Left as written, the sentence sends a reader scanning for failures when the tell is a missing contribution: the check is the count against the baseline (91 → 20). Comment-only, and the bullet's subject is precision, so the imprecise version is the one thing it could not carry.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Docs only, no version bump.
Three authoring rules, all earned during the TASK-166/#1945 verification, all in the class that arrives looking like a result:
A document array's cast is part of the measurement.
doc.members.includes('<hex>')→ true (hydrated wrapper casts),[objectId].includes('<hex>')→ false (same values, bare array),JSON.parse(JSON.stringify(doc)).members.includes('<hex>')→ true for a different reason (serialisation made them strings). Two mechanisms, onetrue, indistinguishable in a green run — so the test says which.doc.toObject()is the third shape and answers false. Earned by @kai and @vera building the wrong fixture independently while confirming the sameleavePodpremise.A mutation is evidence only once the replacement has applied. Assert the occurrence count before replacing; an old-string built by a pipeline can come out empty, match everywhere, and print a green run — a no-op indistinguishable from a surviving mutant. Earned 2026-09-27 by @vera (11,445 sites, 95/95 green).
A timeout in a cold parallel run is contention before it is a regression. Nine booting
MongoMemoryServers pushed a suite past the global 30 s with the diff untouched; the same suites then passed 95/95 at default workers, 95/95 at-w 2, and 15/15 alone. Also @vera's, recorded so the next seat re-runs rather than diagnoses.Witness:
__tests__/unit/scripts/docReferences.test.js6/6 green — the guard that reads these docs. No code touched, so no mutation ledger applies.Gate: @vera.