Merge pull request #65 from feO2x/mutation-triage-test-quality-rule

Mutation triage test quality rule

Author: feO2x

tests/AGENTS.md

@@ -85,8 +85,16 @@ Exactly three permitted responses to a survivor:
 
 Never restructure production code to make a mutant killable: performance outranks mutation score here, and the low-allocation `in`/`ref`/`Span` style produces awkward survivors by nature.
 
+The same discipline applies to test code. A test written under response 1 must stand on its own as a statement about the contract: name it for the behavior it pins down, and assert only on what the public API promises. Nothing in a test may refer to a mutant — no mutant ID, no line number, no mention of Stryker in a name, comment, or assertion message. The source suppression from response 2 is the only place a mutant is named.
+
+If a survivor can only be killed by asserting on something incidental — exact message composition, member or property ordering, a call count, a value the contract does not fix — it is response 2, not response 1. Suppress it with that reasoning. Such a test raises the score once and then constrains an implementation detail forever, which is a worse position than the survivor: the next legitimate refactoring breaks it, and the failure carries no information about the contract.
+
 ### Blind spots — do not read as adequate coverage
 
-- ~9.5% of mutants fail to compile (mostly the `out`/`ref` style); Stryker's Safe Mode then discards every mutant in the enclosing method: `Dragon4.GenerateDigits`, `ResultJsonReader.ReadStatusValue`/`ReadIndexValue`, `ErrorsExtensions.WriteRichErrors`, and all of `LightResult.cs` receive no mutation coverage. Tool limitation, not a test defect — a high score in `Numbers/` is not verified behavior.
+- ~9.5% of mutants fail to compile (mostly the `out`/`ref` style); Stryker's Safe Mode then discards every mutant in the enclosing method, which receives no mutation coverage at all. Tool limitation, not a test defect — a high score in `Numbers/` is not verified behavior. Observed at the baseline: `Dragon4.GenerateDigits`, `ResultJsonReader.ReadStatusValue`/`ReadIndexValue`, `ErrorsExtensions.WriteRichErrors`, and all of `LightResult.cs` (11 of 11 mutants).
+
+  Treat that list as observed, not fixed: any new `out`/`ref` code joins it silently. Stryker announces it as `[INF] Safe Mode! Stryker will remove all mutations in <method>` on the console only — no log file is written — and the discarded mutants are simply absent from the JSON report. The durable way to recover the current set is to filter the report for `"status": "CompileError"`; those sites are the only trace left, and their enclosing methods are the ones running blind.
+
+  When changing a method in that set, mutation score carries no information about it and line coverage only proves execution. Adequacy has to be argued by hand: enumerate the behaviors the method promises and point at the test constraining each one. State that reasoning in the pull request, because no tool in this repository can check it.
 - `Timeout` counts as killed. The pinned 30,000 ms additional timeout reduced `Validation.OpenApi` from 21 timeouts to zero in two consecutive concurrency-8 runs, but no finite value makes classification independent of hardware and load. Investigate any future timeout as either a genuine hang or insufficient headroom; do not assume it represents a killed mutant.
 - The MTP runner is a preview (stryker-mutator/stryker-net#3094); verify surprising results against a plain `dotnet test` run.