fix: run-tests now fails a run when a fixture's one-time setup or teardown throws

.agents/skills/uloop-run-tests/SKILL.md

@@ -6,7 +6,7 @@ description: "Run Unity Test Runner and report detailed results. Use for EditMod
 
 # uloop run-tests
 
-Execute Unity Test Runner. When tests fail or end inconclusive, NUnit XML results with failure messages, stack traces, and inconclusive reasons are automatically saved. Read the XML file at `XmlPath` for detailed diagnosis.
+Execute Unity Test Runner. When a test or suite fails or a test ends inconclusive, NUnit XML results with failure messages, stack traces, and inconclusive reasons are automatically saved. Read the XML file at `XmlPath` for detailed diagnosis.
 
 `uloop run-tests` automatically compiles pending script changes before running tests. Pass `--skip-compile` only while validating active hot-reload patches, because the compile clears those patches; otherwise let the default compile surface errors and run against current scripts. `--skip-compile` skips only the CLI-side compile: Unity still imports script edits saved since the last compile, and that import reloads the domain as soon as the run releases its assembly lock, discarding active patches and ending the request.
 
@@ -46,9 +46,9 @@ exact matches the full test name (Namespace.Class.Method). class runs every test
 
 Returns JSON with:
 
-- `Success` (boolean): Whether every test passed or was skipped; `false` when any failed or was inconclusive
+- `Success` (boolean): Whether every test passed or was skipped; `false` when a test or suite failed or a test was inconclusive
 - `Status` (string): Machine-readable execution status such as `Passed`, `Failed`, `Inconclusive`, `NoTestsFound`, or `ExecutionFailed`
-- `HasFailures` (boolean): Whether any discovered test failed
+- `HasFailures` (boolean): Whether any discovered test or suite failed
 - `Message` (string): Summary message
 - `NoTestsFound` (boolean): Whether Unity Test Runner discovered zero matching tests
 - `NoTestsFoundExplanation` (string): Agent-facing explanation when `NoTestsFound` is true; empty otherwise
@@ -58,19 +58,15 @@ Returns JSON with:
 - `FailedCount` (number): Failed tests
 - `SkippedCount` (number): Skipped tests
 - `InconclusiveCount` (number): Inconclusive tests (an `Assume` was not met)
-- `XmlPath` (string): Path to NUnit XML result file. Empty string when no XML was saved (typically on `Success: true`); populated only when tests failed or were inconclusive and the XML file exists on disk.
+- `XmlPath` (string or null): Path to NUnit XML result file. `null` when no XML was saved (typically on `Success: true`); set only when a test or suite failed or a test was inconclusive and the file exists on disk.
 - `ClearedPausePointIds` (string[], optional): IDs of pause points that were cleared before test execution. Omitted from JSON when no pause points were active.
 - `FailedTests` (array, optional): Up to 10 failed leaf tests with `FullName`, `Message`, and when the stack trace contains a path:line location, `File` and `Line`. Omitted when no tests failed. When `FailedCount` is greater than 10, `Message` ends with `first 10 of N failures listed; see XmlPath for full results.`
 - `SkippedTests` (string[], optional): Up to 10 full names of skipped leaf tests. Omitted when no tests were skipped. When `SkippedCount` is greater than 10, only the first 10 names are listed.
 - `InconclusiveTests` (array, optional): Up to 10 inconclusive leaf tests with `FullName` and `Message`. Omitted when no test was inconclusive. When `InconclusiveCount` is greater than 10, only the first 10 are listed; the XML at `XmlPath` has every message.
+- `FailedSuites` (array, optional): Up to 10 suites that failed outside their tests (e.g. a `OneTimeSetUp` or `OneTimeTearDown` threw), with the `FailedTests` fields. The run is `Failed` even when `FailedCount` is 0. Omitted when none.
 - `ProposedTestAsmdef` (object, optional): `AssetPath` and `Content` of a ready-to-write test `.asmdef` (test-assembly wiring plus references to the project's assemblies under test). Present only when an unfiltered run found no tests and no test assembly exists for the TestMode.
 - `CompileNote` (string, optional): States that the automatic compile ran and succeeded before the tests and names `--skip-compile` as the opt-out. When the compile response carried a Warning (for example active hot-reload changes dropped by the domain reload), the note repeats it. Omitted when `--skip-compile` was passed; a failed compile returns the compile error response instead.
 
 ### XML Result File
 
-When tests fail or end inconclusive, NUnit XML results are automatically saved to `{project_root}/.uloop/outputs/TestResults/<timestamp>.xml`. The XML contains per-test-case results including:
-
-- Test name and full name
-- Pass/fail/skip status and duration
-- For failed tests: `<message>` (assertion error) and `<stack-trace>`
-- For inconclusive tests: `<reason><message>` (the assumption that was not met)
+Saved to `{project_root}/.uloop/outputs/TestResults/<timestamp>.xml`. What it records, including failed suites: `references/xml-results.md`.

.agents/skills/uloop-run-tests/references/xml-results.md

@@ -0,0 +1,13 @@
+# run-tests XML result file
+
+When a test or suite fails or a test ends inconclusive, run-tests saves NUnit XML results to `{project_root}/.uloop/outputs/TestResults/<timestamp>.xml` and returns the path in `XmlPath`. A run in which every test passed or was skipped saves no XML, and `XmlPath` is `null`.
+
+The XML contains per-test-case results including:
+
+- Test name and full name
+- Pass/fail/skip status and duration
+- For failed tests: `<message>` (assertion error) and `<stack-trace>`
+- For inconclusive tests: `<reason><message>` (the assumption that was not met)
+- For failed suites: `<failure>` with `<message>` and `<stack-trace>` on the `<test-suite>`. A fixture whose `OneTimeTearDown` threw keeps its error only here and in `FailedSuites`: its test cases keep their own results, passed or failed.
+
+The response lists at most 10 entries in each of `FailedTests`, `InconclusiveTests`, and `FailedSuites`; the XML keeps every one.

.claude/skills/uloop-run-tests/SKILL.md

@@ -6,7 +6,7 @@ description: "Run Unity Test Runner and report detailed results. Use for EditMod
 
 # uloop run-tests
 
-Execute Unity Test Runner. When tests fail or end inconclusive, NUnit XML results with failure messages, stack traces, and inconclusive reasons are automatically saved. Read the XML file at `XmlPath` for detailed diagnosis.
+Execute Unity Test Runner. When a test or suite fails or a test ends inconclusive, NUnit XML results with failure messages, stack traces, and inconclusive reasons are automatically saved. Read the XML file at `XmlPath` for detailed diagnosis.
 
 `uloop run-tests` automatically compiles pending script changes before running tests. Pass `--skip-compile` only while validating active hot-reload patches, because the compile clears those patches; otherwise let the default compile surface errors and run against current scripts. `--skip-compile` skips only the CLI-side compile: Unity still imports script edits saved since the last compile, and that import reloads the domain as soon as the run releases its assembly lock, discarding active patches and ending the request.
 
@@ -46,9 +46,9 @@ exact matches the full test name (Namespace.Class.Method). class runs every test
 
 Returns JSON with:
 
-- `Success` (boolean): Whether every test passed or was skipped; `false` when any failed or was inconclusive
+- `Success` (boolean): Whether every test passed or was skipped; `false` when a test or suite failed or a test was inconclusive
 - `Status` (string): Machine-readable execution status such as `Passed`, `Failed`, `Inconclusive`, `NoTestsFound`, or `ExecutionFailed`
-- `HasFailures` (boolean): Whether any discovered test failed
+- `HasFailures` (boolean): Whether any discovered test or suite failed
 - `Message` (string): Summary message
 - `NoTestsFound` (boolean): Whether Unity Test Runner discovered zero matching tests
 - `NoTestsFoundExplanation` (string): Agent-facing explanation when `NoTestsFound` is true; empty otherwise
@@ -58,19 +58,15 @@ Returns JSON with:
 - `FailedCount` (number): Failed tests
 - `SkippedCount` (number): Skipped tests
 - `InconclusiveCount` (number): Inconclusive tests (an `Assume` was not met)
-- `XmlPath` (string): Path to NUnit XML result file. Empty string when no XML was saved (typically on `Success: true`); populated only when tests failed or were inconclusive and the XML file exists on disk.
+- `XmlPath` (string or null): Path to NUnit XML result file. `null` when no XML was saved (typically on `Success: true`); set only when a test or suite failed or a test was inconclusive and the file exists on disk.
 - `ClearedPausePointIds` (string[], optional): IDs of pause points that were cleared before test execution. Omitted from JSON when no pause points were active.
 - `FailedTests` (array, optional): Up to 10 failed leaf tests with `FullName`, `Message`, and when the stack trace contains a path:line location, `File` and `Line`. Omitted when no tests failed. When `FailedCount` is greater than 10, `Message` ends with `first 10 of N failures listed; see XmlPath for full results.`
 - `SkippedTests` (string[], optional): Up to 10 full names of skipped leaf tests. Omitted when no tests were skipped. When `SkippedCount` is greater than 10, only the first 10 names are listed.
 - `InconclusiveTests` (array, optional): Up to 10 inconclusive leaf tests with `FullName` and `Message`. Omitted when no test was inconclusive. When `InconclusiveCount` is greater than 10, only the first 10 are listed; the XML at `XmlPath` has every message.
+- `FailedSuites` (array, optional): Up to 10 suites that failed outside their tests (e.g. a `OneTimeSetUp` or `OneTimeTearDown` threw), with the `FailedTests` fields. The run is `Failed` even when `FailedCount` is 0. Omitted when none.
 - `ProposedTestAsmdef` (object, optional): `AssetPath` and `Content` of a ready-to-write test `.asmdef` (test-assembly wiring plus references to the project's assemblies under test). Present only when an unfiltered run found no tests and no test assembly exists for the TestMode.
 - `CompileNote` (string, optional): States that the automatic compile ran and succeeded before the tests and names `--skip-compile` as the opt-out. When the compile response carried a Warning (for example active hot-reload changes dropped by the domain reload), the note repeats it. Omitted when `--skip-compile` was passed; a failed compile returns the compile error response instead.
 
 ### XML Result File
 
-When tests fail or end inconclusive, NUnit XML results are automatically saved to `{project_root}/.uloop/outputs/TestResults/<timestamp>.xml`. The XML contains per-test-case results including:
-
-- Test name and full name
-- Pass/fail/skip status and duration
-- For failed tests: `<message>` (assertion error) and `<stack-trace>`
-- For inconclusive tests: `<reason><message>` (the assumption that was not met)
+Saved to `{project_root}/.uloop/outputs/TestResults/<timestamp>.xml`. What it records, including failed suites: `references/xml-results.md`.

.claude/skills/uloop-run-tests/references/xml-results.md

@@ -0,0 +1,13 @@
+# run-tests XML result file
+
+When a test or suite fails or a test ends inconclusive, run-tests saves NUnit XML results to `{project_root}/.uloop/outputs/TestResults/<timestamp>.xml` and returns the path in `XmlPath`. A run in which every test passed or was skipped saves no XML, and `XmlPath` is `null`.
+
+The XML contains per-test-case results including:
+
+- Test name and full name
+- Pass/fail/skip status and duration
+- For failed tests: `<message>` (assertion error) and `<stack-trace>`
+- For inconclusive tests: `<reason><message>` (the assumption that was not met)
+- For failed suites: `<failure>` with `<message>` and `<stack-trace>` on the `<test-suite>`. A fixture whose `OneTimeTearDown` threw keeps its error only here and in `FailedSuites`: its test cases keep their own results, passed or failed.
+
+The response lists at most 10 entries in each of `FailedTests`, `InconclusiveTests`, and `FailedSuites`; the XML keeps every one.

Assets/Tests/Editor/RunTestsResponseContractTests.cs

@@ -265,6 +265,114 @@ public void FromResult_WhenResultHasInconclusiveLeaves_CopiesCountAndDetails()
             Assert.That(empty.InconclusiveTests, Is.Null);
         }
 
+        /// <summary>
+        /// What: FailedSuites is omitted from JSON when no suite failed and serializes each suite's
+        /// name and message when one did.
+        /// </summary>
+        [Test]
+        public void RunTestsResponse_WhenSerialized_OmitsOrIncludesFailedSuites()
+        {
+            RunTestsResponse withoutFailedSuites = CreateFailedSuitesResponse(null);
+            RunTestsResponse withFailedSuites = CreateFailedSuitesResponse(new[]
+            {
+                new SerializableTestResult.FailedTestDetail
+                {
+                    FullName = "Example.Tests.TearDownFixture",
+                    Message = "TearDown : System.InvalidOperationException : teardown failed"
+                }
+            });
+
+            JObject withoutJson = JObject.Parse(
+                JsonConvert.SerializeObject(
+                    withoutFailedSuites,
+                    Formatting.None,
+                    UnityCliLoopJsonResponseSerializerSettings.Settings));
+            JObject withJson = JObject.Parse(
+                JsonConvert.SerializeObject(
+                    withFailedSuites,
+                    Formatting.None,
+                    UnityCliLoopJsonResponseSerializerSettings.Settings));
+            JArray failedSuites = (JArray)withJson["FailedSuites"];
+
+            Assert.That(withoutJson.Property("FailedSuites"), Is.Null);
+            Assert.That(failedSuites, Is.Not.Null);
+            Assert.That(failedSuites.Count, Is.EqualTo(1));
+            JObject first = (JObject)failedSuites[0];
+            Assert.That(first["FullName"]?.Value<string>(), Is.EqualTo("Example.Tests.TearDownFixture"));
+            Assert.That(
+                first["Message"]?.Value<string>(),
+                Is.EqualTo("TearDown : System.InvalidOperationException : teardown failed"));
+        }
+
+        /// <summary>
+        /// What: the response built from a stored result carries its failed suites, and leaves
+        /// FailedSuites unset when the result lists none.
+        /// </summary>
+        [Test]
+        public void FromResult_WhenResultHasFailedSuites_CopiesThem()
+        {
+            SerializableTestResult withFailedSuites = new SerializableTestResult
+            {
+                success = false,
+                status = RunTestsExecutionStatus.Failed,
+                hasFailures = true,
+                message = "Test execution completed with status: Failed",
+                noTestsFoundExplanation = string.Empty,
+                completedAt = "2026-01-01T00:00:00.0000000Z",
+                testCount = 1,
+                passedCount = 1,
+                failedSuites = new[]
+                {
+                    new SerializableTestResult.FailedTestDetail
+                    {
+                        FullName = "Example.Tests.TearDownFixture",
+                        Message = "TearDown : System.InvalidOperationException : teardown failed"
+                    }
+                }
+            };
+            SerializableTestResult withoutFailedSuites = new SerializableTestResult
+            {
+                success = true,
+                status = RunTestsExecutionStatus.Passed,
+                message = "Test execution completed with status: Passed",
+                noTestsFoundExplanation = string.Empty,
+                completedAt = "2026-01-01T00:00:00.0000000Z",
+                testCount = 1,
+                passedCount = 1,
+                failedSuites = new SerializableTestResult.FailedTestDetail[0]
+            };
+
+            RunTestsResponse copied = RunTestsResponseFactory.FromResult(withFailedSuites);
+            RunTestsResponse empty = RunTestsResponseFactory.FromResult(withoutFailedSuites);
+
+            Assert.That(copied.FailedSuites, Is.Not.Null);
+            Assert.That(copied.FailedSuites.Length, Is.EqualTo(1));
+            Assert.That(copied.FailedSuites[0].FullName, Is.EqualTo("Example.Tests.TearDownFixture"));
+            Assert.That(empty.FailedSuites, Is.Null);
+        }
+
+        private static RunTestsResponse CreateFailedSuitesResponse(
+            SerializableTestResult.FailedTestDetail[] failedSuites)
+        {
+            return new RunTestsResponse(
+                success: failedSuites == null,
+                message: "Test execution completed",
+                completedAt: "2026-01-01T00:00:00.0000000Z",
+                testCount: 1,
+                passedCount: 1,
+                failedCount: 0,
+                skippedCount: 0,
+                inconclusiveCount: 0,
+                xmlPath: null,
+                status: failedSuites == null ? RunTestsExecutionStatus.Passed : RunTestsExecutionStatus.Failed,
+                hasFailures: failedSuites != null,
+                noTestsFound: false,
+                noTestsFoundExplanation: string.Empty)
+            {
+                FailedSuites = failedSuites
+            };
+        }
+
         /// <summary>
         /// What: an empty Warning is omitted from production JSON so the key cannot reappear unnoticed.
         /// </summary>