run-tests: a run whose OneTimeTearDown throws is reported as passed when every test passed

[hatayama]

Summary

When a fixture's [OneTimeTearDown] throws and every test in the run passed, uloop run-tests reports Status: Passed and Success: true. Unity records the teardown exception on the fixture's suite result and leaves its tests' results alone, while run-tests decides the status from the leaf tests only. The exception message is not reported anywhere: FailedTests is omitted and no XML is saved.

Reproduce

using System;
using NUnit.Framework;

public class OneTimeTearDownThrows
{
    [OneTimeTearDown]
    public void TearDownOnce()
    {
        throw new InvalidOperationException("teardown failed");
    }

    [Test]
    public void Passes()
    {
        Assert.That(1, Is.EqualTo(1));
    }
}

uloop run-tests --filter-type regex --filter-value OneTimeTearDownThrows

Observed

"FailedCount": 0,
"PassedCount": 1,
"Status": "Passed",
"Success": true,
"TestCount": 1,
"XmlPath": null

Adding a second fixture with a failing test to the same run makes run-tests save the XML. In that XML the fixture above has result="Failed" while its only test case has result="Passed", so Unity does report the fixture as failed. The XML does not contain the teardown exception message either, because the exporter writes messages for test cases only.

Observed with the package at 3.9.1 on Unity 2022.3.62f3 in EditMode.

Expected

The run is not reported as passed, and the teardown exception message is visible in the response or in the XML.

Notes

• Cause: SerializableTestResultConverter.Classify returns FullyPassed when no leaf failed and passedTests + skippedTests == totalTests, whatever the root status is. The counts, FailedTests, and ShouldSaveResultXml all look at leaf tests only.
• fix: run-tests no longer reports runs with inconclusive tests as passed #3020 covered a neighboring case: a Failed root with an inconclusive leaf and no failed leaf is reported as Failed. The case in this issue, with no failed and no inconclusive leaf, predates fix: run-tests no longer reports runs with inconclusive tests as passed #3020.
• A fix probably needs to treat a Failed root as a failure even when no leaf failed, and to surface the failing suite's message, for example as a FailedTests entry for the suite and a <failure> on the suite in the XML.