Skip to content

Feat/ci list failed tests - #154

Merged
shossain786 merged 2 commits into
seleniumboot:masterfrom
PUSHPAK-JAISWAL:feat/ci-list-failed-tests
Oct 15, 2026
Merged

shossain786 merged 2 commits into
seleniumboot:masterfrom
PUSHPAK-JAISWAL:feat/ci-list-failed-tests

Conversation

@PUSHPAK-JAISWAL

Copy link
Copy Markdown
Contributor

What & why

When tests fail during a GitHub Actions run, identifying which tests failed and locating their interactive trace files currently requires downloading and unpacking artifact zip archives.

This PR adds automatic job summary reporting:

  • Auto-detects GitHub Actions execution via CiEnvironmentDetector.isGitHubActions().
  • When tracing.enabled: true is set and tests fail, appends a GitHub Flavored Markdown table (Test, Error, Trace) directly to $GITHUB_STEP_SUMMARY.
  • Each table row highlights the failed test ID, the sanitized error message, and the relative path to the trace file (e.g. target/traces/ClassName/testMethod-trace.html or build/traces/... for Gradle).
  • Fully no-ops outside GitHub Actions, when tracing.enabled is false, or when no tests fail.
  • Integrated into both TestNG (SuiteExecutionListener) and JUnit 5 (SeleniumBootLauncherListener).

Closes #137
Related: #129, #130

Type of change

  • Bug fix
  • New feature (opt-in, aligned with the framework philosophy)
  • Documentation
  • Internal refactor / test coverage

Checklist

  • All existing tests pass (mvn clean verify)
  • New behaviour is covered by unit tests
  • No new external dependencies introduced without prior discussion
  • Code follows existing conventions (no new frameworks, minimal abstraction)
  • No breaking changes to @SeleniumBootApi public APIs (or a major version bump is justified below)
  • Docs updated if user-facing behaviour changed (docs-site/)

Notes for reviewers

  • Markdown table escaping: Newlines (\r\n, \n) in error messages are normalized to spaces and pipe characters (|) are escaped (\|) to prevent broken table row formatting in GitHub Actions.
  • Cross-platform path resolution: Trace paths are normalized to forward slashes (/) across Windows and Unix runners, resolving against ReportPaths.baseDir() (target or build).
  • File appending: Uses StandardCharsets.UTF_8 in append mode (FileWriter(file, true)) so any existing step summary output from earlier steps in the workflow is preserved.
  • Testability: An override property (-Dseleniumboot.github.step.summary) is supported so the summary export logic can be tested in CI/CD and unit tests without needing process-level environment variable mutation.
  • Docs:
    • Added step summary documentation and example to docs-site/docs/ci/github-actions.md.
    • Added the Trace Viewer guide in docs-site/docs/guides/trace-viewer.md and registered it in docs-site/sidebars.js (resolving docs(guide): document the Trace Viewer #129).
    • Updated CHANGELOG.md and docs-site/docs/changelog.md under [Unreleased].

@shossain786
shossain786 merged commit 66c1a07 into seleniumboot:master Oct 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(ci): list failed tests with trace links in the GitHub Actions job summary

2 participants