[Feature Request] Export & Visualize Test Results in GitHub Actions UI

[Sinitca-Aleksandr]

Problem Statement

Our GitHub Actions pipeline executes tests successfully, but the results are only visible within raw job logs. This creates friction for developers who need to:

• Quickly identify failed tests in a Pull Request without parsing through verbose logs
• Track test flakiness or performance trends across commits
• Receive immediate, contextual feedback directly within the GitHub interface

Proposed Solution

Update the CI workflow to export test results in JUnit XML format and leverage GitHub Actions' native reporting capabilities to visualize results directly in the GitHub UI.

Key outcomes:

• Test summary visible in the "Checks" tab of workflow runs
• Inline annotations on Pull Request files for failed tests
• Optional historical trend tracking via artifact retention

Implementation Plan

1. Configure Test Runner to Output JUnit XML

Examples for common frameworks:

# PyTest
- name: Run tests
  run: pytest --junitxml=test-results.xml

2. Publish Results Using a GitHub Action

Option A: dorny/test-reporter (rich UI with annotations)

- name: Publish Test Report
  uses: dorny/test-reporter@v1
  if: always()
  with:
    name: Test Results
    path: test-results.xml
    reporter: java-junit
    fail-on-error: true

Option B: EnricoMi/publish-unit-test-result-action (detailed statistics and trends)

- name: Publish Unit Test Results
  uses: EnricoMi/publish-unit-test-result-action@v2
  if: always()
  with:
    files: test-results.xml
    check_name: Test Report
    comment_mode: always

3. Upload Raw Results as Artifact (Optional, for debugging)

- name: Upload Raw Results
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: test-results-raw
    path: test-results.xml
    retention-days: 14

Acceptance Criteria

• Test results are generated in JUnit XML format
• A test summary appears in the Checks tab of each workflow run
• Failed tests are displayed as inline annotations on relevant files in Pull Requests
• Workflow executes reporting steps on both success and failure (using if: always())
• No sensitive data (secrets, tokens, PII) is included in exported reports
• Documentation (README.md or CONTRIBUTING.md) is updated with instructions for accessing results

Expected UI Output

After implementation, contributors should see:

In the Checks tab:

• Test Results section showing pass/fail/skip counts
• Expandable details for each failed test with error messages

In Pull Request diffs:

• Inline comments on code lines associated with failing tests
• Clear error output to accelerate debugging

References

• dorny/test-reporter documentation: https://github.com/dorny/test-reporter
• publish-unit-test-result-action documentation: https://github.com/EnricoMi/publish-unit-test-result-action
• GitHub Actions workflow commands: https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions

Rationale for This Approach

Benefit	Explanation
Native GitHub UI	Results appear where developers already work; no external tools or logins required
PR-Centric Feedback	Annotations directly on code changes reduce context switching and speed up fixes
Zero Infrastructure Overhead	Leverages GitHub's built-in artifact storage and Checks API
Extensible Design	The same JUnit XML output can later feed external dashboards if requirements evolve