Use SARIF as the common report format for PR review and HTML rendering

[wborn]

Motivation

The SAT plugin currently consumes the XML reports produced by Checkstyle, PMD, and SpotBugs and generates custom HTML reports.

SARIF should become the common report format used by SAT. This would allow the same analysis results to be used for:

• GitHub code scanning and pull-request annotations;
• compatible IDE integrations;
• a standalone HTML report;
• integration of additional static-analysis tools that already produce SARIF.

A new tool that produces compatible SARIF output could be integrated without implementing another analyzer-specific XML transformation and HTML report generator.

GitHub pull-request integration

The generated SARIF output could be uploaded to GitHub code scanning from GitHub Actions:

permissions:
  contents: read
  security-events: write

steps:
  - name: Run static analysis
    run: ./mvnw clean verify -Pcheck

  - name: Upload SAT results
    if: always()
    uses: github/codeql-action/upload-sarif@v4
    with:
      sarif_file: target/code-analysis/sat-results.sarif
      category: openhab-sat

GitHub would then:

• add a Code scanning results check to pull requests;
• show new findings on changed lines as pull-request annotations;
• allow findings to be discussed and dismissed through the GitHub interface;
• retain findings as repository code-scanning alerts;
• track findings that disappear from a later analysis as fixed.

GitHub primarily highlights newly introduced findings in pull requests. It does not provide an equally prominent list of all findings removed by a pull request, so an explicit added/fixed summary could be considered separately.

The generated report could also be opened in SARIF-compatible IDE integrations. For example, the SARIF Viewer for Visual Studio Code can display findings in the editor and Problems view. SARIF integrations are also available for JetBrains IDEs.

Existing analyzer support

All three analysis tools currently used by SAT can produce SARIF, although their Maven integrations are not equally mature.

Tool	SARIF support
Checkstyle	Checkstyle supports SARIF, and maven-checkstyle-plugin supports <outputFileFormat>sarif</outputFileFormat>.
PMD	PMD provides native SARIF output. First-class support in maven-pmd-plugin is tracked by apache/maven-pmd-plugin#712.
SpotBugs	spotbugs-maven-plugin supports sarifOutput, sarifOutputDirectory, sarifOutputFilename, and sarifFullPath.

The Maven integrations do not provide a consistent way to produce both their existing XML output and SARIF in the same execution:

• SpotBugs can enable XML and SARIF independently;
• Checkstyle selects one output format per execution;
• the Maven PMD Plugin does not yet expose first-class SARIF output.

Maintaining XML for the existing HTML pipeline alongside SARIF would therefore require analyzer-specific handling and, in some cases, additional analysis executions or conversions.

For that reason, the existing custom HTML pipeline should be replaced rather than permanently maintained alongside the SARIF pipeline.

Relevant documentation:

• Maven Checkstyle Plugin output format
• PMD report formats
• Maven PMD Plugin SARIF issue
• SpotBugs Maven Plugin SARIF parameters
• GitHub SARIF support
• Uploading SARIF to GitHub
• Code-scanning alerts in pull requests
• SARIF Viewer for Visual Studio Code
• Qodana IDE plugin

Alternative: use an external code-quality platform

Before replacing the current SAT report implementation, it may be worth discussing whether openHAB should instead adopt an external code-quality platform such as SonarQube Cloud.

Such a platform can provide pull-request analysis, inline findings, quality gates, code-quality metrics, coverage reporting, duplication detection, technical-debt tracking, and historical dashboards without SAT having to implement and maintain all of those features itself.

The possible directions are therefore:

1. continue using SAT and make SARIF its common report format;
2. use SAT for analysis while importing its results into an external platform;
3. replace some or all SAT reporting and pull-request integration with an external platform.

Using GitHub code scanning directly would keep the integration close to the repositories and allow SAT to remain the source of the findings. Using a platform such as SonarQube Cloud could provide broader quality-management functionality, but would introduce another service, configuration, issue-management interface, and potentially overlapping PR feedback.

Custom openHAB checks

An external platform would not necessarily replace the execution of SAT's existing custom Checkstyle, PMD, and SpotBugs checks.

SonarQube Cloud can import results produced by external analyzers, including Checkstyle, PMD, SpotBugs, and SARIF reports. This means SAT could continue running the custom openHAB checks and provide their findings to SonarQube Cloud.

However, these findings would remain external issues:

• SonarQube Cloud would not execute the custom openHAB checks itself;
• the custom rules would not become native Sonar rules;
• they could not be enabled, disabled, or configured through a Sonar quality profile;
• configuration and suppressions would remain managed by SAT;
• changing the status of an imported issue in SonarQube Cloud would not update SAT or its suppression configuration.

Replacing SAT completely with SonarQube Cloud's native analysis would therefore mean losing the custom checks unless they were replaced with equivalent Sonar rules or reimplemented for the external platform.

SARIF import limitations

SonarQube Cloud can import an aggregate SARIF report using:

sonar.sarifReportPaths=target/code-analysis/sat-results.sarif

There are currently some limitations to its generic SARIF integration:

• imported issues are treated as external issues;
• imported rules are not included in Sonar quality profiles;
• SonarQube Cloud currently assigns the SECURITY software quality to issues imported using its generic SARIF importer, even when a Checkstyle or PMD finding represents maintainability or code style;
• severity is derived from the rule-level SARIF configuration rather than each individual result's level.

The dedicated Checkstyle, PMD, and SpotBugs report importers may therefore represent some findings more accurately than the generic SARIF importer.

This means that adopting SonarQube Cloud does not automatically justify removing all analyzer-native report output. SAT might need to retain selected native reports specifically for SonarQube Cloud integration, even if SARIF becomes its main public report format.

Relevant documentation:

• External analyzer reports in SonarQube Cloud
• Importing SARIF reports into SonarQube Cloud

This decision does not necessarily block SARIF support. Producing standardized SARIF output would keep SAT results portable and allow them to be consumed by GitHub, IDE integrations, HTML viewers, and compatible external platforms.

Proposed output

If SAT remains responsible for producing and presenting the analysis results, it should generate one aggregate SARIF file:

target/code-analysis/
├── sat-results.sarif
└── report.html

The SARIF file should contain one run for each analysis tool:

sat-results.sarif
├── Checkstyle run
├── PMD run
├── SpotBugs run
└── additional tool runs

Results from different Maven modules should be combined within the run for their analyzer rather than creating one run per module.

Using one aggregate file provides a simple artifact for GitHub Actions, IDEs, build archives, and local HTML rendering. It also avoids requiring users to manage separate files and GitHub analysis categories for each tool.

Separate SARIF files could remain an implementation detail or a fallback for unusually large analyses, but the normal public output should be the aggregate report.

Possible configuration:

<configuration>
  <sarifOutputFile>
    ${session.executionRootDirectory}/target/code-analysis/sat-results.sarif
  </sarifOutputFile>
</configuration>

The exact parameter name and default location can be decided during implementation.

Possible implementation approach

Use the native SARIF output provided by Checkstyle and SpotBugs.

Until the Maven PMD Plugin provides first-class SARIF support, SAT could either invoke PMD's native SARIF support directly or translate the existing PMD XML report into SARIF.

SAT would then combine the analyzer results into one SARIF 2.1.0 document with a separate run for each analyzer.

The aggregation would need to normalize:

• repository-relative source paths;
• tool and rule identifiers;
• severity levels;
• fingerprints used to identify findings across analyses;
• module ordering when Maven runs in parallel.

The aggregate output should remain deterministic regardless of the order in which Maven modules complete.

Additional analysis tools that produce SARIF could subsequently be integrated by adding their run to the aggregate document rather than implementing tool-specific report transformations and HTML rendering.

HTML report rendering

The standalone HTML report should be rendered from the aggregate SARIF data instead of using the current custom HTML builders and XSLT transformations.

Microsoft provides the MIT-licensed @microsoft/sarif-web-component, a React-based viewer for SARIF files.

Its basic API accepts one or more SARIF logs:

import { Viewer } from "@microsoft/sarif-web-component";

<Viewer logs={sarifLogs} />

The frontend could be compiled during the SAT project build and packaged as static JavaScript and CSS resources inside the Maven plugin. Projects using SAT would therefore not need Node.js or npm during their Maven builds.

SAT could generate a standalone HTML file containing:

• the compiled SARIF viewer;
• the required CSS;
• the aggregate SARIF data embedded as JSON.

For example:

<div id="viewer"></div>

<script id="sarif-data" type="application/json">
  { ... generated SARIF ... }
</script>

Embedding the SARIF data and frontend assets would allow the report to be opened directly from the filesystem without requiring a web server or external network resources.

The resulting architecture would be:

Checkstyle SARIF ─┐
PMD SARIF ────────┼─→ aggregate SAT SARIF
SpotBugs SARIF ───┤              │
Other tools ──────┘              ├─→ GitHub code scanning
                                 ├─→ IDE SARIF viewers
                                 ├─→ compatible external platforms
                                 └─→ standalone report.html

A prototype should verify that the component works well with:

• multiple analyzer runs;
• large multi-module repositories;
• repository-relative source paths;
• local source navigation;
• Checkstyle, PMD, and SpotBugs rule metadata.

The component currently uses an older React-based frontend stack, so its dependencies and long-term maintenance should also be evaluated before making it a required part of SAT.

If suitable, the new report generation could replace:

• the analyzer-specific XML-to-HTML transformations;
• the custom HTML table builders;
• summary_bundles.html;
• the current ad hoc HTML aggregation logic.

Benefits

This would provide:

• inline static-analysis feedback during pull-request review;
• one common report artifact for GitHub, IDEs, and local viewing;
• a standard format supported by multiple development tools;
• persistent tracking and triage of findings;
• clearer separation between newly introduced and existing findings;
• integration with SARIF-compatible IDE viewers;
• compatibility with external platforms that can consume SARIF;
• easier integration of additional SARIF-producing analysis tools;
• reusable HTML rendering through an existing SARIF viewer;
• less custom report-generation code to maintain;
• deterministic aggregation across Maven modules;
• an integration path that does not depend on custom PR comments.