PowerUtils.BenchmarkDotnet.Reporter

Tool to analyze and compare .NET benchmark reports

PowerUtils.BenchmarkDotnet.Reporter is a command-line tool used to analyze and compare .NET benchmark reports generated by BenchmarkDotNet. This tool is designed to run locally or in CI/CD pipelines, providing a simple way to visualize and compare benchmark results.

• Prepare environment
  • Install Tool
  • Update Tool
  • List installed tools
  • Uninstall Tool
• Export reports using BenchmarkDotNet
• How to use
  • Run the tool
  • Commands
    • compare
• Configuration
• Documentation
• GitHub Actions Setup
• Acknowledgments
• Contribution

Prepare environment

Install Tool

This package is available through Nuget Repository: https://www.nuget.org/packages/PowerUtils.BenchmarkDotnet.Reporter

Locally

# Create a tool manifest file in the current directory
dotnet new tool-manifest
# Install the tool in the current directory
dotnet tool install PowerUtils.BenchmarkDotnet.Reporter

Globally

dotnet tool install --global PowerUtils.BenchmarkDotnet.Reporter

Update Tool

Locally

dotnet tool update PowerUtils.BenchmarkDotnet.Reporter

Globally

dotnet tool update --global PowerUtils.BenchmarkDotnet.Reporter

List installed tools

Locally

dotnet tool list

Globally

dotnet tool list --global

Uninstall Tool

Locally

dotnet tool uninstall PowerUtils.BenchmarkDotnet.Reporter

Globally

dotnet tool uninstall --global PowerUtils.BenchmarkDotnet.Reporter

Export reports using BenchmarkDotNet

To use the PowerUtils.BenchmarkDotnet.Reporter tool, you first need to generate benchmark reports using BenchmarkDotNet and export them in JSON format. You can do this by looking at the documentation of BenchmarkDotNet here or following the examples below:

NOTE: Now the PowerUtils.BenchmarkDotnet.Reporter tool supports all the Json Formatters provided by BenchmarkDotNet, so you can choose the one that best suits your needs.

// TheBenchmark.cs
using BenchmarkDotNet.Attributes;

[MemoryDiagnoser]
[JsonExporterAttribute.Full]
[JsonExporterAttribute.FullCompressed]
[JsonExporterAttribute.Brief]
[JsonExporterAttribute.BriefCompressed]
public class TheBenchmark
{
    [Benchmark]
    public void TheMethod()
    {
        // Your benchmark code here
    }
}


// Program.cs
using BenchmarkDotNet.Running;

BenchmarkRunner.Run<TheBenchmark>();

Or

// TheBenchmark.cs
using BenchmarkDotNet.Attributes;

[MemoryDiagnoser]
public class TheBenchmark
{
    [Benchmark]
    public void TheMethod()
    {
        // Your benchmark code here
    }
}


// Program.cs
using BenchmarkDotNet.Running;

var config = ManualConfig
    .Create(DefaultConfig.Instance)
    .AddExporter(JsonExporter.Full)
    .AddExporter(JsonExporter.FullCompressed)
    .AddExporter(JsonExporter.Brief)
    .AddExporter(JsonExporter.BriefCompressed);

BenchmarkRunner.Run<TheBenchmark>(config);

How to use

Run the tool

Locally

dotnet pbreporter [command] [options]

Globally

pbreporter [command] [options]

.NET 10+ (without installation)

dnx PowerUtils.BenchmarkDotnet.Reporter [command] [options]

Starting with .NET 10, you can run the tool directly using dnx without installing it first.

Commands

compare

Compares two benchmark reports and generates a report with the differences.

Example:

pbreporter compare -b baseline-full.json -t target-full.json

For the full option reference, threshold syntax, usage examples, output metrics, and exit codes, see the compare Command Reference.

Configuration

Every option on every command - CLI arguments, environment variables, and a YAML config file, follow a single, consistent precedence: CLI arguments > environment variables > config file. See Configuration for the full naming convention, precedence details, and a worked example (including running compare with no -b/-t at all).

Documentation

• compare Command Reference - full options, threshold syntax, usage examples, output metrics, exit codes
• Configuration - environment variables and YAML config file
• GitHub Actions Setup Guide - CI/CD integration
• Test Data Documentation - sample benchmark reports for contributors

GitHub Actions Setup

PowerUtils.BenchmarkDotnet.Reporter can be easily integrated into your CI/CD pipeline to automatically analyze and compare benchmark performance. For detailed instructions on setting up GitHub Actions workflows, see the GitHub Actions Setup Guide.

Acknowledgments

This project was inspired by and based on the ResultsComparer tool from the .NET performance repository.

Contribution

If you have any questions, comments, or suggestions, please open an issue or create a pull request.

Want to contribute to this tool? Check out the Test Data Documentation to see sample benchmark reports available for testing your changes and developing new features.