[JuhoErvasti]

Description

I think this'll be a useful addition to the ongoing optimization work; this adds a workflow which runs when a PR is opened from branch perf* to main and adds a benchmark report as a comment to the PR.

Should allow us to better keep track of changes. It also lays the groundwork for eventually adding testing for performance regressions, personally I'd like to test this out first and introduce hard limits on regressions when more of the optimizations have already been done.

The report only includes cases with a relative change of at least 25% (either faster or slower). This is an arbitrary number and may have to be adjusted, but we can't go with too low of a number due performance variance in the CI runners.

I've tested this in a personal fork, see here (or no changes).

Type of change

• Bug fix

• New feature

• Other

• Non-breaking change

• Breaking change

Developer checklist

• A CHANGELOG entry has been included (only when change is visible to users)

[JuhoErvasti]

@nlssjritakallio
Ready for review, I changed this a bit so that all the results are reported, but the less meaningful results are hidden in a </details> block, I figure it doesn't hurt to have them available, but hidden by default:

[sritakallio]

Nice tool!