Repository navigation
Conversation
…metrics)
A benchmark could not surface a module counter that explains its result:
the runners only keep memory, CPU and the search memory/disk fields.
- add an optional `info_metrics` list to the benchmark definition (section and
fields), read once after the client tool finished
- log the values as `INFO metrics: {...}` and export them with the other end
of run metrics, in both run-remote and run-local
- sum across shards, skip malformed entries and missing sections, so this
never fails a run
- document the key and cover the collection helper with tests
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
🤖 Automated first-pass review — a human maintainer's review is still required before merge. This is a useful, well-scoped feature and the commit history already does most of the work. Moving the best-effort guard into
Everything else looks good: malformed or non-list config is skipped with a warning, missing sections are simply absent, and |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #574 +/- ##
==========================================
+ Coverage 50.20% 50.36% +0.16%
==========================================
Files 74 74
Lines 9194 9224 +30
==========================================
+ Hits 4616 4646 +30
Misses 4578 4578 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review feedback: the "never fails a run" promise only held in run-remote. - move the guard into collect_info_metrics, so run-remote and run-local cannot drift: INFO errors are logged and yield no metrics, and a value that is not a list is ignored with a warning - drop the now redundant try/except at the run-remote call site - document that numeric scalar values are summed across shards - cover a non-list value and an INFO error with tests Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ics) - collect_info_metrics wrapped every shard in one try, so a TimeoutError from a single slow shard dropped the values of all of them on a cluster - keep the single call for the normal path; only when it raises and there is more than one connection, read the shards one by one, skip the failing ones with a warning and sum the rest, so partial results are still reported - document the behaviour in docs/Readme.md - test the partial and the all-shards-failing cases, and an entry list that is entirely malformed - test the wiring in run_local: a benchmark with info_metrics set reaches the end of run metrics (run_remote needs real infrastructure and stays untested) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
@JoanFM pls review when u can |
JoanFM
left a comment
There was a problem hiding this comment.
Small change, I think this is a great feature!
- review feedback: instead of retrying shard by shard in collect_info_metrics, give collect_redis_metrics an ignore_shard_errors argument so it skips a failing section of a shard itself and the other shards still report - collect_info_metrics goes back to a single call, which also keeps the _shard_N suffix of dict-valued fields when a shard fails - default behaviour is unchanged: without the flag only a ConnectionError is tolerated and any other error still raises - test the flag on collect_redis_metrics directly Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A benchmark could not surface a module counter that explains its result: the runners only keep memory, CPU and the search memory/disk fields.
info_metricslist to the benchmark definition (section and fields), read once after the client tool finishedINFO metrics: {...}and export them with the other end of run metrics, in both run-remote and run-localignore_shard_errorsargument tocollect_redis_metrics: a failing section of one shard is logged and skipped, so the other shards of a cluster still report. Without it only aConnectionErroris tolerated, as before🤖 Generated with Claude Code