Repository navigation
MOD-19327: Read numbers out of the reply of a command at the end of a run (command_metrics) - #575
Conversation
… run (command_metrics) Some counters are not in INFO, only in the reply of a command, for example the debug info of a RediSearch vector index (FT.DEBUG VECSIM_INFO). The info_metrics of a benchmark cannot reach them. - add an optional command_metrics list to the benchmark definition: a name, the command and the dotted paths of the numbers to read out of its reply - read the reply of alternating names and values (a value may be such a list again) as nested names - run the command once per shard after the client tool finished, sum the numbers across shards, log them as Command metrics and export them with the other end of run metrics, in run-remote and run-local - a malformed entry, a path that is missing or not a number, or a command that fails on a shard is logged and left out, so this never fails a run - document the key and cover the helper and the run-local wiring with tests Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
🤖 Automated first-pass review — a human maintainer's review is still required before merge. This looks good to me, and it follows the
A few small things, none of them blockers:
Coverage: the run-remote wiring isn't tested, which the description says up front. Thanks for the thorough description and the tests. Happy to see this land once someone who knows the run-remote export path has taken a look. (I'd mark this as COMMENTED, leaning toward approval.) |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #575 +/- ##
==========================================
+ Coverage 50.33% 50.62% +0.28%
==========================================
Files 74 74
Lines 9224 9288 +64
==========================================
+ Hits 4643 4702 +59
- Misses 4581 4586 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Follow-up of #574.
info_metricsreadsINFOfields. Some counters are not inINFO, only in the reply of a command, for example the debug info of a RediSearch vector index (FT.DEBUG VECSIM_INFO: id holes, next id, compaction moves and time of the disk vector index). A benchmark could not surface them.command_metricslist to the benchmark definition: aname, thecommand(argument list) and the dottedfieldspaths of the numbers to read out of its replyBACKEND_INDEX.NUM_ID_HOLESis a path into itCommand metrics: {...}and export them with the other end of run metrics, in both run-remote and run-localThe metric names are
<name>_<path with the dots as underscores>.Tests: the helper (nested reply, sums across shards, missing or non-numeric paths, malformed entries, a non-list value, a failing shard, every shard failing) and the run-local wiring against a real
redis-server. The run-remote wiring needs real infrastructure and is not covered.🤖 Generated with Claude Code