Allow engine flags to be specified multiple times - #328
Conversation
Change `--engine-flags` to accept multiple values and pair them with the engines
as follows:
- A single engine with multiple flag sets is paired with each set
- Multiple engines with zero or one flag set pairs each engine with the flag
set (if any)
- N engines and N flag sets are paired together by position: `engines[i]` with
`flags[i]`
- All other combinations are errors.
This allows, for example, comparing the performance effects of different flags
for the same engine, e.g. the DRC vs copying collector:
```
sightglass-cli benchmark \
-e engines/wasmtime/libengine.dylib \
--engine-flags "-Ccollector=copying" \
--engine-flags "-Ccollector=drc" \
-- benchmarks/splay/splay.wasm
```
|
Thanks for this; the semantics you've described make reasonable sense each in isolation, but I worry that this interface will be too confusing/complex for a user to invoke and have reasonable expectations for. Especially the transparent shift between one-flags-applies-to-all and flags-apply-pairwise is a mistake waiting to happen: forgetting some flag options could switch modes transparently and cause unexpected flags to apply to later engines. I wonder if we could disambiguate intent with two different flags:
What's more, they could also compose, which might be really useful: specify common flags once, but also add flags to each run individually. This seems to me to be a minor change to your logic, and overall simplifies the semantics / makes them more understandable. What do you think? |
I like this, will implement it shortly |
|
@cfallin ready for re-review! |
2 participants
Change
--engine-flagsto accept multiple values and pair them with the engines as follows:A single engine with multiple flag sets is paired with each set
Multiple engines with zero or one flag set pairs each engine with the flag set (if any)
N engines and N flag sets are paired together by position:
engines[i]withflags[i]All other combinations are errors.
This allows, for example, comparing the performance effects of different flags for the same engine, e.g. the DRC vs copying collector: