Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m0/ci/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,5 +136,5 @@ The historical samples locate candidates; they are not controlled benchmarks.
- [Replay full-run failures against the selected gate](/quest/m0/ci/select-audit.md) - every dependency edge `just ci` misses is found and fixed with a test
- [First shared FFI release](/quest/m0/ci/nightly-ffi-reuse.md) - condition: the backport merges and the next `moq-ffi-v*` tag runs `Release FFI`; verify the wrappers
- [Fewer BTreeMap instantiations](/quest/m0/ci/btree-keys.md) - moq-net shares BTreeMap key and value types, kept only if its CPU time drops
- [Measure moving unit tests out](/quest/m0/ci/test-layout.md) - report what moq-net would save by not compiling its library twice for unit tests
- [Halve the slowest tests' CPU](/quest/m0/ci/slow-tests.md) - the slowest 1% of tests take under half their 289 CPU-seconds, with coverage kept
- [Measure moving unit tests out](/quest/m0/ci/test-layout.md) - deprioritized: keep inline unit tests unless a profile shows moq-net's second library build is worth a migration
44 changes: 28 additions & 16 deletions quest/m0/ci/test-layout.md
Original file line number Diff line number Diff line change
@@ -1,24 +1,36 @@
# [S] Measure moving unit tests out of hub crates
# [L] Measure moving unit tests out of hub crates

## Goal

A report on what CI CPU time moq would save if hub crates, starting with moq-net,
stopped compiling their library twice: once as the library and once as the unit
test target, with unit tests about 52% of moq-net's `src` lines. It estimates the
saving, from a prototype on moq-net, of `[lib] test = false` plus one
integration-test crate that reaches private items through a hidden module or a
feature (moq-net already has a `fuzz` feature for this). It recommends whether to
restructure, and how.
Decide from measured CI CPU time whether hub crates, starting with moq-net,
should move unit tests into external harnesses (`[lib] test = false`) so the
library stops compiling a second time as the unit-test target. Until that
evidence exists, moq keeps its inline unit tests.

## Plan

Decided with the maintainer on 2026-10-25: measure before restructuring. It is
potentially the largest single win, since every moq-net change in CI compiles
both targets, but it changes how every test is written.
Decided with the maintainer on 2026-10-25: measure before restructuring.

- Count how many unit tests use private items, and what a test-only module would
have to expose.
- Measure CPU for a moq-net edit plus its tests, before and after, with mbx warm.
- Weigh the churn, and say whether moq-net is the only crate worth doing.
Deprioritized 2026-10-26: a first pass found no evidence of savings and a
large migration cost, so keep the inline layout. Load-bearing facts, at main
5516a406f4:

Public API: none, unless the hidden test surface leaks; say so if it does.
- About half of moq-net's source is test code, and its library harness has
about 2.7k tests. moq-mux has a similar share.
- Roughly 1k of those tests name private items. Moving them needs a
test-support surface that does not exist. The `fuzz` feature is not one: it
is public and implies `async`, which would break the no-default-feature
boundary.
- A four-case prototype worked mechanically but measured nothing: the shared
host was loaded and cache state differed between runs.
- Interop, bare-FIN, the feed/fetch benchmark, and `sh/rs/net-features.sh`
select library tests, so a migration must carry them too.

Revisit only if a profile, after the thin-generics and BTreeMap work, shows the
unit-test target's library build is still a large share of a moq-net edit's
CPU. A credible comparison then needs a settled test-support design (or
private-state tests kept inline), a prototype that preserves every compiled test
across feature sets, and identical edit-plus-tests runs with mbx warm on a quiet
host.

Public API: none, unless the test-support surface leaks; say so if it does.
Loading