diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5022c2c..ea1b27f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,4 +10,5 @@ 3. New API needs tests. Overwrite-on-full, empty/full, move-only types, and allocators are required edges. 4. Do not add third-party test frameworks or runtime dependencies. Benches may optionally find Boost to compare against `boost::circular_buffer`; that does not make Boost a test or library dependency. -5. After hot-path changes, run `./scripts/dev.sh bench` if Boost is available. Do not treat a single noisy ratio as a pass/fail; use the grouped geomeans (hot/slide, scan, structural) plus the scenarios you meant to change. CI `release-bench` runs the Boost comparison on Ubuntu GCC/Clang (x64 and arm64) and macOS Clang and uploads `bench-*.txt`; same rule — do not gate merges on those ratios. GCC work on a Mac: `colima start && ./scripts/dev.sh bench-gcc`, then `./scripts/dev.sh compare-bench` against `./scripts/dev.sh fetch-ci-bench`. +5. After hot-path changes, follow [docs/BENCH.md](docs/BENCH.md): one idea, full Boost suite, keep-or-revert, then a remote `release-bench` if the change is GCC-shaped. Apple Clang does not predict Ubuntu GCC. +6. Comments, commit messages, GitHub issues, and PR titles/bodies are **English**. diff --git a/README.md b/README.md index 6427033..68dc929 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,7 @@ Hot `push_*` / `pop_*` bump `first_` / `last_` around `[buff_, limit_)`. A full If CMake finds Boost, the bench compares `eqz::circular_buffer` to `boost::circular_buffer` on the same workloads (best of 5). It prints ns/op and geomean `eqz/boost` (`< 1` means eqz is faster), grouped as hot/slide, scan, and structural. Boost is optional and is not a library or test dependency. -CI job `release-bench` installs Boost and runs the suite on several compilers (Ubuntu GCC 13/Clang, Ubuntu arm64 GCC, macOS Apple Clang). Shared runners are noisy; those logs are a record, not a pass/fail gate on ratios. +CI job `release-bench` installs Boost and runs the suite on several compilers (Ubuntu GCC 13/Clang, Ubuntu arm64 GCC, macOS Apple Clang). Shared runners are noisy; those logs are a record, not a pass/fail gate on ratios. Keep-or-revert, how to read GCC vs Clang, and when to push an experiment branch: [docs/BENCH.md](docs/BENCH.md). On a Mac, Apple Clang numbers do not predict Ubuntu GCC. Closest local twin: @@ -121,10 +121,10 @@ tests/ zero-dependency tests benches/ micro-benchmarks examples/ usage + Josephus scripts/dev.sh configure / build / test / bench / format / sanitize -docs/ coding standards and design +docs/ coding standards, design, and bench process ``` -See [docs/DESIGN.md](docs/DESIGN.md) and [docs/CODING_STANDARDS.md](docs/CODING_STANDARDS.md). +See [docs/DESIGN.md](docs/DESIGN.md), [docs/CODING_STANDARDS.md](docs/CODING_STANDARDS.md), and [docs/BENCH.md](docs/BENCH.md). ## License diff --git a/docs/BENCH.md b/docs/BENCH.md new file mode 100644 index 0000000..2b51786 --- /dev/null +++ b/docs/BENCH.md @@ -0,0 +1,79 @@ +# Benchmarking and hot-path optimization + +How we decide whether a micro-opt stays. The suite lives in `benches/bench_circular_buffer.cpp`. Tricks already measured slower or wrong are listed under **Hot path** in [DESIGN.md](DESIGN.md) — do not re-try them without new evidence. + +## Suite + +Release build (`CMAKE_BUILD_TYPE=Release` → GCC/Clang **`-O3 -DNDEBUG`**). Assertions (`EQZ_ASSERT`) are off. CI and `./scripts/dev.sh bench` use this preset. Debug (`default`) is not a bench. + +The binary does **not** turn on LTO, PGO, or `-march=native`. The hot path is header-only and already in the bench TU; LTO is not a missing switch. + +If CMake finds Boost, the run compares `eqz::circular_buffer` to `boost::circular_buffer` (and a small `vector_ring` / `deque` on a few scenarios). Boost is optional and is not a library or test dependency. + +Defaults: `n=1e6`, `cap=4096`, best-of-5. Ratio is **eqz ns/op ÷ Boost ns/op** (`< 1` means eqz is faster). Output groups: + +| Group | What it stresses | +| --- | --- | +| `hot/slide` | overwrite `push_*`, fill, queue, strings, `try_push` | +| `scan` | linear / wrapped iterate and `operator[]` | +| `structural` | copy, insert, `linearize`, move | + +The banner prints `compiler=`, `stdlib=libc++|libstdc++`, `alloc_empty=yes|no`. `alloc_empty=yes` is required for the trivial-`T` `memcpy` copy path. + +```bash +./scripts/dev.sh bench # Apple Clang / local toolchain +colima start && ./scripts/dev.sh bench-gcc # Linux GCC 13 + libstdc++ in Docker +./scripts/dev.sh fetch-ci-bench # latest successful CI artifacts +./scripts/dev.sh compare-bench a.txt b.txt [a_label b_label] +``` + +CI job `release-bench` (workflow `ci.yml`) installs Boost and uploads `bench-/bench.txt` for: + +- `linux-x64-gcc13` +- `linux-x64-clang-libstdcxx` +- `linux-x64-clang-libcxx` +- `linux-arm64-gcc13` +- `macos-arm64-clang` + +Shared runners are noisy. Those logs are a record, not a merge gate on raw geomean. + +## Keep-or-revert + +One change, one decision. Do not stack unproven patches. + +1. Save a baseline log (`./scripts/dev.sh bench | tee build/bench-rounds/00-baseline.txt`). +2. Apply **one** idea. Rebuild only `bench_circular_buffer` if the header is the only change. +3. Run the **full** suite. Do not keep a patch because one scenario moved. +4. Keep if the scenarios you meant to change improved **and** Apple Clang / Linux Clang group geomeans did not regress. Look at **eqz ns/op**, not only the ratio (Boost moves too). +5. Otherwise restore the header (`git checkout -- include/eqz/circular_buffer.hpp`) and try the next idea from the same baseline. + +Local Apple Clang does **not** predict Ubuntu GCC 13. A keep on a Mac is necessary, not sufficient, for a GCC-shaped change. + +## Remote check + +After a candidate survives local keep-or-revert: + +1. Put that **single** idea on its own branch (do not combine R1+R2 on one push). +2. Push so `ci.yml` runs tests **and** all five `release-bench` jobs. +3. Download artifacts (`gh run download ` or `./scripts/dev.sh fetch-ci-bench`). +4. Compare each platform to `main`, especially `linux-x64-gcc13` and `macos-arm64-clang`. + +Same Ubuntu x64 runner, Clang 18 vs GCC 13, is a **compiler** split. Clang + libc++ vs Clang + libstdc++ is usually a wash. Do not retune wrap / `bump` / `drop` from a GCC-only ratio. + +## How to read the numbers + +- **Merge gates** (soft, from the Clang-tuned tree): macOS Apple Clang geomean stays in the same band (~0.57) with scan still 7/7; Linux Clang `hot/slide` stays ≤ ~1.0. GCC geomean is **not** a merge gate. +- Prefer **eqz ns/op** when Boost is bimodal. On GCC 13, `slide/u64/4096` Boost has been ~5–7 ns while `slide/u64/4097` Boost is ~1.1 ns; eqz ~5 ns on both looks like a 4096 WIN and a 4097 loss for the same code. +- `try_push/full` printing `0.00 ns/op` is a timer floor. A jump to ~0.7 ns with Boost still `0.00` explodes the ratio (~100–500×) and drags geomean; that is not an overwrite regression. Exclude it when comparing geomeans, or look at eqz ns. +- `copy/full` / `copy/wrapped` allocate 4096 × 8 B, `memcpy` (when `trivial_memcpy` fires), then `delete`, every iteration. A few hundred ns looks like memcpy; ~11 µs looks like first-touch / heap noise **or** a TU that stopped taking memcpy. The same source has shown both on GCC 13 CI. Confirm with Godbolt (`memcpy@PLT`) before touching the copy path. Do not treat one 0.5× or 6.9× as memcpy flipping on or off. +- `fill/not_full` and `burst/clear_refill` are inlinable lambdas. A cold `call` that Boost inlines will show up here as ~4× on GCC even when Apple Clang is flat. + +## Compiler-specific holes + +If Clang is near Boost and GCC is 3–4× on the same `emplace_*` shape: + +1. Godbolt **x86-64 GCC 13.3** and **Clang 18**, `-O3 -std=c++20 -DNDEBUG`, on a small probe **and** the full bench TU. +2. List distinct ideas (one call removed, one attribute dropped, locals, …). Try each independently from baseline — local full bench, then a remote branch each. +3. A patch that fixes GCC overwrite but sends GCC `copy/full` to ~11 µs is a discard, even if `slide/u64/4097` looks perfect. + +Keep `length_error` for `capacity()==0` in a cold outlined helper. Do not inline `throw` into `emplace_*`. diff --git a/docs/CODING_STANDARDS.md b/docs/CODING_STANDARDS.md index 087d899..539d6fc 100644 --- a/docs/CODING_STANDARDS.md +++ b/docs/CODING_STANDARDS.md @@ -81,6 +81,7 @@ Exception safety: - Internals: comment *why*, not what the next line does. - Comments are **English**. - No large blocks of commented-out code. Design notes belong in `docs/DESIGN.md`. +- How we bench and keep-or-revert hot-path changes: [docs/BENCH.md](BENCH.md). ## Tests @@ -90,5 +91,6 @@ Exception safety: ## Commits -- One concern per commit. Messages in the imperative mood, with the reason. +- One concern per commit. Messages in the imperative mood, with the reason, in **English**. +- GitHub issues, PR titles, and PR bodies are **English**. - Do not commit `build/`, `compile_commands.json`, or IDE directories. diff --git a/docs/DESIGN.md b/docs/DESIGN.md index 3fa8598..df5c3fb 100644 --- a/docs/DESIGN.md +++ b/docs/DESIGN.md @@ -74,7 +74,7 @@ Fill the buffer before `std::iota(begin(), end(), 1)`, or call `resize(n)` first - Skip `allocator_traits::destroy` when `T` is trivially destructible. - `EQZ_ALWAYS_INLINE` on bump/drop/`ptr_at`/emplace/`overwrite_at`/`[]`/`front`/`back`/`empty`/`full`/`pop_*`. gcc/clang only; other compilers get `inline`. -Measured slower or incorrect on the Boost suite, so these stay out: +Measured slower or incorrect on the Boost suite, so these stay out (do not re-try without new Godbolt + keep-or-revert evidence; process in [BENCH.md](BENCH.md)): - Power-of-two `capacity` + bitmask wrap. - Boost-style `T*` linear iterators (logical index iterators stay).