Repository navigation
Conversation
acamarata
force-pushed
the
p7/P7-CI-46
branch
2 times, most recently
from
October 19, 2026 04:06
525700e to
8b121a8
Compare
Contributor
Author
|
Parked ready, not merged. Required check passes; independent Sonnet review (clean checkout) found no must-fix and its should-fix items are in. |
… toolchain (P7-CI-46)
…scenario (P7-CI-46)
…very hosted baseline (P7-CI-46)
acamarata
force-pushed
the
p7/P7-CI-46
branch
from
October 20, 2026 12:08
3875be0 to
1dd4c91
Compare
added 5 commits
October 20, 2026 08:24
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
free/ci/bench: Go benchmarks plus thecibenchprocess harness that times the V15 engine against the frozen v1.4 gate (startup, detect, plan, store open, executor overhead, end-to-end on a tiny Go and a medium Go plus Node fixture, service start behind-services). It emitsperfbench/v1result JSON that the cli's ownperfbench checkreads. Budgets:perf-budget.json(linux/amd64) andperf-budget-darwin-arm64.json, each the worst p95 of three 20-sample runs plus 25 percent.ci-bench.ymlruns on PRs that touchfree/ci/: harness tests, measure,perfbench checkfrom a pinned cli checkout, the relative budget, and controls (a result ten times over budget must fail both checks).Result you need to read first
The relative budget (V15 tiny median <= v1.4 tiny median x 1.10 + 250 ms) is missed on both platforms: linux/amd64 about 1.3 s against a 0.59 s limit, macos-arm64 about 1.5 s. Cause (measured, not guessed): the exec funnel forks
ps -Athree times per tracker snapshot for every funnel call, and a bare run makes about 22 funnel git calls; strace counts 135psexecs in one tiny run. A miss fails the Ticket unless the owner accepts a written exception, soci-bench.ymlfails its ratio step until the engine is fixed or an exception is committed inbench/ratio-exceptions.json(an entry withoutaccepted_bydoes not count). Evidence and the exception request: hqplan/p7/evidence/CI-46-perf.md.Also found: on Linux the engine runs every job under
ulimit -vfrom a 256 MB default demand, which killsgoandnodeat start. The fixtures setrequires.mem_mbto work around it.Scope notes
architecture/layers.yaml(rows forbenchandbench/cmd/cibench),architecture/layers_test.go(os/exec exemption forbench/spawn.goonly),internal/nodes/provision/single_exec_test.go(bench/spawn.go:spawn). The timing tool must not run inside the engine funnel whose overhead it measures.perfbench checkdefaults to-min-n 20; the Ticket's N=10 needs-min-n 10on that verify line. The workflow uses 20 runs and the default.