Cleanup bench - #118
Merged
Merged
Conversation
The benchmarks measured external checkouts named through the environment, so nobody but their author could run them. They now measure two go-swagger generated projects carried in the repository as one archive, unpacked on demand: dockerctl, a light client over a reasonably large API, and kubeapi, a heavy server over a very large one. Both vendor their dependencies, so a measurement needs a clone and nothing else. Corpus: - a corpus package unpacks the archive on demand and stamps the result with its digest, so a re-rolled archive is never measured stale; - an entry that would escape the destination fails the extraction rather than being relocated inside it; - every load sets GOWORK=off, the trees sitting inside a workspace that does not list them. Harness: - the phase split and the profiling wrapper run over either corpus, not just the client, and every report is behind CODESCAN_BENCH=1; - loader-benchmark builds one probe per released version instead of a single baseline, so the history is a column of the same table; - the discovery and working-set axes go with the exploration they instrumented. Documentation: - one README under the benchmarks package replaces four documents that recorded how the measurements were arrived at. It reports the loader comparison warm and cold, and what six months moved: on kubeapi, 7.1 s and 4555 MB allocated under v0.33.3 against 0.97 s and 447 MB today, for the identical document. The second corpus used to be go-swagger itself, which is a CLI tool rather than an API and whose figures nobody could reproduce. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
A maintainers page recapping the measurement work: the two independent gains since the code left go-swagger — the annotation grammar and the package loader — and how the three loader configurations fare warm and cold. Mermaid xychart diagrams over the kubeapi corpus, which is the server stub go-swagger generates from the kubernetes API manifest, not kubernetes itself. Every bar on the page comes from a scan that emitted the same 222 definitions and 260 paths, which is what makes them comparable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
❌ 2 Tests Failed:
View the top 2 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
The corpus extractor refused an entry by asking the host whether its name was
absolute, and the answer differs: filepath.IsAbs("/x") is false on Windows,
which wants a volume, so the rooted entry Linux refused was accepted there. A
guard whose verdict depends on where it runs is the wrong shape for a guard,
whatever it lets through.
The name is now judged as the slash path the tar format defines — no leading
separator, no ".." element, and no backslash or colon, those being a separator
and a drive marker on one platform and ordinary filename characters on
another. One verdict everywhere.
The containment property is then restated on the joined path, redundantly and
on purpose: it is what this function exists to guarantee, and it is asserted on
the value it returns rather than inferred from the checks above it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
Find resolved the archive first and judged the name afterwards, so asking for a corpus that does not exist paid for 55 MB of extraction before being told it was a typo. It also quietly defeated the measurement gate. The test covering that refusal needs no corpus and was therefore left ungated, but the call behind it unpacked one — on every CI job of every platform, which is the cost the gate exists to keep out of an ordinary `go test ./...`. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Frederic BIDON <fredbi@yahoo.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change type
Please select: 🆕 New feature or enhancement|🔧 Bug fix'|📃 Documentation update
Short description
Fixes
Full description
Checklist