Skip to content

bent: isolate compile benchmark caches - #44

Merged
zhouguangyuan0718 merged 1 commit into
xgo-dev:mainfrom
zhouguangyuan0718:codex/isolate-benchmark-cache
Aug 22, 2026
Merged

bent: isolate compile benchmark caches#44
zhouguangyuan0718 merged 1 commit into
xgo-dev:mainfrom
zhouguangyuan0718:codex/isolate-benchmark-cache

Conversation

@zhouguangyuan0718

Copy link
Copy Markdown
Collaborator

Summary

  • add a Bent BuildCache = "stdlib" policy that prewarms only the standard-library dependencies used by the selected suites
  • clone the standard-library seed into a private Go/LLGo cache for every benchmark/configuration build, outside the timed interval
  • disable setup-go cross-run cache restore for the LLGo binary-size workflow and apply the policy to all six compiler configurations

This keeps downloaded module sources available while preventing compiled third-party packages and target packages from leaking across CI runs or benchmark/configuration pairs. Runtime performance and compatibility jobs are unchanged.

Validation

  • go test ./cmd/bent
  • go test -race ./cmd/bent
  • go test ./...
  • go vet ./cmd/bent
  • actionlint and YAML parse
  • Bent Go build-only smoke with a TOML suite
  • pinned LLGo standard-library prewarm smoke in isolated cache directories

@fennoai fennoai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Isolate benchmark build cache

Solid, well-scoped change. The core design — prewarm a stdlib-only seed, then hard-link-clone a private cache per measured build with cache setup/teardown kept outside the timed window — is correct and validated by TestIsolatedBuildCacheDoesNotLeak. The trim.txt copy-instead-of-link special case shows good awareness of Go's cache-mutation behavior, and validateBuildCache cleanly rejects the UseBuildCache + BuildCache conflict. Docs largely match behavior.

I confirmed the measurement boundary is respected: the clone happens before start := time.Now() and cleanup is deferred, so neither contaminates build-real-ns/op.

Findings below are improvements, none blocking.

Comment thread cmd/bent/build_cache.go
cmd := exec.Command("go", args...)
cmd.Dir = bench.BuildDir()
cmd.Env = env
output, err := cmd.CombinedOutput()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] standardLibraryPackages parses merged stderr as package names

cmd.CombinedOutput() merges stderr into the scanned output, and the scanner only filters empty/unsafe/builtin lines. Any stderr line go list emits on success (toolchain download notices, deprecation warnings, etc.) that looks like an import path gets added to packages and later passed to go install, causing a confusing prewarm failure. Prefer cmd.Output() to parse stdout only, keeping stderr for the error message.

Comment thread cmd/bent/build_cache.go
Comment on lines +57 to +60
runsDir := filepath.Join(dirs.buildCaches, "runs")
if err := os.MkdirAll(runsDir, 0o775); err != nil {
return "", func() {}, err
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Orphaned per-build clone dirs accumulate under build-cache/runs

Each build creates a fresh MkdirTemp(runsDir, "cache-") and relies on the deferred cleanup() to remove it. That cleanup is bypassed on os.Exit paths (e.g. the build-bench-file write error in compileOne) and on any crash/kill, leaving full hard-linked cache trees behind. Unlike per-config seeds (which are RemoveAll'd in prepareCacheSeed), runs/ is never pruned, so orphaned trees accumulate across aborted runs and consume disk. Consider clearing runs/ at startup.

Comment thread ci/llgo-size/README.md
Comment on lines +28 to +30
compiled by an earlier pair. Native Go and LLGo follow the same cache policy;
the workflow also disables setup-go's cross-run build-cache restore. LLGo's
package-cache fingerprint still separates the LTO and plugin configurations.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] setup-go cache:false not applied to pr-validation job despite README claim

The README states "the workflow also disables setup-go's cross-run build-cache restore." This cache: false is set on the matrix job but the pr-validation job also runs Bent with the stdlib cache policy and reports build times, yet its setup-go step omits cache: false. It's functionally harmless (isolated builds and prewarm use private cloned cache dirs), but the blanket README wording overstates coverage. Either scope the sentence to the matrix job or add cache: false here for consistency.

@zhouguangyuan0718
zhouguangyuan0718 merged commit a8f1266 into xgo-dev:main Aug 22, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant