Skip to content

ci: cache tool downloads of the container tests in a local proxy #7670

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
viceice merged 4 commits into from
Oct 8, 2026
Merged

ci: cache tool downloads of the container tests in a local proxy #7670

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d2989fd
ci: cache tool downloads of the container tests in a local proxy
viceice Oct 8, 2026
1ae44d6
ci: show the download proxy cache stats in the job summary
viceice Oct 8, 2026
1ac7df5
ci: print the download proxy cache stats to the job log too
viceice Oct 8, 2026
5648e41
ci: list the download proxy hosts and add a local proxy test
viceice Oct 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/actions/download-proxy/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: 'Download proxy'
description: 'Starts a caching proxy for tool downloads and uses it as CONTAINERBASE_CDN'

runs:
using: 'composite'

steps:
- name: Start download proxy
shell: bash
env:
ACTION_PATH: ${{ github.action_path }}
run: |
echo "::group::Preparing download proxy"
set -ex
if ! command -v nginx; then
sudo apt-get -qq update
sudo apt-get -qq install -y nginx
fi
sudo mkdir -p /etc/containerbase-cdn /var/cache/containerbase-cdn /var/log/containerbase-cdn
sudo cp "${ACTION_PATH}/nginx.conf" "${ACTION_PATH}/allowed-hosts.map" /etc/containerbase-cdn/
sudo nginx -t -c /etc/containerbase-cdn/nginx.conf
sudo nginx -c /etc/containerbase-cdn/nginx.conf
echo "CONTAINERBASE_CDN=http://host.docker.internal:8099" >> "${GITHUB_ENV}"
echo "::endgroup::"

- name: Check download proxy
shell: bash
run: |
curl -sSf -o /dev/null http://127.0.0.1:8099/nodejs.org/dist/index.json
29 changes: 29 additions & 0 deletions .github/actions/download-proxy/allowed-hosts.map
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Hosts the download proxy may fetch from: the tool download and lookup hosts
# from docs/custom-registries.md, plus the hosts their downloads redirect to.
api.adoptium.net 1;
api.nuget.org 1;
builds.dotnet.microsoft.com 1;
cdn.dl.k8s.io 1;
dist.nuget.org 1;
dl.google.com 1;
dl.k8s.io 1;
download.docker.com 1;
download.swift.org 1;
downloads.gradle.org 1;
downloads.haskell.org 1;
downloads.lightbend.com 1;
get.helm.sh 1;
github.com 1;
go.dev 1;
mise.jdx.dev 1;
nodejs.org 1;
objects.githubusercontent.com 1;
pypi.org 1;
registry.npmjs.org 1;
release-assets.githubusercontent.com 1;
releases.hashicorp.com 1;
repo.maven.apache.org 1;
rubygems.org 1;
services.gradle.org 1;
static.rust-lang.org 1;
storage.googleapis.com 1;
11 changes: 11 additions & 0 deletions .github/actions/download-proxy/errors.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash

# Prints what the download proxy rejected or failed to fetch, so a missing
# host in allowed-hosts.map shows up in the job log.

set -e

echo "::group::Download proxy errors"
sudo cat /var/log/containerbase-cdn/error.log || true
sudo grep -E '" (403|5[0-9]{2}) ' /var/log/containerbase-cdn/access.log || true
echo "::endgroup::"
109 changes: 109 additions & 0 deletions .github/actions/download-proxy/nginx.conf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Caching reverse proxy for CONTAINERBASE_CDN in the container tests.
# The CLI rewrites https://<host>/<path> to <cdn>/<host>/<path>, so this proxy
# maps the first path segment back to the upstream host and caches the result.
worker_processes auto;
error_log /var/log/containerbase-cdn/error.log warn;
pid /run/containerbase-cdn.pid;

events {
worker_connections 1024;
}

http {
# with the requested host, the host after redirects and the cache status,
# which stats.sh sums up
log_format cdn '$remote_addr [$time_local] "$request" $status $body_bytes_sent host=$upstream_host final=$final_host cache=$upstream_cache_status';
access_log /var/log/containerbase-cdn/access.log cdn;

# systemd-resolved stub of the runner
resolver 127.0.0.53 ipv6=off valid=300s;

# keep space for the docker builds on the runner disk
proxy_cache_path /var/cache/containerbase-cdn levels=1:2 keys_zone=cdn:10m
max_size=4g min_free=2g inactive=1d use_temp_path=off;

# split the raw request uri, so encoded characters like `%2F` in a path
# reach the upstream unchanged
map $request_uri $upstream_host {
default '';
~^/(?<cdn_host>[^/?]+)/ $cdn_host;
}

map $request_uri $upstream_rest {
default '';
~^/[^/?]+/(?<cdn_rest>.*)$ $cdn_rest;
}

# only known download hosts, so this is no open proxy
map $upstream_host $upstream_allowed {
default 0;
include /etc/containerbase-cdn/allowed-hosts.map;
}

# recomputed for every redirect hop, map results are cached otherwise
map $upstream_http_location $redirect_host {
volatile;
default '';
~^https://(?<location_host>[^/:]+)(:443)?/ $location_host;
}

map $redirect_host $redirect_allowed {
volatile;
default 0;
include /etc/containerbase-cdn/allowed-hosts.map;
}

server {
listen 8099;

# the runner itself and the docker networks of the builds
allow 127.0.0.1;
allow 172.16.0.0/12;
deny all;

proxy_ssl_server_name on;
proxy_ssl_verify on;
# the default depth of 1 rejects chains with an intermediate certificate
proxy_ssl_verify_depth 4;
proxy_ssl_trusted_certificate /etc/ssl/certs/ca-certificates.crt;
proxy_http_version 1.1;
proxy_read_timeout 300s;
# GitHub's redirects carry long signed urls and large security headers
proxy_buffer_size 64k;
proxy_buffers 8 64k;
proxy_busy_buffers_size 128k;
proxy_cache cdn;
# cache by the requested url, also for responses fetched after a redirect
proxy_cache_key $request_uri;
# cache downloads for the job, whatever the upstream cache headers say
proxy_cache_valid 200 1d;
proxy_ignore_headers Cache-Control Expires Set-Cookie;
proxy_hide_header Set-Cookie;
proxy_cache_lock on;
proxy_cache_lock_timeout 300s;
add_header X-Cache-Status $upstream_cache_status always;

# follow upstream redirects here, so the client never leaves the proxy
proxy_intercept_errors on;
recursive_error_pages on;

location / {
set $final_host $upstream_host;
if ($upstream_allowed = 0) {
return 403 "host not allowed: $upstream_host\n";
}
proxy_pass https://$upstream_host/$upstream_rest;
error_page 301 302 303 307 308 = @redirect;
}

location @redirect {
set $final_host $redirect_host;
if ($redirect_allowed = 0) {
return 403 "redirect not allowed: $upstream_http_location\n";
}
set $redirect_location $upstream_http_location;
proxy_pass $redirect_location;
error_page 301 302 303 307 308 = @redirect;
}
}
}
64 changes: 64 additions & 0 deletions .github/actions/download-proxy/stats.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/bin/bash

# Prints how many requests the download proxy served from its cache and which
# hosts the builds downloaded from, to the job log and the job summary.
# Usage: stats.sh [access log], defaults to the log on the runner.

set -e

log="${1:-/var/log/containerbase-cdn/access.log}"
summary="${GITHUB_STEP_SUMMARY:-/dev/null}"

# reads the access log, with sudo when it belongs to root as on the runner
read_log() {
if [ -r "${log}" ]; then
cat "${log}"
else
sudo cat "${log}"
fi
}

# the proxy wasn't started, eg. the job failed before
if [ ! -f "${log}" ]; then
exit 0
fi

{
echo "### Download proxy cache"
echo ""
echo "| Cache status | Requests |"
echo "| --- | --- |"
read_log | grep -o 'cache=[A-Z]*' | sort | uniq -c | while read -r count status; do
status="${status#cache=}"
echo "| ${status:-none} | ${count} |"
done

echo ""
echo "### Download proxy hosts"
echo ""
echo "| Host | Requests | Cache hits | Redirects to |"
Comment on lines +35 to +39

@qodo-code-review qodo-code-review Bot Oct 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Cached builds hide download hosts 📎 Requirement gap ◔ Observability

stats.sh prints Download proxy hosts from the current access log without warning that cached
build steps generate no proxy requests. When Bake reuses an image layer, its download hosts are
absent from the table, so someone using the summary to plan network allowlisting may miss hosts
needed for a clean build.
Agent Prompt 
## Issue description
The new host table does not explain that cached build steps can omit hosts needed by a clean build.

## Fix Focus Areas
- .github/actions/download-proxy/stats.sh[35-40]

## Recommended Fix
Add a visible note beside the host table stating that it reflects only requests observed by the proxy in this job and that cached build steps can leave required hosts out. Keep the note in both the job log and summary.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Dismiss ↗ | View ↗

echo "| --- | --- | --- | --- |"
# fields are host=, final= and cache=, see the log_format in nginx.conf
read_log | awk '
{
host = ""; final = ""; cache = ""
for (i = 1; i <= NF; i++) {
if ($i ~ /^host=/) host = substr($i, 6)
if ($i ~ /^final=/) final = substr($i, 7)
if ($i ~ /^cache=/) cache = substr($i, 7)
}
# requests without a host, like a readiness check of `/`
if (host == "") next
requests[host]++
if (cache == "HIT") hits[host]++
if (final != "" && final != host && index(targets[host], final) == 0) {
targets[host] = targets[host] (targets[host] == "" ? "" : ", ") final
}
}
END {
for (host in requests) {
printf "| %s | %d | %d | %s |\n", host, requests[host], hits[host], targets[host]
}
}
' | sort
} | tee -a "${summary}"
73 changes: 73 additions & 0 deletions .github/actions/download-proxy/test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
#!/bin/bash

# Tests the download proxy config locally: starts nginx with nginx.conf in a
# docker container on port 8099, checks downloads, redirects, encoded paths,
# the cache and the host allowlist, then prints the stats.
# Usage: .github/actions/download-proxy/test.sh (needs docker and curl)

set -e

dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
logs="$(mktemp -d)"
name=containerbase-cdn-test
proxy=http://127.0.0.1:8099
failed=0

# removes the container and the log folder
# shellcheck disable=SC2329 # called by the EXIT trap
cleanup() {
docker rm -f "${name}" > /dev/null 2>&1 || true
rm -rf "${logs}"
}
trap cleanup EXIT

# the nginx image runs as root, so the log folder must be writable for it
chmod 777 "${logs}"

docker run -d --name "${name}" --network host \
--tmpfs /var/cache/containerbase-cdn \
-v "${logs}:/var/log/containerbase-cdn" \
-v "${dir}:/etc/containerbase-cdn:ro" \
nginx:stable nginx -c /etc/containerbase-cdn/nginx.conf -g "daemon off;" > /dev/null

# waits until the proxy answers
for _ in $(seq 1 30); do
curl -s -o /dev/null "${proxy}/" && break
sleep 1
done

# requests a path through the proxy and compares status and cache status
# usage: check <path> <expected status> [expected cache status]
check() {
local path="$1" status="$2" cache="${3:-}"
local result
result="$(curl -s -o /dev/null -w '%{http_code} %header{x-cache-status}' "${proxy}/${path}")"
local got_status="${result%% *}" got_cache="${result#* }"
if [ "${got_status}" = "${status}" ] && { [ -z "${cache}" ] || [ "${got_cache}" = "${cache}" ]; }; then
echo "ok ${got_status} ${got_cache} ${path}"
else
echo "FAIL ${got_status} ${got_cache} ${path} (expected ${status} ${cache})"
failed=1
fi
}

# GitHub release asset, redirects to release-assets.githubusercontent.com
check github.com/nubjs/nub/releases/download/v0.9.3/nub-linux-x64.tar.gz.sha256 200 MISS
check github.com/nubjs/nub/releases/download/v0.9.3/nub-linux-x64.tar.gz.sha256 200 HIT
# two redirects
check github.com/containerbase/maven-prebuild/releases/latest/download/version 200
# encoded `%2F` in the release tag
check 'github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv5.8.3/checksums.txt' 200
# redirects to cdn.dl.k8s.io
check dl.k8s.io/release/v1.37.1/bin/linux/amd64/kubectl.sha256 200
# lookup with a query string
check 'api.adoptium.net/v3/info/release_versions?architecture=x64&image_type=jdk&os=linux&page_size=1&release_type=ga&version=%5B21%2C22%29' 200
# missing file is passed through
check github.com/nubjs/nub/releases/download/v0.0.0-missing/nub-linux-x64.tar.gz 404
# host not in allowed-hosts.map
check example.com/ 403

echo ""
"${dir}/stats.sh" "${logs}/access.log"

exit "${failed}"
35 changes: 35 additions & 0 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,9 @@ jobs:
- name: Check system
uses: ./.github/actions/check

- name: ⚙️ Download proxy
uses: ./.github/actions/download-proxy

- name: ⚙️ Init
run: |
echo "OWNER=${OWNER,,}" >> "${GITHUB_ENV}"
Expand All @@ -269,6 +272,14 @@ jobs:
with:
args: test-distro

- name: Download proxy errors
if: failure()
run: .github/actions/download-proxy/errors.sh

- name: Download proxy cache stats
if: always()
run: .github/actions/download-proxy/stats.sh

# test old distros on arm64
base-arm64:
runs-on: ubuntu-24.04-arm
Expand Down Expand Up @@ -300,6 +311,9 @@ jobs:
- name: Check system
uses: ./.github/actions/check

- name: ⚙️ Download proxy
uses: ./.github/actions/download-proxy

- name: ⚙️ Init
run: |
echo "OWNER=${OWNER,,}" >> "${GITHUB_ENV}"
Expand All @@ -309,6 +323,14 @@ jobs:
with:
args: test-base

- name: Download proxy errors
if: failure()
run: .github/actions/download-proxy/errors.sh

- name: Download proxy cache stats
if: always()
run: .github/actions/download-proxy/stats.sh

lang:
runs-on: ${{ matrix.arch.os }}
name: ${{ matrix.lang }} (${{ matrix.arch.name }})
Expand Down Expand Up @@ -387,11 +409,24 @@ jobs:
--set settings.cache-from+=type=gha,scope=${{ needs.setup.outputs.uid }}-${{ matrix.arch.tag }}
build-docker

# only after the base image, so it builds with the same args as in the
# `base` job and is taken from the GHA cache
- name: ⚙️ Download proxy
uses: ./.github/actions/download-proxy

- name: test distro
uses: ./.github/actions/bake
with:
args: test-${{ matrix.arch.name }}

- name: Download proxy errors
if: failure()
run: .github/actions/download-proxy/errors.sh

- name: Download proxy cache stats
if: always()
run: .github/actions/download-proxy/stats.sh

# Catch-all required check for the test matrix, `base` is listed too because
# `lang` is skipped when it fails
test-success:
Expand Down
Loading