Repository navigation
security: remediate 2026-10-20 emSign CA incident in bundled certifi/pip and uv #284
Description
New 2026-10-20 CA-compromise advisories now block routine AMD64 CI
Observed failure: PR #282 documentation-only final head 1a3ff05f60c11c8f1c15510a38ecebf746262e84, Build AMD64 run 39546063063. The exact-head image build, 53-command/role functional smoke, SPDX SBOM generation and image Trivy scans completed; the mandatory fixable-CRITICAL gate then failed. CI log:
ERROR: fixable CRITICAL vulnerabilities in base image:
- GHSA-6q48-6c7w-9gw2 | certifi | 2026.6.17 -> 2026.10.19 | Python
ERROR: fixable CRITICAL vulnerabilities in Remote Dev image:
- GHSA-6q48-6c7w-9gw2 | certifi | 2026.6.17 -> 2026.10.19 | Python
ERROR: vulnerability gate found 2 fixable CRITICAL finding(s)
These are two image reports of one advisory/package family, not proof of two different underlying vulnerable installs. The log says 12 additional no-fix critical findings per image remain visible but do not trigger the current fixable-critical gate. Full artifact trivy-base.json / trivy-remote-dev.json should be inspected to identify exact vulnerable Target package path. Do not guess origin from a package name alone.
Temporal cause: Certifi GHSA published 2026-10-20, after the last successful CI run. This is not evidence that the docs/Compose digest-pinning correction caused the vulnerability. Official reports:
- GHSA-6q48-6c7w-9gw2
- https://discuss.python.org/t/information-for-python-users-regarding-emudhra-emsign-ca-certificate-security-incident/109576
Broader CA incident / second pinned dependency: current versions.env, mise.toml, mise.lock, Dockerfile still pin uv 0.13.1, separately affected by GHSA-hq85-5cpj-mr62. Upstream Python announcement explicitly recommends certifi >=2026.10.19, pip >=26.2.2, uv >=0.13.4, and rebuilding affected images. Available official upstream releases on 2026-10-21: uv 0.13.4 (2026-10-20) and python-build-standalone 20261020 (includes a pip 26.2.1->26.2.2 update). Investigate which exact Python standalone builds/versions for both linux-x64 and linux-arm64 include the fixed CA store.
Required focused security PR (separate from #282)
- Obtain full Trivy JSON report artifact(s) and locate certifi path(s). Confirm whether Python-standalone packaged
pip/_vendor/certifior another embedded component is responsible; validate presence of a safe patched distribution. Do not install an unverified wheel or delete scanner evidence. - Review pinned replacement Python standalone binaries / pip vendoring, all architecture-specific immutable URLs/checksums/provenance and runtime versions. Regenerate genuine
mise.lockand any affected PyPI/uv sidecars with pinned toolchain; updateversions.env,mise.toml, Docker build arguments, updater/check scripts, inventory/licensing/notices, source release documentation as appropriate. Maintain Python 3.14 default + 3.15 selectable contract and project precedence/fail-closed behavior. - Upgrade affected pinned
uvto at least 0.13.4 and ensure its own vendored certificate bundle is corrected. Inspect if other vendored CA consumers are affected. - Rebuild both image stages from the new reproducible locks, run full functional smokes for Codex + Antigravity, verify valid licenses/notices/SPDX, and require new exact-head Trivy scans + no-fixable-critical gate green; explicitly confirm
certifi 2026.6.17is absent from the affected package path(s). No temporarytrivyignore, gate bypass or lowering of severity. - Open a dedicated PR from current main; do not change stable/latest tags, deploy to TrueNAS or mix fixes with documentation PR docs: TrueNAS persistence setup and post-PR2 release alignment #282. Once merged with normal approval, rebase/revalidate docs: TrueNAS persistence setup and post-PR2 release alignment #282 (whose failure is upstream base-image security drift) before considering any merge. Security incident may warrant out-of-cycle maintenance: recurring third-party license and distribution review #53 third-party review.
Why: safeguard TLS trust while maintaining the existing strict security gate, rather than regressing its policy.
Activity
security: repin CA-safe uv and Python standalone builds (#284)
eXPerience83 commentedon Oct 21, 2026
Evidence update — exact original Trivy/SPDX artifacts and active security PR
Downloaded full official #39546063063 artifact
security-and-sbom-reports(artifact 12322085226; both base/final SPDX and Trivy). The reported vulnerablecertifi 2026.6.17appears in bothTarget: Python,Type: python-pkg, same base-image layersha256:17194096868e81a619be2627d344460a9931744f2943b7a900db2edd54f8da20, aspkg:pypi/certifi@2026.6.17. Package records were reported as SBOM-discovered and don't expose a per-fileFilePath. Both generated SPDX reports also explicitly listpip 26.2.1(twice for the bundled Python interpreters), alongsidecertifi 2026.6.17. This is consistent with the two older Python standalone distributions and verifies the issue is present in the common base before runtime addition; the updated Python-build-standalone source is not selected speculatively.Focused draft security PR #286 created from current main with:
First CI 39550085130 passed repo configuration validation and failed early on a newly introduced Dockerfile shell interpolation typo in the pip security check's cache cleanup (not a lock resolver error). This typo has been corrected at head
1778353fcb2f02e081113b99d3b467ef422c992b; new full CI and true archive-derived notice validation are pending.PR #283 still proposes uv 0.13.3, which fails the 0.13.4 CA advisory floor, and its latest PR-run failure is the same Trivy certifi gate. It is separate, non-security upstream maintenance (mise/Context7/Antigravity review metadata) and should be superseded rather than merged unchanged. #282 docs PR continues separate and must be revalidated after remediation. No stable/TrueNAS deployment.
ci(security): one-shot regenerate exact Python CA legal evidence (#284)