Repository navigation
Distrust compromised emSign/Comodo AAA roots, bump pip/uv/certifi (NVP-512) - #24
Conversation
eMudhra intermediate CA private keys are recoverable (disclosed 2026-10-17). Their chains end in "emSign ECC Root CA - G3" and "AAA Certificate Services". certifi already dropped them, but the Debian system trust store used by OpenSSL (aiohttp default context, curl, psql, git) did not until ca-certificates 20250419~deb12u2. - Upgrade ca-certificates and blocklist both roots in /etc/ca-certificates.conf (defence in depth), then regenerate /etc/ssl/certs. - pip >= 26.2.2 and uv >= 0.13.4: both carry their own CA bundle. Ref: https://emudhra.we-have-a-problem.com/ (NVP-512) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
First certifi release that removes "emSign ECC Root CA - G3" while keeping the three unaffected emSign roots. Downstream images (navigator-api, dataintegrator, worker, fieldsync) inherit this lock, so the floor has to live here. (NVP-512) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request addresses the eMudhra/emSign CA security incident by upgrading ca-certificates in the Dockerfile, blocklisting the compromised roots, upgrading pip and uv to versions with safe CA bundles, and enforcing a minimum version of certifi in pyproject.toml. The review feedback highlights two critical issues: first, the certificate name for the Comodo AAA root in the sed command is incorrect, which would prevent the blocklist from being applied; second, upgrading pip globally as a non-root user without the --user flag will likely result in a permission error.
| RUN apt-get update -y && apt-get install -y --only-upgrade ca-certificates && \ | ||
| sed -i -e 's#^mozilla/emSign_ECC_Root_CA_-_G3.crt#!&#' \ | ||
| -e 's#^mozilla/Comodo_AAA_Services_root.crt#!&#' /etc/ca-certificates.conf && \ | ||
| update-ca-certificates --fresh && \ | ||
| apt-get clean && rm -rf /var/lib/apt/lists/* |
There was a problem hiding this comment.
The certificate name for the Comodo AAA root in the Debian ca-certificates package is mozilla/AAA_Certificate_Services.crt, not mozilla/Comodo_AAA_Services_root.crt. Because of this, the sed command will not match the entry in /etc/ca-certificates.conf, and the defense-in-depth blocklist will fail to disable the Comodo AAA root if the package upgrade is not available (e.g., due to a stale mirror).
RUN apt-get update -y && apt-get install -y --only-upgrade ca-certificates && \
sed -i -e 's#^mozilla/emSign_ECC_Root_CA_-_G3.crt#!&#' \
-e 's#^mozilla/AAA_Certificate_Services.crt#!&#' /etc/ca-certificates.conf && \
update-ca-certificates --fresh && \
apt-get clean && rm -rf /var/lib/apt/lists/*
| # Install UV using pip (no-cache so pip doesn't bloat the layer). | ||
| # pip >= 26.2.2 and uv >= 0.13.4 ship CA bundles without the compromised emSign | ||
| # roots (both carry their own trust store and ignore the system one). | ||
| RUN pip install --no-cache-dir --upgrade "pip>=26.2.2" "uv>=0.13.4" |
There was a problem hiding this comment.
Since this command is run as the non-root user troc, attempting to upgrade pip globally without the --user flag can result in a PermissionError when pip tries to uninstall the existing system-wide pip installation. Adding the --user flag ensures that the upgraded pip and uv are installed in the user's home directory (/home/troc/.local/bin), which is already prepended to the PATH in the next line.
RUN pip install --no-cache-dir --user --upgrade "pip>=26.2.2" "uv>=0.13.4"
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Why
eMudhra/emSign CA incident (disclosed 2026-10-17, https://emudhra.we-have-a-problem.com/): the private keys of eight eMudhra intermediates are recoverable from the certificates. Browsers block the chains via CT, but Python/OpenSSL clients only stop trusting them when their trust store is updated.
All Navigator Python images (navigator-api, navigator-api-ai, dataintegrator, dataintegrator-worker*, navigator-fieldsync) inherit from this image. Current
flowcore:main(2026-10-19) has certifi 2026.10.18 (OK) but Debianca-certificates 20250419~deb12u1, so the system store used by aiohttp, curl and psql still trustsemSign ECC Root CA - G3andAAA Certificate Services. pip 25.0.1 and uv 0.13.1 also predate the fix.Jira: NVP-512
What
ca-certificates(bookworm-security now ships20250419~deb12u2) and blocklist both roots in/etc/ca-certificates.confas defence in depth, thenupdate-ca-certificates --fresh.pip>=26.2.2,uv>=0.13.4(both carry their own CA bundle).certifi>=2026.10.19floor. Downstream images inherit thisuv.lock, so the floor must live here.Local build verification (
docker buildon 2026-10-20, exit 0)ca-certificates/etc/ssl/certsG3 / AAAssl.create_default_context()(aiohttp path)Rollout
Merge → Docker Hub publishes
flowcore:main→ rebuild navigator-api, navigator-dataintegrator, navigator-dataintegrator-worker and fieldsync → verify staging pods.Rollback
Revert this PR and republish
main; no data or infra changes.🤖 Generated with Claude Code