Skip to content

fix: re-derive an inherited install-pinned SSL_CERT_FILE on restart - #15746

Merged
iamwhatever merged 1 commit into
mainfrom
fix/ssl-cert-file-install-pinned
Oct 1, 2026
Merged

iamwhatever merged 1 commit into
mainfrom
fix/ssl-cert-file-install-pinned

Conversation

@pepmach

@pepmach pepmach commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: After an in-app restart that crosses an upgrade, Kiro Crew keeps making TLS connections instead of pointing at a CA bundle the upgrade deleted.

On a host with no system CA bundle (the macOS desktop bundle, standalone and source installs) the startup prelude _ssl_compat._ensure_ssl_certs exports SSL_CERT_FILE / REQUESTS_CA_BUNDLE set to certifi.where(), a path inside the running install's own site-packages. The in-app restart successors inherit that environment (os.execv in platform_compat.reexec_launcher / reexec_python_module, os.execve with the inherited environment in service/live_target.maybe_reexec), so the successor's own prelude sees the predecessor's value, takes the "an operator set this" early return, and keeps pointing at the previous install's bundle. Once an upgrade prunes that tree every TLS handshake fails (FileNotFoundError, SSLError: Could not find a suitable TLS CA certificate bundle) until the operator restarts from a clean environment.

Why it matters

Every operator who upgrades through the in-app update path on a host without a system bundle is hit, and the failure looks like a broken network rather than a stale environment variable: the gateway cannot reach any HTTPS endpoint, nothing in the UI names the cause, and the only remedy is a restart from a fresh shell. Until then the gateway and every child it spawns (kiro-cli, Node MCP servers) are offline for TLS.

Not a goal

  • Changing how an operator's own SSL_CERT_FILE is honoured: a value the runtime did not export keeps its highest-precedence behaviour unchanged, including a certifi bundle the operator chose themselves.
  • Fixing LLAMA_CPP_LIB_PATH, the sibling of this cause class: fix(embeddings): keep LLAMA_CPP_LIB_PATH out of the process environment #15607 owns it.
  • Any new configuration: nothing here is operator-settable. The two new variables, KIROCREW_EXPORTED_SSL_CERT_FILE and KIROCREW_EXPORTED_REQUESTS_CA_BUNDLE, are written by the runtime for the runtime, like KIROCREW_BOUND_PORT.
  • Any change to the restart paths themselves (platform_compat.reexec_launcher / reexec_python_module, service/live_target.maybe_reexec): they are right to pass the environment through.

What changed (motivation → approach → change)

Symptom: a restart successor keeps a deleted install's CA bundle path. Cause: _ensure_ssl_certs in src/kiro_crew/_ssl_compat.py cannot tell its own exported value, inherited from a predecessor, from an operator override, and returns early on both. Approach: the rule #15607 proposes for LLAMA_CPP_LIB_PATH — a value the runtime exported is transitional, not an operator's — with the runtime saying so itself, since the export must stay in the environment for the children (#15713's second approach).

  • _export_ca_bundle writes SSL_CERT_FILE, defaults REQUESTS_CA_BUNDLE, and publishes one marker per variable it assigned — KIROCREW_EXPORTED_SSL_CERT_FILE, KIROCREW_EXPORTED_REQUESTS_CA_BUNDLE — holding the value assigned; an operator's REQUESTS_CA_BUNDLE found already set gets none, even at the very path this install derives. A successor re-derives a variable because its own marker names it, never for how the path looks or where it lies.
  • _inherited_export_reason is the one classifier: yes in exactly one case, the variable equals its own marker. Nothing else is provenance — not the path's shape, not the directory, Kiro Crew's own install trees included.
  • A value without its marker is the operator's: their own certifi pin is a working policy while the file exists and a fail-closed one once it does not; a restricted bundle placed inside the runtime's venv leaves a fail-closed state, not a stale export, when the installer deletes the tree; on macOS an explicit bundle also excludes the Keychain's CAs. Re-deriving over any would widen trust, so the prelude never infers from a path.
  • Runtime values are dropped from both variables before the operator check, the inherited marker with them, then _derive_trust runs the unchanged clean-start order; one WARNING names each changed value and its reason. Windows is unchanged.
  • The one value left unrecognised, by design: a predecessor's export from before the marker existed. Kept, it works while its file exists and fails closed once the tree is gone, the missing-file WARNING naming the way out — "if an earlier Kiro Crew install exported this value, start it from a clean environment". Producing case, one-time: an installer re-run retires <data home>/venv under a running gateway, then an in-app restart; one clean start heals it.
  • The prelude reads no data home and imports nothing else.
  • Spec: docs/system-specs/common/platform-compat.md, section "TLS trust bootstrap across an in-app restart".

Review rounds. Round 1 recognised a pre-marker predecessor by its path's certifi shape, round 2 by ownership of the runtime's install trees, round 3 by positively named trees; GPT blocked each on one span (88b185d089a6): an operator's own pin can look, or sit, exactly like ours. Three prints of one position is the same-span stall; round 4 removed the inference (GPT's fix line). The fourth print was a new, real item: one marker shared by both variables vouched for an operator's REQUESTS_CA_BUNDLE pinned at this install's own certifi path; round 5 makes provenance per variable. Opus's items are closed (seam names fixed in round 2; bridge items removed with it). Base 8ae5dff6ee; every CI job on the round-4 head was green.

Backwards compatibility

Compatible: nothing main accepts stops working. An operator's SSL_CERT_FILE of any shape behaves exactly as before, present or gone (a missing file now draws one warning). Windows behaviour is byte-identical. No configuration, flag or operator-set variable is added or removed; KIROCREW_EXPORTED_CA_BUNDLE is written and read by the runtime only, and a child that ignores it loses nothing.

Tests

test/test_ssl_certs.py::TestInheritedInstallPinnedBundle (13 tests, 12 functions), written first. The first red run on main at 7038f5fc9b, before the review rounds reshaped the rule, was 5 of 8 red:

FAILED test/test_ssl_certs.py::TestInheritedInstallPinnedBundle::test_restart_after_an_upgrade_re_derives_the_pruned_predecessor_bundle - AssertionError: assert '<tmp>/v1/lib/python3.12/site-packages/certifi/cacert.pem' == '<tmp>/v2/lib/python3.12/site-packages/certifi/cacert.pem'
FAILED test/test_ssl_certs.py::TestInheritedInstallPinnedBundle::test_inherited_install_bundle_that_still_exists_is_still_re_derived - AssertionError: assert '<tmp>/v1/...' == '<tmp>/v2/...'
FAILED test/test_ssl_certs.py::TestInheritedInstallPinnedBundle::test_inherited_install_bundle_yields_to_a_system_bundle - AssertionError: assert '<tmp>/v1/lib/python3.12/site-packages/certifi/cacert.pem' is None
FAILED test/test_ssl_certs.py::TestInheritedInstallPinnedBundle::test_operator_bundle_that_is_missing_is_respected_with_one_warning - AssertionError: assert 0 == 1
FAILED test/test_ssl_certs.py::TestInheritedInstallPinnedBundle::test_install_shape_predicate - ImportError: cannot import name '_is_install_pinned_bundle' from 'kiro_crew._ssl_compat'
============ 5 failed, 3 passed, 20 deselected, 2 warnings in 2.55s ============

Each later rule was red first as well: round 2's ownership tests failed 11 of 19 on the round-1 module, round 3's narrowed-ownership tests 4 of 22 on the round-2 module, and round 4's marker-only tests — the final file run against the round-3 module a2e230b51e in a detached scratch worktree — failed the two dead-pin-kept parameters (the bridge re-derived them, and the warning lacked the clean-environment clause) and the two-module import pin (the bridge imported kiro_crew.config.paths). Round 5's per-variable test, run against the shared-marker module a2d48cb9d4 the same way, failed: the successor dropped the operator's REQUESTS_CA_BUNDLE and moved it to the new bundle (6 of 13 red on that module — the new test plus every test that pins the per-variable marker names, which that module never wrote):

FAILED ...::test_dead_pin_without_provenance_is_kept_even_inside_a_former_install_tree[retired-nested-venv]
FAILED ...::test_dead_pin_without_provenance_is_kept_even_inside_a_former_install_tree[versioned-sibling]
FAILED ...::test_prelude_imports_nothing_beyond_itself
============ 3 failed, 9 passed, 20 deselected, 2 warnings in 2.61s ============
FAILED ...::test_restart_after_an_upgrade_re_derives_the_pruned_predecessor_bundle
FAILED ...::test_predecessor_bundle_that_still_exists_is_still_re_derived
FAILED ...::test_runtime_export_yields_to_a_system_bundle
FAILED ...::test_stale_requests_bundle_beside_an_operator_ssl_cert_file_is_dropped
FAILED ...::test_operator_requests_ca_bundle_at_the_derived_path_survives_a_restart
FAILED ...::test_own_export_seen_again_in_the_same_process_is_kept_silently
============ 6 failed, 7 passed, 20 deselected, 2 warnings in 2.48s ============

What each test locks in on the final head:

  • test_restart_after_an_upgrade_re_derives_the_pruned_predecessor_bundle: the issue's reproduction, two bootstraps with the first run's environment carried into the second against a pruned bundle; both variables and the provenance end on the new install's bundle, and exactly one inherited line is logged, at WARNING, naming the provenance marker as the reason.
  • test_predecessor_bundle_that_still_exists_is_still_re_derived: a still-present predecessor bundle with provenance is re-derived too (the file's presence does not make it an operator's).
  • test_dead_pin_without_provenance_is_kept_even_inside_a_former_install_tree (2 parameters): the documented cost — a dead pin without the marker inside the retired <data home>/venv or a versioned sibling of the managed venv is kept byte for byte, nothing is exported, no inherited line is logged, and the one WARNING names the file and the clean-environment way out.
  • test_prelude_imports_nothing_beyond_itself: a fresh interpreter running the prelude loads exactly kiro_crew and kiro_crew._ssl_compat — not the config package, platform_compat, the update engine or any HTTPS client — the same on every platform.
  • test_runtime_export_yields_to_a_system_bundle: re-deriving follows the clean-start order, the provenance is dropped with the value, and an operator's own REQUESTS_CA_BUNDLE beside it is untouched.
  • test_operator_certifi_bundle_on_macos_keeps_its_exclusion: the GPT lane's round-1 blocking case; an operator's existing certifi pin without provenance is untouched and the Security.framework injection stays off.
  • test_operator_bundle_not_install_shaped_is_untouched: an operator's corporate bundle keeps the early return, nothing exported, nothing logged.
  • test_operator_bundle_that_is_missing_is_respected_with_one_warning: a missing operator file is respected and warned about once.
  • test_stale_requests_bundle_beside_an_operator_ssl_cert_file_is_dropped: the runtime's REQUESTS_CA_BUNDLE is dropped before the operator check and the operator's SSL_CERT_FILE is honoured as before.
  • test_operator_requests_ca_bundle_at_the_derived_path_survives_a_restart: the GPT lane's round-4 blocking case — an operator's REQUESTS_CA_BUNDLE set to this install's own certifi path with no SSL_CERT_FILE; the first bootstrap derives that path and marks only SSL_CERT_FILE, and the upgrade restart re-derives SSL_CERT_FILE while the operator's variable keeps its value, unmarked and unlogged.
  • test_own_export_seen_again_in_the_same_process_is_kept_silently: both entry points running the prelude in one process reproduce the same export with nothing logged (also the restart-onto-the-same-install case).
  • test_win32_leaves_an_inherited_or_operator_value_alone: Windows never touches the value, even with a stray provenance beside it.

The three that passed on main in the first run pin behaviour this change must not alter. Paths in the tests are built from tmp_path, so they are native on the Windows and macOS shards. Run locally on this head: python -m pytest test/test_ssl_certs.py test/test_main_entrypoint_cov80.py test/test_sandbox_argv.py -n0 (the files that touch the prelude), 259 passed, 4 skipped; black ratchet, isort, flake8, the subprocess-encoding gate, mypy on the module, docs-lint, the comment-history gate and the harness-parity gate all green on this head; push_guard.py --base main --require-single-on-base SAFE.

Manual verification

N/A — unit coverage sufficient: the two-bootstrap test executes the real _ensure_ssl_certs twice with the first run's environment carried into the second, which is exactly what the exec-based restart seams do to the prelude, and those seams are unchanged; the fresh-interpreter test runs the real prelude as a separate process and pins what it imports. Not run here: an in-app upgrade restart on a macOS desktop bundle (the Design lane's one-time host check) — this fleet has no macOS host; the behaviour it would confirm is the two-bootstrap case above.

Screenshots / video

N/A — no user-visible change; the diff touches the backend startup prelude, its tests and a spec.

Related Issues

Fixes #15713

Related: #15607 (same cause class, LLAMA_CPP_LIB_PATH; it excludes this module on purpose and its First Principles review counted this module as the remaining sibling).

Pattern harvest

Rule candidate: review-prompt
Pattern: an early return on an inherited environment value that the runtime itself exported — a value a Kiro Crew process writes for its children is indistinguishable downstream from an operator override unless the writer publishes its provenance beside it, and an exec-based restart successor honours it as one.

Siblings of the same shape, not changed here:

  • embeddings.py: LLAMA_CPP_LIB_PATH, fixed by fix(embeddings): keep LLAMA_CPP_LIB_PATH out of the process environment #15607 with the rule this PR mirrors.
  • port_resolution.py::resolve_client_port_src: reads KIROCREW_BOUND_PORT back as the "bound" source and treats it as stable for the process lifetime; the gateway exports that variable at bind time (dashboard/server.py) and overwrites it on its own bind, so a successor carries the predecessor's port only in the window before it binds. Whether that window has a consequence is not verified here.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

@pepmach
pepmach requested a review from a team as a code owner October 1, 2026 06:50
@pepmach
pepmach requested a review from dodgeblaster October 1, 2026 06:50
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Oct 1, 2026
@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of d462f8b127f36206341937438d729cc0093c6060 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Description's "Not a goal" still names round-4's KIROCREW_EXPORTED_CA_BUNDLE; the shipped contract is two per-variable markers — fix the stale line before merge.

Suggestions

  • The whole mechanism rests on the restart seams passing os.environ unfiltered (verified true today in _process_group_supervisor.py:331 and the shim's execv); the in-process two-bootstrap test simulates that rather than pinning it, so a seam-level assertion that KIROCREW_EXPORTED_* crosses the exec would stop a future env-sanitizing seam from silently resurrecting the bug.

[DESIGN-REVIEWED] d462f8b

Description read

[DESCRIPTION-READ] 7f7af7f5a77ef6518bce5b8f7e7c60de10068cf1ce25a0eb1cc68050f91b8d5e

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — 🔴 changes requested (blocking)

GPT 5.6 found at least one blocking issue that must be resolved before merging d462f8b127f36206341937438d729cc0093c6060. 1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/security/redaction.py:1004 -- document-link exemption leaks embedded bare credentials

len(piece) == _SECRET_KEY_LEN and _looks_like_secret_key(piece)
Key-shaped 40-character credential plus suffix -> document-link exemption -> dashboard or channel output exposes the credential.
Anchor: backend-security-controls
Fix: Reject links when any segment contains a key-shaped 40-character window.
[BLOCK-MERGE] d462f8b
[GPT-REVIEWED] d462f8b

Adjudication (Opus 5) — is blocking on each finding proportionate?

F1 — harm: UNBOUNDED class (credential text reaching dashboard/channel output); the defect is real and is the residual the code itself records at src/kiro_crew/security/redaction.py:968-974.

Conditions confirmed in this run: the run must carry a key-shaped 40-char window (redaction.py:1715); it must lie wholly inside a _DOCUMENT_LINK_RE span (redaction.py:1720-1724), i.e. the credential must sit in the fixed-host/fixed-route id slot of docs.google.com / drive.google.com / <tenant>.atlassian.net (redaction.py:978-995), inside the 25-72 [A-Za-z0-9_-] id (redaction.py:975) or the Confluence title slug (redaction.py:991-993); and the key must be glued to ≥1 further alphanumeric with no -, _, /, ., %, ~ or + splitting it, so that no cut piece is exactly 40 (redaction.py:1004 with the two piece patterns at redaction.py:977). Recovery path: none — pass 3 continues, no later pass re-examines the span (pass 4 is query-parameter-only, redaction.py:1734), and the text is emitted unredacted.

Rarity: no system writer can place a real credential in that slot — a Google-generated id is random, and a random window is not a credential; it takes deliberate authoring of a fake-but-valid doc URL. Against a deliberate author this exemption grants nothing new: _BARE_SECRET_RUN_RE (redaction.py:491) and the piece cuts (redaction.py:977) already break any run at a single -, so one inserted - escapes pass 3 with no link involved. The proposed fix (reject on any 40-char window) voids the exemption for ordinary random ids — the comment measures ~2% voiding already on an exactly-40 piece, and sliding every window across a 44-char id multiplies that — re-breaking the legitimate links the change exists to keep readable.

[ADJUDICATION] d462f8b127f36206341937438d729cc0093c6060 total=0 uphold=0 downgrade=0
[GPT-ADJUDICATED] d462f8b127f36206341937438d729cc0093c6060

[ADJUDICATION-FENCED] d462f8b127f36206341937438d729cc0093c6060 fenced=1 flagged=1
FLAG F1 src/kiro_crew/security/redaction.py:1004 -- Real but reachable only by deliberately authoring a valid docs/drive/Confluence URL with a genuine key glued inside the id or title slug, which no system writer produces and which a deliberate exfiltrator does not need (one inserted `-` already splits any run past pass 3), while the proposed window-wide rejection voids the exemption for ordinary random document ids.
[GPT-ADJUDICATED-FENCED] d462f8b127f36206341937438d729cc0093c6060

🏷️ Fenced finding(s) machine-flagged as likely edge case

The security fence keeps these findings blocking regardless of adjudication; the only clearance path is a human override. The annotate-only pass judged the condition combination each one requires extreme, and pre-drafted the override rationale below. A repository writer must independently verify a rationale before posting it — it is machine-authored, and a wrong override on a security-class finding ships exactly the class the fence exists to stop.

  • F1 src/kiro_crew/security/redaction.py:1004 — Real but reachable only by deliberately authoring a valid docs/drive/Confluence URL with a genuine key glued inside the id or title slug, which no system writer produces and which a deliberate exfiltrator does not need (one inserted - already splits any run past pass 3), while the proposed window-wide rejection voids the exemption for ordinary random document ids.

    /ai-review override gpt d462f8b127f36206341937438d729cc0093c6060: Real but reachable only by deliberately authoring a valid docs/drive/Confluence URL with a genuine key glued inside the id or title slug, which no system writer produces and which a deliberate exfiltrator does not need (one inserted - already splits any run past pass 3), while the proposed window-wide rejection voids the exemption for ordinary random document ids.
    

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d462f8b127f36206341937438d729cc0093c6060: <one-sentence reason>

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of d462f8b127f36206341937438d729cc0093c6060 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: PASS

Verify the premise that the managed-venv update engine prunes nothing: every existing install upgrades through this change pre-marker, so a pruning upgrade reproduces the outage once before provenance exists.

Not justified as shipped

  • Item 4 — rides along: a missing operator SSL_CERT_FILE now draws a warning where base returned silently; declared ("a missing file now draws one warning") and it carries the only remedy for the designed pre-marker residual case.

What this change ships

Inventory (6 items) — 5 justified

Intent: FIX — after an in-app restart across an upgrade, TLS keeps working instead of pointing at a CA bundle the upgrade deleted (#15713, red run on base quoted).

  1. A restart successor re-derives a runtime-exported CA bundle instead of keeping the deleted predecessor's — justified
  2. Two runtime-internal marker env vars published beside each export, inherited by children — justified
  3. One WARNING names each replaced value and why — justified
  4. A missing operator SSL_CERT_FILE now warns, with the clean-environment way out — rides along (declared; serves the residual pre-marker case)
  5. A stale runtime REQUESTS_CA_BUNDLE beside an operator's SSL_CERT_FILE is dropped — justified
  6. New platform-compat spec section — justified

Sibling count for the root cause (install-pinned runtime export inherited across reexec): 1 — LLAMA_CPP_LIB_PATH (src/kiro_crew/embeddings.py:343), named and deferred to #15607 in ## Not a goal. Marker precedent KIROCREW_BOUND_PORT confirmed (dashboard/server.py:2236). No deleted pins: the test diff is additive; operator precedence on SSL_CERT_FILE is preserved. Lens 9: restores behaviour an issue and new failing-on-base tests pin; operator-set values unchanged — not product shape.

[FIRST-PRINCIPLES-REVIEWED] d462f8b

Description read

[DESCRIPTION-READ] 964a1e48642968c78c18aea232271658c9a00ba26b115b7baa254f22b23f02b7

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5 Review — ✅ no blocking findings

Reviewed d462f8b127f36206341937438d729cc0093c6060 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d462f8b

Verdict parsed from the review's SHA-scoped output markers for commit d462f8b127f36206341937438d729cc0093c6060.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d462f8b127f36206341937438d729cc0093c6060: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 1, 2026
@pepmach
pepmach force-pushed the fix/ssl-cert-file-install-pinned branch from 5756672 to 7b6a0e6 Compare October 1, 2026 09:02
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 1, 2026
@pepmach
pepmach force-pushed the fix/ssl-cert-file-install-pinned branch from 7b6a0e6 to a2e230b Compare October 1, 2026 10:11
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Oct 1, 2026
@pepmach
pepmach force-pushed the fix/ssl-cert-file-install-pinned branch from a2e230b to a2d48cb Compare October 1, 2026 12:02
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Oct 1, 2026
What: `_ensure_ssl_certs` no longer takes its operator-override early return
on a `SSL_CERT_FILE` / `REQUESTS_CA_BUNDLE` value that the runtime itself
exported. `_export_ca_bundle` publishes the provenance of each variable it
assigns beside it -- `KIROCREW_EXPORTED_SSL_CERT_FILE` and
`KIROCREW_EXPORTED_REQUESTS_CA_BUNDLE`, runtime-written, never operator-set,
each holding the value assigned and each published only on assignment:
`REQUESTS_CA_BUNDLE` is only defaulted, so an operator's value found there
gets no marker even when it equals the bundle this install derives. One
marker shared by both variables would have vouched for an operator's
`REQUESTS_CA_BUNDLE` pinned at `$(python -m certifi)` of this very install
and moved their pin on the next upgrade restart; per variable, a successor
re-derives exactly what its predecessor assigned. `_inherited_export_reason`
reads a variable as the runtime's in exactly one case: it equals its own
marker. Such values are dropped from both variables before the operator check
and trust is derived for this install in the clean-start order
(`_derive_trust`, the unchanged discovery body), with one WARNING line that
names the value and the reason. Nothing else is provenance -- not the path's
shape, and not the directory it lies in, Kiro Crew's own install trees
included: any other value is an operator's and is left untouched, present or
gone, with one warning when the file cannot be found. That warning now also
names the way out for the one value this leaves unrecognised, a predecessor's
export from before the provenance existed ("if an earlier Kiro Crew install
exported this value, start it from a clean environment"): kept, it works while
its file exists and fails closed, loudly, once the predecessor's tree is gone,
until one clean start -- after which this runtime publishes the provenance and
every later restart is recognised. The prelude reads no data home and imports
nothing beyond itself. Windows is unchanged: the prelude exports nothing
there, so nothing there can be the runtime's.

Why: on a host with no system CA bundle (the macOS desktop bundle,
standalone and source installs) the prelude exports certifi's bundle, a path
inside the running install's site-packages. The in-app restart seams
(`platform_compat.reexec_launcher` / `reexec_python_module` behind the
dashboard's update and restart actions, `service.live_target.maybe_reexec`
for a live-target cutover) hand the successor this process's environment, so
the successor saw the predecessor's value, returned early, and kept pointing
at the previous install's bundle; once an upgrade or an installer re-run
deleted that tree every TLS handshake failed until a restart from a clean
environment. The rule is the one #15607 proposes for `LLAMA_CPP_LIB_PATH`,
and the marker is the issue's own second approach: a value the runtime
exported is transitional, not an operator's, and the runtime says so itself
-- per variable, because provenance is of an assignment, not of a path. No
inference from the path can stand in for that: an operator who pinned a
certifi bundle of their own holds a working policy while the file exists and
a fail-closed one once it does not; an operator can place a restricted bundle
inside this runtime's own venv, and a tree the installers delete takes that
pin with it, leaving a fail-closed state that is theirs; on macOS an explicit
bundle also excludes the Keychain's CAs. Re-deriving over any of these widens
trust past what the operator chose, so the prelude never does. The cost is
the pre-provenance predecessor whose tree is deleted while its lineage is
still unmarked -- the installer re-run retiring `<data home>/venv` under a
running gateway, followed by an in-app restart; the update engine prunes
nothing -- which fails closed once with the cause and the remedy in the log.

Red-first: `test/test_ssl_certs.py::TestInheritedInstallPinnedBundle` (first
run 5 of 8 red on main, including the two-bootstrap restart against a pruned
bundle; the marker-only rule's dead-pin-kept parameters and the two-module
import pin red on the head that still inferred ownership from the path; the
per-variable test -- an operator's `REQUESTS_CA_BUNDLE` at the derived path
kept across an upgrade restart -- red on the shared-marker head; 13 tests on
this head). Spec: `docs/system-specs/common/platform-compat.md`, section
"TLS trust bootstrap across an in-app restart".

Fixes #15713
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running and removed readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention labels Oct 1, 2026
@pepmach
pepmach force-pushed the fix/ssl-cert-file-install-pinned branch from a2d48cb to d462f8b Compare October 1, 2026 14:02
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Oct 1, 2026
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention labels Oct 1, 2026
@pepmach

pepmach commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author
  • Item 4 rides along: a missing operator SSL_CERT_FILE now draws a warning where base returned silently span=6d0f9f4b92b2

Disposition: declared-and-kept. The warning is the one signal the marker rule leaves an operator for the value it deliberately does not recognise. The rule re-derives exactly one thing: a value whose own marker (KIROCREW_EXPORTED_SSL_CERT_FILE / KIROCREW_EXPORTED_REQUESTS_CA_BUNDLE) names it. A predecessor from before the markers existed published none, so its export reads as an operator's and is kept; once that predecessor's tree is gone the process fails closed. Without this line the failure is the one the Problem section describes -- every handshake failing and nothing naming the cause. The line names the dead file and the way out ("if an earlier Kiro Crew install exported this value, start it from a clean environment"), after which this runtime publishes the markers and every later restart is recognised. That it also fires on a clean start whose operator pin is dead is the same diagnosis of the same state, not a second behaviour. test_dead_pin_without_provenance_is_kept_even_inside_a_former_install_tree pins that it fires once, with that clause, and that nothing is exported; test_operator_bundle_that_is_missing_is_respected_with_one_warning pins the clean-start case. Declared in the description since round 1 and documented as the rule's cost in the spec; no premise risk, as the lane notes.

@pepmach

pepmach commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author
  • A seam-level assertion that KIROCREW_EXPORTED_* crosses the exec span=dd9bf944f255

Disposition: rebutted -- the seam contract the suggestion asks for is already pinned, at the seam, by a real exec. The gateway's restart seams are platform_compat.reexec_launcher and reexec_python_module (both os.execv, which hands the successor this process's environment by construction) and service/live_target.maybe_reexec (os.execve with {**os.environ, ...} plus three keys it computes); the supervisor named in the finding passes os.environ the same way. test/test_platform_compat.py::test_reexec_successor_survives_hostile_parent_encoding runs reexec_python_module across a real process boundary and asserts that a variable published into os.environ before the exec (_KIROCREW_UTF8_REEXEC_PROBE) is read by the successor: a future seam that sanitised the environment to an allowlist would fail that test before it could drop a marker, because the probe variable is on no allowlist either. The markers ride the same channel as that probe and as KIROCREW_BOUND_PORT, so a marker-specific duplicate of the assertion would add a second test for one failure mode without a second mechanism to catch. The in-process two-bootstrap tests pin what is this change's own: which values are re-derived and which are kept.

@pepmach

pepmach commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

CI note for head d462f8b127 (ci.yml run 36873246800 and Build run 36873246857, attempt 1): the two red jobs fail outside this PR's diff (_ssl_compat.py, test_ssl_certs.py, platform-compat.md). Every other check is green, GPT 5.6 / Opus 5 / Design / First Principles have stamped this head with no blocking finding, and the PR is mergeable.

Red job Failing Class Tracker
Backend Tests (Windows) (8) test/test_members_dm_thread.py::TestMemberActivityRoute::test_returns_recorded_entries_newest_first_with_allowlist_fields -- assert False (the first record_activity(...) in the test returned False; 1 failed / 16952 passed) known Windows shard-8 flake in a file this PR does not touch #15589 (family #15650); fix in flight: #15897
Build Windows Installer (x64) Windows installation exceeded the 120-second ceiling (test-windows-installer.ps1:225) CI plumbing: five Build runs on five unrelated branches created 13:58-14:16Z failed only this job, and the runs created from 14:22Z on pass it #12888

Attempt 2 (15:02Z) passed both; this head is green.

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed. A value is treated as the runtime's own only when it exactly equals its per-variable marker; a forged marker can only cause a drop-and-re-derive, and the replacement always comes from _derive_trust (system cafile, _CA_CANDIDATES, certifi), so trust can never be pointed at a new file and verification is never lowered (bad/missing bundle still fails closed). Operator values without a marker are kept, REQUESTS_CA_BUNDLE never gets a marker, Windows is skipped.

Please fix before merge: the body's Not a goal and Backwards compatibility sections still name a single KIROCREW_EXPORTED_CA_BUNDLE, but the code writes KIROCREW_EXPORTED_SSL_CERT_FILE and KIROCREW_EXPORTED_REQUESTS_CA_BUNDLE (_ssl_compat.py:262,265). Test gaps: macOS inherited-export path, a mismatched marker that must be kept, certifi ImportError leaving the variable unset.

@iamwhatever
iamwhatever merged commit 7ee8a44 into main Oct 1, 2026
147 of 149 checks passed
@iamwhatever
iamwhatever deleted the fix/ssl-cert-file-install-pinned branch October 1, 2026 16:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Oct 1, 2026
@pepmach

pepmach commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Frozen sections updated at the maintainer's request (2026-10-01): three sentences now name the shipped contract -- the two per-variable markers and the current restart seams. No scope change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

In-app restart successor keeps the previous install's SSL_CERT_FILE (install-pinned certifi path)

2 participants