Skip to content

fix: route stt and wheel downloads through the shared redirect policy - #32318

Merged
KiroCrewCaptain merged 1 commit into
mainfrom
fix/22230-shared-redirect-policy
Nov 7, 2026
Merged

KiroCrewCaptain merged 1 commit into
mainfrom
fix/22230-shared-redirect-policy

Conversation

@bolichen97

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: The whisper model, ffmpeg decoder and update-engine downloads follow redirects under the shared download policy, so a hop can no longer go to plaintext, or off the release CDN's host.

The speech model and ffmpeg decoder fetch (stt/models.py) and the update engine's manifest and wheel fetch (platform/wheel_engine_transfer.py) still opened their own connection with urllib.request.urlopen. Its default redirect handler follows a hop to any host and down to plain http://. #10018 listed both sites; #21683 moved only the two app installs onto asset_downloader.

Why it matters

asset_downloader is the one place that should answer which redirects a pinned download may follow. The update engine runs inside the gateway, which can reach the operator's internal network, so a redirect it follows to another host is a request nobody approved. A hop to plaintext lets anyone on the path watch and alter that leg before the digest check. The digest still guarded the installed bytes, but these two copies did not follow the shared rule.

Not a goal

  • Moving either loop onto asset_downloader.download_to, which Route the stt model and wheel_engine downloads through asset_downloader.download_to #22230 also asks for and which stays open there. Each loop has bounds that download_to lacks and that their tests pin: the speech fetch refuses an oversized chunk before writing it, can be cancelled, and stages through mkstemp; the wheel fetch has a total deadline and an ApplyCancel that shuts the socket. download_to would need all of them first, so only the connection moves here.
  • The other direct urlopen callers that fetch no sha256-pinned artifact (see Sibling sweep).
  • Changing asset_downloader's two policies themselves.

What changed (motivation → approach → change)

Both sites now open their connection through asset_downloader.build_opener, each with the policy that module defines for its kind of url:

Download URL comes from Redirect policy
Whisper model, ffmpeg decoder (stream_pinned_payload) a source constant, or the operator's KIROCREW_WHISPER_MODEL_BASE_URL may change host (Hugging Face and GitHub releases both redirect to their own download host), never leaves https
Update manifest and wheel (_read_bounded) the release CDN stays on the CDN's host and port (the CDN serves both itself)
Same, while the operator's KIROCREW_CDN_BASE is set the process environment (every update url is built from it then) may change host, never leaves https

A refused hop fails the transfer as before, with a message naming both hosts. The facade platform/wheel_engine.py re-exports asset_downloader, because its facade test requires every project name an owner imports to stay bound on the facade. The stt spec, the update RFC and build_opener's list of callers in asset_downloader.py describe the new route. Tests answer the connection through a shared test/opener_helpers.py instead of patching urlopen.

Backwards compatibility

Breaking: a redirect to plaintext from any of these downloads, and a redirect off the release CDN's host while no KIROCREW_CDN_BASE mirror is set, is now refused. Writer sweep on origin/main @ f80c7b8 (the redirect sources are the endpoints these urls name): huggingface.co model urls answer 302 to another https host, still followed; GitHub release decoder urls answer 302 to another https host, still followed; updates.crew.kiro.dev and download.crew.kiro.dev answer the manifest and wheel with a 200; an operator mirror under KIROCREW_WHISPER_MODEL_BASE_URL or KIROCREW_CDN_BASE may still redirect to another https host.

Reader: src/kiro_crew/stt/models.py:ModelStore.ensure -- chat -- the model host's redirect to its download host is still followed (test_the_model_hosts_redirect_to_its_download_host_is_followed)
Reader: src/kiro_crew/stt/decoder.py:maybe_autofetch -- chat -- the GitHub release redirect is a cross-host https hop, which stream_pinned_payload still follows
Reader: src/kiro_crew/platform/wheel_engine.py:download_verified_wheel -- release -- the release CDN serves the wheel with no redirect (test_a_redirect_off_the_cdn_host_is_refused, test_an_operator_mirror_may_redirect_to_another_https_host)
Reader: src/kiro_crew/platform/wheel_engine.py:fetch_verified_manifest -- release -- the feed serves the manifest with no redirect

Tests

  • test_stt_engine_download.py: the model host's redirect to another https host is followed and installed; a redirect to http:// is refused with nothing requested from it and nothing left on disk. Both run the real opener over a fake transport.
  • test_wheel_engine.py: a redirect off the release CDN's host is refused and nothing is installed; under KIROCREW_CDN_BASE a mirror's redirect to another https host is followed and the verified wheel lands.
  • The existing download tests (digest mismatch installs nothing, size ceiling, cancel, deadline, loopback reads) answer the connection through opener_helpers.answer_with, with their assertions unchanged.

Determinism: the four new tests passed in five runs, each in a different shuffled order (-n 0). They use no clock, thread or network: the transport is replaced per test with monkeypatch.

Proof: PROVEN test/test_stt_engine_download.py::test_a_redirect_to_plaintext_is_refused, test/test_stt_engine_download.py::test_a_present_model_is_verified_against_its_pin_not_just_its_size, test/test_stt_engine_download.py::test_the_pin_check_is_not_cached_against_forgeable_metadata, test/test_stt_engine_download.py::test_a_failed_download_is_reported_not_raised, test/test_wheel_engine.py::TestWheelDownload::test_sha_mismatch_refused_and_file_removed (head 9a3ef70)

Manual verification

Checked the live endpoints: https://updates.crew.kiro.dev/feed/stable/latest-cli.json and the wheel url it names both answer 200 with no redirect; https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin answers 302 to another https host.

Related Issues

Part of #22230

Refs #10018, #21683.

Sibling sweep

Swept: git grep -n "urlopen(" -- src/ (each hit read for a sha256-pinned fetch)
Sibling: src/kiro_crew/stt/decoder.py -- not affected -- it fetches through stream_pinned_payload, which this PR moves onto the shared opener
Sibling: src/kiro_crew/eval/bench/datasets.py:249 -- not affected -- a developer's benchmark command, never run by the gateway, fetching source-constant urls that the shared policy lets change host anyway; the sha256 pin decides

Pattern harvest

Rule candidate: semgrep
Pattern: a sha256-pinned download that calls urllib.request.urlopen directly instead of asset_downloader.build_opener

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

The whisper model and ffmpeg decoder fetch (stt.models.stream_pinned_payload)
and the update engine's manifest and wheel fetch (wheel_engine_transfer's
_read_bounded) opened their connection with urllib.request.urlopen, whose
default redirect handler follows a hop to any host, plaintext included.

Both now open it through asset_downloader.build_opener, the one place that
answers which redirects a pinned transfer follows. The stt urls are source
constants or the operator's environment override and their upstreams
redirect to a download host of their own, so they take the cross-host,
https-only policy. The release CDN serves the manifest and wheel itself, so
that transfer keeps the same-host policy, except while an operator's
KIROCREW_CDN_BASE mirror is set. Each loop keeps its own bounds (pre-write size
ceiling, cancel, total deadline, mkstemp staging), which download_to does
not carry, so only the connection moves.

Part of #22230
@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 7, 2026 12:31
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 7, 2026
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 9a3ef7089d8845f6f6778c04b298f51377e3be63 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 9a3ef7089d8845f6f6778c04b298f51377e3be63 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design-level concerns: both fetches now use the shared redirect policy, and each is given the policy matching where its url comes from.

Notes

  • _read_bounded reads KIROCREW_CDN_BASE again on its own instead of using the base the url was built from. It is correct today because wheel_url is pinned to the canonical artifact_base. If a later caller of _read_bounded passes a url not built from cdn_bases, it will quietly get the cross-host policy while that env var is set.

[DESIGN-REVIEWED] 9a3ef70

Description read

[DESCRIPTION-READ] 5593637ebe27f4763f5f00892fb32f947ad042cda2aa381ff7fd45ef927578b3

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 9a3ef7089d8845f6f6778c04b298f51377e3be63 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

One sibling the sweep missed: cli_server_update.py:789 fetches the same release feed with a direct urllib.request.urlopen, which follows redirects to any host.

Notes

  • Item 2 — undeclared sibling: the PR moves the manifest fetch in _read_bounded onto the shared opener, but kirocrew update fetches the same {feed_base}/feed/{channel}/latest-cli.json at src/kiro_crew/cli_server_update.py:789 through urllib.request.urlopen. I grepped urlopen( under src/ and found one unfixed hit in the update path. The Sibling sweep lists only decoder.py and datasets.py. Subtraction: replace that urllib.request.urlopen call with the same asset_downloader.build_opener(...).open, or name it in the sweep and defer it to Route the stt model and wheel_engine downloads through asset_downloader.download_to #22230.

What this change ships

Inventory (7 items) — 6 justified

Intent: stop pinned stt and update downloads from following redirects to plaintext or off the CDN host. FIX.

  1. The whisper model and ffmpeg decoder downloads now refuse a redirect to http:// but still follow one to another https host — justified
  2. The update manifest and wheel downloads now refuse a redirect off the CDN host — justified (but the feed fetch in kirocrew update was left unfixed)
  3. With KIROCREW_CDN_BASE set, an update download may follow a redirect to another https host — justified
  4. The wheel_engine facade now re-exports asset_downloader, which the facade test requires — justified
  5. A shared test helper, test/opener_helpers.py, replaces the urlopen patches in 4 test files — justified
  6. Four new tests cover a followed redirect and a refused one — justified
  7. The stt spec, the update RFC and build_opener's docstring now describe the new route — justified

[FIRST-PRINCIPLES-REVIEWED] 9a3ef70

Description read

[DESCRIPTION-READ] c37a9f3f08d749d2f48e70ddf604f92e6ce60d5fd271063d0216f1d8d45a1dd8

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 9a3ef7089d8845f6f6778c04b298f51377e3be63 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 9a3ef70

Verdict parsed from the review's SHA-scoped output markers for commit 9a3ef7089d8845f6f6778c04b298f51377e3be63.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 9a3ef7089d8845f6f6778c04b298f51377e3be63 false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 9a3ef7089d8845f6f6778c04b298f51377e3be63 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9a3ef70

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9a3ef7089d8845f6f6778c04b298f51377e3be63 false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 7, 2026 12:44
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Nov 7, 2026

@KiroCrewCaptain KiroCrewCaptain left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for the on-call. Review lanes pass on 9a3ef70, no conflict, no open change request.

@KiroCrewCaptain
KiroCrewCaptain merged commit 9ea1ed8 into main Nov 7, 2026
96 checks passed
@KiroCrewCaptain
KiroCrewCaptain deleted the fix/22230-shared-redirect-policy branch November 7, 2026 19:43
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 7, 2026
buluoray pushed a commit that referenced this pull request Nov 8, 2026
…#32318)

The whisper model and ffmpeg decoder fetch (stt.models.stream_pinned_payload)
and the update engine's manifest and wheel fetch (wheel_engine_transfer's
_read_bounded) opened their connection with urllib.request.urlopen, whose
default redirect handler follows a hop to any host, plaintext included.

Both now open it through asset_downloader.build_opener, the one place that
answers which redirects a pinned transfer follows. The stt urls are source
constants or the operator's environment override and their upstreams
redirect to a download host of their own, so they take the cross-host,
https-only policy. The release CDN serves the manifest and wheel itself, so
that transfer keeps the same-host policy, except while an operator's
KIROCREW_CDN_BASE mirror is set. Each loop keeps its own bounds (pre-write size
ceiling, cancel, total deadline, mkstemp staging), which download_to does
not carry, so only the connection moves.

Part of #22230

(cherry picked from commit 9ea1ed8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants