Repository navigation
refactor(apps): route the papyrus and pptx_maker archive downloads through asset_downloader - #21683
Conversation
Sensitive Change ReviewSensitive files changed:
|
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Before merge, check one thing: What this change shipsInventory (8 items) — 8 justifiedIntent: let one function decide whether the Papyrus and PPTX Maker install archives are verified before anything uses them. This is a refactor that consolidates code (closes #10018).
[FIRST-PRINCIPLES-REVIEWED] a15b0b4 Description read[DESCRIPTION-READ] 642ad339d850844676db3d8d881242761ae9816b2d775eb339c619bcaa163fcd That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Verify that the shared opener's TLS context (CA-discovery fallback) and its default [DESIGN-REVIEWED] a15b0b4 Description read[DESCRIPTION-READ] b792d5f6fd348107f392626cf1f917e0302f44e13ab53b262b9ee3427705d110 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
3c692d3 to
59083aa
Compare
|
self-added: yes
|
Sensitive change review
|
59083aa to
afbd6c6
Compare
|
self-added: yes
|
|
Sensitive change review
|
buluoray
left a comment
There was a problem hiding this comment.
Approving at afbd6c6. Apart from the rebase, the only change since 59083aa is the new test TestRedaction::test_returned_reason_never_carries_the_credentialed_url, so the earlier check stands: both apps go through download_to with their sha256 pins and extract only when ok is true, extraction filters are unchanged, redirects stay https-only, and every failure path is fail-closed.
The new test closes the gap on the returned reason: putting the raw url or the exception text back into either download_to failure return makes the InvalidURL or HTTPError case fail. asset_downloader.py is still filed as non-egress, with a comment saying the callers carry the string to the dashboard; the behaviour test now pins that boundary, which is acceptable, though moving the module to the sinks list would let the drift guard catch it too. Related tests pass on this head (269 passed). CI was still running with 0 failed when I checked.
|
|
Review-ready at afbd6c6.
This Crew will not merge, approve, or enable auto-merge; merging is the maintainer's call. |
afbd6c6 to
d89419e
Compare
|
Rebased to clear a merge conflict (afbd6c6 -> d89419e, base 88914fd). The conflict was one hunk in |
Sensitive change review
|
|
CI note for d89419e:
|
|
|
…rough asset_downloader
d89419e to
a15b0b4
Compare
Sensitive change review
|
|
|
|
Review-ready at a15b0b4.
This Crew will not merge, approve, or enable auto-merge; merging is the maintainer's call. |
Problem / Motivation
Goal: The Papyrus (Tectonic) and PPTX Maker (engine) installs fetch their pinned archives through the shared
asset_downloader.download_to, so one piece of code decides whether downloaded bytes are verified before anything uses them.Both app installers still carry their own copy of the transfer: their own opener, their own redirect handler, their own stream-and-hash loop and their own error wording. The embedding model and feature videos already use
asset_downloader.download_to(#9972). Issue #10018 asks for the last two.Why it matters
"Did we verify this before installing it?" has one answer only if one function answers it. With three copies, a fix to one (redaction, redirect policy, a size bound, a staging-file check) has to be repeated by hand in the others, and the copies already drifted: the app copies wrote the archive straight to its final name, while the shared one stages it, opens it without following links, and renames it only after the digest matches.
Not a goal
stt/models.pyandplatform/wheel_engine.py. The issue body lists them as other private downloaders, but its "done" criteria name only these two app call sites.wheel_engine.pyalso has an open PR (feat(session): bound owner-session recursion with lineage quotas #21132) touching it.memory.embed_model_urlconfig knob from the issue thread; the comment there says it belongs in its own change.What changed (motivation → approach → change)
Papyrus
_download_toand PPTX Makerdownload_archivenow callasset_downloader.download_towith their existing sha256 pin, timeout and chunk size. PPTX Maker passes its size floor asmin_bytes; Papyrus passes its progress and "verifying" callbacks. The archive lands at its path only after the digest matches, so extraction still never sees unverified bytes.Both pass
allow_cross_host_redirects=True. Their pinned GitHub URLs redirect to another GitHub host, and their old handlers already allowed any https hop. The docstrings insrc/kiro_crew/asset_downloader.pynow name a source constant as the second shape that option is for, next to an operator's environment override. No code there changes.Deleted: both private redirect handlers and both private
redact_urlcopies; their tests now callasset_downloader.redact_url. Twosecurity_postureallowlist entries go with them, because neither module calls a redactor any more.Download error text now reads
Tectonic <asset>: HTTPS download failed: <Type> from <host>andREFUSING the engine archive for <commit>: <reason>. Nothing was extracted, built or run.Archive-shape refusals (archive rejected: …,engine archive rejected: …) are unchanged.Backwards compatibility
Compatible: same URLs, same pins, same redirect policy, same archive checks. Only the wording of a download failure changes, and nothing parses it.
Reader: src/kiro_crew/apps/builtins/pptx_maker/backend/engine_source.py:_HttpsOnlyRedirectHandler -- app -- removed; its only reader was
test_a_non_https_redirect_is_refused, replaced bytest_redirects_may_change_host_but_never_leave_httpsagainst the shared handlerReader: src/kiro_crew/apps/builtins/papyrus/backend/tectonic.py:_HttpsOnlyRedirectHandler -- app -- removed; no reader outside the module on origin/main
Reader: src/kiro_crew/apps/builtins//backend/:redact_url -- app -- removed; no reader under src/, website/ or scripts/ on origin/main. Its only readers were six test lines, which now call
asset_downloader.redact_url(same host+port output) with the same assertionsReader: src/kiro_crew/security_posture.py:NON_EGRESS_REDACTION_MODULES -- release -- two entries removed;
test_allowlist_has_no_stale_entriesrequires it, andtest_every_redactor_call_site_is_a_registered_sink_or_allowlistedstill passesReader: "implausibly small" (pptx download message) -- app -- now "downloaded file too small"; read only by
test_a_truncated_transfer_is_refused_even_if_it_hashed, updatedTests
test/test_papyrus_tectonic.pyandtest/test_pptx_maker_engine_source.py, new:asset_downloader.download_towith its pin andallow_cross_host_redirects=True;http://hop.test/test_asset_downloader.py, new:download_to's returned reason, which both apps put on the dashboard, carries the exception type and host but no credential from the url, forInvalidURLandHTTPError(userinfo, path token and signed query all checked).urllib.requestdirectly instead of reaching it through the app module. The credential-redaction tests (InvalidURL,OSError,TimeoutError,ValueErrorwith a secret in the url) pass unchanged against the shared path.test_asset_downloader.pyunder two (seeds 1, 2: 61 passed each). They use no sleeps, clocks or network.run_scoped_tests.py --surface backend(36 related files, 3893 passed),test_flake_pattern_ratchet.py, isort, flake8, mypy on the four changed modules, the black ratchet, comment-history, docs-lint, and the repo semgrep rules on the diff.Manual verification
N/A — unit coverage sufficient: the change swaps the transfer under two callers, and every branch (success, mismatch, too small, transport error, corrupt archive) runs through the real shared function with only the socket mocked. No real download was made.
Related Issues
Closes #10018
Follow-up: #22230 moves
stt/models.pyandplatform/wheel_engine_transfer.pyonto the same shared downloader.Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)