Skip to content

feat: host feature-video clips on a signed CDN manifest - #9972

Merged
bolichen97 merged 1 commit into
mainfrom
feat/feature-videos-hosted-backend
Sep 13, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
feat/feature-videos-hosted-backend

Conversation

@iamwhatever

@iamwhatever iamwhatever commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

A feature-intro clip could only ship inside the wheel. The catalog was a static
tuple compiled into the package and the media rode along with it, so adding one
video meant cutting a release — and every install paid for every clip, watched or
not. That is the right shape for two clips and the wrong one for a library.

Why it matters

Hosting the clips decouples recording one from cutting a release, and takes the
media out of the wheel: an install with the feature off (the default) never
fetches a clip, and an install with it on fetches its own release's clips in the
background, once, and plays them from disk after that. But a hosted clip is a file
the gateway fetches from the open internet, so acting on it safely is the whole
problem this PR solves.

What changed (motivation → approach → change)

Clips now come from a manifest published per release folder on the CDN, signed
with the same offline key cli.sh pins for the update feed under a distinct
schema string, and cached in ~/.kiro/crew/feature-videos/<release>/ by a
background pass at gateway boot. The static catalog stays as the fallback for an
install that has never fetched a manifest.

Three properties make it safe to act on bytes from a CDN.

  1. The manifest is verified, and an unverified one is discarded whole — not
    "the bad entries are dropped", because the signature covers the document.
    platform/feed_trust.py's verification core is now shared: the new
    verify_document_signature differs from the CLI feed's entry point only in
    allowing a nested payload and taking its own size cap. One trust root, and the
    two document kinds stay non-interchangeable because each requires its own
    schema inside the signed payload. One key with a third consumer is a
    rotation fact, so the release runbook (docs/build/release.md) and the
    signing README name it beside the existing two.
  2. Every clip is sha256-pinned by that signed manifest, so the CDN is not
    trusted. A tampered object can only fail verification; it cannot reach the cache.
  3. The browser never fetches from the CDN. Only the gateway does, through one
    opener that verifies TLS, refuses any redirect off the exact origin (hostname
    and port), and checks the sha256 before a file is installed. A hosted clip
    that is not on disk yet is not offered — not from the CDN, not from anywhere —
    so every byte a <video> plays was verified here first. A manifest replaces
    the bundled list rather than adding to it, so on the launch where a manifest has
    arrived but no clip has landed yet, nothing plays; the pass runs in the
    background and the next launch shows what it landed. That one quiet launch is
    the trade, chosen over ever offering a bundled clip and its hosted successor as
    two videos. This is a validation invariant (validate_asset_path admits only
    this origin), not a fourth governance consult — the three chokepoints below are
    the whole governance surface.

Governance. New capabilities.feature_videos_download scope (fail-closed, in
the egress family with telemetry / publish), consulted at three chokepoints
because any one alone is a half-control: each manifest request (the fallback walk
can make up to four, and re-asks before every one, so a withdrawal between two
candidates stops the next), each clip request (the poster and the clip each take
their own audited answer, so a withdrawal landing during one transfer is not
followed by the next request on the old permit), and
POST /api/feature-videos/fetch-all (403, rather than a task that would deny
itself). All three are server-side, and that is the whole surface: the browser is
never handed a CDN url, so there is no client-side fetch for the ceiling to miss.
The operator's kill switch is re-read per entry in the same loop, so turning the
feature off mid-pass stops the pass the same way a withdrawn ceiling does. The 60s
memo (download_permitted_cached) has exactly one reader, the download_enabled
field on /status, which the settings panel polls while a transfer runs. Already-
cached clips keep playing under a denial: withdrawing bytes already on disk is a
separate decision this row does not make. No CSP change: media-src stays
'self' blob:. The answer is deliberately not repeated on
GET /api/dashboard/config, which every install fetches on every load and where an
audited read would spend a governance decision on a readout nobody consumes.

Read, write and delete under the cache share one anchor. checked_cache_root()
refuses a cache root that is itself a symlink and returns its canonical path; every
cache write (ensure_cache_dir, which the manifest store and both clip transfers ask
for the folder), the serving route (resolve_served_path) and the eviction pass
(release_folders and the rmtree in _remove_release) derive their containment
from it, and each additionally refuses a symlinked release folder and re-checks that
the folder it is about to write, serve or delete resolves to <canonical root>/<release>. Without that, a planted <root>/<release> link would make the
manifest store and clip staging land wherever the link points, and a linked ROOT
would hand the eviction pass real directories outside the cache to delete. A link is
refused rather than followed even when its target is inside the cache; the write
side opens the root no-follow and proves the descriptor's real path is the canonical
root BEFORE creating anything under it, creates the release folder relative to that
held root, and applies the owner-only mode through the folder's own descriptor
(fchmod; on Windows the DACL by name under a handle that blocks the rename) — a
mkdir or chmod by name after a by-name check would follow a link planted in
between, and the chmod has no undo; and the delete side re-checks immediately before
rmtree, because that call is the irreversible one.

The check is then settled on the descriptor, not the name. A by-name check
followed by a by-name open leaves a window: rename the checked release folder (or
the root) away, put a link at its name, and the open — however carefully the LEAF is
handled — follows the link and finds an ordinary file or folder at the end. Both
sides now ask the kernel about the thing they already hold open. Reading: the
serving route opens root, release folder and file as a chain of no-follow opens,
each relative to the descriptor before it (the root on its own name, so a root
swapped for a link is refused rather than followed — a by-name resolve() of the
root would have handed back the link's target as if it were the root), and requires
the open file's real path (pinned_fs.fd_real_path — /proc/self/fd on Linux,
F_GETPATH on macOS, GetFinalPathNameByHandleW on Windows) to be the held root's
real path plus <release>/<name>, and its link count to be 1, so a root or folder
swapped for a link after the resolver's checks is a 404 and a hard link into the
cache (a second name for an outside inode, invisible to every path rule) is not
served. Writing: every write under the
cache — both transfers, the install rename, the receipts, the manifest store — runs
inside feature_videos_manifest.pinned_release_dir, which creates and checks the
folder by name, then holds it open (asset_downloader.pin_target_dir: the ancestor
chain pinned one openat at a time on POSIX, a handle that blocks renames of the
folder and everything above it on Windows), proves the DESCRIPTOR's real path is
<canonical root>/<release>, and hands the downloader a PinnedTargetDir so the
staging open, the resume hash, the install rename and the lockdown are
descriptor-relative on POSIX and by name under the held handle on Windows. The real
path is what makes the pin sufficient: a swap landing before the open resolves is
followed by the resolve and pinned perfectly, and only the kernel's answer for the
inode already held can tell. A platform that cannot read it refuses. Deleting:
eviction removes a release folder through pinned_fs.remove_tree_pinned — the
folder is opened with its ancestor chain pinned, the removal is approved only when
that open directory's real path is <canonical root>/<release>, and every entry
under it is removed against the inodes one scan found; on Windows the folder is
held open (the handle blocks a rename of it and of every ancestor), proven the same
way, emptied under that hold and rmdired last. Reading back: the cached manifest
is read no-follow through a pinned parent and capped at the 1 MiB the network fetch
already enforces, so a link planted at its name is refused rather than read to
exhaustion. And every link check under the cache refuses a Windows junction the way
it refuses a symlink (platform_compat.is_link_or_junction): a junction lstat's as
a plain directory, and a non-admin user can make one. Pinned by tests that rename
the pinned folder away and plant a link at its name mid-transfer (bytes land in the
moved folder, the link's target receives nothing), swap the root between the check
and the open (refused by the real path — on the write side and on the delete side),
swap the release folder after the serving route's resolution (404), swap it after
the eviction listing (nothing deleted, the link's target intact), hard-link a file
into the cache (404), plant a link at the cached manifest's name (no manifest), and
pad a cached manifest past the cap (no manifest). download_to keeps its by-name behaviour for a caller that passes no
target_dir (the model download; #10018 tracks the siblings), named
TargetDir so which callers hold which guarantee is visible at the call site.

A re-published clip is fetched again. Each verified install leaves a receipt
beside the file (.<name>.sha256) naming the pin it passed, and is_cached requires
presence, the declared clip size and a receipt equal to the pin the manifest in
force carries. Without the receipt, a signed re-publish that keeps a basename and
changes its bytes — same clip size, new content; or any re-encoded poster, which has
no size to compare — would read as cached and be served stale for the life of the
release, with nothing downstream re-hashing. The receipt answers the question a
re-hash would (was THIS content verified?) at the cost of one tiny read on the polled
route; a file with no receipt is never counted as cached; the receipt's own name is
not servable (dot prefix, .sha256 suffix). A receipt that cannot be written fails
that ENTRY with a stated reason — the media stays, reads as not cached, and the next
pass re-fetches it — rather than raising out of the background pass and leaving the
status stuck on downloading with every later entry skipped.

Selection draws uniformly among eligible clips instead of taking the first in
catalog order. Order stopped being a statement about the user: with a hosted
library, catalog order is publication order, so the first entry would always win
and the rest would only be reachable by retiring the ones ahead of it.
Eligibility is unchanged and still deterministic — a retired clip, or one for a
feature you already use, still cannot appear. Every candidate is already on disk,
so the pick costs no egress and no spinner.

The transfer engine moved out of embeddings.py into asset_downloader.py,
shared by both callers, with resume added: a partial stays in a .part staging
file nobody serves and the next boot continues it. embeddings.py keeps everything
that is about the model (which url, the pins, the Ollama salvage, the retry ladder,
the progress dict), and redact_model_url keeps its own path-preserving redaction
because kirocrew doctor prints it to name which model file resolved. One
behaviour change for the model transfer, intended:
it now goes through the shared
opener, so its redirect policy is the shared one. The old inline urlopen followed
any redirect from any url. Now: the CDN default and the agent-writable
memory.embed_model_url config knob may redirect only within their own host and
port; the KIROCREW_EMBED_MODEL_URL environment override may redirect to
another https host, never to plaintext. The split is about who chose the url. An
env value was set by whoever launched the gateway, and their own mirror commonly
answers with a 302 to a storage host (an artifact store fronting a bucket, a bucket
redirecting to its regional endpoint) — refusing that would break the ordinary
mirror shape for no gain, since the sha256 pin decides what is installed. A config
value can be written by the agent's own tools and the CDN is not ours to trust with a
destination, so both keep the host pin. All four cases are pinned by tests. A second private
downloader would be a second place for "did we verify this before installing it?"
to be answered differently. This
consolidation is partial on purpose.
papyrus/backend/tectonic.py and
pptx_maker/backend/engine_source.py keep their own stream-hash-verify-install
loops and adopt only make_ssl_context() from the shared module (a
behaviour-preserving dedup of their SSL-context copies — the reason those two
app-backend files appear in this diff). stt/models.py is not touched at all: it
keeps its bare urlopen and its own loop, as does
platform/wheel_engine.py::_download_to_file. Moving all four onto download_to is
tracked in #10018 and is not
this PR's job: each has its own install contract to re-verify, and folding four
more call sites into a feature-video change would bury the review of both.

The staging file is never written through a link, on any platform, with no
check-to-open window.
A fresh transfer removes whatever sits at the staging name
(a link is removed as a link; a directory is refused) and creates it exclusively
(O_CREAT | O_EXCL), so a link planted in between fails the create itself. A resumed
transfer opens the partial ONCE, for read and append, and then compares the
descriptor to the name — fstat against lstat, device and file index — and
refuses when they are not the same object. O_NOFOLLOW already refuses a link on POSIX; the descriptor comparison
is what covers Windows, where that flag does not exist and os.open follows a
reparse point. The descriptor must also be the inode's only name (st_nlink == 1):
a hard link at the staging name is not a link to follow and IS the named inode, so
it passes both rules above, yet appending to it would write into a file that also
answers to another name and a digest failure would unlink only the staging name.
And what reaches the final name is the inode that was hashed, or nothing: the
digest covers the bytes written to one descriptor, the rename is by name, so after
the rename the installed name is opened no-follow and its identity compared to that
descriptor's — a file swapped in at the staging name between the last write and
the rename is unlinked and the transfer fails instead of landing under a digest it
never met. On POSIX that descriptor stays open until the check is done, because an
inode with an open descriptor cannot be handed back to a file recreated at the
same name (Windows must close it before the rename, and its file ids carry a
reuse sequence number instead). A resumed transfer is held to the same rule from
the other side: the prefix is hashed through the very descriptor the rest of the
body is then appended through, and that descriptor is the one the install checks
— the partial is never re-opened by name after it was hashed, so a same-size file
swapped in at the name between the hash and the append is neither appended to nor
installed. All of it is pinned by tests.

A redirect is followed only to the same origin — hostname AND effective port —
unless the url is the operator's own.
_SameHostRedirectHandler, the default for
every caller and the only policy for a url from a CDN, a signed manifest or a config
file, refuses a redirect off https, to another hostname, or to another port on the
same hostname (:8443 on the trusted name is a different service, and on a box that
co-locates an internal listener with that name it is the same SSRF one port over).
An explicit :443 is the same origin as no port. A non-numeric port is refused
rather than raised. build_opener(allow_cross_host_redirects=True) is the one
opt-in: it swaps in _HttpsOnlyRedirectHandler, which still refuses plaintext but
lets the host change, and only a url from the gateway's process environment
(KIROCREW_EMBED_MODEL_URL, KIROCREW_FEATURE_VIDEOS_MANIFEST_URL) is passed
with it — see the model paragraph above for why that line is drawn at "who set it".
A refused redirect is a named failure, not a bare HTTPError 302: both handlers
raise RedirectRefused, whose reason carries the two hostnames and the policy in
force (never a path, query or credential), and download_to returns it verbatim —
so the dashboard's model-download status and the feature-video failure state read
"redirect from to refused: … honoured only when set through the
environment override" instead of a transport error that looks like a broken CDN.
The production CDN itself answers the model GET with a direct 200 (checked with a
HEAD against d3j0sthz5doyui.cloudfront.net); no install depends on a hop.

The manifest mirror override is an environment variable, not a config key.
KIROCREW_FEATURE_VIDEOS_MANIFEST_URL is the only override channel. There is no
dashboard.feature_videos_manifest_url: config.json is writable by the agent's own
tools, and a url the agent can choose is a GET the gateway makes to any https host on
the agent's behalf — before the signature check has anything to check. That is a
request target, not a preference, so it belongs to whoever launches the gateway. (The
existing memory.embed_model_url config knob has the same shape and predates this PR;
it is recorded in #10018 alongside the other downloader siblings.) It relocates the
MANIFEST fetch and nothing else: cdn_base is inside the signed document, so a copied
manifest still names the public CDN, and serving clips from another host needs a
manifest signed with that host in it; the user doc says exactly that, and does not
call this an air-gap story. The override names
the manifest directly, so it is fetched once, and the release it declares need not
match the running version — but it must be one the running build reads back (this
release, this minor's .0, or the .0 of up to three earlier minors). A document
outside that window is refused with a logged reason instead of being cached under a
folder the next start would never open.

/next answers {video, enabled} and nothing else. There is no source
field — every offer is played from this origin, so it could only ever say local,
and a value with one possible reading has no reader — and no download_enabled:
whether bytes may be pulled decides what the background pass lands, never what this
route offers, and the settings panel reads it from /status. The merged frontend
(#9995) declares both optional and defines absent as local / off, so this is
contract-safe. There is also no probe route: every offer's files were proved on
disk by the request that offered it, so the dialog opens on the /next answer
alone. The frontend's featureVideoProbe client and its source === 'remote'
branch are now unreachable code on main; removing them is a frontend-only
follow-up and is not attempted from this branch.

New endpoints:

  • POST /api/feature-videos/fetch-all — runs the pass now, rate limit lifted. It
    refuses instead of starting a task that would no-op: 409 feature_disabled when
    the kill switch is off, 403 governance_denied when the ceiling forbids the
    fetch, so the settings panel never polls for bytes that are not coming.
  • GET /feature-videos/<release>/<file> — the cache reader. A dynamic route, never
    add_static and never a listing: it validates both URL components against the
    same rules the manifest applies — including the suffix rule, so only a name the
    parser could have admitted for a clip (.mp4) or a poster (.jpg / .jpeg /
    .png / .webp) is served, the Content-Type comes from a fixed table keyed by
    that suffix, and X-Content-Type-Options: nosniff is set. The release folder is
    in the user's data home, so a file another local process plants there is
    reachable by name; with the suffix rule a planted x.html is a 404, and nothing
    this route answers is a type the browser executes on the dashboard's origin. It
    anchors containment to the canonical cache root and refuses a symlink at the
    release folder and at the file alike. A manifest names files, so a link has no
    legitimate reader — it is refused rather than followed, even when its target
    sits inside the cache. The body is then streamed from a descriptor opened ONCE
    with no-follow semantics (O_NOFOLLOW; the reparse-point flag on Windows) and
    type-checked with fstat — not through FileResponse, which reopens by path
    after the check and would follow a link planted in between. Range is honoured
    (206 + Content-Range, 416 for an unsatisfiable one) so the player can seek, and
    HEAD carries the headers with no body. /feature-videos/ is in
    SPA_FALLBACK_EXCLUDED_PREFIXES: it is a data namespace, so a request with no
    session gets the auth refusal, never index.html with a 200 that a <video> would
    render as nothing.

dashboard.feature_videos_enabled keeps its default of false.

Contract

This is the backend half of a three-way split. The publishing script is
PR #9964; the frontend is
PR #9995, already on main,
— this branch is rebased on it and its FeatureVideo / FeatureVideoNext /
FeatureVideoStatus types were checked field by field against what the handlers
emit. /status carries every field FeatureVideoStatus names; /next omits the
two optional ones (source, download_enabled) whose absent reading the types
define.

Schema agreement with #9964 — verified field by field against its
scripts/feature-videos/_manifest.py:
same schema string, same top-level
fields (schema, release, cdn_base, generated_at, entries, signature),
same entry fields (id, feature, title, description, file, poster,
sha256, poster_sha256, bytes, duration_s, doc, used_when,
min_version), same canonical byte form (sorted keys, compact separators, ASCII,
trailing newline, nested values included), same trust root.

Four consumer-side decisions worth stating, all of them "accept what the publisher
can emit":

  • key_id is optional here, and pinned when present. feat: sign feature-video release folders for the CDN #9964 records it on the
    KMS path and omits it for a local staging key. The key is pinned, so key_id
    never established trust — requiring it would make a publisher that omits the hint
    indistinguishable from a forgery.
  • The signed-payload cap is 256 KiB, the publisher's own. A tighter cap would
    discard a valid release, and the discard is whole-document — the failure would be
    "no videos", not "one clip missing". The fetch cap sits above it at 1 MiB, since
    the document is the payload plus a signature.

Every bound, by name

Stated together so neither side has to infer one from the other. Each is also
documented at its definition in src/kiro_crew/feature_videos_manifest.py.

Bound Constant Value Owner
Signed payload — the bytes a signature covers _SIGNED_PAYLOAD_MAX_BYTES 256 KiB matched to the publisher's DEFAULT_MAX_PAYLOAD_BYTES
Whole document — the fetch buffer, the outer bound _MANIFEST_MAX_BYTES 1 MiB consumer; must exceed the payload cap, since a document is payload + signature
One clip's declared bytes _MAX_ENTRY_BYTES 64 MiB consumer; looser than the publisher's 25 MiB per-file default, so it refuses nothing the tool emits
Entry count _MAX_ENTRIES 1000 consumer coherence bound only; the payload cap is the real limit
One clip's duration_s _MAX_DURATION_S 3600 s consumer; an out-of-range, non-finite, or float-overflowing value reads as 0.0 ("unknown") instead of ending the parse — float() of a 310-digit integer raises, and one signed manifest must not take every entry down with it
File and poster names unique across the manifest — consumer; a basename is a cache path, so two entries sharing one would overwrite each other's verified bytes and one offer would play the other's clip. The second entry is dropped with a logged reason

64 KiB is not the cap. It was the value before I read #9964, and it is not what
this PR ships — the code went to 256 KiB in the same change that reconciled the
schema. A publisher that self-limits to 64 KiB is safe (anything it emits verifies)
and needs no change on either side, but it must not document 64 KiB as the consumer's
limit, and the outer bound is 1 MiB rather than 256 KiB. The relationship is the part
worth holding onto: the consumer's payload cap must be ≥ the publisher's, because
exceeding it discards the whole manifest rather than one entry.

  • A release may be a bare numeric version of any depth (1–4 components), which
    is what feat: sign feature-video release folders for the CDN #9964 accepts. The fetch path still only ever asks for
    major.minor.patch folders.
  • The entry-count bound is a coherence check at 1000, far above what the payload
    cap already allows, so a document the publisher would emit cannot trip it instead.
  • The cache ceiling is clamped to a byte count a disk could hold (MAX_CACHE_BYTES,
    8 PiB). A configured megabyte figure is compared against that ceiling before the
    multiply, because the multiply alone overflows a finite 1e308 to inf. A value that
    is not a finite positive number reads as "no size cap", the same answer a missing key
    gives — a ceiling nobody can act on must not become a reason to delete clips.

Signature envelope — unchanged, and now written down

Confirming for the publish-tool session: verify_document_signature adds no new
canonicalization rule
. It and the CLI feed's verify_manifest_signature are two
thin entry points over one _verify_signature core, and the only two things either
passes in are is a nested payload allowed and what is the size cap. The key, the
canonical-JSON shape, the trailing newline, the ASCII encoding and the openssl
invocation are not parameters.

The exact bytes to sign are now spelled out as a numbered five-step block in
src/kiro_crew/platform/feed_trust.py's module docstring — copy from there
rather than reading it off the code. One step is worth repeating here because it is
the one that fails silently and only for some entries: the payload is serialized
ensure_ascii=True and encoded ASCII, so a tool that emitted UTF-8 with
ensure_ascii=False produces a document that verifies for every ASCII-only entry
and never for one carrying a non-ASCII title.

Shared fixture

test/fixtures/feature-videos/manifest.json          # signed, two entries
test/fixtures/feature-videos/signing-key.pub.pem    # public half of the throwaway key
test/fixtures/feature-videos/regenerate.py          # re-signs the fixture with a fresh key
test/feature_video_fixture.py                       # the helper both trees' tests import

The fixture manifest deliberately exercises what two independent encoders disagree
about: two entries (so nested-list order is pinned), keys written out of
alphabetical order (so sort_keys is doing real work), a non-ASCII title, and
one entry with used_when/min_version set beside one with both empty.

test/feature_video_fixture.py is the seam to import, and it carries both halves:

  • verify — load_fixture_manifest() + pin_fixture_key(monkeypatch) points
    feed_trust's pins at the committed public key, so verifying the committed bytes
    needs nothing else;
  • sign — mint_throwaway_key(dir) and sign_document(private, doc, scratch),
    the latter being exactly the canonical byte form above, for the signing half of a
    round-trip test.

One deviation from the ask, and the reason: the PRIVATE key is not committed. A
private-key PEM in the diff is what p/secrets exists to catch, and the SAST job
runs --config p/secrets --error on the diff — so committing one reds the gate, and
the only way to make it pass would be a .semgrepignore entry carving out our own
file. mint_throwaway_key() covers the signing half instead, and a test asserts the
fixture directory contains no private key so this cannot regress. Two further guards
keep the fixture from ever becoming an authority: a test asserts the fixture key id
and PEM differ from the real PINNED_KEY_ID / PINNED_PUBLIC_KEY_B64, and
regenerate.py rotates the key freely because nothing pins its identity — the
fixture is proof of a byte format, never of trust.

Regenerating (only needed if the schema changes):

python3 test/fixtures/feature-videos/regenerate.py

TestSharedFixture in test/test_feature_videos_hosted.py covers: the committed
fixture verifies and parses; the non-ASCII entry survives; re-serializing the file
with different indentation still verifies (the signature covers the canonical form,
not the stored bytes — a verifier that hashed the file as stored would pass this
fixture and fail every real release); tampering fails; the helper's local
canonicalization agrees with the verifier byte for byte while a UTF-8 /
unsorted / pretty-printed payload signed with the same key does not; and the two
authority guards above.

One deliberate deviation from the work item's wording: /status keeps state
as the {id: {status, ts}} engagement map and reports the transfer's step as a new
download_state. Renaming state would have been the smaller diff and the larger
break — the settings panel already reads it.

Release fallback, as documented: this release, then this minor line's .0 when
the running patch is not 0, then the .0 of up to three earlier minors. Never
across a major boundary, which is where a stale clip is most likely to show UI that
no longer exists.

Flagged signals

  • ! config/infra file changed — one new dashboard.feature_videos_cache_max_mb
    key (the manifest mirror override is env-only, see above), config-baseline.json
    regenerated from the dataclass registry, and one line in conftest.py extending
    the existing no-download floor to
    KIROCREW_SKIP_FEATURE_VIDEO_DOWNLOAD so no test can reach a CDN. No default
    behaviour changes: the feature stays off.

Tests

test_asset_downloader.py (39) — resume from a partial, a server that ignores the
Range (restarts rather than appending), a partial at/past the declared size, a
corrupt prefix caught end-to-end by the digest, sha mismatch, size overshoot,
too-small, non-https and file:// refusal, rate-limit pacing, owner-only install,
progress/verify callbacks, url redaction.

test_feature_videos_hosted.py (189) — real signatures, via a generated RSA
pair with feed_trust's pins repointed at it (the same fixture shape
test_feed_trust.py uses), so the positive path runs genuine openssl verification
rather than a stub that would pass for a manifest nobody signed. Covers: tampering
one entry discarding the whole manifest, another key, a disagreeing key_id, the
two document kinds not being interchangeable, every unsafe entry shape dropped
individually while good siblings survive (a reused file or poster basename included,
case-folded), release-candidate order and the major
boundary, fetch fallback and the location-mismatch refusal, the on-disk cache being
re-verified (an edited cdn_base is refused), owner-only permissions, eviction
order / running-release immunity, the download pass, the shared containment anchor (a
symlinked release folder or cache root is refused for writes and nothing lands
outside; the clip transfer is refused through the same gate; a linked root lists no
eviction candidates, _remove_release refuses even when handed the path, and the
serving route answers nothing), the override manifest rule (a release in
the fallback window is fetched once and read back after a restart; one outside it is
refused), a withdrawal between two fallback requests stopping the second, the
serving route's suffix rule (a planted .html / .svg / .js / extensionless
file under the release folder is a 404 even though it is a regular file on disk;
the served type comes from the suffix table, not the bytes; the table equals the
parser's admitted suffixes), the install receipt (a re-published clip of the same
size, or a re-published poster, is not cached; files without a receipt and a
malformed receipt are not cached; the receipt is never served; an unwritable receipt
fails the entry, not the pass), the serving route's streaming (full body, four
Range shapes with Content-Range, four 416 shapes, HEAD without a body, and the
check-to-open window: a file swapped for a symlink or a directory after resolution
is a 404 and the target is never read; the release folder swapped for a link after
resolution is a 404; a hard link into the cache is a 404; a descriptor whose real
path cannot be read is a 404), the pinned write side (a transfer through a pinned
folder lands in that folder after its name is swapped for a link, and the link's
target receives nothing; a link or a file at the folder name cannot be pinned; the
root swapped between the check and the open is refused by the descriptor's real
path; no real path, no write — the entry fails with a reason and a path-addressed
receipt goes through the same pin), a non-string doc dropped rather than crashing
the parse, static-route path
hardening including a symlink
escape, every off-origin src refused (the manifest's own CDN host included), an
uncached hosted clip never offered over the network (the module imports no url
client at all), a manifest with nothing landed yet offering nothing that launch,
governance denial at each chokepoint including between an entry's poster and its
clip, and /next answering exactly {video, enabled} with no governance read (a
cached clip still plays locally under a denial).

test_feature_videos.py — four existing tests updated where the change reshapes
them (first-in-order → an eligible pick; stability → whole-set reachability plus a
seeded uniformity check; the grown /status body; a verdict withdrawing whichever
clip was shown). Every other assertion's intent is untouched.

test_embeddings.py — the urlopen monkeypatch target follows the transfer into
asset_downloader, plus four tests driving the production redirect handlers over a
fake transport: the CDN default and the config-knob url refuse a cross-host 302 (the
second host is never requested, nothing installed, and the status error names the
refused hop and the remedy); the env override follows one to another https host and
installs the pinned bytes; the env override still refuses a 302 to plaintext. One
assertion changed on purpose: the too-small branch used to
read the staged size after unlinking it, so every undersized download surfaced as
a generic transport error; the shared engine reads it before, and now says "too
small". The test's own docstring called that a known quirk.

Local gates green: 356 tests across the four feature-video/downloader files, plus
test_embeddings.py, test_feed_trust.py, test_social_share_governance.py,
test_dashboard_security_headers.py, test_route_registry.py, the governance
suite, and the dashboard-config knob suites (~1,900 tests). black (baseline
respected), isort, flake8, mypy, docs-lint, brand, harness-parity,
feature-map, changelog-history, sync-io-in-async, comment-history,
agent-sdk-boundary, lockdown-before-publish, testpaths-coverage,
subprocess-encoding.

Docs

Owning specs updated in this commit: governance.md (the new scope, its three
chokepoints, and why no CSP change is needed),
learn-cron-dashboard.md (the endpoints, the selection rule, the cache and the
serving route), cli.md (one key, two signed document kinds),
memory-skills-hooks.md (where the transfer now lives),
docs/feature-map/README.md. User-facing: a "Hosted Clips" section in
src/kiro_crew/docs/feature-videos.md and one row in configuration.md.

Note src/kiro_crew/docs/feature-videos.md is also touched by #9964 — both edits
are additive to different sections; whichever lands second rebases.

Linked issue

no linked issue: this is one work item of an internally-tracked goal, dispatched
with its own written spec. PR #9964
is a sibling work item of the same goal, not an issue this closes.

@iamwhatever
iamwhatever requested review from a team and hoang-phan98 September 10, 2026 21:33
@iamwhatever

Copy link
Copy Markdown
Contributor Author

Intent: Let a feature-intro clip ship without cutting a release, by serving clips from a signed per-release CDN manifest and caching them locally — with the signature, a per-file sha256 pin, and a host pin from that signed document as the three things that make CDN bytes safe to act on, and one fail-closed governance scope that can withdraw the whole fetch.
Not a goal: Turning the feature on (dashboard.feature_videos_enabled stays false), producing the manifest (that is #9964), any frontend change, or recording clips.

@iamwhatever
iamwhatever force-pushed the feat/feature-videos-hosted-backend branch 2 times, most recently from 0413697 to 047cbce Compare September 10, 2026 22:25
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 10, 2026
@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 0b091e18b6711d8f3a260ed15309373c76be10a0 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Sound, well-argued design; one seam is off — the model-download redirect pin on the agent-writable config knob breaks real mirrors while protecting nothing.

Watch

The consolidation gives memory.embed_model_url a same-origin redirect policy the old inline urlopen never had, but _resolve_model_url accepts any https:// host from that knob unvalidated — an agent that can write config already points the initial GET anywhere, so refusing a cross-host 302 constrains no attacker while refusing what the PR itself calls "the ordinary mirror shape" (a 302 to a storage host). Consequence: an existing install whose config-set mirror hops cross-host regresses at its next model fetch, and the only remedy is migrating the value to the env var. The "who chose the url" line is coherent for the CDN default; for the config knob it yields breakage with no marginal security.
Clears when: the config-knob URL gets the same https→https redirect allowance as the env override, or the regression is explicitly accepted and named in the release notes alongside the remedy the error message already states.

Suggestions

  • Schema separation between the two signed document kinds holds only at the consumers; anyone with signing access via the routine video-publishing pipeline can mint a document carrying the update feed's schema. A distinct pinned key id for feature-video manifests (verifier already parameterizes per entry point) would make that isolation structural rather than a KMS access-policy fact — reasonable as a fast follow with feat: sign feature-video release folders for the CDN #9964.

[DESIGN-REVIEWED] 0b091e1

@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 0b091e18b6711d8f3a260ed15309373c76be10a0 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All evidence gathered. Emitting the review.

First-Principles-Verdict: CONCERNS

The random-draw selection rewrite rides along — it deletes a deliberate ordered-walk pin on a preference whose zero option, by the diff's own words, costs nothing.

Not justified as shipped

  • Item 3 (which clip plays is now a uniform random draw) — rides along: hosting needs no selection change. The deleted pin read "selection walks the catalog in order and returns the FIRST entry that is eligible, so two requests one second apart cannot disagree"; the replacement concedes "Not because order starves a clip — it does not," and its benefit ("spreads first impressions… early feedback") names no reader of that feedback. Preference, not constraint.

What this change ships

Inventory (10 items) — 9 justified

Intent: ship feature-intro clips from a signed, per-release CDN manifest that the gateway downloads, verifies and serves locally, instead of bundling media in the wheel — ADDITION.

  1. Clips download in the background at boot into ~/.kiro/crew/feature-videos/<release>/, sha256-pinned by a signed manifest — justified
  2. A verified manifest replaces the bundled catalog; the launch before clips land shows nothing — justified
  3. Which eligible clip plays is now a uniform random draw, not first-in-catalog-order — rides along: preference-based rewrite of a deliberate pin
  4. /status gains download fields and POST /api/feature-videos/fetch-all — justified
  5. New serving route GET /feature-videos/<release>/<name> (SPA fallback excluded) — justified
  6. New config key dashboard.feature_videos_cache_max_mb with oldest-first whole-release eviction — justified
  7. New fail-closed governance scope capabilities.feature_videos_download, three chokepoints — justified
  8. New operator env var KIROCREW_FEATURE_VIDEOS_MANIFEST_URL — justified
  9. Feedback now accepts recently-offered ids so a verdict survives a manifest refresh — justified
  10. Shared transfer engine extracted from embeddings.py; three _make_ssl_context copies deleted; feed_trust core shared via verify_document_signature — justified

Watch

  • Item 3 above. Clears when: it ships separately with a named signal that ordered offering failed someone, or select_next keeps the ordered walk.
  • Two sibling private download loops remain after the extraction — papyrus/backend/tectonic.py::_download_to and pptx_maker/backend/engine_source.py::download_archive both keep their own opener/hash/install bodies (counted: 2; only make_ssl_context was consolidated). The module docstring defers them explicitly. Clears when: both transfers read through asset_downloader.download_to.

Subtractions

  • Delete the frontend's remote-clip branch: StartupVideoModal.tsx:114's source === 'remote'/probe path, client.ts featureVideoProbe + FeatureVideoProbe + FeatureVideo.source. After this PR they have zero producers — Offer.payload emits no source key and no /api/feature-videos/probe route exists anywhere in src/ (grepped: 0 matches) — and the new "no third, off-origin shape" invariant contradicts the code they keep alive.

[FIRST-PRINCIPLES-REVIEWED] 0b091e1

@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 0b091e18b6711d8f3a260ed15309373c76be10a0 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 0b091e1

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 0b091e18b6711d8f3a260ed15309373c76be10a0: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 0b091e18b6711d8f3a260ed15309373c76be10a0 — this comment is updated in place on each push.

Review details

I've reviewed the security-critical paths of this PR: the signature verification core (feed_trust._verify_signature / verify_document_signature), the verified download engine (asset_downloader.download_to with its symlink/hardlink/redirect/ceiling guards), the manifest parser and fetch walk (feature_videos_manifest), the cache serving route (resolve_served_path / _open_served, no-follow descriptor chain, suffix-pinned content types), the offer/eligibility logic, the governance scope, and the auth/route wiring.

The discovery pass produced no candidates. Working through the changed code myself, every place a real defect could hide — nested-payload canonicalization vs. the flat feed shape, key_id-optional trust, cross-host redirect handling gated on the env-only override, sha256-before-install ordering, TOCTOU on the release/staging directories, the _issued_ids bound keeping the state-file key set closed, _byte_range edges, the fail-closed governance chokepoints, and the data-home serving route being excluded from SPA fallback so an unauthenticated GET is refused rather than handed the shell — is guarded, and I could not derive a concrete input → call path → observable wrong outcome at the required bar for any of them.

No findings.

[OPUS-REVIEWED] 0b091e1

Verdict parsed from the review's SHA-scoped output markers for commit 0b091e18b6711d8f3a260ed15309373c76be10a0.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 0b091e18b6711d8f3a260ed15309373c76be10a0: <one-sentence reason>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 10, 2026
@iamwhatever
iamwhatever force-pushed the feat/feature-videos-hosted-backend branch from 047cbce to d4ca87d Compare September 10, 2026 23:15
@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: _open_staging_nofollow() — O_NOFOLLOW + lstat refusal

  • resumable staging follows agent-planted symlinks — fixed (span=6a49b6aee0db)

Fixed in d4ca87d. Staging is opened through _open_staging_nofollow(): O_NOFOLLOW so the kernel refuses a symlink at open time on POSIX, plus an is_symlink pre-check for Windows where the flag does not exist.
The class, not the site: EVERY write this module makes to a staging path now goes through that one opener, so a finding of the form "the staging path is plantable" is closed wherever the path comes from — the derived .part name, a caller-supplied staging=, resume and fresh alike.
One branch is closed differently and deliberately: a non-resuming caller unlinks the staging path first, and unlink removes the LINK, never its target — so the plant is destroyed rather than written through. Both branches are pinned (TestStagingSymlink, 2 tests).

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: _SameHostRedirectHandler + build_opener()

  • cross-host redirects permit blind SSRF — fixed (span=5047636a7afc)

Fixed in d4ca87d. asset_downloader.build_opener() is now the only way this package makes an outbound request, and it carries _SameHostRedirectHandler, which refuses any redirect that changes the host or leaves https.
The class, not the three sites: the finding named the manifest client, and the same defect was in the media transfer and the remote probe. All three call build_opener(), so they cannot differ on redirect policy — a fourth client added later inherits it or it has no opener at all.
A url here is authorized against ONE host (a signed manifest's cdn_base, or an https-only operator override), so any finding of the form "the request can leave the authorized host" is covered by this ruling.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: redact_url() drops the path

  • URL redaction preserves path credentials — fixed (span=6a49b6aee0db)

Fixed in d4ca87d. redact_url() returns scheme and host only. Userinfo and the query string were already dropped; the path was the component that looks safe and is not, since a private mirror can put a token in a path segment and this string reaches the gateway log on every transfer.
The class: no url component other than scheme+host reaches a log from this module. What a log reader needs instead is the caller's label (feature-video clip <id>), which is not attacker-controlled.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: per-entry download_denied() inside the pass

  • downloads continue after governance tightens — fixed (span=9b60a14221dc)

Fixed in d4ca87d. The pass re-evaluates the ceiling before every entry, and stops with download_state=denied the moment it denies.
The class is the read/write split, and it now holds both ways: an ACTION chokepoint (each clip download, fetch-all) takes the audited download_denied() and is never stale, while a POLLED route (/status, /next) takes download_permitted_cached() so a progress readout cannot write one governance_decision SEL row per poll. Any finding of the form "this outbound request runs under a permission the ceiling has withdrawn" is covered.
Pinned by a test that permits the pre-loop check and the first clip, then denies: the first clip lands, the rest do not.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: _issued_ids — bounded ledger of ids /next handed out

  • manifest refresh loses permanent verdicts — fixed (span=9c325bfaec09)

Fixed in d4ca87d. /next records the id it offers in a bounded ledger (_ISSUED_ID_LIMIT = 64, oldest evicted), and /feedback accepts an id that is either known or recently offered. Recording a verdict drops the id, since the row on disk supersedes the offer.
Why bounded rather than accepting any well-formed id: the id set is the ONE thing keeping the state file's key set closed, so widening it by "what was offered" keeps that property while widening it by "anything that parses" would not.
One residual, stated rather than papered over: a gateway restart between /next and the click loses the ledger, so that one verdict 400s and the clip is offered again. The alternative is persisting a second file for a promo flag, which costs more than the defect.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: cwd=directory on both openssl children

  • file-creating children inherit the repository CWD — fixed (span=0024792d7f7d)

Fixed in d4ca87d — cwd=directory on both openssl invocations.
Noting the adjudicator FLAGged this one as having no reachable trigger, and agreeing with that reading: both calls pass absolute -out paths and genpkey/pkey write only to -out. It is fixed anyway because the fix is two keyword arguments and the rule it serves (a test's children must not inherit the checkout) is worth holding unconditionally — cheaper than carrying the argument about reachability.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none — doc wording

  • doc claims one bad entry cannot be dropped — fixed (span=3757f9060477)

Fixed in d4ca87d. The doc now separates the two failures it had conflated: a bad SIGNATURE discards the whole document (the signature covers the document, so there is no verifying most of it), while a malformed ENTRY inside a correctly signed manifest is dropped alone with the reason logged.
That is what verified_manifest() and parse_manifest() actually do, and the asymmetry is deliberate: a bad entry costs one clip, a bad document would make every path derived from it wrong.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: max_bytes + DEFAULT_MAX_BYTES ceiling in download_to

  • the poster download passes no size, so an endless body fills the disk — fixed (span=df324d43c742)

Fixed in d4ca87d, and at the class rather than the call site. download_to() gained max_bytes (a BOUND, distinct from size, which is a declared length and also discards an over-long partial), and the effective ceiling is size or max_bytes or DEFAULT_MAX_BYTES — so no transfer runs unbounded whatever a caller forgets. The poster passes max_bytes=MAX_POSTER_BYTES (8 MiB).
The finding's reasoning is right and worth recording: bytes reach the staging file as they arrive, so an endless body wins the race against an end-of-stream digest. Any finding of the form "this transfer has no byte ceiling" is covered by the default.
Pinned by three tests: an endless body abandoned at an explicit ceiling, the same with no ceiling declared, and max_bytes not discarding a resumable partial the way size does.

@iamwhatever

iamwhatever commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — reuses feed_trust's pinned key

  • one key signs two trust domains of different ceremony — resolved by owner (span=7be4fa01611b)

Ruled by the maintainer: the shared key IS the intended policy for this work item. The CLI feed's offline key signs video manifests too, kept apart by a distinct schema inside the signed payload. No separate pinned key before the first publish.
The fixture arrangement is accepted as built: the public PEM and a signed manifest are committed under test/fixtures/feature-videos/, and the private half is minted per run by regenerate.py. A committed private PEM was the literal ask; it trips SAST p/secrets --error on the diff, so the generator stands in for it and the owner has confirmed that substitution.
What is true independent of the ruling: the two documents are not interchangeable. Each carries its own schema inside the signed payload and each consumer refuses the other's, pinned by a test in both directions.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: download_permitted_cached() — 60s TTL'd memo

  • every /status poll writes a SEL governance row — fixed (span=639a39881347)

Fixed in d4ca87d, taking the shape this item's Clears-when named. download_permitted_cached() re-evaluates at most once per 60s and otherwise serves the memo; /status and /next use it, and the action chokepoints (each clip download, fetch-all) keep the audited download_denied().
The item was right about the cost and right about the seam. An audited evaluation per poll writes a governance_decision row and a disk write per poll, and buries the rows that record a real decision — while the polled question is only "draw this control or not".
The cold path evaluates rather than serving the fail-closed default, so a fresh process reports the truth instead of claiming the feature is off. Pinned by three tests: 25 polls cost one evaluation, a TTL expiry costs a second, and a cold process evaluates.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: none — rewritten rationale, behaviour unchanged

  • the random draw's justification is not the property it claimed — fixed (span=26488f47a17e)

The critique of the JUSTIFICATION is correct and the rationale is rewritten in d4ca87d. A verdict is permanent and each launch retires the clip it showed, so a fixed order reaches the whole library too — "the rest would only be reachable by retiring the ones ahead" described normal operation, not starvation. That sentence is gone.
The BEHAVIOUR stays, on a different and defensible ground: publication order is the same for everyone, so a deterministic pick shows every install the same first clip and the newest entry is the last thing anybody sees. The draw spreads first impressions across the set, which is what makes early feedback on a new clip arrive at all. That is a property about the library, not about one install.
It is also a spec'd deliverable of this work item, so reverting it is a maintainer call — see the ruling on span 7d71a7648abf.

@iamwhatever

iamwhatever commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none

  • a deterministic pick keeps the deleted property at zero cost — rejected by owner (span=7d71a7648abf)

Ruled by the maintainer: the random draw stays. Picking one clip at random on start is a stated requirement of the work item this PR implements, so a deterministic order is a reviewer preference about which correct behaviour to ship, not a defect in the shipped one. select_next() keeps random.choice over the eligible set.
This ruling covers the class, not one call site. Any finding whose remedy is to replace the draw with a fixed order — first-eligible, newest-first, round-robin, a seeded sequence — is answered by it, wherever the draw moves in the file. A finding about the draw's correctness (an eligible clip it can never reach, a verdict it ignores, an unseeded source of nondeterminism in a test) is a different claim and is not covered.
The premise was granted and stays granted: determinism would buy a stable answer across two requests a second apart, and nothing about the draw prevents starvation — permanence already does that. The owner weighed the requirement above the stability.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none

  • keep_releases ships defaulted to a no-op and the env var is a second spelling — rebutted (span=0ac39a45844c)

Both knobs are named deliverables of this work item, not speculative additions: it asked for dashboard.feature_videos_keep_releases (default 0 = keep all) and for a KIROCREW_FEATURE_VIDEOS_MANIFEST_URL override by name. Dropping them would fail the spec, which is a maintainer call and not a reviewer one.
On the env var being a second spelling: it is the repo's existing pattern for exactly this pair of needs, not a new one. The model download has KIROCREW_EMBED_MODEL_URL (env) beside memory.embed_model_url (config) for the same reason — an air-gapped deployment sets an env var on the process, a managed install ships a config file, and both resolve in one function (manifest_url()), so there is one resolution order and no second code path.
This ruling covers the class: a config knob or env override this work item named is in scope for this PR, and a finding that it has one consumer today is answered by that rather than by removing it.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none

  • drop keep_releases and evict()'s keep_releases branch — rebutted (span=d7184d99326a)

Keeping it: dashboard.feature_videos_keep_releases is a named deliverable of this work item, so removing it fails the spec — see the ruling on span 0ac39a45844c for the class-level reasoning, which covers every finding of the form "this knob has one consumer and a no-op default".
On the substance, the two bounds are not redundant. The size budget answers "how much disk", keep_releases answers "how many releases", and an operator who wants only the running release's clips cannot express that with a byte budget — they would have to compute it from clip sizes they do not control. Both are bounded and both spare the running release; a test pins each independently.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none

  • drop KIROCREW_FEATURE_VIDEOS_MANIFEST_URL — rebutted (span=3259334d410b)

Keeping it, for the reason recorded on span 0ac39a45844c: the env override is a named deliverable of this work item, and it mirrors KIROCREW_EMBED_MODEL_URL beside memory.embed_model_url — the repo's existing answer to "an air-gapped process sets an env var, a managed install ships config".
It is one spelling of a value, not a second code path: both sources resolve in manifest_url() alone, in one documented order, and both are refused unless https. The signature is checked either way, so neither can introduce an unsigned clip.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — two private copies deleted

  • the PR's own root cause has two unfixed siblings — fixed (span=7d8a3d90bebf)

Fixed in d4ca87d. Both private _make_ssl_context bodies are gone: tectonic.py and engine_source.py now call asset_downloader.make_ssl_context(), and their duplicate _SSL_CA_PATHS tuples and _ssl_context_has_ca_trust imports went with them.
The item is right that leaving them contradicted this PR's own stated root cause, and both files were already in the diff, so the fix is subtraction rather than new surface: three copies of "find the system CA bundle" become one.
The full download_to unification is NOT done here and is not being deferred into an untracked promise: those two callers stream to different shapes (a Tectonic tarball, a pptx engine archive) with their own staging and verification, so folding them in is a separate change with its own review surface. What this PR owed was the shared helper it created, and that is paid.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — two private copies deleted

  • delete the two private _make_ssl_context bodies — fixed (span=123f4afa04f8)

Done in d4ca87d, exactly as written: both bodies deleted, both call sites switched to the now-public asset_downloader.make_ssl_context(), and the _SSL_CA_PATHS duplicates and now-unused ssl / _ssl_context_has_ca_trust imports removed with them.
Verified rather than assumed: test_papyrus_tectonic.py and test_pptx_maker_routes.py pass on the change, and flake8 confirms no orphaned import is left behind.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — stated in the description for the release notes; failure is self-describing

  • Watch — config-set redirecting model mirror now fails — called out for release notes (span=8d6caec566f3)

Same Watch as fe57e314e6bf / 5605c1ffa84e and their predecessors, same answer: the change and its remedy (set the mirror through KIROCREW_EMBED_MODEL_URL) are stated in the description, which is what this repository's release-time CHANGELOG is written from, and the failure is self-describing on first boot — RedirectRefused puts redirect from <host> to <host> refused: … honoured only when set through the environment override in the model download's status.error, shown by the dashboard progress panel.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: consumer list mirrored into docs/build/release.md and packaging/signing/README.md

  • Suggestion — mirror the key's consumer list into the release runbook — taken (span=8b0c06643413)

Taken in 564c13f: docs/build/release.md (the CLI-channel section, right after the pointer to the rotation procedure) now names the three consumers of the pinned key — cli.sh, the update-feed reader, the feature-video manifest — and the re-sign obligation for every hosted manifest a shipped release still reads back; packaging/signing/README.md's rotation paragraph names the third party in the overlap window. The description points at both.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Not justified — random draw replaces first-in-catalog order — owner ruling (rejected as a change) (span=e86203b866c3)

Unchanged, by ruling: the owner kept the draw as a stated requirement on span 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf). The deleted pins recorded the rule this requirement replaces; eligibility stays deterministic, the pick does not.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Watch — random draw has no report behind it — the owner confirmed it as the requirement (span=bca6a2896b30)

Clears by owner confirmation on 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf): the draw is the product requirement, not a fix for a report, and the description states it so.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — sibling consolidation is #10018

Not taken here, and already recorded: tectonic.py and engine_source.py are on #10018 with the other downloader siblings. Routing two document engines through asset_downloader.build_opener changes their redirect policy and needs their own tests; folding that into a feature-video PR would bury the review of both.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the cache root is opened no-follow and held, its descriptor's real path must equal the canonical root, and the release folder is created relative to that descriptor (mkdir/open with dir_fd; Windows: by name under pin_directory's handle)

  • Swapped cache root permits path traversal — fixed (flagged residual closed anyway) (span=5047636a7afc)

Fixed in d913f9a, taking the Fix line as written even though adjudication flagged the residual as an empty directory: create_and_open_dir_pinned resolves the root with realpath and a root swapped for a link after the by-name check was followed into creating <target>/<release> before the witness refused.
Now _create_release_dir_held(root, release) opens the ROOT first with no-follow semantics on its own name (pinned_fs.open_dir_pinned; a link there is CacheDirRefused), requires pinned_fs.fd_real_path(root_fd) to equal the canonical root, and only then creates and opens the release folder relative to that held descriptor (os.mkdir / os.open with dir_fd, O_NOFOLLOW on the leaf). Nothing is resolved by name once the root is held, so a swapped root is refused, not created under. Windows: pin_directory(root) (refuses a reparse point), same witness, then the leaf by name under that held handle, which blocks the root's rename. The leaf's own witness and fchmod follow as before.
Test: the root passes its by-name check and is then swapped for a link to a folder outside the cache — CacheDirRefused, and the outside folder stays empty with its mode untouched.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — needs a maintainer decision; owner has deferred it until the PR is otherwise green

  • Watch — one pinned key, three consumers, a signing oracle — needs-a-decision (deferred by owner) (span=038fc7ab87b3)

Same question as spans 70d4fd0c3b70 / 000ff498b759 / 3ca6f7d010cb on earlier heads. The owner has ruled on the timing: decide once everything else is green, not before. The facts stand — schema stops substitution, not signing; the third consumer is now named in docs/build/release.md and the signing README so a rotation finds it. The two outcomes stay as offered: an affirmation from the update-feed trust-root owner in the description, or a separate pinned key for the video document, whose backend half is a small follow-up.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — stated in the description for the release notes; failure is self-describing

  • Watch — config-set redirecting model mirror now fails — called out for release notes (span=dd7f47156d8f)

Same Watch as 8d6caec566f3 / fe57e314e6bf and their predecessors, same answer: the change and the env-override remedy are stated in the description, which is what this repository's release-time CHANGELOG is written from, and the failure names its own remedy in status.error on first boot.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Not justified — random draw replaces first-in-catalog order — owner ruling (rejected as a change) (span=06b5d9d0124b)

Unchanged, by ruling: the owner kept the draw as a stated requirement on span 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf). The deleted pins recorded the rule this requirement replaces; eligibility stays deterministic, the pick does not.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Watch — the draw's premise is asserted, not evidenced — the owner set it as the requirement (span=19cb15032a7d)

Clears by owner confirmation on 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf): the draw is the product requirement rather than a remedy for a measured harm, and the description states it as such. pool[0] was the shipped rule it replaces.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — frontend-only follow-up, out of this backend PR

  • Subtraction — delete the frontend's remote-clip path — deferred to the frontend follow-up (span=a1522bd251f4)

Not taken here, deliberately: this PR is the backend half and touches no website/ file. The /probe route and the remote src shape are gone on the server, and the frontend's optional source/download_enabled fields read as absent, which their types define as local and off — dead but harmless until the frontend PR removes featureVideoProbe and the remoteBlocked branch, reviewed against its own screenshots.

@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of d913f9a98ea31ddd759a7d6c73fb9d815ef48a09 and found no blocking issues.

This comment is updated in place on each push.

Review details

FINDING -- src/kiro_crew/feature_videos_cache.py:550 -- "cached, total = ..." skips post-download eviction, so newly downloaded clips can leave old releases above the configured cache budget until restart -> Fix: call _evict_now after the download loop.
[GPT-REVIEWED] d913f9a

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d913f9a98ea31ddd759a7d6c73fb9d815ef48a09: <one-sentence reason>

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: _evict_now runs again after the download loop, before the counts

  • Post-download eviction skipped, old releases can sit above the budget until restart — fixed (span=a92b323dd162)

Fixed in 8d35856. Correct: eviction ran once, before the loop, so the bytes the pass itself wrote never counted against the budget until the next boot. _evict_now now runs a second time after the loop (the current release is still never evicted). Test: the two eviction calls are observed with the current folder empty and then full.

Feature-intro clips were a static tuple compiled into the package, with the
media riding in the wheel. That is the right shape for two clips and the
wrong one for a library: every added video needed a release.

Clips now come from a manifest published per release folder on the CDN,
signed with the same offline key cli.sh pins for the update feed under a
distinct schema string, and cached in ~/.kiro/crew/feature-videos/<release>/
by a background pass at gateway boot. The static catalog stays as the
fallback for an install that never fetched a manifest.

Three properties make it safe to act on bytes from a CDN. The manifest is
verified and an unverified one is discarded WHOLE, because the signature
covers the document. Every clip is sha256-pinned by that signed manifest, so
a tampered object can only fail verification. And the one case where the
browser fetches media off this origin -- an eligible clip that is not cached
yet -- is pinned to the host the signed manifest declares.

Fetching any of it is governed by capabilities.feature_videos_download,
fail-closed, consulted at four chokepoints: the manifest fetch, the clip
download, the src handed to the browser, and POST /api/feature-videos/
fetch-all. Already-cached clips keep playing under a denial.

Selection now draws uniformly among eligible clips instead of taking the
first in catalog order. Order stopped being a statement about the user: with
a hosted library, catalog order is publication order, so the first entry
would always win and the rest would only be reachable by retiring the ones
ahead of it. Eligibility is unchanged and still deterministic.

The streamed-and-verified transfer itself moves out of embeddings.py into
asset_downloader.py, shared by both callers, with resume added: a partial
stays in a .part staging file nobody serves and the next boot continues it.
embeddings.py keeps everything that is about the model. Its too-small branch
also stops mis-reporting -- the staged size is now read before the unlink,
where the old inline copy read it after and degraded every undersized
download to a generic transport error.

New: GET /api/feature-videos/probe (server-side reachability, because the
modal's <video> is preload="none" and a client cannot discover a dead src
before the user's "Got it" records a permanent verdict), POST /api/feature-
videos/fetch-all, and GET /feature-videos/<release>/<file> -- a dynamic
route, never add_static and never a listing, validating both URL components
against the manifest's own rules and re-checking containment after resolve().

dashboard.feature_videos_enabled keeps its default of false.
@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: yes
mechanism: the serving route opens root → release → file as a chain of no-follow opens, each relative to the held descriptor before it; the witness compares the file's real path against the HELD root's real path, not a by-name resolve

  • Cache-root swap moves the serving boundary outside the cache — fixed (span=5047636a7afc)

Fixed in 0b091e1. Correct, and the adjudication traced it exactly: checked_cache_root() did _refuse_link(root) and then a separate by-name resolve(), so a root swapped for a link in that window resolved to the link's target, the resolver anchored <root>/<release>/<name> there, and _open_served compared the opened file's real path against that poisoned expectation — a match, so the outside file was served.
_open_served(release, name) no longer opens a path. It opens the cache ROOT on its own name with O_NOFOLLOW (pinned_fs.open_dir_pinned, ancestors pinned one openat at a time), the release folder relative to that descriptor (O_DIRECTORY | O_NOFOLLOW), and the file relative to the folder (O_NOFOLLOW | O_NONBLOCK); a root that became a link is refused by the open, not followed. The witness is now descriptor-to-descriptor: the file's fd_real_path must equal the held root's fd_real_path plus <release>/<name>, so nothing computed by name is trusted. st_nlink == 1 and the regular-file check stay. Windows: pin_directory on root and folder (refuses reparse points, blocks their rename), leaf via open_file_no_reparse under the held handles. The resolver keeps its by-name pre-checks for the flat 404 and the content type only; its canonical path is never re-walked.
Test: the root passes _refuse_link and is then, inside the resolve window, replaced by a link to an outside folder holding a same-named release and clip — 404, and the outside bytes are never read. The test fails on the previous head's code and passes on this one.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — needs a maintainer decision; owner has deferred it until the PR is otherwise green

  • Watch — shared update-feed key widens the update-trust perimeter — needs-a-decision (deferred by owner) (span=f1cbea5160b2)

Same question as spans 038fc7ab87b3 / 70d4fd0c3b70 / 000ff498b759 / 3ca6f7d010cb on earlier heads. The owner has ruled on the timing: decide once everything else is green. The facts stand — schema stops substitution, not signing; the third consumer is named in docs/build/release.md and the signing README. The two outcomes stay as offered: ratification by the signing-key owner in the description, or a separate pinned key for the video document (small backend follow-up).

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — stated in the description for the release notes; failure is self-describing

  • Watch — config-set redirecting model mirror now fails — called out for release notes (span=1f56965b4ca6)

Same Watch as dd7f47156d8f / 8d6caec566f3 and their predecessors, same answer: the change and the env-override remedy are stated in the description, which is what this repository's release-time CHANGELOG is written from, and the failure names its own remedy in status.error on first boot.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Not justified — random draw replaces first-in-catalog order — owner ruling (rejected as a change) (span=67675f6179a5)

Unchanged, by ruling: the owner kept the draw as a stated requirement on span 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf). Eligibility stays deterministic, the pick does not.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Watch — the draw rewrites a recorded decision — the owner confirmed it as the requirement (span=7d6fe167023c)

Clears by its first clause: the owner confirmed the random-draw decision on 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf), and the description states it as the requirement.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Subtraction — defer the random draw (3-line revert) — declined by owner ruling (span=da593ac3c8ea)

Declined on the same ruling (#9972 — round-0 disposition on span 7d71a7648abf): the draw is the requirement this PR ships, so the revert would drop a deliverable, not scaffolding. The _deterministic_pick seam exists so the eligibility tests stay exact under a random pick.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — the policy split is the point; regression + remedy stated in the description for the release notes

  • Watch — config-knob model url gets no cross-host redirect allowance — kept, stated for release notes (span=59c28b6cb3bd)

Kept, second clause: the regression is accepted and named in the description together with the remedy the error message states (KIROCREW_EMBED_MODEL_URL). The allowance is granted by WHO set the url, not by its scheme: the env value was set by whoever launches the gateway, the config value by anything that can edit config.json, the agent's own tools included. An agent that can point the initial GET anywhere is already the reason the config knob is recorded on #10018 as the same shape this PR removed for the manifest; widening its redirect policy to match the operator's would move in the other direction. Same class as 1f56965b4ca6 / dd7f47156d8f on earlier heads.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — needs a maintainer decision; owner has deferred it until the PR is otherwise green, which it now is

  • Suggestion — a distinct pinned key id for feature-video manifests — needs-a-decision, now due (span=f1dcba5c128b)

Same question as f1cbea5160b2 / 038fc7ab87b3 / 70d4fd0c3b70 / 000ff498b759 / 3ca6f7d010cb. The owner deferred it until everything else was green; this head is, so the question is being put to the owner now. Two outcomes: ratification of the shared key by the update-feed trust-root owner, stated in the description; or a separate pinned key for the video document kind, whose backend half is a small follow-up (the verifier already accepts a per-document key_id).

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Not justified — random draw replaces first-in-catalog order — owner ruling (rejected as a change) (span=fb90c6eafb52)

Unchanged, by ruling: the owner kept the draw as a stated requirement on span 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf). Eligibility stays deterministic, the pick does not.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — owner ruling stands

  • Watch — the draw should ship separately — the owner confirmed it as part of this work item (span=edcb83098298)

Clears by owner confirmation on 7d71a7648abf (#9972 — round-0 disposition on span 7d71a7648abf): the draw is a requirement of this work item, stated in the description.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — sibling consolidation is #10018

Acknowledged and recorded: tectonic.py::_download_to and engine_source.py::download_archive are on #10018 with the other downloader siblings. Routing two document engines through asset_downloader.download_to changes their redirect and staging behaviour and needs their own tests; folding that into a feature-video PR would bury the review of both.

@iamwhatever

Copy link
Copy Markdown
Contributor Author

self-added: no
mechanism: none — frontend-only follow-up, out of this backend PR

  • Subtraction — delete the frontend's remote-clip branch — deferred to the frontend follow-up (span=19dc02628b4e)

Not taken here, deliberately: this PR is the backend half and touches no website/ file. The remote src shape and /probe route are gone on the server, and the frontend's optional source/download_enabled fields read as absent, which their types define as local and off — dead but harmless until the frontend PR removes featureVideoProbe and the source === 'remote' branch, reviewed against its own screenshots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants