Skip to content

fix: keep a running model download's status when a custom model is set - #35634

Merged
bolichen97 merged 1 commit into
mainfrom
fix/34429-embed-status-mid-download
Nov 12, 2026
Merged

bolichen97 merged 1 commit into
mainfrom
fix/34429-embed-status-mid-download

Conversation

@bolichen97

@bolichen97 bolichen97 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: A bundled embedding-model download that is already running keeps reporting its own progress when a custom embedding model is configured while it runs.

ModelDownloadManager.ensure_model lets concurrent callers share one in-flight download, and that download owns the manager's status while it holds the lock. The custom-model check breaks this: when memory.embed_model_path is set while the bundled model is still downloading, a second ensure_model call writes custom_model with attempt: 0 over the running download's status. The embedding status then reads download_step: custom_model; the next progress update restores the step but keeps attempt: 0 for the rest of that attempt, and during a retry backoff nothing restores it until the next attempt starts.

Why it matters

The embedding-status endpoint reports this status as download_step and download_attempt, and the Enable handler reads it to decide whether a download is already in flight. The Memory card does not show the wrong step, because with a custom model configured it takes its step from the custom file instead. Today the one caller that can reach the overwrite is the dashboard's Enable/Retry, and only when the configured custom file is missing or unreadable (otherwise it never calls ensure_model) and the click lands between a download taking the lock and writing its first step (otherwise it adopts the running download); the download then reports attempt: 0 for the rest of its first attempt. ensure_model is what promises concurrent callers one shared download, so the status guarantee belongs there rather than in each caller's own check. Low severity: the download itself is not affected.

Not a goal

What changed (motivation → approach → change)

A running bundled download now keeps reporting its own step and attempt when a custom embedding model is configured while it runs.

The overwrite came from the custom-model check in ModelDownloadManager.ensure_model (embeddings_download.py): it wrote status before taking the download lock, while the running download writes that same status under the lock. The check now writes custom_model only when no download holds the lock. It still returns False and never starts the bundled download. The memory-skills-hooks spec states the rule.

Backwards compatibility

Compatible: the return value is unchanged, and so is every status written when no download is running; only an overwrite of a running download's status is dropped.

Removes nothing: no key, record, path or function goes away, and the custom_model step is still written whenever no download holds the lock.

Tests

  • test_refusal_leaves_a_running_download_status_alone (test/test_custom_embedding_model.py): a bundled download blocked mid-transfer holds the lock with step: downloading; a second ensure_model after a custom model is configured returns False, the step stays downloading, and the download then finishes ready.
  • The existing test_interactive_download_refuses still pins custom_model when no download is running.

Proof: PROVEN test/test_custom_embedding_model.py::TestDownloadSuppression::test_refusal_leaves_a_running_download_status_alone (head 53bfe8f)

Determinism, on Linux at this head:

Run Result
the new test, 20 repeats, -n0 0 failures
test/test_custom_embedding_model.py and test/test_embeddings_download.py, 10 repeats, -n 4 --dist loadgroup --max-worker-restart=2 0 failures (59 passed each)
every test in both files (59) shuffled, seed 34429, -n0 59 passed

Manual verification

N/A — unit coverage sufficient: the overwrite is one in-process status write, and the test reproduces it with a real download thread held mid-transfer.

Screenshots / video

N/A — backend-only change; no dashboard file is touched.

Related Issues

Closes #34429

Sibling sweep

Swept: git grep -nE "status = {" -- src/kiro_crew/embeddings_download.py
Sibling: src/kiro_crew/embeddings_download.py:184 -- changed here
Sibling: src/kiro_crew/stt/models.py:964 -- not affected -- #34835 (closing #34431) fixed it on main, which this branch carries
Sibling: src/kiro_crew/stt/decoder.py:412 -- not affected -- its pre-lock write fires only when KIROCREW_SKIP_MODEL_DOWNLOAD=1, and then no fetch ever starts

The other two pre-lock writes in embeddings_download.py, ready at lines 173 and 387, are not in the class: each fires only when the bundled file is already installed, the step a finished download reports.

Pattern harvest

Rule candidate: review-prompt
Pattern: "a caller writes a status field that a lock-holding task owns, before taking that lock"

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

@bolichen97
bolichen97 requested review from a team and KiroCrewCaptain as code owners November 11, 2026 22:36
@bolichen97
bolichen97 requested a review from buluoray November 11, 2026 22:36
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision 53bfe8fc57fe7aac7e3fc9d75088b661124385da touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of 53bfe8fc57fe7aac7e3fc9d75088b661124385da — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. This is a one-condition fix at the write that caused the overwrite. The linked issue and a new test pin the defect, and the spec line is the mandated same-commit update.

What this change ships

Inventory (3 items) — 3 justified

Intent: keep a running bundled embedding-model download reporting its own step and attempt when someone configures a custom model mid-download. This is a FIX.

  1. A running download's status is no longer replaced by custom_model/attempt: 0 when a second Enable/Retry call finds a custom model configured. — justified
  2. The memory-skills-hooks spec now says when custom_model is written and when it is not. — justified
  3. A new test holds a real download mid-transfer, then checks that the status stays downloading and finishes ready. — justified

[FIRST-PRINCIPLES-REVIEWED] 53bfe8f

Description read

[DESCRIPTION-READ] cf7b7404571fc9fe1071b0a276b792760f8ab71c91df73c56513597f1c29c104

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of 53bfe8fc57fe7aac7e3fc9d75088b661124385da — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the guard sits in ensure_model, the code that owns the shared-download promise. It changes only what happens while a download holds the lock.

[DESIGN-REVIEWED] 53bfe8f

Description read

[DESCRIPTION-READ] 7fe47107c1673580abc87fa0576a92b47584a8c8a95af98103067a9e0f6c851c

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed 53bfe8fc57fe7aac7e3fc9d75088b661124385da — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 53bfe8f

Verdict parsed from the review's SHA-scoped output markers for commit 53bfe8fc57fe7aac7e3fc9d75088b661124385da.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 53bfe8fc57fe7aac7e3fc9d75088b661124385da false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of 53bfe8fc57fe7aac7e3fc9d75088b661124385da and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 53bfe8f

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 53bfe8fc57fe7aac7e3fc9d75088b661124385da false-positive: <reason, 20 to 500 characters>

@bolichen97
bolichen97 enabled auto-merge (squash) November 11, 2026 22:47

@buluoray buluoray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve on 256684e. When a custom embedding model is configured while the bundled model is still downloading, a second setup call no longer overwrites the running download's step and attempt. The check and the write run on the event loop with no await between them, so the download thread cannot slip in between. The return value and the custom-model step written when no download runs are unchanged. The new test holds a real download mid-transfer, turns red when the guard is removed, and waits on events rather than sleeping. No security surface is touched. Decisive lines: src/kiro_crew/embeddings_download.py:183-184, test/test_custom_embedding_model.py:479-507

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 11, 2026
ModelDownloadManager.ensure_model wrote step custom_model before it
took the download lock, so a call that found a custom embedding model
configured while the bundled model was still downloading replaced that
download's status with custom_model until the download wrote it again,
and the attempt number the download carries forward was lost.

The custom-model branch still returns False, but it writes its status
only when no download holds the lock; a running download keeps the
status it owns until it ends.

Closes #34429
@bolichen97
bolichen97 force-pushed the fix/34429-embed-status-mid-download branch from 256684e to 53bfe8f Compare November 12, 2026 08:29
@bolichen97

Copy link
Copy Markdown
Contributor Author

Rebased: main moved in docs/system-specs/modules/memory-skills-hooks.md (plus 288 files in the directories this PR touches) since this head's CI ran (f9788c5 -> e47278c); CI re-running.

The four red backend shards on 256684e failed two tests that were red on main itself, test/test_wait_discipline_ratchet.py::test_the_scan_matches_known_sites and test/test_updates_handlers_facade.py::test_every_name_served_from_outside_the_family_is_visible_to_type_checkers; #35625 fixed both on main, and this base carries it. Both pass locally on 53bfe8f, and the patch itself is unchanged (git range-diff shows =). The failed CodeQL Analyze (python) run was a rate-limited upload, not a finding.

[operator: bolichen97]

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 12, 2026
@bolichen97
bolichen97 merged commit 962da06 into main Nov 12, 2026
97 checks passed
@bolichen97
bolichen97 deleted the fix/34429-embed-status-mid-download branch November 12, 2026 09:35
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Embedding model status shows custom_model while the bundled model is still downloading

2 participants