Repository navigation
feat(app-kit): add a batch embed to ctx.embed - #36450
bolichen97 wants to merge 1 commit into
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: the change only adds members to the existing Notes
[DESIGN-REVIEWED] 2789d03 Description read[DESCRIPTION-READ] 2710875a09061b12b2a234f73617bf2bbb77f716aa1e26518c31f49d9a866877 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. Only the Notes
What this change shipsInventory (7 items) — 6 justifiedIntent: let App Kit apps batch-embed texts with the gateway's shared model and learn the vector space and readiness, without the private API or a second model copy. ADDITION.
[FIRST-PRINCIPLES-REVIEWED] 2789d03 Description read[DESCRIPTION-READ] 6afd87956a00970a73783c82d61c832c87d46049e1cd8b9f721cf18d5c9ba11b That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
7242cb2 to
61f2648
Compare
|
Rebased: main moved in docs/app-kit/api-reference.md, docs/system-specs/modules/app-kit-platform.md, src/kiro_crew/apps/app_sdk_level.py and website/src/app-sdk/sdkLevel.ts since this head was judged (af5ca42 -> 187b4da); CI re-running. Main's #33589 took App Kit seam level 7 for [operator: bolichen97] |
Code change approver recommendation: A person should decideWhy: the decider estimates a 12% chance that this change introduces a defect, within the 12% limit for approving on its own, but it changes 421 lines, more than the 400 it may approve on its own. Additive change: EmbedSDK gains embed_batch and embed_batch_sync. They return an EmbedBatchResult carrying model_id, dim, space_signature and a reason. They call the backend's existing embed_batch at PRIORITY_BULK and run on mc-membg under an admission of their own with one slot, plus one added pool thread, so consolidation and the backfill and repair loops never wait on app batches. The backend signatures (llama and Ollama) and their None semantics match how the PR handles them. The seam level rises to 8 on both sides, kept equal by the mirror test, and no reader hard-codes 7. Specs are updated in the same commit. Tests cover each reason and the pool placement, and would fail without the change. No regression on the existing members. The CI red looks like an unrelated timing flake, and the merge conflict is a process item. An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
61f2648 to
9a26e77
Compare
|
Rebased: main moved in docs/app-kit/api-reference.md and docs/system-specs/modules/app-kit-platform.md since this head's CI ran (187b4da -> c000f31); CI re-running. Main's #36341 (the backend SDK) and #36338 (the desktop window keys) both joined App Kit seam level 7, which made this PR conflict in those two docs. Level 7 now lists all of main's seams, and the batch members stay seam level 8. The new base also carries #36504, the fix for the docs-lint red on main that failed [operator: bolichen97] |
|
[operator: bolichen97] The only red on this head (9a26e77) is This PR does not touch the deny gate. The same test fails its bound on many unrelated heads, and shard 7 is also red on main's latest completed runs. The fix for that test is open in #28293, so this PR waits for it to land rather than re-running the shard. Every review lane passes on this head. |
An app that groups or deduplicates its records needs many vectors at once, the identity of the space they live in, and a clear answer when the model cannot serve yet. ctx.embed only offered one text at a time at normal priority, which puts an app's corpus loop beside the host's own memory writes. ctx.embed gains embed_batch and embed_batch_sync: up to 64 texts per call at the shared model's bulk priority, an EmbedBatchResult with the vectors, model id, width and space signature, a reason when it carries none (not_ready whenever the backend has no embedding available), and an optional instruction prefix. The async form runs on the background memory pool, because paced bulk work would hold an embed pool worker that turn context waits on for minutes, under a one-slot admission and a thread of its own, so consolidation never waits behind an app's queue of batches. No manifest permission and no route, like the rest of the seam. The batch members join the open App Kit seam level 10, so an app feature-detects them, never the level alone. Closes #23174 Co-authored-by: Jacob Largent <65189184+JLargent42@users.noreply.github.com>
|
[operator: bolichen97] Rebased onto current main to resolve the merge conflict in |
9a26e77 to
2789d03
Compare
Problem / Motivation
Goal: Let App Kit apps embed many texts in one call with the gateway's shared embedding model, and get back the vector space's identity and a clear "not ready" answer.
ctx.embedembeds one text at a time and reports onlymodel_id. An app that groups, deduplicates or shortlists its records has no batch call, no width or space signature to tell when its stored vectors went stale, no way to tell "the model is still downloading" from "the embed failed", and no instruction prefix for the instruction-tuned bundled model. Today it either imports the privatekiro_crew.embeddingsAPI or loads a second ~700 MB copy of the model.Why it matters
Apps that need embeddings either lean on a private API that breaks on a wheel upgrade, ship their own model (a second resident copy, in a different vector space), or skip embeddings. Serving batches from the one shared model removes both, and running them behind the host's own memory work keeps an app's corpus loop from slowing memory recall.
Not a goal
ctx, and serving them needs a local route, which is a separate request.ctx.embedmembers (available,model_id,embed_sync,run).What changed (motivation → approach → change)
An app can now embed up to 64 texts in one call with the gateway's own model, learn which vector space the vectors live in, and hear "not ready" instead of guessing while the model downloads. Approach: extend the existing
ctx.embedseam rather than add a route, because in-gateway app code already holds the context. Change: two new members onEmbedSDK(src/kiro_crew/apps/embed_sdk.py).await ctx.embed.embed_batch(texts, *, instruction=None)mc-membg), one app batch at a time under an admission and a thread of its own (_APP_EMBED_BATCH_WORKERSinsrc/kiro_crew/executors.py), so a long batch never holds a worker that chat turns or the host's memory writes wait onctx.embed.embed_batch_sync(...)EmbedBatchResultready, one vector per text in input order,model_id,dim,space_signaturereasonwhen not readyinvalid_input,too_many_texts,on_event_loop,not_ready(model downloading, loading, swapping, absent, or the bulk queue full),failedinstructionInstruct: <instruction>\nQuery:Nothing raises into the app. The batch members join the open App Kit seam level 10 (
apps/app_sdk_level.py), which they share with the agent tool gate's fence on an installed app'scredentialMasks, withpermissions.chatLaunchand withpermissions.browserSites, soAPP_SDK_LEVELstays 10 and an app feature-detects the batch members onctx.embed, never the level alone. The app guide (docs/app-kit/api-reference.md), the App Kit spec and the memory spec's pool section (memory-skills-hooks.md) describe the call.Builds on #31077 by @JLargent42.
Backwards compatibility
Compatible: the change only adds
EmbedSDKmembers; every existing member,EmbedSDK()with no arguments andAPP_SDK_LEVELbehave as before.Removes nothing: no path, key, function or validator is removed or tightened.
Tests
test/test_embed_sdk.pyadds cases for: vectors with model id, width and space signature; bulk priority behind memory recall; any app context batch-embeds without a grant; the 64-text cap; malformed texts and instruction; the instruction prefix;not_readyfor an absent, cold or unconstructible backend;failedfor a raise or malformed vectors, never raising; the blocking form refusing on the event loop; the async form running onmc-membgwhile consolidation's two admission slots are both held.Fail-before:
test_async_batch_embed_runs_on_the_background_memory_poolfails withTimeoutErrorwhenembed_batchgoes back to consolidation's admission (run_in_memory_background_pool), and passes with this code (an Ubuntu Linux host run on baseaf5ca42876, with thisembed_sdk.py,executors.pyand test file byte for byte).Determinism:
test/test_embed_sdk.pypassed 20 runs in a row at-n0and a shuffled run (seed 206958309).test/test_flake_pattern_ratchet.py71 passed; semgrep 1.78.0 on the diff, 0 findings.Gates, run on an Ubuntu Linux host on this patch's code and test files on base
09e5482863: black, isort, flake8,mypy src/kiro_crew/, the comment-history, brand, harness-parity, feature-map, file-line-cap, sync-io, loop-bound-lock, docs-lint and memory-store-seam checks all exit 0. The related tests (test_embed_sdk.py,test_app_sdk_level_mirror.py,test_app_backend_sdk.py, every test file that mentionssdk_level,APP_SDK_LEVEL, Bot grants or app memory providers, the App Kit manifest, credential-mask and adoption-pin tests, the embedding preemption tests, the memory kind-routing, origin-gate and tier-policy tests, and every test file that drivesmemory_background) pass: 1,524 passed in 33 files.scripts/local-gate.pyran its three backend batches and its frontend batch: 68,636 backend tests passed and 218 skipped, and 15,731 frontend tests passed in 530 files. Its 5 backend reds (test_find_ui_auto,test_host_git_repo_exec_neutralization, and three cases oftest_kas_orchestrator_memory.py::TestTheRealSessionNewRequest::test_an_unconfirmed_resume_is_evicted_without_deleting_its_record) fail the same way alone on a clean checkout of the base on that host. On this head, whoseembed_sdk.py,executors.pyandtest/test_embed_sdk.pyare byte for byte the ones that host ran, black, isort and flake8 on the changed Python files and the light repository checks (comment-history, brand, harness-parity, feature-map, docs-lint, memory-store-seam and the rest) exit 0, and the seam-level assertions oftest_app_sdk_level_mirror.py,test_app_backend_sdk.pyand the shipped adoption-pin level test, re-run as a script, hold at level 10.Manual verification
Smoke test on an Ubuntu Linux host, on this patch's code on base
09e5482863, against a real isolated gateway (throwawayKIROCREW_HOMEandKIRO_HOME, loopback port, owner session) with a throwaway user app whose routes callawait ctx.embed.embed_batch(...). The gateway downloaded the pinned model itself (its sha256 matches the pin) and served it in-process:reason: too_many_texts,ctx.sdk_level10POST /api/memory/enable-embeddings, 3 textsreason: not_readyready: true, 3 vectors,dim1024,model_idqwen3-embedding:0.6b,space_signatureefb9254d4de91136,ctx.sdk_level10asyncio.gatherof three 3-text callsinstruction; 0 textsreason: invalid_inputThe model finished loading inside the enable call, so this run did not catch the cold-load
not_readywindow;test_batch_embed_reports_not_readycovers it.ctx.sdk_levelis 10 on this head too, where the batch members share main's level 10.Screenshots / video
N/A — the diff changes no frontend file, so nothing on screen changes.
Related Issues
Closes #23174
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)