Repository navigation
feat(app-kit): add a gated batch embed to ctx.embed - #31077
JLargent42 wants to merge 1 commit into
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS Nothing blocks. The title still says "gated" while the diff and description ship an ungated call, and Notes
What this change shipsInventory (6 items) — 4 justifiedIntent: let App Kit apps batch-embed text with the shared model and learn the vector-space identity and a not-ready state, without a private import (#23174). ADDITION.
[FIRST-PRINCIPLES-REVIEWED] f0b1db8 Description read[DESCRIPTION-READ] 1ba4bbd7e2c5a1be5b179760ee6b0c6d96c9772efdc6af87b8d80dee224d2ba9 That sha256 names the PR description this verdict read. Recompute it with |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS No design blocker: the change adds to the existing Notes
[DESIGN-REVIEWED] f0b1db8 Description read[DESCRIPTION-READ] 97ebb78f908cdf8adad2c08ad36f02a21546fbcb2d13092df7bc73f2c5d662bb That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — 🔴 changes requested (blocking)GPT 6.1 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- src/kiro_crew/dashboard/agent_ipc.py:241 -- A spent pickup becomes reusable after its holder exits
[BLOCK-MERGE] 7a02370 Adjudication (Opus 5.5) — is blocking on each finding proportionate?F1 (fenced). This is an unbounded, security-class harm: it bypasses the MCP tool-exclusion ceiling.
False positive or not applicable? A repository writer can comment: |
|
Re GPT 6.1 blocking finding (embed_sdk.py:250, batch permission decisions bypass SEL): fixed in 7a02370. |
7a02370 to
f0b1db8
Compare
|
Updated in f0b1db8 (PR folded to one commit):
|
|
[operator: bolichen97] Continuing this in #36450; thanks for the start. |
|
The KiroCrew conductor (largjaco's v2 drive-to-green session) now owns this PR under the 24h orphan rule; the desk session that opened it has been offline since 11-06. I'll rebase onto main, fix CI and answer review here. |
Code change approver recommendation: A person should decideWhy: the decider raised 2 possible problems and estimates a 55% chance that this change introduces a defect; a person should check them. The adapter is correct and well tested for single calls. The problem is the bulk-priority design: the async path puts throttled, deadline-free bulk inference on the shared 8-worker mc-embed pool that every chat turn uses to build its prompt, and app bulk jobs crowd the shared inference queue's non-interactive slots. A few concurrent app batches can therefore stall chat turns and get host memory writes refused, which contradicts the PR's fair-use claim and repeats the starvation incident documented in executors.py. No test covers concurrency. Requesting changes, for example a dedicated pool or concurrency limit for app batches and a queue-busy answer mapped to not_ready. Things to check (possible problems the decider raised; at this confidence many turn out to be fine):
An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit |
|
The KiroCrew conductor is yielding this PR to #36450, which continues it; no further pushes from here. Anything in this PR that #36450 lacks (tests, edge cases) is listed below. Gaps found by comparing the two diffs: none found. #36450 has the same reasons ( |
Problem / Motivation
Goal: Let App Kit apps batch-embed text with the gateway's shared embedding model and get the vector-space identity and a clear not-ready answer back.
ctx.embedtoday offers only a single-textembed_syncandmodel_id. An app that groups, deduplicates or shortlists text (for example clustering recurring tickets) has no batch call, no width or space signature to tell when its stored vectors went stale, no way to tell "model still downloading" from "embed failed", and no instruction prefix for the instruction-tuned bundled model. The workaround is importing the privatekiro_crew.embeddingsAPI or loading a second ~700 MB copy of the model.Why it matters
Apps that need embeddings either depend on a private API that breaks on a wheel upgrade, ship their own model (a second resident copy, in a different vector space), or skip embeddings. Serving batches from the one shared model avoids both, and fair-use limits keep app work from delaying the host's own memory recall.
Not a goal
ctx.embedmembers (available,model_id,embed_sync,run).embed_syncloads the same model at normal priority; fair use comes from bulk priority and the batch cap instead.What changed (motivation → approach → change)
Goal → extend the existing
EmbedSDKseam rather than add a route, since in-gateway app code already holds the context → new batch members:EmbedSDK.embed_batch_sync(texts, instruction=None)and its pooledawait embed_batch(...)(src/kiro_crew/apps/embed_sdk.py). They return anEmbedBatchResult:ready,vectors(one per text, in order),model_id,dim,space_signature(embeddings_shared.embedding_space_signature), and areasonwhen not ready:invalid_input,too_many_texts,on_event_loop,not_ready,failed. Never raises into the app.embed_batchruns atPRIORITY_BULK, behind memory recall (PRIORITY_INTERACTIVE) and memory writes; at most 64 texts per call; each text cut at_MAX_EMBED_CHARS. A cold model answers bulk work withNonewhile the load it kicked runs, which reads asnot_readyso the app can fall back and retry.Instruct: <instruction>\nQuery:.EmbedSDK(andScrubSDK): in-gateway app code could import the embedder anyway, and the single-text members already load the same model.docs/app-kit/api-reference.md,docs/system-specs/modules/app-kit-platform.md.Backwards compatibility
Compatible: only adds new
EmbedSDKmembers;EmbedSDK()with no arguments and every existing member behave as before.Removes nothing: no existing path, key, function or validator is removed or tightened.
Tests
test/test_embed_sdk.pyadds 18 cases: vectors with model id, width and space signature; bulk priority behind recall; any app context (no manifest grant) can batch-embed; the 64-text cap; malformed texts and instruction; the instruction prefix;not_readyfor an absent, cold or unconstructible backend;failedfor raised,None, short or ragged results; refusal on the event loop; the async form running on the embed pool.Proof: PROVEN the new tests in test/test_embed_sdk.py fail on the previous gated head 7a02370 (18 failed, 8 passed) and pass with the change (26 passed).
Local gate after removing the grant:
scripts/local-gate.pygreen (236 passed, 3 skipped; frontend untouched). black, subprocess-encoding, isort, flake8, mypy, comment-history and brand gates green.Earlier Local gate (
scripts/local-gate.pyplan, related backend tests 1529 files and frontend 888 files, run batch by batch after rebasing on5e431e9ed0): every batch green except host-environment reds, each proven to fail identically (same test ids) at the merge-base5e431e9ed0in a separate worktree:test/metrics/test_otlp_wire_e2e.py::test_every_declared_instrument_reaches_the_exporter,test/test_crew_log_emit_redaction_and_flag.py(2),test/test_crew_log_lease.py(2),test/test_design_tweak_devproc.py(3),test/test_display_battery_rules_version.py::test_the_battery_executes_only_rule_sources,file_explorertest_server (40),test_auto_improvement_members(1),test_enterprise_policy_fetch(52),test_governance_distribution_enterprise_source(6),test_kiro_prerequisite_login_probe(9),test_managed_launcher_restart(6),test_mcp_slim_entry(13). Frontend related batches green (890 files, 22655 tests). black, subprocess-encoding, isort, flake8, mypy, comment-history and brand gates green.Manual verification
N/A — unit coverage sufficient: the SDK is a thin adapter over the shared backend, exercised with fakes for every result branch; no route or UI changed.
Related Issues
Closes #23174
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
Contributed under the project's Apache-2.0 license.