Skip to content

feat(app-kit): add a gated batch embed to ctx.embed - #31077

Open
JLargent42 wants to merge 1 commit into
mainfrom
feat/app-embed-batch-23174
Open

JLargent42 wants to merge 1 commit into
mainfrom
feat/app-embed-batch-23174

Conversation

@JLargent42

@JLargent42 JLargent42 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Problem / Motivation

Goal: Let App Kit apps batch-embed text with the gateway's shared embedding model and get the vector-space identity and a clear not-ready answer back.

ctx.embed today offers only a single-text embed_sync and model_id. An app that groups, deduplicates or shortlists text (for example clustering recurring tickets) has no batch call, no width or space signature to tell when its stored vectors went stale, no way to tell "model still downloading" from "embed failed", and no instruction prefix for the instruction-tuned bundled model. The workaround is importing the private kiro_crew.embeddings API or loading a second ~700 MB copy of the model.

Why it matters

Apps that need embeddings either depend on a private API that breaks on a wheel upgrade, ship their own model (a second resident copy, in a different vector space), or skip embeddings. Serving batches from the one shared model avoids both, and fair-use limits keep app work from delaying the host's own memory recall.

Not a goal

  • No new HTTP route and no new listener. An app that wants to serve vectors over HTTP does it from its own routes, behind the existing app auth.
  • No change to the single-text ctx.embed members (available, model_id, embed_sync, run).
  • No manifest permission. A grant could only gate the batch path, which yields to the host's memory work, while the ungated embed_sync loads the same model at normal priority; fair use comes from bulk priority and the batch cap instead.

What changed (motivation → approach → change)

Goal → extend the existing EmbedSDK seam rather than add a route, since in-gateway app code already holds the context → new batch members:

  • EmbedSDK.embed_batch_sync(texts, instruction=None) and its pooled await embed_batch(...) (src/kiro_crew/apps/embed_sdk.py). They return an EmbedBatchResult: ready, vectors (one per text, in order), model_id, dim, space_signature (embeddings_shared.embedding_space_signature), and a reason when not ready: invalid_input, too_many_texts, on_event_loop, not_ready, failed. Never raises into the app.
  • Fair use: the shared backend's embed_batch runs at PRIORITY_BULK, behind memory recall (PRIORITY_INTERACTIVE) and memory writes; at most 64 texts per call; each text cut at _MAX_EMBED_CHARS. A cold model answers bulk work with None while the load it kicked runs, which reads as not_ready so the app can fall back and retry.
  • Optional instruction (≤ 512 chars) prefixed as Instruct: <instruction>\nQuery:.
  • Ungated like the rest of EmbedSDK (and ScrubSDK): in-gateway app code could import the embedder anyway, and the single-text members already load the same model.
  • Docs: docs/app-kit/api-reference.md, docs/system-specs/modules/app-kit-platform.md.

Backwards compatibility

Compatible: only adds new EmbedSDK members; EmbedSDK() with no arguments and every existing member behave as before.

Removes nothing: no existing path, key, function or validator is removed or tightened.

Tests

test/test_embed_sdk.py adds 18 cases: vectors with model id, width and space signature; bulk priority behind recall; any app context (no manifest grant) can batch-embed; the 64-text cap; malformed texts and instruction; the instruction prefix; not_ready for an absent, cold or unconstructible backend; failed for raised, None, short or ragged results; refusal on the event loop; the async form running on the embed pool.

Proof: PROVEN the new tests in test/test_embed_sdk.py fail on the previous gated head 7a02370 (18 failed, 8 passed) and pass with the change (26 passed).

Local gate after removing the grant: scripts/local-gate.py green (236 passed, 3 skipped; frontend untouched). black, subprocess-encoding, isort, flake8, mypy, comment-history and brand gates green.

Earlier Local gate (scripts/local-gate.py plan, related backend tests 1529 files and frontend 888 files, run batch by batch after rebasing on 5e431e9ed0): every batch green except host-environment reds, each proven to fail identically (same test ids) at the merge-base 5e431e9ed0 in a separate worktree: test/metrics/test_otlp_wire_e2e.py::test_every_declared_instrument_reaches_the_exporter, test/test_crew_log_emit_redaction_and_flag.py (2), test/test_crew_log_lease.py (2), test/test_design_tweak_devproc.py (3), test/test_display_battery_rules_version.py::test_the_battery_executes_only_rule_sources, file_explorer test_server (40), test_auto_improvement_members (1), test_enterprise_policy_fetch (52), test_governance_distribution_enterprise_source (6), test_kiro_prerequisite_login_probe (9), test_managed_launcher_restart (6), test_mcp_slim_entry (13). Frontend related batches green (890 files, 22655 tests). black, subprocess-encoding, isort, flake8, mypy, comment-history and brand gates green.

Manual verification

N/A — unit coverage sufficient: the SDK is a thin adapter over the shared backend, exercised with fakes for every result branch; no route or UI changed.

Related Issues

Closes #23174

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributed under the project's Apache-2.0 license.

@JLargent42
JLargent42 requested review from a team and KiroCrewCaptain as code owners November 6, 2026 04:39
@JLargent42
JLargent42 requested a review from Zedmor November 6, 2026 04:39
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

UX Review (Opus 5.5) — ⏭️ skipped

Revision f0b1db8e950be2a465a437ff55c6cb62d0ee8a1f touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of f0b1db8e950be2a465a437ff55c6cb62d0ee8a1f — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. The title still says "gated" while the diff and description ship an ungated call, and to_dict() has no derived need.

Notes

  • Item 6 — undeclared: the title "add a gated batch embed" contradicts the description's "Ungated like the rest of EmbedSDK" and the test assert not hasattr(Permissions(), "embeddings"). Subtraction: delete "gated" from the title so it matches the commit subject, "add an ungated batch embed".
  • Item 5 — zero consumers: EmbedBatchResult.to_dict() has 0 callers outside its test (grep EmbedBatchResult|to_dict in apps/), and its only support is "for an app route to return as-is". An app can call dataclasses.asdict. Subtraction: delete to_dict and its docs sentence.

What this change ships

Inventory (6 items) — 4 justified

Intent: let App Kit apps batch-embed text with the shared model and learn the vector-space identity and a not-ready state, without a private import (#23174). ADDITION.

  1. Apps can embed up to 64 texts per call, sync or pooled, at bulk priority behind memory work — justified
  2. Batch results carry model id, width and space signature for spotting stale vectors — justified
  3. Not-ready, failed, invalid, too-many and on-event-loop are reported as separate reasons — justified
  4. An optional instruction prefix, at most 512 characters, for the instruction-tuned model — justified
  5. A to_dict() JSON form on the result — zero consumers
  6. The PR title says "gated" but the call is ungated — undeclared

[FIRST-PRINCIPLES-REVIEWED] f0b1db8

Description read

[DESCRIPTION-READ] 1ba4bbd7e2c5a1be5b179760ee6b0c6d96c9772efdc6af87b8d80dee224d2ba9

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Opus 5.5) — ✅ PASS

Design-level review of f0b1db8e950be2a465a437ff55c6cb62d0ee8a1f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

No design blocker: the change adds to the existing EmbedSDK seam, removes nothing, and reuses the shared backend's bulk priority and cold-load behaviour.

Notes

  • The PR title says "a gated batch embed", but the body, docs and the test_every_app_context_can_batch_embed_without_a_grant test say it is ungated. Change the title to match before merge, because it becomes the squash commit subject.
  • The Instruct: …\nQuery: prefix only suits the bundled instruction-tuned model. With a custom model the same instruction gives skewed vectors, and space_signature won't show it. The docs should say to drop instruction when the model is not the bundled one.

[DESIGN-REVIEWED] f0b1db8

Description read

[DESCRIPTION-READ] 97ebb78f908cdf8adad2c08ad36f02a21546fbcb2d13092df7bc73f2c5d662bb

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Opus 5.5 Review — ✅ no blocking findings

Reviewed f0b1db8e950be2a465a437ff55c6cb62d0ee8a1f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] f0b1db8

Verdict parsed from the review's SHA-scoped output markers for commit f0b1db8e950be2a465a437ff55c6cb62d0ee8a1f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable f0b1db8e950be2a465a437ff55c6cb62d0ee8a1f false-positive: <reason, 20 to 500 characters>

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

GPT 6.1 Review — 🔴 changes requested (blocking)

GPT 6.1 found at least one blocking issue that must be resolved before merging 7a023705356eee6f9ae64355df1a2638b613e8f4. 1 of 1 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/dashboard/agent_ipc.py:241 -- A spent pickup becomes reusable after its holder exits

if previous is not None and previous.pid != peer_pid and self._alive(previous):
Trigger: On macOS, an agent-shell descendant reads the MCP server’s retained startup pickup through KERN_PROCARGS2, terminates that server, and submits the pickup to /api/ipc/redeem. Seatbelt permits these process operations, and CapabilityStore.redeem accepts the shell’s runtime ancestry.
Outcome: The shell receives its own valid capability and bypasses MCP tool exclusions by calling internal endpoints directly, including /api/memory/recall when memory_recall is excluded. The endpoint checks session identity and memory-read mode, not that tool exclusion.
Anchor: residual/security
Fix: Keep redeemed pickups spent; provide a fresh pickup through the trusted issuer for legitimate server restarts.

[BLOCK-MERGE] 7a02370
[GPT-REVIEWED] 7a02370

Adjudication (Opus 5.5) — is blocking on each finding proportionate?

F1 (fenced). This is an unbounded, security-class harm: it bypasses the MCP tool-exclusion ceiling.

  • Conditions confirmed: the pickup reaches the server through the environment (src/kiro_crew/acp/ipc_pickup.py:66). Removing it in-process (src/kiro_crew/agent_ipc_client.py:93) does not clear the copy of the startup environment that macOS keeps and hands out through KERN_PROCARGS2. Once the holder is dead, redeem hands out a fresh capability (src/kiro_crew/dashboard/agent_ipc.py:241-246); its only check is that the caller descends from the runtime (:224).

  • Rarity: none of this is extreme. It needs a same-user process on macOS and a kill. Both are within reach of an agent shell that a prompt injection controls. So no FLAG override rationale holds.

    [ADJUDICATION] 7a02370 total=0 uphold=0 downgrade=0
    [GPT-ADJUDICATED] 7a02370

    [ADJUDICATION-FENCED] 7a02370 fenced=1 flagged=0
    UPHOLD-FENCED F1 src/kiro_crew/dashboard/agent_ipc.py:241 -- The pickup stays in the kernel-retained startup environment (ipc_pickup.py:66), which a same-user process can read on macOS, and redeem re-issues a capability once the holder is dead (agent_ipc.py:241-246), so a compromised agent shell can plausibly do this in real operation.
    [GPT-ADJUDICATED-FENCED] 7a02370

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 7a023705356eee6f9ae64355df1a2638b613e8f4 false-positive: <reason, 20 to 500 characters>

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Nov 6, 2026
@JLargent42

Copy link
Copy Markdown
Contributor Author

Re GPT 6.1 blocking finding (embed_sdk.py:250, batch permission decisions bypass SEL): fixed in 7a02370. build_app_context now passes the app name into EmbedSDK, and both embed_batch and embed_batch_sync decide through one shared check, EmbedSDK._batch_permitted. That check writes every decision to SEL via log_api_access(caller="app:<name>", operation="embed_batch", outcome="allowed"|"denied", resources="app=<name> texts=<n>"). The write is best-effort, so a failed audit changes neither answer. The async path audits once (it no longer re-checks inside the pooled sync call). New tests in test/test_embed_sdk.py cover the denied row, the single allowed row on sync and async, an audit-sink failure, and the attribution through build_app_context; all 5 fail without the fix. The spec and api-reference now note the SEL rows and that the grant is a fair-use declaration, not an access control.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 6, 2026
@JLargent42
JLargent42 force-pushed the feat/app-embed-batch-23174 branch from 7a02370 to f0b1db8 Compare November 6, 2026 06:50
@JLargent42

Copy link
Copy Markdown
Contributor Author

Updated in f0b1db8 (PR folded to one commit):

  • First Principles Review (BLOCK): agreed that the permissions.embeddings grant protected nothing an app could not already do one call at a time. It is removed, together with batch_granted, _batch_permitted, the per-call security-log writes and REASON_NOT_PERMITTED, and their doc rows. Batch embed is now available to every app context with the fair-use bulk priority and batch cap the issue asked for. New test test_every_app_context_can_batch_embed_without_a_grant; 18 batch tests fail on the old head and pass now.
  • GPT 6.1 Review (blocking): the finding points at src/kiro_crew/dashboard/agent_ipc.py:241, which this PR does not touch (the diff is embed_sdk.py, test_embed_sdk.py and two docs). It looks like a real residual in existing pickup-redeem code on main, so it should be tracked as its own issue rather than fixed here.

@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Nov 6, 2026
@bolichen97

Copy link
Copy Markdown
Contributor

[operator: bolichen97] Continuing this in #36450; thanks for the start.

@JLargent42

Copy link
Copy Markdown
Contributor Author

The KiroCrew conductor (largjaco's v2 drive-to-green session) now owns this PR under the 24h orphan rule; the desk session that opened it has been offline since 11-06. I'll rebase onto main, fix CI and answer review here.

@KiroCrewCaptain

Copy link
Copy Markdown
Collaborator

Code change approver recommendation: A person should decide

Why: the decider raised 2 possible problems and estimates a 55% chance that this change introduces a defect; a person should check them.

The adapter is correct and well tested for single calls. The problem is the bulk-priority design: the async path puts throttled, deadline-free bulk inference on the shared 8-worker mc-embed pool that every chat turn uses to build its prompt, and app bulk jobs crowd the shared inference queue's non-interactive slots. A few concurrent app batches can therefore stall chat turns and get host memory writes refused, which contradicts the PR's fair-use claim and repeats the starvation incident documented in executors.py. No test covers concurrency. Requesting changes, for example a dedicated pool or concurrency limit for app batches and a queue-busy answer mapped to not_ready.

Things to check (possible problems the decider raised; at this confidence many turn out to be fine):

  • One 64-text app batch holds an mc-embed worker for minutes, mostly in duty-cycle pauses. Concurrent app calls, such as gathering split work as the docs advise or concurrent hits on an app route, can fill all 8 workers and stall every chat turn's prompt build. That is the documented starvation the se (src/kiro_crew/apps/embed_sdk.py, medium)
  • The description and docs say batch work runs 'behind memory recall and memory writes'. In fact a few concurrent app batches fill the shared inference queue's 6 non-interactive slots. Host PRIORITY_NORMAL memory writes are then refused ('embedding queue is busy') and stored without vectors, and the a (src/kiro_crew/apps/embed_sdk.py, medium)

An automated recommendation, not a review: a maintainer still decides. It approves on its own only when the estimated chance of a defect is at most 12% and the change has at most 400 changed lines. Decider v3c (217fe607) · commit f0b1db8e95 · 351 changed lines. If this recommendation is wrong, reply here: replies are how the approver is evaluated.

@JLargent42

Copy link
Copy Markdown
Contributor Author

The KiroCrew conductor is yielding this PR to #36450, which continues it; no further pushes from here. Anything in this PR that #36450 lacks (tests, edge cases) is listed below.

Gaps found by comparing the two diffs: none found. #36450 has the same reasons (invalid_input, too_many_texts, on_event_loop, not_ready, failed), the 64-text and 512-character instruction caps, and a counterpart for each test here (identity, ordering behind memory recall, no grant, size cap, malformed input, instruction prefix, not-ready, never-raises, event-loop refusal, async pool). The only item #36450 lacks is EmbedBatchResult.to_dict(), which the First Principles review on this PR flagged as having no callers, so it is not worth carrying over.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

readiness: action required A blocking check or review needs attention

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose the shared embedding model to apps (SDK function or local route)

3 participants