Repository navigation
feat(knowledge): query-time ACL + revocation isolation in the shared KB retrieval chain - #11219
bolichen97 wants to merge 1 commit into
Conversation
Design Review (Fable 5) — 🟡 CONCERNSDesign-level review of Design-Verdict: CONCERNS Existing connector-ingested content goes dark on upgrade — the deny ships before any hook, binding, or re-admission path exists to ever show it again. Watch
[DESIGN-REVIEWED] f97873c |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — 🔴 BLOCKPremise-level review of First-Principles-Verdict: BLOCK Imported libraries become invisible with the restore path "not invented here" and no RFC; plus rows/vendor/resolver plumbing whose only consumers are unlanded PRs. Not justified as shipped
What this change shipsInventory (10 items) — 3 justifiedIntent: give the shared knowledge library per-user, provider-checked visibility at query time — framed as a FIX for a cross-user leak, but no base path ingests a shared source (
Blockers
[FIRST-PRINCIPLES-REVIEWED] f97873c |
GPT 5.6 Review — 🔴 changes requested (blocking)GPT 5.6 found at least one blocking issue that must be resolved before merging This comment is updated in place on each push. BLOCKING -- test/test_knowledge_acl_w01_integration.py:28 -- Integration suites silently skip when the resolver is absent BLOCKING -- src/kiro_crew/knowledge/rows.py:65 -- Blank rows are checkpointed without content BLOCKING -- src/kiro_crew/dashboard/handlers/knowledge.py:1173 -- Related-item lookup does not authorize its subject [BLOCK-MERGE] f97873c Adjudication (Opus 4.8) — is blocking on each finding proportionate?F1 opens at F2 (fenced): F3 (fenced): False positive or not applicable? A repository writer can comment: |
UX Review (Fable 5) — ⏭️ skippedRevision |
…tor-sf-structured
…ntract Integrated chat408 ACL row API f5bc46a (PR kirodotdev#11219) by normal git merge and wired the per-row ingest contract. Verified the contract against the f5bc46a body -- ONE discrepancy from the relayed summary: the SourceRow field is `resource_ref` (an acl.ProviderResourceRef), not `ref`; managed is a read-only property fixed True (not a passable field). Everything else matches. - supports_rows() -> True: Salesforce is structured, so the sync scheduler drives fetch_rows -> ingest_rows -> per-row set_item_acl, never the single text blob that would collapse many rows' grants into one chunk. - fetch_rows(source) -> (rows, snapshot, checkpoint): each SourceRow carries its OWN key/text/resource_ref/grant. subjects is the EMPTY tuple -- a fail-closed deny-all, NEVER acl.PUBLIC_SUBJECT and never a blank default: Salesforce per-record sharing is established at query time by the shared revalidation hook, not inferred here. tenant is the vendor org id (non-empty). - Snapshot semantics kept SEPARATE per path: SOQL object path is incremental (snapshot=False, since-watermark checkpoint); Report path is a full snapshot (snapshot=True, no watermark). Neither reuses the other's refresh strategy. - Refactored the live read into _collect_rows shared by fetch_rows and the legacy fetch (kept for the abstract BaseConnector.fetch). - props['checkpoint'] is READ as the "ACL landed" signal (shared sync writes it only after full persistence); this connector never writes it. Real SyncScheduler ingest proof (TestSyncSchedulerRealIngest) drives the WHOLE chain over real TLS: loopback TLS server -> SalesforceProductionRunner -> real W01 executor -> fetch_rows -> SyncScheduler.sync_source -> ingest_rows -> store.set_item_acl per row. Reads back store.get_item_grants and asserts each grant is managed=True, subjects==[] (explicit deny, not public), carries the row's own salesforce ProviderResourceRef, and that props['checkpoint'] advanced to the max SystemModstamp only on full success. Both SOQL (2 real TLS pages) and Report (full snapshot) paths proven. 127 tests pass; fresh make backend + fresh venv green (57). black/isort/flake8/pyflakes clean. No shared hunk edited.
de20850 to
36bb7bd
Compare
36bb7bd to
38174bc
Compare
…d KB retrieval chain Adds per-item ACL to the knowledge retrieval gate: a managed cloud/structured item is served only to a query principal whose per-candidate provider binding resolves and revalidates; trusted-local items keep serving the on-host personal library. Provenance-based classification (sourceless and unstamped => managed, fail-closed), an unforgeable trusted-creator stamp, and provenance that survives delete, dedup and bundle round-trip. Per-row structured ingest (SourceRow + connector_row_state + ingest_rows/sync) carries each row key, text, ProviderResourceRef and grant to its own item, with the checkpoint advancing only after full persistence. The shared dashboard handler registers the GitHub/Google/Salesforce connectors import-guarded and injects each vendor runner factory by the connector own keyword. Query principal is derived only from the auth middleware established identity (request app/user/is_dashboard_user), never a raw header; an authenticated app or the dashboard owner becomes a stable verified principal, and no established identity denies. The W01 AccessGrant is bridged into an AccessContext, and the verified-principal marker (QueryPrincipal.verified + principal_is_verified) supplies the trust contract W01 ControlPlaneBindingResolver requires.
|
暂停关闭(Connector Production Stack 项目于 2026-09-20 整体暂停)
本次关闭不代表该工作被否决,只是项目暂停收口。 |
Summary
Query-time ACL and revocation isolation wired into the existing shared Knowledge Library retrieval chain — the query-time half of the Connector Production Stack's shared KB/ACL contracts (KB-01 / ACL-02 / ACL-03 / ACL-05 / ACL-06 / ACL-09 / KB-10). No second KB, no types-only framework: it wires into the existing
KnowledgeStore/HybridRetrieverand the real query call boundaries.HybridRetriever.search'ssource_id/namespacefilters are relevance labels ("NOT a security boundary"); the graph leg was unfiltered; the store had no principal/tenant/grant state — so a KB that ingests shared/managed sources could leak one user's content to another at query time.The gate
Item-scoped, provenance-classified, fail-closed. One
KnowledgeStoremixes trusted-local material with managed cloud/structured items, so the gate decides per item, from EVIDENCE, not from the entry point or a source_type name:sources.trust_class(local_admitted|managed) is written once at creation from the creating path's own knowledge (initial_trust_class): the real on-host local creators (local_folder, local_file, obsidian_vault, artifact, agent, doc) stamplocal_admitted; every remote connector / unknown / misspelled type defaults tomanaged. A one-time migration adds the column (defaultmanaged) and backfillslocal_admittedonto the local-creator types.acl.is_managed_trust_class(has_source, trust_class): sourceless ⇒ managed (absence of a source is not proof of no external ACL);local_admitted⇒ local;managed/ unstamped / unknown / dangling (missing source row ⇒None) ⇒ managed. The query-time gate reads the STAMP only — it never re-guesses trust from a source_type string, so a new/misspelled cloud type, or a managed source whose per-item grant forgotmanaged=True, cannot be waved through as local.acl.LOCAL_LIBRARY(the local single-user context) sees trusted-local items without a grant but is denied managed items — it carries no verifiable provider-mapped subject. A managed item is always checked against a real current subject/tenant and revalidated, regardless of entry point.Revalidation / staleness chain (managed items). A static ingest grant + a local
acl_versionbump is not proof the provider still grants access:acl.RevalidationHook(the provider live-permission interface — a named dependency, not implemented here) +RevalidationOutcome{FRESH,REVOKED,UNVERIFIABLE}. A managed grant is served only when the hook saysFRESHOR the storedfresh_as_ofis within the staleness window.REVOKEDdenies even a subject-matching grant; a wired hook returning/raisingUNVERIFIABLEis a hard deny (no stamp fall-back); with no hook wired a stale/never-revalidated managed grant is denied.fresh_as_ofis meaningful only as a trusted confirmation record (set_item_acl/mark_item_acl_revalidated) — a manual mark / fresh flag is not treated as a completed real-time revocation check, and no other session/binding identity stands in for the current querying subject.store:item_aclgainsmanaged+fresh_as_of;revoke_item_aclresetsfresh_as_of+ bumpsacl_version;mark_item_acl_revalidatedrecords a provider confirmation or observed revocation; grant rows are removed with their item (KB-10).Every leg gated.
search(access_context=...)filters every fused candidate — keyword + vector + the unfiltered graph leg + the protected keyword rescue + citation/location enrichment — before the result window is cut.Non-search reads + export gated too. The gate is applied to every read that returns item content, not just
search: the browse listing (ACL-filtered BEFORE pagination so hidden counts do not leak and visible rows are not pushed off later pages), item detail / content / related / entity-items / per-item export (denied ⇒ 404, so existence does not leak), and the full-library export (bundle items AND the sources/entities/relations/locations reachable only from hidden items are stripped, unconditionally, so an empty visible set cannot leak the rest).Import trust invariant. An untrusted
.knowledgebundle's claim, missing field, or naming an existing source id NEVER produces local trust or a non-managed authorization: an imported source is local only if the same id already exists here aslocal_admitted; every newly-inserted imported item gets a forced managed grant regardless of the source it names; a bundle-provideditem_aclis forcedmanaged(explicitmanaged=0ignored) and is restored ONLY for items this import newly inserts (a pre-existing item's grant is never overwritten); malformed entries are skipped, not crashed.Ingest half (per-row contract).
knowledge/rows.py(SourceRow) +IngestionPipeline.ingest_rows+ theconnector_row_stateledger +BaseConnector.supports_rows/fetch_rowscarry each structured row's own identity + content + ACL grant to its item group, writing the grant and the ledger (and the prior group's delete) in one transaction. Its consumers are the vendor connector PRs (Google Drivesupports_rows→True; GitHub_source_row_from→SourceRow).Call boundaries.
mcp_tools/knowledge.py+dashboard/handlers/knowledge.pypassLOCAL_LIBRARY(they serve the on-host personal library; managed items mixed in are still gated/denied there). A shared/multi-tenant caller resolves subject+tenant from the authenticated caller — for a managed item, the provider-mapped identity (W01Binding.subject_ref/tenant_reffrom the verifier, NOT a raw session key and NOT L08's operation-scope handle) — and wires aRevalidationHook.Testing
test/test_knowledge_query_acl.pydrives the real store + gate: provenance classification (sourceless / local_admitted / managed / unknown / dangling), a MIXED library (local doc readable, managed cloud doc gated), adversarial cases (a brand-new unknown cloud type is gated; a managed source whose grant omittedmanaged=Trueis still gated and revoke-denied), two identities, cross-tenant, graph-leg leakage, and the full revalidation chain (fresh/revoked/error, staleness, mark-revalidated), plus migration backfill. 1004 knowledge tests pass in a clean source-tree-external venv; end-to-end provenance + unknown-type + migration validation from that venv; flake8 clean.Not in scope / untouched
This PR ships the query-time gate PLUS the per-row ingest contract it governs, so it does edit
knowledge/connectors/base.py(addssupports_rows/fetch_rows) and addsknowledge/rows.py+ingest_rows. It does NOT touchconnections/control_plane/**or vendor connector dirs, and does not implement a vendor connector. The provider live-revalidation hook and the provider identity mapping (W01 binding) are named, fail-closed dependencies — declared, not faked. Per-provider live cloud verification, and a verifiable one-step local-backup restore (imported bundles are managed until re-admitted locally), remain separate required items.