Skip to content

feat(connections): W01 L09 control-plane executor, real transport, neutral data channel - #11286

Closed
bolichen97 wants to merge 3 commits into
mainfrom
feat/connector-control-plane-executor
Closed

bolichen97 wants to merge 3 commits into
mainfrom
feat/connector-control-plane-executor

Conversation

@bolichen97

@bolichen97 bolichen97 commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

W01 · L09 — the executor that wires the four control-plane judgments, plus the real transport composition, neutral data channel, and the L04 live-store send path

This is the integration point of W01: it turns the upstream primitives from unconsumed helpers into a real executor that runs the four judgments in order before any transport call, resolves the credential per-binding from L04's trusted live store, and delivers the data a consumer needs on the success path.

Base / provenance

Opened against main. #11066 squash-merged the W01 L01 control-plane seam (+ W02 GitHub, W05 Graph) into main and deleted the feat/connector-control-plane-seam branch, so this PR was re-based onto current main by content analysis (the L01 seam files operation.py / context.py / errors.py and the vendors/** trees are already in main and are not re-proposed). The upstream dependency slices this L09 work imports are not yet in main and are carried here so the executor can import and its suites run; each is preserved from its own source:

Slice Source
L02 binding binding.py
L04 lifecycle lifecycle.py + acl_binding_resolver.py — commit-level integration of L04's a8e907ee9d3d664a1a8df50d92564b922f9cd162 (git cherry-pick), so L04's implementation and its own ~47 tests now execute in this PR's tree (they pass). Conflicts resolved by union; lifecycle.py is a8 verbatim (code AST == a8, comment-only edits for gate compliance). BindingResolutionError lives in binding.py (L02). The F2 store-seal and L08's F3 handle-shape validation are integrated in this PR (see the commit 2 notes and the ownership section).
L05 auth-modes auth_modes.py
L06 policy policy.py
L07 writes writes.py
L08 handle handle.py
L09 (this slice) executor.py, production.py, the neutral-data-channel additions to result.py / control-plane __init__.py / connections.md, and the test_connections_control_plane*.py suites

Commits (this PR — 2)

  1. 44900dcd2 — the W01 L02/L04/L05/L06/L07/L08/L09 control plane on current main: the executor wiring the four judgments (ensure_usable → permit_operation → decide → replay_decision) before any transport call; the real production composition; the L04 live-store per-binding send path (build_production_transport takes store: BindingStore + gate: BindingCustodyGate; per call the gate matches custody, then store.select_secret(binding, reader=vault) fences against the LIVE store and takes secret_ref from the store's record — binding_secret_ref is neither imported nor called, AST-checked); the neutral data channel; the single authoritative cursor; the response-metadata allowlist; guarded redirects with cross-origin credential strip; and write_outcome=unknown on timeout/ambiguity.
  2. 09292555c — the review-response commit folded onto the stack (a8 L04 commit-level integration, L08 F3, the F2 store seal, plus the earlier F1/F4/500/CodeQL/Inclusive fixes), rebased onto current main (04727ebe0, which carries fix(security): word a resolver stall as unverifiable, not a path match #11221 and fix(dashboard): drop the unused json import left by #9840 #11415):
    • F1 (class-level, production.py BindingCustodyGate) — the authorization carrier is frozen at the moment of validation: __post_init__ replaces self.binding with a deep read-only snapshot (_frozen_binding, incl. the nested secret_ref) and trusted_binding_for hands out a read-only mapping. Root cause: the authorization value travelled as a mutable dict and each consumer decided separately whether to re-read it, so a per-field comparison only moved the seam; removing the mutable carrier makes "validate then mutate the dict" unrepresentable. Regression reproduces the post-construction swap; mutation (freeze disabled) fails it.
    • HTTP 500 → unknown (production.py) — 500 is now in _AMBIGUOUS_TRANSIT_STATUSES. A 500 on a non-idempotent write carries no proof of pre/post-commit, and the executor's own contract refuses letting a caller infer "not applied" from a 5xx, so the safe default is write_outcome=unknown (L07 then REPLAY_REFUSEs unless the caller asserts idempotence). A vendor owner that knows its API commits-first still narrows via its own write_outcome. Controlled-TLS test: server commits then 500 → unknown + REPLAY_REFUSE; read-500 stays None; idempotent override still ALLOWs.
    • F4 (test_connections_control_plane.py) — the two spawned refresh workers get cwd=tmp_path and are terminate/wait-reaped in a finally.
    • CodeQL py/clear-text-logging — removed an evidence print that interpolated a revealed secret; assertions carry the evidence with no plaintext to a log.
    • Inclusive Language — one comment sanity-check → confidence-check (comment-only).
    • L08 F3 (handle.py) — commit-level integration of L08's 94841250a: ensure_usable validates the DerivedHandle shape → HandleTamperedError on a missing/wrong-typed field, and is_expired gets the _handle_id_is_lookupable predicate (a malformed handle answers True, does not raise). L08's own 10 counter-example tests ride along and pass in this tree.
    • F2 store seal — the binding store moves to its own top-level directory config_dir()/control-plane-bindings/ (bindings.json + bindings.lock), out of the agent-writable connections/ dir. lifecycle.py (the two path constants + docstrings only), security/paths.py (one directory entry on _WRITE_PROTECTED_HOME_PATHS), and sandbox.py (the same top-level leaf in _CREW_READONLY_LEAVES + _CREW_PRECREATE_READONLY_DIR_LEAVES + _CREW_NOFOLLOW_READONLY_DIR_LEAVES). Behavior-level tests: file-tool gate refuses store/lock/temp/future-child and keeps reads open; Linux launcher READONLY_DIRS (config-level) and macOS Seatbelt deny-write/link (text-level). NOT verified at the real-kernel level (no test spawns a sandbox and checks a write's rc), and Windows has no launcher seal (file-tool gate only) — so F2's OS-layer coverage is incomplete and this PR does not claim it complete.

Schema versions (pin these)

  • RESULT_SCHEMA_VERSION = 3
  • EXECUTOR_SCHEMA_VERSION = 4
  • PRODUCTION_SCHEMA_VERSION = 4
  • LIFECYCLE_SCHEMA_VERSION = 1

Invariants (all covered by tests)

  • Four judgments run before any transport call; a denied gate emits zero transport calls (and reaches neither sender, vault, nor payload).
  • Routing uses only the trusted TrustedHandleView — never the mutable handle.
  • The credential is resolved per binding from the live store (select_secret → assert_live): a revoked binding, a stale/forged generation, or a swapped identity/mode/secret_ref refuses before a socket exists; there is no slug fallback on the send path. A PageWalk re-fences every page.
  • HTTP 412 is preserved as a structured signal; an unknown condition is not given a fabricated precondition.
  • 500 / 502 / 503 / 504 / timeout / exception / a server-committed-then-dropped disconnect on a non-idempotent write records write_outcome = unknown (500 with no proof of non-application defaults to unknown; a read never claims one), which the L07 replay gate refuses to blindly replay unless the caller asserts idempotence.
  • Redirects are checked per hop (same-origin, no https→downgrade, egress allowlist) and any cross-origin hop is stripped of credential headers.
  • Response metadata reaches the caller through an allowlist; Set-Cookie / Authorization / WWW-Authenticate and other credential/session headers can never appear.
  • The success path carries data: OperationResult.payload is one of collection (items) / single object / raw bytes (Office bytes never text-coerced), with the cursor on the envelope, not the payload.

Outward API (for the W02 GitHub and W05 MS downstreams)

class OperationResult(TypedDict):
    status: ResultStatus          # "ok" | "partial"
    next_cursor: str | None       # the SINGLE authoritative cursor
    payload: OperationPayload | None

OperationPayload = CollectionPayload | ObjectPayload | BytesPayload
# CollectionPayload(items)  — cursor lives on the envelope, not here
# ObjectPayload(object)
# BytesPayload(data, media_type, filename)  — never text-coerced

# build_production_transport(*, gate: BindingCustodyGate, store: BindingStore,
#                            vault, locator, http_send=…, decode=…, timeout_seconds=…)
ExecutionOutcome:  .result .error .precondition .view .write_outcome .metadata  (+ .ok, .payload)
ResponseMetadata = Mapping[str, str]   # lowercase keys, allowlisted, read-only
execute(...) -> ExecutionOutcome       # + advance_page(PageWalk), classify_error

Verification

297 control-plane tests pass (the guard suites + executor / production / data-channel / metadata / real-TLS e2e suites), including the two-binding-different-secret, revoke→zero-calls, no-slug-fallback, page-2-re-fence, and committed-then-dropped→unknown regressions on the real TLS path; the disconnect fix is mutation-verified (reverting the except tuple fails both new tests). black --target-version py312, flake8, isort, mypy all clean (return codes captured non-piped). No vendors/**, platform/governance.py, or registry old-API changes.

Build / fresh-install verified on the current head: real python -m build → wheel; zipfile confirms executor.py / production.py / lifecycle.py are in the wheel; a fresh venv install; run from a cwd outside the source tree with PYTHONPATH unset resolves the module in site-packages; and a representative call from the installed wheel on a real TLS path exercises both the new live-store fencing (a real secret reaches the wire once; a revoke then fences the next send to zero calls) and the committed-then-dropped→unknown + replay-refused path. The installed package reports RESULT=3 / EXECUTOR=4 / PRODUCTION=4.

Named gaps (not claimed as done)

L03 (auth code / PKCE / state / callback replay protection) is still required and unimplemented — the feature surface is known-incomplete. principal→binding resolution is single-binding (refuses others — not a multi-identity runtime); the vault entry name is per-binding (L02's binding_scoped_secret_ref, adopted here from e8e11aa23); the principal→binding ACL adapter (BindingResolver.resolve → AccessContext, whose signature differs from lifecycle.BindingStore.resolve) is L04's follow-up and is not wired here; MS's own 412 / readback / baseline / locator semantics live under vendors/microsoft/** (this only preserves the shared signal) and MS owns removing its own disconnect xfail; document/provider-level query ACL is a separately-owned gap.

Do not auto-merge while any known finding is open.

Fenced-finding ownership (as of HEAD 09292555c)

  • F1 / F4 — mine, fixed here (see commit 2 above).
  • F2 (binding-store write seal) — integrated in this PR (see commit 2's "F2 store seal"). The store moved to a top-level control-plane-bindings/ directory and is registered on the platform write-protection list (security/paths.py) and the OS-sandbox read-only leaf lists (sandbox.py); the two lifecycle.py path constants were changed under a one-time cross-owner authorization (their logic otherwise untouched). OS-layer verification is config/text-level only (see the F2 caveat above); not claimed complete.
  • F3 (handle.py) — integrated in this PR via L08's commit 94841250a (see commit 2's "L08 F3"). No longer "not modified here".
  • Later GPT round on 09292555c raised two further fenced findings (production.py:683 encoded-credential echo — Opus adjudication FLAG; production.py:1623 malformed-JSON→{} — Opus UPHELD-FENCED) and one non-fenced executor.py:641 top-level-imports finding; these are open and not yet addressed in this HEAD.

@bolichen97
bolichen97 requested review from a team and patrigao and removed request for a team September 16, 2026 10:39
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 16, 2026
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 1d2300a167b0467c53068e04337baa345efa1172 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Half the diff — the #11349 Google Drive/knowledge-ACL squash at HEAD — is outside this PR's described scope, and the body still says HEAD is 09292555c.

Watch

  • The description covers commits 44900dcd2/09292555c ("Commits (this PR — 2)", "Fenced-finding ownership (as of HEAD 09292555c)"), but HEAD 1d2300a16 adds a third commit carrying an entire second feature — Drive vendor tree, knowledge/acl.py, store schema, retrieval, MCP and sandbox edits (~26 files). Approvers of the described executor are silently approving an undescribed feature, and the 2-commit/one-logical-change rule is breached.
    Clears when: the body is rewritten to cover the Drive/ACL commit at this HEAD, or that commit lands through its own PR.
  • The description itself withholds readiness: "Do not auto-merge while any known finding is open" and names two open fenced findings on production.py (encoded-credential echo, adjudicated FLAG; malformed-JSON→{}, UPHELD-FENCED). Merging as-is contradicts the author's own gate.
    Clears when: both open production.py findings are fixed or explicitly waived in the body.
  • The F2 binding-store READONLY seal is verified only at config/text level ("NOT verified at the real-kernel level", "Windows has no launcher seal") while AGENTS.md makes the OS layer the enforcement point — until verified, the new ceiling leaf rests on the tool gate alone.
    Clears when: a real-sandbox write-refusal test for control-plane-bindings/ lands, or the security spec records the accepted residual.

[DESIGN-REVIEWED] 1d2300a

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 16, 2026
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 1d2300a167b0467c53068e04337baa345efa1172 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

First-Principles-Verdict: CONCERNS

An undeclared second feature — the Google Drive knowledge source + query-time ACL (head commit 1d2300a16, squash of #11349) — merges behind a description stating "Commits (this PR — 2)".

Not justified as shipped

  • Item 7 — undeclared: the Drive vendor tree rides in 1d2300a16, absent from the fetched title/description; it is also the executor's only real consumer (3 files — host_factory.py, operations.py, acl_probe.py; grepped control_plane.executor|build_production_transport over src/, zero consumers elsewhere).
    Subtraction: defer commit 1d2300a16 to feat(connections): Google Drive knowledge source — provider + per-file rows + sync + query-time ACL (D1) #11349's own merge, leaving this PR the two commits it declares.
    Clears when: the description declares the D1 commit, or the branch drops 1d2300a16 and feat(connections): Google Drive knowledge source — provider + per-file rows + sync + query-time ACL (D1) #11349 merges separately.
  • Item 8 — undeclared: same commit, same silence; ships knowledge sync/ingestion surface.
    Clears when: same as item 7.
  • Item 9 — undeclared: same commit; ships a persisted item_acl table + migration and new MCP/dashboard call contracts. The harm is real (the commit documents the unfiltered graph leg), but the description never mentions any of it.
    Clears when: same as item 7.
  • Item 10 — undeclared: docs rider of the same commit (two new Google Drive spec files).

What this change ships

Inventory (10 items) — 5 justified

Intent: run connector operations through the governed control plane (four judgments, live-store credential, real HTTPS) and deliver payload data — ADDITION. Note: the workflow-supplied patch conflates ~970 files of already-merged main history; the branch's real delta vs merge-base 04727ebe0 is 65 files, reviewed here.

  1. Connector operations now run four control-plane judgments before any transport call — justified
  2. Real HTTPS transport: per-binding live-store credential, guarded redirects, response-header allowlist — justified
  3. Success results now carry payload data with a single authoritative cursor — justified
  4. Ambiguous write failures (500/timeout/disconnect) record write_outcome=unknown, refusing blind replay — justified
  5. Upstream slices L02/L04–L08 carried so the executor imports resolve — rides along (declared, load-bearing)
  6. Binding store moved to write-protected top-level control-plane-bindings/, sealed in the sandbox — justified
  7. Google Drive connector vendor tree (real Drive API v3 wire operations) — undeclared
  8. Google Drive knowledge source: per-file rows and sync in the Knowledge Library — undeclared
  9. Knowledge retrieval now gates every candidate through a fail-closed subject/tenant ACL — undeclared
  10. Two new Google Drive spec docs — undeclared

[FIRST-PRINCIPLES-REVIEWED] 1d2300a

@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — 🔴 changes requested (blocking)

GPT 5.6 found at least one blocking issue that must be resolved before merging 1d2300a167b0467c53068e04337baa345efa1172. 3 of 4 blocking finding(s) are security-class and were withheld from adjudication, so the blocking verdict stands.

This comment is updated in place on each push.

BLOCKING -- src/kiro_crew/knowledge/sync.py:65 -- Successful structured syncs fail during checkpoint advancement (origin: validation)
with self._sync_outcome_lock:
fully persisted rows -> _advance_checkpoint -> missing attribute -> false failure status and checkpoint loss
Anchor: residual/crash-data-loss-corruption
Fix: Use the store’s atomic property-revision mechanism instead of the removed lock.

BLOCKING -- src/kiro_crew/knowledge/connectors/google_drive.py:294 -- Failed content reads become authoritative snapshot omissions
return None
transient content failure -> snapshot omits existing file -> ingest_rows deletes its indexed data
Anchor: residual/crash-data-loss-corruption
Fix: Mark the snapshot incomplete and prevent deletion when a content fetch fails.

BLOCKING -- src/kiro_crew/connections/vendors/google_drive/decode.py:71 -- Malformed collections become authoritative empty snapshots
items = tuple(f for f in raw if isinstance(f, Mapping)) if isinstance(raw, list) else ()
2xx response with non-list files -> empty snapshot -> previously indexed Drive rows are deleted
Anchor: residual/crash-data-loss-corruption
Fix: Reject a non-list files field as a malformed response.

BLOCKING -- src/kiro_crew/dashboard/handlers/knowledge.py:3157 -- Optional connectors are instantiated on the gateway boot path (origin: validation)
_register_optional_connector(
gateway launch -> setup_knowledge_routes -> module import and connector construction -> dashboard socket binds later
Anchor: no-new-work-on-gateway-boot-path
Fix: Defer optional connector import and construction to a memoized first-use accessor.

[BLOCK-MERGE] 1d2300a
[GPT-REVIEWED] 1d2300a

Adjudication (Opus 4.8) — is blocking on each finding proportionate?

F4 is anchored to no-new-work-on-gateway-boot-path, which I confirmed carries blocking: true in AUTOSDE.yaml:129-130. The finding names subsystem construction (_register_optional_connector → module import + connector instantiation) reached from setup_knowledge_routes in dashboard/handlers/knowledge.py — squarely inside the rule's file-pattern and its point-2 prohibition on "subsystem construction as a side effect of route registration." The rule's flag is authoritative and outranks my weighing.

Fenced notes: F1 — self._sync_outcome_lock (sync.py:65) is never assigned in SyncScheduler.__init__ (sync.py:19-22) nor anywhere in the file; _advance_checkpoint raises AttributeError on every fully-persisted structured sync, so no rarity argument exists. F2 — a DriveOperationError on content fetch (google_drive.py:288-294) is an ordinary transient condition, not an extreme one. F3 — a non-list files field (decode.py:70-71) is producible by an external API/proxy, not only by the system's own writer.

harm: F1/F2/F3 unbounded (crash-data-loss-corruption); F4 governed by blocking rule.
conditions: F4 confirmed at AUTOSDE.yaml:130 + knowledge.py:3157; F1 confirmed missing attr at sync.py:19-22,65.
fix cost: not weighed — F4 anchored to a blocking rule; fenced findings retain unbounded harm.

[ADJUDICATION] 1d2300a167b0467c53068e04337baa345efa1172 total=1 uphold=1 downgrade=0
UPHOLD F4 src/kiro_crew/dashboard/handlers/knowledge.py:3157 reason=autosde-blocking-rule
[GPT-ADJUDICATED] 1d2300a167b0467c53068e04337baa345efa1172
[ADJUDICATION-FENCED] 1d2300a167b0467c53068e04337baa345efa1172 fenced=3 flagged=0
UPHOLD-FENCED F1 src/kiro_crew/knowledge/sync.py:65 -- `_sync_outcome_lock` is never assigned, so every fully-persisted structured sync raises AttributeError and loses the checkpoint — a universal, not rare, condition.
UPHOLD-FENCED F2 src/kiro_crew/knowledge/connectors/google_drive.py:294 -- a transient content-fetch failure is an ordinary condition, and dropping the row from a full snapshot deletes previously indexed data.
UPHOLD-FENCED F3 src/kiro_crew/connections/vendors/google_drive/decode.py:71 -- a non-list `files` field is producible by an external API/proxy, so the malformed-response-to-mass-deletion path is not beyond what real operation can produce.
[GPT-ADJUDICATED-FENCED] 1d2300a167b0467c53068e04337baa345efa1172

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 1d2300a167b0467c53068e04337baa345efa1172: <one-sentence reason>

@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from 423528c to bbb75ee Compare September 16, 2026 10:51
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 16, 2026
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 1d2300a167b0467c53068e04337baa345efa1172 — this comment is updated in place on each push.

Review details

No blocking findings; one advisory: an unbounded in-memory issuance registry with no eviction path.

FINDING — src/kiro_crew/connections/control_plane/handle.py:400 — _ISSUED[handle_id] = _IssuanceRecord(...) is written on every derive_handle (host_factory.py:9146 calls it once per Drive sync round, per source) and no code path anywhere deletes, pops, caps, or expiry-sweeps _ISSUED, so a long-lived gateway grows the registry without bound as expired records are never reclaimed → Fix: evict a record when it is found expired in ensure_usable/is_expired, and/or bound _ISSUED with expiry-driven eviction at mint in derive_handle (all inside this new file).

[OPUS-REVIEWED] 1d2300a

Verdict parsed from the review's SHA-scoped output markers for commit 1d2300a167b0467c53068e04337baa345efa1172.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 1d2300a167b0467c53068e04337baa345efa1172: <one-sentence reason>

bolichen97 added a commit that referenced this pull request Sep 16, 2026
… rows fetched through W01's executor/transport

Locate a GitHub operation, dispatch it through W01's real executor and
production transport, decode the reply into the neutral payload channel, and
page a real structured fetch across two pages — with no naked sender anywhere.
Wire the knowledge GitHub structured source's live path onto that transport so a
real fetch returns real rows and a real refresh runs.

## What changed

Three new leaves under vendors/github/, plus the knowledge connector wiring and
docs, all consume-only over the W01 control plane (executor.py + production.py at
423528c; schema versions EXECUTOR=4 / PRODUCTION=3 / RESULT=3 / OPERATION=2,
pinned and asserted at compose time):

- vendors/github/locator.py: operation_id + params -> one concrete HttpRequest,
  GitHub's implementation of the transport's injected RequestLocator (mirrors the
  Graph locator). Endpoint templates come from the existing DESCRIPTORS table;
  path segments are individually validated and URL-encoded; it holds no
  credential.
- vendors/github/decoder.py: a GitHub 2xx payload -> OperationResult, GitHub's
  ResultDecode. It puts the fetched records into the ONE neutral data channel — a
  CollectionPayload for a list reply (JSON array, wrapped collection, or GraphQL
  nodes/edges), an ObjectPayload for a single object — via result_with_payload,
  and sets the SINGLE authoritative next_cursor on the envelope (the Link
  rel="next" URL for REST, the GraphQL endCursor for cursor tools). The
  collection carries no cursor of its own.
- vendors/github/dispatch.py: projects a GithubOperationDescriptor into the
  executor's OperationDescriptor, composes the production transport with GitHub's
  locator and decoder, and drives execute() and PageWalk/advance_page. The full
  W01 gate chain and per-binding custody (BindingSecretSelector) run inside the
  executor.
- knowledge/connectors/github_structured.py: PR-3 delta on top of the PR-2
  domain model. GithubStructuredConnector takes an optional
  GithubTransportProvider; fetch/detect_changes drive a REAL W01 PageWalk for the
  repo-scoped pull-request and commit entities (invoked, authorized per page, the
  walk advancing on W01's single next_cursor), read the rows off
  ExecutionOutcome.payload, convert them with PR-2's converters, fold them through
  PR-2's diff_rows, and render (text, metadata). detect_changes reports changed
  iff the since-window returned rows. No payload copy, no second envelope, no
  out-of-band capture, no cache. Built with no provider, fetch/detect_changes
  still refuse NotImplementedError (PR-2's parked contract, unchanged).

No auth, custody, retry, fencing, error classification, or pagination engine is
re-implemented — all W01's. Nothing under vendors/github/ or the connector
touches urllib / requests / httpx: the sole sender is W01's
build_production_transport.

## Paging: a single cursor, and a question

The walk advances on OperationResult.next_cursor alone. GitHub REST list
endpoints carry their next-page position only in the Link response header, so the
vendor decoder reads Link to POPULATE that single next_cursor; the cursor is then
single and authoritative on the envelope and nothing downstream re-parses Link
(ExecutionOutcome.metadata is a rate-limit-only allowlist). Because a GitHub REST
list literally cannot page without Link, this decoder reading is reported as a
question to the conductor rather than worked around.

## Scope: issues and check-runs reported, not written

The authorized scope addition (issues list + check-runs list) needs new rows in
the shared 45-op descriptors.py (owned by the W02 vendor-data slice), so the
exact rows and official endpoints are reported to the conductor for coordination
rather than written here. The wired kinds are pull requests and commits, both
already in the table with repo-scoped REST list paths.

## Tests

- test_connector_github_dispatch.py (22): the load-bearing test drives a real
  two-page gh_list_pull_requests fetch through the unmodified urllib_http_send
  over a self-signed HTTPS loopback, custody from the real encrypted SecretVault,
  asserting two pages, a genuine cursor advance, and the vault-resolved credential
  on the wire. Plus locator shaping/refusals, both decoder contracts (now
  payload-carrying), descriptor projection, denied-gate-sends-nothing,
  wrong-binding refusal.
- test_knowledge_github_structured_live.py: fetch returns REAL rows across >=2
  pages over TLS (keyed, rendered, credential on wire); detect_changes is True
  when the since-window returns rows; no-provider refuses NotImplementedError; a
  provider with no binding returns an empty-but-valid dataset.

## Incorporated dependency

PR-2 (feat/knowledge-github-structured, db7011f) supplies the structured-source
domain model — typed rows, primary keys, converters, since+key incremental diff,
resumable checkpoint, rendering. It is incorporated below this commit by
cherry-pick; PR-2 stays independently parked. This PR rebases to drop that commit
once PR-2 lands on main.

## Pattern harvest

- A vendor RequestLocator that string-concatenates a caller-supplied id into a
  URL path without a per-segment separator guard is an injection shape; the rule
  is "a path template filled from caller input encodes each segment with safe=''
  and rejects a '/' inside one segment".
- A vendor decode that returns next_cursor=None for a body it never inspected
  claims completeness it cannot know; the rule is "next_cursor is None only on a
  provider signal (no Link rel=next, or hasNextPage false), never as a default".
- A cursor carried in two places (a payload copy AND the envelope) is a second
  thing that can disagree; the rule is "the continuation lives on
  OperationResult.next_cursor and nowhere else — a payload struct carries rows,
  not a cursor" (W01 enforces this by removing CollectionPayload.next_cursor).

Non-generalizable: the pinned W01 schema-version integers (4/3/3/2) are a
coupling assertion, not a rule — dispatch.assert_schema_versions() makes a future
bump a loud compose-time failure.

Base is feat/connector-control-plane-executor (PR #11286, a non-main base), so CI
cannot fire; local gates plus the repo's review profile are the evidence until
the dependency lands on main, at which point this same PR retargets to main.
bolichen97 added a commit that referenced this pull request Sep 16, 2026
… rows fetched through W01's executor/transport

Locate a GitHub operation, dispatch it through W01's real executor and
production transport, decode the reply into the neutral payload channel, and
page a real structured fetch across two pages — with no naked sender anywhere.
Wire the knowledge GitHub structured source's live path onto that transport so a
real fetch returns real rows and a real refresh runs.

## What changed

Three new leaves under vendors/github/, plus the knowledge connector wiring and
docs, all consume-only over the W01 control plane (executor.py + production.py at
423528c; schema versions EXECUTOR=4 / PRODUCTION=3 / RESULT=3 / OPERATION=2,
pinned and asserted at compose time):

- vendors/github/locator.py: operation_id + params -> one concrete HttpRequest,
  GitHub's implementation of the transport's injected RequestLocator (mirrors the
  Graph locator). Endpoint templates come from the existing DESCRIPTORS table;
  path segments are individually validated and URL-encoded; it holds no
  credential.
- vendors/github/decoder.py: a GitHub 2xx payload -> OperationResult, GitHub's
  ResultDecode. It puts the fetched records into the ONE neutral data channel — a
  CollectionPayload for a list reply (JSON array, wrapped collection, or GraphQL
  nodes/edges), an ObjectPayload for a single object — via result_with_payload,
  and sets the SINGLE authoritative next_cursor on the envelope (the Link
  rel="next" URL for REST, the GraphQL endCursor for cursor tools). The
  collection carries no cursor of its own.
- vendors/github/dispatch.py: projects a GithubOperationDescriptor into the
  executor's OperationDescriptor, composes the production transport with GitHub's
  locator and decoder, and drives execute() and PageWalk/advance_page. The full
  W01 gate chain and per-binding custody (BindingSecretSelector) run inside the
  executor.
- knowledge/connectors/github_structured.py: PR-3 delta on top of the PR-2
  domain model. GithubStructuredConnector takes an optional
  GithubTransportProvider; fetch/detect_changes drive a REAL W01 PageWalk for the
  repo-scoped pull-request and commit entities (invoked, authorized per page, the
  walk advancing on W01's single next_cursor), read the rows off
  ExecutionOutcome.payload, convert them with PR-2's converters, fold them through
  PR-2's diff_rows, and render (text, metadata). detect_changes reports changed
  iff the since-window returned rows. No payload copy, no second envelope, no
  out-of-band capture, no cache. Built with no provider, fetch/detect_changes
  still refuse NotImplementedError (PR-2's parked contract, unchanged).

No auth, custody, retry, fencing, error classification, or pagination engine is
re-implemented — all W01's. Nothing under vendors/github/ or the connector
touches urllib / requests / httpx: the sole sender is W01's
build_production_transport.

## Paging: a single cursor, and a question

The walk advances on OperationResult.next_cursor alone. GitHub REST list
endpoints carry their next-page position only in the Link response header, so the
vendor decoder reads Link to POPULATE that single next_cursor; the cursor is then
single and authoritative on the envelope and nothing downstream re-parses Link
(ExecutionOutcome.metadata is a rate-limit-only allowlist). Because a GitHub REST
list literally cannot page without Link, this decoder reading is reported as a
question to the conductor rather than worked around.

## Scope: issues and check-runs reported, not written

The authorized scope addition (issues list + check-runs list) needs new rows in
the shared 45-op descriptors.py (owned by the W02 vendor-data slice), so the
exact rows and official endpoints are reported to the conductor for coordination
rather than written here. The wired kinds are pull requests and commits, both
already in the table with repo-scoped REST list paths.

## Tests

- test_connector_github_dispatch.py (22): the load-bearing test drives a real
  two-page gh_list_pull_requests fetch through the unmodified urllib_http_send
  over a self-signed HTTPS loopback, custody from the real encrypted SecretVault,
  asserting two pages, a genuine cursor advance, and the vault-resolved credential
  on the wire. Plus locator shaping/refusals, both decoder contracts (now
  payload-carrying), descriptor projection, denied-gate-sends-nothing,
  wrong-binding refusal.
- test_knowledge_github_structured_live.py: fetch returns REAL rows across >=2
  pages over TLS (keyed, rendered, credential on wire); detect_changes is True
  when the since-window returns rows; no-provider refuses NotImplementedError; a
  provider with no binding returns an empty-but-valid dataset.

## Incorporated dependency

PR-2 (feat/knowledge-github-structured, db7011f) supplies the structured-source
domain model — typed rows, primary keys, converters, since+key incremental diff,
resumable checkpoint, rendering. It is incorporated below this commit by
cherry-pick; PR-2 stays independently parked. This PR rebases to drop that commit
once PR-2 lands on main.

## Pattern harvest

- A vendor RequestLocator that string-concatenates a caller-supplied id into a
  URL path without a per-segment separator guard is an injection shape; the rule
  is "a path template filled from caller input encodes each segment with safe=''
  and rejects a '/' inside one segment".
- A vendor decode that returns next_cursor=None for a body it never inspected
  claims completeness it cannot know; the rule is "next_cursor is None only on a
  provider signal (no Link rel=next, or hasNextPage false), never as a default".
- A cursor carried in two places (a payload copy AND the envelope) is a second
  thing that can disagree; the rule is "the continuation lives on
  OperationResult.next_cursor and nowhere else — a payload struct carries rows,
  not a cursor" (W01 enforces this by removing CollectionPayload.next_cursor).

Non-generalizable: the pinned W01 schema-version integers (4/3/3/2) are a
coupling assertion, not a rule — dispatch.assert_schema_versions() makes a future
bump a loud compose-time failure.

Base is feat/connector-control-plane-executor (PR #11286, a non-main base), so CI
cannot fire; local gates plus the repo's review profile are the evidence until
the dependency lands on main, at which point this same PR retargets to main.
Base automatically changed from feat/connector-control-plane-seam to main September 16, 2026 16:31
@github-actions github-actions Bot added merge conflict Branch has merge conflicts with its base — author must resolve before merge readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 16, 2026
bolichen97 added a commit that referenced this pull request Sep 16, 2026
…urrent main

Re-based onto current main (6dd9db1) after #11066 squash-merged the L01
control-plane seam + W02 GitHub + W05 Graph into main and deleted the
feat/connector-control-plane-seam branch. GitHub auto-retargeted PR #11286 to
main, which then read dirty against the stale seam base.

How the alignment was decided (content/patch analysis, not commit counts)
- merge-base(prev-head, main) = a91a25f; `git cherry -v` marked all four of
  the old stacked commits `+` because #11066 was a SQUASH merge (different
  patch-id), so cherry/rebase auto-drop does not apply -- content analysis was
  required.
- Compared this stack's tree against origin/main file-by-file in the connector
  scope. operation.py / context.py / errors.py are BYTE-IDENTICAL to main (the
  L01 seam) and untouched here. The W02 (vendors/github) and W05
  (vendors/microsoft) trees are byte-identical to main and are NOT reproposed.
- The four files that "differ" from main -- control_plane/__init__.py,
  result.py, test_connections_control_plane.py, connections.md -- were each
  verified main == seam(2140476), i.e. main carries exactly the seam base this
  slice's versions were built on, with zero further main-side edits. So this
  slice's versions are strict supersets and apply as a clean fast-forward, no
  3-way conflict.
- Everything else the stale base showed as "changed" (warm.py, .github/**,
  AGENTS.md, docs/**, conftest.py, ...) was main's own 108-commit advance
  inverted by the stale base -- NOT this slice's work -- and is deliberately
  left at main's version (e.g. warm.py keeps main's newer PID-reuse hardening).

Delivered on top of main (the dependency slices this L09 work imports, none yet
in main; each preserved from its own source):
- binding.py            (L02)
- lifecycle.py          (L04, VERBATIM from f174887fc; byte-identical, sha256
                         90b4931a...), plus the connective BindingResolutionError
                         class added to binding.py so lifecycle.py can import it
- auth_modes.py         (L05)
- policy.py             (L06)
- writes.py             (L07)
- handle.py             (L08)
- executor.py, production.py + the neutral data channel additions to result.py /
  control_plane __init__.py + connections.md   (L09, this slice)
- test_connections_control_plane*.py suites

L09 send path (this slice's core work)
- build_production_transport takes store: BindingStore + gate: BindingCustodyGate
  (was selector: BindingSecretSelector). Per call: (1) gate custody match refuses
  before locator/store/vault; (2) locator; (3) store.select_secret(binding,
  reader=vault) FENCES against the LIVE store (revoked / stale-or-forged
  generation / swapped identity/mode/secret_ref refuse) and takes secret_ref from
  the STORE's record. binding_secret_ref is neither imported nor called in
  production.py (AST-checked). A fence refusal / resolution failure emits ZERO
  calls. PageWalk re-runs the whole chain per page.

Evidence (real TLS server + real on-disk SecretVault + real BindingStore)
- two bindings -> two different secrets on the wire; B's identity refused with 0
  calls; revoke -> next send 0 calls; generation rotation -> 0 calls; no slug
  fallback (AST + absent-entry refusal); PageWalk revoke between page 1 and 2 ->
  page 2 emits 0 calls, cursor path never requested. Mutation-verified: the
  revoked flag and the generation check are redundant load-bearing fences.

Invariants (all green): four judgments before dispatch, deny -> 0 calls; routing
only via TrustedHandleView; 412 structured, no fabricated precondition;
500/timeout/exception -> write_outcome unknown; redirects per-hop with
cross-origin credential strip and no https downgrade; response metadata
allowlisted (no credential/session headers); single authoritative cursor
OperationResult.next_cursor; three payload shapes, Office bytes byte-equal.

Schema: EXECUTOR_SCHEMA_VERSION=4, PRODUCTION_SCHEMA_VERSION=4,
RESULT_SCHEMA_VERSION=3, LIFECYCLE_SCHEMA_VERSION=1.

Named in-flight boundaries (NOT implemented, not claimed): L03 (auth
code/PKCE/state/callback replay protection) is still required and unimplemented.
The principal->binding ACL adapter (BindingResolver.resolve ->
AccessContext, whose signature differs from lifecycle.BindingStore.resolve) is
L04's follow-up, not wired here. MS 412/readback/locator semantics
(vendors/microsoft/**) and document/provider-level query ACL remain other
owners' work.

Gates: 295 passed; black/flake8/isort/mypy clean.
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from 0591b97 to 1648430 Compare September 16, 2026 17:50
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 16, 2026
bolichen97 added a commit that referenced this pull request Sep 16, 2026
…ctors

The structured vendor connectors (GitHub/Google/Salesforce) are NOT constructed
no-arg: each takes an injected runner/factory (Google operations_factory, SF
call_runner) that composes the W01 control-plane executor with per-source
credential custody + real transport (controlled TLS). Constructed without it the
connector's own validate/fetch refuse fail-closed, so a no-arg registration
would be a dead, misleading entry.

_register_optional_connector now takes runner_factory and constructs the
connector WITH it; setup_knowledge_routes reads a {source_type: runner_factory}
map from app['knowledge_connector_runners'] (installed by the host once the W01
executor, PR #11286, is available) and registers a vendor ONLY when its module
imports AND its runner is installed. Absent module or absent runner => not
registered (fail-closed, never public). No vendor code copied.

Adds an end-to-end test: the handler seam constructs a connector WITH an
injected runner and the SAME instance drives the real SyncScheduler ->
fetch_rows -> ingest_rows -> store -> ACL query chain (scripted runner over the
documented offline path; live W01 TLS is PR #11286's executor, not on main).
@github-actions github-actions Bot added the readiness: action required A blocking check or review needs attention label Sep 16, 2026
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from 4d050bd to af3f04c Compare September 16, 2026 23:03
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 16, 2026
bolichen97 added a commit that referenced this pull request Sep 16, 2026
… rows fetched through W01's executor/transport

Locate a GitHub operation, dispatch it through W01's real executor and
production transport, decode the reply into the neutral payload channel, and
page a real structured fetch across two pages — with no naked sender anywhere.
Wire the knowledge GitHub structured source's live path onto that transport so a
real fetch returns real rows and a real refresh runs.

## What changed

Three new leaves under vendors/github/, plus the knowledge connector wiring and
docs, all consume-only over the W01 control plane (executor.py + production.py at
423528c; schema versions EXECUTOR=4 / PRODUCTION=3 / RESULT=3 / OPERATION=2,
pinned and asserted at compose time):

- vendors/github/locator.py: operation_id + params -> one concrete HttpRequest,
  GitHub's implementation of the transport's injected RequestLocator (mirrors the
  Graph locator). Endpoint templates come from the existing DESCRIPTORS table;
  path segments are individually validated and URL-encoded; it holds no
  credential.
- vendors/github/decoder.py: a GitHub 2xx payload -> OperationResult, GitHub's
  ResultDecode. It puts the fetched records into the ONE neutral data channel — a
  CollectionPayload for a list reply (JSON array, wrapped collection, or GraphQL
  nodes/edges), an ObjectPayload for a single object — via result_with_payload,
  and sets the SINGLE authoritative next_cursor on the envelope (the Link
  rel="next" URL for REST, the GraphQL endCursor for cursor tools). The
  collection carries no cursor of its own.
- vendors/github/dispatch.py: projects a GithubOperationDescriptor into the
  executor's OperationDescriptor, composes the production transport with GitHub's
  locator and decoder, and drives execute() and PageWalk/advance_page. The full
  W01 gate chain and per-binding custody (BindingSecretSelector) run inside the
  executor.
- knowledge/connectors/github_structured.py: PR-3 delta on top of the PR-2
  domain model. GithubStructuredConnector takes an optional
  GithubTransportProvider; fetch/detect_changes drive a REAL W01 PageWalk for the
  repo-scoped pull-request and commit entities (invoked, authorized per page, the
  walk advancing on W01's single next_cursor), read the rows off
  ExecutionOutcome.payload, convert them with PR-2's converters, fold them through
  PR-2's diff_rows, and render (text, metadata). detect_changes reports changed
  iff the since-window returned rows. No payload copy, no second envelope, no
  out-of-band capture, no cache. Built with no provider, fetch/detect_changes
  still refuse NotImplementedError (PR-2's parked contract, unchanged).

No auth, custody, retry, fencing, error classification, or pagination engine is
re-implemented — all W01's. Nothing under vendors/github/ or the connector
touches urllib / requests / httpx: the sole sender is W01's
build_production_transport.

## Paging: a single cursor, and a question

The walk advances on OperationResult.next_cursor alone. GitHub REST list
endpoints carry their next-page position only in the Link response header, so the
vendor decoder reads Link to POPULATE that single next_cursor; the cursor is then
single and authoritative on the envelope and nothing downstream re-parses Link
(ExecutionOutcome.metadata is a rate-limit-only allowlist). Because a GitHub REST
list literally cannot page without Link, this decoder reading is reported as a
question to the conductor rather than worked around.

## Scope: issues and check-runs reported, not written

The authorized scope addition (issues list + check-runs list) needs new rows in
the shared 45-op descriptors.py (owned by the W02 vendor-data slice), so the
exact rows and official endpoints are reported to the conductor for coordination
rather than written here. The wired kinds are pull requests and commits, both
already in the table with repo-scoped REST list paths.

## Tests

- test_connector_github_dispatch.py (22): the load-bearing test drives a real
  two-page gh_list_pull_requests fetch through the unmodified urllib_http_send
  over a self-signed HTTPS loopback, custody from the real encrypted SecretVault,
  asserting two pages, a genuine cursor advance, and the vault-resolved credential
  on the wire. Plus locator shaping/refusals, both decoder contracts (now
  payload-carrying), descriptor projection, denied-gate-sends-nothing,
  wrong-binding refusal.
- test_knowledge_github_structured_live.py: fetch returns REAL rows across >=2
  pages over TLS (keyed, rendered, credential on wire); detect_changes is True
  when the since-window returns rows; no-provider refuses NotImplementedError; a
  provider with no binding returns an empty-but-valid dataset.

## Incorporated dependency

PR-2 (feat/knowledge-github-structured, db7011f) supplies the structured-source
domain model — typed rows, primary keys, converters, since+key incremental diff,
resumable checkpoint, rendering. It is incorporated below this commit by
cherry-pick; PR-2 stays independently parked. This PR rebases to drop that commit
once PR-2 lands on main.

## Pattern harvest

- A vendor RequestLocator that string-concatenates a caller-supplied id into a
  URL path without a per-segment separator guard is an injection shape; the rule
  is "a path template filled from caller input encodes each segment with safe=''
  and rejects a '/' inside one segment".
- A vendor decode that returns next_cursor=None for a body it never inspected
  claims completeness it cannot know; the rule is "next_cursor is None only on a
  provider signal (no Link rel=next, or hasNextPage false), never as a default".
- A cursor carried in two places (a payload copy AND the envelope) is a second
  thing that can disagree; the rule is "the continuation lives on
  OperationResult.next_cursor and nowhere else — a payload struct carries rows,
  not a cursor" (W01 enforces this by removing CollectionPayload.next_cursor).

Non-generalizable: the pinned W01 schema-version integers (4/3/3/2) are a
coupling assertion, not a rule — dispatch.assert_schema_versions() makes a future
bump a loud compose-time failure.

Base is feat/connector-control-plane-executor (PR #11286, a non-main base), so CI
cannot fire; local gates plus the repo's review profile are the evidence until
the dependency lands on main, at which point this same PR retargets to main.
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 16, 2026
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from af3f04c to 573cf0f Compare September 16, 2026 23:54
@github-actions github-actions Bot added readiness: checking Automated validation is still running and removed readiness: action required A blocking check or review needs attention labels Sep 16, 2026
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from 573cf0f to b3efa41 Compare September 16, 2026 23:58
@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Sep 17, 2026
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from b3efa41 to 38b6c03 Compare September 17, 2026 01:24
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 17, 2026
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from 38b6c03 to 918ee05 Compare September 17, 2026 03:17
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: action required A blocking check or review needs attention and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Sep 17, 2026
@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Security Scope Review -- 🟡 CONCERNS

Which legitimate operations does 7effc4c534026c0ebe6db39022c0ccfe20ab7071 newly refuse? Updated in place on each push. A script-confirmed regression blocks; the model proposes candidates and never decides a refusal.

Verdict basis: the reviewer's own judgement; no confirmed regression.

Security scope review -- adjudicated candidates

  • macos-latest (platform posix): 11 classified, 0 newly refused, 11 still allowed.
  • ubuntu-latest (platform posix): 11 classified, 0 newly refused, 11 still allowed.
  • windows-latest (platform windows): 6 classified, 0 newly refused, 6 still allowed.

No candidate flipped from allowed to refused.

Candidates written. The four tiers the classifier adjudicates are textually unchanged by this diff — the tightening's actual enforcement points are the file-edit write gate and the OS sandbox mount, neither of which a shell-command verdict can reach. Emitting the review.

Scope-Verdict: CONCERNS

TIGHTENING: A new path fence around the connections control-plane binding store — the top-level control-plane-bindings/ data-home directory is added to _WRITE_PROTECTED_HOME_PATHS (src/kiro_crew/security/paths.py, file-edit tool gate) and to the OS sandbox's read-only, pre-created, NOFOLLOW leaves (src/kiro_crew/sandbox.py _CREW_READONLY_LEAVES / _CREW_PRECREATE_READONLY_DIR_LEAVES / _CREW_NOFOLLOW_READONLY_DIR_LEAVES) — refusing the one threat the hunk names: "a sandboxed agent that could WRITE a forged record here would have the resolver trust it and grant access to an account it was never authorized for" (a forged bindings.json record that BindingStore.assert_live/select_secret would trust as the single source of a connector credential's binding/secret_ref).

ADJUDICATED: 16 candidates, 0 confirmed newly-refused
No confirmed rows were handed back for this run. Consistent with that, none of the four checks the classifier applies to a shell command line (sensitive_path_refusal, is_sensitive_bash_command, audit_bash_exfiltration, is_denied) is textually changed by this diff — the new fence entries feed is_sensitive_write_path (edit-gate only) and the sandbox mask, so the adjudicated composite is expected to classify identically at base and head. The candidates still probe the boundary: store reads (the write-only tier's read allowance), the lock sidecar, KIROCREW_HOME re-anchoring, the fenced name as grep data, the new test lane, and same-name writes into maintainer-owned scratch homes outside the fence.

PLATFORM COVERAGE: posix (10 rows) — home-tilde reads of the store and lock, $KIROCREW_HOME env spelling, tar backup sweep, the sanctioned pytest invocation for the new sandbox-mask test, and scratch-home mkdir/redirect writes, covering the posix-separator, home-anchored entries the diff authors. windows (5 rows) — %USERPROFILE% and %KIROCREW_HOME% backslash spellings, type/dir, .venv\Scripts\python.exe test lane, and a %TEMP% cmd-redirection scratch write, covering the other tokenizer for every path assumption the diff makes (the entries are authored with / separators and matched via the shared both-separator anchoring). any (1 row) — the platform-neutral grep. One assumption no shell candidate can probe on any host: the Linux bind-mount seal itself, which refuses at the OS rather than at any classifiable gate — named below. Per-leg verdicts are the deterministic fold's to report, not this section's.

UNADJUDICATED:

  1. The file-edit write gate. The new _WRITE_PROTECTED_HOME_PATHS entries ({.kiro/crew,.kirocrew}/control-plane-bindings, plus the KIROCREW_HOME re-anchor) are consulted only on the edit path: hooks.on_tool_call → the is_edit_call branch (src/kiro_crew/hooks.py:1000) → edit_target_candidates → is_sensitive_write_path (src/kiro_crew/hooks.py:1046) → _path_in_home_dirs(..., _SENSITIVE_HOME_DIRS + _WRITE_PROTECTED_HOME_PATHS). The classifier's path-fence tier is sensitive_path_refusal, which reads _SENSITIVE_HOME_DIRS alone — unchanged — so no shell verdict can settle this guard. Candidate legitimate inputs I would have submitted: a file-edit targeting the agent-writable sibling ~/.kiro/crew/connections/ (the mint manifest / status state file — the directory the store was deliberately moved out of, which must stay editable); an edit of a workspace file inside a project directory named control-plane-bindings/ (the fence must stay home-anchored, not name-anchored); an edit of ~/.kiro/crew/control-plane-bindings.md (shares the prefix without a separator — the entry + os.sep boundary of the shared matcher); the Windows spelling C:\Users\me\.kiro\crew\connections\state.json.
  2. The OS sandbox seal. The control-plane-bindings leaf is now bind-mounted read-only, pre-created, and NOFOLLOW for the entire sandboxed process tree — a refusal delivered by the kernel (EROFS), not by any gate the classifier can read. Candidate legitimate inputs: (a) a BindingStore.insert/rotate/revoke executed by a Crew process that itself runs inside the sandbox — lifecycle.py explicitly contemplates "a scheduled refresh racing an on-demand one", and sandbox.py's own comment says boot_platform() resolves keystones inside the sandbox for a script cron; if any scheduled refresh runs from a sandboxed cron, rotation writes die at the mount with no gate refusal to diagnose from. (b) a user whose control-plane-bindings directory is a symlink (data home on shared or synced storage): the leaf is NOFOLLOW, the same class of loss the playwright-cli precedent accepts, but here it lands on credential rotation rather than a bundled executable. Both need a human to exercise; neither could be encoded as a shell row.

[SCOPE-REVIEWED] 7effc4c

…urrent main

Re-based onto current main (6dd9db1) after #11066 squash-merged the L01
control-plane seam + W02 GitHub + W05 Graph into main and deleted the
feat/connector-control-plane-seam branch. GitHub auto-retargeted PR #11286 to
main, which then read dirty against the stale seam base.

How the alignment was decided (content/patch analysis, not commit counts)
- merge-base(prev-head, main) = a91a25f; `git cherry -v` marked all four of
  the old stacked commits `+` because #11066 was a SQUASH merge (different
  patch-id), so cherry/rebase auto-drop does not apply -- content analysis was
  required.
- Compared this stack's tree against origin/main file-by-file in the connector
  scope. operation.py / context.py / errors.py are BYTE-IDENTICAL to main (the
  L01 seam) and untouched here. The W02 (vendors/github) and W05
  (vendors/microsoft) trees are byte-identical to main and are NOT reproposed.
- The four files that "differ" from main -- control_plane/__init__.py,
  result.py, test_connections_control_plane.py, connections.md -- were each
  verified main == seam(2140476), i.e. main carries exactly the seam base this
  slice's versions were built on, with zero further main-side edits. So this
  slice's versions are strict supersets and apply as a clean fast-forward, no
  3-way conflict.
- Everything else the stale base showed as "changed" (warm.py, .github/**,
  AGENTS.md, docs/**, conftest.py, ...) was main's own 108-commit advance
  inverted by the stale base -- NOT this slice's work -- and is deliberately
  left at main's version (e.g. warm.py keeps main's newer PID-reuse hardening).

Delivered on top of main (the dependency slices this L09 work imports, none yet
in main; each preserved from its own source):
- binding.py            (L02)
- lifecycle.py          (L04, VERBATIM from f174887fc; byte-identical, sha256
                         90b4931a...), plus the connective BindingResolutionError
                         class added to binding.py so lifecycle.py can import it
- auth_modes.py         (L05)
- policy.py             (L06)
- writes.py             (L07)
- handle.py             (L08)
- executor.py, production.py + the neutral data channel additions to result.py /
  control_plane __init__.py + connections.md   (L09, this slice)
- test_connections_control_plane*.py suites

L09 send path (this slice's core work)
- build_production_transport takes store: BindingStore + gate: BindingCustodyGate
  (was selector: BindingSecretSelector). Per call: (1) gate custody match refuses
  before locator/store/vault; (2) locator; (3) store.select_secret(binding,
  reader=vault) FENCES against the LIVE store (revoked / stale-or-forged
  generation / swapped identity/mode/secret_ref refuse) and takes secret_ref from
  the STORE's record. binding_secret_ref is neither imported nor called in
  production.py (AST-checked). A fence refusal / resolution failure emits ZERO
  calls. PageWalk re-runs the whole chain per page.

Evidence (real TLS server + real on-disk SecretVault + real BindingStore)
- two bindings -> two different secrets on the wire; B's identity refused with 0
  calls; revoke -> next send 0 calls; generation rotation -> 0 calls; no slug
  fallback (AST + absent-entry refusal); PageWalk revoke between page 1 and 2 ->
  page 2 emits 0 calls, cursor path never requested. Mutation-verified: the
  revoked flag and the generation check are redundant load-bearing fences.

Invariants (all green): four judgments before dispatch, deny -> 0 calls; routing
only via TrustedHandleView; 412 structured, no fabricated precondition;
500/timeout/exception -> write_outcome unknown; redirects per-hop with
cross-origin credential strip and no https downgrade; response metadata
allowlisted (no credential/session headers); single authoritative cursor
OperationResult.next_cursor; three payload shapes, Office bytes byte-equal.

Schema: EXECUTOR_SCHEMA_VERSION=4, PRODUCTION_SCHEMA_VERSION=4,
RESULT_SCHEMA_VERSION=3, LIFECYCLE_SCHEMA_VERSION=1.

Named in-flight boundaries (NOT implemented, not claimed): L03 (auth
code/PKCE/state/callback replay protection) is still required and unimplemented.
The principal->binding ACL adapter (BindingResolver.resolve ->
AccessContext, whose signature differs from lifecycle.BindingStore.resolve) is
L04's follow-up, not wired here. MS 412/readback/locator semantics
(vendors/microsoft/**) and document/provider-level query ACL remain other
owners' work.

Gates: 295 passed; black/flake8/isort/mypy clean.
@bolichen97
bolichen97 force-pushed the feat/connector-control-plane-executor branch from 918ee05 to 0929255 Compare September 17, 2026 03:38
@github-actions github-actions Bot removed the readiness: action required A blocking check or review needs attention label Sep 17, 2026
…re seal + GPT round fixes

W01 control-plane, on current main (04727eb). Second commit of the 2-commit stack,
folding the a8 L04 integration, L08's F3, the F2 store seal, and this round's GPT
review-response fixes.

GPT 5.6 round on the prior head raised three findings in my code; all addressed:

F1 (production.py response_metadata) — a reflected credential no longer has to be
RAW to be caught. The kept-value screen now drops a header value containing the
outbound credential in ANY form: raw, base64 (standard or url-safe, with or without
padding), or percent-encoded (_credential_echo_forms). Closes the FLAGged encoded-echo
escape without relaxing the allowlist. Regression + mutation (raw-only screen fails
the encoded-echo test).

F2 (production.py decode_json_body) — a NON-EMPTY 2xx body that is not valid UTF-8
JSON, or parses to a non-object, is now REFUSED with the new MalformedResponseBodyError
instead of collapsing to {}. A silent {} on a 2xx is indistinguishable from a genuine
empty body, so a caller read a reported success carrying no data with no retry signal
— silent data loss (truncation, 200-served HTML error pages, gzip faults). An EMPTY
body is still a real empty result and yields {}. Regression + mutation (revert-to-{}
fails the refuse tests).

executor.py top-level-imports — `import logging` and `from kiro_crew.sel import
SecurityEvent, sel` moved to module scope (kiro_crew.sel is stdlib-only and
same-package, so no import-time dependency risk). Only the SEL USE stays guarded, so a
runtime SEL fault still soft-fails a decided call. The SEL audit test now patches the
executor module's `sel` name (module-top binding).

F2 store-seal real-kernel test (test_sandbox_governance_mask.py) — a NEW test spawns
the real namespace launcher (sandbox.namespace_argv) and records a child's ACTUAL write
outcome: store/lock/temp writes must be denied by the kernel read-only bind while the
store stays readable, and a legitimate pre-sandbox backend write works. It SKIPS with
the reason when unprivileged user/mount namespaces are unavailable (a launch that never
sandboxed is not a write-denial pass), distinguishing env-launch-failure from an
effective seal. macOS (Seatbelt text) and Windows (no launcher seal) remain out of
scope and are NOT claimed complete.

Preserved: F1 deep-frozen carrier (incl. nested secret_ref), 500 in
_AMBIGUOUS_TRANSIT_STATUSES, L08 F3 (HandleTamperedError), and all prior tests.
…e rows + sync + query-time ACL (D1) (#11349)

* fix(connections): L09 disconnect-unknown + #11286 CI reds (CodeQL, F1/F2/F4, gate hygiene) on current main

Combined fix commit above the aligned base 1648430 (W01 L02/L04/L05/L06/L07/L08/L09
control plane re-based onto current main). Folds the server-committed-then-dropped
disconnect fix and this round's CI-red resolutions into one commit; the tree is
byte-for-byte identical to the pre-squash HEAD 9d3c2efa0.

Disconnect -> unknown (production.py): the ambiguous-outcome `except` tuple also
names `ConnectionError` and `http.client.HTTPException`, so a raw
`http.client.RemoteDisconnected` (a ConnectionResetError AND a BadStatusLine, but
NOT a URLError) records `write_outcome=unknown` and the L07 replay gate refuses a
blind re-send. Real-TLS + injected-sender regressions; mutation-verified.

CodeQL (3 highs, this slice's tests): TLS test servers pin
`minimum_version = TLSVersion.TLSv1_2`; the fencing test stops logging the secret
value (references the name, asserts it resolved).

F2 (GPT BLOCKING, real): an allowlisted response header whose VALUE echoes the
outbound bearer would reach the caller via ExecutionOutcome.metadata. The
allowlist gates the NAME; `response_metadata` now takes the outbound
`sent_credential` and drops any kept value that contains it. Mutation-verified.

F4 (GPT BLOCKING, real, test): the two rotation workers run in their own process
group (start_new_session), cwd under tmp_path, and are terminated+waited (then
killed) in a finally on every exit path.

F1 (integration miss, not new work): adopted L02's binding.py VERBATIM from
`e8e11aa23` (byte-identical) — `binding_scoped_secret_ref` / `_secret_scope` /
`resolve_binding_for_principal`, and `create_binding` records a per-binding
scoped secret_ref (retains `BindingResolutionError` that lifecycle.py imports).
Test harness reconciled to the per-binding scheme. F3 (lifecycle.py:347) is L04's.

Gate hygiene (added lines): Brand Name `KiroCrew`->`Kiro Crew`; 32 history-narration
comment markers reworded to present tense across 11 files. Sibling-slice files
edited by authorization, COMMENTS ONLY (AST-invariant): lifecycle.py (code AST ==
f174887fc), auth_modes.py / writes.py (code AST == e8e11aa23), handle test.

Integrated slices carried here (not yet in main): L02 binding (e8e11aa23-derived),
L04 lifecycle (lifecycle.py verbatim from f174887fc), L05 auth_modes, L06 policy,
L07 writes, L08 handle; L09 executor/production/result/data-channel.

Schema: RESULT=3, EXECUTOR=4, PRODUCTION=4, LIFECYCLE=1.
Gates: 299 passed; black/flake8/isort/mypy clean (rc non-piped). No vendors/**,
platform/governance.py, or registry changes.
Named in-flight boundary: L03 (auth code/PKCE/state/callback replay protection)
still required and unimplemented; F3 belongs to L04.

* feat(knowledge): query-time ACL + revocation isolation in the retrieval chain

The shared Knowledge Library retriever had no notion of a querying identity:
HybridRetriever.search's source_id/namespace filters are relevance labels (their
own docstrings say NOT a security boundary), the graph leg was entirely
unfiltered, and the store schema carried no principal/tenant/grant state. A KB
that ingests shared sources under an admin/service identity could therefore leak
one user's content to another at query time.

Add a fail-closed query-time ACL gate wired into the real retrieval chain:

- knowledge/acl.py: AccessContext (verified subject+tenant, from the
  authenticated caller, never self-reported) + AclPolicy. SubjectTenantAclPolicy
  denies a missing/unreadable/revoked grant, a tenant mismatch (same email in a
  different tenant is a different identity), and a subject not on the grant list;
  the only allow-all is the explicit local single-user context.
- store.py: item_acl table (subjects/tenant/acl_version), set/revoke/batch-read
  APIs. revoke keeps the row with an empty subject set and bumps acl_version so
  the next query denies and any version-keyed cache is invalidated; the row is
  removed with its item so a grant never outlives the content.
- retrieval.py: search() gains access_context and gates every fused candidate
  (keyword + vector + the unfiltered graph leg + the protected keyword rescue +
  citation/location enrichment) BEFORE the result window is cut. No context =>
  local single-user bypass, preserving the personal-library behaviour.
- mcp_tools/knowledge.py + dashboard/handlers/knowledge.py: pass an explicit
  AccessContext at the call boundary (local library = single-user context); a
  shared/multi-tenant caller resolves subject+tenant from the authenticated
  caller and passes a real context.

Tests: test_knowledge_query_acl.py drives the real store+gate over two
identities, cross-tenant, same-bare-id-different-source, revoke-then-query,
cache-invalidation, unreadable-grant fail-closed, graph-leg leakage, and
backward-compat. Existing call-site assertions updated for the new kwarg.

* fix(knowledge): item-scoped ACL bypass + managed-item revalidation chain

Root re-read the first cut and found two boundary holes; fix both before ship.

Root cause 1 -- the bypass was call-surface-scoped, not item-scoped. The local
single-user context skipped the grant read and admitted EVERY candidate, but one
KnowledgeStore mixes trusted-local material with managed cloud/structured items,
so a 'personal dashboard' path could still surface a managed item with no check.

  - acl.py: bypass is no longer 'see everything'. is_managed_source_type() +
    an explicit MANAGED_SOURCE_TYPES set classify each item; the retriever
    re-derives managed from the LIVE source (grant flag OR managed type OR a
    dangling source row = fail-closed), while a sourceless/local item stays
    trusted-local. LOCAL_LIBRARY (renamed from ALLOW_ALL, kept as an alias) now
    sees trusted-local items without a grant but is DENIED managed items -- it
    carries no verifiable provider-mapped subject. A managed item is always
    checked against a real current subject regardless of entry point.
  - retrieval._acl_filter: per-item classify (sourceless/local vs managed vs
    dangling) then gate; the graph leg, keyword rescue and enrichment all consume
    the filtered list.
  - store.get_item_source_types now returns (has_source, source_type) so the
    gate can tell a sourceless local item from a dangling managed one.

Root cause 2 -- a static ingest grant + a locally-bumped acl_version is not proof
the provider still grants access. Add a revalidation/staleness chain:

  - acl.RevalidationHook (the provider live-permission interface -- a named
    DEPENDENCY, not implemented here) + RevalidationOutcome{FRESH,REVOKED,
    UNVERIFIABLE}. A managed grant is served only when FRESH from the hook OR its
    stored fresh_as_of is within the staleness window. REVOKED denies even a
    subject-matching grant; a wired hook returning/raising UNVERIFIABLE is a hard
    deny (no fall-back to the stored stamp); with NO hook wired a stale/
    never-revalidated managed grant is denied. A static grant can never pose as
    a live check.
  - store: item_acl gains managed + fresh_as_of; set_item_acl takes both;
    revoke resets fresh_as_of; new mark_item_acl_revalidated records a provider
    confirmation (or observed revocation). Migration backfills legacy rows
    (managed=0/fresh_as_of=0 => a managed legacy grant is stale until refreshed).

Tests: test_knowledge_query_acl rewritten to drive the REAL store+gate over a
MIXED library (local doc readable, managed cloud doc gated), the sourceless /
dangling-source classification, and the full revalidation chain (hook
fresh/revoked/error, staleness, mark-revalidated). 818 knowledge tests pass in a
clean source-tree-external venv; flake8 clean.

W01 note: a managed item's AccessContext subject/tenant must be the
PROVIDER-mapped identity (Binding.subject_ref/tenant_ref from the verifier), not
a raw session key; the caller boundary resolves it and the revalidation hook is
where the provider probe lives. Neither is faked here.

* fix(knowledge): classify managed-vs-local from source provenance, not a type list

Root's third read found the second cut still fail-open: MANAGED_SOURCE_TYPES was
an enumeration and is_managed_source_type only tested membership, with the doc
saying 'unknown type => trusted-local'. A new/misspelled cloud source_type, or a
managed source whose per-item grant forgot managed=True, was therefore treated
as local -- the same hole, relocated from the whole call surface to
unenumerated types. The store type list I cited as an 'admitted-local
authoritative set' was actually the orphan-reclaim GC rule, not a trust
directory.

Redo the classification on PROVENANCE evidence, fail-closed by construction:

- store: sources gains a trust_class column (local_admitted | managed), written
  ONCE at creation from the creating path's own knowledge (initial_trust_class),
  defaulting to MANAGED for anything not a known LOCAL production creator
  (local_folder/local_file/obsidian_vault/artifact/agent/doc -- the real on-host
  ingestion creators, not a runtime type guess). A remote connector source is
  created managed. Migration adds the column defaulting managed and backfills
  local_admitted onto exactly those local-creator types -- one-time migration
  semantics, gated on the column being freshly added.
- acl: replace is_managed_source_type with is_managed_trust_class(has_source,
  trust_class). Sourceless => local; local_admitted stamp => local; managed /
  unstamped / unknown / dangling (missing source row => trust_class None) =>
  managed. The query-time gate reads the STAMP only; it never re-guesses trust
  from a source_type string, so a new/misspelled cloud type cannot be waved
  through. The per-item grant's managed flag is still OR'd in as a backstop.
- retrieval: _acl_filter classifies from get_item_trust (has_source,
  trust_class); get_item_grants also returns trust_class.

Adversarial regressions added: a brand-new unknown source_type is stamped
managed and gated; a managed source whose grant omitted managed=True is still
gated (revoke hook denies it); migration backfill verified. 1004 knowledge
tests pass in a clean source-tree-external venv; end-to-end provenance +
unknown-type + migration validation from that venv; flake8 clean.

The revalidation-chain fix from the prior commit stands: fresh_as_of is only
meaningful as a trusted confirmation record (set_item_acl/mark_item_acl_revalidated),
and with no provider resolver/revalidator wired a managed grant is denied --
the fresh flag/manual mark is NOT treated as a completed real-time revocation
check, and no other session/binding identity stands in for the current querying
subject.

* docs(knowledge): flag sourceless-item trust axis for central review; model sourced fixtures

Per Root's brief, the 'no source does not prove no external ACL' axis needs a
CENTRAL independent review, not another list tweak. Rather than silently flip
sourceless->managed (which regresses the single-user local library and breaks
the retriever's sourceless test idiom) or silently keep it, this documents the
axis in is_managed_trust_class (sourceless stays trusted-local for now, with the
rationale -- no managed connector produces a sourceless item -- and the
conservative alternative -- flip to managed + migrate legacy sourceless rows --
spelled out and held for the review), and keeps the keyword_leg fixtures on a
local_folder source (models production: every ingested item has a source),
correct under either resolution.

No behaviour change vs the pushed head for managed items. 795 knowledge tests
pass; flake8 clean.

* fix(knowledge): sourceless=managed per decision; trusted stamp issuer; provenance survives bundle round-trip

Root decided the reviewed axis: 'no source therefore trusted-local' is NOT a
shared-query authorization rule. Implement it fail-closed and harden the stamp
against forgery + preserve managed provenance across every path.

- acl.is_managed_trust_class: sourceless (has_source False) -> MANAGED. Only a
  source stamped local_admitted is trusted-local. Absence of provenance evidence
  is not local; the test habit of omitting a source is not production trust.
- store.add_source: the local_admitted stamp is issued from the creator TYPE, not
  a caller-assertable field. An explicit trust_class kwarg may only NARROW to
  managed; a local_admitted request for a non-local type is refused down to
  managed. External API input / a source_type rename / a forged kwarg cannot mint
  local trust -- the stamp does not prove the issuer, the creator type does.
- Managed provenance is durable: re-label/dedup that moves items.source_id never
  rewrites item_acl.managed (no path does), so a managed item reassigned under a
  local source stays gated. update_source cannot mutate trust_class (not in
  _SOURCE_COLUMNS).
- export_all/import_bundle now carry trust_class AND item_acl grants, so a bundle
  round-trip does not silently un-gate a managed item. A bundled trust_class is
  re-checked by issuer (local only for a local creator type); a bundled managed
  grant imports with fresh_as_of forced to 0 (stale) so it must be re-revalidated
  on the importing store, never served off the exporter's old confirmation.

Tests: sourceless->managed, forged-stamp-refused, managed-survives-reassignment,
bundle-round-trip (local stays visible, managed stays stale-denied) added to
test_knowledge_query_acl. Retriever-mechanics tests that used sourceless items
migrated to a trusted-local source (models production; every ingested item has a
source) -- an isolated-fixture test migration, not a live-library change and not
a gate relaxation. 1306 knowledge tests pass; flake8 clean; source-tree-external
e2e validated.

* feat(knowledge): make the provider-revalidation input contract explicit

Per Root: form the per-source service / resource-locator / credential-binding
inputs the RevalidationHook needs and their call sites, not just 'wait for W01'.

Add two named input structures to acl.py:
- ProviderResourceRef(provider, account, resource_id, locator): WHICH provider
  object a managed item came from -- the resource locator a live-permission
  probe needs. Documents the minimal locator per provider family
  (sharepoint/onedrive/google_drive/salesforce/github_structured). This is
  ingest-side provenance the connector owner must persist alongside the grant.
- RevalidationRequest(ctx, item_id, grant, resource, credential_ref): the full
  input a hook needs. ctx MUST be the provider-mapped subject/tenant (W01
  Binding.subject_ref/tenant_ref), not a raw session key and not one binding for
  every source. credential_ref is an OPAQUE handle (vault key / connection id)
  the host resolves at probe time -- never a raw secret -- and which binding is
  correct is W01's trusted association for (provider, account, subject).

The retriever still calls revalidate(ctx, item_id, grant) (the keys it holds at
query time); the hook docstring now spells out that a real implementation looks
up the resource ref + credential by item_id to build the full request, and that
those two are the named ingest-side + W01 dependencies -- until they exist the
hook cannot answer FRESH and every managed grant stays denied (fail-closed).

flake8 clean; ACL suite green.

* feat(knowledge): wire query-time ACL identity + revalidator seam into dashboard search

Root returned the dashboard/handlers/knowledge.py write slot (GitHub owner's
registration import+map is saved separately on their branch; not merged/copied
here). Turn the two dashboard search call sites from a hardcoded local-library
bypass into a real, fail-closed wiring seam:

- _knowledge_access_context(request): resolves the query-time AccessContext. Uses
  an installed app['knowledge_identity_resolver'] (the W01-backed seam that maps
  the authenticated caller to a provider subject/tenant) when present; otherwise
  the on-host personal library runs under acl.LOCAL_LIBRARY (trusted-local
  visible, managed denied). A resolver that returns None or raises fails closed
  to LOCAL_LIBRARY -- never an exception out of the handler, never a
  managed-item leak. It never derives a managed identity from a raw dashboard/
  session id; that is the resolver's job and its absence keeps managed denied.
- _knowledge_revalidator(request): pulls an optional acl.RevalidationHook off
  app['knowledge_revalidator']; None until a deployment installs one (managed
  grants then fall back to their staleness stamp -> stale -> denied).
- Both threaded into list_items search, _search_until_exhausted, and
  search_for_context: HybridRetriever(..., revalidator=...) +
  search(access_context=...). No call site hardcodes the context any more.

So a shared/multi-tenant deployment installs the resolver + hook and the same
call sites enforce provider-mapped ACL with live revalidation, with zero further
handler changes; the personal library keeps working unchanged today.

Tests: test_knowledge_query_acl_wiring covers resolver present/absent/None/raises
and revalidator present/absent; existing HybridRetriever stubs updated for the
new revalidator kwarg. 1313 knowledge tests pass; flake8 clean; source-tree-
external e2e shows the seam feeding the resolved context + revalidator into the
gate (fresh->visible, revoked->denied, local-lib->managed denied).

* feat(knowledge): per-candidate provider binding + persist ProviderResourceRef

Root's decision: one request must NOT resolve a single provider identity for the
whole library, and persisting the ProviderResourceRef is this shared store's own
responsibility (not an external blocker). GitHub is issue/PR/commit/check-run;
Graph carries full container/tenant/endpoint.

Store (my responsibility, implemented now):
- item_acl gains a resource_ref column; set_item_acl(resource_ref=...) accepts a
  ProviderResourceRef / dict / json str and persists it; get_item_grants returns
  it; migration adds the column; export_all/import_bundle carry it. This is the
  real ingest -> grant linkage a candidate item is revalidated through.

ACL model:
- acl.ProviderResourceRef gains to_json/from_json and provider-accurate locator
  docs: github {issue|pull{owner,repo,number}, commit{owner,repo,sha},
  check_run{owner,repo,check_run_id}} (NOT a path); every MS Graph provider
  {tenantId, container ids, endpoint}; salesforce {instanceUrl,sobjectType,
  recordId}; drive/gmail/zoom/slack/asana object keys.
- acl.QueryPrincipal (the whole-query authenticated caller) + LOCAL_PRINCIPAL,
  and acl.BindingResolver.resolve(principal, provider, account) -> AccessContext
  | None: maps the principal to the binding it holds on ONE candidate's own
  (provider, account); None => deny (never reuse another provider's binding).

Retriever:
- HybridRetriever(binding_resolver=...); search(query_principal=...). _acl_filter
  resolves the managed context PER CANDIDATE from its resource_ref's
  (provider, account) via the resolver, then revalidates against THAT context. A
  managed item with no resource_ref (under a resolver), no binding, a
  wrong-account/other-provider binding, a resolver error, or a local principal is
  denied. No resolver => direct-context path (unchanged) for tests/simple hosts.

Dashboard: adds _knowledge_query_principal + _knowledge_binding_resolver seams
(app['knowledge_query_principal'] / ['knowledge_binding_resolver']) threaded into
both search call sites alongside the existing revalidator seam.

Tests: test_knowledge_query_acl_binding (matching-provider-only visible,
wrong-account no leak, missing resource_ref denied, resolver error denied, local
principal denies managed, resource_ref round-trips); wiring tests extended. 1324
knowledge tests pass; flake8 clean; source-tree-external e2e validated.

Remaining named deps (fail-closed): W01 BindingResolver + QueryPrincipal resolver
implementations; per-provider RevalidationHook; connector ingest calling
set_item_acl(managed=True, resource_ref=...).

* feat(knowledge): generic per-row ingest contract binding row->item/chunk->ACL

Root: the fetch()->(text,meta) path collapsed a structured source's rows into
one blob whose per-row refs lived only in source.properties, so set_item_acl and
meta['rows'] never became real per-row ACL. Persisting the per-row link is this
shared store's responsibility, not an external blocker. Add a minimal generic
per-row contract in the EXISTING framework (reusing the per-document group
pattern the artifact/agent aggregates use), no second sync/runtime, no vendor
change:

- knowledge/rows.py: SourceRow{key, text, resource_ref: ProviderResourceRef,
  subjects, tenant, managed, title} -- one fetched row carrying its own
  identity + content + ACL grant; a managed row MUST carry a resource_ref
  (else unqueryable). RowResult / RowsIngestOutcome{fully_persisted, deleted_keys}.
- store: connector_row_state(source_id,row_key,content_hash,item_ids,status)
  PK(source_id,row_key), registered in the doc-state tables so dedup/adoption
  treat it consistently; get_connector_row_state / set_connector_row_state.
- ingestion.ingest_rows(rows, source_id, snapshot): each row keys its own item
  group; unchanged rows (hash match) skipped; a changed row is chunked+extracted
  on ITS OWN text and its per-user grant + ProviderResourceRef is written in the
  SAME off-loop finalize unit as the item write (new ingest_text on_items hook),
  so item, grant and row-ledger advance together. Different-permission rows never
  share a chunk or a ref. snapshot=True deletes rows the full fetch dropped
  (items+grant); snapshot=False (incremental) never deletes an absent row.
- connectors/base.py: optional supports_rows()/fetch_rows(source) ->
  (rows, snapshot, checkpoint). sync.py: sync_source takes the rows path when
  supports_rows() is True (strict, so a MagicMock connector keeps the fetch
  path), and advances the source CHECKPOINT (connector resume token in
  properties) ONLY when outcome.fully_persisted -- a partial/failed/cancelled
  round leaves the checkpoint and re-attempts next sync; old managed grants are
  never erased on an incremental round. Plain-document connectors unchanged.

Tests: test_knowledge_rows_ingest drives the REAL SyncScheduler -> ingest_rows ->
store -> HybridRetriever chain: two rows with different permissions each isolated
(binding resolver + revalidation), incremental keeps unchanged rows, full
snapshot deletes a dropped row (items+grant gone), partial failure does NOT
advance the checkpoint, checkpoint advances only on full success, local
principal denies ingested managed rows. embedder=None, isolated tmp store. 1330
knowledge tests pass; flake8 clean.

Remaining named deps (fail-closed): W01 BindingResolver + QueryPrincipal
implementations; per-provider RevalidationHook; vendor connectors implementing
supports_rows/fetch_rows to emit SourceRow (SF, GitHub PR3) -- consumed without
pipeline change.

* fix(knowledge): SourceRow never invents a grant from a missing field

Root: SourceRow.subjects defaulted to (PUBLIC_SUBJECT,), so a connector that
omitted the grant produced a PUBLIC row -- inventing authorization from a
missing field, which violates 'unknown must not invent authorization'. A blank
tenant or managed=False must not let a cloud row escape managed provenance.

- subjects is now REQUIRED (no default). An EMPTY tuple is a valid, explicit
  deny-all; a MISSING grant is a constructor TypeError, never public. PUBLIC is
  only reachable by the connector passing acl.PUBLIC_SUBJECT on purpose.
- tenant is REQUIRED non-empty: a blank tenant is rejected (a cloud row must be
  tenant-scoped; PUBLIC_TENANT must be given explicitly with a basis).
- managed is now a read-only property that is ALWAYS True -- a SourceRow is by
  definition a managed cloud/structured row, so a connector cannot pass
  managed=False to bypass managed provenance/revalidation. (Plain-document
  connectors use the text path, not SourceRow.)
- resource_ref stays required.

Tests (centralised negatives, same PR): subjects-required-no-public-default,
explicit-empty-subjects-is-deny-all-not-public (ingested + queried, nobody incl.
matching-tenant subject and local library sees it), blank-tenant-rejected,
missing-resource_ref-rejected, managed-always-true-no-bypass. 1335 knowledge
tests pass; flake8 clean.

* knowledge: register real GitHub/Google/Salesforce connectors at the shared handler seam

Wire the three structured vendor connectors into setup_knowledge_routes'
connectors map via an import-guarded registration (_register_optional_connector):
each is instantiated and keyed by its own source_type() only when its module is
importable. A not-yet-landed vendor module (or one whose transport deps are
absent) is simply not registered -- add_source rejects the source_type and the
retrieval gate never sees it (fail-closed, never public). No vendor
implementation is copied here; only the import + instantiation live in the
shared handler, so each vendor auto-activates when its own PR lands on main.

* knowledge: inject the W01-backed runner when registering vendor connectors

The structured vendor connectors (GitHub/Google/Salesforce) are NOT constructed
no-arg: each takes an injected runner/factory (Google operations_factory, SF
call_runner) that composes the W01 control-plane executor with per-source
credential custody + real transport (controlled TLS). Constructed without it the
connector's own validate/fetch refuse fail-closed, so a no-arg registration
would be a dead, misleading entry.

_register_optional_connector now takes runner_factory and constructs the
connector WITH it; setup_knowledge_routes reads a {source_type: runner_factory}
map from app['knowledge_connector_runners'] (installed by the host once the W01
executor, PR #11286, is available) and registers a vendor ONLY when its module
imports AND its runner is installed. Absent module or absent runner => not
registered (fail-closed, never public). No vendor code copied.

Adds an end-to-end test: the handler seam constructs a connector WITH an
injected runner and the SAME instance drives the real SyncScheduler ->
fetch_rows -> ingest_rows -> store -> ACL query chain (scripted runner over the
documented offline path; live W01 TLS is PR #11286's executor, not on main).

* knowledge: register vendor connector on import, inject runner when installed

Refine vendor-connector registration so it composes cleanly with the GitHub
structured-source PR handler hunk (which registers github unconditionally and
must stay edition-overridable). Each connector takes its live-read dependency as
an OPTIONAL injected arg (GitHub transport_provider, Google operations_factory,
SF call_runner) and self-enforces fail-closed when absent (GitHub refuses at
fetch; Google and SF at validate). _register_optional_connector now registers a
connector whenever its module imports, injecting the runner factory only when the
host installed one; an absent module is still not registered (fail-closed). This
keeps GitHub registered and overridable AND the ACL runner injection, with no
hard import of a not-yet-landed module.

* knowledge: document the connector runner factory as per-source, no stand-in

Clarify the shared-handler registration seam per the per-source runner-factory
decision: the value installed under knowledge_connector_runners is a PER-SOURCE
factory the connector invokes as factory(source) to build that source's own
runner (Google operations_factory(source); Salesforce per-source call runner),
so one installed factory serves many sources without sharing a credential
binding. Also state explicitly that this handler never synthesises a stand-in
empty runner to feign activation: an absent factory leaves the connector on its
own default (self-refusing live reads, fail-closed); an absent module leaves the
source_type unregistered. Comment/docstring only; no behavior change.

* knowledge: bridge W01 AccessGrant into a real AccessContext at the ACL seam

Root found a real mismatch: W01's ControlPlaneBindingResolver.resolve returns an
AccessGrant record (subject/tenant/groups from the trusted store's verified
refs) which is NOT an AccessContext -- it has no subject_ids, the exact set the
policy subject test reads (grant.subjects & ctx.subject_ids). Feeding it raw
would AttributeError.

Add acl.bridge_binding_resolver: wraps any resolver returning an AccessGrant and
maps it field-for-field into a genuine AccessContext (subject, tenant, groups,
bypass_acl=False), None-preserving and fail-closed on a subjectless/tenantless
grant; an AccessContext is passed through unchanged. The handler now wraps
app[knowledge_binding_resolver] through this bridge, so a host installing W01's
resolver gets a working gate with the policy UNCHANGED.

Tests drive the real AccessGrant shape (no subject_ids) through the bridge into
the real HybridRetriever.search: granted item visible, wrong-account denied,
group-from-grant satisfies the subject test, None grant denied; plus the handler
seam bridges an installed AccessGrant resolver and passes an AccessContext
through. 489 knowledge tests pass, flake8 clean.

* test: real W01 ControlPlaneBindingResolver + BindingStore integration through the ACL bridge

Adds test_knowledge_acl_w01_integration.py, which drives the ACTUAL W01
control-plane store and resolver rather than an AccessGrant-shaped stand-in:
seed a VERIFIED binding via create_binding + BindingStore.insert, resolve it
through the real ControlPlaneBindingResolver, bridge the returned AccessGrant to
an AccessContext with acl.bridge_binding_resolver, and run the real
HybridRetriever ACL gate. Cases: bound principal admitted, wrong-account denied
(the store returns no binding for that deployment), revoked binding fenced
(store returns None after revoke), unbound principal denied.

The W01 control-plane package is not on this branch; it arrives in the merged
candidate (this branch + the W01 L04 branch). The module importorskips when it
is absent, so the standalone branch stays green (1 skipped) and the merged
candidate runs the real integration (4 passed). Only the identity-proof verifier
IO is substituted per its Protocol; the store, resolver, revoke fencing and
account/deployment keying are the real ones -- an unmodified account genuinely
returns no binding, not a faked pass.

* knowledge: inject the per-source runner factory by each connector's own keyword

The connectors do not share one injection keyword: Google takes
operations_factory, GitHub takes transport_provider, and Salesforce takes a
POSITIONAL call_runner plus a KEYWORD-ONLY runner_factory. The prior positional
inject would have put the per-source factory into Salesforce's call_runner (a
single pre-composed runner), mis-typing it. _register_optional_connector now
takes inject_kw and injects the factory by that keyword; the registration loop
maps each source_type to its connector's own keyword. Verified against the real
SalesforceStructuredConnector (SF 74e54): the factory lands in runner_factory and
call_runner stays None. Tests cover keyword injection, the SF-shape no-mis-wire,
default-when-absent, and absent/broken module fail-closed.

* feat(connections): Google Drive knowledge source — real provider + per-file SourceRow + sync/refresh + query-time ACL probe

Adds the Google Drive provider stream's knowledge slice (D1), all four segments
connected, every outbound call routed through W01's execute/PageWalk (vendor side
only assembles: no token, session, HTTP, vault, refresh or revoke).

vendors/google_drive/:
- descriptors.py  W01 OperationDescriptors (service_id=google_drive, effect=read)
- locator.py      RequestLocator: (descriptor,args) -> HttpRequest (both shared-drive flags)
- decode.py       ResultDecode: 2xx HttpReply -> OperationResult (nextPageToken->next_cursor)
- drive_api.py    Drive v3 request builders + mime classification + shortcut/export/media
- errors.py       Drive failure -> neutral ErrorClass (reason-level classifier, documented)
- pagination.py   distinct page cursor vs resume/resync token readers
- operations.py   sequences over an injected OperationRunner (list walk, content, changes+resync)
- acl_probe.py    GoogleDriveRevalidationHook: real per-query probe AS the subject, revoke-next-query

knowledge/connectors/google_drive.py: GoogleDriveConnector — one SourceRow per file
(fileId/driveId/mimeType/modifiedTime/version + fingerprint), snapshot vs incremental
via changes.list, stale-token resync -> full-snapshot fallback, subjects from REAL
permission principals (anyone=public, missing=empty deny-all, never invents a grant).

Change-token resync keys on W01's neutral `input` class (Google documents no TTL/status
for an expired token; W01 redacts the provider reason) — documented gap in the spec, no
fabricated code. Non-input failures still raise.

Tests run the REAL W01 execute/PageWalk over a scripted transport that runs the real
locator.locate + decode.for_operation (56 tests): multi-page walk, export/media/shortcut,
resync, duplicate cursor, removed change, malformed response, missing perms,
content-fetch failure, revoke-then-deny. Ceiling code_complete: no live evidence, no
real Google request, no business write.

Owning spec: docs/system-specs/modules/connector-google-drive.md.
Registration (shared knowledge handler is the ACL owner's, not edited here) delivered as
a concrete factory/hunk PROPOSAL: connector-google-drive-registration-proposal.md.
@bolichen97

Copy link
Copy Markdown
Contributor Author

暂停关闭(Connector Production Stack 项目于 2026-09-20 整体暂停)

  • 所属工作流:W01
  • 关闭时状态:需 rebase(dirty)+ 大量红检查(14 项,含 Coverage / PR Hygiene / GPT / Security scope / macOS / Windows installer 等)。
  • 分支 feat/connector-control-plane-executor 未删除,head 1d2300a167b0 保留。
  • 恢复方式:gh pr reopen 11286;若 reopen 被拒(base 变动/分支落后),基于同一分支对 main 新开 PR 即可。
  • 说明:本分支是多条下游 stacked 片(如 feat(connections): W02 PR-3 — real GitHub operation invoked + rows fetched via W01 transport #11292)的依赖根,恢复时建议先恢复本条并 rebase 到 main。
  • 交接文档:connector-campaign-20260914/wind-down/HANDOVER-PROMPT-20260920.md

本次关闭不代表该工作被否决,只是项目暂停收口。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge conflict Branch has merge conflicts with its base — author must resolve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants