migrate the context gatekeeper to kernel-owned observer tracking

docs/integration-testing.md

@@ -123,14 +123,15 @@ module at `fixtures/bundled-blueprints.ts`, and hands that fixture its list thro
 installer, and the check that decides whether to run it, are the Workshop's own.
 
 A gatekeeper's `main` is such an input too, so a gatekeeper can be upgraded from an older release:
-`build:upgrade-base` bundles gatekeeper-google as an earlier revision of the repo had it, the
-Google hooks suite boots that bundle, and `upgradeGoogle()` (`__tests__/google-fake.ts`) swaps in
-this tree's build with `server.update(current => …)`, replacing only that Worker's config.
+`build:upgrade-base` bundles gatekeeper-google and gatekeeper-context as earlier revisions of the
+repo had them, the Google hooks and Context observer upgrade suites boot those bundles, and
+`redeployWorker()` (`src/harness.ts`) swaps in this tree's build with `server.update(current => …)`,
+replacing only that Worker's config.
 
 `update()` resolves at the first reload, not the last. It hands each Worker its new config in
 turn, and each rebuilds and reloads on its own, so the reload it waited for can be running one
 Worker's new build beside the other's old one, with a second reload still to come. That second
-restart breaks the sessions and the requests the test has opened since. `upgradeGoogle()` gives
+restart breaks the sessions and the requests the test has opened since. `redeployWorker()` gives
 every Worker's config a mark unique to the update, and waits until each reports it through
 `getWorker(name).getEnv()`, the one state no reload can leave half-applied.
 `Harness.redeployWorkshop()` doesn't wait this way yet.

docs/observers.md

@@ -893,7 +893,7 @@ its resource types.
 | **confluence** | Space | **C** | Verify space access; track observed pages and blog posts because content restrictions may be narrower. |
 | **confluence** | Page / Blog Post | **C** | Verify bound-content access; track observed child pages because they may have stricter restrictions than their parent. |
 | **zoominfo** | Account | **A** | Always throw. The whole-account binding exposes licensed, entitlement-dependent and account-specific intelligence, and ZoomInfo provides no ACL oracle proving another account can read every historical result. |
-| **context** | Context Library singleton | **C** | Track collections whose contents a read revealed (search results, a collection's listing, a document); verify each is in the observer's sharing domain and is public, is everyone-viewable, or grants the observer's Context account a viewer, editor, or owner role. Uses the kit's default observer cap to bound per-read verifier fanout. The top-level collection listing tracks nothing: its titles, descriptions and counts are discovery metadata the agent catalog already advertises unobserved. The denial never names the collection, which the refused collaborator cannot read. |
+| **context** | Context Library singleton | **C**, kernel-tracked (§10) | Never called: the connection advertises `supportsObserverTracking`. Each read reports the collections whose contents it revealed (search results, a collection's listing, a document) as `["collection", sharingDomain, collectionId]` subresources, and `verifyObserverAccess` denies each one the observer's Context account cannot read now: outside its sharing domain, or neither public, everyone-viewable, nor granted to it as viewer, editor, or owner. The top-level collection listing reports `[]`: its titles, descriptions and counts are discovery metadata the agent catalog already advertises unobserved. `getHistoricalSubresources` exports every `observedCollection:` marker the binding's own collection tracker left before the move, pending ones included, so those markers are never deleted. The Workshop's denial never names the collection. |
 | **x** | Account | **C** | Admit any observer with a working X connection: public posts, profiles, public Lists and follow graphs need only an X account. Reads private to the account -- bookmarks, likes, mute state, private Lists, and any page holding a protected account's post -- authorize against one synthetic `owner` collection, which admits only an observer connected as the same X user. |
 | **x** | Post / List | **C** | As Account, plus a check on every open that the observer's own X account can see the bound post or List (`canViewPost` / `canViewList`). Each check is a read billed to the observer's connection, so a passed check is trusted for an hour; a failed one never is. |
 | **x** | Profile | **C** | As Account: a public profile needs only an X account, and the `owner` collection covers a protected user's posts. Accepted over-permissiveness for every X binding: an author who has blocked the observer is still readable through the gadget, since checking each author per observer per read would cost a read each and what leaks is public. |
@@ -937,7 +937,7 @@ This is why the broad bindings split the way they do:
 A connection whose `describe()` returns `supportsObserverTracking: true` is tracked by the
 Workshop itself instead of the protocol above (`src/observer-ledger.ts`). Connections without the
 flag keep the legacy `addObserver`/`removeObserver`/`excludeObservers` protocol unchanged, and one
-workspace can mix the two. No shipping gatekeeper opts in yet.
+workspace can mix the two. The Context Library is the first shipping gatekeeper to opt in (§9.2).
 
 - **The audit log is the requirement store.** Every tracked observation reports `subresources`:
   opaque, stable, non-secret IDs, fully qualified within their vendor (`[]` is valid; omitting it,
@@ -1038,7 +1038,9 @@ workspace can mix the two. No shipping gatekeeper opts in yet.
   "use" collaborators only for gadgets that still reach the connection.
 
 The integration fixture's tracked mode (`/control/observer-tracking`) exercises this end to end in
-`packages/integration-tests/__tests__/observer-tracking.test.ts`.
+`packages/integration-tests/__tests__/observer-tracking.test.ts`, and
+`context-observer-upgrade.test.ts` upgrades the real Context Library from the release that tracked
+its own observers, importing what that release recorded.
 
 ---
 

packages/gatekeeper-context/__tests__/collection-sharing.test.ts

@@ -1,7 +1,9 @@
 import { createExecutionContext, env, runDurableObjectAlarm, runInDurableObject } from "cloudflare:test";
+import type { RpcStub } from "cloudflare:workers";
 import { beforeEach, describe, expect, it } from "vitest";
-import { ObservationGate, privateObservers } from "@gadgets/gatekeeper-kit/observers";
-import type { GatekeeperUserSelectionExchange } from "@gadgets/workshop-shared/gatekeeper";
+import type {
+  GatekeeperUserSelectionExchange, ObservationAuthorizer,
+} from "@gadgets/workshop-shared/gatekeeper";
 import { ContextApiImpl } from "../src/context-api.js";
 import { canReadCollection, collectionPermissions } from "../src/collection-access.js";
 import { ContextCollectionDurableObject } from "../src/context-collection.js";
@@ -70,13 +72,13 @@ async function permissions(id: string, accountId: string) {
 }
 
 function readSession(accountId: string): LibraryReadSession {
+  // Only authorizeObservation() and disposal are reached by the read session.
   let authorizer = {
     async authorizeObservation() {},
     [Symbol.dispose]() {},
-  } as unknown as ConstructorParameters<typeof ObservationGate>[0];
+  } as unknown as RpcStub<ObservationAuthorizer>;
   return new LibraryReadSession(
-    testEnv, testEnv.COLLECTIONS, testEnv.USER_LIBRARIES, domain, accountId,
-    new ObservationGate(authorizer, privateObservers("No observers in this access-check fixture.")));
+    testEnv, testEnv.COLLECTIONS, testEnv.USER_LIBRARIES, domain, accountId, authorizer);
 }
 
 /**

packages/gatekeeper-context/__tests__/context-observers.test.ts

@@ -1,58 +1,103 @@
-import { describe, expect, it } from "vitest";
-import type { ObserverKv } from "@gadgets/gatekeeper-kit/observers";
-import type { GatekeeperUserVerifier } from "@gadgets/workshop-shared/gatekeeper";
+import { env } from "cloudflare:test";
+import { beforeEach, describe, expect, it } from "vitest";
+import type { ContextCollectionDurableObject } from "../src/context-collection.js";
 import {
-  CONTEXT_OBSERVER_DENIED, contextObserverStrategy,
+  collectionSubresource, deniedCollectionSubresources, forgetLegacyObserver,
+  legacyCollectionSubresources,
 } from "../src/context-observers.js";
+import { domainName } from "../src/domain.js";
 
-function makeKv(): ObserverKv & { map: Map<string, unknown> } {
-  let map = new Map<string, unknown>();
-  return {
-    map,
-    get: <T>(key: string) => map.get(key) as T | undefined,
-    put: <T>(key: string, value: T) => void map.set(key, value),
-    delete: (key: string) => map.delete(key),
-    list: <T>({ prefix }: { prefix?: string } = {}) =>
-      [...map.entries()].filter(([key]) => key.startsWith(prefix ?? "")) as Iterable<[string, T]>,
-  } as ObserverKv & { map: Map<string, unknown> };
-}
+// The miniflare bindings vitest.config.ts declares, which Cloudflare.Env does not.
+const testEnv = env as unknown as { COLLECTIONS: DurableObjectNamespace<ContextCollectionDurableObject> };
+const collections = testEnv.COLLECTIONS;
+
+// Storage persists across the file's tests, so each gets its own sharing domain.
+let domain = "";
+let domains = 0;
+beforeEach(() => {
+  domain = `observers-${domains++}.example`;
+});
 
-function verifier(allowed: string[]) {
-  let calls: Array<{ sharingDomain: string; collectionIds: string[] }> = [];
-  let user = {
-    async hasCollectionsAccess(sharingDomain: string, collectionIds: string[]) {
-      calls.push({ sharingDomain, collectionIds });
-      return collectionIds.map(id => allowed.includes(id));
-    },
-  } as unknown as Fetcher<GatekeeperUserVerifier>;
-  return { user, calls };
+const collection = (id: string) => collections.get(collections.idFromName(domainName(domain, id)));
+
+async function createCollection(id: string, visibility: "private" | "public", owner = "alice") {
+  let now = new Date();
+  await collection(id).initialize({
+    id, title: id, description: "", visibility,
+    created: now, lastUpdated: now, documentCount: 0, content: { source: "web" },
+  }, domain, visibility === "private" ? owner : "");
 }
 
-describe("contextObserverStrategy", () => {
-  it("keeps enforcing collections recorded before the move to the kit tracker", async () => {
-    for (let legacy of [true, "observed", "pending"]) {
-      let kv = makeKv();
-      kv.put("observedCollection:private", legacy);
-      let strategy = contextObserverStrategy(kv, "workshop.example");
-
-      await expect(strategy.addObserver("denied", verifier([]).user))
-        .rejects.toThrow(CONTEXT_OBSERVER_DENIED);
-      let allowed = verifier(["private"]);
-      await strategy.addObserver("allowed", allowed.user);
-      expect(allowed.calls).toEqual([
-        { sharingDomain: "workshop.example", collectionIds: ["private"] },
-      ]);
-    }
+const denied = (accountId: string, subresources: string[]) =>
+  deniedCollectionSubresources(collections, domain, accountId, subresources);
+
+describe("deniedCollectionSubresources", () => {
+  it("grants exactly the collections the account can read now", async () => {
+    await createCollection("shared", "private");
+    await collection("shared").share("alice", [{ accountId: "bob", displayName: "bob" }], "viewer");
+    await createCollection("everyone", "private");
+    await collection("everyone").setEveryoneCanView("alice", true);
+    await createCollection("public", "public");
+    await createCollection("unshared", "private");
+    await createCollection("deleted", "private");
+    await collection("deleted").removeAccount("alice");
+
+    let readable = ["shared", "everyone", "public"].map(id => collectionSubresource(domain, id));
+    let unreadable = [
+      ...["unshared", "deleted", "never-created"].map(id => collectionSubresource(domain, id)),
+      // Another sharing domain's collection, even one with this ID.
+      collectionSubresource("elsewhere.example", "public"),
+      JSON.stringify(["document", domain, "public"]),
+      "not a subresource",
+    ];
+    expect((await denied("bob", [...readable, ...unreadable])).toSorted()).toEqual(unreadable.toSorted());
+    expect(await denied("alice", [collectionSubresource(domain, "unshared")])).toEqual([]);
   });
 
-  it("refuses without disclosing which collection the collaborator cannot read", async () => {
-    let kv = makeKv();
-    let strategy = contextObserverStrategy(kv, "workshop.example");
-    let check = await strategy.prepare!(["private-collection-id"]);
-    check.commit();
+  it("denies a collection as soon as the account's access to it is revoked", async () => {
+    await createCollection("shared", "private");
+    await collection("shared").share("alice", [{ accountId: "bob", displayName: "bob" }], "viewer");
+    let shared = collectionSubresource(domain, "shared");
+    expect(await denied("bob", [shared])).toEqual([]);
+
+    await collection("shared").removeMember("alice", "bob");
+
+    expect(await denied("bob", [shared, shared])).toEqual([shared, shared]);
+  });
+});
+
+describe("legacyCollectionSubresources", () => {
+  it("exports every collection the gatekeeper's own tracker recorded, whatever its state", async () => {
+    // The layout the tracker left in a binding's storage: one marker per collection a read revealed,
+    // beside each admitted observer's registration.
+    let storage = new Map<string, unknown>([
+      ["observedCollection:unshared", true],
+      ["observedCollection:shared", "observed"],
+      ["observedCollection:uncertain", "pending"],
+      ["observer:collaborator", { verifier: "stub" }],
+      ["observer-attempt:candidate", { verifier: "stub", at: 0 }],
+      ["observer-nonce:candidate", "nonce"],
+    ]);
+    let kv = {
+      list: ({ prefix = "" }: SyncKvListOptions = {}) =>
+        [...storage].filter(([key]) => key.startsWith(prefix)).values(),
+      delete: (key: string) => storage.delete(key),
+    } as Pick<SyncKvStorage, "list" | "delete">;
+    await createCollection("unshared", "private");
+    await createCollection("shared", "private");
+    await collection("shared").share("alice", [{ accountId: "bob", displayName: "bob" }], "viewer");
+    await createCollection("uncertain", "private");
+
+    // Retiring the tracker's registrations leaves the history to export again if the import fails.
+    forgetLegacyObserver(kv, "collaborator");
+    forgetLegacyObserver(kv, "candidate");
+    let exported = legacyCollectionSubresources(kv, domain);
 
-    let denial = strategy.addObserver("denied", verifier([]).user);
-    await expect(denial).rejects.toThrow(CONTEXT_OBSERVER_DENIED);
-    await expect(denial).rejects.not.toThrow(/private-collection-id/);
+    expect(exported.toSorted()).toEqual(
+      ["shared", "uncertain", "unshared"].map(id => collectionSubresource(domain, id)).toSorted());
+    expect([...storage.keys()].every(key => key.startsWith("observedCollection:"))).toBe(true);
+    expect((await denied("bob", exported)).toSorted()).toEqual(
+      ["uncertain", "unshared"].map(id => collectionSubresource(domain, id)).toSorted());
+    expect(await denied("alice", exported)).toEqual([]);
   });
 });