Skip to content

fix(knowledge): revoke absent documents without rewriting unchanged ACLs - #8179

Merged
waleedlatif1 merged 1 commit into
stagingfrom
fix/reconcile-revoke-bounded-fanout
Sep 23, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
fix/reconcile-revoke-bounded-fanout

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Absent-document revocation in admin-mode reconciliation assigned acl 500 documents per statement. Every acl assignment fires the projection trigger, which rewrites each filled chunk row of the document, so one batch of chunk-heavy documents could outrun the statement timeout, fail the run, and fail the same way on every retry
  • The permission-only page revocation had the same 500-per-statement write and no "already empty" guard, so a document that already granted nobody still fired the trigger
  • New revokeDocumentAcls in sync-persistence.ts, used by both paths, follows the same approach as persistDocumentAcls:
    • acl is assigned only where cardinality(acl) > 0, ACL_CHANGE_BATCH_SIZE (25) documents per statement
    • a document already readable by nobody only has leftover aclRequirements / aclVerifiedAt cleared, ACL_WRITE_BATCH_SIZE per statement, without assigning acl, so it fires no fan-out
  • Unchanged: which documents are revoked, the revoked values ([], [], null), revocation still runs ahead of and independently of the delete cap and suspect-listing guard, one lease-checked transaction per loaded page, soft/hard delete ordering, and docsDeleted

Type of Change

  • Bug fix

Testing

  • New sync-content-pass.postgres.test.ts against the real projection trigger, with a test-only AFTER UPDATE trigger counting fan-out: an absent document with an empty ACL fires nothing while an absent granted one is revoked and its filled projection rows become empty (unfilled rows stay NULL); a 60-document absent backlog is fully revoked with every removal reported; an already-empty ACL with leftover evidence gets it cleared without fan-out. Added to the "Verify Confluence audience migrations and permission queries in PostgreSQL" CI step
  • Unit tests for the statement shape (guard present, 25/500 batch sizes) on the helper and at both call sites
  • Mutation-checked each guard with file copies: dropping the cardinality > 0 guard, the 25 batch size, the evidence-only statement, its not(grants) guard, or reverting either call site each turns the matching tests red
  • lib/knowledge/connectors suites: 788 passed (with the Postgres tests running locally); bun run lint, bun run check:audits (47/47), apps/sim type-check pass

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Sep 23, 2026 12:03am UTC

Request Review

@greptile-apps

greptile-apps Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with no outstanding blocking findings.

Summary

This PR reduces projection-trigger fan-out while revoking document access:

  • Introduces a shared ACL-revocation helper that updates granted documents in batches of 25.
  • Clears stale permission evidence separately for documents whose ACL is already empty.
  • Uses the helper in permission-only refresh and absent-document reconciliation.
  • Adds unit and PostgreSQL integration coverage and runs the new integration suite in CI.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Documents selected for revocation] --> B{ACL grants anyone?}
  B -->|Yes| C[Batch up to 25 documents]
  C --> D[Clear ACL and permission evidence]
  D --> E[Projection trigger updates filled chunk rows]
  B -->|No| F[Batch up to 500 documents]
  F --> G[Clear leftover permission evidence only]
  G --> H[No ACL-triggered projection fan-out]
Loading

Reviews (2) · Last reviewed commit: "fix(knowledge): revoke absent documents ..."

Comment thread apps/sim/lib/knowledge/connectors/sync-content-pass.ts

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 6 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 884c602 into staging Sep 23, 2026
34 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/reconcile-revoke-bounded-fanout branch September 23, 2026 01:41

This branch was successfully deployed

1 active deployment
Preview — 322eef22 Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant