Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,6 @@ jobs:
platforms: linux/amd64
tags: ${{ steps.login-ecr.outputs.registry }}/${{ steps.ecr-repo.outputs.name }}:${{ github.sha }}-dev
build-args: |
SIM_SEARCH_LIVE_DEFAULT=true
MSHIP_PLAN_MODE_DEFAULT=true
max-cache-size-mb: ${{ matrix.cache_mb }}

Expand Down
1 change: 0 additions & 1 deletion .github/workflows/desktop-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ on:
- 'apps/sim/hooks/queries/personal-search-integrations.ts'
- 'apps/sim/hooks/use-search-integration-connection.ts'
- 'apps/sim/hooks/use-github-installation-setup.ts'
- 'apps/sim/app/o/**/integrations/indexed/use-member-enrollment.ts'
- 'apps/sim/lib/api/contracts/desktop-source-connect.ts'
- 'apps/sim/scripts/fixtures/desktop-source-connect.tsx'
- 'apps/sim/app/workspace/**/browser-session/**'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ The private key must include its PEM header, footer, and contents. Sim accepts a

Keep the private key and client secret in the deployment's server configuration; organization admins select installations in Sim without entering these secrets.

The live Search application needs these five variables. If a worker also indexes ordinary GitHub knowledge bases, or Search has explicitly reverted to `SIM_SEARCH_LIVE=false`, configure the variables in that worker’s matching environment too. Updating the app’s secret store does not update a separately configured worker. Live Search does not schedule a GitHub indexing worker.
The live Search application needs these five variables. If a worker also indexes ordinary GitHub knowledge bases, configure the variables in that worker’s matching environment too. Updating the app’s secret store does not update a separately configured worker. Live Search does not schedule a GitHub indexing worker.

The **Client ID** is different from the numeric **App ID**. Use credentials from **Developer settings → GitHub Apps**. `GITHUB_CLIENT_ID` and `GITHUB_CLIENT_SECRET` belong to the separate GitHub sign-in integration and remain unchanged. Search does not read `GITHUB_REPO_CLIENT_ID` or `GITHUB_REPO_CLIENT_SECRET`.

Expand Down
2 changes: 1 addition & 1 deletion apps/docs/content/docs/search/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,6 @@ Conversations remain private to their author. Connecting an external account doe

## Legacy indexing and workspace knowledge bases

Live Search is the default. An operator can explicitly set `SIM_SEARCH_LIVE=false` to restore the legacy indexed Search backend. Its content sync, document processing, and stored permission maintenance are specific to that mode. These guides describe live Search.
Enterprise Search uses live provider queries. Search sources do not run content indexing or stored permission maintenance; ordinary knowledge-base connectors retain their indexing behavior.

Ordinary workspace [knowledge bases](/knowledgebase) still ingest, chunk, and index documents for their own features. Their connector setup, processing status, and access settings remain separate.
2 changes: 1 addition & 1 deletion apps/docs/content/docs/search/mcp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The connection applies to the organization whose URL you copied. Sim checks curr

`chat` accepts questions up to 8,192 characters. Tool responses are limited to 1 MiB. API rate limits apply; follow any retry delay. Provider failures or incomplete retrieval are reported explicitly, and the tools do not fall back to an old index when a live provider is unavailable.

These are the live-backend schemas. If an operator explicitly selects legacy Search with `SIM_SEARCH_LIVE=false`, the server advertises its indexed search/read schemas instead. Refresh your MCP client's tool discovery after changing backends.
These are the Search MCP schemas. Refresh tool discovery in clients that previously connected to the retired indexed backend.

## Reconnect or revoke access

Expand Down
19 changes: 0 additions & 19 deletions apps/sim/app/api/knowledge/connectors/directory-sync/route.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import {
hasMockCondition,
resetEnvFlagsMock,
schemaMock,
setEnvFlags,
} from '@sim/testing'
import { authInternalMock, authInternalMockFns } from '@sim/testing/mocks/auth-internal.mock'
import { dbChainMockFns } from '@sim/testing/mocks/database.mock'
Expand Down Expand Up @@ -108,24 +107,6 @@ describe('connector directory sync scheduler', () => {
await expect(run()).resolves.toMatchObject({ dispatched: 1, failed: 1 })
})

it.each([true, false])(
'excludes Search directories from scheduled pages only when live Search is %s',
async (liveSearch) => {
setEnvFlags({ isLiveEnterpriseSearchEnabled: liveSearch })
mockConnectorRows.mockResolvedValue([])
await run()
expect(
hasMockCondition(
mockWhere.mock.calls[0][0],
(node) =>
node.type === 'eq' &&
node.left === schemaMock.knowledgeBase.isSearchIndex &&
node.right === false
)
).toBe(liveSearch)
}
)

it('does not enqueue a connector another scheduler claimed or paused', async () => {
mockConnectorRows.mockResolvedValue([connector()])
mockClaim.mockResolvedValueOnce([])
Expand Down
64 changes: 1 addition & 63 deletions apps/sim/app/api/knowledge/search/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,68 +6,9 @@ import {
} from '@/lib/api/server/routes'
import { internalKnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies'
import { knowledgeOperations } from '@/lib/knowledge/application/operations'
import { DEFAULT_RERANKER_MODEL } from '@/lib/knowledge/reranker-models'
import { sourceAuthor } from '@/lib/knowledge/search/author'
import { searchScopedKnowledge } from '@/lib/sim-search/indexed'
import { isIndexedOrgSearchEnabled } from '@/lib/sim-search/indexed/gate'
import { searchLiveKnowledge } from '@/lib/sim-search/live/application'

const DIRECT_SEARCH_VECTOR_BUDGET_MS = 3000

const indexedSearchRoute = defineInternalJsonRoute({
contract: searchWorkspaceKnowledgeContract,
auth: internalSessionAuth,
operation: knowledgeOperations.search,
rateLimit: internalRateLimits.none({
reason:
'A person typing queries; the embedding call is metered against the canonical search owner',
}),
errorPolicy: internalKnowledgeErrorPolicies.search,
mapInput: ({ body }, { request }) => ({
workspaceId: body.workspaceId,
organizationId: body.organizationId,
filters: body.filters,
query: body.query,
topK: body.topK,
allowPartialResults: true,
vectorBudgetMs: DIRECT_SEARCH_VECTOR_BUDGET_MS,
/**
* A person's search is reranked by a cross-encoder whenever the workspace or the platform
* holds a key for one; the use case checks that before spending a call, and reranking stays
* best-effort, so a provider outage leaves the fused order in place.
*/
rerankerEnabled: true,
rerankerModel: DEFAULT_RERANKER_MODEL,
surface: 'dashboard' as const,
signal: request.signal,
}),
useCase: searchScopedKnowledge,
present: ({ results, knowledgeBases, retrieval }, { input }) => {
const knowledgeBaseNames = new Map(knowledgeBases.map((kb) => [kb.id, kb.name]))
return {
success: true as const,
data: {
query: input.query ?? '',
retrieval,
results: results.map((result) => ({
documentId: result.documentId,
knowledgeBaseId: result.knowledgeBaseId,
knowledgeBaseName: knowledgeBaseNames.get(result.knowledgeBaseId) ?? '',
documentName: result.documentName,
sourceUrl: result.sourceUrl,
connectorType: result.connectorType,
sourceModifiedAt: result.sourceModifiedAt?.toISOString() ?? null,
author: sourceAuthor(result.metadata),
content: result.content,
chunkIndex: result.chunkIndex,
similarity: result.similarity,
})),
},
}
},
})

const liveSearchRoute = defineInternalJsonRoute({
export const POST = defineInternalJsonRoute({
contract: searchWorkspaceKnowledgeContract,
auth: internalSessionAuth,
operation: knowledgeOperations.search,
Expand All @@ -80,6 +21,3 @@ const liveSearchRoute = defineInternalJsonRoute({
useCase: searchLiveKnowledge,
present: (data) => ({ success: true as const, data }),
})

/** Indexed organization search is dormant unless its gate is on; Live Search serves otherwise. */
export const POST = isIndexedOrgSearchEnabled() ? indexedSearchRoute : liveSearchRoute
20 changes: 0 additions & 20 deletions apps/sim/app/api/knowledge/sim-search/connect/route.ts

This file was deleted.

This file was deleted.

This file was deleted.

23 changes: 0 additions & 23 deletions apps/sim/app/api/knowledge/sim-search/sources/overview/route.ts

This file was deleted.

23 changes: 0 additions & 23 deletions apps/sim/app/api/knowledge/sim-search/sources/progress/route.ts

This file was deleted.

44 changes: 1 addition & 43 deletions apps/sim/app/api/knowledge/sim-search/sources/route.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,32 +5,16 @@ import type { SearchSourceSummary } from '@/lib/api/contracts/knowledge/connecto

const mocks = vi.hoisted(() => ({
execute: vi.fn(),
overview: vi.fn(),
adminOverview: vi.fn(),
}))
vi.mock('@/lib/knowledge/application/organization-search-overview', () => ({
readOrganizationSearchOverview: {
operation: { id: 'knowledge.search.integrations.overview' },
execute: mocks.adminOverview,
},
}))
vi.mock('@/lib/knowledge/application/search-sources', () => ({
listSearchSources: { operation: { id: 'knowledge.search.sources.list' }, execute: mocks.execute },
}))
vi.mock('@/lib/knowledge/application/search-source-overview', () => ({
readSearchSourceOverview: {
operation: { id: 'knowledge.search.sources.overview' },
execute: mocks.overview,
},
}))
vi.mock('@/lib/knowledge/application/search', () => knowledgeSearchUseCaseMock)
vi.mock('@/lib/knowledge/application/upload-sessions', () => ({
KnowledgeDocumentUnsupportedMediaTypeError: class extends Error {},
}))

import { NoWorkspaceAccessError } from '@/lib/core/application/workspace-authorization'
import { OrchestrationError } from '@/lib/core/orchestration/types'
import { GET as getAdminOverview } from '@/app/api/knowledge/sim-search/integrations/overview/route'
import { GET } from '@/app/api/knowledge/sim-search/sources/route'

const WORKSPACE_ID = '7d28e5e2-fb03-4118-9c52-4ab77ccff369'
Expand All @@ -42,15 +26,7 @@ const source = {
accessMode: 'admin',
availability: 'available',
enabled: true,
isSyncing: false,
lastSyncAt: null,
hasSyncError: false,
hasViewerDocuments: false,
viewerFailedDocumentCount: 0,
viewerEmailVerified: true,
viewerAccounts: [],
connectionRequired: false,
viewerMembership: null,
isGitHubInstallation: false,
} satisfies SearchSourceSummary

beforeEach(() => {
Expand Down Expand Up @@ -112,21 +88,3 @@ describe('Search pagination boundary', () => {
expect(mocks.execute).not.toHaveBeenCalled()
})
})

describe('organization administration overview boundary', () => {
it('preserves a role refusal without exposing health data', async () => {
mocks.adminOverview.mockRejectedValue(
new OrchestrationError('forbidden', 'Organization administrator access is required')
)
const response = await getAdminOverview(
createMockRequest(
'GET',
undefined,
{},
`http://localhost/api/knowledge/sim-search/integrations/overview?organizationId=${WORKSPACE_ID}`
)
)
expect(response.status).toBe(403)
expect(await response.json()).not.toHaveProperty('data')
})
})
Loading
Loading