From 39ad75c7a4c9f6ca132aef1c781f5f62b8e9c502 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 14:36:38 -0700 Subject: [PATCH 01/30] chore(skills): vendor codebase-design deep-module vocabulary skill --- .agents/skills/codebase-design/DEEPENING.md | 37 ++++++ .../skills/codebase-design/DESIGN-IT-TWICE.md | 44 +++++++ .agents/skills/codebase-design/SKILL.md | 117 ++++++++++++++++++ .claude/skills/codebase-design | 1 + 4 files changed, 199 insertions(+) create mode 100644 .agents/skills/codebase-design/DEEPENING.md create mode 100644 .agents/skills/codebase-design/DESIGN-IT-TWICE.md create mode 100644 .agents/skills/codebase-design/SKILL.md create mode 120000 .claude/skills/codebase-design diff --git a/.agents/skills/codebase-design/DEEPENING.md b/.agents/skills/codebase-design/DEEPENING.md new file mode 100644 index 00000000000..cd94075cfd7 --- /dev/null +++ b/.agents/skills/codebase-design/DEEPENING.md @@ -0,0 +1,37 @@ +# Deepening + +How to deepen a cluster of shallow modules safely, given its dependencies. Assumes the vocabulary in [SKILL.md](SKILL.md): **module**, **interface**, **seam**, **adapter**. + +## Dependency categories + +When assessing a candidate for deepening, classify its dependencies. The category determines how the deepened module is tested across its seam. + +### 1. In-process + +Pure computation, in-memory state, no I/O. Always deepenable: merge the modules and test through the new interface directly. No adapter needed. + +### 2. Local-substitutable + +Dependencies that have local test stand-ins (PGLite for Postgres, in-memory filesystem). Deepenable if the stand-in exists. The deepened module is tested with the stand-in running in the test suite. The seam is internal; no port at the module's external interface. + +### 3. Remote but owned (Ports & Adapters) + +Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Tests use an in-memory adapter. Production uses an HTTP/gRPC/queue adapter. + +Recommendation shape: *"Define a port at the seam, implement an HTTP adapter for production and an in-memory adapter for testing, so the logic sits in one deep module even though it's deployed across a network."* + +### 4. True external (Mock) + +Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port; tests provide a mock adapter. + +## Seam discipline + +- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified (typically production + test). A single-adapter seam is just indirection. +- **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them. + +## Testing strategy: replace, don't layer + +- Old unit tests on shallow modules become waste once tests at the deepened module's interface exist; delete them. +- Write new tests at the deepened module's interface. The **interface is the test surface**. +- Tests assert on observable outcomes through the interface, not internal state. +- Tests should survive internal refactors, since they describe behaviour, not implementation. If a test has to change when the implementation changes, it's testing past the interface. diff --git a/.agents/skills/codebase-design/DESIGN-IT-TWICE.md b/.agents/skills/codebase-design/DESIGN-IT-TWICE.md new file mode 100644 index 00000000000..af03b2724c6 --- /dev/null +++ b/.agents/skills/codebase-design/DESIGN-IT-TWICE.md @@ -0,0 +1,44 @@ +# Design It Twice + +When the user wants to explore alternative interfaces for a chosen deepening candidate, use this parallel sub-agent pattern. Based on "Design It Twice" (Ousterhout): your first idea is unlikely to be the best. + +Uses the vocabulary in [SKILL.md](SKILL.md): **module**, **interface**, **seam**, **adapter**, **leverage**. + +## Process + +### 1. Frame the problem space + +Before spawning sub-agents, write a user-facing explanation of the problem space for the chosen candidate: + +- The constraints any new interface would need to satisfy +- The dependencies it would rely on, and which category they fall into (see [DEEPENING.md](DEEPENING.md)) +- A rough illustrative code sketch to ground the constraints, not a proposal, just a way to make the constraints concrete + +Show this to the user, then immediately proceed to Step 2. The user reads and thinks while the sub-agents work in parallel. + +### 2. Spawn sub-agents + +Spawn 3+ sub-agents in parallel. Each must produce a **radically different** interface for the deepened module. + +Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint: + +- Agent 1: "Minimize the interface: aim for 1–3 entry points max. Maximise leverage per entry point." +- Agent 2: "Maximise flexibility: support many use cases and extension." +- Agent 3: "Optimise for the most common caller: make the default case trivial." +- Agent 4 (if applicable): "Design around ports & adapters for cross-seam dependencies." + +Include both [SKILL.md](SKILL.md) vocabulary and the repo's domain language (root `AGENTS.md` and the relevant `.claude/rules/*.md`) in the brief so each sub-agent names things consistently with the architecture language and the project's domain language. + +Each sub-agent outputs: + +1. Interface (types, methods, params, plus invariants, ordering, error modes) +2. Usage example showing how callers use it +3. What the implementation hides behind the seam +4. Dependency strategy and adapters (see [DEEPENING.md](DEEPENING.md)) +5. Trade-offs: where leverage is high, where it's thin + +### 3. Present and compare + +Present designs sequentially so the user can absorb each one, then compare them in prose. Contrast by **depth** (leverage at the interface), **locality** (where change concentrates), and **seam placement**. + +After comparing, give your own recommendation: which design you think is strongest and why. If elements from different designs would combine well, propose a hybrid. Be opinionated: the user wants a strong read, not a menu. diff --git a/.agents/skills/codebase-design/SKILL.md b/.agents/skills/codebase-design/SKILL.md new file mode 100644 index 00000000000..8499f52ab60 --- /dev/null +++ b/.agents/skills/codebase-design/SKILL.md @@ -0,0 +1,117 @@ +--- +name: codebase-design +description: Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary. +# Vendored from github.com/mattpocock/skills (MIT, Copyright (c) 2026 Matt Pocock). No agents/openai.yaml by design: this is a vocabulary reference, not a service-integration builder. +--- + +# Codebase Design + +This is a reference, not a process: it supplies vocabulary and principles and stops. It has no loop, artifact, or checkpoint, so never treat it as a driver that starts reading files and proposing refactors on its own — pair it with a task that names the code being designed. + +Design **deep modules**: a lot of behaviour behind a small interface, placed at a clean seam, testable through that interface. Use this language and these principles wherever code is being designed or restructured. The aim is leverage for callers, locality for maintainers, and testability for everyone. + +## Glossary + +Use these terms exactly: don't substitute "component," "service," "API," or "boundary." Consistent language is the whole point. + +**Module**: anything with an interface and an implementation. Deliberately scale-agnostic: a function, class, package, or tier-spanning slice. _Avoid_: unit, component, service. + +**Interface**: everything a caller must know to use the module correctly: the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics. _Avoid_: API, signature (too narrow, they refer only to the type-level surface). + +**Implementation**: what's inside a module, its body of code. Distinct from **Adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Reach for "adapter" when the seam is the topic; "implementation" otherwise. + +**Depth**: leverage at the interface. The amount of behaviour a caller (or test) can exercise per unit of interface they have to learn. A module is **deep** when a large amount of behaviour sits behind a small interface, **shallow** when the interface is nearly as complex as the implementation. + +**Seam** _(Michael Feathers)_: a place where you can alter behaviour without editing in that place; the *location* at which a module's interface lives. Where to put the seam is its own design decision, distinct from what goes behind it. _Avoid_: boundary (overloaded with DDD's bounded context). + +**Adapter**: a concrete thing that satisfies an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside). + +**Leverage**: what callers get from depth. More capability per unit of interface they learn. One implementation pays back across N call sites and M tests. + +**Locality**: what maintainers get from depth. Change, bugs, knowledge, and verification concentrate in one place rather than spreading across callers. Fix once, fixed everywhere. + +## Deep vs shallow + +**Deep module** = small interface + lots of implementation: + +``` +┌─────────────────────┐ +│ Small Interface │ ← Few methods, simple params +├─────────────────────┤ +│ │ +│ Deep Implementation│ ← Complex logic hidden +│ │ +└─────────────────────┘ +``` + +**Shallow module** = large interface + little implementation (avoid): + +``` +┌─────────────────────────────────┐ +│ Large Interface │ ← Many methods, complex params +├─────────────────────────────────┤ +│ Thin Implementation │ ← Just passes through +└─────────────────────────────────┘ +``` + +When designing an interface, ask: + +- Can I reduce the number of methods? +- Can I simplify the parameters? +- Can I hide more complexity inside? + +## Principles + +- **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, mockable, swappable parts; they just aren't part of the interface. A module can have **internal seams** (private to its implementation, used by its own tests) as well as the **external seam** at its interface. +- **The deletion test.** Imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep. +- **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape. +- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it. + +## Designing for testability + +Good interfaces make testing natural: + +1. **Accept dependencies, don't create them.** + + ```typescript + // Testable + function processOrder(order, paymentGateway) {} + + // Hard to test + function processOrder(order) { + const gateway = new StripeGateway(); + } + ``` + +2. **Return results, don't produce side effects.** + + ```typescript + // Testable + function calculateDiscount(cart): Discount {} + + // Hard to test + function applyDiscount(cart): void { + cart.total -= discount; + } + ``` + +3. **Small surface area.** Fewer methods = fewer tests needed. Fewer params = simpler test setup. + +## Relationships + +- A **Module** has exactly one **Interface** (the surface it presents to callers and tests). +- **Depth** is a property of a **Module**, measured against its **Interface**. +- A **Seam** is where a **Module**'s **Interface** lives. +- An **Adapter** sits at a **Seam** and satisfies the **Interface**. +- **Depth** produces **Leverage** for callers and **Locality** for maintainers. + +## Rejected framings + +- **Depth as ratio of implementation-lines to interface-lines** (Ousterhout): rewards padding the implementation. We use depth-as-leverage instead. +- **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow: interface here includes every fact a caller must know. +- **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**. + +## Going deeper + +- **Deepening a cluster given its dependencies**, see [DEEPENING.md](DEEPENING.md): dependency categories, seam discipline, and replace-don't-layer testing. +- **Exploring alternative interfaces**, see [DESIGN-IT-TWICE.md](DESIGN-IT-TWICE.md): spin up parallel sub-agents to design the interface several radically different ways, then compare on depth, locality, and seam placement. diff --git a/.claude/skills/codebase-design b/.claude/skills/codebase-design new file mode 120000 index 00000000000..08b466ebb0a --- /dev/null +++ b/.claude/skills/codebase-design @@ -0,0 +1 @@ +../../.agents/skills/codebase-design \ No newline at end of file From 9f718e4a47ca857b701fedc3b2489fbd14862477 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 14:58:06 -0700 Subject: [PATCH 02/30] chore(cleanup): remove dead client hooks, query exports, and orphaned scheduled-tasks utils --- .../knowledge/hooks/use-knowledge-upload.ts | 4 +- .../utils/calendar-grid.test.ts | 45 --- .../scheduled-tasks/utils/calendar-grid.ts | 237 ---------------- .../scheduled-tasks/utils/recurrence.test.ts | 268 ------------------ .../scheduled-tasks/utils/recurrence.ts | 263 ----------------- .../utils/schedule-events.test.ts | 100 ------- .../scheduled-tasks/utils/schedule-events.ts | 184 ------------ .../w/[workflowId]/utils/index.ts | 1 - .../w/[workflowId]/utils/node-derivation.ts | 21 -- .../w/components/sidebar/hooks/index.ts | 1 - .../sidebar/hooks/use-auto-scroll.ts | 103 ------- apps/sim/hooks/queries/general-settings.ts | 8 - apps/sim/hooks/queries/mothership-admin.ts | 1 - .../hooks/queries/mothership-chats.test.ts | 52 +--- apps/sim/hooks/queries/mothership-chats.ts | 149 +--------- .../hooks/queries/oauth/oauth-connections.ts | 1 - apps/sim/hooks/queries/organization.test.tsx | 30 +- apps/sim/hooks/queries/organization.ts | 20 -- apps/sim/hooks/queries/schedules.ts | 1 - apps/sim/hooks/queries/subscription.ts | 21 -- apps/sim/hooks/queries/tables.ts | 12 - apps/sim/hooks/queries/workspace-files.ts | 1 - apps/sim/hooks/use-scroll-anchor.test.ts | 41 --- apps/sim/hooks/use-scroll-anchor.ts | 172 ----------- 24 files changed, 6 insertions(+), 1730 deletions(-) delete mode 100644 apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.test.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.test.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.test.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/node-derivation.ts delete mode 100644 apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-auto-scroll.ts delete mode 100644 apps/sim/hooks/use-scroll-anchor.test.ts delete mode 100644 apps/sim/hooks/use-scroll-anchor.ts diff --git a/apps/sim/app/workspace/[workspaceId]/knowledge/hooks/use-knowledge-upload.ts b/apps/sim/app/workspace/[workspaceId]/knowledge/hooks/use-knowledge-upload.ts index 10a543c09ac..34320dd67ab 100644 --- a/apps/sim/app/workspace/[workspaceId]/knowledge/hooks/use-knowledge-upload.ts +++ b/apps/sim/app/workspace/[workspaceId]/knowledge/hooks/use-knowledge-upload.ts @@ -1,6 +1,6 @@ import { useCallback, useState } from 'react' import { createLogger } from '@sim/logger' -import { getErrorMessage } from '@sim/utils/errors' +import { getErrorMessage, toError } from '@sim/utils/errors' import { useQueryClient } from '@tanstack/react-query' import type { V2KnowledgeDocumentSummary } from '@/lib/api/contracts/v2/knowledge' import type { KnowledgeDocumentUploadRecipe } from '@/lib/knowledge/upload-metadata' @@ -176,7 +176,7 @@ export function useKnowledgeUpload(options: UseKnowledgeUploadOptions = {}) { } else if (result?.status === 'rejected') { failed.push({ file: files[idx], - error: result.reason instanceof Error ? result.reason : new Error(String(result.reason)), + error: toError(result.reason), }) } }) diff --git a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.test.ts b/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.test.ts deleted file mode 100644 index 428db9421da..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.test.ts +++ /dev/null @@ -1,45 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { - EVENT_CHIP_HEIGHT, - layoutColumn, - visibleRange, -} from '@/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid' - -// June 10, 2026 is a Wednesday. June 1, 2026 is a Monday. -const ANCHOR = new Date(2026, 5, 10) - -describe('visibleRange', () => { - it('pads the rendered span by a day each side to cover timezone offset slop', () => { - // Week of Jun 7–13, 2026 (Sun–Sat); padded to Jun 6 → Jun 14. - const { start, end } = visibleRange('week', ANCHOR) - expect(start).toEqual(new Date(2026, 5, 6)) - expect(end.getDate()).toBe(14) - }) -}) - -describe('layoutColumn', () => { - const at = (h: number, m: number) => ({ start: new Date(2026, 5, 15, h, m) }) - - it('splits events within one pill-height of each other into side-by-side lanes', () => { - const placed = layoutColumn([at(9, 0), at(9, 10)], EVENT_CHIP_HEIGHT) - expect(placed.map((p) => ({ lane: p.lane, lanes: p.lanes }))).toEqual([ - { lane: 0, lanes: 2 }, - { lane: 1, lanes: 2 }, - ]) - }) - - it('reuses a freed lane after the overlap clears and resets the cluster', () => { - const placed = layoutColumn([at(9, 0), at(9, 10), at(12, 0)], EVENT_CHIP_HEIGHT) - expect(placed.map((p) => ({ lane: p.lane, lanes: p.lanes }))).toEqual([ - { lane: 0, lanes: 2 }, - { lane: 1, lanes: 2 }, - { lane: 0, lanes: 1 }, - ]) - }) - - it('sorts by start time before assigning lanes', () => { - const placed = layoutColumn([at(9, 10), at(9, 0)], EVENT_CHIP_HEIGHT) - expect(placed[0].item).toEqual(at(9, 0)) - expect(placed[1].item).toEqual(at(9, 10)) - }) -}) diff --git a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.ts b/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.ts deleted file mode 100644 index fd66e89fda3..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/calendar-grid.ts +++ /dev/null @@ -1,237 +0,0 @@ -import { - addDays, - addMonths, - addWeeks, - eachDayOfInterval, - endOfDay, - endOfMonth, - endOfWeek, - format, - isSameDay, - isSameMonth, - startOfDay, - startOfMonth, - startOfWeek, -} from 'date-fns' - -/** The granularity the calendar is currently rendering. */ -export type CalendarScope = 'day' | 'week' | 'month' - -/** A single day rendered in any view (month cell, week/day column header). */ -export interface CalendarDayCell { - date: Date - isToday: boolean - /** `false` for leading/trailing spillover days outside the focused month. */ - isCurrentMonth: boolean -} - -export interface MonthGrid { - kind: 'month' - /** Calendar rows (4–6) of 7 day cells each, including spillover days. */ - weeks: CalendarDayCell[][] -} - -export interface WeekGrid { - kind: 'week' - days: CalendarDayCell[] - hours: number[] -} - -export interface DayGrid { - kind: 'day' - day: CalendarDayCell - hours: number[] -} - -export type CalendarGrid = MonthGrid | WeekGrid | DayGrid - -/** Sunday-first, matching the emcn `Calendar` picker. */ -export const WEEK_STARTS_ON = 0 as const - -/** Hours of the day rendered as rows in the week/day time grid. */ -export const HOURS: number[] = Array.from({ length: 24 }, (_, hour) => hour) - -/** Fixed pixel height of one hour row in the time grid. */ -export const TIME_SLOT_HEIGHT = 48 - -/** Rendered height of a task pill in the time grid, used for overlap detection. */ -export const EVENT_CHIP_HEIGHT = 22 - -const BASE_WEEKDAYS = ['Sun', 'Mon', 'Tue', 'Wed', 'Thu', 'Fri', 'Sat'] as const - -/** Weekday header labels rotated to honor {@link WEEK_STARTS_ON}. */ -export const WEEKDAY_LABELS: string[] = [ - ...BASE_WEEKDAYS.slice(WEEK_STARTS_ON), - ...BASE_WEEKDAYS.slice(0, WEEK_STARTS_ON), -] - -function toCell(date: Date, today: Date, anchor?: Date): CalendarDayCell { - return { - date, - isToday: isSameDay(date, today), - isCurrentMonth: anchor ? isSameMonth(date, anchor) : true, - } -} - -/** Move the anchor date forward (`delta > 0`) or back by one unit of `scope`. */ -export function advanceAnchor(anchor: Date, scope: CalendarScope, delta: number): Date { - switch (scope) { - case 'month': - return addMonths(anchor, delta) - case 'week': - return addWeeks(anchor, delta) - case 'day': - return addDays(anchor, delta) - } -} - -function buildMonthGrid(anchor: Date, today: Date): MonthGrid { - const start = startOfWeek(startOfMonth(anchor), { weekStartsOn: WEEK_STARTS_ON }) - const end = endOfWeek(endOfMonth(anchor), { weekStartsOn: WEEK_STARTS_ON }) - const days = eachDayOfInterval({ start, end }) - const weeks: CalendarDayCell[][] = [] - for (let i = 0; i < days.length; i += 7) { - weeks.push(days.slice(i, i + 7).map((date) => toCell(date, today, anchor))) - } - return { kind: 'month', weeks } -} - -function buildWeekGrid(anchor: Date, today: Date): WeekGrid { - const start = startOfWeek(anchor, { weekStartsOn: WEEK_STARTS_ON }) - const end = endOfWeek(anchor, { weekStartsOn: WEEK_STARTS_ON }) - const days = eachDayOfInterval({ start, end }).map((date) => toCell(date, today)) - return { kind: 'week', days, hours: HOURS } -} - -function buildDayGrid(anchor: Date, today: Date): DayGrid { - return { kind: 'day', day: toCell(anchor, today), hours: HOURS } -} - -/** - * Pure, React-free derivation of the renderable grid for a given scope and - * anchor. `today` is passed in (never read from the clock here) so the result - * is fully deterministic and unit-testable. - */ -export function buildCalendarGrid(scope: CalendarScope, anchor: Date, today: Date): CalendarGrid { - switch (scope) { - case 'month': - return buildMonthGrid(anchor, today) - case 'week': - return buildWeekGrid(anchor, today) - case 'day': - return buildDayGrid(anchor, today) - } -} - -/** The day span the current view renders (month spans its 4–6 spillover weeks). */ -function visibleSpan(scope: CalendarScope, anchor: Date): { start: Date; end: Date } { - switch (scope) { - case 'month': - return { - start: startOfWeek(startOfMonth(anchor), { weekStartsOn: WEEK_STARTS_ON }), - end: endOfWeek(endOfMonth(anchor), { weekStartsOn: WEEK_STARTS_ON }), - } - case 'week': - return { - start: startOfWeek(anchor, { weekStartsOn: WEEK_STARTS_ON }), - end: endOfWeek(anchor, { weekStartsOn: WEEK_STARTS_ON }), - } - case 'day': - return { start: startOfDay(anchor), end: endOfDay(anchor) } - } -} - -/** - * The instant window that bounds recurrence expansion for the current view, - * padded one day past the rendered span on each side. The grid frame is in the - * viewer's zone, but each occurrence is positioned in its task's own zone, so an - * instant up to a full UTC offset (≤14h) outside the rendered span can still - * fall on a visible day. The pad guarantees those boundary occurrences are - * expanded; `bucketEventsByDay` then places each on its zoned day, so any that - * land off-screen sit in an unrendered bucket and never show. - */ -export function visibleRange(scope: CalendarScope, anchor: Date): { start: Date; end: Date } { - const span = visibleSpan(scope, anchor) - return { start: addDays(span.start, -1), end: addDays(span.end, 1) } -} - -/** Toolbar period label, e.g. `June 2026`, `Jun 7 – 13, 2026`, `June 10, 2026`. */ -export function formatScopeLabel(scope: CalendarScope, anchor: Date): string { - if (scope === 'month') return format(anchor, 'MMMM yyyy') - if (scope === 'day') return format(anchor, 'MMMM d, yyyy') - const start = startOfWeek(anchor, { weekStartsOn: WEEK_STARTS_ON }) - const end = endOfWeek(anchor, { weekStartsOn: WEEK_STARTS_ON }) - if (isSameMonth(start, end)) return `${format(start, 'MMM d')} – ${format(end, 'd, yyyy')}` - return `${format(start, 'MMM d')} – ${format(end, 'MMM d, yyyy')}` -} - -/** Display label for an hour-of-day gutter row, e.g. `7 AM`, `12 PM`. */ -export function formatHourLabel(hour: number): string { - return format(new Date(2000, 0, 1, hour), 'h a') -} - -/** - * Vertical pixel offset of a moment within the day, measured from the top of the - * time grid's slot stack (the `00:00` row). Positions the current-time - * indicator. Pure and clock-free — `date` is passed in so callers control "now". - */ -export function timeToOffset(date: Date): number { - return (date.getHours() + date.getMinutes() / 60) * TIME_SLOT_HEIGHT -} - -/** Wire-format time string for an hour slot, e.g. `07:00`. */ -export function formatSlotTime(hour: number): string { - return `${hour.toString().padStart(2, '0')}:00` -} - -/** A time-grid item placed at its minute, with its column slot within an overlap cluster. */ -export interface PlacedEvent { - item: T - /** Pixel offset from the top of the day column. */ - topPx: number - /** 0-based column index within the overlap cluster. */ - lane: number - /** Total columns the overlap cluster spans (1 when nothing overlaps). */ - lanes: number -} - -/** - * Google-Calendar-style lane assignment for events sharing a day column. Items - * whose pill rectangles (`[topPx, topPx + chipHeight]`) intersect form a cluster - * and split the width into side-by-side lanes; non-overlapping items keep the - * full width. Pure: positions come from {@link timeToOffset}. - */ -export function layoutColumn( - items: T[], - chipHeight: number -): PlacedEvent[] { - const sorted = [...items].sort((a, b) => a.start.getTime() - b.start.getTime()) - const placed: PlacedEvent[] = [] - let cluster: PlacedEvent[] = [] - let laneBottoms: number[] = [] - - const closeCluster = () => { - for (const entry of cluster) entry.lanes = laneBottoms.length - cluster = [] - laneBottoms = [] - } - - for (const item of sorted) { - const topPx = timeToOffset(item.start) - if (laneBottoms.length > 0 && laneBottoms.every((bottom) => topPx >= bottom)) { - closeCluster() - } - let lane = laneBottoms.findIndex((bottom) => topPx >= bottom) - if (lane === -1) { - lane = laneBottoms.length - laneBottoms.push(topPx + chipHeight) - } else { - laneBottoms[lane] = topPx + chipHeight - } - const entry: PlacedEvent = { item, topPx, lane, lanes: 1 } - cluster.push(entry) - placed.push(entry) - } - closeCluster() - return placed -} diff --git a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.test.ts b/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.test.ts deleted file mode 100644 index cb8bab8f1a2..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.test.ts +++ /dev/null @@ -1,268 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { - cronToRecurrence, - expandOccurrences, - type Recurrence, - recurrenceToCron, - recurrenceToScheduleFields, -} from './recurrence' - -const once: Recurrence = { frequency: 'once', weekdays: [], end: { type: 'never' } } - -describe('recurrenceToCron', () => { - it('builds a weekly expression from the selected weekdays, sorted and deduped', () => { - expect( - recurrenceToCron( - { frequency: 'weekly', weekdays: [3, 1, 1], end: { type: 'never' } }, - '2026-06-15', - '08:00' - ) - ).toBe('0 8 * * 1,3') - }) - - it('builds a monthly nth-weekday expression (2026-06-15 is the third Monday)', () => { - expect( - recurrenceToCron( - { frequency: 'monthly', weekdays: [], monthlyMode: 'nth-weekday', end: { type: 'never' } }, - '2026-06-15', - '09:30' - ) - ).toBe('30 9 * * 1#3') - }) - - it('builds a monthly last-weekday expression', () => { - expect( - recurrenceToCron( - { frequency: 'monthly', weekdays: [], monthlyMode: 'last-weekday', end: { type: 'never' } }, - '2026-06-29', - '09:30' - ) - ).toBe('30 9 * * 1#L') - }) - - it('clamps a 5th-occurrence nth-weekday to last-weekday so no month is skipped', () => { - // 2026-06-29 is the fifth Monday of June; `#5` would skip months without one. - expect( - recurrenceToCron( - { frequency: 'monthly', weekdays: [], monthlyMode: 'nth-weekday', end: { type: 'never' } }, - '2026-06-29', - '09:30' - ) - ).toBe('30 9 * * 1#L') - }) -}) - -describe('recurrenceToScheduleFields', () => { - it('resolves a one-time launch to the UTC instant of that wall-clock in the zone', () => { - const fields = recurrenceToScheduleFields(once, '2026-06-15', '09:00', 'America/New_York') - expect(fields.cronExpression).toBeNull() - expect(fields.time).toBe('2026-06-15T13:00:00.000Z') - expect(fields.lifecycle).toBe('persistent') - }) - - it('maps "ends after N" to maxRuns with an until_complete lifecycle', () => { - const fields = recurrenceToScheduleFields( - { frequency: 'daily', weekdays: [], end: { type: 'after', count: 5 } }, - '2026-06-15', - '09:00', - 'UTC' - ) - expect(fields.cronExpression).toBe('0 9 * * *') - expect(fields.maxRuns).toBe(5) - expect(fields.lifecycle).toBe('until_complete') - expect(fields.endsAt).toBeUndefined() - }) - - it('maps "ends on date" to an end-of-day boundary in the zone', () => { - const fields = recurrenceToScheduleFields( - { frequency: 'daily', weekdays: [], end: { type: 'on', date: '2026-07-01' } }, - '2026-06-15', - '09:00', - 'UTC' - ) - expect(fields.endsAt).toBe('2026-07-01T23:59:59.000Z') - expect(fields.maxRuns).toBeUndefined() - expect(fields.lifecycle).toBe('persistent') - }) -}) - -describe('cronToRecurrence', () => { - const anchor = new Date('2026-06-15T09:00:00Z') - - it('recovers daily, weekly, and monthly cadences', () => { - expect( - cronToRecurrence({ - cronExpression: '30 9 * * *', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence.frequency - ).toBe('daily') - - const weekly = cronToRecurrence({ - cronExpression: '0 8 * * 1,3', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence - expect(weekly.frequency).toBe('weekly') - expect(weekly.weekdays).toEqual([1, 3]) - - const monthly = cronToRecurrence({ - cronExpression: '5 7 15 * *', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence - expect(monthly.frequency).toBe('monthly') - expect(monthly.monthlyMode).toBe('day-of-month') - }) - - it('recovers monthly nth-weekday, monthly last-weekday, and yearly cadences', () => { - const nthWeekday = cronToRecurrence({ - cronExpression: '30 9 * * 1#3', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence - expect(nthWeekday.frequency).toBe('monthly') - expect(nthWeekday.monthlyMode).toBe('nth-weekday') - - const lastWeekday = cronToRecurrence({ - cronExpression: '30 9 * * 1#L', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence - expect(lastWeekday.frequency).toBe('monthly') - expect(lastWeekday.monthlyMode).toBe('last-weekday') - - expect( - cronToRecurrence({ - cronExpression: '30 9 15 6 *', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence.frequency - ).toBe('yearly') - }) - - it("accepts croner's alternate Sunday digit (7) for monthly weekday anchors", () => { - const nth = cronToRecurrence({ - cronExpression: '30 9 * * 7#3', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence - expect(nth.frequency).toBe('monthly') - expect(nth.monthlyMode).toBe('nth-weekday') - - const last = cronToRecurrence({ - cronExpression: '30 9 * * 7#L', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence - expect(last.frequency).toBe('monthly') - expect(last.monthlyMode).toBe('last-weekday') - }) - - it('leaves a 5th-occurrence (#5) cron as custom so its month-skipping is preserved', () => { - const { recurrence } = cronToRecurrence({ - cronExpression: '30 9 * * 1#5', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }) - expect(recurrence.frequency).toBe('custom') - expect(recurrence.cron).toBe('30 9 * * 1#5') - }) - - it('falls back to custom for an expression it did not author', () => { - const { recurrence } = cronToRecurrence({ - cronExpression: '*/5 * * * *', - maxRuns: null, - endsAt: null, - anchor, - timezone: 'UTC', - }) - expect(recurrence.frequency).toBe('custom') - expect(recurrence.cron).toBe('*/5 * * * *') - }) - - it('recovers the end boundary from maxRuns and endsAt', () => { - expect( - cronToRecurrence({ - cronExpression: '0 9 * * *', - maxRuns: 5, - endsAt: null, - anchor, - timezone: 'UTC', - }).recurrence.end - ).toEqual({ type: 'after', count: 5 }) - - expect( - cronToRecurrence({ - cronExpression: '0 9 * * *', - maxRuns: null, - endsAt: '2026-07-01T23:59:59Z', - anchor, - timezone: 'UTC', - }).recurrence.end - ).toEqual({ type: 'on', date: '2026-07-01' }) - }) -}) - -describe('expandOccurrences', () => { - const base = { - cronExpression: '0 12 * * *', - timezone: 'UTC', - rangeStart: new Date('2026-06-01T00:00:00Z'), - rangeEnd: new Date('2026-06-03T23:59:59Z'), - from: new Date('2026-05-31T00:00:00Z'), - } - - it('skips excluded occurrences', () => { - const occurrences = expandOccurrences({ ...base, excludedDates: ['2026-06-02T12:00:00.000Z'] }) - expect(occurrences.map((d) => d.toISOString())).toEqual([ - '2026-06-01T12:00:00.000Z', - '2026-06-03T12:00:00.000Z', - ]) - }) - - it('stops at the recurrence end boundary', () => { - const occurrences = expandOccurrences({ ...base, endsAt: new Date('2026-06-02T12:00:00Z') }) - expect(occurrences.map((d) => d.toISOString())).toEqual([ - '2026-06-01T12:00:00.000Z', - '2026-06-02T12:00:00.000Z', - ]) - }) - - it('materializes a monthly nth-weekday cron (third Monday of each month)', () => { - const occurrences = expandOccurrences({ - cronExpression: '30 9 * * 1#3', - timezone: 'UTC', - rangeStart: new Date('2026-06-01T00:00:00Z'), - rangeEnd: new Date('2026-08-31T23:59:59Z'), - from: new Date('2026-05-31T00:00:00Z'), - }) - expect(occurrences.map((d) => d.toISOString())).toEqual([ - '2026-06-15T09:30:00.000Z', - '2026-07-20T09:30:00.000Z', - '2026-08-17T09:30:00.000Z', - ]) - }) - - it('returns nothing for an invalid expression instead of throwing', () => { - expect(expandOccurrences({ ...base, cronExpression: 'not-a-cron' })).toEqual([]) - }) -}) diff --git a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.ts b/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.ts deleted file mode 100644 index 9a132b10f15..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence.ts +++ /dev/null @@ -1,263 +0,0 @@ -import { Cron } from 'croner' -import { zonedWallClock, zonedWallClockToUtc } from '@/lib/core/utils/timezone' - -/** - * Recurrence cadence the modal exposes. `once` is a one-time launch; `custom` - * preserves a cron expression the UI did not author (e.g. a task created - * conversationally) so editing never silently rewrites it. - */ -export type RecurrenceFrequency = 'once' | 'daily' | 'weekly' | 'monthly' | 'yearly' | 'custom' - -/** - * How a monthly recurrence anchors within the month, mirroring a calendar app's - * monthly sub-options. `day-of-month` repeats on the launch date's day number - * (e.g. the 15th); `nth-weekday` on the same ordinal weekday (e.g. the third - * Tuesday, croner `2#3`); `last-weekday` on the final weekday of that kind (e.g. - * the last Tuesday, croner `2#L`). The weekday and ordinal are read from the - * launch date at cron-build time — on edit the launch date is an actual - * occurrence, so the mode round-trips to the same day. - */ -export type MonthlyMode = 'day-of-month' | 'nth-weekday' | 'last-weekday' - -/** When a recurrence stops, mirroring the three calendar-app end options. */ -export type RecurrenceEnd = - | { type: 'never' } - | { type: 'on'; date: string } - | { type: 'after'; count: number } - -export interface Recurrence { - frequency: RecurrenceFrequency - /** Weekly only: weekdays 0 (Sun) – 6 (Sat). Empty falls back to the launch day's weekday. */ - weekdays: number[] - /** Monthly only: how it anchors within the month. Defaults to `day-of-month`. */ - monthlyMode?: MonthlyMode - end: RecurrenceEnd - /** `custom` only: the raw cron expression, passed through unchanged on save. */ - cron?: string -} - -/** Upper bound on occurrences materialized for one schedule in a single view. */ -const MAX_OCCURRENCES_PER_VIEW = 500 - -/** - * Builds the cron expression for a recurrence, evaluated in the schedule's - * timezone against the launch day/time. Returns `null` for a one-time task and - * the preserved expression for a `custom` recurrence. The weekday/day-of-month - * are read from the launch date as a zone-independent calendar date (UTC parse), - * so the cron targets the right day regardless of the device zone. - */ -export function recurrenceToCron( - recurrence: Recurrence, - launchDate: string, - launchTime: string -): string | null { - if (recurrence.frequency === 'once') return null - if (recurrence.frequency === 'custom') return recurrence.cron ?? null - - const [hour, minute] = launchTime.split(':').map(Number) - const launchDay = new Date(`${launchDate}T00:00:00Z`) - - switch (recurrence.frequency) { - case 'daily': - return `${minute} ${hour} * * *` - case 'weekly': { - const days = recurrence.weekdays.length > 0 ? recurrence.weekdays : [launchDay.getUTCDay()] - return `${minute} ${hour} * * ${[...new Set(days)].sort((a, b) => a - b).join(',')}` - } - case 'monthly': { - const weekday = launchDay.getUTCDay() - switch (recurrence.monthlyMode ?? 'day-of-month') { - case 'nth-weekday': { - // A 5th occurrence is always the last weekday of the month; emit `#L` - // rather than `#5` so no month without a 5th occurrence is silently - // skipped (the picker only offers nth for the 1st–4th, but the launch - // date can still drift to a 5th via the footer date picker). - const nth = Math.ceil(launchDay.getUTCDate() / 7) - return nth >= 5 - ? `${minute} ${hour} * * ${weekday}#L` - : `${minute} ${hour} * * ${weekday}#${nth}` - } - case 'last-weekday': - return `${minute} ${hour} * * ${weekday}#L` - default: - return `${minute} ${hour} ${launchDay.getUTCDate()} * *` - } - } - case 'yearly': - return `${minute} ${hour} ${launchDay.getUTCDate()} ${launchDay.getUTCMonth() + 1} *` - } -} - -export interface ScheduleFields { - cronExpression: string | null - time?: string - maxRuns?: number - endsAt?: string - lifecycle: 'persistent' | 'until_complete' -} - -/** - * Translates a recurrence + launch into the wire fields the schedules API - * accepts: a one-time `time`, or a `cronExpression` with an optional end - * boundary (`maxRuns` for "after N", `endsAt` for "on date"). The launch - * date/time and end date are wall-clock in `timezone`, so they resolve to UTC - * instants in that zone — matching how the recurring cron is evaluated. - */ -export function recurrenceToScheduleFields( - recurrence: Recurrence, - launchDate: string, - launchTime: string, - timezone: string -): ScheduleFields { - const cronExpression = recurrenceToCron(recurrence, launchDate, launchTime) - if (!cronExpression) { - return { - cronExpression: null, - time: zonedWallClockToUtc(`${launchDate}T${launchTime}`, timezone).toISOString(), - lifecycle: 'persistent', - } - } - - const { end } = recurrence - return { - cronExpression, - maxRuns: end.type === 'after' ? end.count : undefined, - endsAt: - end.type === 'on' - ? zonedWallClockToUtc(`${end.date}T23:59:59`, timezone).toISOString() - : undefined, - lifecycle: end.type === 'after' ? 'until_complete' : 'persistent', - } -} - -const CRON_FIELD_COUNT = 5 - -/** - * Recovers the modal's recurrence + launch fields from a stored schedule so - * editing reflects what is persisted, read back in the schedule's `timezone`. - * A recurring task's launch clock comes from its cron; a one-time task's comes - * from the stored instant (`anchor`). A cron the UI did not author maps to - * `custom` and round-trips untouched. - */ -export function cronToRecurrence(params: { - cronExpression: string | null - maxRuns: number | null - endsAt: string | null - anchor: Date - timezone: string -}): { recurrence: Recurrence; launchTime: string } { - const { cronExpression, maxRuns, endsAt, anchor, timezone } = params - - const end: RecurrenceEnd = endsAt - ? { type: 'on', date: zonedWallClock(new Date(endsAt), timezone).slice(0, 10) } - : maxRuns - ? { type: 'after', count: maxRuns } - : { type: 'never' } - const anchorTime = zonedWallClock(anchor, timezone).slice(11, 16) - - if (!cronExpression) { - return { - recurrence: { frequency: 'once', weekdays: [], end }, - launchTime: anchorTime, - } - } - - const parts = cronExpression.trim().split(/\s+/) - if (parts.length !== CRON_FIELD_COUNT) { - return { - recurrence: { frequency: 'custom', weekdays: [], end, cron: cronExpression }, - launchTime: anchorTime, - } - } - - const [minute, hour, dayOfMonth, month, dayOfWeek] = parts - const launchTime = `${hour.padStart(2, '0')}:${minute.padStart(2, '0')}` - const isNumeric = (value: string) => /^\d+$/.test(value) - const numbersAreValid = isNumeric(minute) && isNumeric(hour) - - if (numbersAreValid && month === '*') { - if (dayOfMonth === '*' && dayOfWeek === '*') { - return { recurrence: { frequency: 'daily', weekdays: [], end }, launchTime } - } - if (dayOfMonth === '*' && /^[0-6](,[0-6])*$/.test(dayOfWeek)) { - const weekdays = dayOfWeek.split(',').map(Number) - return { recurrence: { frequency: 'weekly', weekdays, end }, launchTime } - } - // Accept croner's alternate Sunday digit (`7`) so externally-authored - // `7#…` crons round-trip; the picker canonicalizes them to `0#…` on save. - // A 5th occurrence (`#5`) is intentionally NOT matched — it falls through to - // `custom` so its month-skipping behavior is preserved verbatim rather than - // silently rewritten to `#L`. - if (dayOfMonth === '*' && /^[0-7]#[1-4]$/.test(dayOfWeek)) { - return { - recurrence: { frequency: 'monthly', weekdays: [], monthlyMode: 'nth-weekday', end }, - launchTime, - } - } - if (dayOfMonth === '*' && /^[0-7]#L$/.test(dayOfWeek)) { - return { - recurrence: { frequency: 'monthly', weekdays: [], monthlyMode: 'last-weekday', end }, - launchTime, - } - } - if (isNumeric(dayOfMonth) && dayOfWeek === '*') { - return { - recurrence: { frequency: 'monthly', weekdays: [], monthlyMode: 'day-of-month', end }, - launchTime, - } - } - } - - if (numbersAreValid && isNumeric(dayOfMonth) && isNumeric(month) && dayOfWeek === '*') { - return { recurrence: { frequency: 'yearly', weekdays: [], end }, launchTime } - } - - return { - recurrence: { frequency: 'custom', weekdays: [], end, cron: cronExpression }, - launchTime, - } -} - -/** - * Materializes a recurring schedule's run instants inside `[rangeStart, rangeEnd]` - * that are still upcoming (after `from`), skipping individually deleted - * occurrences and stopping at the recurrence end. Pure given its inputs. - * - * The lower bound is inclusive: croner's `nextRun(date)` returns the first - * occurrence strictly after `date`, so the search starts one millisecond before - * the bound to admit an occurrence landing exactly on it. - */ -export function expandOccurrences(params: { - cronExpression: string - timezone: string - rangeStart: Date - rangeEnd: Date - from: Date - excludedDates?: string[] | null - endsAt?: Date | null -}): Date[] { - const { cronExpression, timezone, rangeStart, rangeEnd, from, excludedDates, endsAt } = params - - let cron: Cron - try { - cron = new Cron(cronExpression, timezone ? { timezone } : undefined) - } catch { - return [] - } - - const excluded = new Set( - (excludedDates ?? []).map((iso) => new Date(iso).getTime()).filter((ms) => !Number.isNaN(ms)) - ) - const lowerBound = rangeStart.getTime() > from.getTime() ? rangeStart : from - - const occurrences: Date[] = [] - let cursor = new Date(lowerBound.getTime() - 1) - for (let i = 0; i < MAX_OCCURRENCES_PER_VIEW; i++) { - const next = cron.nextRun(cursor) - if (!next || next.getTime() > rangeEnd.getTime()) break - if (endsAt && next.getTime() > endsAt.getTime()) break - if (!excluded.has(next.getTime())) occurrences.push(next) - cursor = next - } - return occurrences -} diff --git a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.test.ts b/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.test.ts deleted file mode 100644 index 3ecef5ef5c4..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.test.ts +++ /dev/null @@ -1,100 +0,0 @@ -import { describe, expect, it } from 'vitest' -import type { WorkspaceScheduleRow } from '@/lib/api/contracts/schedules' -import { - type ScheduledTask, - scheduleToTasks, - taskToCalendarEvent, -} from '@/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events' - -function makeTask(overrides: Partial): ScheduledTask { - return { - id: 't1', - scheduleId: 's1', - sourceUserId: 'user-1', - prompt: 'Summarize yesterday', - runAt: new Date('2026-06-10T14:30:00.000Z'), - timezone: 'UTC', - status: 'pending', - recurring: false, - disabled: false, - ...overrides, - } -} - -const RANGE_START = new Date('2026-06-08T00:00:00.000Z') -const RANGE_END = new Date('2026-06-14T23:59:59.999Z') -const NOW = new Date('2026-06-10T00:00:00.000Z') - -function makeRow(overrides: Partial): WorkspaceScheduleRow { - return { - id: 's1', - sourceType: 'job', - prompt: 'Summarize yesterday', - timezone: 'UTC', - status: 'active', - cronExpression: null, - nextRunAt: null, - lastRanAt: null, - lastFailedAt: null, - excludedDates: null, - endsAt: null, - contexts: null, - ...overrides, - } as WorkspaceScheduleRow -} - -describe('taskToCalendarEvent', () => { - it('shifts the position to the task timezone, not the run instant', () => { - const task = makeTask({ - runAt: new Date('2026-06-10T14:30:00.000Z'), - timezone: 'America/New_York', - }) - const event = taskToCalendarEvent(task) - // 14:30 UTC is 10:30 in New York (EDT, UTC-4) on this date. - expect(event.start.getHours()).toBe(10) - expect(event.start.getMinutes()).toBe(30) - expect(event.start.getDate()).toBe(10) - }) -}) - -describe('scheduleToTasks', () => { - it('marks the last run as error when the latest failure is at or after the last success', () => { - const tasks = scheduleToTasks( - makeRow({ - status: 'completed', - lastRanAt: '2026-06-09T09:00:00.000Z', - lastFailedAt: '2026-06-09T09:00:00.000Z', - }), - RANGE_START, - RANGE_END, - NOW - ) - expect(tasks[0].status).toBe('error') - }) - - it('skips individually-deleted occurrences of a recurring task', () => { - const tasks = scheduleToTasks( - makeRow({ - cronExpression: '0 12 * * *', - excludedDates: ['2026-06-12T12:00:00.000Z'], - }), - RANGE_START, - RANGE_END, - NOW - ) - const runs = tasks.filter((t) => t.status === 'pending').map((t) => t.runAt.toISOString()) - expect(runs).not.toContain('2026-06-12T12:00:00.000Z') - }) - - it('expands a paused recurring schedule as disabled occurrences so it stays resumable', () => { - const tasks = scheduleToTasks( - makeRow({ status: 'disabled', cronExpression: '0 12 * * *' }), - RANGE_START, - RANGE_END, - NOW - ) - const pending = tasks.filter((t) => t.status === 'pending') - expect(pending.length).toBe(5) // Jun 10–14 noon (NOW is Jun 10 00:00) - expect(pending.every((t) => t.disabled)).toBe(true) - }) -}) diff --git a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.ts b/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.ts deleted file mode 100644 index 0361e6325b3..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/scheduled-tasks/utils/schedule-events.ts +++ /dev/null @@ -1,184 +0,0 @@ -import { truncate } from '@sim/utils/string' -import { format } from 'date-fns' -import type { WorkspaceScheduleRow } from '@/lib/api/contracts/schedules' -import { zonedClockDate } from '@/lib/core/utils/timezone' -import { expandOccurrences } from '@/app/workspace/[workspaceId]/scheduled-tasks/utils/recurrence' -import type { ChatContext } from '@/stores/panel' - -/** - * Lifecycle of a scheduled task occurrence: `pending` has not run yet, and - * `error`/`completed` are terminal outcomes of a past run. - */ -export type ScheduledTaskStatus = 'pending' | 'error' | 'completed' - -/** - * One occurrence of a scheduled task as the calendar renders it. A recurring - * schedule expands into many of these; one-time tasks produce a single one. - */ -export interface ScheduledTask { - /** Occurrence-unique key for rendering — not the backend identifier. */ - id: string - /** The persisted schedule id, used to edit or delete the task. */ - scheduleId: string - /** The user whose authority executes the task and who may edit its contents. */ - sourceUserId: string | null - /** The instruction Sim runs. Doubles as the calendar title. */ - prompt: string - /** Resources the prompt `@`-mentions / skills it `/`-invokes, when any. */ - contexts?: ChatContext[] - /** When this occurrence runs (`pending`) or ran (`completed`/`error`). */ - runAt: Date - /** IANA timezone the launch time was captured in. */ - timezone: string - status: ScheduledTaskStatus - /** Whether the task repeats — drives edit seeding and the delete dialog. */ - recurring: boolean - /** - * Whether the parent schedule is paused. A paused recurring task still shows - * its upcoming occurrences (rendered dimmed) so it can be found and resumed; - * it will not run until resumed. Always `false` for past runs and one-time tasks. - */ - disabled: boolean -} - -/** - * A scheduled task positioned on the calendar. Derived from a - * {@link ScheduledTask} via {@link taskToCalendarEvent}; keeps the full `task` - * for the click-through details modal. - */ -export interface CalendarEvent { - id: string - /** - * The occurrence's wall-clock position in the task's own timezone, as a - * device-local {@link zonedClockDate} — a layout coordinate, not the real - * instant. Keeps the calendar showing each task at the local time it was - * scheduled for, matching the modal. The true instant lives in `task.runAt`. - */ - start: Date - title: string - task: ScheduledTask -} - -/** Bucket key for a day cell (`yyyy-MM-dd`). */ -export function dayKey(date: Date): string { - return format(date, 'yyyy-MM-dd') -} - -/** The most recent terminal run of a schedule, or `null` if it has never run. */ -function lastRunMarker( - row: WorkspaceScheduleRow -): { at: Date; status: ScheduledTaskStatus } | null { - const ranAt = row.lastRanAt ? new Date(row.lastRanAt) : null - const failedAt = row.lastFailedAt ? new Date(row.lastFailedAt) : null - if (failedAt && (!ranAt || failedAt.getTime() >= ranAt.getTime())) { - return { at: failedAt, status: 'error' } - } - if (ranAt) return { at: ranAt, status: 'completed' } - return null -} - -function withinRange(date: Date, rangeStart: Date, rangeEnd: Date): boolean { - return date.getTime() >= rangeStart.getTime() && date.getTime() <= rangeEnd.getTime() -} - -/** - * Maps a persisted job schedule into the occurrences visible in `[rangeStart, - * rangeEnd]`: upcoming runs (`pending`, expanded from the recurrence) plus the - * schedule's most recent terminal run. `now` separates upcoming from past. A - * paused (`disabled`) recurring schedule still expands its upcoming occurrences - * — flagged `disabled` so the calendar can render them dimmed and offer Resume — - * since the cadence is intact and only suspended. `completed` schedules expand - * no future runs. - */ -export function scheduleToTasks( - row: WorkspaceScheduleRow, - rangeStart: Date, - rangeEnd: Date, - now: Date -): ScheduledTask[] { - const recurring = Boolean(row.cronExpression) - const paused = row.status === 'disabled' - // double-cast-allowed: contexts persist as open kind/label objects; the calendar consumes them as ChatContext - const contexts = (row.contexts ?? undefined) as unknown as ChatContext[] | undefined - const base = { - scheduleId: row.id, - sourceUserId: row.sourceUserId, - prompt: row.prompt ?? '', - contexts, - timezone: row.timezone, - recurring, - disabled: false, - } - const tasks: ScheduledTask[] = [] - - if (!recurring) { - if (row.status === 'active' && row.nextRunAt) { - const runAt = new Date(row.nextRunAt) - if (withinRange(runAt, rangeStart, rangeEnd)) { - tasks.push({ ...base, id: row.id, runAt, status: 'pending' }) - } - } else { - const marker = lastRunMarker(row) - if (marker && withinRange(marker.at, rangeStart, rangeEnd)) { - tasks.push({ ...base, id: row.id, runAt: marker.at, status: marker.status }) - } - } - return tasks - } - - if ((row.status === 'active' || paused) && row.cronExpression) { - const occurrences = expandOccurrences({ - cronExpression: row.cronExpression, - timezone: row.timezone, - rangeStart, - rangeEnd, - from: now, - excludedDates: row.excludedDates, - endsAt: row.endsAt ? new Date(row.endsAt) : null, - }) - for (const runAt of occurrences) { - tasks.push({ - ...base, - id: `${row.id}:${runAt.toISOString()}`, - runAt, - status: 'pending', - disabled: paused, - }) - } - } - - const marker = lastRunMarker(row) - if (marker && withinRange(marker.at, rangeStart, rangeEnd)) { - tasks.push({ ...base, id: `${row.id}:last`, runAt: marker.at, status: marker.status }) - } - - return tasks -} - -/** - * Adapts a task occurrence into a positioned calendar event, placing it at its - * wall-clock time in the task's own timezone (see {@link CalendarEvent.start}). - * Every occurrence renders identically regardless of status; the details modal - * carries the state. - */ -export function taskToCalendarEvent(task: ScheduledTask): CalendarEvent { - const prompt = task.prompt.trim() - return { - id: task.id, - start: zonedClockDate(task.runAt, task.timezone), - title: prompt ? truncate(prompt, 60) : 'Scheduled task', - task, - } -} - -/** Groups events by calendar day for both the month grid and the time grid. */ -export function bucketEventsByDay(events: CalendarEvent[]): Map { - const map = new Map() - for (const event of events) { - const key = dayKey(event.start) - const bucket = map.get(key) - if (bucket) bucket.push(event) - else map.set(key, [event]) - } - return map -} diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/index.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/index.ts index 8435a995fd6..d7ade05eb8f 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/index.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/index.ts @@ -1,7 +1,6 @@ export * from './auto-layout-utils' export * from './block-protection-utils' export * from './block-ring-utils' -export * from './node-derivation' export * from './node-position-utils' export * from './run-from-block' export * from './workflow-canvas-helpers' diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/node-derivation.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/node-derivation.ts deleted file mode 100644 index e361720baa0..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/node-derivation.ts +++ /dev/null @@ -1,21 +0,0 @@ -import type { BlockState } from '@/stores/workflows/workflow/types' - -export const Z_INDEX = { - ROOT_BLOCK: 10, - CHILD_BLOCK: 1000, -} as const - -export function computeContainerZIndex( - block: Pick, - allBlocks: Record> -): number { - let depth = 0 - let parentId = block.data?.parentId - - while (parentId && depth < 100) { - depth++ - parentId = allBlocks[parentId]?.data?.parentId - } - - return depth -} diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/index.ts b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/index.ts index e3d9983e27a..da1cfa02b03 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/index.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/index.ts @@ -1,4 +1,3 @@ -export { useAutoScroll } from './use-auto-scroll' export { useChatSelection } from './use-chat-selection' export { type DropIndicator, useDragDrop } from './use-drag-drop' export { useFlyoutInlineRename } from './use-flyout-inline-rename' diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-auto-scroll.ts b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-auto-scroll.ts deleted file mode 100644 index 77e91fa27d2..00000000000 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/hooks/use-auto-scroll.ts +++ /dev/null @@ -1,103 +0,0 @@ -import { useCallback, useRef } from 'react' - -/** - * Optimized auto-scroll hook for smooth drag operations - */ -export const useAutoScroll = (containerRef: React.RefObject) => { - const animationRef = useRef(null) - const speedRef = useRef(0) - const lastUpdateRef = useRef(0) - - const animateScroll = useCallback(() => { - const scrollContainer = containerRef.current?.querySelector( - '[data-radix-scroll-area-viewport]' - ) as HTMLElement - if (!scrollContainer || speedRef.current === 0) { - animationRef.current = null - return - } - - const currentScrollTop = scrollContainer.scrollTop - const maxScrollTop = scrollContainer.scrollHeight - scrollContainer.clientHeight - - // Check bounds and stop if needed - if ( - (speedRef.current < 0 && currentScrollTop <= 0) || - (speedRef.current > 0 && currentScrollTop >= maxScrollTop) - ) { - speedRef.current = 0 - animationRef.current = null - return - } - - // Apply smooth scroll - scrollContainer.scrollTop = Math.max( - 0, - Math.min(maxScrollTop, currentScrollTop + speedRef.current) - ) - animationRef.current = requestAnimationFrame(animateScroll) - }, [containerRef]) - - const startScroll = useCallback( - (speed: number) => { - speedRef.current = speed - if (!animationRef.current) { - animationRef.current = requestAnimationFrame(animateScroll) - } - }, - [animateScroll] - ) - - const stopScroll = useCallback(() => { - if (animationRef.current) { - cancelAnimationFrame(animationRef.current) - animationRef.current = null - } - speedRef.current = 0 - }, []) - - const handleDragOver = useCallback( - (e: DragEvent) => { - const now = performance.now() - // Throttle to ~16ms for 60fps - if (now - lastUpdateRef.current < 16) return - lastUpdateRef.current = now - - const scrollContainer = containerRef.current - if (!scrollContainer) return - - const rect = scrollContainer.getBoundingClientRect() - const mouseY = e.clientY - - // Early exit if mouse is outside container - if (mouseY < rect.top || mouseY > rect.bottom) { - stopScroll() - return - } - - const scrollZone = 50 - const maxSpeed = 4 - const distanceFromTop = mouseY - rect.top - const distanceFromBottom = rect.bottom - mouseY - - let scrollSpeed = 0 - - if (distanceFromTop < scrollZone) { - const intensity = (scrollZone - distanceFromTop) / scrollZone - scrollSpeed = -maxSpeed * intensity ** 2 - } else if (distanceFromBottom < scrollZone) { - const intensity = (scrollZone - distanceFromBottom) / scrollZone - scrollSpeed = maxSpeed * intensity ** 2 - } - - if (Math.abs(scrollSpeed) > 0.1) { - startScroll(scrollSpeed) - } else { - stopScroll() - } - }, - [containerRef, startScroll, stopScroll] - ) - - return { handleDragOver, stopScroll } -} diff --git a/apps/sim/hooks/queries/general-settings.ts b/apps/sim/hooks/queries/general-settings.ts index fb6e0c522b2..0db0992f049 100644 --- a/apps/sim/hooks/queries/general-settings.ts +++ b/apps/sim/hooks/queries/general-settings.ts @@ -54,14 +54,6 @@ export function useGeneralSettings() { return query } -/** - * Prefetch general settings into a QueryClient cache. - * Use on hover to warm data before navigation. - */ -export function prefetchGeneralSettings(queryClient: QueryClient) { - queryClient.prefetchQuery(generalSettingsQuery) -} - /** * Convenience selector hooks for individual settings. * These provide a simple API for components that only need a single setting value. diff --git a/apps/sim/hooks/queries/mothership-admin.ts b/apps/sim/hooks/queries/mothership-admin.ts index bdf47a186ff..70d9d2db64d 100644 --- a/apps/sim/hooks/queries/mothership-admin.ts +++ b/apps/sim/hooks/queries/mothership-admin.ts @@ -55,7 +55,6 @@ async function mothershipGet( export const MOTHERSHIP_REQUESTS_STALE_TIME = 60 * 1000 export const MOTHERSHIP_USER_BREAKDOWN_STALE_TIME = 60 * 1000 export const MOTHERSHIP_LICENSE_LIST_STALE_TIME = 60 * 1000 -export const MOTHERSHIP_LICENSE_DETAIL_STALE_TIME = 60 * 1000 export const mothershipKeys = { all: ['mothership-admin'] as const, diff --git a/apps/sim/hooks/queries/mothership-chats.test.ts b/apps/sim/hooks/queries/mothership-chats.test.ts index 1f7c9057c54..bd4d2444f1b 100644 --- a/apps/sim/hooks/queries/mothership-chats.test.ts +++ b/apps/sim/hooks/queries/mothership-chats.test.ts @@ -2,7 +2,6 @@ import { jsonResponse } from '@sim/testing/helpers/http' import { reactQueryMock, reactQueryMockFns } from '@sim/testing/mocks/react-query.mock' import { sleep } from '@sim/utils/helpers' import { beforeEach, describe, expect, it, vi } from 'vitest' -import type { MothershipResource } from '@/lib/mothership/resources/types' const { suspendBrowserScope, suspendTerminalScope, clearChat } = vi.hoisted(() => ({ clearChat: vi.fn(), @@ -24,7 +23,7 @@ vi.mock('@/lib/terminal/transport', () => ({ suspendTerminalScope, })) -import { useDeleteMothershipChats, useRemoveChatResource } from '@/hooks/queries/mothership-chats' +import { useDeleteMothershipChats } from '@/hooks/queries/mothership-chats' const queryClient = reactQueryMockFns.mockQueryClient @@ -95,52 +94,3 @@ describe('tasks query boundary parsing', () => { }) }) }) - -it('removes only the requested workspace alias and forwards its owner', async () => { - const first: MothershipResource = { - type: 'file', - id: 'files/report.csv', - title: 'A', - workspaceId: 'ws-a', - } - const second: MothershipResource = { ...first, title: 'B', workspaceId: 'ws-b' } - let cached = { - id: 'chat-1', - title: null, - messages: [], - activeStreamId: null, - resources: [first, second], - } - queryClient.setQueryData.mockImplementation((_key, update) => { - cached = update(cached) - }) - vi.stubGlobal( - 'fetch', - vi.fn().mockResolvedValue(jsonResponse({ success: true, resources: [first] })) - ) - const mutation = useRemoveChatResource('chat-1') as unknown as { - onMutate: (input: { - chatId: string - resourceType: 'file' - resourceId: string - workspaceId: string - }) => Promise - mutationFn: (input: { - chatId: string - resourceType: 'file' - resourceId: string - workspaceId: string - }) => Promise - } - const input = { - chatId: 'chat-1', - resourceType: 'file' as const, - resourceId: 'files/report.csv', - workspaceId: 'ws-b', - } - await mutation.onMutate(input) - expect(cached.resources).toEqual([first]) - await mutation.mutationFn(input) - expect(JSON.parse(vi.mocked(fetch).mock.calls[0][1]?.body as string)).toEqual(input) - vi.unstubAllGlobals() -}) diff --git a/apps/sim/hooks/queries/mothership-chats.ts b/apps/sim/hooks/queries/mothership-chats.ts index b4eb830978f..48ea0783916 100644 --- a/apps/sim/hooks/queries/mothership-chats.ts +++ b/apps/sim/hooks/queries/mothership-chats.ts @@ -11,7 +11,6 @@ import { import { isApiClientError } from '@/lib/api/client/errors' import { requestJson } from '@/lib/api/client/request' import { - addMothershipChatResourceContract, deleteMothershipChatContract, forkMothershipChatContract, getMothershipChatContract, @@ -20,8 +19,6 @@ import { type MothershipChat, type MothershipChatScope, markMothershipChatReadContract, - removeMothershipChatResourceContract, - reorderMothershipChatResourcesContract, restoreMothershipChatContract, updateMothershipChatContract, } from '@/lib/api/contracts/mothership-chats' @@ -34,7 +31,7 @@ import { isFilePreviewSession, } from '@/lib/mothership/request/session/file-preview-session-contract' import { isStreamBatchEvent, type StreamBatchEvent } from '@/lib/mothership/request/session/types' -import { getChatResourceKey, type MothershipResource } from '@/lib/mothership/resources/types' +import type { MothershipResource } from '@/lib/mothership/resources/types' import { useMothershipQueueStore } from '@/stores/mothership-queue/store' export interface MothershipChatMetadata { @@ -207,14 +204,6 @@ function parseChatHistory(value: unknown): MothershipChatHistory { } } -function parseChatResourcesResponse(value: unknown): { resources: MothershipResource[] } { - assertValid(isRecordLike(value), 'Invalid chat resources response: body must be an object') - - return { - resources: parseResources(value.resources, 'Invalid chat resources response: resources'), - } -} - export function mapChat(chat: MothershipChat): MothershipChatMetadata { const updatedAt = new Date(chat.updatedAt) return { @@ -440,142 +429,6 @@ export function useRenameMothershipChat(owner?: MothershipChatOwner) { }) } -async function addChatResource(params: { - chatId: string - resource: MothershipResource -}): Promise<{ resources: MothershipResource[] }> { - const data = await requestJson(addMothershipChatResourceContract, { - body: { chatId: params.chatId, resource: params.resource }, - }) - return parseChatResourcesResponse(data) -} - -export function useAddChatResource(chatId?: string) { - const queryClient = useQueryClient() - return useMutation({ - mutationFn: addChatResource, - onMutate: async ({ resource }) => { - if (!chatId) return - await queryClient.cancelQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - const previous = queryClient.getQueryData( - mothershipChatKeys.detail(chatId) - ) - if (previous) { - const exists = previous.resources.some( - (r) => getChatResourceKey(r) === getChatResourceKey(resource) - ) - if (!exists) { - queryClient.setQueryData(mothershipChatKeys.detail(chatId), { - ...previous, - resources: [...previous.resources, resource], - }) - } - } - return { previous } - }, - onError: (_err, _variables, context) => { - if (context?.previous && chatId) { - queryClient.setQueryData(mothershipChatKeys.detail(chatId), context.previous) - } - }, - onSettled: () => { - if (chatId) { - queryClient.invalidateQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - } - }, - }) -} - -async function reorderChatResources(params: { - chatId: string - resources: MothershipResource[] -}): Promise<{ resources: MothershipResource[] }> { - const data = await requestJson(reorderMothershipChatResourcesContract, { - body: { chatId: params.chatId, resources: params.resources }, - }) - return parseChatResourcesResponse(data) -} - -export function useReorderChatResources(chatId?: string) { - const queryClient = useQueryClient() - return useMutation({ - mutationFn: reorderChatResources, - onMutate: async ({ resources }) => { - if (!chatId) return - await queryClient.cancelQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - const previous = queryClient.getQueryData( - mothershipChatKeys.detail(chatId) - ) - if (previous) { - queryClient.setQueryData(mothershipChatKeys.detail(chatId), { - ...previous, - resources, - }) - } - return { previous } - }, - onError: (_err, _variables, context) => { - if (context?.previous && chatId) { - queryClient.setQueryData(mothershipChatKeys.detail(chatId), context.previous) - } - }, - onSettled: () => { - if (chatId) { - queryClient.invalidateQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - } - }, - }) -} - -async function removeChatResource(params: { - chatId: string - resourceType: MothershipResource['type'] - resourceId: string - workspaceId?: string -}): Promise<{ resources: MothershipResource[] }> { - const data = await requestJson(removeMothershipChatResourceContract, { - body: params, - }) - return parseChatResourcesResponse(data) -} - -export function useRemoveChatResource(chatId?: string) { - const queryClient = useQueryClient() - return useMutation({ - mutationFn: removeChatResource, - onMutate: async ({ resourceType, resourceId, workspaceId }) => { - if (!chatId) return - await queryClient.cancelQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - const removed: MothershipChatHistory['resources'] = [] - queryClient.setQueryData(mothershipChatKeys.detail(chatId), (prev) => { - if (!prev) return prev - const next: MothershipChatHistory['resources'] = [] - for (const r of prev.resources) { - if ( - getChatResourceKey(r) === - getChatResourceKey({ type: resourceType, id: resourceId, workspaceId }) - ) - removed.push(r) - else next.push(r) - } - return removed.length > 0 ? { ...prev, resources: next } : prev - }) - return { removed } - }, - onError: (_err, _variables, context) => { - if (!chatId || !context?.removed.length) return - queryClient.setQueryData(mothershipChatKeys.detail(chatId), (prev) => - prev ? { ...prev, resources: [...prev.resources, ...context.removed] } : prev - ) - }, - onSettled: () => { - if (chatId) { - queryClient.invalidateQueries({ queryKey: mothershipChatKeys.detail(chatId) }) - } - }, - }) -} - async function markChatRead(chatId: string): Promise { await requestJson(markMothershipChatReadContract, { body: { chatId }, diff --git a/apps/sim/hooks/queries/oauth/oauth-connections.ts b/apps/sim/hooks/queries/oauth/oauth-connections.ts index 87dd6cb86c3..92621e907bf 100644 --- a/apps/sim/hooks/queries/oauth/oauth-connections.ts +++ b/apps/sim/hooks/queries/oauth/oauth-connections.ts @@ -16,7 +16,6 @@ import { getPerRequestOAuthLinkScopes } from '@/lib/oauth/utils' const logger = createLogger('OAuthConnectionsQuery') export const OAUTH_CONNECTIONS_STALE_TIME = 30 * 1000 -export const OAUTH_CONNECTED_ACCOUNTS_STALE_TIME = 60 * 1000 /** * Query key factory for OAuth connection queries. diff --git a/apps/sim/hooks/queries/organization.test.tsx b/apps/sim/hooks/queries/organization.test.tsx index 912187da31a..7439625a497 100644 --- a/apps/sim/hooks/queries/organization.test.tsx +++ b/apps/sim/hooks/queries/organization.test.tsx @@ -35,16 +35,12 @@ import { organizationKeys, useOrganization, useOrganizationBilling, - useOrganizationList, useOrganizationRoster, } from '@/hooks/queries/organization' import { shouldRetryOrganizationBillingSummary } from '@/hooks/queries/organization-billing-summary' -const { - getFullOrganization: mockGetFullOrganization, - list: mockListOrganizations, - setActive: mockSetActiveOrganization, -} = authClientMockFns.mockClient.organization +const { getFullOrganization: mockGetFullOrganization, setActive: mockSetActiveOrganization } = + authClientMockFns.mockClient.organization const mockRequestJson = apiClientRequestMockFns.mockRequestJson @@ -101,11 +97,6 @@ function OrganizationProbe({ organizationId }: { organizationId: string }) { ) } -function MembershipProbe() { - const query = useOrganizationList() - return
{query.error?.message ?? query.data?.map(({ name }) => name).join(', ')}
-} - function renderOrganization(organizationId: string) { act(() => { root.render( @@ -158,23 +149,6 @@ describe('organization identity transitions', () => { expect(container.textContent).not.toContain('Manage organization') }) - it('surfaces membership-list errors for retry instead of returning an empty list', async () => { - mockListOrganizations.mockResolvedValue({ - data: null, - error: { message: 'Membership service unavailable' }, - }) - await act(async () => - root.render( - - - - ) - ) - await flushQueries() - expect(container.textContent).toBe('Membership service unavailable') - expect(queryClient.getQueryState(organizationKeys.lists())?.status).toBe('error') - }) - it('clears organization detail, roster, billing, and actions while the next org loads', async () => { const organizationB = createDeferred<{ data: typeof ORGANIZATION_A }>() const rosterB = createDeferred() diff --git a/apps/sim/hooks/queries/organization.ts b/apps/sim/hooks/queries/organization.ts index 74e10d73b34..849e493abc0 100644 --- a/apps/sim/hooks/queries/organization.ts +++ b/apps/sim/hooks/queries/organization.ts @@ -39,7 +39,6 @@ import { type OrganizationBillingApiResponse, } from '@/lib/api/contracts/subscription' import { client } from '@/lib/auth/auth-client' -import { isOrganizationsEnabled } from '@/lib/core/config/env-flags' import { workspaceCredentialKeys } from '@/hooks/queries/utils/credential-keys' import { organizationKeys } from '@/hooks/queries/utils/organization-keys' import { organizationUsageKeys } from '@/hooks/queries/utils/organization-usage-keys' @@ -50,11 +49,8 @@ const logger = createLogger('OrganizationQueries') const invitationListsKey = ['invitations', 'list'] as const export const ORGANIZATION_ROSTER_STALE_TIME = 30 * 1000 -export const ORGANIZATION_LIST_STALE_TIME = 30 * 1000 export const ORGANIZATION_DETAIL_STALE_TIME = 30 * 1000 -export const ORGANIZATION_SUBSCRIPTION_STALE_TIME = 30 * 1000 export const ORGANIZATION_BILLING_STALE_TIME = 30 * 1000 -export const ORGANIZATION_MEMBERS_STALE_TIME = 30 * 1000 export const ORGANIZATION_MEMBER_USAGE_LIMIT_STALE_TIME = 30 * 1000 /** * Zero: removal impact is a consent disclosure, so every dialog open must @@ -78,22 +74,6 @@ export { organizationKeys } export type { OrganizationRoster, RosterMember, RosterPendingInvitation, RosterWorkspaceAccess } -/** Better Auth owns the authenticated membership-list endpoint. */ -export function useOrganizationList() { - return useQuery({ - queryKey: organizationKeys.lists(), - queryFn: async ({ signal }) => { - const response = await client.organization.list({ fetchOptions: { signal } }) - if (response.error) { - throw new Error(response.error.message || 'Failed to load organizations') - } - return response.data ?? [] - }, - enabled: isOrganizationsEnabled, - staleTime: ORGANIZATION_LIST_STALE_TIME, - }) -} - async function fetchOrganizationRoster( orgId: string, signal?: AbortSignal diff --git a/apps/sim/hooks/queries/schedules.ts b/apps/sim/hooks/queries/schedules.ts index c2d563daaad..8a1db7dd80a 100644 --- a/apps/sim/hooks/queries/schedules.ts +++ b/apps/sim/hooks/queries/schedules.ts @@ -16,7 +16,6 @@ import { deploymentKeys } from '@/hooks/queries/deployments' const logger = createLogger('ScheduleQueries') export const SCHEDULE_LIST_STALE_TIME = 30 * 1000 -export const SCHEDULE_DETAIL_STALE_TIME = 30 * 1000 export const SCHEDULE_BLOCK_STALE_TIME = 30 * 1000 export const scheduleKeys = { diff --git a/apps/sim/hooks/queries/subscription.ts b/apps/sim/hooks/queries/subscription.ts index 84c1ee0c599..e5067c542fb 100644 --- a/apps/sim/hooks/queries/subscription.ts +++ b/apps/sim/hooks/queries/subscription.ts @@ -73,27 +73,6 @@ async function fetchUsageLimitData(signal?: AbortSignal) { }) } -interface UseUsageLimitDataOptions { - /** Whether to enable the query (defaults to true) */ - enabled?: boolean -} - -/** - * Hook to fetch usage limit metadata - * Returns: currentLimit, minimumLimit, canEdit, plan, updatedAt - * Use this for editing usage limits, not for displaying current usage - */ -export function useUsageLimitData(options: UseUsageLimitDataOptions = {}) { - const { enabled = true } = options - - return useQuery({ - queryKey: subscriptionKeys.usage(), - queryFn: ({ signal }) => fetchUsageLimitData(signal), - staleTime: USAGE_LIMIT_STALE_TIME, - enabled, - }) -} - /** * Fetch finalized invoices for the active billing customer (personal or * organization-scoped). diff --git a/apps/sim/hooks/queries/tables.ts b/apps/sim/hooks/queries/tables.ts index 25a9fcb56ad..052673bc1c7 100644 --- a/apps/sim/hooks/queries/tables.ts +++ b/apps/sim/hooks/queries/tables.ts @@ -320,18 +320,6 @@ export function useTable(workspaceId: string | undefined, tableId: string | unde }) } -/** - * Shared table-detail query options so non-component callers (e.g. selector - * providers) can `ensureQueryData` the same cache entry `useTable` populates. - */ -export function getTableDetailQueryOptions(workspaceId: string, tableId: string) { - return { - queryKey: tableKeys.detail(tableId), - queryFn: ({ signal }: { signal?: AbortSignal }) => fetchTable(workspaceId, tableId, signal), - staleTime: TABLE_DETAIL_STALE_TIME, - } -} - export interface TableRunState { dispatches: ActiveDispatch[] runningByRowId: Record diff --git a/apps/sim/hooks/queries/workspace-files.ts b/apps/sim/hooks/queries/workspace-files.ts index db8659491ab..5714ca235d1 100644 --- a/apps/sim/hooks/queries/workspace-files.ts +++ b/apps/sim/hooks/queries/workspace-files.ts @@ -68,7 +68,6 @@ export const workspaceFilesKeys = { export const WORKSPACE_FILES_LIST_STALE_TIME = 30 * 1000 export const WORKSPACE_FILE_CONTENT_STALE_TIME = 30 * 1000 export const WORKSPACE_FILE_BINARY_STALE_TIME = 30 * 1000 -export const WORKSPACE_STORAGE_INFO_STALE_TIME = 60 * 1000 /** Cloud storage (S3/Blob) is env-driven and does not change at runtime. */ export const CLOUD_STORAGE_CONFIGURED_STALE_TIME = Number.POSITIVE_INFINITY diff --git a/apps/sim/hooks/use-scroll-anchor.test.ts b/apps/sim/hooks/use-scroll-anchor.test.ts deleted file mode 100644 index fde41688903..00000000000 --- a/apps/sim/hooks/use-scroll-anchor.test.ts +++ /dev/null @@ -1,41 +0,0 @@ -/** - * Tests for the pure functions extracted from `useScrollAnchor`: - * `computeSpacerShortage` and `shouldReengage`. The hook's DOM-interaction - * behaviour (event listeners, MutationObserver, and the forced-reflow / - * scroll-event race condition fix) requires a real browser layout engine - * and is covered by manual QA. - */ -import { describe, expect, it } from 'vitest' -import { computeSpacerShortage, shouldReengage } from '@/hooks/use-scroll-anchor' - -describe('computeSpacerShortage', () => { - it('subtracts existing spacer height before recomputing shortage', () => { - // spacer was 900 from last update; content grew to 1000 natural height - // scrollHeight = 1000 + 900 = 1900; needed = 500 + 600 = 1100 - // naturalScrollHeight = 1900 - 900 = 1000; shortage = 1100 - 1000 = 100 - expect(computeSpacerShortage(500, 600, 1900, 900)).toBe(100) - }) - - it('never returns a negative value', () => { - expect(computeSpacerShortage(0, 600, 10000, 0)).toBe(0) - expect(computeSpacerShortage(0, 600, 0, 0)).toBe(600) - }) -}) - -describe('shouldReengage', () => { - it('returns false when the spacer is active, even at distanceFromBottom = 0', () => { - // The spacer inflates scrollHeight to exactly targetScrollTop + clientHeight, - // so programmatic scroll restoration always produces distanceFromBottom = 0. - // Without this guard, onScroll would falsely re-engage auto-follow, clear the - // spacer on the next content update, and jump the user to the top. - expect(shouldReengage(0, 1000)).toBe(false) - }) - - it('returns true when the user genuinely reaches the document bottom (no spacer)', () => { - expect(shouldReengage(0, 0)).toBe(true) - }) - - it('returns false when exactly at threshold + 1', () => { - expect(shouldReengage(31, 0)).toBe(false) - }) -}) diff --git a/apps/sim/hooks/use-scroll-anchor.ts b/apps/sim/hooks/use-scroll-anchor.ts deleted file mode 100644 index ec875eb3589..00000000000 --- a/apps/sim/hooks/use-scroll-anchor.ts +++ /dev/null @@ -1,172 +0,0 @@ -import { useCallback, useLayoutEffect, useRef } from 'react' - -const NEAR_BOTTOM_THRESHOLD = 30 - -/** - * Returns the `minHeight` the spacer needs so `scrollTop` can safely reach - * `targetScrollTop` when replace-mode streaming produces temporarily shorter content. - */ -export function computeSpacerShortage( - targetScrollTop: number, - clientHeight: number, - scrollHeight: number, - prevSpacerHeight: number -): number { - const needed = targetScrollTop + clientHeight - const naturalScrollHeight = scrollHeight - prevSpacerHeight - return Math.max(0, needed - naturalScrollHeight) -} - -/** - * Returns whether the scroll container should re-engage auto-follow. - * - * Re-engagement is blocked while the spacer is active. The spacer inflates - * `scrollHeight` to exactly `targetScrollTop + clientHeight`, so programmatic - * scroll restoration produces `distanceFromBottom = 0` — which is artificial - * proximity, not the user genuinely reaching the document bottom. - */ -export function shouldReengage(distanceFromBottom: number, spacerHeight: number): boolean { - return distanceFromBottom <= NEAR_BOTTOM_THRESHOLD && spacerHeight === 0 -} - -/** - * Manages scroll for a streaming file-preview container. - * - * Never-scrolled: auto-follows new content to the bottom (MutationObserver - * keeps it pinned). Scrolled-up: position is locked via a spacer element that - * inflates `scrollHeight` to prevent the browser from clamping `scrollTop` when - * replace-mode streaming temporarily produces a shorter chunk. Scrolled back to - * the bottom: auto-follow re-engages. - * - * @param isStreaming - whether the container is currently receiving streaming content - * @param content - drives spacer recalculation; pass the current text value - */ -export function useScrollAnchor(isStreaming: boolean, content?: string) { - const containerRef = useRef(null) - const spacerRef = useRef(null) - const hasUserScrolledRef = useRef(false) - const stickyRef = useRef(false) - const intendedScrollTopRef = useRef(0) - // Avoids a layout read inside onScroll. - const spacerHeightRef = useRef(0) - - const scrollToBottom = useCallback(() => { - const el = containerRef.current - if (!el) return - el.scrollTop = el.scrollHeight - }, []) - - const onWheel = useCallback((e: WheelEvent) => { - if (e.deltaY >= 0 || hasUserScrolledRef.current) return - hasUserScrolledRef.current = true - stickyRef.current = false - const el = containerRef.current - if (el) intendedScrollTopRef.current = el.scrollTop - }, []) - - const onScroll = useCallback(() => { - const el = containerRef.current - if (!el) return - - if (hasUserScrolledRef.current) { - intendedScrollTopRef.current = el.scrollTop - const distanceFromBottom = el.scrollHeight - el.scrollTop - el.clientHeight - if (shouldReengage(distanceFromBottom, spacerHeightRef.current)) { - hasUserScrolledRef.current = false - stickyRef.current = true - } - return - } - - const distanceFromBottom = el.scrollHeight - el.scrollTop - el.clientHeight - if (distanceFromBottom > NEAR_BOTTOM_THRESHOLD) { - hasUserScrolledRef.current = true - stickyRef.current = false - intendedScrollTopRef.current = el.scrollTop - } else { - stickyRef.current = true - } - }, []) - - const callbackRef = useCallback( - (el: HTMLDivElement | null) => { - const prev = containerRef.current - if (prev) { - prev.removeEventListener('scroll', onScroll) - prev.removeEventListener('wheel', onWheel as EventListener) - } - containerRef.current = el - if (el) { - el.addEventListener('scroll', onScroll, { passive: true }) - el.addEventListener('wheel', onWheel as EventListener, { passive: true }) - } - }, - [onScroll, onWheel] - ) - - useLayoutEffect(() => { - if (!isStreaming) return - const el = containerRef.current - if (!el) return - if (hasUserScrolledRef.current) return - const distanceFromBottom = el.scrollHeight - el.scrollTop - el.clientHeight - stickyRef.current = distanceFromBottom <= NEAR_BOTTOM_THRESHOLD - if (stickyRef.current) scrollToBottom() - }, [isStreaming, scrollToBottom]) - - useLayoutEffect(() => { - if (!isStreaming) return - const el = containerRef.current - if (!el) return - - let rafId = 0 - const guardedScroll = () => { - if (stickyRef.current) scrollToBottom() - } - const onMutation = () => { - if (!stickyRef.current) return - cancelAnimationFrame(rafId) - rafId = requestAnimationFrame(guardedScroll) - } - - const observer = new MutationObserver(onMutation) - observer.observe(el, { childList: true, subtree: true, characterData: true }) - - return () => { - observer.disconnect() - cancelAnimationFrame(rafId) - } - }, [isStreaming, scrollToBottom]) - - useLayoutEffect(() => { - const el = containerRef.current - const spacer = spacerRef.current - if (!el) return - - if (!hasUserScrolledRef.current || !isStreaming) { - spacerHeightRef.current = 0 - if (spacer) spacer.style.minHeight = '0' - return - } - - // Must read before scrollHeight: that forced reflow can synchronously fire 'scroll' and clamp the value. - const targetScrollTop = intendedScrollTopRef.current - - const prevSpacerHeight = spacer ? spacer.offsetHeight : 0 - const shortage = computeSpacerShortage( - targetScrollTop, - el.clientHeight, - el.scrollHeight, - prevSpacerHeight - ) - - spacerHeightRef.current = shortage - if (spacer) spacer.style.minHeight = `${shortage}px` - if (el.scrollTop < targetScrollTop) el.scrollTop = targetScrollTop - }, [content, isStreaming]) - - return { - ref: callbackRef, - spacerRef, - } -} From 23eff1b3784ba6a24e0211074eea4c2499909b69 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:00:20 -0700 Subject: [PATCH 03/30] refactor(queries): concentrate workflow search MCP list fetches into two helpers --- .../hooks/queries/workflow-search-replace.ts | 66 +++++++------------ 1 file changed, 22 insertions(+), 44 deletions(-) diff --git a/apps/sim/hooks/queries/workflow-search-replace.ts b/apps/sim/hooks/queries/workflow-search-replace.ts index 4f837b3117d..a559778a0ff 100644 --- a/apps/sim/hooks/queries/workflow-search-replace.ts +++ b/apps/sim/hooks/queries/workflow-search-replace.ts @@ -393,6 +393,24 @@ export function useWorkflowSearchFileDetails(matches: WorkflowSearchMatch[], wor ) } +/** Shared workspace MCP servers plus the managed catalog, as one list. */ +async function fetchWorkspaceMcpServers(workspaceId: string, signal: AbortSignal) { + const [shared, managed] = await Promise.all([ + requestJson(listMcpServersContract, { query: { workspaceId }, signal }), + requestJson(listManagedMcpCatalogContract, { query: { workspaceId }, signal }), + ]) + return [...shared.data.servers, ...managed.servers] +} + +/** Discovered workspace MCP tools plus the managed catalog tools, as one list. */ +async function fetchWorkspaceMcpTools(workspaceId: string, signal: AbortSignal) { + const [shared, managed] = await Promise.all([ + requestJson(discoverMcpToolsContract, { query: { workspaceId }, signal }), + requestJson(listManagedMcpCatalogContract, { query: { workspaceId }, signal }), + ]) + return [...shared.data.tools, ...managed.tools] +} + export function useWorkflowSearchMcpServerDetails( matches: WorkflowSearchMatch[], workspaceId?: string @@ -402,17 +420,7 @@ export function useWorkflowSearchMcpServerDetails( const serversQuery = useQuery({ queryKey: workflowSearchReplaceKeys.mcpServerListDetails(workspaceId), queryFn: async ({ signal }: { signal: AbortSignal }) => { - const [shared, managed] = await Promise.all([ - requestJson(listMcpServersContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - requestJson(listManagedMcpCatalogContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - ]) - return [...shared.data.servers, ...managed.servers] + return fetchWorkspaceMcpServers(workspaceId as string, signal) }, enabled: Boolean(workspaceId && serverMatches.length > 0), staleTime: WORKFLOW_SEARCH_MCP_SERVER_LIST_STALE_TIME, @@ -447,17 +455,7 @@ export function useWorkflowSearchMcpToolDetails( const toolsQuery = useQuery({ queryKey: workflowSearchReplaceKeys.mcpToolListDetails(workspaceId), queryFn: async ({ signal }: { signal: AbortSignal }) => { - const [shared, managed] = await Promise.all([ - requestJson(discoverMcpToolsContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - requestJson(listManagedMcpCatalogContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - ]) - return [...shared.data.tools, ...managed.tools] + return fetchWorkspaceMcpTools(workspaceId as string, signal) }, enabled: Boolean(workspaceId && toolMatches.length > 0), staleTime: WORKFLOW_SEARCH_MCP_TOOL_LIST_STALE_TIME, @@ -728,17 +726,7 @@ export function useWorkflowSearchMcpServerReplacementOptions( }: { signal: AbortSignal }): Promise => { - const [shared, managed] = await Promise.all([ - requestJson(listMcpServersContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - requestJson(listManagedMcpCatalogContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - ]) - return [...shared.data.servers, ...managed.servers] + return fetchWorkspaceMcpServers(workspaceId as string, signal) }, enabled: Boolean(workspaceId && serverGroups.length > 0), staleTime: WORKFLOW_SEARCH_MCP_SERVER_REPLACEMENT_STALE_TIME, @@ -790,17 +778,7 @@ export function useWorkflowSearchMcpToolReplacementOptions( }: { signal: AbortSignal }): Promise => { - const [shared, managed] = await Promise.all([ - requestJson(discoverMcpToolsContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - requestJson(listManagedMcpCatalogContract, { - query: { workspaceId: workspaceId as string }, - signal, - }), - ]) - return [...shared.data.tools, ...managed.tools] + return fetchWorkspaceMcpTools(workspaceId as string, signal) }, enabled: Boolean(workspaceId && toolGroups.length > 0), staleTime: WORKFLOW_SEARCH_MCP_TOOL_REPLACEMENT_STALE_TIME, From 99a01a8edec8a3b94a295ce17188169d58a04230 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:06:48 -0700 Subject: [PATCH 04/30] chore(cleanup): dedupe app copies of shared package helpers - import subflow helpers from @sim/workflow-persistence/subflow-helpers - drop env-capabilities and service-account-metadata re-export shims - use isWorkflowBlockProtected in block-enablement - move anonymous user identity into @sim/auth/principal - reuse isLightTileColor from @sim/workflow-renderer/tile-icon-color - use @sim/utils sleep/getErrorMessage in sim-cli (bundled) - remove dead normalizeSandboxProvider export --- apps/realtime/src/auth.ts | 10 -- apps/realtime/src/middleware/auth.ts | 3 +- .../auth/oauth2/callback/instagram/route.ts | 2 +- .../api/auth/oauth2/callback/shopify/route.ts | 2 +- apps/sim/app/api/v1/auth.ts | 2 +- .../settings/components/general/general.tsx | 2 +- .../sidebar-footer/sidebar-footer.tsx | 2 +- apps/sim/blocks/icon-color.ts | 22 +-- .../lib/api/server/routes/v2-api-key-auth.ts | 2 +- apps/sim/lib/auth/anonymous.ts | 2 +- apps/sim/lib/auth/constants.ts | 11 -- apps/sim/lib/core/config/api-keys.ts | 2 +- .../core/config/env-capabilities.server.ts | 4 +- .../lib/core/config/env-capabilities.test.ts | 8 +- apps/sim/lib/core/config/env-capabilities.ts | 2 - apps/sim/lib/core/config/env-flags.ts | 6 +- apps/sim/lib/embeddings/client.ts | 10 +- .../knowledge-embedding-family.test.ts | 4 +- .../execution/remote-sandbox/pi-lifetime.ts | 2 +- .../integrations/availability.server.test.ts | 16 +- .../lib/integrations/availability.server.ts | 8 +- .../integrations/service-account-metadata.ts | 2 - .../knowledge/documents/document-processor.ts | 2 +- .../lib/messaging/email/providers/index.ts | 2 +- .../lib/mothership/chat/process-contents.ts | 2 +- apps/sim/lib/oauth/oauth.ts | 10 +- .../apply-workflow-operations.test.ts | 2 +- .../application/apply-workflow-operations.ts | 5 +- .../application/update-workflow-content.ts | 5 +- .../lib/workflows/diff/diff-engine.test.ts | 2 +- apps/sim/lib/workflows/diff/diff-engine.ts | 2 +- .../lib/workflows/editing/block-enablement.ts | 22 +-- apps/sim/lib/workflows/editing/engine.ts | 5 +- .../workflows/persistence/prepare-state.ts | 5 +- .../lib/workflows/persistence/utils.test.ts | 2 +- .../workflows/sanitization/json-sanitizer.ts | 5 +- apps/sim/serializer/index.ts | 5 +- apps/sim/stores/workflows/workflow/store.ts | 13 +- .../stores/workflows/workflow/utils.test.ts | 7 +- apps/sim/stores/workflows/workflow/utils.ts | 140 +----------------- .../stores/workflows/workflow/validation.ts | 5 +- packages/auth/src/principal.ts | 12 ++ packages/sim-cli/src/auth/device-flow.ts | 2 +- .../src/commands/protocol/files-get.test.ts | 2 +- .../src/commands/protocol/logs-follow.test.ts | 2 +- .../src/commands/protocol/logs-follow.ts | 2 +- .../protocol/workflow-run-wait.test.ts | 2 +- .../commands/protocol/workflow-run-wait.ts | 2 +- .../protocol/workspace-operation-wait.ts | 2 +- packages/sim-cli/src/config/profile.test.ts | 2 +- packages/sim-cli/src/embed.test.ts | 2 +- packages/sim-cli/src/embed.ts | 6 +- packages/sim-cli/src/helpers.ts | 12 -- packages/sim-cli/src/http/client.test.ts | 2 +- packages/utils/src/sandbox-references.ts | 9 -- .../src/lib/tile-icon-color.ts | 14 +- scripts/check-utils-enforcement.ts | 1 - 57 files changed, 128 insertions(+), 311 deletions(-) delete mode 100644 apps/sim/lib/core/config/env-capabilities.ts delete mode 100644 apps/sim/lib/integrations/service-account-metadata.ts delete mode 100644 packages/sim-cli/src/helpers.ts diff --git a/apps/realtime/src/auth.ts b/apps/realtime/src/auth.ts index 40491a62af7..3fa013917d8 100644 --- a/apps/realtime/src/auth.ts +++ b/apps/realtime/src/auth.ts @@ -1,16 +1,6 @@ import { createVerifyAuth } from '@sim/auth/verify' import { env } from '@/env' -export const ANONYMOUS_USER_ID = '00000000-0000-0000-0000-000000000000' - -export const ANONYMOUS_USER = { - id: ANONYMOUS_USER_ID, - name: 'Anonymous', - email: 'anonymous@localhost', - emailVerified: true, - image: null, -} as const - export const auth = createVerifyAuth({ secret: env.BETTER_AUTH_SECRET, baseURL: env.BETTER_AUTH_URL, diff --git a/apps/realtime/src/middleware/auth.ts b/apps/realtime/src/middleware/auth.ts index 6ea22f02d91..77919e9a70d 100644 --- a/apps/realtime/src/middleware/auth.ts +++ b/apps/realtime/src/middleware/auth.ts @@ -1,7 +1,8 @@ +import { ANONYMOUS_USER, ANONYMOUS_USER_ID } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import type { Socket } from 'socket.io' -import { ANONYMOUS_USER, ANONYMOUS_USER_ID, auth } from '@/auth' +import { auth } from '@/auth' import { isAuthDisabled } from '@/env' const logger = createLogger('SocketAuth') diff --git a/apps/sim/app/api/auth/oauth2/callback/instagram/route.ts b/apps/sim/app/api/auth/oauth2/callback/instagram/route.ts index afca4590c1f..62603299bbc 100644 --- a/apps/sim/app/api/auth/oauth2/callback/instagram/route.ts +++ b/apps/sim/app/api/auth/oauth2/callback/instagram/route.ts @@ -1,5 +1,6 @@ import { db } from '@sim/db' import { account } from '@sim/db/schema' +import { EnvCapabilityConfigurationError } from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' import { and, eq } from 'drizzle-orm' @@ -7,7 +8,6 @@ import { type NextRequest, NextResponse } from 'next/server' import { instagramCallbackContract } from '@/lib/api/contracts/oauth-connections' import { parseRequest } from '@/lib/api/server' import { getSession } from '@/lib/auth' -import { EnvCapabilityConfigurationError } from '@/lib/core/config/env-capabilities' import { requireConfiguredOAuthClient } from '@/lib/core/config/env-capabilities.server' import { DEFAULT_MAX_ERROR_BODY_BYTES, diff --git a/apps/sim/app/api/auth/oauth2/callback/shopify/route.ts b/apps/sim/app/api/auth/oauth2/callback/shopify/route.ts index 7df3318106e..9eb23807984 100644 --- a/apps/sim/app/api/auth/oauth2/callback/shopify/route.ts +++ b/apps/sim/app/api/auth/oauth2/callback/shopify/route.ts @@ -1,3 +1,4 @@ +import { EnvCapabilityConfigurationError } from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { hmacSha256Hex } from '@sim/security/hmac' @@ -7,7 +8,6 @@ import { shopifyShopDomainSchema, } from '@/lib/api/contracts/oauth-connections' import { getSession } from '@/lib/auth' -import { EnvCapabilityConfigurationError } from '@/lib/core/config/env-capabilities' import { requireConfiguredOAuthClient } from '@/lib/core/config/env-capabilities.server' import { getBaseUrl } from '@/lib/core/utils/urls' import { isSameOrigin } from '@/lib/core/utils/validation' diff --git a/apps/sim/app/api/v1/auth.ts b/apps/sim/app/api/v1/auth.ts index 78c68e1f9dd..c75f00cd1ec 100644 --- a/apps/sim/app/api/v1/auth.ts +++ b/apps/sim/app/api/v1/auth.ts @@ -1,8 +1,8 @@ import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal } from '@sim/auth/principal' +import { ANONYMOUS_USER_ID } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { authenticateApiKeyFromHeader, updateApiKeyLastUsed } from '@/lib/api-key/service' -import { ANONYMOUS_USER_ID } from '@/lib/auth/constants' import { isAuthDisabled } from '@/lib/core/config/env-flags' const logger = createLogger('V1Auth') diff --git a/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx b/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx index 5d1f48293c0..708540bd25d 100644 --- a/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx +++ b/apps/sim/app/workspace/[workspaceId]/settings/components/general/general.tsx @@ -1,6 +1,7 @@ 'use client' import { useEffect, useRef, useState } from 'react' +import { ANONYMOUS_USER_ID } from '@sim/auth/principal' import { Button, Chip, @@ -24,7 +25,6 @@ import Image from 'next/image' import { useRouter } from 'next/navigation' import { useQueryState } from 'nuqs' import { useSession } from '@/lib/auth/auth-client' -import { ANONYMOUS_USER_ID } from '@/lib/auth/constants' import { signOutAndRedirect } from '@/lib/auth/sign-out' import { useDeploymentShape } from '@/lib/core/config/deployment-shape' import { getBrowserTimezone, getTimezoneOptions } from '@/lib/core/utils/timezone' diff --git a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx index 6fd710207d1..68054db6e28 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/sidebar-footer/sidebar-footer.tsx @@ -1,6 +1,7 @@ 'use client' import type { ComponentType } from 'react' +import { ANONYMOUS_USER_ID } from '@sim/auth/principal' import type { DesktopUpdateState } from '@sim/desktop-bridge' import { Chip, @@ -21,7 +22,6 @@ import { BookOpen, Download, HelpCircle, LogOut, Settings } from '@sim/emcn/icon import { useRouter } from 'next/navigation' import { SlackIcon } from '@/components/icons' import { SettingsIntentLink } from '@/components/settings/settings-intent-link' -import { ANONYMOUS_USER_ID } from '@/lib/auth/constants' import { signOutAndRedirect } from '@/lib/auth/sign-out' import { getDesktopUpdates } from '@/lib/desktop' import { rememberSettingsReturnUrl } from '@/lib/navigation/settings-return' diff --git a/apps/sim/blocks/icon-color.ts b/apps/sim/blocks/icon-color.ts index e60bf408cc3..6f85b343c65 100644 --- a/apps/sim/blocks/icon-color.ts +++ b/apps/sim/blocks/icon-color.ts @@ -4,27 +4,7 @@ * these without pulling 282 block configs and the tool registry into its bundle. * Registry-backed icon styling lives in `@/blocks/brand-icon`. */ -import { perceivedBackgroundBrightness } from '@sim/utils/color' - -/** - * Brightness above which a brand tile is "clearly light" and a white foreground - * icon would wash out. Set deliberately high (0.75) so only genuinely light - * tiles flip their icon to dark: it keeps monochrome `currentColor` icons - * legible on their pale tiles (Notion/Mailchimp/Infisical sit at ~0.83+) while - * leaving mid-bright saturated brand tiles (HubSpot orange, amber notes) on the - * white icon they have always used — avoiding a needless app-wide recolor. - */ -const LIGHT_TILE_THRESHOLD = 0.75 - -/** - * True when a block's {@link BlockConfig.bgColor} tile is light enough that a - * white foreground icon would wash out. Gradients use the average brightness - * of their supported color stops; unknown values are treated as dark. - */ -export function isLightTileColor(bgColor: string | null | undefined): boolean { - const brightness = bgColor ? perceivedBackgroundBrightness(bgColor) : null - return brightness !== null && brightness > LIGHT_TILE_THRESHOLD -} +import { isLightTileColor } from '@sim/workflow-renderer/tile-icon-color' /** * Tailwind foreground class for a brand icon rendered inside its diff --git a/apps/sim/lib/api/server/routes/v2-api-key-auth.ts b/apps/sim/lib/api/server/routes/v2-api-key-auth.ts index e04a1844276..7e7b5f79a1f 100644 --- a/apps/sim/lib/api/server/routes/v2-api-key-auth.ts +++ b/apps/sim/lib/api/server/routes/v2-api-key-auth.ts @@ -3,6 +3,7 @@ import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal, } from '@sim/auth/principal' +import { ANONYMOUS_USER_ID } from '@sim/auth/principal' import { db } from '@sim/db' import { apiKey, user } from '@sim/db/schema' import { createLogger } from '@sim/logger' @@ -10,7 +11,6 @@ import { eq } from 'drizzle-orm' import type { V2CredentialHeaders } from '@/lib/api/server/routes/v2-credential-headers' import { hashApiKey } from '@/lib/api-key/crypto' import { updateApiKeyLastUsed } from '@/lib/api-key/service' -import { ANONYMOUS_USER_ID } from '@/lib/auth/constants' import { InvalidOAuthAccessTokenError, type OAuthAccessTokenOptions, diff --git a/apps/sim/lib/auth/anonymous.ts b/apps/sim/lib/auth/anonymous.ts index c4be061bea0..73be753918f 100644 --- a/apps/sim/lib/auth/anonymous.ts +++ b/apps/sim/lib/auth/anonymous.ts @@ -1,9 +1,9 @@ +import { ANONYMOUS_USER, ANONYMOUS_USER_ID } from '@sim/auth/principal' import { db } from '@sim/db' import * as schema from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' import { eq } from 'drizzle-orm' -import { ANONYMOUS_USER, ANONYMOUS_USER_ID } from './constants' const logger = createLogger('AnonymousAuth') diff --git a/apps/sim/lib/auth/constants.ts b/apps/sim/lib/auth/constants.ts index 6e056669329..bb22d317d01 100644 --- a/apps/sim/lib/auth/constants.ts +++ b/apps/sim/lib/auth/constants.ts @@ -1,14 +1,3 @@ -/** Anonymous user ID used when DISABLE_AUTH is enabled */ -export const ANONYMOUS_USER_ID = '00000000-0000-0000-0000-000000000000' - -export const ANONYMOUS_USER = { - id: ANONYMOUS_USER_ID, - name: 'Anonymous', - email: 'anonymous@localhost', - emailVerified: true, - image: null, -} as const - /** * Provider IDs permitted to authenticate (create or link a session) through the * unauthenticated sign-in endpoints `/sign-in/social` and `/sign-in/oauth2`. diff --git a/apps/sim/lib/core/config/api-keys.ts b/apps/sim/lib/core/config/api-keys.ts index 01c04bc096a..e6c87d31038 100644 --- a/apps/sim/lib/core/config/api-keys.ts +++ b/apps/sim/lib/core/config/api-keys.ts @@ -1,5 +1,5 @@ +import { LLM_KEY_POOLS } from '@sim/deployment-config/env-capabilities' import { env } from '@/lib/core/config/env' -import { LLM_KEY_POOLS } from '@/lib/core/config/env-capabilities' /** Whether the platform holds at least one key for a provider, without selecting one. */ export function hasRotatingApiKey(provider: string): boolean { diff --git a/apps/sim/lib/core/config/env-capabilities.server.ts b/apps/sim/lib/core/config/env-capabilities.server.ts index 698a44dca1b..271d46a266e 100644 --- a/apps/sim/lib/core/config/env-capabilities.server.ts +++ b/apps/sim/lib/core/config/env-capabilities.server.ts @@ -4,7 +4,6 @@ * @packageDocumentation */ -import { env } from '@/lib/core/config/env' import { ASYNC_JOBS_CAPABILITY, CACHE_CAPABILITY, @@ -20,7 +19,8 @@ import { STORAGE_CAPABILITY, type WireFallbackOptions, wireFallback, -} from '@/lib/core/config/env-capabilities' +} from '@sim/deployment-config/env-capabilities' +import { env } from '@/lib/core/config/env' export function getConfiguredStorageProviderId() { return requireCapability(STORAGE_CAPABILITY, env).providerId diff --git a/apps/sim/lib/core/config/env-capabilities.test.ts b/apps/sim/lib/core/config/env-capabilities.test.ts index 638c4b0b3ab..944aa75c038 100644 --- a/apps/sim/lib/core/config/env-capabilities.test.ts +++ b/apps/sim/lib/core/config/env-capabilities.test.ts @@ -1,6 +1,3 @@ -import integrationsJson from '@sim/deployment-config/integrations.json' -import { CREDENTIAL_CONFIGURED_OAUTH_SERVICE_IDS } from '@sim/deployment-config/service-account-metadata' -import { describe, expect, it, vi } from 'vitest' import { ASYNC_JOBS_CAPABILITY, CACHE_CAPABILITY, @@ -17,7 +14,10 @@ import { SANDBOX_CAPABILITY, STORAGE_CAPABILITY, wireFallback, -} from '@/lib/core/config/env-capabilities' +} from '@sim/deployment-config/env-capabilities' +import integrationsJson from '@sim/deployment-config/integrations.json' +import { CREDENTIAL_CONFIGURED_OAUTH_SERVICE_IDS } from '@sim/deployment-config/service-account-metadata' +import { describe, expect, it, vi } from 'vitest' import type { Integration } from '@/lib/integrations/types' import { getServiceConfigByServiceId } from '@/lib/oauth/utils' diff --git a/apps/sim/lib/core/config/env-capabilities.ts b/apps/sim/lib/core/config/env-capabilities.ts deleted file mode 100644 index 4a36493790b..00000000000 --- a/apps/sim/lib/core/config/env-capabilities.ts +++ /dev/null @@ -1,2 +0,0 @@ -/** Application compatibility surface for shared deployment capability policy. */ -export * from '@sim/deployment-config/env-capabilities' diff --git a/apps/sim/lib/core/config/env-flags.ts b/apps/sim/lib/core/config/env-flags.ts index 9ad4f597f0d..c50e34abb48 100644 --- a/apps/sim/lib/core/config/env-flags.ts +++ b/apps/sim/lib/core/config/env-flags.ts @@ -3,6 +3,11 @@ * config-boundary dependencies use workspace packages or relative imports. */ +import { + hasEnvCapabilityValue, + inspectCapability, + SANDBOX_CAPABILITY, +} from '@sim/deployment-config/env-capabilities' import { isImmutableDaytonaSnapshotRef, isImmutableE2BTemplateRef, @@ -16,7 +21,6 @@ import { resolveSandboxFeatureAvailability, } from './enterprise-entitlements' import { env, envBoolean, envNumber, getEnv, isFalsy, isTruthy } from './env' -import { hasEnvCapabilityValue, inspectCapability, SANDBOX_CAPABILITY } from './env-capabilities' /** * Is the application running in production mode diff --git a/apps/sim/lib/embeddings/client.ts b/apps/sim/lib/embeddings/client.ts index b7fdfe9a9bd..4596d14750e 100644 --- a/apps/sim/lib/embeddings/client.ts +++ b/apps/sim/lib/embeddings/client.ts @@ -1,3 +1,8 @@ +import { + type FallbackFactories, + KNOWLEDGE_EMBEDDINGS_CAPABILITY, + wireFallback, +} from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { sha256Hex } from '@sim/security/hash' import { chunkArray } from '@sim/utils/helpers' @@ -5,11 +10,6 @@ import { truncate } from '@sim/utils/string' import { getBYOKKey } from '@/lib/api-key/byok' import { getRotatingApiKey } from '@/lib/core/config/api-keys' import { env, envNumber } from '@/lib/core/config/env' -import { - type FallbackFactories, - KNOWLEDGE_EMBEDDINGS_CAPABILITY, - wireFallback, -} from '@/lib/core/config/env-capabilities' import { isHosted } from '@/lib/core/config/env-flags' import { isQuotaExhaustionBody } from '@/lib/core/errors/provider-quota' import { diff --git a/apps/sim/lib/embeddings/knowledge-embedding-family.test.ts b/apps/sim/lib/embeddings/knowledge-embedding-family.test.ts index f22260e2274..559e406f6d7 100644 --- a/apps/sim/lib/embeddings/knowledge-embedding-family.test.ts +++ b/apps/sim/lib/embeddings/knowledge-embedding-family.test.ts @@ -1,9 +1,9 @@ -import { describe, expect, it } from 'vitest' import { inspectCapability, KNOWLEDGE_EMBEDDINGS_CAPABILITY, knowledgeEmbeddingFamily, -} from '@/lib/core/config/env-capabilities' +} from '@sim/deployment-config/env-capabilities' +import { describe, expect, it } from 'vitest' import { DEFAULT_EMBEDDING_MODEL, getEmbeddingModelInfo, diff --git a/apps/sim/lib/execution/remote-sandbox/pi-lifetime.ts b/apps/sim/lib/execution/remote-sandbox/pi-lifetime.ts index a7de2543c8f..d1f30ebdd8e 100644 --- a/apps/sim/lib/execution/remote-sandbox/pi-lifetime.ts +++ b/apps/sim/lib/execution/remote-sandbox/pi-lifetime.ts @@ -4,9 +4,9 @@ * against the same number without importing the provider SDKs. */ +import { inspectCapability, SANDBOX_CAPABILITY } from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { env } from '@/lib/core/config/env' -import { inspectCapability, SANDBOX_CAPABILITY } from '@/lib/core/config/env-capabilities' import { getMaxExecutionTimeout, getRemainingExecutionMs } from '@/lib/core/execution-limits' const logger = createLogger('PiSandboxLifetime') diff --git a/apps/sim/lib/integrations/availability.server.test.ts b/apps/sim/lib/integrations/availability.server.test.ts index 64f700324e2..b976b30ef18 100644 --- a/apps/sim/lib/integrations/availability.server.test.ts +++ b/apps/sim/lib/integrations/availability.server.test.ts @@ -1,10 +1,14 @@ -import integrationsJson from '@sim/deployment-config/integrations.json' -import { resetEnvMock, setEnv } from '@sim/testing/mocks/env.mock' -import { afterAll, describe, expect, it } from 'vitest' import { OAUTH_CLIENT_CAPABILITIES, resolveOAuthClientCapabilityId, -} from '@/lib/core/config/env-capabilities' +} from '@sim/deployment-config/env-capabilities' +import integrationsJson from '@sim/deployment-config/integrations.json' +import { + CREDENTIAL_CONFIGURED_OAUTH_SERVICE_IDS, + SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID, +} from '@sim/deployment-config/service-account-metadata' +import { resetEnvMock, setEnv } from '@sim/testing/mocks/env.mock' +import { afterAll, describe, expect, it } from 'vitest' import { getIntegrationTypesForOAuthServiceId, type IntegrationAvailability, @@ -17,10 +21,6 @@ import { isIntegrationDeploymentAvailable, isIntegrationDeploymentAvailableForVisibility, } from '@/lib/integrations/availability.server' -import { - CREDENTIAL_CONFIGURED_OAUTH_SERVICE_IDS, - SERVICE_ACCOUNT_METADATA_BY_OAUTH_SERVICE_ID, -} from '@/lib/integrations/service-account-metadata' import type { Integration } from '@/lib/integrations/types' import { getServiceConfigByServiceId } from '@/lib/oauth/utils' diff --git a/apps/sim/lib/integrations/availability.server.ts b/apps/sim/lib/integrations/availability.server.ts index 103920d8014..e490cd6be9a 100644 --- a/apps/sim/lib/integrations/availability.server.ts +++ b/apps/sim/lib/integrations/availability.server.ts @@ -1,10 +1,10 @@ -import { stripVersionSuffix } from '@sim/utils/string' -import type { BlockVisibilityState } from '@/lib/core/config/block-visibility' -import { env } from '@/lib/core/config/env' import { inspectOAuthClientCapability, resolveOAuthClientCapabilityId, -} from '@/lib/core/config/env-capabilities' +} from '@sim/deployment-config/env-capabilities' +import { stripVersionSuffix } from '@sim/utils/string' +import type { BlockVisibilityState } from '@/lib/core/config/block-visibility' +import { env } from '@/lib/core/config/env' import { type IntegrationAvailability, resolveIntegrationAvailability, diff --git a/apps/sim/lib/integrations/service-account-metadata.ts b/apps/sim/lib/integrations/service-account-metadata.ts deleted file mode 100644 index 9e13b80eb33..00000000000 --- a/apps/sim/lib/integrations/service-account-metadata.ts +++ /dev/null @@ -1,2 +0,0 @@ -/** Application compatibility surface for shared service-account deployment metadata. */ -export * from '@sim/deployment-config/service-account-metadata' diff --git a/apps/sim/lib/knowledge/documents/document-processor.ts b/apps/sim/lib/knowledge/documents/document-processor.ts index eb00b2807f3..324c4b97c14 100644 --- a/apps/sim/lib/knowledge/documents/document-processor.ts +++ b/apps/sim/lib/knowledge/documents/document-processor.ts @@ -1,3 +1,4 @@ +import { OCR_CAPABILITY, requireCapability } from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { sha256Hex } from '@sim/security/hash' import { toError } from '@sim/utils/errors' @@ -17,7 +18,6 @@ import { } from '@/lib/chunkers' import type { ChunkingStrategy, StrategyOptions } from '@/lib/chunkers/types' import { env } from '@/lib/core/config/env' -import { OCR_CAPABILITY, requireCapability } from '@/lib/core/config/env-capabilities' import { recordProviderCooldown, waitForProviderAdmission, diff --git a/apps/sim/lib/messaging/email/providers/index.ts b/apps/sim/lib/messaging/email/providers/index.ts index 19428079e6a..3e38db6ec4e 100644 --- a/apps/sim/lib/messaging/email/providers/index.ts +++ b/apps/sim/lib/messaging/email/providers/index.ts @@ -1,5 +1,5 @@ +import { EMAIL_CAPABILITY, type FallbackFactories } from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' -import { EMAIL_CAPABILITY, type FallbackFactories } from '@/lib/core/config/env-capabilities' import { wireServerFallback } from '@/lib/core/config/env-capabilities.server' import { createAzureProvider } from '@/lib/messaging/email/providers/azure' import { createGmailProvider } from '@/lib/messaging/email/providers/gmail' diff --git a/apps/sim/lib/mothership/chat/process-contents.ts b/apps/sim/lib/mothership/chat/process-contents.ts index 777ef195c56..3c6a07884a6 100644 --- a/apps/sim/lib/mothership/chat/process-contents.ts +++ b/apps/sim/lib/mothership/chat/process-contents.ts @@ -1,4 +1,5 @@ import { db } from '@sim/db' +import { EnvCapabilityConfigurationError } from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { authorizeWorkflowByWorkspacePermission, @@ -7,7 +8,6 @@ import { import { escapeRegExp } from '@sim/utils/string' import { eq } from 'drizzle-orm' import type { MothershipTableViewContext } from '@/lib/api/contracts/mothership-resources' -import { EnvCapabilityConfigurationError } from '@/lib/core/config/env-capabilities' import { getAllowedIntegrationsFromEnv } from '@/lib/core/config/env-flags' import { OrchestrationError } from '@/lib/core/orchestration/types' import { mapWithConcurrency } from '@/lib/core/utils/concurrency' diff --git a/apps/sim/lib/oauth/oauth.ts b/apps/sim/lib/oauth/oauth.ts index fb06f9f261b..71c20c39ef2 100644 --- a/apps/sim/lib/oauth/oauth.ts +++ b/apps/sim/lib/oauth/oauth.ts @@ -1,3 +1,8 @@ +import { + type OAuthClientCapabilityField, + type OAuthClientCapabilityId, + requireOAuthClientCapability, +} from '@sim/deployment-config/env-capabilities' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import { @@ -69,11 +74,6 @@ import { ZoomIcon, } from '@/components/icons' import { env } from '@/lib/core/config/env' -import { - type OAuthClientCapabilityField, - type OAuthClientCapabilityId, - requireOAuthClientCapability, -} from '@/lib/core/config/env-capabilities' import { isSlackExtendedScopesEnabled } from '@/lib/core/config/env-flags' import { redactExactSensitiveValues } from '@/lib/core/security/redaction' import { diff --git a/apps/sim/lib/workflows/application/apply-workflow-operations.test.ts b/apps/sim/lib/workflows/application/apply-workflow-operations.test.ts index 7c166e4c283..45a08186833 100644 --- a/apps/sim/lib/workflows/application/apply-workflow-operations.test.ts +++ b/apps/sim/lib/workflows/application/apply-workflow-operations.test.ts @@ -88,7 +88,7 @@ vi.mock('@/lib/permission-groups/resolve.server', () => permissionGroupsResolveM vi.mock('@/blocks/visibility/server-context', () => ({ withBlockVisibility: (_state: unknown, run: () => unknown) => run(), })) -vi.mock('@/stores/workflows/workflow/utils', () => ({ +vi.mock('@sim/workflow-persistence/subflow-helpers', () => ({ generateLoopBlocks: () => ({}), generateParallelBlocks: () => ({}), })) diff --git a/apps/sim/lib/workflows/application/apply-workflow-operations.ts b/apps/sim/lib/workflows/application/apply-workflow-operations.ts index ef73c302c92..4052746ebbd 100644 --- a/apps/sim/lib/workflows/application/apply-workflow-operations.ts +++ b/apps/sim/lib/workflows/application/apply-workflow-operations.ts @@ -3,6 +3,10 @@ import { type Principal, resolvePrincipalAttribution } from '@sim/auth/principal import { db } from '@sim/db' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import type { BlockState, WorkflowState } from '@sim/workflow-types/workflow' import { hasWorkspaceSandboxAccess } from '@/lib/billing/core/subscription' import { ForbiddenOperationError, principalAuditSource } from '@/lib/core/application' @@ -56,7 +60,6 @@ import { import { loadWorkflowFromNormalizedTables } from '@/lib/workflows/persistence/utils' import { validateWorkflowState } from '@/lib/workflows/sanitization/validation' import { withBlockVisibility } from '@/blocks/visibility/server-context' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' import { normalizeWorkflowState } from '@/stores/workflows/workflow/validation' const logger = createLogger('ApplyWorkflowOperations') diff --git a/apps/sim/lib/workflows/application/update-workflow-content.ts b/apps/sim/lib/workflows/application/update-workflow-content.ts index a50cda3848b..bd7501c55f0 100644 --- a/apps/sim/lib/workflows/application/update-workflow-content.ts +++ b/apps/sim/lib/workflows/application/update-workflow-content.ts @@ -3,6 +3,10 @@ import { type Principal, resolvePrincipalAttribution } from '@sim/auth/principal import { db } from '@sim/db' import { workflow } from '@sim/db/schema' import { generateId } from '@sim/utils/id' +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import type { BlockState, WorkflowState } from '@sim/workflow-types/workflow' import { and, eq, isNull } from 'drizzle-orm' import { principalAuditSource } from '@/lib/core/application' @@ -24,7 +28,6 @@ import { } from '@/lib/workflows/editing/block-enablement' import { replaceWorkflowNormalizedState } from '@/lib/workflows/persistence/replace-normalized-state' import { loadWorkflowFromNormalizedTables } from '@/lib/workflows/persistence/utils' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' const MAX_WORKFLOW_VARIABLE_OPERATIONS = 100 diff --git a/apps/sim/lib/workflows/diff/diff-engine.test.ts b/apps/sim/lib/workflows/diff/diff-engine.test.ts index ab59c154bd9..7d6c5dc1954 100644 --- a/apps/sim/lib/workflows/diff/diff-engine.test.ts +++ b/apps/sim/lib/workflows/diff/diff-engine.test.ts @@ -35,7 +35,7 @@ vi.mock('@/lib/workflows/autolayout/constants', () => ({ DEFAULT_LAYOUT_OPTIONS: {}, })) -vi.mock('@/stores/workflows/workflow/utils', () => ({ +vi.mock('@sim/workflow-persistence/subflow-helpers', () => ({ generateLoopBlocks: () => ({}), generateParallelBlocks: () => ({}), })) diff --git a/apps/sim/lib/workflows/diff/diff-engine.ts b/apps/sim/lib/workflows/diff/diff-engine.ts index 6a3b3918b6e..63277399e12 100644 --- a/apps/sim/lib/workflows/diff/diff-engine.ts +++ b/apps/sim/lib/workflows/diff/diff-engine.ts @@ -416,7 +416,7 @@ export class WorkflowDiffEngine { // This ensures the nodes arrays in loops/parallels contain the correct (remapped) block IDs, // which is critical for variable resolution in the tag dropdown. const { generateLoopBlocks, generateParallelBlocks } = await import( - '@/stores/workflows/workflow/utils' + '@sim/workflow-persistence/subflow-helpers' ) // Build the proposed state diff --git a/apps/sim/lib/workflows/editing/block-enablement.ts b/apps/sim/lib/workflows/editing/block-enablement.ts index dc8d92a21f9..d1b8a02a748 100644 --- a/apps/sim/lib/workflows/editing/block-enablement.ts +++ b/apps/sim/lib/workflows/editing/block-enablement.ts @@ -1,20 +1,4 @@ -import type { BlockState } from '@sim/workflow-types/workflow' - -/** Whether a block, or any container above it, is locked against edits. */ -export function isBlockProtected(blockId: string, blocksById: Record): boolean { - const block = blocksById[blockId] - if (!block) return false - if (block.locked) return true - - const visited = new Set() - let parentId = block.data?.parentId - while (parentId && !visited.has(parentId)) { - visited.add(parentId) - if (blocksById[parentId]?.locked) return true - parentId = blocksById[parentId]?.data?.parentId - } - return false -} +import { type BlockState, isWorkflowBlockProtected } from '@sim/workflow-types/workflow' /** Whether any container above a block is disabled, which keeps the block from running. */ export function hasDisabledAncestor( @@ -88,7 +72,7 @@ export function decideBlockEnablement( refusal: { reason: 'not_found', message: `Block ${blockId} not found` }, } } - if (isBlockProtected(blockId, blocks)) { + if (isWorkflowBlockProtected(blockId, blocks)) { return { outcome: 'refused', refusal: { @@ -110,7 +94,7 @@ export function decideBlockEnablement( const affectedBlockIds = new Set([blockId]) if (targetBlock.type === 'loop' || targetBlock.type === 'parallel') { for (const descendantId of findDescendants(blockId, blocks)) { - if (!isBlockProtected(descendantId, blocks)) { + if (!isWorkflowBlockProtected(descendantId, blocks)) { affectedBlockIds.add(descendantId) } } diff --git a/apps/sim/lib/workflows/editing/engine.ts b/apps/sim/lib/workflows/editing/engine.ts index db8487f82d6..623f4838074 100644 --- a/apps/sim/lib/workflows/editing/engine.ts +++ b/apps/sim/lib/workflows/editing/engine.ts @@ -1,4 +1,8 @@ import { createLogger } from '@sim/logger' +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import type { BlockState } from '@sim/workflow-types/workflow' import { isEqual } from 'es-toolkit' import type { PermissionGroupConfig } from '@/lib/permission-groups/fields' @@ -8,7 +12,6 @@ import { reindexRewrittenToolCanonicalModes } from '@/lib/workflows/subblocks/vi import { applyAgentToolUsageControlModes } from '@/lib/workflows/tool-input/usage-control' import { getBlock } from '@/blocks/registry' import { validateEdges } from '@/stores/workflows/workflow/edge-validation' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' import { addConnectionsAsEdges, createValidatedEdge, diff --git a/apps/sim/lib/workflows/persistence/prepare-state.ts b/apps/sim/lib/workflows/persistence/prepare-state.ts index 6593bb10a93..193fecadd86 100644 --- a/apps/sim/lib/workflows/persistence/prepare-state.ts +++ b/apps/sim/lib/workflows/persistence/prepare-state.ts @@ -1,7 +1,10 @@ +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import { sanitizeAgentToolsInBlocks } from '@/lib/workflows/sanitization/validation' import { validateEdges } from '@/stores/workflows/workflow/edge-validation' import type { BlockState, WorkflowState } from '@/stores/workflows/workflow/types' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' export interface PreparedWorkflowState { blocks: Record diff --git a/apps/sim/lib/workflows/persistence/utils.test.ts b/apps/sim/lib/workflows/persistence/utils.test.ts index 91d24c8e05e..255c0141ea1 100644 --- a/apps/sim/lib/workflows/persistence/utils.test.ts +++ b/apps/sim/lib/workflows/persistence/utils.test.ts @@ -20,13 +20,13 @@ import { resetDbChainMock, schemaMock, } from '@sim/testing' +import { generateLoopBlocks } from '@sim/workflow-persistence/subflow-helpers' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' import { workflowStateSchema } from '@/lib/api/contracts/workflows' import type { BlockState as AppBlockState, WorkflowState as AppWorkflowState, } from '@/stores/workflows/workflow/types' -import { generateLoopBlocks } from '@/stores/workflows/workflow/utils' /** * Type helper for converting test workflow state to app workflow state. diff --git a/apps/sim/lib/workflows/sanitization/json-sanitizer.ts b/apps/sim/lib/workflows/sanitization/json-sanitizer.ts index d7ecff40435..69878d43c27 100644 --- a/apps/sim/lib/workflows/sanitization/json-sanitizer.ts +++ b/apps/sim/lib/workflows/sanitization/json-sanitizer.ts @@ -1,4 +1,8 @@ import { isRecordLike, sortObjectKeysDeep, toRecord } from '@sim/utils/object' +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import { normalizeWorkflowEdgeSourceHandle } from '@sim/workflow-types/workflow' import type { Edge } from '@xyflow/react' import { getBaseUrl } from '@/lib/core/utils/urls' @@ -11,7 +15,6 @@ import type { Parallel, WorkflowState, } from '@/stores/workflows/workflow/types' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' import { TRIGGER_ROUTING_FIELD, TRIGGER_WEBHOOK_URL_FIELD } from '@/triggers/constants' import { blockAdvertisesWebhookUrl, resolveBlockTriggerId } from '@/triggers/webhook-url' diff --git a/apps/sim/serializer/index.ts b/apps/sim/serializer/index.ts index c90ba4758ac..c9c330361bc 100644 --- a/apps/sim/serializer/index.ts +++ b/apps/sim/serializer/index.ts @@ -1,6 +1,10 @@ import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import { resolveBlockRetryConfig } from '@sim/workflow-types/workflow' import type { Edge } from '@xyflow/react' import { migrateMcpOperationControls } from '@/lib/workflows/migrations/mcp-operation-controls' @@ -21,7 +25,6 @@ import { isCustomBlockType, RESERVED_PARAMS } from '@/blocks/custom/build-config import type { SubBlockConfig } from '@/blocks/types' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' import type { BlockState, Loop, Parallel } from '@/stores/workflows/workflow/types' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' import { getToolParams } from '@/tools/metadata' import { expandSubBlockValueToParams } from '@/tools/param-shape' diff --git a/apps/sim/stores/workflows/workflow/store.ts b/apps/sim/stores/workflows/workflow/store.ts index 01692d1f82a..6305b9d354e 100644 --- a/apps/sim/stores/workflows/workflow/store.ts +++ b/apps/sim/stores/workflows/workflow/store.ts @@ -1,5 +1,10 @@ import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' +import { + clampParallelBatchSize, + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import type { BlockRetryConfig } from '@sim/workflow-types/workflow' import { filterAcyclicEdges, getWorkflowBlockNameConflict } from '@sim/workflow-types/workflow' import type { Edge } from '@xyflow/react' @@ -26,13 +31,7 @@ import type { WorkflowState, WorkflowStore, } from '@/stores/workflows/workflow/types' -import { - clampParallelBatchSize, - findAllDescendantNodes, - generateLoopBlocks, - generateParallelBlocks, - isBlockProtected, -} from '@/stores/workflows/workflow/utils' +import { findAllDescendantNodes, isBlockProtected } from '@/stores/workflows/workflow/utils' import { normalizeWorkflowState } from '@/stores/workflows/workflow/validation' const logger = createLogger('WorkflowStore') diff --git a/apps/sim/stores/workflows/workflow/utils.test.ts b/apps/sim/stores/workflows/workflow/utils.test.ts index 00995f86a01..e73f3ed4d6d 100644 --- a/apps/sim/stores/workflows/workflow/utils.test.ts +++ b/apps/sim/stores/workflows/workflow/utils.test.ts @@ -1,11 +1,8 @@ import { createAgentBlock, createLoopBlock } from '@sim/testing' +import { convertLoopBlockToLoop } from '@sim/workflow-persistence/subflow-helpers' import { describe, expect, it } from 'vitest' import type { BlockState } from '@/stores/workflows/workflow/types' -import { - convertLoopBlockToLoop, - isAncestorProtected, - isBlockProtected, -} from '@/stores/workflows/workflow/utils' +import { isAncestorProtected, isBlockProtected } from '@/stores/workflows/workflow/utils' describe('convertLoopBlockToLoop', () => { it.concurrent('should keep string as-is if not valid JSON', () => { diff --git a/apps/sim/stores/workflows/workflow/utils.ts b/apps/sim/stores/workflows/workflow/utils.ts index 3999e7e131b..37d466b61e5 100644 --- a/apps/sim/stores/workflows/workflow/utils.ts +++ b/apps/sim/stores/workflows/workflow/utils.ts @@ -2,101 +2,7 @@ import { isWorkflowBlockAncestorLocked, isWorkflowBlockProtected, } from '@sim/workflow-types/workflow' -import type { BlockState, Loop, Parallel } from '@/stores/workflows/workflow/types' - -const DEFAULT_LOOP_ITERATIONS = 5 -const DEFAULT_PARALLEL_BATCH_SIZE = 20 -const MAX_PARALLEL_BATCH_SIZE = 20 - -export function clampParallelBatchSize(batchSize: unknown): number { - const parsed = typeof batchSize === 'number' ? batchSize : Number.parseInt(String(batchSize), 10) - if (Number.isNaN(parsed)) { - return DEFAULT_PARALLEL_BATCH_SIZE - } - return Math.max(1, Math.min(MAX_PARALLEL_BATCH_SIZE, parsed)) -} - -/** - * Convert UI loop block to executor Loop format - * - * @param loopBlockId - ID of the loop block to convert - * @param blocks - Record of all blocks in the workflow - * @returns Loop object for execution engine or undefined if not a valid loop - */ -export function convertLoopBlockToLoop( - loopBlockId: string, - blocks: Record -): Loop | undefined { - const loopBlock = blocks[loopBlockId] - if (!loopBlock || loopBlock.type !== 'loop') return undefined - - const loopType = loopBlock.data?.loopType || 'for' - - const loop: Loop = { - id: loopBlockId, - nodes: findChildNodes(loopBlockId, blocks), - iterations: loopBlock.data?.count || DEFAULT_LOOP_ITERATIONS, - loopType, - enabled: loopBlock.enabled, - } - - loop.forEachItems = loopBlock.data?.collection || '' - loop.whileCondition = loopBlock.data?.whileCondition || '' - loop.doWhileCondition = loopBlock.data?.doWhileCondition || '' - - return loop -} - -/** - * Convert UI parallel block to executor Parallel format - * - * @param parallelBlockId - ID of the parallel block to convert - * @param blocks - Record of all blocks in the workflow - * @returns Parallel object for execution engine or undefined if not a valid parallel block - */ -export function convertParallelBlockToParallel( - parallelBlockId: string, - blocks: Record -): Parallel | undefined { - const parallelBlock = blocks[parallelBlockId] - if (!parallelBlock || parallelBlock.type !== 'parallel') return undefined - - const parallelType = parallelBlock.data?.parallelType || 'count' - - const validParallelTypes = ['collection', 'count'] as const - const validatedParallelType = validParallelTypes.includes(parallelType as any) - ? parallelType - : 'collection' - - const distribution = - validatedParallelType === 'collection' ? parallelBlock.data?.collection || '' : undefined - - const count = parallelBlock.data?.count || 5 - const batchSize = clampParallelBatchSize(parallelBlock.data?.batchSize) - - return { - id: parallelBlockId, - nodes: findChildNodes(parallelBlockId, blocks), - distribution, - count, - parallelType: validatedParallelType, - batchSize, - enabled: parallelBlock.enabled, - } -} - -/** - * Find all nodes that are children of this container (loop or parallel) - * - * @param containerId - ID of the container to find children for - * @param blocks - Record of all blocks in the workflow - * @returns Array of node IDs that are direct children of this container - */ -export function findChildNodes(containerId: string, blocks: Record): string[] { - return Object.values(blocks) - .filter((block) => block.data?.parentId === containerId) - .map((block) => block.id) -} +import type { BlockState } from '@/stores/workflows/workflow/types' /** * Find all descendant nodes, including children, grandchildren, etc. @@ -149,47 +55,3 @@ export function isAncestorProtected(blockId: string, blocks: Record): boolean { return isWorkflowBlockProtected(blockId, blocks) } - -/** - * Builds a complete collection of loops from the UI blocks - * - * @param blocks - Record of all blocks in the workflow - * @returns Record of Loop objects for execution engine - */ -export function generateLoopBlocks(blocks: Record): Record { - const loops: Record = {} - - Object.entries(blocks) - .filter(([_, block]) => block.type === 'loop') - .forEach(([id, block]) => { - const loop = convertLoopBlockToLoop(id, blocks) - if (loop) { - loops[id] = loop - } - }) - - return loops -} - -/** - * Builds a complete collection of parallel blocks from the UI blocks - * - * @param blocks - Record of all blocks in the workflow - * @returns Record of Parallel objects for execution engine - */ -export function generateParallelBlocks( - blocks: Record -): Record { - const parallels: Record = {} - - Object.entries(blocks) - .filter(([_, block]) => block.type === 'parallel') - .forEach(([id, block]) => { - const parallel = convertParallelBlockToParallel(id, blocks) - if (parallel) { - parallels[id] = parallel - } - }) - - return parallels -} diff --git a/apps/sim/stores/workflows/workflow/validation.ts b/apps/sim/stores/workflows/workflow/validation.ts index bdfc3b5064f..d08e8cf62c6 100644 --- a/apps/sim/stores/workflows/workflow/validation.ts +++ b/apps/sim/stores/workflows/workflow/validation.ts @@ -1,6 +1,9 @@ +import { + generateLoopBlocks, + generateParallelBlocks, +} from '@sim/workflow-persistence/subflow-helpers' import { validateEdges } from '@/stores/workflows/workflow/edge-validation' import type { WorkflowState } from '@/stores/workflows/workflow/types' -import { generateLoopBlocks, generateParallelBlocks } from '@/stores/workflows/workflow/utils' export interface NormalizationResult { state: WorkflowState diff --git a/packages/auth/src/principal.ts b/packages/auth/src/principal.ts index 99bcd7ca1c5..7f0f7cb1ce7 100644 --- a/packages/auth/src/principal.ts +++ b/packages/auth/src/principal.ts @@ -847,3 +847,15 @@ export function resolvePrincipalAttribution( throw new PrincipalSubjectUserRequiredError(actor.kind) } } + +/** User ID every request acts as when `DISABLE_AUTH` is enabled. */ +export const ANONYMOUS_USER_ID = '00000000-0000-0000-0000-000000000000' + +/** The user record behind {@link ANONYMOUS_USER_ID}, shared by the app and the realtime server. */ +export const ANONYMOUS_USER = { + id: ANONYMOUS_USER_ID, + name: 'Anonymous', + email: 'anonymous@localhost', + emailVerified: true, + image: null, +} as const diff --git a/packages/sim-cli/src/auth/device-flow.ts b/packages/sim-cli/src/auth/device-flow.ts index 18f6e8dafe0..4a25861dd2b 100644 --- a/packages/sim-cli/src/auth/device-flow.ts +++ b/packages/sim-cli/src/auth/device-flow.ts @@ -1,6 +1,6 @@ import { createHash, randomBytes, randomInt } from 'node:crypto' +import { sleep } from '@sim/utils/helpers' import { writeStderr } from '#sim-cli/output/io' -import { sleep } from '../helpers' import { buildUrl, REDIRECT_STATUSES, redirectEndpoint, SimApiError } from '../http/client' import { identityHeaders } from '../telemetry/client-info' diff --git a/packages/sim-cli/src/commands/protocol/files-get.test.ts b/packages/sim-cli/src/commands/protocol/files-get.test.ts index a0ef49c4e36..0269462f7e7 100644 --- a/packages/sim-cli/src/commands/protocol/files-get.test.ts +++ b/packages/sim-cli/src/commands/protocol/files-get.test.ts @@ -12,9 +12,9 @@ import { import { tmpdir } from 'node:os' import { join } from 'node:path' import { Writable } from 'node:stream' +import { sleep } from '@sim/utils/helpers' import { Command } from 'commander' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { sleep } from '../../helpers' import { buildGeneratedCommands } from '../../runtime/build' import { removeStagingOnSignal, saveToFile, streamToFile } from './files-get' import { attachProtocolCommands } from './index' diff --git a/packages/sim-cli/src/commands/protocol/logs-follow.test.ts b/packages/sim-cli/src/commands/protocol/logs-follow.test.ts index dde93e647f4..bdb383787c9 100644 --- a/packages/sim-cli/src/commands/protocol/logs-follow.test.ts +++ b/packages/sim-cli/src/commands/protocol/logs-follow.test.ts @@ -9,7 +9,7 @@ const { mockSleep } = vi.hoisted(() => ({ mockSleep: vi.fn(() => Promise.resolve()), })) -vi.mock('../../helpers', () => ({ sleep: mockSleep })) +vi.mock('@sim/utils/helpers', () => ({ sleep: mockSleep })) vi.mock('../../context', async () => (await import('../../test/context-mock')).contextMock) diff --git a/packages/sim-cli/src/commands/protocol/logs-follow.ts b/packages/sim-cli/src/commands/protocol/logs-follow.ts index d0d374dac87..02043521703 100644 --- a/packages/sim-cli/src/commands/protocol/logs-follow.ts +++ b/packages/sim-cli/src/commands/protocol/logs-follow.ts @@ -1,3 +1,4 @@ +import { sleep } from '@sim/utils/helpers' import { type Command, Option } from 'commander' import { dump } from 'js-yaml' import { printLine, writeStderr } from '#sim-cli/output/io' @@ -7,7 +8,6 @@ import { clientFrom } from '../../context' import { CLI_CONTRACT } from '../../contract/commands' import type { ColumnSpec } from '../../contract/types' import { type ListLogsResponse, V2_OPERATIONS } from '../../generated/v2-api' -import { sleep } from '../../helpers' import { SimApiError, type SimClient } from '../../http/client' import { bool, diff --git a/packages/sim-cli/src/commands/protocol/workflow-run-wait.test.ts b/packages/sim-cli/src/commands/protocol/workflow-run-wait.test.ts index 80b387b7d3b..980ece89cbc 100644 --- a/packages/sim-cli/src/commands/protocol/workflow-run-wait.test.ts +++ b/packages/sim-cli/src/commands/protocol/workflow-run-wait.test.ts @@ -27,7 +27,7 @@ vi.mock('../../context', () => ({ * command reads its deadline from, so a `--wait-timeout 3600` test costs * nothing and the poll schedule is exactly what the assertions say it is. */ -vi.mock('../../helpers', () => ({ +vi.mock('@sim/utils/helpers', () => ({ sleep: (ms: number) => { sleeps.push(ms) clock.now += ms diff --git a/packages/sim-cli/src/commands/protocol/workflow-run-wait.ts b/packages/sim-cli/src/commands/protocol/workflow-run-wait.ts index fb715cc0657..f5785d1c7c0 100644 --- a/packages/sim-cli/src/commands/protocol/workflow-run-wait.ts +++ b/packages/sim-cli/src/commands/protocol/workflow-run-wait.ts @@ -1,3 +1,4 @@ +import { sleep } from '@sim/utils/helpers' import { isRecordLike, toRecordOrNull } from '@sim/utils/object' import { type Command, Option } from 'commander' import { printError, writeStderr } from '#sim-cli/output/io' @@ -7,7 +8,6 @@ import { CLI_CONTRACT } from '../../contract/commands' import type { CommandSpec } from '../../contract/types' import { setSoftExitCode } from '../../embed-context' import { V2_OPERATIONS } from '../../generated/v2-api' -import { sleep } from '../../helpers' import { resolvePath, SimApiError } from '../../http/client' import { renderResult } from '../../runtime/result' diff --git a/packages/sim-cli/src/commands/protocol/workspace-operation-wait.ts b/packages/sim-cli/src/commands/protocol/workspace-operation-wait.ts index 17d0b54dcec..b3d9d62e943 100644 --- a/packages/sim-cli/src/commands/protocol/workspace-operation-wait.ts +++ b/packages/sim-cli/src/commands/protocol/workspace-operation-wait.ts @@ -1,8 +1,8 @@ +import { sleep } from '@sim/utils/helpers' import type { Command } from 'commander' import { clientFrom } from '../../context' import { CLI_CONTRACT } from '../../contract/commands' import { type GetWorkspaceOperationResponse, V2_OPERATIONS } from '../../generated/v2-api' -import { sleep } from '../../helpers' import { resolvePath, SimApiError, type SimClient } from '../../http/client' import { renderResult } from '../../runtime/result' diff --git a/packages/sim-cli/src/config/profile.test.ts b/packages/sim-cli/src/config/profile.test.ts index a3b9547a00f..40377ae6282 100644 --- a/packages/sim-cli/src/config/profile.test.ts +++ b/packages/sim-cli/src/config/profile.test.ts @@ -10,8 +10,8 @@ import { } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' +import { sleep } from '@sim/utils/helpers' import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { sleep } from '../helpers' import { configPath, credentialsPath } from './paths' import { DEFAULT_ENDPOINT, diff --git a/packages/sim-cli/src/embed.test.ts b/packages/sim-cli/src/embed.test.ts index 3234b73c57d..1fa113f3a63 100644 --- a/packages/sim-cli/src/embed.test.ts +++ b/packages/sim-cli/src/embed.test.ts @@ -1,6 +1,6 @@ +import { sleep } from '@sim/utils/helpers' import { describe, expect, it, vi } from 'vitest' import { runEmbeddedCli } from './embed' -import { sleep } from './helpers' const IDENTITY = { endpoint: 'https://sim.internal.test', diff --git a/packages/sim-cli/src/embed.ts b/packages/sim-cli/src/embed.ts index 7c8d8335abb..2cad81cf817 100644 --- a/packages/sim-cli/src/embed.ts +++ b/packages/sim-cli/src/embed.ts @@ -1,3 +1,4 @@ +import { getErrorMessage } from '@sim/utils/errors' import { type Command, CommanderError } from 'commander' import { ProfileConfigError } from './config/index' import { @@ -193,10 +194,7 @@ function renderEmbeddedError(ctx: EmbedContext, error: unknown): number { } return error.exitCode } - ctx.stderr.diagnostic( - // utils-lint-allow: this published standalone CLI cannot import the private @sim/utils package. - `Error: ${sanitize(error instanceof Error ? error.message : String(error))}` - ) + ctx.stderr.diagnostic(`Error: ${sanitize(getErrorMessage(error))}`) return 1 } diff --git a/packages/sim-cli/src/helpers.ts b/packages/sim-cli/src/helpers.ts deleted file mode 100644 index 6b5f77fe373..00000000000 --- a/packages/sim-cli/src/helpers.ts +++ /dev/null @@ -1,12 +0,0 @@ -/** - * Local copies of the shared helpers. - * - * `@sim/utils` is a private workspace package, so the published `sim` package - * cannot depend on it — importing it would resolve in the monorepo and fail for - * anyone installing from npm. - */ - -/** Resolves after `ms` milliseconds. */ -export function sleep(ms: number): Promise { - return new Promise((resolve) => setTimeout(resolve, ms)) -} diff --git a/packages/sim-cli/src/http/client.test.ts b/packages/sim-cli/src/http/client.test.ts index 4bcbf398df4..b08ba9057df 100644 --- a/packages/sim-cli/src/http/client.test.ts +++ b/packages/sim-cli/src/http/client.test.ts @@ -1,6 +1,6 @@ +import { sleep } from '@sim/utils/helpers' import { afterEach, describe, expect, it, vi } from 'vitest' import { V2_OPERATIONS, type V2OperationName } from '../generated/v2-api' -import { sleep } from '../helpers' import { formatApiErrorDetails, requestAllPages, diff --git a/packages/utils/src/sandbox-references.ts b/packages/utils/src/sandbox-references.ts index 9798fb748a4..89d3117c1d2 100644 --- a/packages/utils/src/sandbox-references.ts +++ b/packages/utils/src/sandbox-references.ts @@ -18,15 +18,6 @@ const E2B_TEMPLATE_NAME_PATTERN = /^[a-z0-9][a-z0-9_-]{0,62}$/ const E2B_TEMPLATE_REFERENCE_NAME_PATTERN = /^(?:[a-z0-9][a-z0-9_-]{0,62}\/)?[a-z0-9][a-z0-9_-]{0,62}$/ -/** Normalizes a configured provider and rejects values outside the supported registry. */ -export function normalizeSandboxProvider( - value: string | undefined -): SandboxProviderName | undefined { - if (!value) return undefined - const normalized = value.toLowerCase() - return SANDBOX_PROVIDER_IDS.find((provider) => provider === normalized) -} - /** E2B template families use the provider's untagged, lowercase name grammar. */ export function isValidE2BTemplateName(value: string): boolean { return E2B_TEMPLATE_NAME_PATTERN.test(value) diff --git a/packages/workflow-renderer/src/lib/tile-icon-color.ts b/packages/workflow-renderer/src/lib/tile-icon-color.ts index 3f633f54e96..ecec63a941b 100644 --- a/packages/workflow-renderer/src/lib/tile-icon-color.ts +++ b/packages/workflow-renderer/src/lib/tile-icon-color.ts @@ -3,9 +3,10 @@ import { perceivedBackgroundBrightness } from '@sim/utils/color' /** * Foreground class for a brand icon rendered inside its colored block tile. * - * The brightness maths is `@sim/utils/color`, shared with `@/blocks/icon-color`, - * so the canvas and the rest of the app can never disagree about which tiles are - * light. Only the threshold lives here, and it matches that helper's. + * The single source of truth for which tiles are light: the canvas renders with it + * and the app's `@/blocks/icon-color` builds its Tailwind classes on it, so the two + * can never disagree. It imports only `@sim/utils/color`, keeping the landing + * bundle that reaches it through `@/blocks/icon-color` light. * * Block icons are increasingly drawn with `fill='currentColor'`, so a tile must * give them a foreground that contrasts the (fixed, non-theme) brand @@ -13,7 +14,12 @@ import { perceivedBackgroundBrightness } from '@sim/utils/color' * multi-color icons ignore the class and keep their own fills. */ -/** Tiles brighter than this flip their icon foreground to near-black. */ +/** + * Tiles brighter than this flip their icon foreground to near-black. Set + * deliberately high so only genuinely light tiles (Notion, Mailchimp, Infisical + * sit at ~0.83+) flip, while mid-bright saturated brand tiles (HubSpot orange, + * amber notes) keep the white icon they have always used. + */ const LIGHT_TILE_THRESHOLD = 0.75 /** Whether a provider tile needs dark foreground content for legibility. */ diff --git a/scripts/check-utils-enforcement.ts b/scripts/check-utils-enforcement.ts index a44162f89e1..5f69ab916e0 100644 --- a/scripts/check-utils-enforcement.ts +++ b/scripts/check-utils-enforcement.ts @@ -39,7 +39,6 @@ const ALLOWLISTED_FILES = new Set([ 'packages/utils/src/retry.test.ts', // Published standalone CLIs: `@sim/utils` is private, so they carry local // copies rather than a dependency that only resolves inside the monorepo. - 'packages/sim-cli/src/helpers.ts', 'packages/cli/src/index.ts', 'packages/ts-sdk/src/index.ts', // CJS bundle — cannot use ES module imports From b698002506129bf7f21d753c61b3749f2307fc5a Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:01:03 -0700 Subject: [PATCH 05/30] chore(cleanup): trim log-views to toOverview and use interruptibleSleep in data drains --- .../handlers/pi/cloud/babysit/backend.test.ts | 8 +- .../handlers/pi/cloud/babysit/backend.ts | 6 +- .../lib/data-drains/destinations/bigquery.ts | 6 +- .../lib/data-drains/destinations/datadog.ts | 5 +- apps/sim/lib/data-drains/destinations/gcs.ts | 6 +- .../lib/data-drains/destinations/snowflake.ts | 12 +- .../sim/lib/data-drains/destinations/utils.ts | 20 - .../lib/data-drains/destinations/webhook.ts | 4 +- apps/sim/lib/logs/log-views.test.ts | 259 +-------- apps/sim/lib/logs/log-views.ts | 499 ------------------ 10 files changed, 24 insertions(+), 801 deletions(-) diff --git a/apps/sim/executor/handlers/pi/cloud/babysit/backend.test.ts b/apps/sim/executor/handlers/pi/cloud/babysit/backend.test.ts index 3e49e584f31..e83a0564f4f 100644 --- a/apps/sim/executor/handlers/pi/cloud/babysit/backend.test.ts +++ b/apps/sim/executor/handlers/pi/cloud/babysit/backend.test.ts @@ -1,5 +1,6 @@ import { resetEnvMock, setEnv } from '@sim/testing' import { remoteSandboxMock, remoteSandboxMockFns } from '@sim/testing/mocks/remote-sandbox.mock' +import { utilsHelpersMock } from '@sim/testing/mocks/utils-helpers.mock' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' const { @@ -11,7 +12,6 @@ const { mockRequestReview, mockReviewLanded, mockResolvePiSandboxLifetime, - mockSleepUntilAborted, } = vi.hoisted(() => ({ mockFetchSnapshot: vi.fn(), mockFetchThreads: vi.fn(), @@ -21,13 +21,10 @@ const { mockRequestReview: vi.fn(), mockReviewLanded: vi.fn(), mockResolvePiSandboxLifetime: vi.fn(), - mockSleepUntilAborted: vi.fn(), })) vi.mock('@/lib/execution/remote-sandbox', () => remoteSandboxMock) -vi.mock('@/lib/data-drains/destinations/utils', () => ({ - sleepUntilAborted: mockSleepUntilAborted, -})) +vi.mock('@sim/utils/helpers', () => utilsHelpersMock) vi.mock('@/lib/execution/remote-sandbox/pi-lifetime', async (importOriginal) => { const original = await importOriginal() @@ -230,7 +227,6 @@ describe('runBabysitPiWithOptions', () => { mockRequestReview.mockReset() mockReviewLanded.mockReset() mockResolvePiSandboxLifetime.mockReturnValue(getMaxExecutionTimeout()) - mockSleepUntilAborted.mockResolvedValue(undefined) mockFetchDiagnostics.mockResolvedValue(new Map([['check:ci', 'failure output']])) mockReplyAndResolve.mockResolvedValue({ repliesPosted: 1, diff --git a/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts b/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts index 08d4af65ac5..9a549e430dd 100644 --- a/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts +++ b/apps/sim/executor/handlers/pi/cloud/babysit/backend.ts @@ -6,8 +6,8 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' +import { interruptibleSleep } from '@sim/utils/helpers' import { truncate } from '@sim/utils/string' -import { sleepUntilAborted } from '@/lib/data-drains/destinations/utils' import { type PiSandboxRunner, withPiSandbox } from '@/lib/execution/remote-sandbox' import { resolvePiRunLifetimeMs, @@ -620,7 +620,7 @@ async function waitForHeadConvergence( if (snapshot.headSha === newSha) return 'converged' if (snapshot.headSha !== previousSha) return 'third_party' if (attempt < options.convergenceAttempts - 1) { - await sleepUntilAborted(options.convergenceWaitMs, signal) + await interruptibleSleep(options.convergenceWaitMs, signal) if (signal.aborted) throw new Error('Pi run aborted') } } @@ -647,7 +647,7 @@ async function waitWithSandboxProbe( let remainingMs = durationMs while (remainingMs > 0) { const intervalMs = Math.min(remainingMs, SANDBOX_PROBE_INTERVAL_MS) - await sleepUntilAborted(intervalMs, signal) + await interruptibleSleep(intervalMs, signal) if (signal.aborted) throw new Error('Pi run aborted') const probe = await raceAbort(runner.run('true', { timeoutMs: FINALIZE_TIMEOUT_MS }), signal) if (probe.exitCode !== 0) throw new Error('Babysit sandbox stopped responding') diff --git a/apps/sim/lib/data-drains/destinations/bigquery.ts b/apps/sim/lib/data-drains/destinations/bigquery.ts index c01779c163a..da342c66f44 100644 --- a/apps/sim/lib/data-drains/destinations/bigquery.ts +++ b/apps/sim/lib/data-drains/destinations/bigquery.ts @@ -1,6 +1,7 @@ import { createHash } from 'node:crypto' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { interruptibleSleep } from '@sim/utils/helpers' import { backoffWithJitter, parseRetryAfter } from '@sim/utils/retry' import { JWT } from 'google-auth-library' import { z } from 'zod' @@ -9,7 +10,6 @@ import { parseNdjsonObjects, parseServiceAccount, refineServiceAccountJson, - sleepUntilAborted, } from '@/lib/data-drains/destinations/utils' import type { DeliveryMetadata, DrainDestination } from '@/lib/data-drains/types' @@ -222,7 +222,7 @@ async function insertAll(input: InsertAllInput): Promise { }) /** Drain the body so the keep-alive connection can be reused. */ await response.text().catch(() => '') - await sleepUntilAborted(retryAfterMs, input.signal) + await interruptibleSleep(retryAfterMs, input.signal) if (input.signal.aborted) throw input.signal.reason ?? new Error('Aborted') } catch (error) { /** @@ -238,7 +238,7 @@ async function insertAll(input: InsertAllInput): Promise { retryAfterMs, error: toError(error).message, }) - await sleepUntilAborted(retryAfterMs, input.signal) + await interruptibleSleep(retryAfterMs, input.signal) if (input.signal.aborted) throw input.signal.reason ?? new Error('Aborted') } } diff --git a/apps/sim/lib/data-drains/destinations/datadog.ts b/apps/sim/lib/data-drains/destinations/datadog.ts index 6a56c00ad95..801aec3574b 100644 --- a/apps/sim/lib/data-drains/destinations/datadog.ts +++ b/apps/sim/lib/data-drains/destinations/datadog.ts @@ -1,9 +1,10 @@ import { gzipSync } from 'node:zlib' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { interruptibleSleep } from '@sim/utils/helpers' import { backoffWithJitter, parseRetryAfter } from '@sim/utils/retry' import { z } from 'zod' -import { parseNdjsonObjects, sleepUntilAborted } from '@/lib/data-drains/destinations/utils' +import { parseNdjsonObjects } from '@/lib/data-drains/destinations/utils' import type { DeliveryMetadata, DrainDestination } from '@/lib/data-drains/types' const logger = createLogger('DataDrainDatadogDestination') @@ -184,7 +185,7 @@ async function postWithRetries(input: PostInput): Promise { await response.text().catch(() => '') } if (attempt < MAX_ATTEMPTS) { - await sleepUntilAborted(backoffWithJitter(attempt, retryAfterMs), input.signal) + await interruptibleSleep(backoffWithJitter(attempt, retryAfterMs), input.signal) } } throw lastError instanceof Error ? lastError : new Error('Datadog delivery failed after retries') diff --git a/apps/sim/lib/data-drains/destinations/gcs.ts b/apps/sim/lib/data-drains/destinations/gcs.ts index 40e288e7117..70a6d576915 100644 --- a/apps/sim/lib/data-drains/destinations/gcs.ts +++ b/apps/sim/lib/data-drains/destinations/gcs.ts @@ -1,5 +1,6 @@ import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { interruptibleSleep } from '@sim/utils/helpers' import { generateShortId } from '@sim/utils/id' import { backoffWithJitter, parseRetryAfter } from '@sim/utils/retry' import { JWT } from 'google-auth-library' @@ -10,7 +11,6 @@ import { type ParsedServiceAccount, parseServiceAccount, refineServiceAccountJson, - sleepUntilAborted, } from '@/lib/data-drains/destinations/utils' import type { DrainDestination } from '@/lib/data-drains/types' @@ -155,7 +155,7 @@ async function fetchWithRetry(input: RetryRequestInput): Promise { error: toError(error).message, }) if (attempt < MAX_ATTEMPTS) { - await sleepUntilAborted(backoffWithJitter(attempt, null), input.signal) + await interruptibleSleep(backoffWithJitter(attempt, null), input.signal) continue } throw error @@ -180,7 +180,7 @@ async function fetchWithRetry(input: RetryRequestInput): Promise { const retryAfterMs = parseRetryAfter(response.headers.get('retry-after')) /** Drain the retryable response body so undici can return the socket to the keep-alive pool. */ await response.text().catch(() => '') - await sleepUntilAborted(backoffWithJitter(attempt, retryAfterMs), input.signal) + await interruptibleSleep(backoffWithJitter(attempt, retryAfterMs), input.signal) } throw lastError instanceof Error ? lastError diff --git a/apps/sim/lib/data-drains/destinations/snowflake.ts b/apps/sim/lib/data-drains/destinations/snowflake.ts index 23a8a5665dc..ab43eeac1e1 100644 --- a/apps/sim/lib/data-drains/destinations/snowflake.ts +++ b/apps/sim/lib/data-drains/destinations/snowflake.ts @@ -1,11 +1,11 @@ import { createHash, createPublicKey } from 'node:crypto' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { interruptibleSleep } from '@sim/utils/helpers' import { generateId } from '@sim/utils/id' import { backoffWithJitter, parseRetryAfter } from '@sim/utils/retry' import { importPKCS8, SignJWT } from 'jose' import { z } from 'zod' -import { sleepUntilAborted } from '@/lib/data-drains/destinations/utils' import type { DrainDestination } from '@/lib/data-drains/types' const logger = createLogger('DataDrainSnowflakeDestination') @@ -264,7 +264,7 @@ async function executeStatement(input: ExecuteInput): Promise { error: toError(error).message, }) if (input.signal.aborted || attempt === EXECUTE_MAX_ATTEMPTS) throw error - await sleepUntilAborted( + await interruptibleSleep( backoffWithJitter(attempt, null, { baseMs: EXECUTE_RETRY_BASE_DELAY_MS, maxMs: EXECUTE_RETRY_MAX_DELAY_MS, @@ -318,7 +318,7 @@ async function executeStatement(input: ExecuteInput): Promise { status: response.status, delayMs: delay, }) - await sleepUntilAborted(delay, input.signal) + await interruptibleSleep(delay, input.signal) } throw lastError ?? new Error('Snowflake request failed after retries') } @@ -341,7 +341,7 @@ async function pollStatement(input: PollInput): Promise { while (Date.now() < deadline) { if (input.signal.aborted) throw input.signal.reason ?? new Error('Aborted') if (!skipIntervalSleep) { - await sleepUntilAborted(interval, input.signal) + await interruptibleSleep(interval, input.signal) } skipIntervalSleep = false const jwt = await input.getJwt() @@ -369,7 +369,7 @@ async function pollStatement(input: PollInput): Promise { delayMs: delay, error: toError(error).message, }) - await sleepUntilAborted(delay, input.signal) + await interruptibleSleep(delay, input.signal) skipIntervalSleep = true continue } @@ -401,7 +401,7 @@ async function pollStatement(input: PollInput): Promise { }) /** Drain the body so undici can return the socket to the keep-alive pool between retries. */ await response.text().catch(() => '') - await sleepUntilAborted(delay, input.signal) + await interruptibleSleep(delay, input.signal) skipIntervalSleep = true continue } diff --git a/apps/sim/lib/data-drains/destinations/utils.ts b/apps/sim/lib/data-drains/destinations/utils.ts index eb220a04398..b69e1f6ba6a 100644 --- a/apps/sim/lib/data-drains/destinations/utils.ts +++ b/apps/sim/lib/data-drains/destinations/utils.ts @@ -1,26 +1,6 @@ import { toError } from '@sim/utils/errors' import { z } from 'zod' -/** - * Sleep for `ms` milliseconds, resolving early if `signal` aborts. Used by - * destination retry/poll loops so cancelled drain runs do not hang waiting on - * a `setTimeout` that ignores the abort signal. - */ -export function sleepUntilAborted(ms: number, signal: AbortSignal): Promise { - if (signal.aborted) return Promise.resolve() - return new Promise((resolve) => { - const onAbort = () => { - clearTimeout(timeoutId) - resolve() - } - const timeoutId = setTimeout(() => { - signal.removeEventListener('abort', onAbort) - resolve() - }, ms) - signal.addEventListener('abort', onAbort, { once: true }) - }) -} - export function normalizePrefix(raw: string | undefined): string { if (!raw) return '' const trimmed = raw.replace(/^\/+/, '').replace(/\/+$/, '') diff --git a/apps/sim/lib/data-drains/destinations/webhook.ts b/apps/sim/lib/data-drains/destinations/webhook.ts index 54c480f6dd0..0a61d4a4114 100644 --- a/apps/sim/lib/data-drains/destinations/webhook.ts +++ b/apps/sim/lib/data-drains/destinations/webhook.ts @@ -1,6 +1,7 @@ import { createHmac } from 'node:crypto' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { interruptibleSleep } from '@sim/utils/helpers' import { backoffWithJitter, parseRetryAfter } from '@sim/utils/retry' import { z } from 'zod' import { validateExternalUrl } from '@/lib/core/security/input-validation' @@ -8,7 +9,6 @@ import { secureFetchWithPinnedIP, validateUrlWithDNS, } from '@/lib/core/security/input-validation.server' -import { sleepUntilAborted } from '@/lib/data-drains/destinations/utils' import type { DeliveryMetadata, DrainDestination } from '@/lib/data-drains/types' const logger = createLogger('DataDrainWebhookDestination') @@ -235,7 +235,7 @@ export const webhookDestination: DrainDestination< retryAfterMs = parseRetryAfter(response.headers.get('retry-after')) } if (attempt < MAX_ATTEMPTS) { - await sleepUntilAborted(backoffWithJitter(attempt, retryAfterMs), signal) + await interruptibleSleep(backoffWithJitter(attempt, retryAfterMs), signal) } } throw lastError instanceof Error diff --git a/apps/sim/lib/logs/log-views.test.ts b/apps/sim/lib/logs/log-views.test.ts index 740fa1fd425..50771da77f2 100644 --- a/apps/sim/lib/logs/log-views.test.ts +++ b/apps/sim/lib/logs/log-views.test.ts @@ -1,51 +1,7 @@ -import { - executionPayloadStoreMock, - executionPayloadStoreMockFns, -} from '@sim/testing/mocks/execution-payload-store.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const { - isLargeArrayManifestMock, - isLargeValueRefMock, - readLargeArrayManifestSliceMock, - materializeLargeArrayManifestMock, -} = vi.hoisted(() => ({ - isLargeArrayManifestMock: vi.fn(), - isLargeValueRefMock: vi.fn(), - readLargeArrayManifestSliceMock: vi.fn(), - materializeLargeArrayManifestMock: vi.fn(), -})) - -vi.mock('@/lib/execution/payloads/large-array-manifest-metadata', () => ({ - isLargeArrayManifest: isLargeArrayManifestMock, -})) -vi.mock('@/lib/execution/payloads/large-value-ref', () => ({ - isLargeValueRef: isLargeValueRefMock, -})) -vi.mock('@/lib/execution/payloads/large-array-manifest', () => ({ - readLargeArrayManifestSlice: readLargeArrayManifestSliceMock, - materializeLargeArrayManifest: materializeLargeArrayManifestMock, -})) -vi.mock('@/lib/execution/payloads/store', () => executionPayloadStoreMock) - -import { sleep } from '@sim/utils/helpers' +import { describe, expect, it } from 'vitest' import type { TraceSpan } from '@/lib/logs/types' -import { grepSpans, type LogViewContext, toFull, toOverview, toTrace } from './log-views' +import { toOverview } from './log-views' -const materializeLargeValueRefMock = executionPayloadStoreMockFns.mockMaterializeLargeValueRef - -const ctx: LogViewContext = { - workspaceId: 'ws-1', - workflowId: 'wf-1', - executionId: 'exec-1', -} - -// Fixture helpers — the mocked type guards key off `__sim`. -const manifest = (totalCount: number, preview: unknown[] = []) => ({ - __sim: 'manifest', - totalCount, - preview, -}) const ref = (preview: unknown) => ({ __sim: 'ref', preview, size: 100 }) function span(overrides: Partial = {}): TraceSpan { @@ -60,11 +16,6 @@ function span(overrides: Partial = {}): TraceSpan { } as TraceSpan } -beforeEach(() => { - isLargeArrayManifestMock.mockImplementation((v: any) => v?.__sim === 'manifest') - isLargeValueRefMock.mockImplementation((v: any) => v?.__sim === 'ref') -}) - describe('toOverview', () => { it('keeps timing/cost/hierarchy and omits input/output without materializing refs', () => { const spans: TraceSpan[] = [ @@ -89,211 +40,5 @@ describe('toOverview', () => { expect(out[0]).not.toHaveProperty('input') expect(out[0]).not.toHaveProperty('output') expect(out[0].children?.[0]).toMatchObject({ id: 'child', name: 'Tool' }) - expect(materializeLargeArrayManifestMock).not.toHaveBeenCalled() - expect(materializeLargeValueRefMock).not.toHaveBeenCalled() - }) -}) - -describe('toFull', () => { - it('block scoping returns only the selected subtree', async () => { - const spans: TraceSpan[] = [ - span({ id: 's1', blockId: 'blk-a', name: 'A' }), - span({ id: 's2', blockId: 'blk-b', name: 'B', output: { keep: true } }), - ] - const out = await toFull(spans, ctx, { blockId: 'blk-b' }) - expect(out).toHaveLength(1) - expect(out[0]).toMatchObject({ blockId: 'blk-b', output: { keep: true } }) - }) - - it('materializes a large-array manifest field', async () => { - materializeLargeArrayManifestMock.mockResolvedValue([1, 2, 3]) - const out = await toFull([span({ output: manifest(3) as any })], ctx) - expect(out[0].output).toEqual([1, 2, 3]) - expect(materializeLargeArrayManifestMock).toHaveBeenCalledTimes(1) - }) - - it('falls back to ref preview when a single ref is unavailable', async () => { - materializeLargeValueRefMock.mockResolvedValue(undefined) - const out = await toFull([span({ output: ref('the-preview') as any })], ctx) - expect(out[0].output).toBe('the-preview') - }) -}) - -describe('grepSpans', () => { - it('matches inline output text and error text', async () => { - const spans = [ - span({ output: { msg: 'request timeout occurred' }, errorMessage: 'boom failure' }), - ] - - const outMatch = await grepSpans(spans, 'timeout', ctx) - expect(outMatch.matches.some((m) => m.field === 'output')).toBe(true) - expect(outMatch.truncated).toBe(false) - - const errMatch = await grepSpans(spans, 'boom', ctx) - expect(errMatch.matches.some((m) => m.field === 'error')).toBe(true) - }) - - it('streams a large-array manifest slice-by-slice with advancing offsets', async () => { - // totalCount 500, batch 200 → starts 0, 200, 400 (3 slices). Needle in slice 3. - readLargeArrayManifestSliceMock.mockImplementation(async (_m: unknown, start: number) => { - if (start === 400) return [{ v: 'found the needle here' }] - return [{ v: 'nothing' }] - }) - const spans = [span({ output: manifest(500) as any })] - - const result = await grepSpans(spans, 'needle', ctx) - - expect(result.matches.some((m) => m.field === 'output')).toBe(true) - const starts = readLargeArrayManifestSliceMock.mock.calls.map((c) => c[1]) - expect(starts).toEqual([0, 200, 400]) - }) - - it('caps matches and marks truncated', async () => { - const spans = [span({ id: 'a', name: 'needle one', output: { v: 'needle two' } })] - const result = await grepSpans(spans, 'needle', ctx, { maxMatches: 1 }) - expect(result.matches).toHaveLength(1) - expect(result.truncated).toBe(true) - }) - - it('falls back to ref preview when ref access is rejected (no throw)', async () => { - materializeLargeValueRefMock.mockRejectedValue(new Error('not available in this execution')) - const spans = [span({ output: ref('secret-token') as any })] - const result = await grepSpans(spans, 'secret-token', ctx) - expect(result.matches.some((m) => m.field === 'output')).toBe(true) - }) - - it.each([ - ['character class', 'status=\\d+'], - ['anchor', '^Agent'], - ['alternation', '(openai|anthropic)'], - ['word boundary', '\\bstatus\\b'], - ['bounded quantifier', '\\d{4}-\\d{2}-\\d{2}'], - ['wildcard', 'called.*status'], - ])('interprets %s regex syntax', async (_label, pattern) => { - // None of these patterns occur literally in the span, so a match proves the - // regex was interpreted. `^Agent` anchors against the name field, the rest - // against output — hence the field-agnostic assertion. - const spans = [span({ output: { v: 'called api.openai.com -> status=503 on 2026-01-01' } })] - const result = await grepSpans(spans, pattern, ctx) - expect(result.matches.length).toBeGreaterThan(0) - expect(result.patternNotice).toBeUndefined() - }) - - it('falls back to a literal, with a notice, for syntax RE2 does not implement', async () => { - const spans = [span({ output: { v: 'id: abc and (?=x) literally here' } })] - - const lookahead = await grepSpans(spans, '(?=x)', ctx) - expect(lookahead.matches.some((m) => m.field === 'output')).toBe(true) - expect(lookahead.patternNotice).toContain('RE2') - - // Unbalanced paren is invalid in both engines; still degrades to a literal. - const invalid = await grepSpans([span({ output: { v: 'value a(b' } })], '(', ctx) - expect(invalid.matches.some((m) => m.field === 'output')).toBe(true) - expect(invalid.patternNotice).toContain('RE2') - }) - - it.each([ - ['nested quantifier', '(a+)+$'], - ['duplicate alternation, passes safe-regex2', '(a|a)*b'], - ['adjacent quantifiers, passes every structural screen', 'a*a*b'], - ])('runs a catastrophic pattern in linear time (%s)', async (_label, pattern) => { - // Each blocks the event loop for minutes on a backtracking engine: - // `a*a*b` measured 213s on JSC / 132s on V8 over a 10k-character run. - // RE2 has no backtracking, so these are matched normally and stay fast. - const spans = [span({ output: { v: `${'a'.repeat(10000)}!` } })] - - const start = Date.now() - const result = await grepSpans(spans, pattern, ctx) - const elapsedMs = Date.now() - start - - expect(elapsedMs).toBeLessThan(1000) - expect(result.truncated).toBe(false) - }) - - it('stops scanning and marks truncated once the character budget is exhausted', async () => { - const spans = [ - span({ id: 'a', output: { v: 'x'.repeat(400) } }), - span({ id: 'b', output: { v: 'needle' } }), - ] - const result = await grepSpans(spans, 'needle', ctx, { maxScannedChars: 100 }) - expect(result.matches).toEqual([]) - expect(result.truncated).toBe(true) - }) - - it('stops scanning and marks truncated once the match-time budget is exhausted', async () => { - const spans = [span({ output: { v: 'needle' } })] - const result = await grepSpans(spans, 'needle', ctx, { matchTimeBudgetMs: 0 }) - expect(result.matches).toEqual([]) - expect(result.truncated).toBe(true) - }) - - it('does not charge blob-store I/O to the match-time budget', async () => { - // Each slice read sleeps well past the budget: only time spent matching - // counts, so a slow-but-legitimate grep must still return complete results. - readLargeArrayManifestSliceMock.mockImplementation(async (_m: unknown, start: number) => { - await sleep(30) - return start === 400 ? [{ v: 'found the needle here' }] : [{ v: 'nothing' }] - }) - const spans = [span({ output: manifest(500) as any })] - - const result = await grepSpans(spans, 'needle', ctx, { matchTimeBudgetMs: 50 }) - - expect(result.matches.some((m) => m.field === 'output')).toBe(true) - expect(result.truncated).toBe(false) - }) -}) - -describe('toTrace', () => { - it('collapses loop iterations into one per-block digest line with status counts', () => { - const spans: TraceSpan[] = [ - span({ - id: 'loop', - blockId: 'blk-loop', - name: 'Loop', - type: 'loop', - children: [ - span({ id: 'i1', blockId: 'blk-agent', name: 'Agent', status: 'success', duration: 10 }), - span({ id: 'i2', blockId: 'blk-agent', name: 'Agent', status: 'success', duration: 20 }), - span({ id: 'i3', blockId: 'blk-agent', name: 'Agent', status: 'error', duration: 5 }), - ], - }), - ] - - const digest = toTrace(spans) - - expect(digest).toHaveLength(2) - expect(digest[1]).toMatchObject({ - blockId: 'blk-agent', - name: 'Agent', - executions: 3, - statuses: { success: 2, error: 1 }, - totalDurationMs: 35, - }) - }) -}) - -describe('toFull field projection', () => { - it('narrows spans to whole payload keys', async () => { - const out = await toFull( - [span({ input: { a: 1 }, output: { b: 2 }, errorMessage: 'boom' })], - ctx, - undefined, - ['output', 'error'] - ) - expect(out[0]).toMatchObject({ output: { b: 2 }, error: 'boom' }) - expect(out[0]).not.toHaveProperty('input') - }) - - it('extracts dotted paths under selected', async () => { - const out = await toFull( - [span({ output: { result: { rows: [1, 2, 3], meta: 'big' } } })], - ctx, - undefined, - ['output.result.rows'] - ) - expect(out[0]).not.toHaveProperty('output') - expect((out[0] as { selected?: Record }).selected).toEqual({ - 'output.result.rows': [1, 2, 3], - }) }) }) diff --git a/apps/sim/lib/logs/log-views.ts b/apps/sim/lib/logs/log-views.ts index 07c11fdb737..3bc6657a99e 100644 --- a/apps/sim/lib/logs/log-views.ts +++ b/apps/sim/lib/logs/log-views.ts @@ -1,50 +1,5 @@ -import { compileLinearRegex, isPlainText, literalRegex } from '@/lib/core/security/linear-regex' -import { - materializeLargeArrayManifest, - readLargeArrayManifestSlice, -} from '@/lib/execution/payloads/large-array-manifest' -import { isLargeArrayManifest } from '@/lib/execution/payloads/large-array-manifest-metadata' -import { isLargeValueRef } from '@/lib/execution/payloads/large-value-ref' -import type { LargeValueStoreContext } from '@/lib/execution/payloads/store' -import { materializeLargeValueRef } from '@/lib/execution/payloads/store' import type { TraceSpan } from '@/lib/logs/types' -/** - * Access/materialization context for resolving large-value refs embedded in a - * trace. Built once per request (by the caller) from the fetched execution log. - */ -export type LogViewContext = LargeValueStoreContext - -/** Cap a single (non-array) large-value ref materialization. */ -const SINGLE_REF_MAX_BYTES = 4 * 1024 * 1024 -/** Items per large-array slice while streaming a grep. */ -const ARRAY_SLICE_BATCH = 200 - -const DEFAULT_MAX_MATCHES = 50 -const DEFAULT_MAX_SNIPPET_CHARS = 500 -const DEFAULT_MAX_SLICES_SCANNED = 200 - -/** - * Cumulative time the pattern itself may spend matching, across all spans/fields. - * - * Deliberately counts only time spent matching, not the grep's wall clock: the - * scan awaits blob-store reads (array slices, large-value refs) between matches, - * and charging that I/O to the budget would truncate slow-but-legitimate greps - * under load. Matching is the only part that occupies the event loop, so it is - * the only part worth bounding. - * - * RE2JS trades throughput for its linear-time guarantee — roughly 100x slower - * than the built-in engine, ~25ms per megabyte — so on a very large trace this - * budget is what actually caps the scan rather than a formality. - */ -const DEFAULT_MATCH_TIME_BUDGET_MS = 5_000 -/** - * Total characters a single grep may run the pattern over. Bounds the work one - * request can demand across every span and slice; set well above any realistic - * trace so normal greps never trip it. - */ -const DEFAULT_MAX_SCANNED_CHARS = 64 * 1024 * 1024 - /** Block tree with timing and cost, without input/output. */ export interface OverviewSpan { id: string @@ -73,457 +28,3 @@ export function toOverview(spans: TraceSpan[]): OverviewSpan[] { return node }) } - -/** Condensed per-block digest: names, statuses, counts. */ -export interface TraceDigestEntry { - /** Block id when the spans carry one; the drill-in key for `full` blockIds. */ - blockId?: string - name: string - type: string - /** How many spans (loop iterations included) this block produced. */ - executions: number - /** Span count per status, e.g. { success: 498, error: 2 }. */ - statuses: Record - totalDurationMs: number -} - -/** - * Project trace spans to a flat per-block digest in first-execution order. - * Every span in the tree is counted (loop iterations collapse into their - * block's entry), so a 500-iteration loop is one line, not 500. Never - * materializes refs. - */ -export function toTrace(spans: TraceSpan[]): TraceDigestEntry[] { - const byKey = new Map() - const walk = (list: TraceSpan[]): void => { - for (const s of list) { - const key = s.blockId ?? `${s.type}:${s.name}` - let entry = byKey.get(key) - if (!entry) { - entry = { - ...(s.blockId ? { blockId: s.blockId } : {}), - name: s.name, - type: s.type, - executions: 0, - statuses: {}, - totalDurationMs: 0, - } - byKey.set(key, entry) - } - entry.executions++ - const status = s.status ?? 'unknown' - entry.statuses[status] = (entry.statuses[status] ?? 0) + 1 - entry.totalDurationMs += s.duration ?? 0 - if (s.children && s.children.length > 0) walk(s.children) - } - } - walk(spans) - return Array.from(byKey.values()) -} - -/** Block tree with materialized input/output. */ -export interface FullSpan extends OverviewSpan { - startTime?: string - endTime?: string - input?: unknown - output?: unknown - error?: string - children?: FullSpan[] -} - -export interface BlockSelector { - blockId?: string - /** Multiple drill-in targets at once (ids from the trace digest). */ - blockIds?: string[] - blockName?: string -} - -/** - * Project trace spans to full detail, materializing large-value refs in - * input/output. When a `selector` is given, only the matching span subtree(s) - * are returned (and materialized), so a single block's I/O is loaded instead of - * the whole trace. - */ -export async function toFull( - spans: TraceSpan[], - ctx: LogViewContext, - selector?: BlockSelector, - fields?: string[] -): Promise { - const roots = selectSpans(spans, selector) - const full = await Promise.all(roots.map((s) => fullSpan(s, ctx))) - if (!fields || fields.length === 0) return full - return full.map((s) => projectSpanFields(s, fields)) -} - -/** - * Narrows a full span to the requested fields so the caller loads only what it - * needs. A field is either a whole payload key (`input` / `output` / `error`) - * or a dotted path into one (`output.result.rows`); dotted selections land - * under `selected` keyed by the full path. Span identity/status/timing always - * stay, and children are projected recursively. - */ -function projectSpanFields(span: FullSpan, fields: string[]): FullSpan { - const node: FullSpan = { - id: span.id, - blockId: span.blockId, - name: span.name, - type: span.type, - status: span.status, - durationMs: span.durationMs, - startTime: span.startTime, - endTime: span.endTime, - } - if (span.cost) node.cost = span.cost - const selected: Record = {} - let hasSelected = false - for (const field of fields) { - if (field === 'input' || field === 'output' || field === 'error') { - if (span[field] !== undefined) node[field] = span[field] as never - continue - } - const [head, ...rest] = field.split('.') - if ((head === 'input' || head === 'output') && rest.length > 0) { - let value: unknown = span[head] - for (const key of rest) { - if (value && typeof value === 'object' && !Array.isArray(value)) { - value = (value as Record)[key] - } else if (Array.isArray(value) && /^\d+$/.test(key)) { - value = value[Number(key)] - } else { - value = undefined - break - } - } - selected[field] = value - hasSelected = true - } - } - if (hasSelected) (node as FullSpan & { selected?: Record }).selected = selected - if (span.children && span.children.length > 0) { - node.children = span.children.map((c) => projectSpanFields(c, fields)) - } - return node -} - -function selectSpans(spans: TraceSpan[], selector?: BlockSelector): TraceSpan[] { - if (!selector || (!selector.blockId && !selector.blockIds?.length && !selector.blockName)) { - return spans - } - const idSet = new Set(selector.blockIds ?? []) - if (selector.blockId !== undefined) idSet.add(selector.blockId) - const out: TraceSpan[] = [] - const walk = (list: TraceSpan[]): void => { - for (const s of list) { - const matches = - (s.blockId !== undefined && idSet.has(s.blockId)) || - (selector.blockName !== undefined && s.name === selector.blockName) - if (matches) { - out.push(s) - } else if (s.children && s.children.length > 0) { - walk(s.children) - } - } - } - walk(spans) - return out -} - -async function fullSpan(s: TraceSpan, ctx: LogViewContext): Promise { - const node: FullSpan = { - id: s.id, - blockId: s.blockId, - name: s.name, - type: s.type, - status: s.status, - durationMs: s.duration ?? 0, - startTime: s.startTime, - endTime: s.endTime, - } - if (s.cost) node.cost = s.cost - if (s.errorMessage) node.error = s.errorMessage - if (s.input !== undefined) node.input = await materializeField(s.input, ctx) - if (s.output !== undefined) node.output = await materializeField(s.output, ctx) - if (s.children && s.children.length > 0) { - node.children = await Promise.all(s.children.map((c) => fullSpan(c, ctx))) - } - return node -} - -/** - * Resolve a span field that may be inline OR a large-value ref/manifest. Falls - * back to the ref `preview` (or a placeholder) when the value is unavailable or - * exceeds caps — never throws. - */ -async function materializeField(value: unknown, ctx: LogViewContext): Promise { - if (isLargeArrayManifest(value)) { - try { - return await materializeLargeArrayManifest(value, ctx) - } catch { - return value.preview ?? '[large array unavailable]' - } - } - if (isLargeValueRef(value)) { - try { - const materialized = await materializeLargeValueRef(value, { - ...ctx, - maxBytes: ctx.maxBytes ?? SINGLE_REF_MAX_BYTES, - }) - return materialized === undefined - ? (value.preview ?? '[large value unavailable]') - : materialized - } catch { - return value.preview ?? '[large value unavailable]' - } - } - return value -} - -// Grep (single execution): stream large refs chunk-by-chunk, release each. - -export interface GrepSpanMatch { - spanId: string - blockId?: string - name: string - field: 'name' | 'type' | 'error' | 'input' | 'output' - snippet: string -} - -export interface GrepSpansResult { - matches: GrepSpanMatch[] - /** - * Whether the scan stopped early — because a budget was exhausted, the slice - * cap was hit, or `maxMatches` was reached. It is a "there may be more" flag, - * not proof that trace was left unread: reaching `maxMatches` on the final - * match sets it even when nothing remained. Treat it as a prompt to narrow - * the pattern, never as a count. - */ - truncated: boolean - /** - * Present only when the pattern used syntax RE2 does not implement and was - * therefore matched literally. The tool catalog cannot warn up front — it is - * generated from a contract in another repository — so the caller is told - * here rather than reading zero matches as "not present in the trace". - */ - patternNotice?: string -} - -export interface GrepSpansOptions { - maxMatches?: number - maxSnippetChars?: number - maxSlicesScanned?: number - maxScannedChars?: number - matchTimeBudgetMs?: number -} - -interface GrepState { - matches: GrepSpanMatch[] - slicesScanned: number - scannedChars: number - matchTimeMs: number - truncated: boolean - maxMatches: number - maxSnippetChars: number - maxSlicesScanned: number - maxScannedChars: number - matchTimeBudgetMs: number - find: FindMatch -} - -/** Index of the first case-insensitive match in `text`, or -1. */ -type FindMatch = (text: string) => number - -/** - * Compile a caller-supplied grep pattern into a matcher that cannot backtrack. - * - * Trace text is attacker-influenced — a workflow can emit arbitrarily long - * uniform runs into its own block outputs — and matching runs synchronously on - * the shared event loop, so a backtracking engine lets one request stall every - * other request on the instance. See `@/lib/core/security/linear-regex` for why - * the engine changed rather than the pattern being screened. - * - * A pattern with no metacharacter takes the built-in engine, which is ~100x - * quicker and identical in meaning when there is nothing to interpret. Syntax - * RE2 cannot represent degrades to a literal with a notice, so the caller knows - * its regex was not applied instead of reading zero matches as "not present". - */ -function compilePattern(pattern: string): { find: FindMatch; notice?: string } { - if (isPlainText(pattern)) return { find: literalRegex(pattern, { ignoreCase: true }).find } - - const compiled = compileLinearRegex(pattern, { ignoreCase: true }) - if (compiled) return { find: compiled.find } - - return { - find: literalRegex(pattern, { ignoreCase: true }).find, - notice: - 'Pattern is not valid RE2 syntax (lookahead, lookbehind and backreferences are unsupported), so it was matched as a literal string. Rewrite it without those constructs to search by regex.', - } -} - -function findTimed(text: string, state: GrepState): number { - const started = performance.now() - try { - return state.find(text) - } finally { - state.matchTimeMs += performance.now() - started - } -} - -function snippetAround(text: string, index: number, state: GrepState): string { - const maxChars = state.maxSnippetChars - const half = Math.floor(maxChars / 2) - const start = Math.max(0, index - half) - const end = Math.min(text.length, start + maxChars) - const prefix = start > 0 ? '…' : '' - const suffix = end < text.length ? '…' : '' - return `${prefix}${text.slice(start, end)}${suffix}` -} - -function done(state: GrepState): boolean { - if (state.truncated || state.matches.length >= state.maxMatches) return true - if (state.matchTimeMs >= state.matchTimeBudgetMs) { - state.truncated = true - return true - } - return false -} - -function recordIfMatch( - text: string, - field: GrepSpanMatch['field'], - span: TraceSpan, - state: GrepState -): void { - if (done(state)) return - if (state.scannedChars + text.length > state.maxScannedChars) { - state.truncated = true - return - } - state.scannedChars += text.length - const index = findTimed(text, state) - if (index < 0) return - state.matches.push({ - spanId: span.id, - blockId: span.blockId, - name: span.name, - field, - snippet: snippetAround(text, index, state), - }) - if (state.matches.length >= state.maxMatches) state.truncated = true -} - -async function grepField( - value: unknown, - field: 'input' | 'output', - span: TraceSpan, - ctx: LogViewContext, - state: GrepState -): Promise { - if (done(state)) return - - if (isLargeArrayManifest(value)) { - let start = 0 - while (start < value.totalCount && !done(state)) { - if (state.slicesScanned >= state.maxSlicesScanned) { - state.truncated = true - break - } - let slice: unknown[] | null - try { - slice = await readLargeArrayManifestSlice(value, start, ARRAY_SLICE_BATCH, ctx) - } catch { - // Unavailable chunk: fall back to the manifest preview once and stop. - recordIfMatch(safeStringify(value.preview), field, span, state) - return - } - state.slicesScanned += 1 - if (slice.length === 0) break - recordIfMatch(safeStringify(slice), field, span, state) - start += ARRAY_SLICE_BATCH - // Release the batch before fetching the next so peak memory ~= one batch. - slice = null - } - return - } - - if (isLargeValueRef(value)) { - let materialized: unknown - try { - materialized = await materializeLargeValueRef(value, { - ...ctx, - maxBytes: ctx.maxBytes ?? SINGLE_REF_MAX_BYTES, - }) - } catch { - materialized = undefined - } - const text = - materialized === undefined ? safeStringify(value.preview) : safeStringify(materialized) - recordIfMatch(text, field, span, state) - return - } - - recordIfMatch(safeStringify(value), field, span, state) -} - -function safeStringify(value: unknown): string { - if (value === undefined || value === null) return '' - if (typeof value === 'string') return value - try { - return JSON.stringify(value) - } catch { - return String(value) - } -} - -/** - * Grep a single execution's trace spans for `pattern`. Inline fields are scanned - * directly; large-array I/O is streamed slice-by-slice (each released before the - * next); single large refs are materialized under a byte cap (falling back to - * the ref preview). Only bounded match snippets are accumulated. - * - * `pattern` is matched by a non-backtracking engine — see `compilePattern` — so - * no pattern can blow up on any input. Two budgets bound total work on top of - * that: a character budget and a cumulative match-time budget. Neither counts - * the blob-store I/O this scan awaits, so a slow-but-legitimate grep is not - * truncated merely for being slow. - */ -export async function grepSpans( - spans: TraceSpan[], - pattern: string, - ctx: LogViewContext, - opts?: GrepSpansOptions -): Promise { - const compiled = compilePattern(pattern) - const state: GrepState = { - matches: [], - slicesScanned: 0, - scannedChars: 0, - matchTimeMs: 0, - truncated: false, - maxMatches: opts?.maxMatches ?? DEFAULT_MAX_MATCHES, - maxSnippetChars: opts?.maxSnippetChars ?? DEFAULT_MAX_SNIPPET_CHARS, - maxSlicesScanned: opts?.maxSlicesScanned ?? DEFAULT_MAX_SLICES_SCANNED, - maxScannedChars: opts?.maxScannedChars ?? DEFAULT_MAX_SCANNED_CHARS, - matchTimeBudgetMs: opts?.matchTimeBudgetMs ?? DEFAULT_MATCH_TIME_BUDGET_MS, - find: compiled.find, - } - - const walk = async (list: TraceSpan[]): Promise => { - for (const span of list) { - if (done(state)) return - recordIfMatch(span.name, 'name', span, state) - recordIfMatch(span.type, 'type', span, state) - if (span.errorMessage) recordIfMatch(span.errorMessage, 'error', span, state) - if (span.input !== undefined) await grepField(span.input, 'input', span, ctx, state) - if (span.output !== undefined) await grepField(span.output, 'output', span, ctx, state) - if (span.children && span.children.length > 0) await walk(span.children) - } - } - - await walk(spans) - return { - matches: state.matches, - truncated: state.truncated, - ...(compiled.notice ? { patternNotice: compiled.notice } : {}), - } -} From 4af24d48b8229a5a531d1e9bacd5c9f159b13a92 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:10:41 -0700 Subject: [PATCH 06/30] chore(cleanup): use @sim/utils/object helpers in internal clients and drop dead vanta input-size --- apps/sim/lib/internal/agiloft/operations.ts | 7 +- apps/sim/lib/internal/asana/client.ts | 18 +-- apps/sim/lib/internal/asana/operations.ts | 33 +++--- .../sim/lib/internal/cloudtrail/operations.ts | 5 +- apps/sim/lib/internal/confluence/client.ts | 14 +-- .../sim/lib/internal/confluence/operations.ts | 97 ++++++++-------- apps/sim/lib/internal/gmail/client.ts | 18 +-- apps/sim/lib/internal/gmail/mail.ts | 7 +- apps/sim/lib/internal/google-drive/client.ts | 10 +- .../internal/google-drive/operations.test.ts | 2 - apps/sim/lib/internal/jira/operations.ts | 10 +- apps/sim/lib/internal/jsm/assets.ts | 11 +- apps/sim/lib/internal/jsm/client.ts | 16 +-- apps/sim/lib/internal/jsm/forms.ts | 15 +-- apps/sim/lib/internal/jsm/operations.test.ts | 5 - apps/sim/lib/internal/jsm/service-desk.ts | 23 ++-- .../lib/internal/microsoft-teams/client.ts | 8 +- apps/sim/lib/internal/outlook/client.ts | 11 +- apps/sim/lib/internal/slack/client.ts | 9 +- .../sim/lib/internal/vanta/input-size.test.ts | 22 ---- apps/sim/lib/internal/vanta/input-size.ts | 104 ------------------ apps/sim/lib/slack-search/messages.test.ts | 67 ----------- .../lib/workspaces/organization/utils.test.ts | 33 ------ 23 files changed, 130 insertions(+), 415 deletions(-) delete mode 100644 apps/sim/lib/internal/vanta/input-size.test.ts delete mode 100644 apps/sim/lib/internal/vanta/input-size.ts delete mode 100644 apps/sim/lib/slack-search/messages.test.ts delete mode 100644 apps/sim/lib/workspaces/organization/utils.test.ts diff --git a/apps/sim/lib/internal/agiloft/operations.ts b/apps/sim/lib/internal/agiloft/operations.ts index ba25c691692..0879b2f541a 100644 --- a/apps/sim/lib/internal/agiloft/operations.ts +++ b/apps/sim/lib/internal/agiloft/operations.ts @@ -1,5 +1,5 @@ import { toError } from '@sim/utils/errors' -import { filterUndefined } from '@sim/utils/object' +import { filterUndefined, toRecordOrNull } from '@sim/utils/object' import type { AgiloftAsyncStatusBody, AgiloftAttachBody, @@ -97,10 +97,7 @@ export interface AgiloftOperationContext { function parseRecordData(data: string): Record | null { try { - const parsed = JSON.parse(data) - return typeof parsed === 'object' && parsed !== null && !Array.isArray(parsed) - ? (parsed as Record) - : null + return toRecordOrNull(JSON.parse(data)) } catch { return null } diff --git a/apps/sim/lib/internal/asana/client.ts b/apps/sim/lib/internal/asana/client.ts index 2a5f70fde22..d5b8e9b3f74 100644 --- a/apps/sim/lib/internal/asana/client.ts +++ b/apps/sim/lib/internal/asana/client.ts @@ -1,4 +1,4 @@ -import { toArray } from '@sim/utils/object' +import { toArray, toRecord } from '@sim/utils/object' import { readResponseTextWithLimit } from '@/lib/core/utils/stream-limits' import { AsanaOperationError } from '@/lib/internal/asana/errors' @@ -7,21 +7,11 @@ const ASANA_RESPONSE_MAX_BYTES = 10 * 1024 * 1024 export type AsanaJsonObject = Record -export function asObject(value: unknown): AsanaJsonObject { - return value && typeof value === 'object' && !Array.isArray(value) - ? (value as AsanaJsonObject) - : {} -} - -export function asArray(value: unknown): unknown[] { - return toArray(value) -} - function providerErrorMessage(response: Response, text: string): string { let message = `Asana API error: ${response.status} ${response.statusText}` try { - const data = asObject(JSON.parse(text)) - const firstError = asObject(asArray(data.errors)[0]) + const data = toRecord(JSON.parse(text)) + const firstError = toRecord(toArray(data.errors)[0]) if (Object.keys(firstError).length > 0) { const providerMessage = typeof firstError.message === 'string' && firstError.message ? firstError.message : message @@ -75,7 +65,7 @@ export class AsanaClient { details: text, }) } - return asObject(JSON.parse(text)) + return toRecord(JSON.parse(text)) } async empty(path: string, init: RequestInit, signal?: AbortSignal): Promise { diff --git a/apps/sim/lib/internal/asana/operations.ts b/apps/sim/lib/internal/asana/operations.ts index 617f0fec877..b0585729ffe 100644 --- a/apps/sim/lib/internal/asana/operations.ts +++ b/apps/sim/lib/internal/asana/operations.ts @@ -1,3 +1,4 @@ +import { toArray, toRecord } from '@sim/utils/object' import type { AsanaAddCommentBody, AsanaAddFollowersBody, @@ -15,7 +16,7 @@ import type { AsanaUpdateTaskBody, } from '@/lib/api/contracts/tools/asana' import { validateAlphanumericId } from '@/lib/core/security/input-validation' -import { AsanaClient, type AsanaJsonObject, asArray, asObject } from '@/lib/internal/asana/client' +import { AsanaClient, type AsanaJsonObject } from '@/lib/internal/asana/client' import { AsanaOperationError } from '@/lib/internal/asana/errors' const TASK_OPT_FIELDS = @@ -35,7 +36,7 @@ function validateId(value: string, name: string): void { } function dataObject(result: AsanaJsonObject): AsanaJsonObject { - return asObject(result.data) + return toRecord(result.data) } function optionalString(value: unknown): string | undefined { @@ -51,9 +52,9 @@ function optionalBoolean(value: unknown): boolean | undefined { } function taskSummary(value: unknown) { - const task = asObject(value) - const assignee = asObject(task.assignee) - const createdBy = asObject(task.created_by) + const task = toRecord(value) + const assignee = toRecord(task.assignee) + const createdBy = toRecord(task.created_by) return { gid: requiredString(task.gid), resource_type: optionalString(task.resource_type), @@ -124,7 +125,7 @@ export async function executeAsanaAddComment(input: AsanaAddCommentBody, signal? signal ) const story = dataObject(result) - const createdBy = asObject(story.created_by) + const createdBy = toRecord(story.created_by) return { success: true as const, ts: timestamp(), @@ -152,8 +153,8 @@ export async function executeAsanaAddFollowers(input: AsanaAddFollowersBody, sig ts: timestamp(), gid: requiredString(task.gid), name: requiredString(task.name), - followers: asArray(task.followers).map((value) => { - const follower = asObject(value) + followers: toArray(task.followers).map((value) => { + const follower = toRecord(value) return { gid: requiredString(follower.gid), name: requiredString(follower.name) } }), } @@ -255,8 +256,8 @@ export async function executeAsanaGetProjects(input: AsanaGetProjectsBody, signa return { success: true as const, ts: timestamp(), - projects: asArray(result.data).map((value) => { - const project = asObject(value) + projects: toArray(result.data).map((value) => { + const project = toRecord(value) return { gid: requiredString(project.gid), name: requiredString(project.name), @@ -296,7 +297,7 @@ export async function executeAsanaGetTask(input: AsanaGetTaskBody, signal?: Abor return { success: true as const, ts: timestamp(), - tasks: asArray(result.data).map(taskSummary), + tasks: toArray(result.data).map(taskSummary), next_page: result.next_page, } } @@ -311,8 +312,8 @@ export async function executeAsanaListSections(input: AsanaListSectionsBody, sig return { success: true as const, ts: timestamp(), - sections: asArray(result.data).map((value) => { - const section = asObject(value) + sections: toArray(result.data).map((value) => { + const section = toRecord(value) return { gid: requiredString(section.gid), name: requiredString(section.name), @@ -334,8 +335,8 @@ export async function executeAsanaListWorkspaces( return { success: true as const, ts: timestamp(), - workspaces: asArray(result.data).map((value) => { - const workspace = asObject(value) + workspaces: toArray(result.data).map((value) => { + const workspace = toRecord(value) return { gid: requiredString(workspace.gid), name: requiredString(workspace.name), @@ -363,7 +364,7 @@ export async function executeAsanaSearchTasks(input: AsanaSearchTasksBody, signa return { success: true as const, ts: timestamp(), - tasks: asArray(result.data).map(taskSummary), + tasks: toArray(result.data).map(taskSummary), next_page: result.next_page, } } diff --git a/apps/sim/lib/internal/cloudtrail/operations.ts b/apps/sim/lib/internal/cloudtrail/operations.ts index ad169ea7566..c1f93320cd8 100644 --- a/apps/sim/lib/internal/cloudtrail/operations.ts +++ b/apps/sim/lib/internal/cloudtrail/operations.ts @@ -18,6 +18,7 @@ import { type Trail, } from '@aws-sdk/client-cloudtrail' import { createLogger } from '@sim/logger' +import { isRecordLike } from '@sim/utils/object' import type { AwsCloudtrailCancelQueryBody } from '@/lib/api/contracts/tools/aws/cloudtrail-cancel-query' import type { AwsCloudtrailDescribeQueryBody } from '@/lib/api/contracts/tools/aws/cloudtrail-describe-query' import type { AwsCloudtrailDescribeTrailsBody } from '@/lib/api/contracts/tools/aws/cloudtrail-describe-trails' @@ -103,8 +104,8 @@ function parseCloudTrailEvent(raw: string | undefined): { if (!raw) return { cloudTrailEvent: null, cloudTrailEventRaw: null } try { const parsed: unknown = JSON.parse(raw) - if (parsed !== null && typeof parsed === 'object' && !Array.isArray(parsed)) { - return { cloudTrailEvent: parsed as Record, cloudTrailEventRaw: null } + if (isRecordLike(parsed)) { + return { cloudTrailEvent: parsed, cloudTrailEventRaw: null } } } catch { logger.warn('Failed to parse CloudTrailEvent payload; returning the raw string') diff --git a/apps/sim/lib/internal/confluence/client.ts b/apps/sim/lib/internal/confluence/client.ts index fae557a5d88..fb9d5791443 100644 --- a/apps/sim/lib/internal/confluence/client.ts +++ b/apps/sim/lib/internal/confluence/client.ts @@ -1,4 +1,4 @@ -import { toArray, toRecord } from '@sim/utils/object' +import { toRecord } from '@sim/utils/object' import { validateJiraCloudId } from '@/lib/core/security/input-validation' import { MAX_JSON_API_RESPONSE_BYTES } from '@/lib/core/security/input-validation.server' import { @@ -17,17 +17,9 @@ export interface ConfluenceConnectionConfig { export type JsonObject = Record -export function asObject(value: unknown): JsonObject { - return toRecord(value) -} - -export function asArray(value: unknown): unknown[] { - return toArray(value) -} - export function nested(object: JsonObject, ...keys: string[]): unknown { let value: unknown = object - for (const key of keys) value = asObject(value)[key] + for (const key of keys) value = toRecord(value)[key] return value } @@ -121,7 +113,7 @@ export async function readConfluenceResponseObject( label = 'Confluence response' ): Promise { const text = await readConfluenceResponseText(response, signal, label) - return text ? asObject(JSON.parse(text)) : {} + return text ? toRecord(JSON.parse(text)) : {} } export async function throwConfluenceResponseError( diff --git a/apps/sim/lib/internal/confluence/operations.ts b/apps/sim/lib/internal/confluence/operations.ts index 824f529b244..2faa2a9c034 100644 --- a/apps/sim/lib/internal/confluence/operations.ts +++ b/apps/sim/lib/internal/confluence/operations.ts @@ -1,5 +1,6 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' +import { toArray, toRecord } from '@sim/utils/object' import type { ConfluenceBlogPostOperationBody, ConfluenceCreateCommentBody, @@ -50,8 +51,6 @@ import { } from '@/lib/core/security/input-validation' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' import { - asArray, - asObject, type ConfluenceClient, createConfluenceClient, type JsonObject, @@ -109,7 +108,7 @@ function cappedLimit(value: string | number): string { } function mappedPage(value: unknown): JsonObject { - const page = asObject(value) + const page = toRecord(value) return { id: page.id, title: page.title, @@ -142,8 +141,8 @@ async function findConfluenceSpaceByKey( for (const status of SPACE_STATUSES) { const query = new URLSearchParams({ keys: spaceKey, limit: '1', status }) const data = await client.json(client.apiV2(`/spaces?${query}`), {}, signal) - const match = asArray(data.results) - .map(asObject) + const match = toArray(data.results) + .map(toRecord) .find((space) => space.key === spaceKey) if (match) return match } @@ -315,8 +314,8 @@ export async function executeConfluenceListAttachments( context.signal ) return { - attachments: asArray(data.results).map((value) => { - const attachment = asObject(value) + attachments: toArray(data.results).map((value) => { + const attachment = toRecord(value) return { id: attachment.id, title: attachment.title, @@ -415,7 +414,7 @@ export async function executeConfluenceUploadAttachment( signal, 'Confluence attachment response' ) - const attachment = asObject(asArray(data.results)[0] || data) + const attachment = toRecord(toArray(data.results)[0] || data) return { attachmentId: attachment.id, title: attachment.title, @@ -437,7 +436,7 @@ export async function executeConfluenceAddLabel( jsonInit('POST', [{ prefix: input.prefix || 'global', name: input.labelName }]), context.signal ) - const label = asObject(asArray(data.results)[0] || asArray(data)[0] || data) + const label = toRecord(toArray(data.results)[0] || toArray(data)[0] || data) return { id: label.id ?? '', name: label.name ?? input.labelName, @@ -461,8 +460,8 @@ export async function executeConfluenceListLabels( context.signal ) return { - labels: asArray(data.results).map((value) => { - const label = asObject(value) + labels: toArray(data.results).map((value) => { + const label = toRecord(value) return { id: label.id, name: label.name, prefix: label.prefix || 'global' } }), nextCursor: nextCursor(data), @@ -518,8 +517,8 @@ export async function executeConfluenceListComments( context.signal ) return { - comments: asArray(data.results).map((value) => { - const comment = asObject(value) + comments: toArray(data.results).map((value) => { + const comment = toRecord(value) return { id: comment.id, body: { @@ -621,8 +620,8 @@ export async function executeConfluenceListPageProperties( context.signal ) return { - properties: asArray(data.results).map((value) => { - const property = asObject(value) + properties: toArray(data.results).map((value) => { + const property = toRecord(value) return { id: property.id, key: property.key, @@ -681,8 +680,8 @@ export async function executeConfluenceGetPageAncestors( context.signal ) return { - ancestors: asArray(data.results).map((value) => { - const page = asObject(value) + ancestors: toArray(data.results).map((value) => { + const page = toRecord(value) return { id: page.id, title: page.title, @@ -709,8 +708,8 @@ export async function executeConfluenceGetPageChildren( context.signal ) return { - children: asArray(data.results).map((value) => { - const page = asObject(value) + children: toArray(data.results).map((value) => { + const page = toRecord(value) return { id: page.id, title: page.title, @@ -740,8 +739,8 @@ export async function executeConfluenceGetPageDescendants( context.signal ) return { - descendants: asArray(data.results).map((value) => { - const page = asObject(value) + descendants: toArray(data.results).map((value) => { + const page = toRecord(value) return { id: page.id, title: page.title, @@ -830,8 +829,8 @@ export async function executeConfluencePageVersions( context.signal ) return { - versions: asArray(data.results).map((value) => { - const version = asObject(value) + versions: toArray(data.results).map((value) => { + const version = toRecord(value) return { number: version.number, message: version.message ?? null, @@ -859,7 +858,7 @@ export async function executeConfluenceGetPagesByLabel( context.signal ) return { - pages: asArray(data.results).map(mappedPage), + pages: toArray(data.results).map(mappedPage), labelId: input.labelId, nextCursor: nextCursor(data), } @@ -880,11 +879,11 @@ export async function executeConfluenceSearch( }) const data = await client.json(client.rest(`/search?${query}`), {}, context.signal) return { - results: asArray(data.results).map((value) => { - const result = asObject(value) - const content = asObject(result.content) - const globalContainer = asObject(result.resultGlobalContainer) - const contentSpace = asObject(content.space) + results: toArray(data.results).map((value) => { + const result = toRecord(value) + const content = toRecord(result.content) + const globalContainer = toRecord(result.resultGlobalContainer) + const contentSpace = toRecord(content.space) const space = Object.keys(globalContainer).length ? globalContainer : contentSpace return { id: content.id || result.id, @@ -920,9 +919,9 @@ export async function executeConfluenceSearchInSpace( if (input.contentType) cql += ` AND type = "${escapeCql(input.contentType)}"` const query = new URLSearchParams({ cql, limit: cappedLimit(input.limit) }) const data = await client.json(client.rest(`/search?${query}`), {}, context.signal) - const results = asArray(data.results).map((value) => { - const result = asObject(value) - const content = asObject(result.content) + const results = toArray(data.results).map((value) => { + const result = toRecord(value) + const content = toRecord(result.content) return { id: content.id ?? result.id, title: content.title ?? result.title, @@ -952,8 +951,8 @@ export async function executeConfluenceListBlogPostsInSpace( context.signal ) return { - blogPosts: asArray(data.results).map((value) => { - const post = asObject(value) + blogPosts: toArray(data.results).map((value) => { + const post = toRecord(value) return { id: post.id, title: post.title, @@ -986,8 +985,8 @@ export async function executeConfluenceListPagesInSpace( context.signal ) return { - pages: asArray(data.results).map((value) => { - const page = asObject(value) + pages: toArray(data.results).map((value) => { + const page = toRecord(value) return { ...mappedPage(page), body: page.body ?? null } }), nextCursor: nextCursor(data), @@ -1008,8 +1007,8 @@ export async function executeConfluenceListSpaceLabels( context.signal ) return { - labels: asArray(data.results).map((value) => { - const label = asObject(value) + labels: toArray(data.results).map((value) => { + const label = toRecord(value) return { id: label.id, name: label.name, prefix: label.prefix || 'global' } }), spaceId, @@ -1032,8 +1031,8 @@ export async function executeConfluenceListSpacePermissions( context.signal ) return { - permissions: asArray(data.results).map((value) => { - const permission = asObject(value) + permissions: toArray(data.results).map((value) => { + const permission = toRecord(value) return { id: permission.id, principalType: nested(permission, 'principal', 'type') ?? null, @@ -1060,8 +1059,8 @@ export async function executeConfluenceListBlogPosts( if (input.cursor) query.set('cursor', input.cursor) const data = await client.json(client.apiV2(`/blogposts?${query}`), {}, context.signal) return { - blogPosts: asArray(data.results).map((value) => { - const post = asObject(value) + blogPosts: toArray(data.results).map((value) => { + const post = toRecord(value) return { id: post.id, title: post.title, @@ -1246,7 +1245,7 @@ export async function executeConfluenceDeleteSpace( 'Confluence delete space response', 'DELETE' ) - if (text) longTask = asObject(JSON.parse(text)) + if (text) longTask = toRecord(JSON.parse(text)) } catch { context.signal?.throwIfAborted() } @@ -1267,8 +1266,8 @@ export async function executeConfluenceListSpaces( if (input.cursor) query.set('cursor', input.cursor) const data = await client.json(client.apiV2(`/spaces?${query}`), {}, context.signal) return { - spaces: asArray(data.results).map((value) => { - const space = asObject(value) + spaces: toArray(data.results).map((value) => { + const space = toRecord(value) return { id: space.id, name: space.name, @@ -1316,8 +1315,8 @@ export async function executeConfluenceSpaceProperties( if (input.cursor) query.set('cursor', input.cursor) const data = await client.json(`${base}?${query}`, {}, context.signal) return { - properties: asArray(data.results).map((value) => { - const property = asObject(value) + properties: toArray(data.results).map((value) => { + const property = toRecord(value) return { id: property.id, key: property.key, value: property.value ?? null } }), spaceId, @@ -1326,7 +1325,7 @@ export async function executeConfluenceSpaceProperties( } function mapTask(value: unknown): JsonObject { - const task = asObject(value) + const task = toRecord(value) return { id: task.id, localId: task.localId ?? null, @@ -1390,7 +1389,7 @@ export async function executeConfluenceTasks( query.set('assigned-to', input.assignedTo) } const data = await client.json(client.apiV2(`/tasks?${query}`), {}, context.signal) - return { tasks: asArray(data.results).map(mapTask), nextCursor: nextCursor(data) } + return { tasks: toArray(data.results).map(mapTask), nextCursor: nextCursor(data) } } export async function executeConfluenceGetUser( diff --git a/apps/sim/lib/internal/gmail/client.ts b/apps/sim/lib/internal/gmail/client.ts index 0457fb6fb9c..9d77de93ddd 100644 --- a/apps/sim/lib/internal/gmail/client.ts +++ b/apps/sim/lib/internal/gmail/client.ts @@ -1,4 +1,4 @@ -import { toArray } from '@sim/utils/object' +import { toArray, toRecord } from '@sim/utils/object' import { type ReadResponseWithLimitOptions, readResponseJsonWithLimit, @@ -14,17 +14,9 @@ const RESPONSE_LIMIT: ReadResponseWithLimitOptions = { export type JsonObject = Record -export function asObject(value: unknown): JsonObject { - return value && typeof value === 'object' && !Array.isArray(value) ? (value as JsonObject) : {} -} - -export function asArray(value: unknown): unknown[] { - return toArray(value) -} - export function nested(value: unknown, ...keys: string[]): unknown { let current = value - for (const key of keys) current = asObject(current)[key] + for (const key of keys) current = toRecord(current)[key] return current } @@ -57,7 +49,7 @@ export class GmailClient { error: `Gmail API error: ${response.statusText}`, }) } - return asObject(await readResponseJsonWithLimit(response, RESPONSE_LIMIT)) + return toRecord(await readResponseJsonWithLimit(response, RESPONSE_LIMIT)) } async threadingHeaders( @@ -82,8 +74,8 @@ export class GmailClient { await response.body?.cancel().catch(() => {}) return {} } - const data = asObject(await readResponseJsonWithLimit(response, RESPONSE_LIMIT)) - const headers = asArray(nested(data, 'payload', 'headers')).map(asObject) + const data = toRecord(await readResponseJsonWithLimit(response, RESPONSE_LIMIT)) + const headers = toArray(nested(data, 'payload', 'headers')).map(toRecord) const value = (name: string) => { const header = headers.find( (entry) => typeof entry.name === 'string' && entry.name.toLowerCase() === name diff --git a/apps/sim/lib/internal/gmail/mail.ts b/apps/sim/lib/internal/gmail/mail.ts index 859914d4932..ac5c9407f42 100644 --- a/apps/sim/lib/internal/gmail/mail.ts +++ b/apps/sim/lib/internal/gmail/mail.ts @@ -1,12 +1,13 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' +import { toRecord } from '@sim/utils/object' import type { GmailDraftBody, GmailEditDraftBody, GmailSendBody, } from '@/lib/api/contracts/tools/google' import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits' -import { asObject, GmailClient } from '@/lib/internal/gmail/client' +import { GmailClient } from '@/lib/internal/gmail/client' import { GmailOperationError } from '@/lib/internal/gmail/errors' import { docNotReadyMessage, isDocNotReadyError } from '@/lib/uploads/utils/doc-not-ready' import { processFilesToUserFiles } from '@/lib/uploads/utils/file-utils' @@ -178,7 +179,7 @@ export async function executeGmailDraft(input: GmailDraftBody, context: GmailMai { method: 'POST', body: JSON.stringify({ message }) }, context.signal ) - const draftMessage = asObject(data.message) + const draftMessage = toRecord(data.message) return { success: true, output: { @@ -209,7 +210,7 @@ export async function executeGmailEditDraft( { method: 'PUT', body: JSON.stringify({ id: input.draftId, message }) }, context.signal ) - const draftMessage = asObject(data.message) + const draftMessage = toRecord(data.message) return { success: true, output: { diff --git a/apps/sim/lib/internal/google-drive/client.ts b/apps/sim/lib/internal/google-drive/client.ts index 34b2c379b85..ffb0b38fb08 100644 --- a/apps/sim/lib/internal/google-drive/client.ts +++ b/apps/sim/lib/internal/google-drive/client.ts @@ -54,12 +54,8 @@ export async function requestGoogleDrive( export type JsonObject = Record -export function asObject(value: unknown): JsonObject { - return toRecord(value) -} - export async function responseObject(response: SecureFetchResponse): Promise { - return asObject(await response.json()) + return toRecord(await response.json()) } export async function responseErrorObject( @@ -72,7 +68,7 @@ export async function responseErrorObject( label: 'Google Drive error response', signal, }) - return text ? asObject(JSON.parse(text)) : {} + return text ? toRecord(JSON.parse(text)) : {} } catch { signal?.throwIfAborted() return {} @@ -80,6 +76,6 @@ export async function responseErrorObject( } export function googleApiErrorMessage(data: JsonObject, fallback: string): string { - const error = asObject(data.error) + const error = toRecord(data.error) return typeof error.message === 'string' && error.message ? error.message : fallback } diff --git a/apps/sim/lib/internal/google-drive/operations.test.ts b/apps/sim/lib/internal/google-drive/operations.test.ts index 19ff0299475..0490ff9161b 100644 --- a/apps/sim/lib/internal/google-drive/operations.test.ts +++ b/apps/sim/lib/internal/google-drive/operations.test.ts @@ -6,7 +6,6 @@ const mocks = vi.hoisted(() => ({ })) vi.mock('@/lib/internal/google-drive/client', () => ({ - asObject: (value: unknown) => toRecord(value), googleApiErrorMessage: (data: { error?: { message?: string } }, fallback: string) => data.error?.message || fallback, requestGoogleDrive: mocks.request, @@ -17,7 +16,6 @@ vi.mock('@/lib/internal/google-drive/file-input', () => ({ resolveGoogleDriveUploadFile: mocks.resolveFile, })) -import { toRecord } from '@sim/utils/object' import { executeGoogleDriveExport } from '@/lib/internal/google-drive/operations' import { MAX_EXPORT_BYTES } from '@/tools/google_drive/utils' diff --git a/apps/sim/lib/internal/jira/operations.ts b/apps/sim/lib/internal/jira/operations.ts index d88039ecfa2..fcf3566f7b5 100644 --- a/apps/sim/lib/internal/jira/operations.ts +++ b/apps/sim/lib/internal/jira/operations.ts @@ -26,12 +26,8 @@ export interface JiraOperationContext { type JsonObject = Record -function asObject(value: unknown): JsonObject { - return toRecord(value) -} - function parseObject(text: string): JsonObject { - return asObject(JSON.parse(text)) + return toRecord(JSON.parse(text)) } function optionalObject(text: string): JsonObject { @@ -44,7 +40,7 @@ function optionalObject(text: string): JsonObject { } function nestedString(value: unknown, key: string): string | undefined { - const nested = asObject(value)[key] + const nested = toRecord(value)[key] return typeof nested === 'string' ? nested : undefined } @@ -243,7 +239,7 @@ async function throwResponse(response: Response): Promise { } function attachmentObject(value: unknown) { - const object = asObject(value) + const object = toRecord(value) return { id: typeof object.id === 'string' ? object.id : '', filename: typeof object.filename === 'string' ? object.filename : '', diff --git a/apps/sim/lib/internal/jsm/assets.ts b/apps/sim/lib/internal/jsm/assets.ts index d31fb4fb456..124cd90a560 100644 --- a/apps/sim/lib/internal/jsm/assets.ts +++ b/apps/sim/lib/internal/jsm/assets.ts @@ -1,3 +1,4 @@ +import { toArray } from '@sim/utils/object' import type { ContractBody } from '@/lib/api/contracts' import type { jsmCreateObjectContract, @@ -10,7 +11,7 @@ import type { jsmSearchObjectsAqlContract, jsmUpdateObjectContract, } from '@/lib/api/contracts/tools/jsm' -import { asArray, createJsmAssetsClient } from '@/lib/internal/jsm/client' +import { createJsmAssetsClient } from '@/lib/internal/jsm/client' import { mapAssetObject } from '@/tools/jsm/utils' type ListObjectSchemasInput = ContractBody @@ -50,7 +51,7 @@ export async function executeJsmListObjectSchemas( signal, true ) - const values = asArray(data.values) + const values = toArray(data.values) return { success: true, output: { @@ -88,7 +89,7 @@ export async function executeJsmListObjectTypes(input: ListObjectTypesInput, sig true ) const data = Array.isArray(value) ? {} : (value as Record) - const objectTypes = Array.isArray(value) ? value : asArray(data.values) + const objectTypes = Array.isArray(value) ? value : toArray(data.values) return { success: true, output: { ts: new Date().toISOString(), objectTypes, total: objectTypes.length }, @@ -114,7 +115,7 @@ export async function executeJsmGetObjectTypeAttributes( true ) const data = Array.isArray(value) ? {} : (value as Record) - const attributes = Array.isArray(value) ? value : asArray(data.values) + const attributes = Array.isArray(value) ? value : toArray(data.values) return { success: true, output: { ts: new Date().toISOString(), attributes, total: attributes.length }, @@ -137,7 +138,7 @@ export async function executeJsmSearchObjectsAql( if (input.objectTypeId) body.objectTypeId = input.objectTypeId if (input.objectSchemaId) body.objectSchemaId = input.objectSchemaId const data = await client.json(client.assets('/object/aql'), jsonBody('POST', body), signal, true) - const entries = asArray(data.objectEntries) + const entries = toArray(data.objectEntries) return { success: true, output: { diff --git a/apps/sim/lib/internal/jsm/client.ts b/apps/sim/lib/internal/jsm/client.ts index a44193dd634..39ea6730377 100644 --- a/apps/sim/lib/internal/jsm/client.ts +++ b/apps/sim/lib/internal/jsm/client.ts @@ -1,4 +1,4 @@ -import { toArray } from '@sim/utils/object' +import { toRecord } from '@sim/utils/object' import { validateJiraCloudId } from '@/lib/core/security/input-validation' import { JsmOperationError } from '@/lib/internal/jsm/errors' import { getJiraCloudId, parseAtlassianErrorMessage } from '@/tools/jira/utils' @@ -16,17 +16,9 @@ export interface JsmAssetsConnectionConfig extends JsmConnectionConfig { export type JsonObject = Record -export function asObject(value: unknown): JsonObject { - return value && typeof value === 'object' && !Array.isArray(value) ? (value as JsonObject) : {} -} - -export function asArray(value: unknown): unknown[] { - return toArray(value) -} - export function nested(object: JsonObject, ...keys: string[]): unknown { let value: unknown = object - for (const key of keys) value = asObject(value)[key] + for (const key of keys) value = toRecord(value)[key] return value } @@ -94,7 +86,7 @@ export class JsmClient { signal?: AbortSignal, includeProviderDetails = false ): Promise { - return asObject(await this.value(path, init, signal, includeProviderDetails)) + return toRecord(await this.value(path, init, signal, includeProviderDetails)) } async value( @@ -128,7 +120,7 @@ export class JsmClient { const response = await this.fetch(path, init, signal) if (!response.ok) await throwJsmResponseError(response, includeProviderDetails) const text = await response.text() - return text ? asObject(JSON.parse(text)) : {} + return text ? toRecord(JSON.parse(text)) : {} } } diff --git a/apps/sim/lib/internal/jsm/forms.ts b/apps/sim/lib/internal/jsm/forms.ts index 5ef4c6324ee..cc493475b25 100644 --- a/apps/sim/lib/internal/jsm/forms.ts +++ b/apps/sim/lib/internal/jsm/forms.ts @@ -1,3 +1,4 @@ +import { toArray, toRecord } from '@sim/utils/object' import type { JsmAttachFormBody, JsmCopyFormsBody, @@ -14,7 +15,7 @@ import type { JsmSubmitFormBody, } from '@/lib/api/contracts/tools/jsm' import { validateJiraCloudId, validateJiraIssueKey } from '@/lib/core/security/input-validation' -import { asArray, asObject, createJsmClient, nested } from '@/lib/internal/jsm/client' +import { createJsmClient, nested } from '@/lib/internal/jsm/client' import { JsmOperationError } from '@/lib/internal/jsm/errors' type IssueFormInput = @@ -57,15 +58,15 @@ export async function executeJsmGetIssueForms(input: JsmIssueFormsBody, signal?: signal, true ) - const data = asObject(value) - const forms = Array.isArray(value) ? value : asArray(data.values ?? data.forms) + const data = toRecord(value) + const forms = Array.isArray(value) ? value : toArray(data.values ?? data.forms) return { success: true, output: { ts: new Date().toISOString(), issueIdOrKey: input.issueIdOrKey, forms: forms.map((entry) => { - const form = asObject(entry) + const form = toRecord(entry) return { id: form.id ?? null, name: form.name ?? null, @@ -242,15 +243,15 @@ export async function executeJsmGetFormTemplates( signal, true ) - const data = asObject(value) - const templates = Array.isArray(value) ? value : asArray(data.values) + const data = toRecord(value) + const templates = Array.isArray(value) ? value : toArray(data.values) return { success: true, output: { ts: new Date().toISOString(), projectIdOrKey: input.projectIdOrKey, templates: templates.map((entry) => { - const template = asObject(entry) + const template = toRecord(entry) return { id: template.id ?? null, name: template.name ?? null, diff --git a/apps/sim/lib/internal/jsm/operations.test.ts b/apps/sim/lib/internal/jsm/operations.test.ts index 9048bc18035..11d219d331b 100644 --- a/apps/sim/lib/internal/jsm/operations.test.ts +++ b/apps/sim/lib/internal/jsm/operations.test.ts @@ -19,11 +19,6 @@ const mocks = vi.hoisted(() => { }) vi.mock('@/lib/internal/jsm/client', () => ({ - asArray: (value: unknown) => (Array.isArray(value) ? value : []), - asObject: (value: unknown) => - value && typeof value === 'object' && !Array.isArray(value) - ? (value as Record) - : {}, nested: (value: unknown, ...keys: string[]) => { let current = value for (const key of keys) { diff --git a/apps/sim/lib/internal/jsm/service-desk.ts b/apps/sim/lib/internal/jsm/service-desk.ts index 1218e4436a4..97857d93110 100644 --- a/apps/sim/lib/internal/jsm/service-desk.ts +++ b/apps/sim/lib/internal/jsm/service-desk.ts @@ -1,3 +1,4 @@ +import { toArray, toRecord } from '@sim/utils/object' import type { JsmApprovalsBody, JsmCommentBody, @@ -21,7 +22,7 @@ import { validateEnum, validateJiraIssueKey, } from '@/lib/core/security/input-validation' -import { asArray, asObject, createJsmClient } from '@/lib/internal/jsm/client' +import { createJsmClient } from '@/lib/internal/jsm/client' import { JsmOperationError } from '@/lib/internal/jsm/errors' function validateId(value: string, field: string): void { @@ -143,8 +144,8 @@ export async function executeJsmGetRequestTypeFields( requestTypeId: input.requestTypeId, canAddRequestParticipants: data.canAddRequestParticipants ?? false, canRaiseOnBehalfOf: data.canRaiseOnBehalfOf ?? false, - requestTypeFields: asArray(data.requestTypeFields).map((entry) => { - const field = asObject(entry) + requestTypeFields: toArray(data.requestTypeFields).map((entry) => { + const field = toRecord(entry) return { fieldId: field.fieldId ?? null, name: field.name ?? null, @@ -212,7 +213,7 @@ export async function executeJsmGetRequests(input: JsmRequestsBody, signal?: Abo } function currentStatus(data: Record) { - const value = asObject(data.currentStatus) + const value = toRecord(data.currentStatus) return data.currentStatus ? { status: value.status ?? null, @@ -224,7 +225,7 @@ function currentStatus(data: Record) { function reporter(data: Record, includeActive: boolean) { if (!data.reporter) return null - const value = asObject(data.reporter) + const value = toRecord(data.reporter) return { accountId: value.accountId ?? null, displayName: value.displayName ?? null, @@ -315,8 +316,8 @@ export async function executeJsmGetRequest(input: JsmRequestBody, signal?: Abort createdDate: data.createdDate ?? null, currentStatus: currentStatus(data), reporter: reporter(data, true), - requestFieldValues: asArray(data.requestFieldValues).map((entry) => { - const field = asObject(entry) + requestFieldValues: toArray(data.requestFieldValues).map((entry) => { + const field = toRecord(entry) return { fieldId: field.fieldId ?? null, label: field.label ?? null, @@ -338,7 +339,7 @@ export async function executeJsmAddComment(input: JsmCommentBody, signal?: Abort signal, true ) - const author = asObject(data.author) + const author = toRecord(data.author) return { success: true, output: { @@ -472,9 +473,9 @@ export async function executeJsmAnswerApproval(input: JsmApprovalsBody, signal?: name: data.name ?? null, finalDecision: data.finalDecision ?? null, canAnswerApproval: data.canAnswerApproval ?? null, - approvers: asArray(data.approvers).map((entry) => { - const item = asObject(entry) - const approver = asObject(item.approver) + approvers: toArray(data.approvers).map((entry) => { + const item = toRecord(entry) + const approver = toRecord(item.approver) return { approver: { accountId: approver.accountId ?? null, diff --git a/apps/sim/lib/internal/microsoft-teams/client.ts b/apps/sim/lib/internal/microsoft-teams/client.ts index 82ec249010b..e2eed2148f3 100644 --- a/apps/sim/lib/internal/microsoft-teams/client.ts +++ b/apps/sim/lib/internal/microsoft-teams/client.ts @@ -8,12 +8,8 @@ const MICROSOFT_GRAPH_RESPONSE_MAX_BYTES = 2 * 1024 * 1024 export type MicrosoftTeamsGraphObject = Record -function asObject(value: unknown): MicrosoftTeamsGraphObject { - return toRecord(value) -} - function errorMessage(data: MicrosoftTeamsGraphObject, fallback: string): string { - const error = asObject(data.error) + const error = toRecord(data.error) return typeof error.message === 'string' && error.message ? error.message : fallback } @@ -44,7 +40,7 @@ export class MicrosoftTeamsClient { let data: MicrosoftTeamsGraphObject try { - data = text ? asObject(JSON.parse(text)) : {} + data = text ? toRecord(JSON.parse(text)) : {} } catch (error) { if (!response.ok) throw new MicrosoftTeamsOperationError(fallbackError, response.status) throw new Error(getErrorMessage(error, 'Microsoft Graph returned invalid JSON')) diff --git a/apps/sim/lib/internal/outlook/client.ts b/apps/sim/lib/internal/outlook/client.ts index 2b7a2ac7e2b..22427779c73 100644 --- a/apps/sim/lib/internal/outlook/client.ts +++ b/apps/sim/lib/internal/outlook/client.ts @@ -1,4 +1,5 @@ import { getErrorMessage } from '@sim/utils/errors' +import { toRecord } from '@sim/utils/object' import { DEFAULT_MAX_ERROR_BODY_BYTES, readResponseTextWithLimit, @@ -11,19 +12,13 @@ const MICROSOFT_GRAPH_RESPONSE_MAX_BYTES = 10 * 1024 * 1024 export type OutlookJsonObject = Record -export function asObject(value: unknown): OutlookJsonObject { - return value && typeof value === 'object' && !Array.isArray(value) - ? (value as OutlookJsonObject) - : {} -} - function parseJson(text: string): OutlookJsonObject { if (!text) return {} - return asObject(JSON.parse(text)) + return toRecord(JSON.parse(text)) } function graphErrorMessage(data: OutlookJsonObject, fallback: string): string { - const error = asObject(data.error) + const error = toRecord(data.error) return typeof error.message === 'string' && error.message ? error.message : fallback } diff --git a/apps/sim/lib/internal/slack/client.ts b/apps/sim/lib/internal/slack/client.ts index f40bb4b63b0..23700aec6f8 100644 --- a/apps/sim/lib/internal/slack/client.ts +++ b/apps/sim/lib/internal/slack/client.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' import { isPayloadSizeLimitError, readResponseJsonWithLimit } from '@/lib/core/utils/stream-limits' const MAX_SLACK_JSON_BYTES = 2 * 1024 * 1024 @@ -35,10 +36,6 @@ export function postSlackMessage(accessToken: string, message: SlackMessage, sig return requestSlackApi({ accessToken, method: 'chat.postMessage', body: { ...message }, signal }) } -function isSlackJsonObject(value: unknown): value is SlackJsonObject { - return typeof value === 'object' && value !== null && !Array.isArray(value) -} - export function slackString(data: SlackJsonObject, key: string): string | undefined { const value = data[key] return typeof value === 'string' ? value : undefined @@ -46,7 +43,7 @@ export function slackString(data: SlackJsonObject, key: string): string | undefi export function slackObject(data: SlackJsonObject, key: string): SlackJsonObject | undefined { const value = data[key] - return isSlackJsonObject(value) ? value : undefined + return isRecordLike(value) ? value : undefined } export function slackArray(data: SlackJsonObject, key: string): unknown[] | undefined { @@ -103,7 +100,7 @@ export async function requestSlackApi({ } } signal?.throwIfAborted() - if (!isSlackJsonObject(parsed)) throw new Error('Slack API returned an invalid response') + if (!isRecordLike(parsed)) throw new Error('Slack API returned an invalid response') const scopeHeader = response.headers.get('x-oauth-scopes') return { data: parsed, diff --git a/apps/sim/lib/internal/vanta/input-size.test.ts b/apps/sim/lib/internal/vanta/input-size.test.ts deleted file mode 100644 index c6b0ba97573..00000000000 --- a/apps/sim/lib/internal/vanta/input-size.test.ts +++ /dev/null @@ -1,22 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { isJsonInputWithinLimit } from '@/lib/internal/vanta/input-size' - -describe('Vanta operation input sizing', () => { - it.each([ - null, - { value: 'plain text' }, - { value: 'quotes " and slashes \\' }, - { value: 'emoji 🚀 and control\n' }, - { nested: [{ enabled: true }, undefined, 42] }, - ])('matches JSON byte boundaries without materializing the entire input', (input) => { - const bytes = Buffer.byteLength(JSON.stringify(input) ?? '', 'utf8') - expect(isJsonInputWithinLimit(input, bytes)).toBe(true) - if (bytes > 0) expect(isJsonInputWithinLimit(input, bytes - 1)).toBe(false) - }) - - it('rejects cyclic inputs as invalid JSON', () => { - const cyclic: { self?: unknown } = {} - cyclic.self = cyclic - expect(() => isJsonInputWithinLimit(cyclic, 1024)).toThrow(/circular/i) - }) -}) diff --git a/apps/sim/lib/internal/vanta/input-size.ts b/apps/sim/lib/internal/vanta/input-size.ts deleted file mode 100644 index 4e58ccd25e8..00000000000 --- a/apps/sim/lib/internal/vanta/input-size.ts +++ /dev/null @@ -1,104 +0,0 @@ -function jsonStringBytes(value: string): number { - let bytes = 2 - for (let index = 0; index < value.length; index += 1) { - const code = value.charCodeAt(index) - if ( - code === 0x22 || - code === 0x5c || - code === 0x08 || - code === 0x09 || - code === 0x0a || - code === 0x0c || - code === 0x0d - ) { - bytes += 2 - } else if (code <= 0x1f) { - bytes += 6 - } else if (code <= 0x7f) { - bytes += 1 - } else if (code <= 0x7ff) { - bytes += 2 - } else if (code >= 0xd800 && code <= 0xdbff) { - const next = value.charCodeAt(index + 1) - if (next >= 0xdc00 && next <= 0xdfff) { - bytes += 4 - index += 1 - } else { - bytes += 6 - } - } else if (code >= 0xdc00 && code <= 0xdfff) { - bytes += 6 - } else { - bytes += 3 - } - } - return bytes -} - -function primitiveJsonBytes(value: unknown): number | null { - if (value === null) return 4 - switch (typeof value) { - case 'string': - return jsonStringBytes(value) - case 'boolean': - return value ? 4 : 5 - case 'number': - return Number.isFinite(value) ? String(value).length : 4 - case 'bigint': - throw new TypeError('Do not know how to serialize a BigInt') - case 'undefined': - case 'function': - case 'symbol': - return null - default: - return null - } -} - -function addJsonBytes( - value: unknown, - limit: number, - seen: Set, - arrayEntry = false -): number { - const primitiveBytes = primitiveJsonBytes(value) - if (primitiveBytes !== null) return primitiveBytes - if (value === undefined || typeof value === 'function' || typeof value === 'symbol') { - return arrayEntry ? 4 : 0 - } - if (value instanceof Date) return jsonStringBytes(value.toJSON()) - if (!value || typeof value !== 'object') return 0 - if (seen.has(value)) throw new TypeError('Converting circular structure to JSON') - seen.add(value) - - let bytes = 2 - let emitted = false - if (Array.isArray(value)) { - for (const entry of value) { - if (emitted) bytes += 1 - bytes += addJsonBytes(entry, limit - bytes, seen, true) - emitted = true - if (bytes > limit) break - } - } else { - for (const [key, entry] of Object.entries(value)) { - const entryBytes = addJsonBytes(entry, limit - bytes, seen) - if ( - entryBytes === 0 && - (entry === undefined || typeof entry === 'function' || typeof entry === 'symbol') - ) { - continue - } - if (emitted) bytes += 1 - bytes += jsonStringBytes(key) + 1 + entryBytes - emitted = true - if (bytes > limit) break - } - } - seen.delete(value) - return bytes -} - -export function isJsonInputWithinLimit(input: unknown, limit: number): boolean { - return addJsonBytes(input, limit, new Set()) <= limit -} diff --git a/apps/sim/lib/slack-search/messages.test.ts b/apps/sim/lib/slack-search/messages.test.ts deleted file mode 100644 index 218e6b7f791..00000000000 --- a/apps/sim/lib/slack-search/messages.test.ts +++ /dev/null @@ -1,67 +0,0 @@ -import { describe, expect, it } from 'vitest' -import type { KnowledgeSearchItem } from '@/lib/knowledge/application/search' -import { renderSlackSearchResults } from '@/lib/slack-search/messages' - -const message = { - appId: 'A1', - teamId: 'T1', - eventId: 'Ev1', - channelId: 'D1', - userId: 'U1', - query: 'release notes', - queryTooLong: false, -} -function result(id: string, overrides: Partial = {}): KnowledgeSearchItem { - return { - embeddingId: 'e1', - knowledgeBaseId: 'kb1', - documentId: id, - documentName: `Document ${id}`, - sourceUrl: null, - sourceModifiedAt: null, - connectorType: null, - content: 'snippet', - chunkIndex: 0, - metadata: {}, - similarity: 1, - ...overrides, - } -} - -describe('Slack Search result presentation', () => { - it('renders untrusted text as plain text and only uses valid source URLs', () => { - const reply = renderSlackSearchResults( - { ...message, threadTs: '1.2' }, - 'org1', - [ - result('1', { - documentName: ' *hi*', - sourceUrl: 'javascript:alert(1)', - content: 'x'.repeat(500), - }), - ], - 'https://sim.test' - ) - expect(reply.thread_ts).toBe('1.2') - expect(reply.blocks?.[1]).toMatchObject({ - text: { type: 'plain_text' }, - accessory: { url: 'https://sim.test/o/org1/knowledge/kb1/1' }, - }) - expect(JSON.stringify(reply)).not.toContain('javascript:') - expect(JSON.stringify(reply)).not.toContain('x'.repeat(301)) - }) - it('keeps Unicode queries and long provider URLs within Slack button limits', () => { - const reply = renderSlackSearchResults( - { ...message, query: '界'.repeat(2000) }, - 'org1', - [result('1', { sourceUrl: `https://example.com/${'x'.repeat(3000)}` })], - 'https://sim.test' - ) - expect(reply.blocks?.[1]).toMatchObject({ - accessory: { url: 'https://sim.test/o/org1/knowledge/kb1/1' }, - }) - expect(reply.blocks?.[2]).toMatchObject({ - elements: [{ url: 'https://sim.test/o/org1/search' }], - }) - }) -}) diff --git a/apps/sim/lib/workspaces/organization/utils.test.ts b/apps/sim/lib/workspaces/organization/utils.test.ts deleted file mode 100644 index 1467dde1d08..00000000000 --- a/apps/sim/lib/workspaces/organization/utils.test.ts +++ /dev/null @@ -1,33 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { calculateSeatUsage } from '@/lib/workspaces/organization/utils' - -describe('calculateSeatUsage', () => { - it('does not count external pending workspace invitations as occupied seats', () => { - const seats = calculateSeatUsage({ - id: 'org-1', - name: 'Acme', - slug: 'acme', - createdAt: new Date(), - members: [ - { id: 'member-1', role: 'owner' }, - { id: 'member-2', role: 'member' }, - ], - invitations: [ - { - id: 'inv-1', - email: 'internal@example.com', - status: 'pending', - membershipIntent: 'internal', - }, - { - id: 'inv-2', - email: 'external@example.com', - status: 'pending', - membershipIntent: 'external', - }, - ], - }) - - expect(seats).toEqual({ used: 3, members: 2, pending: 1 }) - }) -}) From f39ba4ab59a849c76784361e25a2bd74734413ea Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:10:42 -0700 Subject: [PATCH 07/30] chore(cleanup): remove dead logs types, organization utils, and workspace permission queries --- apps/sim/lib/logs/execution/logger.ts | 3 +- .../lib/logs/execution/snapshot/service.ts | 4 +- apps/sim/lib/logs/types.ts | 206 ------------------ apps/sim/lib/workspaces/organization/types.ts | 133 ----------- apps/sim/lib/workspaces/organization/utils.ts | 42 ---- .../lib/workspaces/permissions/utils.test.ts | 32 --- apps/sim/lib/workspaces/permissions/utils.ts | 89 -------- apps/sim/lib/workspaces/utils.ts | 10 - .../testing/src/mocks/permissions.mock.ts | 4 - .../src/mocks/workspaces-utils.mock.ts | 2 - 10 files changed, 3 insertions(+), 522 deletions(-) diff --git a/apps/sim/lib/logs/execution/logger.ts b/apps/sim/lib/logs/execution/logger.ts index 7313c0fdcc8..43a57faa7f1 100644 --- a/apps/sim/lib/logs/execution/logger.ts +++ b/apps/sim/lib/logs/execution/logger.ts @@ -74,7 +74,6 @@ import type { ExecutionEnvironment, ExecutionFinalizationPath, ExecutionTrigger, - ExecutionLoggerService as IExecutionLoggerService, TraceSpan, WorkflowExecutionLog, WorkflowState, @@ -411,7 +410,7 @@ function recordedFile(file: TraversedFile) { } } -export class ExecutionLogger implements IExecutionLoggerService { +export class ExecutionLogger { private compactExecutionDataForStorage( executionData: ExecutionData, executionId: string diff --git a/apps/sim/lib/logs/execution/snapshot/service.ts b/apps/sim/lib/logs/execution/snapshot/service.ts index 5bc96a9465c..50bbd886878 100644 --- a/apps/sim/lib/logs/execution/snapshot/service.ts +++ b/apps/sim/lib/logs/execution/snapshot/service.ts @@ -6,7 +6,7 @@ import { generateId } from '@sim/utils/id' import { and, eq, inArray, lt, notExists, sql } from 'drizzle-orm' import { LRUCache } from 'lru-cache' import { consumeRowBudget, type RowBudget } from '@/lib/cleanup/batch-delete' -import type { SnapshotService as ISnapshotService, WorkflowState } from '@/lib/logs/types' +import type { WorkflowState } from '@/lib/logs/types' import { normalizedStringify, normalizeWorkflowState } from '@/lib/workflows/comparison' const logger = createLogger('SnapshotService') @@ -37,7 +37,7 @@ export interface ResolvedSnapshot { const snapshotCacheKey = ({ workflowId, stateHash }: Omit) => `${workflowId}:${stateHash}` -export class SnapshotService implements ISnapshotService { +export class SnapshotService { /** * Resolves the snapshot row holding `state` for `workflowId`, creating the row * only when no identical state (same normalized hash) is stored yet. diff --git a/apps/sim/lib/logs/types.ts b/apps/sim/lib/logs/types.ts index a592c1c2d43..7245ff89f5e 100644 --- a/apps/sim/lib/logs/types.ts +++ b/apps/sim/lib/logs/types.ts @@ -47,7 +47,6 @@ export interface ToolCall { error?: string } -export type BlockInputData = Record export type BlockOutputData = NormalizedBlockOutput | null export interface ExecutionEnvironment { @@ -245,9 +244,6 @@ export interface CompletedWorkflowExecutionLog extends WorkflowExecutionLog { persistedStatus: PersistedWorkflowExecutionStatus } -export type WorkflowExecutionLogInsert = Omit -export type WorkflowExecutionLogSelect = WorkflowExecutionLog - export type TokenInfo = BlockTokens export interface ProviderTiming { @@ -364,205 +360,3 @@ export interface TraceSpan { /** For failed child spans: human-readable error message. */ errorMessage?: string } - -export interface WorkflowExecutionSummary { - id: string - workflowId: string - workflowName: string - executionId: string - trigger: ExecutionTrigger['type'] - status: ExecutionStatus['status'] - startedAt: string - endedAt: string - durationMs: number - - costSummary: { - total: number - inputCost: number - outputCost: number - tokens: number - } - stateSnapshotId: string - errorSummary?: { - blockId: string - blockName: string - message: string - } -} - -export interface WorkflowExecutionDetail extends WorkflowExecutionSummary { - environment: ExecutionEnvironment - triggerData: ExecutionTrigger - blockExecutions: BlockExecutionSummary[] - traceSpans: TraceSpan[] - workflowState: WorkflowState -} - -export interface BlockExecutionSummary { - id: string - blockId: string - blockName: string - blockType: string - startedAt: string - endedAt: string - durationMs: number - status: 'success' | 'error' | 'skipped' - errorMessage?: string - cost?: CostBreakdown - inputSummary: { - parameterCount: number - hasComplexData: boolean - } - outputSummary: { - hasOutput: boolean - outputType: string - hasError: boolean - } -} - -export interface PaginatedResponse { - data: T[] - pagination: { - page: number - pageSize: number - total: number - totalPages: number - hasNext: boolean - hasPrevious: boolean - } -} - -export type WorkflowExecutionsResponse = PaginatedResponse -export type BlockExecutionsResponse = PaginatedResponse - -export interface WorkflowExecutionFilters { - workflowIds?: string[] - folderIds?: string[] - triggers?: ExecutionTrigger['type'][] - status?: ExecutionStatus['status'][] - startDate?: string - endDate?: string - search?: string - minDuration?: number - maxDuration?: number - minCost?: number - maxCost?: number - hasErrors?: boolean -} - -export interface PaginationParams { - page: number - pageSize: number - sortBy?: 'startedAt' | 'durationMs' | 'totalCost' | 'blockCount' - sortOrder?: 'asc' | 'desc' -} - -export interface LogsQueryParams extends WorkflowExecutionFilters, PaginationParams { - includeBlockSummary?: boolean - includeWorkflowState?: boolean -} - -export interface LogsError { - code: 'EXECUTION_NOT_FOUND' | 'SNAPSHOT_NOT_FOUND' | 'INVALID_WORKFLOW_STATE' | 'STORAGE_ERROR' - message: string - details?: Record -} - -export interface ValidationError { - field: string - message: string - value: unknown -} - -export class LogsServiceError extends Error { - public code: LogsError['code'] - public details?: Record - - constructor(message: string, code: LogsError['code'], details?: Record) { - super(message) - this.name = 'LogsServiceError' - this.code = code - this.details = details - } -} - -export interface DatabaseOperationResult { - success: boolean - data?: T - error?: LogsServiceError -} - -export interface BatchInsertResult { - inserted: T[] - failed: Array<{ - item: T - error: string - }> - totalAttempted: number - totalSucceeded: number - totalFailed: number -} - -export interface SnapshotService { - resolveSnapshot( - workflowId: string, - state: WorkflowState, - options?: { fresh?: boolean } - ): Promise<{ id: string; workflowId: string; stateHash: string }> - rememberReferencedSnapshot(snapshot: { id: string; workflowId: string; stateHash: string }): void - computeStateHash(state: WorkflowState): string - cleanupOrphanedSnapshots(olderThanDays: number): Promise -} - -export interface ExecutionLoggerService { - loadTraceSpansForProjection(params: { - executionId: string - workflowId: string - workspaceId: string | null - traceSpans: TraceSpan[] - isResume?: boolean - }): Promise - - prepareTraceSpansForProjection(params: { - executionId: string - workflowId: string - workspaceId: string | null - userId?: string | null - traceSpans: TraceSpan[] - }): Promise - - startWorkflowExecution(params: { - workflowId: string - workspaceId: string - executionId: string - trigger: ExecutionTrigger - environment: ExecutionEnvironment - actorUserId?: string | null - billingAttribution?: BillingAttributionSnapshot - workflowState: WorkflowState - }): Promise - - completeWorkflowExecution(params: { - executionId: string - endedAt: string - totalDurationMs: number - - costSummary: { - totalCost: number - totalInputCost: number - totalOutputCost: number - totalTokens: number - } - finalOutput: BlockOutputData - traceSpans?: TraceSpan[] - workflowInput?: any - executionState?: SerializableExecutionState - finalizationPath?: ExecutionFinalizationPath - completionFailure?: string - isResume?: boolean - level?: 'info' | 'error' - status?: 'completed' | 'failed' | 'cancelled' | 'pending' - actorUserId?: string | null - billingAttribution?: BillingAttributionSnapshot - }): Promise -} diff --git a/apps/sim/lib/workspaces/organization/types.ts b/apps/sim/lib/workspaces/organization/types.ts index e1cd65ca27e..fcb091fea8e 100644 --- a/apps/sim/lib/workspaces/organization/types.ts +++ b/apps/sim/lib/workspaces/organization/types.ts @@ -28,136 +28,3 @@ export interface Organization { createdAt: string | Date [key: string]: unknown } - -interface Subscription { - id: string - plan: string - status: string - seats?: number - referenceId: string - cancelAtPeriodEnd?: boolean - periodEnd?: number | Date - trialEnd?: number | Date - metadata?: any - [key: string]: unknown -} - -interface WorkspaceInvitation { - workspaceId: string - permission: string -} - -interface Workspace { - id: string - name: string - ownerId: string - isOwner: boolean - canInvite: boolean -} - -interface OrganizationFormData { - name: string - slug: string - logo: string -} - -interface MemberUsageData { - userId: string - userName: string - userEmail: string - currentUsage: number - usageLimit: number - percentUsed: number - isOverLimit: boolean - role: string - joinedAt: string -} - -interface OrganizationBillingData { - organizationId: string - organizationName: string - subscriptionPlan: string - subscriptionStatus: string - totalSeats: number - usedSeats: number - seatsCount: number - totalCurrentUsage: number - totalUsageLimit: number - minimumBillingAmount: number - averageUsagePerMember: number - billingPeriodStart: string | null - billingPeriodEnd: string | null - members?: MemberUsageData[] - userRole?: string - billingBlocked?: boolean -} - -interface OrganizationState { - // Core organization data - organizations: Organization[] - activeOrganization: Organization | null - - // Team management - subscriptionData: Subscription | null - userWorkspaces: Workspace[] - - // Organization billing and usage - organizationBillingData: OrganizationBillingData | null - - // Organization settings - orgFormData: OrganizationFormData - - // Loading states - isLoading: boolean - isLoadingSubscription: boolean - isLoadingOrgBilling: boolean - isCreatingOrg: boolean - isInviting: boolean - isSavingOrgSettings: boolean - - // Error states - error: string | null - orgSettingsError: string | null - - // Success states - inviteSuccess: boolean - orgSettingsSuccess: string | null - - // Cache timestamps - lastFetched: number | null - lastSubscriptionFetched: number | null - lastOrgBillingFetched: number | null - - // User permissions - hasTeamPlan: boolean - hasEnterprisePlan: boolean -} - -interface OrganizationStore extends OrganizationState { - loadData: () => Promise - loadOrganizationSubscription: (orgId: string) => Promise - loadOrganizationBillingData: (organizationId: string, force?: boolean) => Promise - loadUserWorkspaces: (userId?: string) => Promise - refreshOrganization: () => Promise - - // Organization management - createOrganization: (name: string, slug: string) => Promise - setActiveOrganization: (orgId: string) => Promise - updateOrganizationSettings: () => Promise - - // Team management - inviteMember: (email: string, workspaceInvitations?: WorkspaceInvitation[]) => Promise - removeMember: (memberId: string) => Promise - cancelInvitation: (invitationId: string) => Promise - - transferSubscriptionToOrganization: (orgId: string) => Promise - - getUserRole: (userEmail?: string) => string - isAdminOrOwner: (userEmail?: string) => boolean - getUsedSeats: () => { used: number; members: number; pending: number } - - setOrgFormData: (data: Partial) => void - - clearError: () => void - clearSuccessMessages: () => void -} diff --git a/apps/sim/lib/workspaces/organization/utils.ts b/apps/sim/lib/workspaces/organization/utils.ts index 49c51f03bfe..f81470f3aec 100644 --- a/apps/sim/lib/workspaces/organization/utils.ts +++ b/apps/sim/lib/workspaces/organization/utils.ts @@ -4,8 +4,6 @@ */ import { isOrgAdminRole } from '@sim/platform-authz/predicates' -import { normalizeEmail } from '@sim/utils/string' -import { quickValidateEmail } from '@/lib/messaging/email/validation' import type { Organization } from '@/lib/workspaces/organization/types' /** @@ -33,39 +31,6 @@ export function isAdminOrOwner( return isOrgAdminRole(role) } -/** - * Calculate seat usage for an organization - */ -export function calculateSeatUsage(organization: Organization | null | undefined): { - used: number - members: number - pending: number -} { - if (!organization) { - return { used: 0, members: 0, pending: 0 } - } - - const membersCount = organization.members?.length || 0 - const pendingInvitationsCount = - organization.invitations?.filter( - (inv) => inv.status === 'pending' && inv.membershipIntent !== 'external' - ).length || 0 - - return { - used: membersCount + pendingInvitationsCount, - members: membersCount, - pending: pendingInvitationsCount, - } -} - -/** - * Get used seats from an organization - * Alias for calculateSeatUsage - */ -export function getUsedSeats(organization: Organization | null | undefined) { - return calculateSeatUsage(organization) -} - /** * Generate a URL-friendly slug from a name */ @@ -76,10 +41,3 @@ export function generateSlug(name: string): string { .replace(/-+/g, '-') // Replace consecutive hyphens with single hyphen .replace(/^-|-$/g, '') // Remove leading and trailing hyphens } - -/** - * Validate email format - */ -export function validateEmail(email: string): boolean { - return quickValidateEmail(normalizeEmail(email)).isValid -} diff --git a/apps/sim/lib/workspaces/permissions/utils.test.ts b/apps/sim/lib/workspaces/permissions/utils.test.ts index c346ce5ac4e..185779d3578 100644 --- a/apps/sim/lib/workspaces/permissions/utils.test.ts +++ b/apps/sim/lib/workspaces/permissions/utils.test.ts @@ -2,7 +2,6 @@ import { db } from '@sim/db' import { describe, expect, it, vi } from 'vitest' import { checkWorkspaceAccess, - getManageableWorkspaces, getUserEntityPermissions, getUsersWithPermissions, getWorkspaceWithOwner, @@ -195,37 +194,6 @@ describe('Permission Utils', () => { }) }) - describe('getManageableWorkspaces', () => { - it('should combine owned and admin workspaces without duplicates', async () => { - const mockOwnedWorkspaces = [ - { id: 'ws1', name: 'My Workspace', ownerId: 'user123' }, - { id: 'ws2', name: 'Another Workspace', ownerId: 'user123' }, - ] - const mockAdminWorkspaces = [ - { id: 'ws1', name: 'My Workspace', ownerId: 'user123' }, // Duplicate (should be filtered) - { id: 'ws3', name: 'Shared Workspace', ownerId: 'other-user' }, - ] - - let callCount = 0 - mockDb.select.mockImplementation(() => { - callCount++ - if (callCount === 1) { - return createMockChain(mockOwnedWorkspaces) // Owned workspaces - } - return createMockChain(mockAdminWorkspaces) // Admin workspaces - }) - - const result = await getManageableWorkspaces('user123') - - expect(result).toHaveLength(3) - expect(result).toEqual([ - { id: 'ws1', name: 'My Workspace', ownerId: 'user123', accessType: 'owner' }, - { id: 'ws2', name: 'Another Workspace', ownerId: 'user123', accessType: 'owner' }, - { id: 'ws3', name: 'Shared Workspace', ownerId: 'other-user', accessType: 'direct' }, - ]) - }) - }) - describe('getWorkspaceWithOwner', () => { /** * Archived visibility is applied in JS, not SQL, so the read can be shared by the diff --git a/apps/sim/lib/workspaces/permissions/utils.ts b/apps/sim/lib/workspaces/permissions/utils.ts index 19fb541f3ca..373583720b3 100644 --- a/apps/sim/lib/workspaces/permissions/utils.ts +++ b/apps/sim/lib/workspaces/permissions/utils.ts @@ -12,7 +12,6 @@ import { import { and, eq, inArray, isNull } from 'drizzle-orm' import { HttpError } from '@/lib/core/utils/http-error' import type { DbOrTx } from '@/lib/db/types' -import { getOrgAdminWorkspaceRows } from '@/lib/workspaces/utils' export type { PermissionType } export interface WorkspaceBasic { @@ -558,91 +557,3 @@ export async function isOrganizationAdminOrOwner( .limit(1) return isOrgAdminRole(row?.role) } - -/** - * Check whether a user is a member (any role) of a specific organization. - * - * @param userId - The ID of the user to check - * @param organizationId - The ID of the organization to check - * @returns Promise - True when the user has an organization membership row - */ -export async function isOrganizationMember( - userId: string, - organizationId: string -): Promise { - const [row] = await db - .select({ id: member.id }) - .from(member) - .where(and(eq(member.userId, userId), eq(member.organizationId, organizationId))) - .limit(1) - return !!row -} - -/** - * Get a list of workspaces that the user has access to - * - * @param userId - The ID of the user to check - * @returns Promise> - A list of workspaces that the user has access to - */ -export async function getManageableWorkspaces(userId: string): Promise< - Array<{ - id: string - name: string - ownerId: string - accessType: 'direct' | 'owner' - }> -> { - const ownedWorkspaces = await db - .select({ - id: workspace.id, - name: workspace.name, - ownerId: workspace.ownerId, - }) - .from(workspace) - .where(and(eq(workspace.ownerId, userId), isNull(workspace.archivedAt))) - - const adminWorkspaces = await db - .select({ - id: workspace.id, - name: workspace.name, - ownerId: workspace.ownerId, - }) - .from(workspace) - .innerJoin(permissions, eq(permissions.entityId, workspace.id)) - .where( - and( - isNull(workspace.archivedAt), - eq(permissions.userId, userId), - eq(permissions.entityType, 'workspace'), - eq(permissions.permissionType, 'admin') - ) - ) - - const orgAdminWorkspaces = (await getOrgAdminWorkspaceRows(userId, 'active')).map((ws) => ({ - id: ws.id, - name: ws.name, - ownerId: ws.ownerId, - })) - - const ownedSet = new Set(ownedWorkspaces.map((w) => w.id)) - const seen = new Set(ownedSet) - const combined: Array<{ - id: string - name: string - ownerId: string - accessType: 'direct' | 'owner' - }> = ownedWorkspaces.map((ws) => ({ ...ws, accessType: 'owner' as const })) - - for (const ws of [...adminWorkspaces, ...orgAdminWorkspaces]) { - if (seen.has(ws.id)) continue - seen.add(ws.id) - combined.push({ ...ws, accessType: 'direct' as const }) - } - - return combined -} diff --git a/apps/sim/lib/workspaces/utils.ts b/apps/sim/lib/workspaces/utils.ts index a84c53af67f..39c3ba85d0a 100644 --- a/apps/sim/lib/workspaces/utils.ts +++ b/apps/sim/lib/workspaces/utils.ts @@ -162,16 +162,6 @@ export async function listAccessibleWorkspaceRowsForUser( ) } -export async function listUserWorkspaces(userId: string, scope: WorkspaceScope = 'active') { - const rows = await listAccessibleWorkspaceRowsForUser(userId, scope) - - return rows.map(({ workspace: ws, permissionType }) => ({ - workspaceId: ws.id, - workspaceName: ws.name, - role: ws.ownerId === userId ? 'owner' : permissionType, - })) -} - export interface ReassignBilledAccountResult { reassigned: Array<{ workspaceId: string; newBilledAccountUserId: string }> unresolved: string[] diff --git a/packages/testing/src/mocks/permissions.mock.ts b/packages/testing/src/mocks/permissions.mock.ts index b34544fb4ba..a850c816d5f 100644 --- a/packages/testing/src/mocks/permissions.mock.ts +++ b/packages/testing/src/mocks/permissions.mock.ts @@ -66,8 +66,6 @@ export const permissionsMockFns = { mockGetWorkspacePermissionsForViewer: vi.fn(), mockHasWorkspaceAdminAccess: vi.fn(), mockIsOrganizationAdminOrOwner: vi.fn(), - mockIsOrganizationMember: vi.fn(), - mockGetManageableWorkspaces: vi.fn(), } /** @@ -96,6 +94,4 @@ export const permissionsMock = { getWorkspacePermissionsForViewer: permissionsMockFns.mockGetWorkspacePermissionsForViewer, hasWorkspaceAdminAccess: permissionsMockFns.mockHasWorkspaceAdminAccess, isOrganizationAdminOrOwner: permissionsMockFns.mockIsOrganizationAdminOrOwner, - isOrganizationMember: permissionsMockFns.mockIsOrganizationMember, - getManageableWorkspaces: permissionsMockFns.mockGetManageableWorkspaces, } diff --git a/packages/testing/src/mocks/workspaces-utils.mock.ts b/packages/testing/src/mocks/workspaces-utils.mock.ts index c14ec296e47..ec42e42e9e1 100644 --- a/packages/testing/src/mocks/workspaces-utils.mock.ts +++ b/packages/testing/src/mocks/workspaces-utils.mock.ts @@ -36,7 +36,6 @@ export const workspacesUtilsMockFns = { mockGetWorkspaceOrganizationId: vi.fn(), mockGetOrgAdminWorkspaceRows: vi.fn(), mockListAccessibleWorkspaceRowsForUser: vi.fn(), - mockListUserWorkspaces: vi.fn(), mockTransferWorkspaceOwnershipToBilledAccountForMemberRemovalTx: vi.fn(), mockReassignWorkflowOwnershipForWorkspaceMemberRemovalTx: vi.fn(), mockReassignBilledAccountForUser: vi.fn( @@ -77,7 +76,6 @@ export const workspacesUtilsMock = { getWorkspaceOrganizationId: workspacesUtilsMockFns.mockGetWorkspaceOrganizationId, getOrgAdminWorkspaceRows: workspacesUtilsMockFns.mockGetOrgAdminWorkspaceRows, listAccessibleWorkspaceRowsForUser: workspacesUtilsMockFns.mockListAccessibleWorkspaceRowsForUser, - listUserWorkspaces: workspacesUtilsMockFns.mockListUserWorkspaces, transferWorkspaceOwnershipToBilledAccountForMemberRemovalTx: workspacesUtilsMockFns.mockTransferWorkspaceOwnershipToBilledAccountForMemberRemovalTx, reassignWorkflowOwnershipForWorkspaceMemberRemovalTx: From 0859414870f9ef55071f2b7b19e9c3a90bb863d6 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:10:43 -0700 Subject: [PATCH 08/30] chore(cleanup): remove dead custom-tool use cases and unused folder, slack-search, skills, and selector helpers --- .../application/use-cases.test.ts | 34 ----- .../lib/custom-tools/application/use-cases.ts | 136 +----------------- apps/sim/lib/folders/cascade.test.ts | 24 ++-- apps/sim/lib/folders/cascade.ts | 21 --- apps/sim/lib/folders/scope.test.ts | 46 +----- apps/sim/lib/folders/scope.ts | 30 +--- apps/sim/lib/folders/subtree.test.ts | 9 -- apps/sim/lib/folders/subtree.ts | 18 --- apps/sim/lib/selectors/server/types.ts | 10 -- apps/sim/lib/skills/access.test.ts | 43 ------ apps/sim/lib/skills/access.ts | 38 ----- apps/sim/lib/slack-search/manifest.test.ts | 21 +-- apps/sim/lib/slack-search/manifest.ts | 43 ------ apps/sim/lib/slack-search/messages.ts | 86 ----------- 14 files changed, 16 insertions(+), 543 deletions(-) diff --git a/apps/sim/lib/custom-tools/application/use-cases.test.ts b/apps/sim/lib/custom-tools/application/use-cases.test.ts index f9f5c4e63e5..9eb3d14594c 100644 --- a/apps/sim/lib/custom-tools/application/use-cases.test.ts +++ b/apps/sim/lib/custom-tools/application/use-cases.test.ts @@ -23,15 +23,12 @@ vi.mock('@/lib/uploads/contexts/workspace', () => workspaceUploadsMock) vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) vi.mock('@sim/audit', () => auditMock) vi.mock('@/lib/workflows/custom-tools/operations', () => ({ - deleteCustomTool: vi.fn(), deleteWorkspaceCustomTool: vi.fn(), getAvailableCustomTool: hoisted.getAvailableTool, - getCustomToolById: vi.fn(), getWorkspaceCustomTool: hoisted.getWorkspaceTool, getWorkspaceCustomToolByTitle: hoisted.getByTitle, listCustomTools: hoisted.listAvailable, listWorkspaceCustomTools: vi.fn(), - updateCustomTool: vi.fn(), updateWorkspaceCustomTool: hoisted.updateWorkspaceTool, upsertCustomTools: hoisted.upsert, })) @@ -41,7 +38,6 @@ import { createWorkspaceCustomToolUseCase, listAvailableCustomToolsUseCase, readAvailableCustomToolByIdOrTitleUseCase, - saveWorkspaceCustomToolUseCase, updateWorkspaceCustomToolUseCase, } from '@/lib/custom-tools/application/use-cases' @@ -370,36 +366,6 @@ describe('custom tool application use cases', () => { expect(mocks.audit).not.toHaveBeenCalled() }) - it('normalizes the in-transaction duplicate-title error for compatibility saves', async () => { - mocks.upsert.mockRejectedValueOnce( - new Error(`A tool with the title "${tool.title}" already exists in this workspace`) - ) - - await expect( - saveWorkspaceCustomToolUseCase.execute({ - principal: { - kind: 'delegated', - serviceId: 'copilot', - subjectUserId: 'user-1', - workspaceId: workspace.workspaceId, - delegationId: 'delegation-1', - audience: CUSTOM_TOOL_DELEGATION_AUDIENCE, - issuedAt: new Date(Date.now() - 1_000), - expiresAt: new Date(Date.now() + 60_000), - }, - input: { - workspaceId: workspace.workspaceId, - title: tool.title, - schema: tool.schema, - code: tool.code, - source: 'tool_input', - }, - }) - ).rejects.toMatchObject({ code: 'conflict' }) - - expect(mocks.audit).not.toHaveBeenCalled() - }) - describe('public update against the shape the response publishes', () => { const storableSchema = { type: 'function', diff --git a/apps/sim/lib/custom-tools/application/use-cases.ts b/apps/sim/lib/custom-tools/application/use-cases.ts index 4d9643837ba..c70e9944c1e 100644 --- a/apps/sim/lib/custom-tools/application/use-cases.ts +++ b/apps/sim/lib/custom-tools/application/use-cases.ts @@ -1,5 +1,5 @@ import { AuditAction, AuditResourceType } from '@sim/audit' -import { type Principal, resolvePrincipalAttribution } from '@sim/auth/principal' +import { resolvePrincipalAttribution } from '@sim/auth/principal' import type { customTools } from '@sim/db/schema' import { getErrorMessage, getPostgresErrorCode } from '@sim/utils/errors' import type { CursorKey, ListSortOrder } from '@/lib/api/list-query' @@ -17,15 +17,12 @@ import { import { loadActiveWorkspaceContext } from '@/lib/uploads/contexts/workspace' import { type CustomToolSortBy, - deleteCustomTool, deleteWorkspaceCustomTool, getAvailableCustomTool, - getCustomToolById, getWorkspaceCustomTool, getWorkspaceCustomToolByTitle, listCustomTools, listWorkspaceCustomTools, - updateCustomTool, updateWorkspaceCustomTool, upsertCustomTools, } from '@/lib/workflows/custom-tools/operations' @@ -60,21 +57,6 @@ async function resolveWorkspaceToolContext( return { ...workspace, tool } } -async function resolveAvailableToolContext(args: { - principal: Exclude - workspaceId: string - toolId: string -}): Promise { - const workspace = await resolveWorkspaceContext(args.workspaceId) - const tool = await getCustomToolById({ - toolId: args.toolId, - userId: requireCustomToolUserId(args.principal), - workspaceId: workspace.workspaceId, - }) - if (!tool) throw new OrchestrationError('not_found', 'Custom tool not found') - return { ...workspace, tool } -} - function customToolConflict(error: unknown): never { if (getPostgresErrorCode(error) === '23505') { throw new OrchestrationError( @@ -220,38 +202,6 @@ export const createWorkspaceCustomToolUseCase = defineAuthorizedWorkspaceUseCase }), }) -export const saveWorkspaceCustomToolUseCase = defineAuthorizedWorkspaceUseCase({ - operation: customToolOperations.save, - resolveContext: ({ input }: { input: CreateWorkspaceCustomToolInput }) => - resolveWorkspaceContext(input.workspaceId), - authorizationOptions, - async execute({ principal, input, context }) { - const attribution = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }) - try { - const tools = await upsertCustomTools({ - tools: [{ title: input.title, schema: input.schema, code: input.code }], - workspaceId: context.workspaceId, - userId: attribution.attributedUserId, - }) - const tool = tools.find((candidate) => candidate.title === input.title) - if (!tool) throw new Error(`Custom tool "${input.title}" missing after a successful save`) - return { tool } - } catch (error) { - return customToolConflict(error) - } - }, - projectAudit: ({ input, result }) => ({ - action: AuditAction.CUSTOM_TOOL_CREATED, - resourceType: AuditResourceType.CUSTOM_TOOL, - resourceId: result.tool.id, - resourceName: result.tool.title, - description: `Created custom tool "${result.tool.title}"`, - metadata: { source: input.source }, - }), -}) - interface UpdateCustomToolFields { title?: string schema?: unknown @@ -321,56 +271,6 @@ export const updateWorkspaceCustomToolUseCase = defineAuthorizedWorkspaceUseCase }), }) -export const updateAvailableCustomToolUseCase = defineAuthorizedWorkspaceUseCase({ - operation: customToolOperations.updateAvailable, - resolveContext: ({ - principal, - input, - }: { - principal: Exclude - input: UpdateWorkspaceCustomToolInput - }) => - resolveAvailableToolContext({ - principal, - workspaceId: input.workspaceId, - toolId: input.toolId, - }), - authorizationOptions, - async execute({ principal, input, context }) { - const title = input.title ?? context.tool.title - await ensureTitleAvailable(context, title) - /** - * Only a supplied schema, unlike the public update above: this surface does - * not parse what it returns, so an edit that merely keeps a legacy stored - * schema still succeeds, while a caller-supplied one is held to the shape - * the public API has to publish. - */ - if (input.schema !== undefined) assertValidCustomToolDeclaration(input.schema) - try { - const tool = await updateCustomTool({ - workspaceId: context.workspaceId, - toolId: context.tool.id, - userId: requireCustomToolUserId(principal), - title, - schema: input.schema ?? context.tool.schema, - code: input.code ?? context.tool.code, - }) - if (!tool) throw new OrchestrationError('not_found', 'Custom tool not found') - return { tool } - } catch (error) { - return customToolConflict(error) - } - }, - projectAudit: ({ input, result }) => ({ - action: AuditAction.CUSTOM_TOOL_UPDATED, - resourceType: AuditResourceType.CUSTOM_TOOL, - resourceId: result.tool.id, - resourceName: result.tool.title, - description: `Updated custom tool "${result.tool.title}"`, - metadata: { source: input.source }, - }), -}) - export interface DeleteWorkspaceCustomToolInput { workspaceId: string toolId: string @@ -399,37 +299,3 @@ export const deleteWorkspaceCustomToolUseCase = defineAuthorizedWorkspaceUseCase metadata: { source: input.source }, }), }) - -export const deleteAvailableCustomToolUseCase = defineAuthorizedWorkspaceUseCase({ - operation: customToolOperations.deleteAvailable, - resolveContext: ({ - principal, - input, - }: { - principal: Exclude - input: DeleteWorkspaceCustomToolInput - }) => - resolveAvailableToolContext({ - principal, - workspaceId: input.workspaceId, - toolId: input.toolId, - }), - authorizationOptions, - async execute({ principal, input, context }) { - const deleted = await deleteCustomTool({ - workspaceId: context.workspaceId, - toolId: context.tool.id, - userId: requireCustomToolUserId(principal), - }) - if (!deleted) throw new OrchestrationError('not_found', 'Custom tool not found') - return { tool: context.tool } - }, - projectAudit: ({ input, result }) => ({ - action: AuditAction.CUSTOM_TOOL_DELETED, - resourceType: AuditResourceType.CUSTOM_TOOL, - resourceId: result.tool.id, - resourceName: result.tool.title, - description: `Deleted custom tool "${result.tool.title}"`, - metadata: { source: input.source }, - }), -}) diff --git a/apps/sim/lib/folders/cascade.test.ts b/apps/sim/lib/folders/cascade.test.ts index ac32791dc44..c8e8986113b 100644 --- a/apps/sim/lib/folders/cascade.test.ts +++ b/apps/sim/lib/folders/cascade.test.ts @@ -5,7 +5,8 @@ import { collectArchivedSubtreeIds, collectCascadeSubtreeIds, type DbOrTx, - restoreFolderCascade, + restoreFolderChildren, + restoreFolderRows, } from '@/lib/folders/cascade' import { FOLDER_RESOURCES, type FolderResourceConfig } from '@/lib/folders/config' import { FolderCollectionLimitExceededError } from '@/lib/folders/errors' @@ -187,7 +188,7 @@ describe('archiveFolderCascade', () => { }) }) -describe('restoreFolderCascade', () => { +describe('folder restore cascade', () => { const dependents = [ { table: DEPENDENT_TABLE as never, @@ -202,16 +203,18 @@ describe('restoreFolderCascade', () => { updates: [[{ id: 'root' }, { id: 'sub' }], [{ id: 'child-1' }, { id: 'child-2' }], []], }) - const counts = await restoreFolderCascade( + const config = makeConfig({ restoreDependents: dependents }) + const folders = await restoreFolderRows(tx, config, 'ws-1', ['root', 'sub'], TIMESTAMP, NOW) + const children = await restoreFolderChildren( tx, - makeConfig({ restoreDependents: dependents }), + config, 'ws-1', ['root', 'sub'], TIMESTAMP, NOW ) - expect(counts).toEqual({ folders: 2, children: 2 }) + expect({ folders, children }).toEqual({ folders: 2, children: 2 }) expect(updateCalls).toHaveLength(3) expect(updateCalls[1].set).toEqual({ archivedAt: null, updatedAt: NOW }) expect(updateCalls[2].table).toBe(DEPENDENT_TABLE) @@ -225,14 +228,9 @@ describe('restoreFolderCascade', () => { it('restores only rows carrying the folder’s own soft-delete timestamp', async () => { const { tx, updateCalls } = makeTx({ updates: [[{ id: 'root' }], [{ id: 'child-1' }], []] }) - await restoreFolderCascade( - tx, - makeConfig({ restoreDependents: dependents }), - 'ws-1', - ['root'], - TIMESTAMP, - NOW - ) + const config = makeConfig({ restoreDependents: dependents }) + await restoreFolderRows(tx, config, 'ws-1', ['root'], TIMESTAMP, NOW) + await restoreFolderChildren(tx, config, 'ws-1', ['root'], TIMESTAMP, NOW) for (const call of updateCalls) { expect(hasMockCondition(call.where, (node) => node.right === TIMESTAMP)).toBe(true) diff --git a/apps/sim/lib/folders/cascade.ts b/apps/sim/lib/folders/cascade.ts index 08db6cab1b3..4a859759ee0 100644 --- a/apps/sim/lib/folders/cascade.ts +++ b/apps/sim/lib/folders/cascade.ts @@ -212,27 +212,6 @@ export async function restoreFolderChildren( return childIds.length } -/** - * Restores a folder subtree and the resources inside it, via the default row-update path. - * - * Only valid for resources without a {@link FolderResourceConfig.restoreChildren} hook — - * those hooks call canonical single-resource restores that open their own transactions and - * therefore must not run nested inside this one. `restoreFolder` sequences that case itself. - */ -export async function restoreFolderCascade( - tx: DbOrTx, - config: FolderResourceConfig, - workspaceId: string, - folderIds: string[], - timestamp: Date, - now: Date -): Promise { - const folders = await restoreFolderRows(tx, config, workspaceId, folderIds, timestamp, now) - const children = await restoreFolderChildren(tx, config, workspaceId, folderIds, timestamp, now) - - return { folders, children } -} - /** * Maps internal cascade counts onto the per-resourceType shape the API returns, so a * `knowledge_base` folder reports `knowledgeBases` and a `table` folder reports `tables`. diff --git a/apps/sim/lib/folders/scope.test.ts b/apps/sim/lib/folders/scope.test.ts index b7745ae91d3..ab1f8f36597 100644 --- a/apps/sim/lib/folders/scope.test.ts +++ b/apps/sim/lib/folders/scope.test.ts @@ -1,9 +1,5 @@ import { describe, expect, it } from 'vitest' -import { - isFolderPathWithinScope, - isWithinFolderIdScope, - isWithinFolderScope, -} from '@/lib/folders/scope' +import { isWithinFolderScope } from '@/lib/folders/scope' describe('isWithinFolderScope', () => { it('includes descendants by default and only the folder itself when told not to', () => { @@ -18,43 +14,3 @@ describe('isWithinFolderScope', () => { expect(isWithinFolderScope(['Reports Archive'], ['Reports'])).toBe(false) }) }) - -describe('isFolderPathWithinScope', () => { - it('reads a slash inside a folder name as one level, not two', () => { - /* One folder genuinely named `Q3/Q4`, not a `Q3` holding a `Q4`. */ - expect(isFolderPathWithinScope('/Reports/Q3%2FQ4', '/Reports/Q3')).toBe(false) - expect(isFolderPathWithinScope('/Reports/Q3%2FQ4', '/Reports/Q3%2FQ4')).toBe(true) - expect(isFolderPathWithinScope('/Reports/Q3%2FQ4/Drafts', '/Reports/Q3%2FQ4')).toBe(true) - expect(isFolderPathWithinScope('/Reports/Q3', '/Reports/Q3%2FQ4')).toBe(false) - }) - - it('matches an exact folder and its descendants', () => { - expect(isFolderPathWithinScope('/Reports', '/Reports')).toBe(true) - expect(isFolderPathWithinScope('/Reports/Q3%20Results', '/Reports')).toBe(true) - expect( - isFolderPathWithinScope('/Reports/Q3%20Results', '/Reports', { includeSubfolders: false }) - ).toBe(false) - }) - - it('rejects a path that is not canonical', () => { - expect(() => isFolderPathWithinScope('Reports', '/Reports')).toThrow() - expect(() => isFolderPathWithinScope('/Reports/', '/Reports')).toThrow() - }) -}) - -describe('isWithinFolderIdScope', () => { - const scope = { folderIds: new Set(['a', 'b']), includeRootItems: false } - - it('rejects an item outside it', () => { - expect(isWithinFolderIdScope('c', scope)).toBe(false) - }) - - /* - * The case the flag exists for: a root item has no id to match, so without - * it every root item silently falls outside every scope. - */ - it('rejects a root item unless the scope includes the root', () => { - expect(isWithinFolderIdScope(null, scope)).toBe(false) - expect(isWithinFolderIdScope(null, { ...scope, includeRootItems: true })).toBe(true) - }) -}) diff --git a/apps/sim/lib/folders/scope.ts b/apps/sim/lib/folders/scope.ts index f784e45334c..db4a5aa1780 100644 --- a/apps/sim/lib/folders/scope.ts +++ b/apps/sim/lib/folders/scope.ts @@ -1,5 +1,3 @@ -import { parseFolderPath } from '@/lib/folders/paths' - export interface FolderScopeOptions { /** * Whether the scope reaches nested folders. Absent means yes — a folder @@ -36,27 +34,11 @@ export function isWithinFolderScope( return scopeSegments.every((segment, index) => segments[index] === segment) } -/** - * Whether a canonical folder path sits inside a canonical folder scope. - * - * For the resource types whose folders are addressed by canonical path on both - * sides — workflows, knowledge bases, and tables. Workspace files store a - * backslash-escaped display path instead and go through - * `isFileInFolderScope`, which decodes its side before comparing. - */ -export function isFolderPathWithinScope( - folderPath: string, - scopePath: string, - options?: FolderScopeOptions -): boolean { - return isWithinFolderScope(parseFolderPath(folderPath), parseFolderPath(scopePath), options) -} - /** * A folder scope already resolved to concrete folder ids. * - * The path predicates above answer "is this folder inside that scope" one - * folder at a time. This is the same question asked of many items at once, + * `isWithinFolderScope` answers "is this folder inside that scope" one folder + * at a time. This is the same question asked of many items at once, * after the paths have been walked to ids: a listing filters against it in * memory, and a query pushes it down into SQL. * @@ -70,11 +52,3 @@ export interface FolderIdScope { /** Items carrying no folder id are in scope. */ includeRootItems: boolean } - -/** Whether an item belongs to a resolved scope. Root items carry no folder id. */ -export function isWithinFolderIdScope( - folderId: string | null | undefined, - scope: FolderIdScope -): boolean { - return folderId ? scope.folderIds.has(folderId) : scope.includeRootItems -} diff --git a/apps/sim/lib/folders/subtree.test.ts b/apps/sim/lib/folders/subtree.test.ts index aac9639477a..7ebdf115972 100644 --- a/apps/sim/lib/folders/subtree.test.ts +++ b/apps/sim/lib/folders/subtree.test.ts @@ -5,7 +5,6 @@ import { collectFolderDepths, type FolderNode, indexFolderChildren, - selectFolderSubtreeRows, } from '@/lib/folders/subtree' const tree: FolderNode[] = [ @@ -98,11 +97,3 @@ describe('collectFolderDepths', () => { expect(() => collectFolderDepths(withCycle, 'root')).not.toThrow() }) }) - -describe('selectFolderSubtreeRows', () => { - it('derives depth from the full tree, so a filtered row set cannot orphan descendants', () => { - const rows = [{ id: 'draft' }] - - expect(selectFolderSubtreeRows(rows, depthTree, 'reports')).toEqual([{ id: 'draft' }]) - }) -}) diff --git a/apps/sim/lib/folders/subtree.ts b/apps/sim/lib/folders/subtree.ts index bde7e766324..90a610a043c 100644 --- a/apps/sim/lib/folders/subtree.ts +++ b/apps/sim/lib/folders/subtree.ts @@ -117,21 +117,3 @@ export function collectFolderDepths( return depths } - -/** - * Narrows already-queried rows to the subtree under `rootId`, preserving the - * query's ordering so a caller's `sortBy` still decides the result order. - * - * `tree` is the workspace's full folder set, deliberately separate from `rows`: - * depths must come from the real hierarchy, or a `search` that excludes an - * intermediate folder would orphan its matching descendants. - */ -export function selectFolderSubtreeRows( - rows: readonly Row[], - tree: readonly FolderNode[], - rootId: string | null, - maxDepth?: number -): Row[] { - const depths = collectFolderDepths(tree, rootId, { maxDepth }) - return rows.filter((row) => depths.has(row.id)) -} diff --git a/apps/sim/lib/selectors/server/types.ts b/apps/sim/lib/selectors/server/types.ts index b361c675c26..114f542e82b 100644 --- a/apps/sim/lib/selectors/server/types.ts +++ b/apps/sim/lib/selectors/server/types.ts @@ -230,13 +230,3 @@ export function requireListRequest( } return request } - -export function requireDetailRequest( - selectorKey: SelectorKey, - request: SelectorRequest -): Extract { - if (request.kind !== 'detail') { - throw new Error(`Selector ${selectorKey} received an unsupported list request`) - } - return request -} diff --git a/apps/sim/lib/skills/access.test.ts b/apps/sim/lib/skills/access.test.ts index e6eb1c3be36..a15f5a059ff 100644 --- a/apps/sim/lib/skills/access.test.ts +++ b/apps/sim/lib/skills/access.test.ts @@ -22,7 +22,6 @@ const { dbState, makeChain } = vi.hoisted(() => { vi.mock('@/lib/workspaces/permissions/utils', () => permissionsMock) import { - checkSkillsUpdateAccess, getEditableSkillIds, getSkillActorContext, listSkillEditors, @@ -180,48 +179,6 @@ describe('listSkillEditors', () => { }) }) -describe('checkSkillsUpdateAccess', () => { - it('returns nothing for an empty id list without querying', async () => { - const result = await checkSkillsUpdateAccess({ workspaceId: 'ws', userId: 'u', skillIds: [] }) - expect(result.existingIds.size).toBe(0) - expect(result.denied).toEqual([]) - expect(dbMock.select).not.toHaveBeenCalled() - }) - - it('partitions resolvable ids and denies skills without an editor row', async () => { - dbState.results = [ - [ - { id: 's-mine', name: 'mine' }, - { id: 's-other', name: 'other' }, - ], - [{ skillId: 's-mine' }], - ] - mockCheckWorkspaceAccess.mockResolvedValue(wsWrite) - - const result = await checkSkillsUpdateAccess({ - workspaceId: 'ws', - userId: 'u', - skillIds: ['s-mine', 's-other', 's-create'], - }) - - expect(result.existingIds).toEqual(new Set(['s-mine', 's-other'])) - expect(result.denied).toEqual([{ id: 's-other', name: 'other' }]) - }) - - it('denies nothing for workspace admins', async () => { - dbState.results = [[{ id: 's-any', name: 'any' }], []] - mockCheckWorkspaceAccess.mockResolvedValue(wsAdmin) - - const result = await checkSkillsUpdateAccess({ - workspaceId: 'ws', - userId: 'admin-user', - skillIds: ['s-any'], - }) - - expect(result.denied).toEqual([]) - }) -}) - describe('removeWorkspaceSkillMembershipsTx', () => { it('returns 0 for an empty workspace list without querying', async () => { const tx = { select: vi.fn(() => makeChain()), delete: vi.fn(() => makeChain()) } diff --git a/apps/sim/lib/skills/access.ts b/apps/sim/lib/skills/access.ts index ca835a47414..37014751b23 100644 --- a/apps/sim/lib/skills/access.ts +++ b/apps/sim/lib/skills/access.ts @@ -141,44 +141,6 @@ export async function listSkillEditors(skillRow: { return editors } -export interface SkillsUpdateAccess { - /** Ids from the request that resolve to existing skills in the workspace. */ - existingIds: Set - /** Existing skills the user may not update (not an editor, not a workspace admin). */ - denied: Array<{ id: string; name: string }> -} - -/** - * Partitions an upsert request's skill ids for authorization: ids that resolve - * to existing workspace skills require skill editor access; unresolved ids are - * creates, gated by workspace write permission instead. - */ -export async function checkSkillsUpdateAccess(params: { - workspaceId: string - userId: string - skillIds: string[] - workspaceAccess?: WorkspaceAccess -}): Promise { - if (params.skillIds.length === 0) return { existingIds: new Set(), denied: [] } - - const rows = await db - .select({ id: skill.id, name: skill.name }) - .from(skill) - .where(and(eq(skill.workspaceId, params.workspaceId), inArray(skill.id, params.skillIds))) - - const existingIds = new Set(rows.map((row) => row.id)) - if (rows.length === 0) return { existingIds, denied: [] } - - const access = await getEditableSkillIds(params.workspaceId, params.userId, { - workspaceAccess: params.workspaceAccess, - }) - const denied = access.canAdminWorkspace - ? [] - : rows.filter((row) => !access.editorSkillIds.has(row.id)) - - return { existingIds, denied } -} - /** * Removes a user's skill editor grants across one or more workspaces when they * leave (workspace removal, org removal/transfer). Rows are editor grants diff --git a/apps/sim/lib/slack-search/manifest.test.ts b/apps/sim/lib/slack-search/manifest.test.ts index 9ad4277605c..9f5698b4aeb 100644 --- a/apps/sim/lib/slack-search/manifest.test.ts +++ b/apps/sim/lib/slack-search/manifest.test.ts @@ -1,8 +1,5 @@ import { describe, expect, it } from 'vitest' -import { - createSharedSlackSearchManifest, - createSlackSearchManifest, -} from '@/lib/slack-search/manifest' +import { createSlackSearchManifest } from '@/lib/slack-search/manifest' describe('Search app manifest', () => { it('preserves existing member grants when updating a bot manifest', () => { @@ -16,19 +13,3 @@ describe('Search app manifest', () => { ]) }) }) - -it('official app declares commands and lifecycle events without member message events', () => { - const manifest = createSharedSlackSearchManifest('https://www.sim.ai') - expect(manifest.features.slash_commands.map((command) => command.command)).toEqual([ - '/query', - '/connect', - ]) - expect( - manifest.features.slash_commands.every( - (command) => command.url === 'https://www.sim.ai/api/webhooks/slack' - ) - ).toBe(true) - expect(manifest.settings.event_subscriptions.bot_events).toContain('tokens_revoked') - expect(manifest.settings.event_subscriptions.bot_events).not.toContain('message.channels') - expect(manifest.settings.event_subscriptions).not.toHaveProperty('user_events') -}) diff --git a/apps/sim/lib/slack-search/manifest.ts b/apps/sim/lib/slack-search/manifest.ts index a8d3a27f153..7ad83e44e3a 100644 --- a/apps/sim/lib/slack-search/manifest.ts +++ b/apps/sim/lib/slack-search/manifest.ts @@ -89,46 +89,3 @@ export function createSlackSearchManifest( }, } } - -/** Adds the official app's commands and lifecycle events to the common manifest. */ -export function createSharedSlackSearchManifest(origin: string) { - const manifest = createSlackSearchManifest( - SLACK_SEARCH_DEFAULT_NAME, - SLACK_SEARCH_DEFAULT_DESCRIPTION, - origin - ) - const webhook = new URL(SLACK_SEARCH_WEBHOOK_PATH, origin).href - return { - ...manifest, - features: { - ...manifest.features, - slash_commands: [ - { - command: '/query', - description: 'Ask Sim Search a question privately', - usage_hint: '[question]', - url: webhook, - should_escape: false, - }, - { - command: '/connect', - description: 'Connect your personal sources in Sim', - usage_hint: '[provider]', - url: webhook, - should_escape: false, - }, - ], - }, - settings: { - ...manifest.settings, - event_subscriptions: { - ...manifest.settings.event_subscriptions, - bot_events: [ - ...manifest.settings.event_subscriptions.bot_events, - 'app_uninstalled', - 'tokens_revoked', - ], - }, - }, - } -} diff --git a/apps/sim/lib/slack-search/messages.ts b/apps/sim/lib/slack-search/messages.ts index 9b37b0ca725..9981a215034 100644 --- a/apps/sim/lib/slack-search/messages.ts +++ b/apps/sim/lib/slack-search/messages.ts @@ -1,30 +1,7 @@ -import { truncate } from '@sim/utils/string' import type { SlackJsonObject, SlackMessage } from '@/lib/internal/slack/client' -import type { KnowledgeSearchItem } from '@/lib/knowledge/application/search' import type { SlackSearchMessage } from '@/lib/slack-search/types' import { slackSearchThreadTimestamp } from '@/lib/slack-search/types' -function sourceUrl(result: KnowledgeSearchItem, organizationId: string, baseUrl: string): string { - if (result.sourceUrl) { - try { - const url = new URL(result.sourceUrl) - if ( - (url.protocol === 'https:' || url.protocol === 'http:') && - !url.username && - !url.password && - url.href.length <= 3000 - ) - return url.href - } catch { - /** An invalid provider URL uses the canonical document link. */ - } - } - return new URL( - `/o/${encodeURIComponent(organizationId)}/knowledge/${encodeURIComponent(result.knowledgeBaseId)}/${encodeURIComponent(result.documentId)}`, - baseUrl - ).href -} - export function slackSearchReply( message: SlackSearchMessage, text: string, @@ -68,66 +45,3 @@ export function renderSlackSearchRedirect( ] ) } - -/** Presents the first five documents, preserving the ranking of their best returned chunks. */ -export function renderSlackSearchResults( - message: SlackSearchMessage, - organizationId: string, - results: KnowledgeSearchItem[], - baseUrl: string -): SlackMessage { - const seen = new Set() - const documents: KnowledgeSearchItem[] = [] - for (const result of results) { - if (seen.has(result.documentId)) continue - seen.add(result.documentId) - documents.push(result) - if (documents.length === 5) break - } - const blocks: SlackJsonObject[] = [ - { - type: 'section', - text: { - type: 'plain_text', - text: documents.length ? 'Search results' : 'No results found that you can access.', - }, - }, - ] - for (const [index, result] of documents.entries()) { - blocks.push({ - type: 'section', - text: { - type: 'plain_text', - text: `${index + 1}. ${truncate(result.documentName || 'Untitled document', 150)}\n${truncate(result.content.replace(/\s+/g, ' ').trim(), 300)}`, - }, - accessory: { - type: 'button', - action_id: `sim_search.open_document.${index}`, - text: { type: 'plain_text', text: 'Open' }, - url: sourceUrl(result, organizationId, baseUrl), - }, - }) - } - const searchUrl = new URL(`/o/${encodeURIComponent(organizationId)}/search`, baseUrl) - searchUrl.searchParams.set('q', message.query) - /** Slack caps button URLs at 3,000 characters; long queries still get a link to Search. */ - if (searchUrl.href.length > 3000) searchUrl.search = '' - blocks.push({ - type: 'actions', - elements: [ - { - type: 'button', - action_id: 'sim_search.open_search', - text: { type: 'plain_text', text: 'Open in Sim Search' }, - url: searchUrl.href, - }, - ], - }) - return slackSearchReply( - message, - documents.length - ? `Found ${documents.length} search results. Open Slack to view them.` - : 'No results found that you can access.', - blocks - ) -} From ad11f0c6a67ffd9318305043efbaa54930d2de01 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:12:24 -0700 Subject: [PATCH 09/30] chore(cleanup): remove unreachable application use cases and API helpers Delete test-only Copilot/executor use cases (bulk table rows by filter, credential delete-many, credential-group invite send/link, sandbox and connected-accounts adapters) with their operation entries, the unused public API route wrapper, parseToolRequest, the invitation management error policy, the execution invitation rate limit, and dead table/chat route helpers. --- apps/sim/app/api/chat/utils.ts | 24 -- apps/sim/app/api/files/utils.ts | 2 +- .../app/api/public-api-route-handler.test.ts | 191 --------- apps/sim/app/api/public-api-route-handler.ts | 76 ---- apps/sim/app/api/table/row-wire.ts | 73 +--- apps/sim/lib/api/server/index.ts | 1 - apps/sim/lib/api/server/tool-validation.ts | 75 ---- .../application/create-invite-link.test.ts | 145 ------- .../application/create-invite-link.ts | 68 ---- .../application/manage-groups.test.ts | 31 -- .../application/operations.ts | 18 - .../application/send-invite.test.ts | 168 -------- .../application/send-invite.ts | 77 ---- .../lib/credential-groups/enrollments.test.ts | 64 --- apps/sim/lib/credential-groups/enrollments.ts | 20 - apps/sim/lib/credential-groups/rate-limit.ts | 12 - .../delete-many-credentials.test.ts | 112 ------ .../application/delete-many-credentials.ts | 114 ------ .../lib/credentials/application/operations.ts | 8 - .../application/manage-invitation.test.ts | 16 +- .../invitations/management-error-policy.ts | 22 -- .../execute-sandbox-use-case.test.ts | 67 ---- .../application/execute-sandbox-use-case.ts | 14 - .../application/load-connected-accounts.ts | 28 -- .../application/copilot-bulk-rows.test.ts | 196 ---------- .../table/application/copilot-bulk-rows.ts | 363 ------------------ .../credential-groups-enrollments.mock.ts | 2 - scripts/check-api-validation-contracts.ts | 2 +- 28 files changed, 7 insertions(+), 1982 deletions(-) delete mode 100644 apps/sim/app/api/public-api-route-handler.test.ts delete mode 100644 apps/sim/app/api/public-api-route-handler.ts delete mode 100644 apps/sim/lib/api/server/tool-validation.ts delete mode 100644 apps/sim/lib/credential-groups/application/create-invite-link.test.ts delete mode 100644 apps/sim/lib/credential-groups/application/create-invite-link.ts delete mode 100644 apps/sim/lib/credential-groups/application/send-invite.test.ts delete mode 100644 apps/sim/lib/credential-groups/application/send-invite.ts delete mode 100644 apps/sim/lib/credentials/application/delete-many-credentials.test.ts delete mode 100644 apps/sim/lib/credentials/application/delete-many-credentials.ts delete mode 100644 apps/sim/lib/invitations/management-error-policy.ts delete mode 100644 apps/sim/lib/mothership/application/execute-sandbox-use-case.test.ts delete mode 100644 apps/sim/lib/mothership/application/execute-sandbox-use-case.ts delete mode 100644 apps/sim/lib/mothership/application/load-connected-accounts.ts delete mode 100644 apps/sim/lib/table/application/copilot-bulk-rows.test.ts delete mode 100644 apps/sim/lib/table/application/copilot-bulk-rows.ts diff --git a/apps/sim/app/api/chat/utils.ts b/apps/sim/app/api/chat/utils.ts index 005aedfc91f..7775d3032a8 100644 --- a/apps/sim/app/api/chat/utils.ts +++ b/apps/sim/app/api/chat/utils.ts @@ -26,30 +26,6 @@ export async function setChatAuthCookie( }) } -/** - * Check if user has permission to create a chat for a specific workflow - */ -export async function checkWorkflowAccessForChatCreation( - workflowId: string, - userId: string -): Promise<{ hasAccess: boolean; workflow?: any }> { - const authorization = await authorizeWorkflowByWorkspacePermission({ - workflowId, - userId, - action: 'admin', - }) - - if (!authorization.workflow) { - return { hasAccess: false } - } - - if (authorization.allowed) { - return { hasAccess: true, workflow: authorization.workflow } - } - - return { hasAccess: false } -} - /** * Check if user has access to view/edit/delete a specific chat */ diff --git a/apps/sim/app/api/files/utils.ts b/apps/sim/app/api/files/utils.ts index 0bd29f54288..7805849ed5c 100644 --- a/apps/sim/app/api/files/utils.ts +++ b/apps/sim/app/api/files/utils.ts @@ -40,7 +40,7 @@ export class InvalidRequestError extends Error { } } -export const contentTypeMap: Record = { +const contentTypeMap: Record = { txt: 'text/plain', csv: 'text/csv', json: 'application/json', diff --git a/apps/sim/app/api/public-api-route-handler.test.ts b/apps/sim/app/api/public-api-route-handler.test.ts deleted file mode 100644 index 1922e4f164e..00000000000 --- a/apps/sim/app/api/public-api-route-handler.test.ts +++ /dev/null @@ -1,191 +0,0 @@ -import { getMockLogger, loggerMock } from '@sim/testing/mocks/logger.mock' -import { createMockRequest, requestUtilsMockFns } from '@sim/testing/mocks/request.mock' -import { v1MiddlewareMock, v1MiddlewareMockFns } from '@sim/testing/mocks/v1-middleware.mock' -import { type NextRequest, NextResponse } from 'next/server' -import { beforeEach, describe, expect, it, vi } from 'vitest' -import { z } from 'zod' -import { defineRouteContract } from '@/lib/api/contracts' -import { recordRateLimitSnapshot } from '@/lib/api/server/rate-limit-context' - -const { mockHandler } = vi.hoisted(() => ({ - mockHandler: vi.fn(), -})) - -vi.mock('@/app/api/v1/middleware', () => v1MiddlewareMock) - -import { withPublicApiRouteHandler } from '@/app/api/public-api-route-handler' - -const mockCheckRateLimit = v1MiddlewareMockFns.mockCheckRateLimit -const requestContextState: { - current: { requestId: string; method?: string; path?: string } | undefined -} = { current: undefined } -const mockLoggerError = vi.fn() - -loggerMock.getRequestContext.mockImplementation(() => requestContextState.current) -loggerMock.runWithRequestContext.mockImplementation( - async ( - context: { requestId: string; method?: string; path?: string }, - callback: () => T | Promise - ): Promise => { - requestContextState.current = context - try { - return await callback() - } finally { - requestContextState.current = undefined - } - } -) -getMockLogger('RouteHandler').error.mockImplementation((...arguments_: unknown[]) => - mockLoggerError(requestContextState.current?.requestId, ...arguments_) -) - -requestUtilsMockFns.mockGenerateRequestId.mockImplementation( - () => requestContextState.current?.requestId ?? 'outer-request-id' -) - -const RATE_LIMIT = { - allowed: true, - limit: 400, - remaining: 399, - resetAt: new Date('2026-08-06T20:00:00.000Z'), - userId: 'user-1', - keyType: 'personal' as const, -} - -const queryContract = defineRouteContract({ - method: 'POST', - path: '/api/test/:itemId', - params: z.object({ itemId: z.string().min(1) }), - query: z.object({ limit: z.coerce.number().int().positive() }), - body: z.object({ name: z.string().min(1) }), - response: { mode: 'json', schema: z.object({ ok: z.boolean() }) }, -}) - -const listContract = defineRouteContract({ - method: 'GET', - path: '/api/test', - query: z.object({ workspaceId: z.string().min(1) }), - response: { mode: 'json', schema: z.object({ ok: z.boolean() }) }, -}) - -const POST = withPublicApiRouteHandler({ - contract: queryContract, - rateLimitEndpoint: 'table-rows', - parseOptions: { - maxBodyBytes: 32, - payloadTooLargeResponse: () => - NextResponse.json({ error: 'Custom payload limit response' }, { status: 413 }), - }, - handler: async (arguments_) => { - mockHandler(arguments_) - return NextResponse.json({ ok: true }) - }, -}) - -const GET = withPublicApiRouteHandler({ - contract: listContract, - rateLimitEndpoint: 'tables', - handler: async (arguments_) => { - mockHandler(arguments_) - return NextResponse.json({ ok: true }) - }, -}) - -const FAILING_GET = withPublicApiRouteHandler({ - contract: listContract, - rateLimitEndpoint: 'tables', - handler: async () => { - throw new Error('handler failed') - }, -}) - -function postRequest(body: string): NextRequest { - return createMockRequest({ - method: 'POST', - url: '/api/test/item-1?limit=10', - headers: { 'Content-Type': 'application/json' }, - rawBody: body, - }) -} - -function listRequest(query = 'workspaceId=workspace-1'): NextRequest { - return createMockRequest({ url: `/api/test?${query}` }) -} - -describe('withPublicApiRouteHandler', () => { - beforeEach(() => { - mockCheckRateLimit.mockImplementation(async (request: NextRequest) => { - recordRateLimitSnapshot(request, RATE_LIMIT) - return RATE_LIMIT - }) - }) - - it.each([ - ['authentication failure', 401], - ['rate-limit denial', 429], - ])('short-circuits %s before reading or parsing the body', async (_label, status) => { - mockCheckRateLimit.mockImplementation(async (request: NextRequest) => { - if (status === 401) { - return { - allowed: false, - limit: 0, - remaining: 0, - resetAt: new Date('2026-08-06T20:00:00.000Z'), - error: 'API key required', - } - } - - recordRateLimitSnapshot(request, RATE_LIMIT) - return { ...RATE_LIMIT, allowed: false, remaining: 0, retryAfterMs: 30_000 } - }) - const request = postRequest('{not valid json') - - const response = await POST(request, { params: { itemId: 'item-1' } }) - - expect(response.status).toBe(status) - expect(request.bodyUsed).toBe(false) - expect(mockHandler).not.toHaveBeenCalled() - expect(mockCheckRateLimit).toHaveBeenCalledWith(request, 'table-rows') - if (status === 401) { - expect(response.headers.get('X-RateLimit-Limit')).toBe('0') - } else { - expect(response.headers.get('Retry-After')).toBe('30') - expect(response.headers.get('X-RateLimit-Limit')).toBe('400') - } - }) - - it('fails fast when an allowed rate-limit result has no user ID', async () => { - mockCheckRateLimit.mockResolvedValue({ ...RATE_LIMIT, userId: undefined }) - - const response = await GET(listRequest()) - - expect(response.status).toBe(500) - expect(mockHandler).not.toHaveBeenCalled() - }) - - it('returns a contract validation response after authentication', async () => { - const response = await POST(postRequest(JSON.stringify({ name: '' })), { - params: { itemId: 'item-1' }, - }) - - expect(response.status).toBe(400) - expect(response.headers.get('X-RateLimit-Limit')).toBe('400') - expect(mockHandler).not.toHaveBeenCalled() - }) - - it('keeps rate-limit and request headers on unhandled endpoint errors', async () => { - const response = await FAILING_GET(listRequest()) - - expect(response.status).toBe(500) - await expect(response.json()).resolves.toEqual({ - error: { code: 'INTERNAL_ERROR', message: 'Internal server error' }, - }) - expect(response.headers.get('x-request-id')).toBe('outer-request-id') - expect(response.headers.get('X-RateLimit-Limit')).toBe('400') - expect(mockLoggerError).toHaveBeenCalledWith( - 'outer-request-id', - 'Unhandled route error', - expect.objectContaining({ error: 'handler failed' }) - ) - }) -}) diff --git a/apps/sim/app/api/public-api-route-handler.ts b/apps/sim/app/api/public-api-route-handler.ts deleted file mode 100644 index 99beb0f48d8..00000000000 --- a/apps/sim/app/api/public-api-route-handler.ts +++ /dev/null @@ -1,76 +0,0 @@ -import type { NextRequest, NextResponse } from 'next/server' -import type { AnyApiRouteContract } from '@/lib/api/contracts' -import { type ParsedRequest, type ParseRequestOptions, parseRequest } from '@/lib/api/server' -import { generateRequestId } from '@/lib/core/utils/request' -import { withRouteHandler } from '@/lib/core/utils/with-route-handler' -import { type ApiEndpoint, type AuthorizedRequest, checkRateLimit } from '@/app/api/v1/middleware' -import { v2Error, v2RateLimitError, v2ValidationError } from '@/app/api/v2/lib/response' - -interface PublicApiRouteContext { - params?: - | Promise> - | Record -} - -interface PublicApiRouteHandlerArguments { - request: NextRequest - input: ParsedRequest - auth: AuthorizedRequest -} - -interface PublicApiRouteHandlerOptions { - contract: C - rateLimitEndpoint: ApiEndpoint - parseOptions?: ParseRequestOptions - handler: ( - arguments_: PublicApiRouteHandlerArguments - ) => Promise | NextResponse | Response -} - -type PublicApiNextRouteHandler = ( - request: NextRequest, - context?: PublicApiRouteContext -) => Promise - -/** - * Wraps an API-key-authenticated public route with request context, rate - * limiting, authentication, and contract parsing before invoking the route's - * authorization and business logic. Unexpected endpoint errors are logged once - * by the shared route handler and rendered as the canonical v2 500 envelope. - */ -export function withPublicApiRouteHandler({ - contract, - rateLimitEndpoint, - parseOptions, - handler, -}: PublicApiRouteHandlerOptions): PublicApiNextRouteHandler { - const wrapped = withRouteHandler( - async (request, context) => { - const requestId = generateRequestId() - const rateLimit = await checkRateLimit(request, rateLimitEndpoint) - if (!rateLimit.allowed) return v2RateLimitError(rateLimit) - - if (!rateLimit.userId) { - throw new Error('Allowed public API request is missing a user ID') - } - const userId = rateLimit.userId - - const parsed = await parseRequest(contract, request, context ?? {}, { - validationErrorResponse: v2ValidationError, - ...parseOptions, - }) - if (!parsed.success) return parsed.response - - return handler({ - request, - input: parsed.data, - auth: { requestId, userId, rateLimit }, - }) - }, - { - unhandledErrorResponse: () => v2Error('INTERNAL_ERROR', 'Internal server error'), - } - ) - - return async (request, context) => wrapped(request, context) -} diff --git a/apps/sim/app/api/table/row-wire.ts b/apps/sim/app/api/table/row-wire.ts index 880f5f5695e..af9b61658e2 100644 --- a/apps/sim/app/api/table/row-wire.ts +++ b/apps/sim/app/api/table/row-wire.ts @@ -1,78 +1,9 @@ import type { SessionPrincipal, WorkflowExecutionDelegatedPrincipal } from '@sim/auth/principal' -import { AuthType, type AuthTypeValue } from '@/lib/auth/hybrid' -import type { - Filter, - RowData, - Sort, - SortSpec, - TablePredicate, - TableRow, - TableSchema, -} from '@/lib/table' +import type { TableRow, TableSchema } from '@/lib/table' import type { TableRowDataKeying } from '@/lib/table/application/rows' import { namedRowMapper } from '@/lib/table/cell-format' -import { - buildIdByName, - filterNamesToIds, - rowDataNameToId, - sortNamesToIds, - sortSpecNamesToIds, -} from '@/lib/table/column-keys' -import { predicateToStorage, resolveFilterSelectValues } from '@/lib/table/select-values' import { toWireTimestamp } from '@/lib/table/wire' -export interface RowWireTranslators { - /** Inbound row data: wire keys → storage column ids. */ - dataIn: (data: RowData) => RowData - /** Outbound row data: storage column ids → wire keys. */ - dataOut: (data: RowData) => RowData - /** Inbound filter: wire field refs → storage column ids. */ - filterIn: (filter: Filter) => Filter - /** Inbound sort: wire field refs → storage column ids. */ - sortIn: (sort: Sort) => Sort - /** Inbound v2 predicate: wire field refs → storage column ids. */ - predicateIn: (predicate: TablePredicate) => TablePredicate - /** Inbound v2 sort spec: wire field refs → storage column ids. */ - sortSpecIn: (sort: SortSpec) => SortSpec -} - -/** - * Wire-keying translators for the internal table row routes, which serve two - * caller kinds: the first-party UI (session auth) speaks stable column ids and - * passes through untouched, while workflow tool executions (internal JWT) speak - * column names — tool enrichment surfaces names to the LLM — and translate - * name↔id at this boundary, mirroring the public v1 routes. - */ -export function rowWireTranslators( - authType: AuthTypeValue | undefined, - schema: TableSchema -): RowWireTranslators { - if (authType !== AuthType.INTERNAL_JWT) { - const identity = (value: T): T => value - return { - dataIn: identity, - dataOut: identity, - filterIn: identity, - sortIn: identity, - predicateIn: identity, - sortSpecIn: identity, - } - } - const idByName = buildIdByName(schema) - return { - dataOut: namedRowMapper(schema.columns), - dataIn: (data) => rowDataNameToId(data, idByName), - // Rekey field refs name → id, then resolve select operand names → ids. Both - // grammars need that second step: a select cell stores an option id, so a - // filter written with the option NAME matches nothing without it. - filterIn: (filter) => - resolveFilterSelectValues(filterNamesToIds(filter, idByName), schema.columns), - sortIn: (sort) => sortNamesToIds(sort, idByName), - predicateIn: (predicate) => predicateToStorage(predicate, schema), - sortSpecIn: (sort) => sortSpecNamesToIds(sort, idByName), - } -} - /** * The principal kinds the internal table row routes admit — the auth policy * yields exactly these two. Typed as the union rather than `Principal` so a @@ -108,8 +39,6 @@ export function presentRowForPrincipal( schema: TableSchema, principal: TableRowRoutePrincipal ) { - // Only the outbound mapper is needed here; building the full translator set - // would also index the schema name→id for inbound paths a presenter cannot reach. const dataOut = rowKeyingForPrincipal(principal) === 'names' ? namedRowMapper(schema.columns) : identity return { diff --git a/apps/sim/lib/api/server/index.ts b/apps/sim/lib/api/server/index.ts index 23766528140..698c8e52f51 100644 --- a/apps/sim/lib/api/server/index.ts +++ b/apps/sim/lib/api/server/index.ts @@ -1,2 +1 @@ -export * from './tool-validation' export * from './validation' diff --git a/apps/sim/lib/api/server/tool-validation.ts b/apps/sim/lib/api/server/tool-validation.ts deleted file mode 100644 index a9dca78ac2c..00000000000 --- a/apps/sim/lib/api/server/tool-validation.ts +++ /dev/null @@ -1,75 +0,0 @@ -import { type NextRequest, NextResponse } from 'next/server' -import type { AnyApiRouteContract } from '@/lib/api/contracts' -import { parseRequest } from '@/lib/api/server/validation' - -export type ToolValidationErrorFormat = 'firstError' | 'details' | 'toolDetails' - -interface ToolValidationLogger { - warn(message: string, metadata?: Record): void -} - -export interface ParseToolRequestOptions { - errorFormat?: ToolValidationErrorFormat - logger?: ToolValidationLogger - logMessage?: string -} - -/** - * Parse a tool route request against its contract and produce a tool-shaped - * 400 response on validation failure. - * - * Three error envelope variants are supported via `errorFormat`: - * - `firstError` → `{ error: }` - * - `details` → `{ error: 'Invalid request data', details: }` (default) - * - `toolDetails` → `{ success: false, error: 'Invalid request data', details: }` - * - * For `toolDetails`, an invalid-JSON body is also wrapped as - * `{ success: false, error: 'Request body must be valid JSON' }` so the caller - * sees a consistent envelope across both failure modes. The other formats fall - * back to the default `{ error: 'Request body must be valid JSON' }` shape. - */ -export async function parseToolRequest( - contract: C, - request: NextRequest, - options: ParseToolRequestOptions = {} -) { - const errorFormat: ToolValidationErrorFormat = options.errorFormat ?? 'details' - - return parseRequest( - contract, - request, - {}, - { - invalidJsonResponse: - errorFormat === 'toolDetails' - ? () => - NextResponse.json( - { success: false, error: 'Request body must be valid JSON' }, - { status: 400 } - ) - : undefined, - validationErrorResponse: (error) => { - options.logger?.warn(options.logMessage ?? 'Invalid request data', { errors: error.issues }) - - if (errorFormat === 'firstError') { - return NextResponse.json( - { error: error.issues[0]?.message ?? 'Invalid request' }, - { status: 400 } - ) - } - - if (errorFormat === 'toolDetails') { - return NextResponse.json( - { success: false, error: 'Invalid request data', details: error.issues }, - { status: 400 } - ) - } - - return NextResponse.json( - { error: 'Invalid request data', details: error.issues }, - { status: 400 } - ) - }, - } - ) -} diff --git a/apps/sim/lib/credential-groups/application/create-invite-link.test.ts b/apps/sim/lib/credential-groups/application/create-invite-link.test.ts deleted file mode 100644 index d349a619dc0..00000000000 --- a/apps/sim/lib/credential-groups/application/create-invite-link.test.ts +++ /dev/null @@ -1,145 +0,0 @@ -import { - createExecutorPrincipal, - createSessionPrincipal, -} from '@sim/testing/factories/principal.factory' -import { - credentialGroupsEnrollmentsMock, - credentialGroupsEnrollmentsMockFns, -} from '@sim/testing/mocks/credential-groups-enrollments.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const hoisted = vi.hoisted(() => ({ - requireAvailable: vi.fn(), - resolveGroup: vi.fn(), -})) - -vi.mock('@/lib/credential-groups/application/context', () => ({ - requireCredentialGroupsAvailable: hoisted.requireAvailable, - resolveWorkspaceAccountsContext: hoisted.resolveGroup, -})) - -vi.mock('@/lib/credential-groups/enrollments', () => credentialGroupsEnrollmentsMock) - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -import { createCredentialGroupInviteLink } from '@/lib/credential-groups/application/create-invite-link' - -const mocks = { - ...hoisted, - createInvitationLink: credentialGroupsEnrollmentsMockFns.mockCreateCredentialGroupInvitationLink, -} - -const resolvePermission = workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission - -const context = { - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - credentialGroupId: 'group-1', - name: 'Support', - status: 'active' as const, - options: [], -} - -function executorPrincipal(workspaceId = 'workspace-1') { - return createExecutorPrincipal({ - subjectUserId: 'admin-1', - workspaceId, - audience: 'sim:credential-groups', - delegationContext: { kind: 'workflow_execution', workflowId: 'workflow-1' }, - }) -} - -describe('createCredentialGroupInviteLink', () => { - beforeEach(() => { - mocks.resolveGroup.mockResolvedValue(context) - resolvePermission.mockResolvedValue('admin') - mocks.requireAvailable.mockResolvedValue(undefined) - mocks.createInvitationLink.mockResolvedValue({ - enrollment: { - id: 'enrollment-1', - email: 'person@example.com', - status: 'invited', - }, - invitationLink: 'https://sim.ai/credential-groups/enroll/token-1', - }) - }) - - it('rejects unsupported principals before loading the group', async () => { - const principal = createSessionPrincipal({ userId: 'admin-1' }) - - await expect( - createCredentialGroupInviteLink.execute({ - principal, - input: { workspaceId: 'workspace-1', email: 'person@example.com' }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - expect(mocks.resolveGroup).not.toHaveBeenCalled() - }) - - it('issues an unattributed link for an actorless run', async () => { - // A schedule (or a webhook with no external subject) reaches this with a real - // admin-scoped delegation and no person on it. The delegation is the authority; - // the issuer is only recorded, and `created_by` is nullable — so this issues the - // link with no issuer rather than refusing, which is what it did when the - // subject was demanded here. - const { subjectUserId: _subject, ...base } = executorPrincipal() - // What actually authorizes an actorless caller: the delegation is running a - // deployment. No user is consulted anywhere in that decision. - const actorless = { - ...base, - delegationContext: { - kind: 'workflow_execution' as const, - workflowId: 'workflow-1', - principal: { - kind: 'system' as const, - serviceId: 'schedule' as const, - workspaceId: 'workspace-1', - workflowId: 'workflow-1', - }, - currentWorkflow: { - workflowId: 'workflow-1', - mode: 'deployment' as const, - deploymentVersionId: 'version-1', - }, - }, - } - - const result = await createCredentialGroupInviteLink.execute({ - principal: actorless, - input: { workspaceId: 'workspace-1', email: 'person@example.com' }, - }) - - expect(result.invitationLink).toBe('https://sim.ai/credential-groups/enroll/token-1') - expect(mocks.createInvitationLink).toHaveBeenCalledWith( - 'workspace-1', - 'group-1', - undefined, - 'person@example.com' - ) - }) - - it('rejects delegation scoped to another workspace', async () => { - await expect( - createCredentialGroupInviteLink.execute({ - principal: executorPrincipal('workspace-2'), - input: { workspaceId: 'workspace-1', email: 'person@example.com' }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - expect(mocks.createInvitationLink).not.toHaveBeenCalled() - }) - - it('requires the current subject to remain a workspace admin', async () => { - resolvePermission.mockResolvedValue('write') - - await expect( - createCredentialGroupInviteLink.execute({ - principal: executorPrincipal(), - input: { workspaceId: 'workspace-1', email: 'person@example.com' }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - expect(mocks.createInvitationLink).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/credential-groups/application/create-invite-link.ts b/apps/sim/lib/credential-groups/application/create-invite-link.ts deleted file mode 100644 index d731629be96..00000000000 --- a/apps/sim/lib/credential-groups/application/create-invite-link.ts +++ /dev/null @@ -1,68 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { resolvePrincipalSubjectUserId } from '@sim/auth/principal' -import { isValidEmailSyntax, normalizeEmail } from '@sim/utils/string' -import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { credentialGroupDelegationPolicy } from '@/lib/credential-groups/application/authorization' -import { - requireCredentialGroupsAvailable, - resolveWorkspaceAccountsContext, -} from '@/lib/credential-groups/application/context' -import { credentialGroupOperations } from '@/lib/credential-groups/application/operations' -import { - CredentialGroupEnrollmentError, - createCredentialGroupInvitationLink, -} from '@/lib/credential-groups/enrollments' - -export interface CreateCredentialGroupInviteLinkInput { - workspaceId: string - email: string -} - -/** Issues a fresh bearer invitation link without delivering an email. */ -export const createCredentialGroupInviteLink = defineAuthorizedWorkspaceUseCase({ - operation: credentialGroupOperations.createInviteLink, - resolveContext: ({ input }: { input: CreateCredentialGroupInviteLinkInput }) => - resolveWorkspaceAccountsContext(input.workspaceId), - authorizationOptions: { delegation: credentialGroupDelegationPolicy }, - execute: async ({ principal, input, context }) => { - if (context.status !== 'active') { - throw new OrchestrationError('conflict', 'Credential group is disabled') - } - const email = normalizeEmail(input.email) - if (!isValidEmailSyntax(email)) { - throw new OrchestrationError('validation', 'Email must be a valid address') - } - await requireCredentialGroupsAvailable(context.workspaceId) - - try { - return await createCredentialGroupInvitationLink( - context.workspaceId, - context.credentialGroupId, - // Attribution, not authority: the delegation's admin-scoped Credential Group - // grant is what permits this. An actorless run records no issuer. - resolvePrincipalSubjectUserId(principal), - email - ) - } catch (error) { - if (error instanceof CredentialGroupEnrollmentError) { - throw new OrchestrationError( - error.status === 404 ? 'not_found' : error.status === 409 ? 'conflict' : 'internal', - error.message - ) - } - throw error - } - }, - projectAudit: ({ context, result }) => ({ - action: AuditAction.CREDENTIAL_GROUP_UPDATED, - resourceType: AuditResourceType.CREDENTIAL_GROUP, - resourceId: context.credentialGroupId, - resourceName: context.name, - description: `Generated an invitation link for ${result.enrollment.email}`, - metadata: { - email: result.enrollment.email, - enrollmentId: result.enrollment.id, - }, - }), -}) diff --git a/apps/sim/lib/credential-groups/application/manage-groups.test.ts b/apps/sim/lib/credential-groups/application/manage-groups.test.ts index 4488867f32c..75cdcccb137 100644 --- a/apps/sim/lib/credential-groups/application/manage-groups.test.ts +++ b/apps/sim/lib/credential-groups/application/manage-groups.test.ts @@ -41,8 +41,6 @@ import { getCredentialGroupSettings, getWorkspaceAccountsSettings, } from '@/lib/credential-groups/application/manage-groups' -import { loadCopilotConnectedAccounts } from '@/lib/mothership/application/load-connected-accounts' -import { requireTrustedCopilotExecutionContext } from '@/lib/mothership/auth/application-delegation' const mocks = { ...hoisted, @@ -70,14 +68,6 @@ const enrollmentPrincipal: CredentialGroupEnrollmentPrincipal = { invitationTokenHash: 'hash-1', } -const copilotContext = requireTrustedCopilotExecutionContext({ - userId: 'admin-1', - workspaceId: 'workspace-1', - toolCallId: 'tool-1', - copilotToolExecution: true, - chatId: 'chat-1', -}) - function copilotPrincipal(overrides: Partial = {}): DelegatedPrincipal { return { ...createDelegatedPrincipal({ @@ -133,27 +123,6 @@ describe('Credential Group Settings application operations', () => { expect(mocks.list).not.toHaveBeenCalled() }) - it('reauthorizes connected-account reads after the acting admin is demoted', async () => { - await loadCopilotConnectedAccounts(copilotContext) - mocks.list.mockClear() - resolvePermission.mockResolvedValue('read') - await expect(loadCopilotConnectedAccounts(copilotContext)).rejects.toMatchObject({ - code: 'forbidden', - }) - expect(mocks.list).not.toHaveBeenCalled() - expect(mocks.listEnrollments).not.toHaveBeenCalled() - }) - - it('rechecks account availability on each authorized read', async () => { - await loadCopilotConnectedAccounts(copilotContext) - mocks.list.mockClear() - mocks.requireAvailable.mockRejectedValue(new OrchestrationError('not_found', 'Unavailable')) - await expect(loadCopilotConnectedAccounts(copilotContext)).rejects.toMatchObject({ - code: 'not_found', - }) - expect(mocks.list).not.toHaveBeenCalled() - }) - it.each([ { audience: 'sim:knowledge' }, { workspaceId: 'other-workspace' }, diff --git a/apps/sim/lib/credential-groups/application/operations.ts b/apps/sim/lib/credential-groups/application/operations.ts index fd9a51e9777..c3c6e40c01b 100644 --- a/apps/sim/lib/credential-groups/application/operations.ts +++ b/apps/sim/lib/credential-groups/application/operations.ts @@ -121,24 +121,6 @@ export const credentialGroupOperations = { principalKinds: ['delegated'], delegatedServices: ['executor'], }), - // permission-group-exempt: enrolls an outside person in a credential group, not a member in a workspace, so invitations.send does not name it - sendInvite: defineWorkspaceOperation({ - id: 'credential_groups.invites.send', - minimumRole: 'admin', - workspaceApiKey: 'deny', - capability: 'none', - principalKinds: ['delegated'], - delegatedServices: ['executor'], - }), - // permission-group-exempt: mints an enrollment link for an outside person, not a workspace invitation, so invitations.send does not name it - createInviteLink: defineWorkspaceOperation({ - id: 'credential_groups.invites.link.create', - minimumRole: 'admin', - workspaceApiKey: 'deny', - capability: 'none', - principalKinds: ['delegated'], - delegatedServices: ['executor'], - }), // permission-group-exempt: workspace admin already decides this, and no group key names the credential-groups section startSlackConfiguration: defineWorkspaceOperation({ id: 'credential_groups.slack_configuration.start', diff --git a/apps/sim/lib/credential-groups/application/send-invite.test.ts b/apps/sim/lib/credential-groups/application/send-invite.test.ts deleted file mode 100644 index 7e6f0373f9a..00000000000 --- a/apps/sim/lib/credential-groups/application/send-invite.test.ts +++ /dev/null @@ -1,168 +0,0 @@ -import type { WorkflowExecutionDelegatedPrincipal } from '@sim/auth/principal' -import { - createExecutorPrincipal, - createSessionPrincipal, -} from '@sim/testing/factories/principal.factory' -import { - credentialGroupsEnrollmentsMock, - credentialGroupsEnrollmentsMockFns, -} from '@sim/testing/mocks/credential-groups-enrollments.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const hoisted = vi.hoisted(() => ({ - requireAvailable: vi.fn(), - resolveGroup: vi.fn(), -})) - -vi.mock('@/lib/credential-groups/application/context', () => ({ - requireCredentialGroupsAvailable: hoisted.requireAvailable, - resolveWorkspaceAccountsContext: hoisted.resolveGroup, -})) - -vi.mock('@/lib/credential-groups/enrollments', () => credentialGroupsEnrollmentsMock) - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -import { sendCredentialGroupInvite } from '@/lib/credential-groups/application/send-invite' - -const mocks = { - ...hoisted, - inviteEnrollment: credentialGroupsEnrollmentsMockFns.mockInviteCredentialGroupEnrollment, - loadInviter: credentialGroupsEnrollmentsMockFns.mockLoadCredentialGroupInviterIdentity, -} - -const resolvePermission = workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission - -const context = { - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - credentialGroupId: 'group-1', - name: 'Support', - status: 'active' as const, - options: [], -} - -function executorPrincipal(): WorkflowExecutionDelegatedPrincipal { - return createExecutorPrincipal({ - subjectUserId: 'admin-1', - audience: 'sim:credential-groups', - delegationContext: { - kind: 'workflow_execution', - workflowId: 'workflow-1', - principal: createSessionPrincipal({ userId: 'admin-1' }), - currentWorkflow: { workflowId: 'workflow-1', mode: 'draft' }, - }, - }) -} - -/** A deployed run whose only actor is the external identity that triggered it. */ -function unattendedPrincipal( - principal: NonNullable['principal'] -): WorkflowExecutionDelegatedPrincipal { - const { subjectUserId: _subject, ...base } = executorPrincipal() - return { - ...base, - delegationContext: { - kind: 'workflow_execution', - workflowId: 'workflow-1', - principal, - currentWorkflow: { - workflowId: 'workflow-1', - mode: 'deployment', - deploymentVersionId: 'version-1', - }, - }, - } -} - -function slackPrincipal(): WorkflowExecutionDelegatedPrincipal { - return unattendedPrincipal({ - kind: 'system', - serviceId: 'webhook', - workspaceId: 'workspace-1', - workflowId: 'workflow-1', - webhookId: 'webhook-1', - provider: 'slack', - subject: { kind: 'external_user', provider: 'slack', tenantId: 'T123', subjectId: 'U123' }, - }) -} - -function invite(principal: WorkflowExecutionDelegatedPrincipal) { - return sendCredentialGroupInvite.execute({ - principal, - input: { workspaceId: 'workspace-1', email: 'person@example.com' }, - }) -} - -describe('sendCredentialGroupInvite', () => { - beforeEach(() => { - mocks.resolveGroup.mockResolvedValue(context) - resolvePermission.mockResolvedValue('admin') - mocks.requireAvailable.mockResolvedValue(undefined) - mocks.loadInviter.mockResolvedValue({ name: 'Ada Lovelace', email: 'ada@example.com' }) - mocks.inviteEnrollment.mockResolvedValue({ - id: 'enrollment-1', - email: 'person@example.com', - status: 'invited', - }) - }) - - it('invites without naming an inviter on a Slack-triggered run', async () => { - const result = await invite(slackPrincipal()) - - expect(result.enrollment.id).toBe('enrollment-1') - expect(mocks.loadInviter).not.toHaveBeenCalled() - expect(mocks.inviteEnrollment).toHaveBeenCalledWith( - 'workspace-1', - 'group-1', - undefined, - undefined, - 'person@example.com' - ) - }) - - it('names the human a session-actor run acts as', async () => { - await invite(executorPrincipal()) - - expect(mocks.loadInviter).toHaveBeenCalledWith('admin-1') - expect(mocks.inviteEnrollment).toHaveBeenCalledWith( - 'workspace-1', - 'group-1', - 'admin-1', - 'Ada Lovelace', - 'person@example.com' - ) - }) - - it('falls back to the inviter email when they have no name', async () => { - mocks.loadInviter.mockResolvedValue({ name: ' ', email: 'ada@example.com' }) - - await invite(executorPrincipal()) - - expect(mocks.inviteEnrollment).toHaveBeenCalledWith( - 'workspace-1', - 'group-1', - 'admin-1', - 'ada@example.com', - 'person@example.com' - ) - }) - - it('requires the current subject to remain a workspace admin', async () => { - resolvePermission.mockResolvedValue('write') - - await expect(invite(executorPrincipal())).rejects.toMatchObject({ code: 'forbidden' }) - expect(mocks.inviteEnrollment).not.toHaveBeenCalled() - }) - - it('rejects a delegation asserting a subject its run never had', async () => { - const spoofed = slackPrincipal() - spoofed.subjectUserId = 'invented-user' - - await expect(invite(spoofed)).rejects.toMatchObject({ code: 'forbidden' }) - expect(mocks.inviteEnrollment).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/credential-groups/application/send-invite.ts b/apps/sim/lib/credential-groups/application/send-invite.ts deleted file mode 100644 index 8fb9e40bd97..00000000000 --- a/apps/sim/lib/credential-groups/application/send-invite.ts +++ /dev/null @@ -1,77 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { resolvePrincipalSubjectUserId } from '@sim/auth/principal' -import { isValidEmailSyntax, normalizeEmail } from '@sim/utils/string' -import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { - credentialGroupDelegationPolicy, - requireCredentialGroupWorkflowActor, -} from '@/lib/credential-groups/application/authorization' -import { - requireCredentialGroupsAvailable, - resolveWorkspaceAccountsContext, -} from '@/lib/credential-groups/application/context' -import { credentialGroupOperations } from '@/lib/credential-groups/application/operations' -import { - CredentialGroupEnrollmentError, - inviteCredentialGroupEnrollment, - loadCredentialGroupInviterIdentity, -} from '@/lib/credential-groups/enrollments' - -export interface SendCredentialGroupInviteInput { - workspaceId: string - email: string -} - -export const sendCredentialGroupInvite = defineAuthorizedWorkspaceUseCase({ - operation: credentialGroupOperations.sendInvite, - resolveContext: ({ input }: { input: SendCredentialGroupInviteInput }) => - resolveWorkspaceAccountsContext(input.workspaceId), - authorizationOptions: { delegation: credentialGroupDelegationPolicy }, - authorizeResource({ principal }) { - requireCredentialGroupWorkflowActor(principal) - }, - execute: async ({ principal, input, context }) => { - if (context.status !== 'active') { - throw new OrchestrationError('conflict', 'Credential group is disabled') - } - const email = normalizeEmail(input.email) - if (!isValidEmailSyntax(email)) { - throw new OrchestrationError('validation', 'Email must be a valid address') - } - await requireCredentialGroupsAvailable(context.workspaceId) - - // Attribution, not authority. An actorless or Slack-triggered run names no - // inviter rather than borrowing its actor, so the email claims no one invited. - const userId = resolvePrincipalSubjectUserId(principal) - const inviter = userId ? await loadCredentialGroupInviterIdentity(userId) : null - const inviterName = inviter?.name?.trim() || inviter?.email - - try { - const enrollment = await inviteCredentialGroupEnrollment( - context.workspaceId, - context.credentialGroupId, - userId, - inviterName, - email - ) - return { enrollment } - } catch (error) { - if (error instanceof CredentialGroupEnrollmentError) { - throw new OrchestrationError( - error.status === 404 ? 'not_found' : error.status === 409 ? 'conflict' : 'internal', - error.message - ) - } - throw error - } - }, - projectAudit: ({ input, context, result }) => ({ - action: AuditAction.CREDENTIAL_GROUP_UPDATED, - resourceType: AuditResourceType.CREDENTIAL_GROUP, - resourceId: context.credentialGroupId, - resourceName: context.name, - description: `Invited ${result.enrollment.email} to connect accounts`, - metadata: { email: normalizeEmail(input.email), enrollmentId: result.enrollment.id }, - }), -}) diff --git a/apps/sim/lib/credential-groups/enrollments.test.ts b/apps/sim/lib/credential-groups/enrollments.test.ts index 49391fd47b3..95e82372725 100644 --- a/apps/sim/lib/credential-groups/enrollments.test.ts +++ b/apps/sim/lib/credential-groups/enrollments.test.ts @@ -45,7 +45,6 @@ import { isFeatureEnabled } from '@/lib/core/config/feature-flags' import { bindCredentialGroupEnrollmentUser, completeCredentialGroupEnrollment, - createCredentialGroupInvitationLink, createCredentialGroupSelfEnrollmentLink, deleteCredentialGroupEnrollment, getAuthorizedCredentialGroupOAuthContext, @@ -362,61 +361,6 @@ describe('listCredentialGroupEnrollments', () => { }) }) -describe('createCredentialGroupInvitationLink', () => { - beforeEach(() => { - resetDbChainMock() - }) - - it('issues a fresh enrollment token without sending email', async () => { - const issued = { - ...ENROLLMENT, - email: 'person@example.com', - status: 'invited' as const, - sentAt: null, - completedAt: null, - } - dbChainMockFns.limit - .mockResolvedValueOnce([ - { - workspaceId: 'workspace-1', - workspaceName: 'Workspace', - groupId: 'group-1', - groupName: 'Group', - groupStatus: 'active', - options: [{ id: 'option-1', status: 'active' }], - }, - ]) - .mockResolvedValueOnce([]) - dbChainMockFns.returning.mockResolvedValueOnce([issued]) - - const result = await createCredentialGroupInvitationLink( - 'workspace-1', - 'group-1', - 'user-1', - ' Person@Example.COM ' - ) - - expect(result.enrollment).toMatchObject({ - id: ENROLLMENT.id, - email: 'person@example.com', - status: 'invited', - sentAt: null, - }) - expect(new URL(result.invitationLink).pathname).toMatch( - /^\/credential-groups\/enroll\/[0-9a-f-]+$/ - ) - expect(dbChainMockFns.values).toHaveBeenCalledWith( - expect.objectContaining({ - credentialGroupId: 'group-1', - email: 'person@example.com', - createdBy: 'user-1', - sentAt: null, - }) - ) - expect(sendEmail).not.toHaveBeenCalled() - }) -}) - describe('verified self enrollment', () => { beforeEach(() => { resetDbChainMock() @@ -446,14 +390,6 @@ describe('verified self enrollment', () => { ) }) - it('continues to require an account type for external invitations', async () => { - dbChainMockFns.limit.mockResolvedValueOnce([group]).mockResolvedValueOnce([]) - await expect( - createCredentialGroupInvitationLink('workspace-1', 'group-1', 'admin', ENROLLMENT.email) - ).rejects.toThrow('Add an account type') - expect(dbChainMockFns.insert).not.toHaveBeenCalled() - }) - it('refuses a disabled group for self enrollment', async () => { dbChainMockFns.limit.mockResolvedValueOnce([{ ...group, groupStatus: 'disabled' }]) await expect( diff --git a/apps/sim/lib/credential-groups/enrollments.ts b/apps/sim/lib/credential-groups/enrollments.ts index 4a06c489db3..2d1299075aa 100644 --- a/apps/sim/lib/credential-groups/enrollments.ts +++ b/apps/sim/lib/credential-groups/enrollments.ts @@ -968,26 +968,6 @@ export async function createCredentialGroupSelfEnrollmentLink( } } -export async function createCredentialGroupInvitationLink( - scopeInput: string | ResourceScope, - groupId: string, - /** See {@link issueInvitation}: the issuer is attribution, never the authority. */ - userId: string | undefined, - email: string, - /** See {@link inviteCredentialGroupEnrollment}. */ - revokedEnrollment: RevokedEnrollmentPolicy = 'reactivate' -): Promise { - const scope = credentialGroupScope(scopeInput) - const context = await getInvitationContext(scope, groupId) - const issued = await issueInvitation(context, userId, normalizeEmail(email), { - revokedEnrollment, - }) - return { - enrollment: toCredentialGroupEnrollment(issued.enrollment), - invitationLink: issued.invitationLink, - } -} - export async function resendCredentialGroupEnrollment( scopeInput: string | ResourceScope, groupId: string, diff --git a/apps/sim/lib/credential-groups/rate-limit.ts b/apps/sim/lib/credential-groups/rate-limit.ts index 373531c0ad5..c1865c3e5b8 100644 --- a/apps/sim/lib/credential-groups/rate-limit.ts +++ b/apps/sim/lib/credential-groups/rate-limit.ts @@ -136,15 +136,3 @@ export async function enforceCredentialGroupInvitationRouteRateLimit( ) } } - -/** Applies the shared invitation budget to non-HTTP workflow execution. */ -export async function enforceCredentialGroupInvitationExecutionRateLimit( - workspaceId: string -): Promise { - const result = await rateLimiter.checkRateLimitDirect( - credentialGroupInvitationRateLimitKey(workspaceId), - CREDENTIAL_GROUP_INVITATION_RATE_LIMIT, - { failClosed: true } - ) - if (!result.allowed) throw new RateLimitError('Credential Group invitation rate limit exceeded') -} diff --git a/apps/sim/lib/credentials/application/delete-many-credentials.test.ts b/apps/sim/lib/credentials/application/delete-many-credentials.test.ts deleted file mode 100644 index 6e7ab663804..00000000000 --- a/apps/sim/lib/credentials/application/delete-many-credentials.test.ts +++ /dev/null @@ -1,112 +0,0 @@ -import { createDelegatedPrincipal } from '@sim/testing/factories/principal.factory' -import { - credentialsAccessMock, - credentialsAccessMockFns, -} from '@sim/testing/mocks/credentials-access.mock' -import { posthogServerMock } from '@sim/testing/mocks/posthog-server.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { - workspaceContextMock, - workspaceContextMockFns, -} from '@sim/testing/mocks/workspace-context.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const hoisted = vi.hoisted(() => ({ - deleteCredential: vi.fn(), -})) - -vi.mock('@/lib/workspaces/application/workspace-context', () => workspaceContextMock) -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) -vi.mock('@/lib/credentials/access', () => credentialsAccessMock) -vi.mock('@/lib/credentials/orchestration', () => ({ - deleteCredentialRecord: hoisted.deleteCredential, -})) -vi.mock('@/lib/posthog/server', () => posthogServerMock) - -import { deleteManyCredentialsUseCase } from '@/lib/credentials/application/delete-many-credentials' - -const mocks = { - ...hoisted, - getActor: credentialsAccessMockFns.mockGetCredentialActorContext, - resolvePermission: workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission, -} - -const workspace = { - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', -} -const principal = createDelegatedPrincipal({ audience: 'sim:credentials' }) - -function oauthCredential(id: string, workspaceId = 'workspace-1') { - return { - id, - workspaceId, - type: 'oauth' as const, - displayName: `OAuth ${id}`, - description: null, - providerId: 'google-email', - accountId: `account-${id}`, - envKey: null, - envOwnerUserId: null, - encryptedServiceAccountKey: null, - createdBy: 'user-1', - createdAt: new Date('2026-08-14T12:00:00.000Z'), - updatedAt: new Date('2026-08-14T12:00:00.000Z'), - } -} - -describe('deleteManyCredentialsUseCase', () => { - beforeEach(() => { - workspaceContextMockFns.mockLoadActiveWorkspaceApplicationContext.mockResolvedValue(workspace) - mocks.resolvePermission.mockResolvedValue('read') - mocks.deleteCredential.mockResolvedValue(true) - }) - - it('deletes only OAuth credentials administered in the delegated workspace', async () => { - const allowed = oauthCredential('credential-1') - mocks.getActor - .mockResolvedValueOnce({ - credential: allowed, - member: { role: 'admin' }, - hasWorkspaceAccess: true, - isAdmin: true, - }) - .mockResolvedValueOnce({ - credential: oauthCredential('credential-2', 'workspace-2'), - member: { role: 'admin' }, - hasWorkspaceAccess: true, - isAdmin: true, - }) - - const result = await deleteManyCredentialsUseCase.execute({ - principal, - input: { - workspaceId: 'workspace-1', - credentialIds: ['credential-1', 'credential-2'], - }, - }) - - expect(result).toEqual({ - deleted: ['credential-1'], - failed: ['credential-2'], - deletedCredentials: [allowed], - }) - expect(mocks.deleteCredential).toHaveBeenCalledOnce() - }) - - it('rejects duplicate IDs before loading any credential', async () => { - await expect( - deleteManyCredentialsUseCase.execute({ - principal, - input: { - workspaceId: 'workspace-1', - credentialIds: ['credential-1', 'credential-1'], - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - expect(mocks.getActor).not.toHaveBeenCalled() - expect(mocks.deleteCredential).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/credentials/application/delete-many-credentials.ts b/apps/sim/lib/credentials/application/delete-many-credentials.ts deleted file mode 100644 index 993f1795ef2..00000000000 --- a/apps/sim/lib/credentials/application/delete-many-credentials.ts +++ /dev/null @@ -1,114 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { requirePrincipalSubjectUserId } from '@sim/auth/principal' -import { createLogger } from '@sim/logger' -import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { getCredentialActorContext } from '@/lib/credentials/access' -import { credentialDelegationPolicy } from '@/lib/credentials/application/authorization' -import { credentialOperations } from '@/lib/credentials/application/operations' -import { deleteCredentialRecord } from '@/lib/credentials/orchestration' -import type { CredentialRow } from '@/lib/credentials/queries' -import { captureServerEvent } from '@/lib/posthog/server' -import { loadActiveWorkspaceApplicationContext } from '@/lib/workspaces/application/workspace-context' - -const logger = createLogger('DeleteManyCredentialsApplication') -const MAX_CREDENTIAL_DELETE_BATCH = 20 - -export interface DeleteManyCredentialsInput { - workspaceId: string - credentialIds: string[] -} - -export interface DeleteManyCredentialsResult { - deleted: string[] - failed: string[] - deletedCredentials: CredentialRow[] -} - -export const deleteManyCredentialsUseCase = defineAuthorizedWorkspaceUseCase({ - operation: credentialOperations.deleteMany, - resolveContext: async ({ input }: { input: DeleteManyCredentialsInput }) => { - const context = await loadActiveWorkspaceApplicationContext(input.workspaceId) - if (!context) throw new OrchestrationError('not_found', 'Workspace not found') - return context - }, - authorizationOptions: { delegation: credentialDelegationPolicy }, - async execute({ principal, input, context }): Promise { - if (input.credentialIds.length === 0) { - throw new OrchestrationError('validation', 'At least one credential ID is required') - } - if (input.credentialIds.length > MAX_CREDENTIAL_DELETE_BATCH) { - throw new OrchestrationError( - 'validation', - `At most ${MAX_CREDENTIAL_DELETE_BATCH} credentials can be deleted at once` - ) - } - if (new Set(input.credentialIds).size !== input.credentialIds.length) { - throw new OrchestrationError('validation', 'Credential IDs must be unique') - } - - const userId = requirePrincipalSubjectUserId(principal) - const deleted: string[] = [] - const failed: string[] = [] - const deletedCredentials: CredentialRow[] = [] - - for (const credentialId of input.credentialIds) { - try { - const access = await getCredentialActorContext(credentialId, userId) - if ( - !access.credential || - access.credential.workspaceId !== context.workspaceId || - !access.hasWorkspaceAccess || - !access.isAdmin || - access.credential.type !== 'oauth' - ) { - failed.push(credentialId) - continue - } - const didDelete = await deleteCredentialRecord({ - credential: access.credential, - reason: 'copilot_delete', - }) - if (!didDelete) { - failed.push(credentialId) - continue - } - deleted.push(credentialId) - deletedCredentials.push(access.credential) - } catch (error) { - logger.error('Failed to delete credential in Copilot batch', { credentialId, error }) - failed.push(credentialId) - } - } - - return { deleted, failed, deletedCredentials } - }, - projectAudit: ({ result }) => - result.deletedCredentials.map((credential) => ({ - action: AuditAction.CREDENTIAL_DELETED, - resourceType: AuditResourceType.CREDENTIAL, - resourceId: credential.id, - resourceName: credential.displayName, - description: `Deleted oauth credential "${credential.displayName}" (copilot_delete)`, - metadata: { - reason: 'copilot_delete', - credentialType: credential.type, - providerId: credential.providerId, - accountId: credential.accountId, - }, - })), - afterSuccess: ({ principal, context, result }) => { - for (const credential of result.deletedCredentials) { - captureServerEvent( - requirePrincipalSubjectUserId(principal), - 'credential_deleted', - { - credential_type: 'oauth', - provider_id: credential.providerId ?? credential.id, - workspace_id: context.workspaceId, - }, - { groups: { workspace: context.workspaceId } } - ) - } - }, -}) diff --git a/apps/sim/lib/credentials/application/operations.ts b/apps/sim/lib/credentials/application/operations.ts index 7a20c196f02..86d5111d7fc 100644 --- a/apps/sim/lib/credentials/application/operations.ts +++ b/apps/sim/lib/credentials/application/operations.ts @@ -183,14 +183,6 @@ export const credentialOperations = { }), 'admin' ), - deleteMany: defineWorkspaceOperation({ - id: 'credentials.delete_many', - minimumRole: 'read', - workspaceApiKey: 'deny', - capability: 'integrations.manage', - principalKinds: ['delegated'], - delegatedServices: ['copilot'], - }), saveDraft: defineWorkspaceOperation({ id: 'credentials.drafts.save', minimumRole: 'write', diff --git a/apps/sim/lib/invitations/application/manage-invitation.test.ts b/apps/sim/lib/invitations/application/manage-invitation.test.ts index ee765c282a4..bc1d7ed9ef5 100644 --- a/apps/sim/lib/invitations/application/manage-invitation.test.ts +++ b/apps/sim/lib/invitations/application/manage-invitation.test.ts @@ -48,7 +48,6 @@ import { resendInvitation, resendWorkspaceInvitation, } from '@/lib/invitations/application/manage-invitation' -import { invitationManagementErrorPolicy } from '@/lib/invitations/management-error-policy' const mocks = { orgAdmin: billingOrganizationMockFns.mockIsOrganizationOwnerOrAdmin, @@ -231,14 +230,7 @@ it.each([ { status: 404, code: 'not_found' }, { status: 409, code: 'conflict' }, { status: 502, code: 'internal' }, -])( - 'projects typed invitation status $status without losing domain metadata', - ({ status, code }) => { - const error = new InvitationManagementError(status, 'Action unavailable', true) - expect(asOrchestrationError(error)).toMatchObject({ code, message: 'Action unavailable' }) - expect(invitationManagementErrorPolicy.project(error)).toEqual({ - status, - body: { error: 'Action unavailable', upgradeRequired: true }, - }) - } -) +])('maps typed invitation status $status to its orchestration code', ({ status, code }) => { + const error = new InvitationManagementError(status, 'Action unavailable', true) + expect(asOrchestrationError(error)).toMatchObject({ code, message: 'Action unavailable' }) +}) diff --git a/apps/sim/lib/invitations/management-error-policy.ts b/apps/sim/lib/invitations/management-error-policy.ts deleted file mode 100644 index 266135e8e56..00000000000 --- a/apps/sim/lib/invitations/management-error-policy.ts +++ /dev/null @@ -1,22 +0,0 @@ -import { internalOrchestrationErrorPolicy } from '@/lib/api/server/routes' -import { ForbiddenOperationError } from '@/lib/core/application/forbidden' -import { InvitationManagementError } from '@/lib/invitations/application/manage-invitation' - -/** Preserves invitation action status and upgrade guidance across internal adapters. */ -export const invitationManagementErrorPolicy = { - project(error: unknown) { - if (error instanceof ForbiddenOperationError) - return { status: 403, body: { error: error.message, details: { code: error.detailCode } } } - if (error instanceof InvitationManagementError) - return { - status: error.status, - body: { - error: error.message, - ...(error.upgradeRequired !== undefined - ? { upgradeRequired: error.upgradeRequired } - : {}), - }, - } - return internalOrchestrationErrorPolicy.project(error) - }, -} diff --git a/apps/sim/lib/mothership/application/execute-sandbox-use-case.test.ts b/apps/sim/lib/mothership/application/execute-sandbox-use-case.test.ts deleted file mode 100644 index d15b03e77e8..00000000000 --- a/apps/sim/lib/mothership/application/execute-sandbox-use-case.test.ts +++ /dev/null @@ -1,67 +0,0 @@ -import { describe, expect, it, vi } from 'vitest' -import { customToolOperations } from '@/lib/custom-tools/application/operations' -import { executeCopilotSandboxUseCase } from '@/lib/mothership/application/execute-sandbox-use-case' -import { sandboxOperations } from '@/lib/sandboxes/application/operations' - -const trustedContext = { - userId: 'user-1', - workspaceId: 'workspace-1', - chatId: 'chat-1', - executionId: 'execution-1', - toolCallId: 'tool-call-1', - copilotToolExecution: true, -} as const - -describe('executeCopilotSandboxUseCase', () => { - it('normalizes trusted Copilot authority into the sandbox delegation', async () => { - const execute = vi.fn().mockResolvedValue({ sandboxes: [] }) - const useCase = { operation: sandboxOperations.list, execute } - const input = { workspaceId: trustedContext.workspaceId } - - await expect(executeCopilotSandboxUseCase(trustedContext, useCase, input)).resolves.toEqual({ - sandboxes: [], - }) - expect(execute).toHaveBeenCalledWith({ - principal: expect.objectContaining({ - kind: 'delegated', - serviceId: 'copilot', - subjectUserId: trustedContext.userId, - workspaceId: trustedContext.workspaceId, - delegationId: `copilot-tool:${trustedContext.toolCallId}`, - audience: 'sim:sandboxes', - resourceScope: expect.objectContaining({ - chatId: trustedContext.chatId, - executionId: trustedContext.executionId, - }), - }), - input, - }) - }) - - it('rejects an untrusted Copilot marker before application execution', () => { - const execute = vi.fn() - const useCase = { operation: sandboxOperations.create, execute } - - expect(() => - executeCopilotSandboxUseCase({ ...trustedContext, copilotToolExecution: false }, useCase, { - workspaceId: trustedContext.workspaceId, - name: 'data-tools', - language: 'python', - dependencies: [], - }) - ).toThrow('trusted Copilot execution context') - expect(execute).not.toHaveBeenCalled() - }) - - it("refuses a use case from another domain's registry", () => { - const execute = vi.fn() - const useCase = { operation: customToolOperations.list, execute } - - expect(() => - executeCopilotSandboxUseCase(trustedContext, useCase, { - workspaceId: trustedContext.workspaceId, - }) - ).toThrow('Unregistered Copilot sandbox operation') - expect(execute).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/mothership/application/execute-sandbox-use-case.ts b/apps/sim/lib/mothership/application/execute-sandbox-use-case.ts deleted file mode 100644 index bed6566f9f0..00000000000 --- a/apps/sim/lib/mothership/application/execute-sandbox-use-case.ts +++ /dev/null @@ -1,14 +0,0 @@ -import { createCopilotApplicationAdapter } from '@/lib/mothership/application/application-adapter' -import { COPILOT_APPLICATION_DELEGATION_TTL_MS } from '@/lib/mothership/auth/application-delegation' -import { sandboxDelegationPolicy } from '@/lib/sandboxes/application/authorization' -import { sandboxOperations } from '@/lib/sandboxes/application/operations' - -export const executeCopilotSandboxUseCase = createCopilotApplicationAdapter({ - domain: 'sandbox', - delegation: { - audience: sandboxDelegationPolicy.audience, - ttlMs: COPILOT_APPLICATION_DELEGATION_TTL_MS, - createDelegationId: (context) => `copilot-tool:${context.toolCallId}`, - }, - operations: sandboxOperations, -}) diff --git a/apps/sim/lib/mothership/application/load-connected-accounts.ts b/apps/sim/lib/mothership/application/load-connected-accounts.ts deleted file mode 100644 index e550634de5b..00000000000 --- a/apps/sim/lib/mothership/application/load-connected-accounts.ts +++ /dev/null @@ -1,28 +0,0 @@ -import { workspaceAccountsSettingsDelegationPolicy } from '@/lib/credential-groups/application/authorization' -import { getWorkspaceAccountsSettings } from '@/lib/credential-groups/application/manage-groups' -import { credentialGroupOperations } from '@/lib/credential-groups/application/operations' -import { createCopilotApplicationAdapter } from '@/lib/mothership/application/application-adapter' -import { - COPILOT_APPLICATION_DELEGATION_TTL_MS, - type TrustedCopilotExecutionContext, -} from '@/lib/mothership/auth/application-delegation' - -const executeWorkspaceAccountsUseCase = createCopilotApplicationAdapter({ - domain: 'connected accounts', - delegation: { - audience: workspaceAccountsSettingsDelegationPolicy.audience, - ttlMs: COPILOT_APPLICATION_DELEGATION_TTL_MS, - createDelegationId: (context) => `copilot-tool:${context.toolCallId}`, - }, - operations: { workspaceSettings: credentialGroupOperations.workspaceSettings }, -}) - -/** Reads the workspace's account configuration with the acting admin's current access. */ -export async function loadCopilotConnectedAccounts(context: TrustedCopilotExecutionContext) { - const { credentialGroup } = await executeWorkspaceAccountsUseCase( - context, - getWorkspaceAccountsSettings, - { workspaceId: context.workspaceId } - ) - return credentialGroup -} diff --git a/apps/sim/lib/table/application/copilot-bulk-rows.test.ts b/apps/sim/lib/table/application/copilot-bulk-rows.test.ts deleted file mode 100644 index cdda26a1cbb..00000000000 --- a/apps/sim/lib/table/application/copilot-bulk-rows.test.ts +++ /dev/null @@ -1,196 +0,0 @@ -import { createDelegatedPrincipal } from '@sim/testing/factories/principal.factory' -import { auditMock, auditMockFns } from '@sim/testing/mocks/audit.mock' -import { backgroundTaskMock } from '@sim/testing/mocks/background-task.mock' -import { idMock, idMockFns } from '@sim/testing/mocks/id.mock' -import { tableMock, tableMockFns } from '@sim/testing/mocks/table.mock' -import { - tableApplicationContextMock, - tableApplicationContextMockFns, -} from '@sim/testing/mocks/table-application-context.mock' -import { - tableApplicationRowsMock, - tableApplicationRowsMockFns, -} from '@sim/testing/mocks/table-application-rows.mock' -import { tableEventsMock, tableEventsMockFns } from '@sim/testing/mocks/table-events.mock' -import { - tableJobsServiceMock, - tableJobsServiceMockFns, -} from '@sim/testing/mocks/table-jobs-service.mock' -import { - tableRowsSecretProvenanceMock, - tableRowsSecretProvenanceMockFns, -} from '@sim/testing/mocks/table-rows-secret-provenance.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' -import type { TableDefinition } from '@/lib/table/types' - -vi.mock('@sim/audit', () => auditMock) - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -vi.mock('@sim/utils/id', () => idMock) - -vi.mock('@/lib/core/utils/background', () => backgroundTaskMock) - -vi.mock('@/lib/table', () => tableMock) - -vi.mock('@/lib/table/application/context', () => tableApplicationContextMock) - -vi.mock('@/lib/table/application/rows', () => tableApplicationRowsMock) - -vi.mock('@/lib/table/column-keys', () => ({ buildIdByName: () => new Map() })) -vi.mock('@/lib/table/delete-runner', () => ({ - markTableDeleteFailed: vi.fn(), - runTableDelete: vi.fn(), -})) -vi.mock('@/lib/table/events', () => tableEventsMock) -vi.mock('@/lib/table/jobs/service', () => tableJobsServiceMock) -vi.mock('@/lib/table/mutation-locks', () => ({ - assertRowDelete: vi.fn(), - assertRowUpdate: vi.fn(), - patchColumnIds: () => [], -})) -vi.mock('@/lib/table/rows/secret-provenance', () => tableRowsSecretProvenanceMock) -vi.mock('@/lib/table/update-runner', () => ({ - markTableUpdateFailed: vi.fn(), - runTableUpdate: vi.fn(), -})) - -import { - copilotDeleteRowsByFilter, - copilotUpdateRowsByFilter, -} from '@/lib/table/application/copilot-bulk-rows' - -const mocks = { - deleteByFilter: tableMockFns.mockDeleteRowsByFilter, - updateByFilter: tableMockFns.mockUpdateRowsByFilter, - markJob: tableJobsServiceMockFns.mockMarkTableJobRunningInWorkspace, - releaseJob: tableJobsServiceMockFns.mockReleaseJobClaimInWorkspace, - resolveContext: tableApplicationContextMockFns.mockResolveActiveTableContext, - translateFilter: tableApplicationRowsMockFns.mockTablePredicateNamesToFilter, - audit: auditMockFns.mockRecordAudit, - resolvePermission: workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission, - signal: tableEventsMockFns.mockSignalTableRowsChanged, -} - -idMockFns.mockGenerateId.mockReturnValue('job-12345678') -tableMockFns.mockRowDataNameToId.mockImplementation((data) => data) -tableRowsSecretProvenanceMockFns.mockCreateExactEmptyTableRowSecretProvenance.mockReturnValue({ - complete: true, - columns: {}, -}) - -const table: TableDefinition = { - id: 'table-1', - name: 'People', - description: null, - schema: { columns: [{ id: 'column-1', name: 'name', type: 'string' }] }, - metadata: null, - rowCount: 2, - maxRows: 100, - workspaceId: 'workspace-1', - createdBy: 'owner-1', - archivedAt: null, - createdAt: new Date('2026-08-01T00:00:00.000Z'), - updatedAt: new Date('2026-08-01T00:00:00.000Z'), -} - -const principal = createDelegatedPrincipal({ - delegationId: 'copilot-tool:tool-1', - audience: 'sim:tables', - resourceScope: { tableId: 'table-1' }, -}) - -const input = { - tableId: 'table-1', - assertedWorkspaceId: 'workspace-1', - filter: { all: [] as [] }, - data: { name: 'Ada' }, - limit: 1, -} - -describe('Copilot bulk row application use cases', () => { - beforeEach(() => { - mocks.resolvePermission.mockResolvedValue('write') - mocks.resolveContext.mockResolvedValue({ - tableId: table.id, - table, - workspaceId: table.workspaceId, - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mocks.translateFilter.mockReturnValue({}) - mocks.updateByFilter.mockResolvedValue({ affectedCount: 1, affectedRowIds: ['row-1'] }) - mocks.deleteByFilter.mockResolvedValue({ affectedCount: 1, affectedRowIds: ['row-1'] }) - mocks.markJob.mockResolvedValue(true) - mocks.releaseJob.mockResolvedValue(true) - }) - - it('rejects delegated resource-scope mismatches before mutation', async () => { - await expect( - copilotUpdateRowsByFilter.execute({ - principal: { ...principal, resourceScope: { tableId: 'table-other' } }, - input, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.updateByFilter).not.toHaveBeenCalled() - }) - - it('rejects current permission loss before mutation', async () => { - mocks.resolvePermission.mockResolvedValueOnce('read') - - await expect(copilotUpdateRowsByFilter.execute({ principal, input })).rejects.toMatchObject({ - code: 'forbidden', - }) - expect(mocks.updateByFilter).not.toHaveBeenCalled() - }) - - it('projects audit and shared effects only from an authoritative mutation result', async () => { - await copilotUpdateRowsByFilter.execute({ principal, input }) - - expect(mocks.audit).toHaveBeenCalledTimes(1) - expect(mocks.audit).toHaveBeenCalledWith( - expect.objectContaining({ - action: 'table.updated', - resourceId: 'table-1', - metadata: expect.objectContaining({ operation: 'tables.rows.update_many', rowsUpdated: 1 }), - }) - ) - expect(mocks.signal).toHaveBeenCalledWith('table-1') - - vi.clearAllMocks() - mocks.resolvePermission.mockResolvedValue('write') - mocks.resolveContext.mockResolvedValue({ - tableId: table.id, - table, - workspaceId: table.workspaceId, - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mocks.translateFilter.mockReturnValue({}) - mocks.updateByFilter.mockResolvedValue({ affectedCount: 0, affectedRowIds: [] }) - - await copilotUpdateRowsByFilter.execute({ principal, input }) - expect(mocks.audit).not.toHaveBeenCalled() - expect(mocks.signal).not.toHaveBeenCalled() - }) - - it('releases inline delete claims and audits the committed count', async () => { - await copilotDeleteRowsByFilter.execute({ - principal, - input: { - tableId: 'table-1', - assertedWorkspaceId: 'workspace-1', - filter: { all: [] }, - limit: 1, - }, - }) - - expect(mocks.deleteByFilter).toHaveBeenCalledTimes(1) - expect(mocks.releaseJob).toHaveBeenCalledWith('table-1', 'workspace-1', 'job-12345678') - expect(mocks.audit).toHaveBeenCalledTimes(1) - }) -}) diff --git a/apps/sim/lib/table/application/copilot-bulk-rows.ts b/apps/sim/lib/table/application/copilot-bulk-rows.ts deleted file mode 100644 index 4720ea898d0..00000000000 --- a/apps/sim/lib/table/application/copilot-bulk-rows.ts +++ /dev/null @@ -1,363 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { resolvePrincipalAttribution } from '@sim/auth/principal' -import { createLogger } from '@sim/logger' -import { getErrorMessage } from '@sim/utils/errors' -import { generateId } from '@sim/utils/id' -import { capabilityGovernedPrincipalUserId } from '@/lib/core/application' -import { isTriggerDevEnabled } from '@/lib/core/config/env-flags' -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { runDetached } from '@/lib/core/utils/background' -import { - deleteRowsByFilter, - type Filter, - queryRows, - type RowData, - rowDataNameToId, - TABLE_LIMITS, - type TableDeleteJobPayload, - type TablePredicate, - type TableUpdateJobPayload, - updateRowsByFilter, -} from '@/lib/table' -import { defineAuthorizedTableUseCase } from '@/lib/table/application/authorized-table-use-case' -import { resolveActiveTableContext } from '@/lib/table/application/context' -import { tableOperations } from '@/lib/table/application/operations' -import { tablePredicateNamesToFilter } from '@/lib/table/application/rows' -import { buildIdByName } from '@/lib/table/column-keys' -import { markTableDeleteFailed, runTableDelete } from '@/lib/table/delete-runner' -import { signalTableRowsChanged } from '@/lib/table/events' -import { - markTableJobRunningInWorkspace, - releaseJobClaimInWorkspace, -} from '@/lib/table/jobs/service' -import { assertRowDelete, assertRowUpdate, patchColumnIds } from '@/lib/table/mutation-locks' -import { createExactEmptyTableRowSecretProvenance } from '@/lib/table/rows/secret-provenance' -import { markTableUpdateFailed, runTableUpdate } from '@/lib/table/update-runner' -import { uniqueColumnsInPatch } from '@/lib/table/validation' - -const logger = createLogger('CopilotBulkRowsApplication') - -interface CopilotBulkRowsInput { - tableId: string - assertedWorkspaceId: string -} - -export interface CopilotUpdateRowsByFilterInput extends CopilotBulkRowsInput { - filter: TablePredicate - data: RowData - limit?: number -} - -export type CopilotUpdateRowsByFilterResult = - | { kind: 'inline'; affectedCount: number; affectedRowIds: string[] } - | { kind: 'background'; affectedCount: number; jobId: string } - -export interface CopilotDeleteRowsByFilterInput extends CopilotBulkRowsInput { - filter: TablePredicate - limit?: number -} - -export type CopilotDeleteRowsByFilterResult = - | { kind: 'inline'; affectedCount: number; affectedRowIds: string[] } - | { kind: 'background'; doomedCount: number; jobId: string; bounded: boolean } - -function requestId(): string { - return generateId().slice(0, 8) -} - -function validateLimit(limit: number | undefined): void { - if (limit !== undefined && (!Number.isSafeInteger(limit) || limit < 1)) { - throw new OrchestrationError('validation', 'Limit must be an integer of at least 1') - } -} - -async function releaseClaim(tableId: string, workspaceId: string, jobId: string): Promise { - const released = await releaseJobClaimInWorkspace(tableId, workspaceId, jobId) - if (!released) throw new Error('Table job claim was no longer active') -} - -async function withReleasedClaim( - tableId: string, - workspaceId: string, - jobId: string, - run: () => Promise -): Promise { - let result: T - try { - result = await run() - } catch (error) { - try { - await releaseClaim(tableId, workspaceId, jobId) - } catch (cleanupError) { - logger.error('Failed to release table job claim after operation failure', { - tableId, - workspaceId, - jobId, - error: getErrorMessage(cleanupError), - }) - } - throw error - } - await releaseClaim(tableId, workspaceId, jobId) - return result -} - -async function releaseClaimAfterDispatchFailure(params: { - tableId: string - workspaceId: string - jobId: string -}): Promise { - try { - await releaseClaim(params.tableId, params.workspaceId, params.jobId) - } catch (cleanupError) { - logger.error('Failed to release table job claim after dispatch failure', { - ...params, - error: getErrorMessage(cleanupError), - }) - } -} - -async function dispatchUpdateJob(params: { - jobId: string - tableId: string - workspaceId: string - filter: Filter - data: RowData - cutoff: Date - maxRows?: number -}): Promise { - if (isTriggerDevEnabled) { - try { - const [{ tableUpdateTask }, { tasks }, { resolveTriggerRegion }] = await Promise.all([ - import('@/background/table-update'), - import('@trigger.dev/sdk'), - import('@/lib/core/async-jobs/region'), - ]) - await tasks.trigger( - 'table-update', - { ...params, cutoff: params.cutoff.toISOString() }, - { - tags: [`tableId:${params.tableId}`, `jobId:${params.jobId}`], - region: await resolveTriggerRegion(), - } - ) - } catch (error) { - await releaseClaimAfterDispatchFailure(params) - throw error - } - return - } - runDetached('table-update', () => - runTableUpdate(params).catch(async (error) => { - await markTableUpdateFailed(params.tableId, params.jobId, error) - throw error - }) - ) -} - -async function dispatchDeleteJob(params: { - jobId: string - tableId: string - workspaceId: string - filter: Filter - cutoff: Date - maxRows?: number -}): Promise { - if (isTriggerDevEnabled) { - try { - const [{ tableDeleteTask }, { tasks }, { resolveTriggerRegion }] = await Promise.all([ - import('@/background/table-delete'), - import('@trigger.dev/sdk'), - import('@/lib/core/async-jobs/region'), - ]) - await tasks.trigger( - 'table-delete', - { ...params, cutoff: params.cutoff.toISOString() }, - { - tags: [`tableId:${params.tableId}`, `jobId:${params.jobId}`], - region: await resolveTriggerRegion(), - } - ) - } catch (error) { - await releaseClaimAfterDispatchFailure(params) - throw error - } - return - } - runDetached('table-delete', () => - runTableDelete(params).catch(async (error) => { - await markTableDeleteFailed(params.tableId, params.jobId, error) - throw error - }) - ) -} - -export const copilotUpdateRowsByFilter = defineAuthorizedTableUseCase({ - operation: tableOperations.updateRows, - resolveContext: ({ input }: { input: CopilotUpdateRowsByFilterInput }) => - resolveActiveTableContext(input), - async execute({ principal, input, context }): Promise { - validateLimit(input.limit) - const idData = rowDataNameToId(input.data, buildIdByName(context.table.schema)) - const filter = tablePredicateNamesToFilter(input.filter, context.table) - const patchTouchesUnique = uniqueColumnsInPatch(context.table.schema, idData).length > 0 - const inlineEligible = - input.limit !== undefined && input.limit <= TABLE_LIMITS.MAX_BULK_OPERATION_SIZE - - if (!inlineEligible && !patchTouchesUnique) { - const { totalCount } = await queryRows( - context.table, - { filter, limit: 1, withExecutions: false }, - requestId() - ) - const matchCount = totalCount ?? 0 - const target = input.limit === undefined ? matchCount : Math.min(input.limit, matchCount) - if (target > TABLE_LIMITS.MAX_BULK_OPERATION_SIZE) { - const cutoff = new Date() - const jobId = generateId() - const payload: TableUpdateJobPayload = { - filter, - data: idData, - cutoff: cutoff.toISOString(), - affectedCount: target, - maxRows: input.limit, - } - assertRowUpdate(context.table, patchColumnIds(idData)) - const claimed = await markTableJobRunningInWorkspace( - context.tableId, - context.workspaceId, - jobId, - 'update', - payload - ) - if (!claimed) { - throw new OrchestrationError('conflict', 'A job is already in progress for this table') - } - await dispatchUpdateJob({ - jobId, - tableId: context.tableId, - workspaceId: context.workspaceId, - filter, - data: idData, - cutoff, - maxRows: input.limit, - }) - return { kind: 'background', affectedCount: target, jobId } - } - } - - const result = await updateRowsByFilter( - context.table, - { - filter, - data: idData, - limit: input.limit, - actorUserId: resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId, - capabilityGovernedUserId: capabilityGovernedPrincipalUserId(principal), - secretProvenance: createExactEmptyTableRowSecretProvenance(idData), - }, - requestId() - ) - return { kind: 'inline', ...result } - }, - projectAudit({ context, result }) { - if (result.kind !== 'inline' || result.affectedCount === 0) return [] - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: context.tableId, - resourceName: context.table.name, - description: `Updated ${result.affectedCount} row(s) in table "${context.table.name}"`, - metadata: { op: 'bulk_update', rowsUpdated: result.affectedCount }, - } - }, - afterSuccess({ context, result }) { - if (result.kind === 'inline' && result.affectedCount > 0) { - signalTableRowsChanged(context.tableId) - } - }, -}) - -export const copilotDeleteRowsByFilter = defineAuthorizedTableUseCase({ - operation: tableOperations.deleteRows, - resolveContext: ({ input }: { input: CopilotDeleteRowsByFilterInput }) => - resolveActiveTableContext(input), - async execute({ input, context }): Promise { - validateLimit(input.limit) - const filter = tablePredicateNamesToFilter(input.filter, context.table) - const inlineEligible = - input.limit !== undefined && input.limit <= TABLE_LIMITS.MAX_BULK_OPERATION_SIZE - - if (!inlineEligible) { - const { totalCount } = await queryRows( - context.table, - { filter, limit: 1, withExecutions: false }, - requestId() - ) - const matchCount = totalCount ?? 0 - const target = input.limit === undefined ? matchCount : Math.min(input.limit, matchCount) - if (target > TABLE_LIMITS.MAX_BULK_OPERATION_SIZE) { - const doomedCount = Math.min(target, context.table.rowCount) - const cutoff = new Date() - const jobId = generateId() - const bounded = input.limit !== undefined - const payload: TableDeleteJobPayload = bounded - ? { filter, cutoff: cutoff.toISOString(), maxRows: input.limit } - : { filter, cutoff: cutoff.toISOString(), doomedCount } - assertRowDelete(context.table) - const claimed = await markTableJobRunningInWorkspace( - context.tableId, - context.workspaceId, - jobId, - 'delete', - payload - ) - if (!claimed) { - throw new OrchestrationError('conflict', 'A job is already in progress for this table') - } - await dispatchDeleteJob({ - jobId, - tableId: context.tableId, - workspaceId: context.workspaceId, - filter, - cutoff, - maxRows: input.limit, - }) - return { kind: 'background', doomedCount, jobId, bounded } - } - } - - const jobId = generateId() - const claimed = await markTableJobRunningInWorkspace( - context.tableId, - context.workspaceId, - jobId, - 'delete' - ) - if (!claimed) { - throw new OrchestrationError('conflict', 'A job is already in progress for this table') - } - const result = await withReleasedClaim(context.tableId, context.workspaceId, jobId, () => - deleteRowsByFilter(context.table, { filter, limit: input.limit }, requestId()) - ) - return { kind: 'inline', ...result } - }, - projectAudit({ context, result }) { - if (result.kind !== 'inline' || result.affectedCount === 0) return [] - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: context.tableId, - resourceName: context.table.name, - description: `Deleted ${result.affectedCount} row(s) from table "${context.table.name}"`, - metadata: { op: 'bulk_delete', rowsDeleted: result.affectedCount }, - } - }, - afterSuccess({ context, result }) { - if (result.kind === 'inline' && result.affectedCount > 0) { - signalTableRowsChanged(context.tableId) - } - }, -}) diff --git a/packages/testing/src/mocks/credential-groups-enrollments.mock.ts b/packages/testing/src/mocks/credential-groups-enrollments.mock.ts index 2627636e14a..96c7f72278d 100644 --- a/packages/testing/src/mocks/credential-groups-enrollments.mock.ts +++ b/packages/testing/src/mocks/credential-groups-enrollments.mock.ts @@ -39,7 +39,6 @@ export const credentialGroupsEnrollmentsMockFns = { mockLoadCredentialGroupInviterIdentity: vi.fn(), mockInviteCredentialGroupEnrollment: vi.fn(), mockCreateCredentialGroupSelfEnrollmentLink: vi.fn(), - mockCreateCredentialGroupInvitationLink: vi.fn(), mockResendCredentialGroupEnrollment: vi.fn(), mockDeleteCredentialGroupEnrollment: vi.fn(), mockRevokeCredentialGroupEnrollment: vi.fn(), @@ -74,7 +73,6 @@ export const credentialGroupsEnrollmentsMock = { loadCredentialGroupInviterIdentity: fns.mockLoadCredentialGroupInviterIdentity, inviteCredentialGroupEnrollment: fns.mockInviteCredentialGroupEnrollment, createCredentialGroupSelfEnrollmentLink: fns.mockCreateCredentialGroupSelfEnrollmentLink, - createCredentialGroupInvitationLink: fns.mockCreateCredentialGroupInvitationLink, resendCredentialGroupEnrollment: fns.mockResendCredentialGroupEnrollment, deleteCredentialGroupEnrollment: fns.mockDeleteCredentialGroupEnrollment, revokeCredentialGroupEnrollment: fns.mockRevokeCredentialGroupEnrollment, diff --git a/scripts/check-api-validation-contracts.ts b/scripts/check-api-validation-contracts.ts index a716c21615e..119815284d9 100644 --- a/scripts/check-api-validation-contracts.ts +++ b/scripts/check-api-validation-contracts.ts @@ -198,7 +198,7 @@ const DECLARATIVE_ROUTE_BUILDER_USAGE_PATTERN = /\b(?:defineInternalJsonRoute|defineV2JsonRoute|defineInternalBinaryRoute|defineV2BinaryRoute|defineScimRoute)\s*\(/ const SERVER_VALIDATION_IMPORT_PATTERN = /\bfrom\s+['"]@\/lib\/api\/server(?:\/validation)?['"]/ const SCHEMA_PARSE_PATTERN = /\b\w+Schema\.(?:safeParse|parse)\(/ -const CONTRACT_SERVER_HELPER_PATTERN = /\b(?:parseToolRequest|validateShimEnvelope)\(/ +const CONTRACT_SERVER_HELPER_PATTERN = /\bvalidateShimEnvelope\(/ const CANONICAL_HELPER_USAGE_PATTERN = /\b(?:isZodError|validationErrorResponse|validationErrorResponseFromError|getValidationErrorMessage)\s*\(/ const CONTRACT_MAP_PARSE_PATTERN = From 4479c3775d82b179dffc07a3c56a8c1b371b7f32 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:14:21 -0700 Subject: [PATCH 10/30] chore(cleanup): remove dead application use cases and utils, share workflow context resolver - Delete unreachable platform-context, workflow VFS, deployment-overview, chat undeploy, and Copilot block-output/upstream-reference use cases plus their workflowOperations entries - Delete dead fetch-deadline and scheduling utils and unused exported helpers - Add resolvePrincipalWorkflowContext and use it at the 16 identical resolvers - Share processCodeFailure between the Daytona and E2B adapters - Use escapeRegExp and generateId in place of inline equivalents --- .../routes/in-process-transport.test.ts | 42 +- .../api/server/routes/in-process-transport.ts | 31 +- .../application/delete-chat-deployment.ts | 8 +- .../application/operations.ts | 10 +- .../config/env-capabilities.server.test.ts | 2 - .../core/config/env-capabilities.server.ts | 4 - .../lib/core/rate-limiter/route-helpers.ts | 17 - apps/sim/lib/core/security/csp.ts | 23 - apps/sim/lib/core/utils.test.ts | 81 +- .../sim/lib/core/utils/fetch-deadline.test.ts | 69 -- apps/sim/lib/core/utils/fetch-deadline.ts | 123 --- .../lib/core/utils/response-format.test.ts | 5 +- apps/sim/lib/core/utils/response-format.ts | 31 - apps/sim/lib/core/utils/scheduling.ts | 51 -- apps/sim/lib/core/utils/timezone.ts | 5 - apps/sim/lib/core/utils/validation.ts | 21 - .../execution-deadline-header.test.ts | 11 +- .../execution/execution-deadline-header.ts | 11 - .../payloads/large-execution-value.test.ts | 23 +- .../payloads/large-execution-value.ts | 55 -- .../execution/remote-sandbox/code-failure.ts | 27 + .../lib/execution/remote-sandbox/daytona.ts | 23 +- apps/sim/lib/execution/remote-sandbox/e2b.ts | 23 +- .../execution/remote-sandbox/session-lock.ts | 4 +- .../remote-sandbox/workspace-sandboxes.ts | 6 - .../application/authorization.ts | 12 - .../platform-context/application/context.ts | 14 - .../application/operations.ts | 35 - .../platform-context-use-cases.test.ts | 209 ----- .../application/read-account-billing.ts | 22 - .../application/read-enterprise-context.ts | 89 -- apps/sim/lib/vfs/limits.ts | 61 -- .../application/apply-workflow-operations.ts | 19 +- .../workflows/application/chat-deployments.ts | 66 +- .../workflows/application/delete-workflow.ts | 11 +- .../lib/workflows/application/deployments.ts | 30 +- .../application/inspect-workflow-tools.ts | 15 +- .../application/list-workflow-versions.ts | 16 +- .../workflows/application/operations.test.ts | 1 - .../lib/workflows/application/operations.ts | 64 -- .../workflows/application/principal-scope.ts | 24 + .../read-workflow-copilot-metadata.test.ts | 121 --- .../read-workflow-copilot-metadata.ts | 265 +----- .../application/read-workflow-definition.ts | 17 +- .../read-workflow-deployment-overview.test.ts | 182 ---- .../read-workflow-deployment-overview.ts | 130 --- .../application/read-workflow-graph.ts | 10 +- .../application/read-workflow-version.ts | 16 +- .../workflows/application/read-workflow.ts | 21 +- .../application/replace-workflow-state.ts | 15 +- .../application/run-workflow-from-copilot.ts | 21 +- .../application/update-workflow-content.ts | 22 +- .../update-workflow-deployment-settings.ts | 17 +- .../workflows/application/update-workflow.ts | 24 +- .../application/workflow-vfs.test.ts | 191 ---- .../lib/workflows/application/workflow-vfs.ts | 847 ------------------ apps/sim/lib/workflows/comparison/describe.ts | 45 - .../comparison/format-description.test.ts | 29 +- apps/sim/lib/workflows/editing/validation.ts | 61 -- .../executor/execution-queries.test.ts | 91 -- .../workflows/executor/execution-queries.ts | 78 -- apps/sim/lib/workflows/utils.ts | 18 - scripts/check-egress-boundary.ts | 2 - 63 files changed, 141 insertions(+), 3476 deletions(-) delete mode 100644 apps/sim/lib/core/utils/fetch-deadline.test.ts delete mode 100644 apps/sim/lib/core/utils/fetch-deadline.ts delete mode 100644 apps/sim/lib/core/utils/scheduling.ts create mode 100644 apps/sim/lib/execution/remote-sandbox/code-failure.ts delete mode 100644 apps/sim/lib/platform-context/application/authorization.ts delete mode 100644 apps/sim/lib/platform-context/application/context.ts delete mode 100644 apps/sim/lib/platform-context/application/operations.ts delete mode 100644 apps/sim/lib/platform-context/application/platform-context-use-cases.test.ts delete mode 100644 apps/sim/lib/platform-context/application/read-account-billing.ts delete mode 100644 apps/sim/lib/platform-context/application/read-enterprise-context.ts delete mode 100644 apps/sim/lib/vfs/limits.ts delete mode 100644 apps/sim/lib/workflows/application/read-workflow-copilot-metadata.test.ts delete mode 100644 apps/sim/lib/workflows/application/read-workflow-deployment-overview.test.ts delete mode 100644 apps/sim/lib/workflows/application/read-workflow-deployment-overview.ts delete mode 100644 apps/sim/lib/workflows/application/workflow-vfs.test.ts delete mode 100644 apps/sim/lib/workflows/application/workflow-vfs.ts delete mode 100644 apps/sim/lib/workflows/executor/execution-queries.test.ts diff --git a/apps/sim/lib/api/server/routes/in-process-transport.test.ts b/apps/sim/lib/api/server/routes/in-process-transport.test.ts index 8a30f6a34a8..4c4c622de99 100644 --- a/apps/sim/lib/api/server/routes/in-process-transport.test.ts +++ b/apps/sim/lib/api/server/routes/in-process-transport.test.ts @@ -1,4 +1,4 @@ -import { NextRequest, NextResponse } from 'next/server' +import { NextRequest } from 'next/server' import { describe, expect, it, vi } from 'vitest' const { handlers } = vi.hoisted(() => ({ @@ -21,11 +21,9 @@ vi.mock('@/lib/api/server/routes/v2-route-table.generated', () => ({ })) import { - createInProcessTransport, dispatchInProcessV2Request, matchV2Route, } from '@/lib/api/server/routes/in-process-transport' -import { isInternalRequest } from '@/lib/api/server/routes/internal-request' describe('in-process transport', () => { it('prefers the more literal pattern and decodes dynamic segments', async () => { @@ -35,33 +33,6 @@ describe('in-process transport', () => { expect(matchV2Route('/api/v2/nowhere')).toBeNull() }) - it('dispatches a v2 request to its route handler in-process, marked internal', async () => { - handlers.getBlock.mockImplementation( - async (request: Request, context: { params: Promise> }) => - NextResponse.json({ - internal: isInternalRequest(request), - key: request.headers.get('x-api-key'), - query: new URL(request.url).searchParams.get('workspaceId'), - params: await context.params, - }) - ) - const transport = createInProcessTransport() - - const response = await transport('http://internal/api/v2/blocks/agent?workspaceId=ws-1', { - method: 'GET', - headers: { 'x-api-key': 'secret' }, - }) - - expect(response.status).toBe(200) - expect(await response.json()).toEqual({ - internal: true, - key: 'secret', - query: 'ws-1', - params: { blockId: 'agent' }, - }) - expect(handlers.listBlocks).not.toHaveBeenCalled() - }) - it('dispatches HEAD through GET while retaining HEAD for authorization-only behavior', async () => { handlers.getBlock.mockImplementation(async (request: Request) => { expect(request.method).toBe('HEAD') @@ -73,15 +44,4 @@ describe('in-process transport', () => { expect(response?.status).toBe(200) expect(handlers.getBlock).toHaveBeenCalledOnce() }) - - it('falls through to fetch for anything outside the v2 table', async () => { - const network = vi.spyOn(globalThis, 'fetch').mockResolvedValue(new Response('elsewhere')) - const transport = createInProcessTransport() - - const response = await transport('http://internal/api/files/serve/abc', { method: 'GET' }) - - expect(await response.text()).toBe('elsewhere') - expect(network).toHaveBeenCalledTimes(1) - expect(handlers.getBlock).not.toHaveBeenCalled() - }) }) diff --git a/apps/sim/lib/api/server/routes/in-process-transport.ts b/apps/sim/lib/api/server/routes/in-process-transport.ts index 052837c58e7..0a78a5972dc 100644 --- a/apps/sim/lib/api/server/routes/in-process-transport.ts +++ b/apps/sim/lib/api/server/routes/in-process-transport.ts @@ -1,21 +1,18 @@ -import { NextRequest } from 'next/server' -import { markInternalRequest } from '@/lib/api/server/routes/internal-request' +import { escapeRegExp } from '@sim/utils/string' +import type { NextRequest } from 'next/server' import { V2_ROUTES } from '@/lib/api/server/routes/v2-route-table.generated' /** - * A `fetch` that answers the server's own v2 requests in-process. + * Answers the server's own v2 requests in-process. * * The embedded CLI and the agent-cli engines are typed v2 clients. Pointing them at * the server's URL made every tool call a network round trip through the proxy, * API-key authentication, the abuse rate limits, and the proxy body ceiling — a - * grep over one block definition cost seconds and tripped the per-key limit. This - * transport resolves the request's path against the generated route table and - * invokes the route handler directly, with the request marked internal so - * admission authenticates it but does not rate-limit it. Contracts, use cases, - * presenters, and error envelopes are untouched: the handler that runs is the one - * the network path would run. - * - * Anything outside the v2 table falls through to real `fetch`. + * grep over one block definition cost seconds and tripped the per-key limit. A + * caller's transport resolves the request's path against the generated route + * table and invokes the route handler directly. Contracts, use cases, presenters, + * and error envelopes are untouched: the handler that runs is the one the network + * path would run. */ type RouteHandler = ( @@ -49,7 +46,7 @@ const COMPILED: CompiledRoute[] = V2_ROUTES.map((route) => { const param = /^\{(.+)\}$/.exec(segment) if (!param?.[1]) { literals += 1 - return segment.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + return escapeRegExp(segment) } params.push(param[1]) return '([^/]+)' @@ -93,13 +90,3 @@ export async function dispatchInProcessV2Request( if (typeof handler !== 'function') return undefined return (handler as RouteHandler)(request, { params: Promise.resolve(matched.params) }) } - -export function createInProcessTransport(): typeof fetch { - return async (input, init) => { - const request = new NextRequest( - new Request(input instanceof Request ? input.clone() : input, init) - ) - markInternalRequest(request) - return (await dispatchInProcessV2Request(request)) ?? fetch(input, init) - } -} diff --git a/apps/sim/lib/chat-deployments/application/delete-chat-deployment.ts b/apps/sim/lib/chat-deployments/application/delete-chat-deployment.ts index 578ff2f1204..b4959b6ec61 100644 --- a/apps/sim/lib/chat-deployments/application/delete-chat-deployment.ts +++ b/apps/sim/lib/chat-deployments/application/delete-chat-deployment.ts @@ -17,12 +17,8 @@ export interface DeleteChatDeploymentInput { } /** - * Stops one chat deployment serving. - * - * Keyed on the deployment rather than on its workflow, which is what - * `workflows.chat.undeploy` takes. Both end in `performChatUndeploy`; they stay - * separate operations because a caller holding a deployment id cannot name the - * workflow the other requires, and the reverse. + * Stops one chat deployment serving, keyed on the deployment rather than on + * its workflow. * * The workflow's own deployment is untouched — only the chat surface stops. */ diff --git a/apps/sim/lib/chat-deployments/application/operations.ts b/apps/sim/lib/chat-deployments/application/operations.ts index 0f2365c9344..4673eb3023d 100644 --- a/apps/sim/lib/chat-deployments/application/operations.ts +++ b/apps/sim/lib/chat-deployments/application/operations.ts @@ -14,11 +14,11 @@ import { defineWorkspaceOperation } from '@/lib/core/application/workspace-opera * `list` takes it too: a group with the chat deployment surface withheld should * not still be told which workflows are published on it. * - * `workflows.chat.deploy` and `workflows.chat.undeploy` remain the entry points - * for the surfaces that name a workflow and ask for it to be published — the - * internal deploy route and the Copilot tool. They converge on the same domain - * effect as `replace` and `delete` but authorize a workflow the caller is - * deploying rather than a chat surface the caller is configuring. + * `workflows.chat.deploy` remains the entry point for the surfaces that name a + * workflow and ask for it to be published — the internal deploy route and the + * Copilot tool. It converges on the same domain effect as `replace` but + * authorizes a workflow the caller is deploying rather than a chat surface the + * caller is configuring. */ const CHAT_DEPLOYMENT_LIST_POLICY = { principalKinds: [ diff --git a/apps/sim/lib/core/config/env-capabilities.server.test.ts b/apps/sim/lib/core/config/env-capabilities.server.test.ts index a13a291bcad..d10ff373010 100644 --- a/apps/sim/lib/core/config/env-capabilities.server.test.ts +++ b/apps/sim/lib/core/config/env-capabilities.server.test.ts @@ -1,7 +1,6 @@ import { resetEnvMock, setEnv } from '@sim/testing' import { afterAll, beforeEach, describe, expect, expectTypeOf, it } from 'vitest' import { - getConfiguredSandboxProviderId, getSelectedSandboxProviderId, inspectConfiguredOAuthClient, requireConfiguredOAuthClient, @@ -71,7 +70,6 @@ describe('server environment capabilities', () => { setEnv({ SANDBOX_PROVIDER: 'daytona', DAYTONA_API_KEY: 'daytona-key' }) expect(getSelectedSandboxProviderId()).toBe('daytona') - expect(() => getConfiguredSandboxProviderId()).toThrow(/DAYTONA_FUNCTION_SNAPSHOT_ID/) }) it('rejects an unknown sandbox provider during selection', () => { diff --git a/apps/sim/lib/core/config/env-capabilities.server.ts b/apps/sim/lib/core/config/env-capabilities.server.ts index 271d46a266e..28af2740442 100644 --- a/apps/sim/lib/core/config/env-capabilities.server.ts +++ b/apps/sim/lib/core/config/env-capabilities.server.ts @@ -26,10 +26,6 @@ export function getConfiguredStorageProviderId() { return requireCapability(STORAGE_CAPABILITY, env).providerId } -export function getConfiguredSandboxProviderId() { - return requireCapability(SANDBOX_CAPABILITY, env).providerId -} - /** * Selects the sandbox adapter without requiring a Function-specific base image. * Each adapter validates the API key and image required by the requested sandbox diff --git a/apps/sim/lib/core/rate-limiter/route-helpers.ts b/apps/sim/lib/core/rate-limiter/route-helpers.ts index 2bed1f0a3ef..be7b21c770a 100644 --- a/apps/sim/lib/core/rate-limiter/route-helpers.ts +++ b/apps/sim/lib/core/rate-limiter/route-helpers.ts @@ -157,23 +157,6 @@ export async function enforceResourceRateLimit( return buildRateLimitResponse(resetAt) } -/** - * Apply a per-workspace token bucket. Use for routes whose cost is borne by the - * workspace rather than the acting user — a shared budget any member spends - * against, so N admins cannot each get a full allowance. - */ -export async function enforceWorkspaceRateLimit( - bucketName: string, - workspaceId: string, - config: TokenBucketConfig = DEFAULT_USER_ROUTE_LIMIT -): Promise { - const key = `route:${bucketName}:workspace:${workspaceId}` - const { allowed, resetAt } = await rateLimiter.checkRateLimitDirect(key, config) - if (allowed) return null - logger.warn('Workspace rate limit exceeded', { bucket: bucketName, workspaceId }) - return buildRateLimitResponse(resetAt) -} - /** * Apply a per-user limit when a userId is present, else fall back to per-IP. * Use for routes whose auth path may legitimately resolve without a userId diff --git a/apps/sim/lib/core/security/csp.ts b/apps/sim/lib/core/security/csp.ts index 1614c81b756..6e3b1b5408b 100644 --- a/apps/sim/lib/core/security/csp.ts +++ b/apps/sim/lib/core/security/csp.ts @@ -337,26 +337,3 @@ export function getChatEmbedCSPPolicy(): string { 'frame-ancestors': ['*'], }) } - -/** - * Add a source to a specific directive (modifies build-time directives) - */ -export function addCSPSource(directive: keyof CSPDirectives, source: string): void { - if (!buildTimeCSPDirectives[directive]) { - buildTimeCSPDirectives[directive] = [] - } - if (!buildTimeCSPDirectives[directive]!.includes(source)) { - buildTimeCSPDirectives[directive]!.push(source) - } -} - -/** - * Remove a source from a specific directive (modifies build-time directives) - */ -export function removeCSPSource(directive: keyof CSPDirectives, source: string): void { - if (buildTimeCSPDirectives[directive]) { - buildTimeCSPDirectives[directive] = buildTimeCSPDirectives[directive]!.filter( - (s: string) => s !== source - ) - } -} diff --git a/apps/sim/lib/core/utils.test.ts b/apps/sim/lib/core/utils.test.ts index 8013430e183..98649bca108 100644 --- a/apps/sim/lib/core/utils.test.ts +++ b/apps/sim/lib/core/utils.test.ts @@ -30,8 +30,7 @@ afterAll(resetEnvMock) import { getRotatingApiKey } from '@/lib/core/config/api-keys' import { decryptSecret, encryptSecret } from '@/lib/core/security/encryption' -import { convertScheduleOptionsToCron } from '@/lib/core/utils/scheduling' -import { getInvalidCharacters, isValidName, validateName } from '@/lib/core/utils/validation' +import { validateName } from '@/lib/core/utils/validation' vi.mock('crypto', () => ({ createCipheriv: vi.fn().mockReturnValue({ @@ -73,54 +72,6 @@ describe('encryption and decryption', () => { }) }) -describe('convertScheduleOptionsToCron', () => { - it.concurrent('should convert minutes schedule to cron', () => { - const result = convertScheduleOptionsToCron('minutes', { minutesInterval: '5' }) - expect(result).toBe('*/5 * * * *') - }) - - it.concurrent('should convert hourly schedule to cron', () => { - const result = convertScheduleOptionsToCron('hourly', { hourlyMinute: '30' }) - expect(result).toBe('30 * * * *') - }) - - it.concurrent('should convert daily schedule to cron', () => { - const result = convertScheduleOptionsToCron('daily', { dailyTime: '15:30' }) - expect(result).toBe('15 30 * * *') - }) - - it.concurrent('should convert weekly schedule to cron', () => { - const result = convertScheduleOptionsToCron('weekly', { - weeklyDay: 'MON', - weeklyDayTime: '09:30', - }) - expect(result).toBe('09 30 * * 1') - }) - - it.concurrent('should convert monthly schedule to cron', () => { - const result = convertScheduleOptionsToCron('monthly', { - monthlyDay: '15', - monthlyTime: '12:00', - }) - expect(result).toBe('12 00 15 * *') - }) - - it.concurrent('should use custom cron expression directly', () => { - const customCron = '*/15 9-17 * * 1-5' - const result = convertScheduleOptionsToCron('custom', { cronExpression: customCron }) - expect(result).toBe(customCron) - }) - - it.concurrent('should throw error for unsupported schedule type', () => { - expect(() => convertScheduleOptionsToCron('invalid', {})).toThrow('Unsupported schedule type') - }) - - it.concurrent('should use default values when options are not provided', () => { - const result = convertScheduleOptionsToCron('daily', {}) - expect(result).toBe('00 09 * * *') - }) -}) - describe('formatDuration', () => { it.concurrent('should format milliseconds correctly', () => { const result = formatDuration(500) @@ -160,36 +111,6 @@ describe('validateName', () => { }) }) -describe('isValidName', () => { - it.concurrent('should return true for valid names', () => { - expect(isValidName('test_name')).toBe(true) - expect(isValidName('test123')).toBe(true) - expect(isValidName('test name')).toBe(true) - expect(isValidName('TestName')).toBe(true) - expect(isValidName('')).toBe(true) - }) - - it.concurrent('should return false for invalid names', () => { - expect(isValidName('test@name')).toBe(false) - expect(isValidName('test-name')).toBe(false) - expect(isValidName('test#name')).toBe(false) - expect(isValidName('test$name')).toBe(false) - expect(isValidName('test%name')).toBe(false) - }) -}) - -describe('getInvalidCharacters', () => { - it.concurrent('should return invalid characters', () => { - const result = getInvalidCharacters('test@#$name') - expect(result).toEqual(['@', '#', '$']) - }) - - it.concurrent('should return unique invalid characters', () => { - const result = getInvalidCharacters('test@@##name') - expect(result).toEqual(['@', '#']) - }) -}) - describe('getRotatingApiKey', () => { it.concurrent('rotates the TypeSafe key pool through the shared selector', () => { expect(getRotatingApiKey('typesafe')).toMatch(/^test-typesafe-key-[1-3]$/) diff --git a/apps/sim/lib/core/utils/fetch-deadline.test.ts b/apps/sim/lib/core/utils/fetch-deadline.test.ts deleted file mode 100644 index 8d9a685df1b..00000000000 --- a/apps/sim/lib/core/utils/fetch-deadline.test.ts +++ /dev/null @@ -1,69 +0,0 @@ -import { Agent } from 'undici' -import { describe, expect, it } from 'vitest' -import { isTransportTimeoutError, withCallerOwnedDeadline } from '@/lib/core/utils/fetch-deadline' - -describe('withCallerOwnedDeadline', () => { - /* - * The pinned Bun ignores a positive numeric `timeout` and honors only the - * boolean/zero form, so anything other than `false` here silently leaves the - * 300s default in force — which is the outage this module exists to prevent. - */ - it('disarms the transport timer rather than negotiating a value', () => { - expect(withCallerOwnedDeadline({}).timeout).toBe(false) - }) - - /* - * Tests run under Node, where `timeout: false` is ignored and undici's - * 300s `headersTimeout` default is what killed async (Trigger.dev) sandbox - * runs. The dispatcher is what disarms it there; dropping it regresses every - * worker-side internal route call longer than five minutes. - */ - it('attaches a dispatcher on runtimes whose fetch is undici', () => { - expect(withCallerOwnedDeadline({}).dispatcher).toBeInstanceOf(Agent) - }) - - it('reuses one dispatcher across calls so pooled connections are shared', () => { - expect(withCallerOwnedDeadline({}).dispatcher).toBe(withCallerOwnedDeadline({}).dispatcher) - }) - - it('preserves the init the caller already built', () => { - const signal = new AbortController().signal - const init = withCallerOwnedDeadline({ method: 'POST', body: 'x', signal }) - expect(init.method).toBe('POST') - expect(init.body).toBe('x') - expect(init.signal).toBe(signal) - }) - - it('does not mutate the caller’s init', () => { - const original: RequestInit = { method: 'POST' } - withCallerOwnedDeadline(original) - expect('timeout' in original).toBe(false) - expect('dispatcher' in original).toBe(false) - }) -}) - -describe('isTransportTimeoutError', () => { - it('recognizes the runtime timeout', () => { - const error = new Error('The operation timed out.') - error.name = 'TimeoutError' - expect(isTransportTimeoutError(error)).toBe(true) - }) - - it('recognizes a severed connection', () => { - expect(isTransportTimeoutError(new TypeError('fetch failed'))).toBe(true) - }) - - it('does not claim a cancellation', () => { - const error = new Error('aborted') - error.name = 'AbortError' - expect(isTransportTimeoutError(error)).toBe(false) - }) - - it.each([ - ['an unrelated TypeError', new TypeError('x is not a function')], - ['a plain error', new Error('boom')], - ['a non-error', 'fetch failed'], - ])('does not claim %s', (_label, value) => { - expect(isTransportTimeoutError(value)).toBe(false) - }) -}) diff --git a/apps/sim/lib/core/utils/fetch-deadline.ts b/apps/sim/lib/core/utils/fetch-deadline.ts deleted file mode 100644 index 52ce39b406c..00000000000 --- a/apps/sim/lib/core/utils/fetch-deadline.ts +++ /dev/null @@ -1,123 +0,0 @@ -import { Agent, type Dispatcher } from 'undici/index.js' - -/** - * Keeps the transport deadline from undercutting the application deadline. - * - * Both runtimes this code executes under ship a five-minute transport default - * that is not raised by an `AbortSignal`, so it acts as an absolute deadline - * for the peer to begin answering. Any request whose peer legitimately works - * before it replies dies at five minutes no matter what deadline the caller - * computed for it: - * - * - Bun's HTTP client arms an idle timer defaulting to 300s. It does not - * re-arm while awaiting response headers. `timeout: false` disarms it. - * - Node's fetch is undici, whose default dispatcher arms `headersTimeout` - * and `bodyTimeout`, both defaulting to 300e3. An expiry surfaces as - * `TypeError: fetch failed` with cause `HeadersTimeoutError` - * (`UND_ERR_HEADERS_TIMEOUT`). A request-scoped `dispatcher` that arms - * neither timer disarms it. - * - * This bit production twice, once per runtime. Workflow function blocks are - * bounded by a plan deadline (50 minutes on enterprise, 7 days async), but the - * executor's call into the internal function route inherited the transport - * default instead, so every sandbox run longer than five minutes failed with a - * bare `fetch failed` that read as user-code failure rather than a transport - * cap. The first fix (`timeout: false`) covered the app server, which runs - * Bun; async executions run in Trigger.dev workers (`runtime: 'node-24'` in - * `trigger.config.ts`), where that option is silently ignored and the same - * five-minute death reappeared as - * `Transport failure calling function_execute after 300401ms`. - * - * The timers are therefore disarmed rather than re-negotiated: callers on this - * path already own an in-process deadline (an `AbortController` armed with the - * plan timeout), and a second, shorter, invisible deadline underneath it is - * exactly the bug. Disarming leaves one enforcement point instead of two that - * disagree. - * - * Bun accepts only the boolean/zero form of `timeout`. Measured on Bun 1.3.14 - * against a server that withholds response headers, with the numeric behavior - * rechecked on Bun 1.4.2, so the numbers below are the real deadline rather - * than an inferred one: - * - * no option -> THREW 300028ms (TimeoutError) <- the 300s default - * timeout: false -> RESOLVED 310031ms <- disarmed - * timeout: 1000 -> RESOLVED 3008ms on a 3s request <- numeric ignored - * - * So a positive numeric `timeout` silently changes nothing on this version; the - * numeric idle-deadline form and `BUN_CONFIG_HTTP_IDLE_TIMEOUT` both exist only - * on Bun's `main`. Do not "improve" this into a numeric pass-through until the - * pinned version supports it, and re-measure with the probe above if you do. - * - * Measured on Node 23.11 (built-in fetch, bundled undici 6.21.2) driving an - * npm `undici@7.29.0` `Agent` — a wider version split than the node-24 workers - * run, so the cross-copy `dispatcher` handoff is proven, not assumed: - * - * timeout: false only -> THREW 300996ms (fetch failed, - * HeadersTimeoutError) <- ignored - * dispatcher armed at 200ms -> THREW 1011ms <- honored - * dispatcher with 0/0 vs a 310s server -> RESOLVED 310016ms <- disarmed - * - * `bun-types@1.4.1` declares `timeout` on `BunFetchRequestInit`, but the shared - * DOM lib does not declare that Bun extension or undici's `dispatcher`. The - * interface below therefore stays local so this cross-runtime helper does not - * depend on either runtime's ambient types. - */ - -/** - * `RequestInit` plus each runtime's transport-timer control, which the DOM lib - * does not declare. Bun reads `timeout` (`false` disarms its idle timer) and - * ignores `dispatcher`; Node's undici fetch reads `dispatcher` and ignores - * `timeout`. - */ -export interface DeadlineRequestInit extends RequestInit { - timeout?: number | boolean - dispatcher?: Dispatcher -} - -let callerOwnedDeadlineDispatcher: Dispatcher | undefined - -/** - * The shared dispatcher whose header/body timers are disarmed, for runtimes - * whose fetch is undici. Constructed lazily so Bun — where `dispatcher` is - * ignored and `timeout: false` does the disarming — never pays for it, and - * shared so repeated internal-route calls reuse its keep-alive connections. - */ -function getCallerOwnedDeadlineDispatcher(): Dispatcher | undefined { - if (typeof process !== 'undefined' && process.versions?.bun) { - return undefined - } - callerOwnedDeadlineDispatcher ??= new Agent({ headersTimeout: 0, bodyTimeout: 0 }) - return callerOwnedDeadlineDispatcher -} - -/** - * Disarms the transport timers so the caller's own deadline is the only one - * in force. - * - * Only use this where the caller genuinely enforces a deadline in-process — - * an `AbortSignal` wired to a timer or an execution budget. Without one, a - * request to a peer that never answers would hang until the socket dies. - */ -export function withCallerOwnedDeadline(init: RequestInit): DeadlineRequestInit { - const dispatcher = getCallerOwnedDeadlineDispatcher() - return { ...init, timeout: false, ...(dispatcher ? { dispatcher } : {}) } -} - -/** - * Whether a caught error is the transport giving up rather than the request - * being cancelled or the peer erroring. - * - * Bun reports both an unanswered request and a truncated body as - * `TimeoutError: The operation timed out.`, and surfaces a severed connection - * as a bare `fetch failed`. Node's undici reports its expired header/body - * timers and severed connections alike as `TypeError: fetch failed`, with the - * distinguishing `HeadersTimeoutError`/`BodyTimeoutError` only on `cause` — - * none of which name the hop, the elapsed time, or the fact that a cap was - * hit. Callers use this to annotate before rethrowing so a transport cap - * cannot masquerade as a failure of the work itself. - */ -export function isTransportTimeoutError(error: unknown): error is Error { - if (!(error instanceof Error)) return false - if (error.name === 'TimeoutError') return true - return error.name === 'TypeError' && error.message === 'fetch failed' -} diff --git a/apps/sim/lib/core/utils/response-format.test.ts b/apps/sim/lib/core/utils/response-format.test.ts index 646b3b8ccd8..c336c735c72 100644 --- a/apps/sim/lib/core/utils/response-format.test.ts +++ b/apps/sim/lib/core/utils/response-format.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest' -import { extractFieldValues, traverseObjectPath } from '@/lib/core/utils/response-format' +import { traverseObjectPath } from '@/lib/core/utils/response-format' import { LARGE_ARRAY_MANIFEST_VERSION, type LargeArrayManifest, @@ -42,9 +42,6 @@ describe('response format traversal', () => { const manifest = createManifest() expect(traverseObjectPath({ output: { rows: manifest } }, 'output.rows.length')).toBe(100_000) - expect( - extractFieldValues({ output: { rows: manifest } }, ['block-1_output.rows.length'], 'block-1') - ).toEqual({ 'output.rows.length': 100_000 }) }) it('does not perform indexed manifest reads in sync traversal', () => { diff --git a/apps/sim/lib/core/utils/response-format.ts b/apps/sim/lib/core/utils/response-format.ts index a5d4a38f30b..caf1c9a97b5 100644 --- a/apps/sim/lib/core/utils/response-format.ts +++ b/apps/sim/lib/core/utils/response-format.ts @@ -82,37 +82,6 @@ export function parseResponseFormatSafely( } } -/** - * Extract field values from a parsed JSON object based on selected output paths - * Used for both workspace and chat client field extraction - */ -export function extractFieldValues( - parsedContent: any, - selectedOutputs: string[], - blockId: string -): Record { - const extractedValues: Record = {} - - for (const outputId of selectedOutputs) { - const blockIdForOutput = extractBlockIdFromOutputId(outputId) - - if (blockIdForOutput !== blockId) { - continue - } - - const path = extractPathFromOutputId(outputId, blockIdForOutput) - - if (path) { - const current = traverseObjectPathInternal(parsedContent, path) - if (current !== undefined) { - extractedValues[path] = current - } - } - } - - return extractedValues -} - /** * Extract block ID from output ID * Handles both formats: "blockId" and "blockId_path" or "blockId.path" diff --git a/apps/sim/lib/core/utils/scheduling.ts b/apps/sim/lib/core/utils/scheduling.ts deleted file mode 100644 index ad24ae7d2ba..00000000000 --- a/apps/sim/lib/core/utils/scheduling.ts +++ /dev/null @@ -1,51 +0,0 @@ -/** - * Converts schedule options to a cron expression - */ -export function convertScheduleOptionsToCron( - scheduleType: string, - options: Record -): string { - switch (scheduleType) { - case 'minutes': { - const interval = options.minutesInterval || '15' - // For example, if options.minutesStartingAt is provided, use that as the start minute. - return `*/${interval} * * * *` - } - case 'hourly': { - // When scheduling hourly, take the specified minute offset - return `${options.hourlyMinute || '00'} * * * *` - } - case 'daily': { - // Expected dailyTime in HH:MM - const [minute, hour] = (options.dailyTime || '00:09').split(':') - return `${minute || '00'} ${hour || '09'} * * *` - } - case 'weekly': { - // Expected weeklyDay as MON, TUE, etc. and weeklyDayTime in HH:MM - const dayMap: Record = { - MON: 1, - TUE: 2, - WED: 3, - THU: 4, - FRI: 5, - SAT: 6, - SUN: 0, - } - const day = dayMap[options.weeklyDay || 'MON'] - const [minute, hour] = (options.weeklyDayTime || '00:09').split(':') - return `${minute || '00'} ${hour || '09'} * * ${day}` - } - case 'monthly': { - // Expected monthlyDay and monthlyTime in HH:MM - const day = options.monthlyDay || '1' - const [minute, hour] = (options.monthlyTime || '00:09').split(':') - return `${minute || '00'} ${hour || '09'} ${day} * *` - } - case 'custom': { - // Use the provided cron expression directly - return options.cronExpression - } - default: - throw new Error('Unsupported schedule type') - } -} diff --git a/apps/sim/lib/core/utils/timezone.ts b/apps/sim/lib/core/utils/timezone.ts index fe4e19c104e..f41593c1761 100644 --- a/apps/sim/lib/core/utils/timezone.ts +++ b/apps/sim/lib/core/utils/timezone.ts @@ -215,11 +215,6 @@ export function zonedWallClock(instant: Date, timeZone: string): string { return `${formatIsoYear(wall.year)}-${pad(wall.month)}-${pad(wall.day)}T${pad(wall.hour)}:${pad(wall.minute)}` } -/** The current wall-clock time in `timeZone` as a naive `yyyy-MM-ddTHH:mm` string. */ -export function wallClockNow(timeZone: string): string { - return zonedWallClock(new Date(), timeZone) -} - /** * A `Date` whose device-local fields (year…minute) equal the wall-clock time of * `instant` in `timeZone`. It deliberately does NOT represent the same instant — diff --git a/apps/sim/lib/core/utils/validation.ts b/apps/sim/lib/core/utils/validation.ts index ddb3136c6fa..265780601df 100644 --- a/apps/sim/lib/core/utils/validation.ts +++ b/apps/sim/lib/core/utils/validation.ts @@ -31,27 +31,6 @@ export function validateName(name: string): string { .replace(/\s+/g, ' ') // Collapse multiple spaces into single spaces } -/** - * Checks if a name contains invalid characters - * - * @param name - The name to check - * @returns True if the name is valid, false otherwise - */ -export function isValidName(name: string): boolean { - return /^[a-zA-Z0-9_\s]*$/.test(name) -} - -/** - * Gets a list of invalid characters in a name - * - * @param name - The name to check - * @returns Array of invalid characters found - */ -export function getInvalidCharacters(name: string): string[] { - const invalidChars = name.match(/[^a-zA-Z0-9_\s]/g) - return invalidChars ? [...new Set(invalidChars)] : [] -} - /** * Escapes non-ASCII characters in JSON string for HTTP header safety. * Dropbox API requires characters 0x7F and all non-ASCII to be escaped as \uXXXX. diff --git a/apps/sim/lib/execution/execution-deadline-header.test.ts b/apps/sim/lib/execution/execution-deadline-header.test.ts index 917e89f7205..901da160baf 100644 --- a/apps/sim/lib/execution/execution-deadline-header.test.ts +++ b/apps/sim/lib/execution/execution-deadline-header.test.ts @@ -2,27 +2,18 @@ import { describe, expect, it } from 'vitest' import { INTERNAL_EXECUTION_DEADLINE_HEADER, parseExecutionDeadlineHeader, - parseRemainingExecutionDeadlineMs, } from '@/lib/execution/execution-deadline-header' describe('internal execution deadline header', () => { - it('returns the remaining budget from an absolute deadline', () => { + it('parses an absolute deadline', () => { const headers = new Headers({ [INTERNAL_EXECUTION_DEADLINE_HEADER]: '12000' }) expect(parseExecutionDeadlineHeader(headers)).toBe(12000) - expect(parseRemainingExecutionDeadlineMs(headers, 5000)).toBe(7000) - }) - - it('returns a one millisecond budget for an expired deadline', () => { - const headers = new Headers({ [INTERNAL_EXECUTION_DEADLINE_HEADER]: '4000' }) - - expect(parseRemainingExecutionDeadlineMs(headers, 5000)).toBe(1) }) it.each(['', 'invalid', '1.5', '-1'])('ignores invalid deadline %j', (value) => { const headers = new Headers({ [INTERNAL_EXECUTION_DEADLINE_HEADER]: value }) expect(parseExecutionDeadlineHeader(headers)).toBeUndefined() - expect(parseRemainingExecutionDeadlineMs(headers, 5000)).toBeUndefined() }) }) diff --git a/apps/sim/lib/execution/execution-deadline-header.ts b/apps/sim/lib/execution/execution-deadline-header.ts index 15536d80bfb..2d1b2dc81a3 100644 --- a/apps/sim/lib/execution/execution-deadline-header.ts +++ b/apps/sim/lib/execution/execution-deadline-header.ts @@ -16,14 +16,3 @@ export function parseExecutionDeadlineHeader(headers: Headers): number | undefin const deadline = Number(rawDeadline) return Number.isSafeInteger(deadline) && deadline > 0 ? deadline : undefined } - -/** Returns the remaining trusted workflow budget carried by an internal request. */ -export function parseRemainingExecutionDeadlineMs( - headers: Headers, - now: number = Date.now() -): number | undefined { - const deadline = parseExecutionDeadlineHeader(headers) - if (deadline === undefined) return undefined - - return Math.max(1, deadline - now) -} diff --git a/apps/sim/lib/execution/payloads/large-execution-value.test.ts b/apps/sim/lib/execution/payloads/large-execution-value.test.ts index 6b7559cc45b..86e07cdb30b 100644 --- a/apps/sim/lib/execution/payloads/large-execution-value.test.ts +++ b/apps/sim/lib/execution/payloads/large-execution-value.test.ts @@ -4,10 +4,7 @@ import { LARGE_ARRAY_MANIFEST_VERSION, type LargeArrayManifest, } from '@/lib/execution/payloads/large-array-manifest-metadata' -import { - collectLargeValueExecutionIds, - collectLargeValueKeys, -} from '@/lib/execution/payloads/large-execution-value' +import { collectLargeValueKeys } from '@/lib/execution/payloads/large-execution-value' import { LARGE_VALUE_REF_MARKER, LARGE_VALUE_REF_VERSION, @@ -41,23 +38,7 @@ function largeArrayManifest(executionId: string): LargeArrayManifest { } } -describe('collectLargeValueExecutionIds', () => { - it('collects deduplicated execution IDs from nested refs and manifests', () => { - const executionIds = collectLargeValueExecutionIds({ - blockStates: { - upstream: { - output: { - directRef: largeValueRef('lv_ABCDEFGHIJKL', 'execution-a'), - inheritedManifest: largeArrayManifest('execution-b'), - duplicateRef: largeValueRef('lv_NOPQRSTUVWXY', 'execution-a'), - }, - }, - }, - }) - - expect(executionIds).toEqual(['execution-a', 'execution-b']) - }) - +describe('collectLargeValueKeys', () => { it('collects deduplicated storage keys from nested refs and manifests', () => { const keys = collectLargeValueKeys({ directRef: largeValueRef('lv_ABCDEFGHIJKL', 'execution-a'), diff --git a/apps/sim/lib/execution/payloads/large-execution-value.ts b/apps/sim/lib/execution/payloads/large-execution-value.ts index e3b0ad1adf0..9e3860c4a10 100644 --- a/apps/sim/lib/execution/payloads/large-execution-value.ts +++ b/apps/sim/lib/execution/payloads/large-execution-value.ts @@ -26,67 +26,12 @@ export function parseLargeExecutionValue(value: unknown): LargeExecutionValue | } } -/** - * Finds execution IDs referenced by large values embedded in persisted execution state. - */ -export function collectLargeValueExecutionIds(value: unknown): string[] { - const executionIds = new Set() - collectLargeValueExecutionIdsInto(value, executionIds, new WeakSet()) - return Array.from(executionIds) -} - export function collectLargeValueKeys(value: unknown): string[] { const keys = new Set() collectLargeValueKeysInto(value, keys, new WeakSet()) return Array.from(keys) } -function collectLargeValueExecutionIdsInto( - value: unknown, - executionIds: Set, - seen: WeakSet -): void { - if (!value || typeof value !== 'object') { - return - } - - if (seen.has(value)) { - return - } - seen.add(value) - - if (isLargeValueRef(value)) { - addExecutionId(value, executionIds) - collectLargeValueExecutionIdsInto(value.preview, executionIds, seen) - return - } - - if (isLargeArrayManifest(value)) { - for (const chunk of value.chunks) { - addExecutionId(chunk.ref, executionIds) - } - collectLargeValueExecutionIdsInto(value.preview, executionIds, seen) - return - } - - if (Array.isArray(value)) { - for (const item of value) { - collectLargeValueExecutionIdsInto(item, executionIds, seen) - } - return - } - - for (const item of Object.values(value)) { - collectLargeValueExecutionIdsInto(item, executionIds, seen) - } -} - -function addExecutionId(ref: LargeValueRef, executionIds: Set): void { - if (ref.executionId) { - executionIds.add(ref.executionId) - } -} - function collectLargeValueKeysInto(value: unknown, keys: Set, seen: WeakSet): void { if (!value || typeof value !== 'object') { return diff --git a/apps/sim/lib/execution/remote-sandbox/code-failure.ts b/apps/sim/lib/execution/remote-sandbox/code-failure.ts new file mode 100644 index 00000000000..a7cc62f873e --- /dev/null +++ b/apps/sim/lib/execution/remote-sandbox/code-failure.ts @@ -0,0 +1,27 @@ +import type { SandboxCodeResult, SandboxCommandResult } from '@/lib/execution/remote-sandbox/types' + +/** + * Converts a failed code command into a code result, naming the error from the + * last `Name: message` line of the traceback (stderr, else stdout). + */ +export function processCodeFailure(result: SandboxCommandResult): SandboxCodeResult { + const traceback = result.stderr || result.stdout + const errorLine = traceback + .split('\n') + .reverse() + .find((line) => /^[A-Za-z_$][\w.$]*(?:Error|Exception|Interrupt|Exit)?:\s*/.test(line.trim())) + ?.trim() + const separator = errorLine?.indexOf(':') ?? -1 + const parsedErrorLine = errorLine ?? '' + const name = separator > 0 ? parsedErrorLine.slice(0, separator) : 'Error' + const value = + separator > 0 + ? parsedErrorLine.slice(separator + 1).trim() + : parsedErrorLine || 'Execution failed' + return { + text: '', + stdout: result.stdout, + stderr: result.stderr, + error: { name, value, traceback }, + } +} diff --git a/apps/sim/lib/execution/remote-sandbox/daytona.ts b/apps/sim/lib/execution/remote-sandbox/daytona.ts index 683dfae61e1..0d709dc8ac9 100644 --- a/apps/sim/lib/execution/remote-sandbox/daytona.ts +++ b/apps/sim/lib/execution/remote-sandbox/daytona.ts @@ -16,6 +16,7 @@ import { isNonRetryableExecutionError, SandboxLaunchIndeterminateError, } from '@/lib/execution/non-retryable-error' +import { processCodeFailure } from '@/lib/execution/remote-sandbox/code-failure' import { appendStreamedSandboxOutput, assertSandboxProcessOutputWithinLimit, @@ -73,28 +74,6 @@ function assertSafeProcessEnvironment(envs: Record | undefined): } } -function processCodeFailure(result: SandboxCommandResult): SandboxCodeResult { - const traceback = result.stderr || result.stdout - const errorLine = traceback - .split('\n') - .reverse() - .find((line) => /^[A-Za-z_$][\w.$]*(?:Error|Exception|Interrupt|Exit)?:\s*/.test(line.trim())) - ?.trim() - const separator = errorLine?.indexOf(':') ?? -1 - const parsedErrorLine = errorLine ?? '' - const name = separator > 0 ? parsedErrorLine.slice(0, separator) : 'Error' - const value = - separator > 0 - ? parsedErrorLine.slice(separator + 1).trim() - : parsedErrorLine || 'Execution failed' - return { - text: '', - stdout: result.stdout, - stderr: result.stderr, - error: { name, value, traceback }, - } -} - function snapshotFor(kind: SandboxKind, imageRef?: string): string { if (kind === 'mothership') { const snapshot = env.DAYTONA_SHELL_SNAPSHOT_ID?.trim() diff --git a/apps/sim/lib/execution/remote-sandbox/e2b.ts b/apps/sim/lib/execution/remote-sandbox/e2b.ts index 838196b91b0..7d6d5868933 100644 --- a/apps/sim/lib/execution/remote-sandbox/e2b.ts +++ b/apps/sim/lib/execution/remote-sandbox/e2b.ts @@ -28,6 +28,7 @@ import { sandboxCliToolRecipes, sandboxCliVerificationCommand, } from '@/lib/execution/remote-sandbox/cli-tools.server' +import { processCodeFailure } from '@/lib/execution/remote-sandbox/code-failure' import { recordSandboxProcess, reportUnsettledSandboxProcess, @@ -176,28 +177,6 @@ function reachedE2BProviderLimit( ) } -function processCodeFailure(result: SandboxCommandResult): SandboxCodeResult { - const traceback = result.stderr || result.stdout - const errorLine = traceback - .split('\n') - .reverse() - .find((line) => /^[A-Za-z_$][\w.$]*(?:Error|Exception|Interrupt|Exit)?:\s*/.test(line.trim())) - ?.trim() - const separator = errorLine?.indexOf(':') ?? -1 - const parsedErrorLine = errorLine ?? '' - const name = separator > 0 ? parsedErrorLine.slice(0, separator) : 'Error' - const value = - separator > 0 - ? parsedErrorLine.slice(separator + 1).trim() - : parsedErrorLine || 'Execution failed' - return { - text: '', - stdout: result.stdout, - stderr: result.stderr, - error: { name, value, traceback }, - } -} - function functionTemplateRef(): string { const templateRef = env.E2B_FUNCTION_TEMPLATE_ID if (!templateRef) { diff --git a/apps/sim/lib/execution/remote-sandbox/session-lock.ts b/apps/sim/lib/execution/remote-sandbox/session-lock.ts index 5e6ef9d7135..714036ade95 100644 --- a/apps/sim/lib/execution/remote-sandbox/session-lock.ts +++ b/apps/sim/lib/execution/remote-sandbox/session-lock.ts @@ -1,6 +1,6 @@ -import { randomUUID } from 'node:crypto' import { setTimeout as delay } from 'node:timers/promises' import { createLogger } from '@sim/logger' +import { generateId } from '@sim/utils/id' import { acquireLock, extendLock, releaseLock } from '@/lib/core/config/redis' const logger = createLogger('SandboxSessionLock') @@ -17,7 +17,7 @@ export async function withSandboxSessionLock( signal.throwIfAborted() localOwners.add(key) const lockKey = `sandbox-session:${key}` - const owner = randomUUID() + const owner = generateId() const lost = new AbortController() const leaseSignal = AbortSignal.any([signal, lost.signal]) let acquired = false diff --git a/apps/sim/lib/execution/remote-sandbox/workspace-sandboxes.ts b/apps/sim/lib/execution/remote-sandbox/workspace-sandboxes.ts index d092c4a7947..b24a5cd5567 100644 --- a/apps/sim/lib/execution/remote-sandbox/workspace-sandboxes.ts +++ b/apps/sim/lib/execution/remote-sandbox/workspace-sandboxes.ts @@ -293,12 +293,6 @@ export async function listWorkspaceSandboxesPage(params: { return { data: await attachBuildStatus(page.data), nextCursorKeys: page.nextCursorKeys } } -/** The whole set, name-ordered, for the surfaces that render every sandbox at once. */ -export async function listWorkspaceSandboxes(workspaceId: string): Promise { - const page = await listWorkspaceSandboxesPage({ workspaceId }) - return page.data -} - /** * Reads one sandbox back, scoped to its workspace. Fetches the single row rather * than filtering a full list — this runs after every create and update. diff --git a/apps/sim/lib/platform-context/application/authorization.ts b/apps/sim/lib/platform-context/application/authorization.ts deleted file mode 100644 index 11fe1a00bf5..00000000000 --- a/apps/sim/lib/platform-context/application/authorization.ts +++ /dev/null @@ -1,12 +0,0 @@ -import type { DelegatedPrincipal } from '@sim/auth/principal' -import type { ActiveWorkspaceApplicationContext } from '@/lib/workspaces/application/workspace-context' - -export const PLATFORM_CONTEXT_DELEGATION_AUDIENCE = 'sim:platform-context' - -export const platformContextDelegationPolicy = { - audience: PLATFORM_CONTEXT_DELEGATION_AUDIENCE, - isWithinScope: ( - principal: DelegatedPrincipal, - context: ActiveWorkspaceApplicationContext - ): boolean => principal.workspaceId === context.workspaceId, -} as const diff --git a/apps/sim/lib/platform-context/application/context.ts b/apps/sim/lib/platform-context/application/context.ts deleted file mode 100644 index f25cc85b882..00000000000 --- a/apps/sim/lib/platform-context/application/context.ts +++ /dev/null @@ -1,14 +0,0 @@ -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { - type ActiveWorkspaceApplicationContext, - loadActiveWorkspaceApplicationContext, -} from '@/lib/workspaces/application/workspace-context' - -/** Loads canonical active workspace state before authorizing a live platform-context read. */ -export async function resolvePlatformContextWorkspace( - workspaceId: string -): Promise { - const context = await loadActiveWorkspaceApplicationContext(workspaceId) - if (!context) throw new OrchestrationError('not_found', 'Workspace not found') - return context -} diff --git a/apps/sim/lib/platform-context/application/operations.ts b/apps/sim/lib/platform-context/application/operations.ts deleted file mode 100644 index b1dcf4c6a52..00000000000 --- a/apps/sim/lib/platform-context/application/operations.ts +++ /dev/null @@ -1,35 +0,0 @@ -import { defineWorkspaceOperation } from '@/lib/core/application/workspace-operation' - -const LIVE_PLATFORM_CONTEXT_PRINCIPAL_POLICY = { - principalKinds: ['delegated'], - delegatedServices: ['copilot'], -} as const - -/** - * What Sim reads about itself before it can answer at all: the workspace's plan, - * its seat and usage state, and whether the organization is on the enterprise - * tier. No permission-group key names them, and a member whose group withheld - * them would get an agent that cannot tell them why anything is unavailable — - * withholding the description of a restriction is not the same as applying one. - */ -export const platformContextOperations = { - // permission-group-exempt: the plan and usage state every answer is framed against; withholding it blanks the agent rather than restricting it - readAccountBilling: defineWorkspaceOperation({ - id: 'platform_context.account_billing.read', - minimumRole: 'read', - workspaceApiKey: 'deny', - capability: 'none', - ...LIVE_PLATFORM_CONTEXT_PRINCIPAL_POLICY, - }), - // permission-group-exempt: reports which enterprise features the organization has, the frame the restrictions themselves are described in - readEnterpriseContext: defineWorkspaceOperation({ - id: 'platform_context.enterprise.read', - minimumRole: 'read', - workspaceApiKey: 'deny', - capability: 'none', - ...LIVE_PLATFORM_CONTEXT_PRINCIPAL_POLICY, - }), -} as const - -export type PlatformContextOperation = - (typeof platformContextOperations)[keyof typeof platformContextOperations] diff --git a/apps/sim/lib/platform-context/application/platform-context-use-cases.test.ts b/apps/sim/lib/platform-context/application/platform-context-use-cases.test.ts deleted file mode 100644 index d9cd60744ec..00000000000 --- a/apps/sim/lib/platform-context/application/platform-context-use-cases.test.ts +++ /dev/null @@ -1,209 +0,0 @@ -import { - createDelegatedPrincipal, - createWorkspaceApiKeyPrincipal, -} from '@sim/testing/factories/principal.factory' -import { - permissionGroupsResolveMock, - permissionGroupsResolveMockFns, -} from '@sim/testing/mocks/permission-groups-resolve.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { - workspaceContextMock, - workspaceContextMockFns, -} from '@sim/testing/mocks/workspace-context.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const hoisted = vi.hoisted(() => ({ - getAccountBillingSnapshot: vi.fn(), - getWorkspaceHostContextForViewer: vi.fn(), -})) - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -vi.mock('@/lib/workspaces/application/workspace-context', () => workspaceContextMock) - -vi.mock('@/lib/billing/core/account-billing-snapshot', () => ({ - getAccountBillingSnapshot: hoisted.getAccountBillingSnapshot, -})) - -vi.mock('@/lib/workspaces/host-context', () => ({ - getWorkspaceHostContextForViewer: hoisted.getWorkspaceHostContextForViewer, -})) - -vi.mock('@/lib/permission-groups/resolve.server', () => permissionGroupsResolveMock) - -import { DEFAULT_PERMISSION_GROUP_CONFIG } from '@/lib/permission-groups/fields' -import { readAccountBilling } from '@/lib/platform-context/application/read-account-billing' -import { readEnterpriseContext } from '@/lib/platform-context/application/read-enterprise-context' - -const mocks = { - ...hoisted, - loadWorkspace: workspaceContextMockFns.mockLoadActiveWorkspaceApplicationContext, - resolvePermission: workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission, - resolveVerifiedUserAccessControlContext: - permissionGroupsResolveMockFns.mockResolveVerifiedUserAccessControlContext, -} - -const workspace = { - workspaceId: 'workspace-1', - workspaceOrganizationId: 'org-1', - allowPersonalApiKeys: true, - billedAccountUserId: 'owner-1', -} - -function copilotPrincipal() { - return createDelegatedPrincipal({ audience: 'sim:platform-context' }) -} - -describe('platform context application use cases', () => { - beforeEach(() => { - mocks.loadWorkspace.mockResolvedValue(workspace) - mocks.resolvePermission.mockResolvedValue('read') - }) - - it('authorizes a current Copilot subject before reading account billing', async () => { - const snapshot = { - plan: 'pro', - billingScope: 'user', - organizationId: null, - usage: {}, - credits: {}, - } - mocks.getAccountBillingSnapshot.mockResolvedValue(snapshot) - - await expect( - readAccountBilling.execute({ - principal: copilotPrincipal(), - input: { workspaceId: 'workspace-1' }, - }) - ).resolves.toBe(snapshot) - - expect(mocks.resolvePermission).toHaveBeenCalledWith( - 'user-1', - 'workspace-1', - 'org-1', - undefined, - { forUpdate: undefined } - ) - expect(mocks.getAccountBillingSnapshot).toHaveBeenCalledWith('user-1') - }) - - it.each([ - { - name: 'workspace API key', - principal: createWorkspaceApiKeyPrincipal(), - }, - { - name: 'executor delegation', - principal: { ...copilotPrincipal(), serviceId: 'executor' as const }, - }, - ])('rejects a $name before loading protected account context', async ({ principal }) => { - await expect( - readAccountBilling.execute({ principal, input: { workspaceId: 'workspace-1' } }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.loadWorkspace).not.toHaveBeenCalled() - expect(mocks.getAccountBillingSnapshot).not.toHaveBeenCalled() - }) - - it('does not load enterprise context when current workspace access is absent', async () => { - mocks.resolvePermission.mockResolvedValue(null) - - await expect( - readEnterpriseContext.execute({ - principal: copilotPrincipal(), - input: { workspaceId: 'workspace-1' }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.getWorkspaceHostContextForViewer).not.toHaveBeenCalled() - expect(mocks.resolveVerifiedUserAccessControlContext).not.toHaveBeenCalled() - }) - - it('projects enterprise context only after authorization', async () => { - mocks.getWorkspaceHostContextForViewer.mockResolvedValue({ - workspace: { - id: 'workspace-1', - name: 'Customer Support', - workspaceMode: 'collaborative', - }, - hostOrganizationId: 'org-1', - ownerBilling: { plan: 'enterprise', isEnterprise: true }, - viewer: { - permission: 'admin', - isHostOrganizationMember: false, - isHostOrganizationAdmin: false, - organizationRole: null, - }, - }) - mocks.resolveVerifiedUserAccessControlContext.mockResolvedValue({ - entitled: true, - permissionGroup: null, - config: DEFAULT_PERMISSION_GROUP_CONFIG, - }) - - await expect( - readEnterpriseContext.execute({ - principal: copilotPrincipal(), - input: { workspaceId: 'workspace-1' }, - }) - ).resolves.toMatchObject({ - workspace: { - id: 'workspace-1', - capabilities: { canRead: true, canEdit: true, canDeploy: true }, - }, - organization: { - id: 'org-1', - relationship: 'external', - canManageOrganization: false, - }, - accessControl: { entitled: true }, - }) - expect(mocks.getWorkspaceHostContextForViewer).toHaveBeenCalledWith('workspace-1', 'user-1') - }) - - it('allows read-role execution but hides deployment when every deploy surface is hidden', async () => { - mocks.getWorkspaceHostContextForViewer.mockResolvedValue({ - workspace: { - id: 'workspace-1', - name: 'Customer Support', - workspaceMode: 'collaborative', - }, - hostOrganizationId: 'org-1', - ownerBilling: { plan: 'enterprise', isEnterprise: true }, - viewer: { - permission: 'read', - isHostOrganizationMember: true, - isHostOrganizationAdmin: false, - organizationRole: 'member', - }, - }) - mocks.resolveVerifiedUserAccessControlContext.mockResolvedValue({ - entitled: true, - permissionGroup: null, - config: { - ...DEFAULT_PERMISSION_GROUP_CONFIG, - hideDeployApi: true, - hideDeployMcp: true, - hideDeployChatbot: true, - }, - }) - - await expect( - readEnterpriseContext.execute({ - principal: copilotPrincipal(), - input: { workspaceId: 'workspace-1' }, - }) - ).resolves.toMatchObject({ - workspace: { - capabilities: { - canRead: true, - canEdit: false, - canRun: true, - canDeploy: false, - canManageWorkspace: false, - }, - }, - }) - }) -}) diff --git a/apps/sim/lib/platform-context/application/read-account-billing.ts b/apps/sim/lib/platform-context/application/read-account-billing.ts deleted file mode 100644 index ee3bd7ed698..00000000000 --- a/apps/sim/lib/platform-context/application/read-account-billing.ts +++ /dev/null @@ -1,22 +0,0 @@ -import { requirePrincipalSubjectUserId } from '@sim/auth/principal' -import { - type AccountBillingSnapshot, - getAccountBillingSnapshot, -} from '@/lib/billing/core/account-billing-snapshot' -import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' -import { platformContextDelegationPolicy } from '@/lib/platform-context/application/authorization' -import { resolvePlatformContextWorkspace } from '@/lib/platform-context/application/context' -import { platformContextOperations } from '@/lib/platform-context/application/operations' - -export interface ReadAccountBillingInput { - workspaceId: string -} - -export const readAccountBilling = defineAuthorizedWorkspaceUseCase({ - operation: platformContextOperations.readAccountBilling, - resolveContext: ({ input }: { input: ReadAccountBillingInput }) => - resolvePlatformContextWorkspace(input.workspaceId), - authorizationOptions: { delegation: platformContextDelegationPolicy }, - execute: async ({ principal }): Promise => - getAccountBillingSnapshot(requirePrincipalSubjectUserId(principal)), -}) diff --git a/apps/sim/lib/platform-context/application/read-enterprise-context.ts b/apps/sim/lib/platform-context/application/read-enterprise-context.ts deleted file mode 100644 index cbca10f2ca6..00000000000 --- a/apps/sim/lib/platform-context/application/read-enterprise-context.ts +++ /dev/null @@ -1,89 +0,0 @@ -import { requirePrincipalSubjectUserId } from '@sim/auth/principal' -import { permissionSatisfies } from '@sim/platform-authz/workspace' -import { defineAuthorizedWorkspaceUseCase } from '@/lib/core/application' -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { capabilityDeniedBy } from '@/lib/permission-groups/capability-assertions' -import { getActivePermissionGroupRestrictions } from '@/lib/permission-groups/features' -import { platformContextDelegationPolicy } from '@/lib/platform-context/application/authorization' -import { resolvePlatformContextWorkspace } from '@/lib/platform-context/application/context' -import { platformContextOperations } from '@/lib/platform-context/application/operations' -import { getWorkspaceHostContextForViewer } from '@/lib/workspaces/host-context' -import { resolveVerifiedUserAccessControlContext } from '@/ee/access-control/utils/permission-check' - -const ENTERPRISE_PERMISSION_DOCUMENTATION = [ - { - title: 'Roles and permissions', - path: 'docs/platform/permissions.mdx', - url: 'https://docs.sim.ai/platform/permissions', - }, - { - title: 'Enterprise Access Control', - path: 'docs/platform/enterprise/access-control.mdx', - url: 'https://docs.sim.ai/platform/enterprise/access-control', - }, -] as const - -export interface ReadEnterpriseContextInput { - workspaceId: string -} - -export const readEnterpriseContext = defineAuthorizedWorkspaceUseCase({ - operation: platformContextOperations.readEnterpriseContext, - resolveContext: ({ input }: { input: ReadEnterpriseContextInput }) => - resolvePlatformContextWorkspace(input.workspaceId), - authorizationOptions: { delegation: platformContextDelegationPolicy }, - async execute({ principal, context }) { - const userId = requirePrincipalSubjectUserId(principal) - const hostContext = await getWorkspaceHostContextForViewer(context.workspaceId, userId) - if (!hostContext) { - throw new OrchestrationError('not_found', 'Workspace not found or you do not have access.') - } - - const accessControl = await resolveVerifiedUserAccessControlContext( - userId, - context.workspaceId, - hostContext.hostOrganizationId - ) - const canWrite = permissionSatisfies(hostContext.viewer.permission, 'write') - const canAdmin = permissionSatisfies(hostContext.viewer.permission, 'admin') - const allDeploymentSurfacesHidden = - capabilityDeniedBy('deploy.api', accessControl.config) && - capabilityDeniedBy('deploy.mcp', accessControl.config) && - capabilityDeniedBy('deploy.chat', accessControl.config) - - return { - workspace: { - id: hostContext.workspace.id, - name: hostContext.workspace.name, - mode: hostContext.workspace.workspaceMode, - permission: hostContext.viewer.permission, - capabilities: { - canRead: true, - canEdit: canWrite, - canRun: true, - canDeploy: canAdmin && !allDeploymentSurfacesHidden, - canManageWorkspace: canAdmin, - }, - }, - organization: hostContext.hostOrganizationId - ? { - id: hostContext.hostOrganizationId, - relationship: hostContext.viewer.isHostOrganizationMember ? 'internal' : 'external', - role: hostContext.viewer.organizationRole ?? null, - canManageOrganization: hostContext.viewer.isHostOrganizationAdmin, - canManageBilling: hostContext.viewer.isHostOrganizationAdmin, - plan: hostContext.ownerBilling.plan, - isEnterprise: hostContext.ownerBilling.isEnterprise, - } - : null, - accessControl: { - entitled: accessControl.entitled, - governingPermissionGroup: accessControl.permissionGroup, - effectiveConfig: accessControl.config, - activeRestrictions: getActivePermissionGroupRestrictions(accessControl.config), - }, - documentation: ENTERPRISE_PERMISSION_DOCUMENTATION, - resolvedAt: new Date().toISOString(), - } - }, -}) diff --git a/apps/sim/lib/vfs/limits.ts b/apps/sim/lib/vfs/limits.ts deleted file mode 100644 index 931531274b3..00000000000 --- a/apps/sim/lib/vfs/limits.ts +++ /dev/null @@ -1,61 +0,0 @@ -export const MAX_VFS_PATH_ITEMS = 100 -export const MAX_VFS_PATH_LENGTH = 4096 -export const MAX_VFS_TOTAL_PATH_BYTES = 64 * 1024 -export const MAX_VFS_PATH_SEGMENTS = 64 -export const MAX_VFS_SEGMENT_LENGTH = 255 - -export class VfsPathLimitError extends Error { - constructor(message: string) { - super(message) - this.name = 'VfsPathLimitError' - } -} - -function byteLength(value: string): number { - return new TextEncoder().encode(value).length -} - -export function validateVfsPathSegments(segments: readonly string[]): void { - if (segments.length > MAX_VFS_PATH_SEGMENTS) { - throw new VfsPathLimitError(`VFS paths cannot exceed ${MAX_VFS_PATH_SEGMENTS} segments`) - } - for (const segment of segments) { - if (segment.length === 0 || byteLength(segment) > MAX_VFS_SEGMENT_LENGTH) { - throw new VfsPathLimitError( - `VFS path segments must be between 1 and ${MAX_VFS_SEGMENT_LENGTH} bytes` - ) - } - } -} - -export function validateVfsPathBatch(paths: readonly string[]): void { - if (paths.length > MAX_VFS_PATH_ITEMS) { - throw new VfsPathLimitError(`VFS commands cannot exceed ${MAX_VFS_PATH_ITEMS} paths`) - } - let totalBytes = 0 - for (const path of paths) { - const pathBytes = byteLength(path) - totalBytes += pathBytes - if (pathBytes > MAX_VFS_PATH_LENGTH) { - throw new VfsPathLimitError(`VFS paths cannot exceed ${MAX_VFS_PATH_LENGTH} bytes`) - } - const segments = path - .trim() - .replace(/^\/+|\/+$/g, '') - .split('/') - .filter(Boolean) - .map((segment) => { - try { - return decodeURIComponent(segment) - } catch { - return segment - } - }) - validateVfsPathSegments(segments) - } - if (totalBytes > MAX_VFS_TOTAL_PATH_BYTES) { - throw new VfsPathLimitError( - `VFS command paths cannot exceed ${MAX_VFS_TOTAL_PATH_BYTES} total bytes` - ) - } -} diff --git a/apps/sim/lib/workflows/application/apply-workflow-operations.ts b/apps/sim/lib/workflows/application/apply-workflow-operations.ts index 4052746ebbd..83210d9a0bf 100644 --- a/apps/sim/lib/workflows/application/apply-workflow-operations.ts +++ b/apps/sim/lib/workflows/application/apply-workflow-operations.ts @@ -16,12 +16,9 @@ import { MAX_PLAN_REQUIRED } from '@/lib/execution/remote-sandbox/entitlement' import { resolvePermissionGroupConfig } from '@/lib/permission-groups/config-scope.server' import { notifyWorkflowUpdated } from '@/lib/realtime/notify' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { - type ActiveWorkflowApplicationContext, - resolveActiveWorkflowApplicationContext, -} from '@/lib/workflows/application/context' +import type { ActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { withWorkflowBlockScope } from '@/lib/workflows/application/workflow-block-scope' import { requireMutableWorkflow } from '@/lib/workflows/application/workflow-mutability' import { WorkflowOperationsNotAppliedError } from '@/lib/workflows/application/workflow-operations-error' @@ -256,17 +253,7 @@ async function resolveBaseGraph( */ export const applyWorkflowOperations = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.applyOperations, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: ApplyWorkflowOperationsInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }): Promise { if (input.operations.length === 0) { throw new OrchestrationError('validation', 'operations cannot be empty') diff --git a/apps/sim/lib/workflows/application/chat-deployments.ts b/apps/sim/lib/workflows/application/chat-deployments.ts index 44b27bede10..933808d6a10 100644 --- a/apps/sim/lib/workflows/application/chat-deployments.ts +++ b/apps/sim/lib/workflows/application/chat-deployments.ts @@ -1,6 +1,5 @@ import { AuditAction, AuditResourceType } from '@sim/audit' import { - type Principal, requirePrincipalSubjectUserId, resolvePrincipalAttribution, toPrincipalActor, @@ -16,10 +15,9 @@ import { } from '@/lib/chat-deployments/queries' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' -import { performChatDeploy, performChatUndeploy } from '@/lib/workflows/orchestration' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' +import { performChatDeploy } from '@/lib/workflows/orchestration' import { formatInternalOutputSelector } from '@/lib/workflows/streaming/output-selector' import { validateChatDeployAuth } from '@/ee/access-control/utils/permission-check' @@ -55,11 +53,6 @@ export interface DeployWorkflowChatInput { idempotencyKey?: string } -export interface UndeployWorkflowChatInput { - workflowId: string - assertedWorkspaceId?: string -} - function parseChatOutputConfigs(value: unknown[] | undefined): ChatOutputConfig[] | undefined { if (value === undefined) return undefined if ( @@ -89,22 +82,9 @@ function parseChatOutputConfigs(value: unknown[] | undefined): ChatOutputConfig[ return value } -function resolveWorkflowContext({ - principal, - input, -}: { - principal: Principal - input: I -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - export const deployWorkflowChat = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.deployChat, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { const existingDeployment = await getLiveChatDeploymentForWorkflow(context.workflowId) @@ -256,43 +236,3 @@ export const deployWorkflowChat = defineAuthorizedWorkflowUseCase({ }, }), }) - -export const undeployWorkflowChat = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.undeployChat, - resolveContext: resolveWorkflowContext, - async execute({ principal, context }) { - const deployment = await getLiveChatDeploymentForWorkflow(context.workflowId) - if (!deployment) { - throw new OrchestrationError('not_found', 'No active chat deployment found for this workflow') - } - - const attribution = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }) - const result = await performChatUndeploy({ - chatId: deployment.id, - userId: attribution.attributedUserId, - workspaceId: context.workspaceId, - projectLegacyAudit: false, - }) - if (!result.success) { - /** Only a genuinely absent deployment is concealed; anything else propagates. */ - const message = result.error ?? 'Failed to undeploy chat' - if (result.errorCode !== 'not_found') throw new Error(message) - throw new OrchestrationError('not_found', message) - } - return { workflowId: context.workflowId, deployment: toChatDeploymentView(deployment) } - }, - projectAudit: ({ result }) => ({ - action: AuditAction.CHAT_DELETED, - resourceType: AuditResourceType.CHAT, - resourceId: result.deployment.id, - resourceName: result.deployment.title || result.deployment.id, - description: `Deleted chat deployment "${result.deployment.title || result.deployment.id}"`, - metadata: { - workflowId: result.workflowId, - identifier: result.deployment.identifier || undefined, - authType: result.deployment.authType || undefined, - }, - }), -}) diff --git a/apps/sim/lib/workflows/application/delete-workflow.ts b/apps/sim/lib/workflows/application/delete-workflow.ts index 4a845fc9fcd..08c24205b7a 100644 --- a/apps/sim/lib/workflows/application/delete-workflow.ts +++ b/apps/sim/lib/workflows/application/delete-workflow.ts @@ -1,13 +1,12 @@ import { AuditAction, AuditResourceType } from '@sim/audit' -import { type Principal, resolvePrincipalAttribution } from '@sim/auth/principal' +import { resolvePrincipalAttribution } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { assertWorkflowMutable, WorkflowLockedError } from '@sim/platform-authz/workflow' import { OrchestrationError } from '@/lib/core/orchestration/types' import { notifyWorkflowDeleted, notifyWorkspaceWorkflowsChanged } from '@/lib/realtime/notify' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { requireWorkflowTransition } from '@/lib/workflows/application/transition-result' import { deleteWorkflowRecord } from '@/lib/workflows/orchestration' @@ -20,11 +19,7 @@ export interface DeleteWorkflowInput { export const deleteWorkflow = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.delete, - resolveContext: ({ principal, input }: { principal: Principal; input: DeleteWorkflowInput }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, context }) { try { await assertWorkflowMutable(context.workflowId) diff --git a/apps/sim/lib/workflows/application/deployments.ts b/apps/sim/lib/workflows/application/deployments.ts index a3e1185d7ca..0b7ce9cd071 100644 --- a/apps/sim/lib/workflows/application/deployments.ts +++ b/apps/sim/lib/workflows/application/deployments.ts @@ -1,5 +1,5 @@ import { AuditAction, AuditResourceType } from '@sim/audit' -import { type Principal, resolvePrincipalAttribution, toPrincipalActor } from '@sim/auth/principal' +import { resolvePrincipalAttribution, toPrincipalActor } from '@sim/auth/principal' import { assertWorkflowMutable, WorkflowLockedError } from '@sim/platform-authz/workflow' import { OrchestrationError, type OrchestrationErrorCode } from '@/lib/core/orchestration/types' import { listLiveWorkflowMcpToolsForWorkflow } from '@/lib/mcp/queries' @@ -7,9 +7,8 @@ import { notifyWorkflowReverted } from '@/lib/realtime/notify' import { listDeployedWebhookUrls } from '@/lib/webhooks/deployed-urls' import { requireWorkflowExecutionUserId } from '@/lib/workflows/application/authorization' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { checkNeedsRedeployment } from '@/lib/workflows/deployment-status' import { getWorkflowDeploymentSummary, @@ -68,19 +67,6 @@ export interface UpdateWorkflowVersionInput { description?: string | null } -function resolveWorkflowContext({ - principal, - input, -}: { - principal: Principal - input: I -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - function throwDeploymentFailure( result: { error?: string; errorCode?: OrchestrationErrorCode }, fallback: string @@ -104,7 +90,7 @@ async function requireMutableWorkflow(workflowId: string): Promise { export const deployWorkflow = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.deploy, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { await requireMutableWorkflow(context.workflowId) const attribution = resolvePrincipalAttribution(principal, { @@ -132,7 +118,7 @@ export const deployWorkflow = defineAuthorizedWorkflowUseCase({ export const undeployWorkflow = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.undeploy, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { if (!context.workflow.isDeployed) { throw new OrchestrationError('validation', 'Workflow is not deployed') @@ -175,7 +161,7 @@ export const undeployWorkflow = defineAuthorizedWorkflowUseCase({ export const activateWorkflowVersion = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.activateVersion, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { if (input.transition === 'rollback' && !context.workflow.isDeployed) { throw new OrchestrationError('validation', 'Workflow is not deployed') @@ -226,7 +212,7 @@ export const activateWorkflowVersion = defineAuthorizedWorkflowUseCase({ export const readWorkflowDeploymentStatus = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.read, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ context }) { const deploymentSummary = await getWorkflowDeploymentSummary(context.workflowId) const isDeployed = deploymentSummary.activeDeployment !== null @@ -249,7 +235,7 @@ export const readWorkflowDeploymentStatus = defineAuthorizedWorkflowUseCase({ export const revertWorkflowVersion = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.revertVersion, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { const userId = requireWorkflowExecutionUserId(principal) await requireMutableWorkflow(context.workflowId) @@ -288,7 +274,7 @@ export const revertWorkflowVersion = defineAuthorizedWorkflowUseCase({ export const updateWorkflowVersion = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.updateVersion, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ input, context }) { const updated = await updateDeploymentVersionMetadata({ workflowId: context.workflowId, diff --git a/apps/sim/lib/workflows/application/inspect-workflow-tools.ts b/apps/sim/lib/workflows/application/inspect-workflow-tools.ts index 01820b8e6f2..81cd3c9594b 100644 --- a/apps/sim/lib/workflows/application/inspect-workflow-tools.ts +++ b/apps/sim/lib/workflows/application/inspect-workflow-tools.ts @@ -18,9 +18,8 @@ import { projectIntegrationCatalog } from '@/lib/mothership/integrations/applica import { OPERATION_SUBBLOCK_ID } from '@/lib/permission-groups/operation-access' import { getSkillUseCase } from '@/lib/skills/application/use-cases' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { loadWorkflowGraph, type ReadWorkflowGraphInput, @@ -403,17 +402,7 @@ async function inspectSelections( /** No execution context is fabricated: discovery is explicitly for the caller reading this draft. */ export const inspectWorkflowTools = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.inspectTools, - resolveContext: ({ - principal, - input, - }: { - principal: InspectionPrincipal - input: InspectWorkflowToolsInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }): Promise { input.signal?.throwIfAborted() const limit = input.limit ?? 20 diff --git a/apps/sim/lib/workflows/application/list-workflow-versions.ts b/apps/sim/lib/workflows/application/list-workflow-versions.ts index 9c65a7ca53a..f0228831c0c 100644 --- a/apps/sim/lib/workflows/application/list-workflow-versions.ts +++ b/apps/sim/lib/workflows/application/list-workflow-versions.ts @@ -1,10 +1,8 @@ -import type { Principal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { isDeploymentOperationStatus } from '@/lib/workflows/deployment-lifecycle' import { listWorkflowVersions as listStoredWorkflowVersions } from '@/lib/workflows/persistence/utils' @@ -21,17 +19,7 @@ export interface ListWorkflowVersionsInput { export const listWorkflowVersions = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.listVersions, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: ListWorkflowVersionsInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { if ( input.limit !== undefined && diff --git a/apps/sim/lib/workflows/application/operations.test.ts b/apps/sim/lib/workflows/application/operations.test.ts index 265d0f5786c..98a1716dd5a 100644 --- a/apps/sim/lib/workflows/application/operations.test.ts +++ b/apps/sim/lib/workflows/application/operations.test.ts @@ -49,7 +49,6 @@ describe('workflow operation registry', () => { } expect(workflowOperations.deployChat.delegatedServices).toEqual(['copilot']) - expect(workflowOperations.undeployChat.delegatedServices).toEqual(['copilot']) expect(workflowOperations.revertVersion.delegatedServices).toEqual(['copilot']) }) diff --git a/apps/sim/lib/workflows/application/operations.ts b/apps/sim/lib/workflows/application/operations.ts index 14aed5e74ce..9ad2572b420 100644 --- a/apps/sim/lib/workflows/application/operations.ts +++ b/apps/sim/lib/workflows/application/operations.ts @@ -56,14 +56,6 @@ export const workflowOperations = { capability: 'none', ...WORKFLOW_READ_PRINCIPAL_POLICY, }), - // permission-group-exempt: reporting where a workflow is already deployed is a read of existing state; a group withholds the act of deploying, not the record of it - readDeploymentOverview: defineWorkspaceOperation({ - id: 'workflows.deployment_overview.read', - minimumRole: 'read', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), /** Full diagnostics retain the caller's subject for protected reference and secret reads. */ // permission-group-exempt: lint reads workflow content under the existing workflow and secret authorization policies readLint: defineWorkspaceOperation({ @@ -91,22 +83,6 @@ export const workflowOperations = { capability: 'none', ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, }), - // permission-group-exempt: reading a block's declared outputs is workflow content; Chat itself is withheld by copilot.use at the chat surface - readCopilotBlockOutputs: defineWorkspaceOperation({ - id: 'workflows.copilot.block_outputs.read', - minimumRole: 'read', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), - // permission-group-exempt: resolving which upstream blocks a block may reference is workflow content; Chat itself is withheld by copilot.use at the chat surface - readCopilotUpstreamReferences: defineWorkspaceOperation({ - id: 'workflows.copilot.upstream_references.read', - minimumRole: 'read', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), // permission-group-exempt: the workflow module has no hide key, so creating a workflow is governed by workspace role alone create: defineWorkspaceOperation({ id: 'workflows.create', @@ -219,38 +195,6 @@ export const workflowOperations = { capability: 'none', ...ALL_WORKFLOW_PRINCIPAL_POLICY, }), - // permission-group-exempt: the workflow file tree has no hide key; arranging it is governed by workspace role - createVfsFolders: defineWorkspaceOperation({ - id: 'workflows.vfs.folders.create', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), - // permission-group-exempt: the workflow file tree has no hide key; arranging it is governed by workspace role - moveVfsItems: defineWorkspaceOperation({ - id: 'workflows.vfs.move', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), - // permission-group-exempt: the workflow file tree has no hide key; arranging it is governed by workspace role - copyVfsItems: defineWorkspaceOperation({ - id: 'workflows.vfs.copy', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), - // permission-group-exempt: the workflow file tree has no hide key; arranging it is governed by workspace role - deleteVfsItems: defineWorkspaceOperation({ - id: 'workflows.vfs.delete', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), // permission-group-exempt: duplicating copies a graph the caller may already read into the same workspace, so it crosses no capability boundary duplicate: defineWorkspaceOperation({ id: 'workflows.duplicate', @@ -362,14 +306,6 @@ export const workflowOperations = { capability: 'deploy.chat', ...HUMAN_WORKFLOW_PRINCIPAL_POLICY, }), - undeployChat: defineWorkspaceOperation({ - id: 'workflows.chat.undeploy', - oauthScope: 'api:write', - minimumRole: 'admin', - workspaceApiKey: 'deny', - capability: 'deploy.chat', - ...HUMAN_WORKFLOW_PRINCIPAL_POLICY, - }), /** * Toggling unauthenticated public execution is an admin-role change a human * key-holder may legitimately make from a script, so personal API keys are diff --git a/apps/sim/lib/workflows/application/principal-scope.ts b/apps/sim/lib/workflows/application/principal-scope.ts index f98347c0874..f73dc2d29f8 100644 --- a/apps/sim/lib/workflows/application/principal-scope.ts +++ b/apps/sim/lib/workflows/application/principal-scope.ts @@ -1,4 +1,8 @@ import type { Principal } from '@sim/auth/principal' +import { + type ActiveWorkflowApplicationContext, + resolveActiveWorkflowApplicationContext, +} from '@/lib/workflows/application/context' /** Leaves scoped-principal workspace mismatches to canonical authorization so they remain 403s. */ export function assertedWorkflowWorkspaceId( @@ -10,3 +14,23 @@ export function assertedWorkflowWorkspaceId( } return assertedWorkspaceId } + +/** + * Resolves the active workflow a use-case input names, asserting the caller's + * workspace only where {@link assertedWorkflowWorkspaceId} keeps it. Pass it as + * `resolveContext: resolvePrincipalWorkflowContext`. + */ +export function resolvePrincipalWorkflowContext< + I extends { workflowId: string; assertedWorkspaceId?: string }, +>({ + principal, + input, +}: { + principal: Principal + input: I +}): Promise { + return resolveActiveWorkflowApplicationContext({ + workflowId: input.workflowId, + assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), + }) +} diff --git a/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.test.ts b/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.test.ts deleted file mode 100644 index 5bf8a91181d..00000000000 --- a/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.test.ts +++ /dev/null @@ -1,121 +0,0 @@ -import { - workflowContextMock, - workflowContextMockFns, -} from '@sim/testing/mocks/workflow-context.mock' -import { - workflowsPersistenceUtilsMock, - workflowsPersistenceUtilsMockFns, -} from '@sim/testing/mocks/workflows-persistence-utils.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import type { Mock } from 'vitest' -import { beforeEach, describe, expect, it, vi } from 'vitest' -import { getBlock } from '@/blocks/registry' - -const mocks = vi.hoisted(() => ({ - outputPaths: vi.fn(), -})) - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -vi.mock('@/lib/workflows/application/context', () => workflowContextMock) - -vi.mock('@/lib/workflows/persistence/utils', () => workflowsPersistenceUtilsMock) - -vi.mock('@/lib/workflows/blocks/block-outputs', () => ({ - getEffectiveBlockOutputPaths: mocks.outputPaths, -})) - -vi.mock('@/lib/workflows/blocks/block-path-calculator', () => ({ - BlockPathCalculator: { findAllPathNodes: vi.fn().mockReturnValue([]) }, -})) - -vi.mock('@/lib/workflows/blocks/block-reference-tags', () => ({ - getBlockReferenceTags: vi.fn().mockReturnValue([]), -})) - -vi.mock('@/lib/workflows/triggers/run-options', () => ({ - resolveTriggerRunOptions: vi.fn().mockReturnValue([]), - toPublicRunOption: vi.fn((value) => value), -})) - -vi.mock('@/lib/workflows/triggers/trigger-utils', () => ({ - hasTriggerCapability: vi.fn().mockReturnValue(false), -})) - -import { readCopilotWorkflowBlockOutputs } from '@/lib/workflows/application/read-workflow-copilot-metadata' - -const mockGetBlock = getBlock as Mock -mockGetBlock.mockReturnValue(undefined) - -const mockLoadDraft = workflowsPersistenceUtilsMockFns.mockLoadWorkflowFromNormalizedTables - -const mockResolvePermission = workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission -const mockResolveContext = workflowContextMockFns.mockResolveActiveWorkflowApplicationContext - -const principal = { - kind: 'delegated' as const, - serviceId: 'copilot' as const, - subjectUserId: 'user-1', - workspaceId: 'workspace-1', - delegationId: 'tool-1', - audience: 'sim:workflows', - issuedAt: new Date('2026-08-01T00:00:00Z'), - expiresAt: new Date('2999-08-01T00:00:00Z'), -} - -describe('Copilot workflow metadata application queries', () => { - beforeEach(() => { - mockResolveContext.mockResolvedValue({ - workflowId: 'workflow-1', - workflow: { - id: 'workflow-1', - workspaceId: 'workspace-1', - variables: { - variable1: { id: 'variable-1', name: 'Customer Name', type: 'plain' }, - }, - }, - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mockResolvePermission.mockResolvedValue('read') - mockLoadDraft.mockResolvedValue({ - blocks: { - 'agent-1': { type: 'agent', name: 'Support Agent', subBlocks: {} }, - }, - edges: [], - loops: {}, - parallels: {}, - }) - mockGetBlock.mockReturnValue({ category: 'core' }) - mocks.outputPaths.mockReturnValue(['content']) - }) - - it('rechecks current permission before loading workflow state', async () => { - mockResolvePermission.mockResolvedValue(null) - - await expect( - readCopilotWorkflowBlockOutputs.execute({ - principal, - input: { workflowId: 'workflow-1', blockIds: ['agent-1'] }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mockLoadDraft).not.toHaveBeenCalled() - }) - - it('rejects oversized block selections before loading workflow state', async () => { - await expect( - readCopilotWorkflowBlockOutputs.execute({ - principal, - input: { - workflowId: 'workflow-1', - blockIds: Array.from({ length: 101 }, (_, index) => `block-${index}`), - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - - expect(mockLoadDraft).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.ts b/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.ts index 62e2a4f9de9..d6b7f83d323 100644 --- a/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.ts +++ b/apps/sim/lib/workflows/application/read-workflow-copilot-metadata.ts @@ -1,112 +1,27 @@ -import type { Principal } from '@sim/auth/principal' import { mergeSubblockStateWithValues } from '@sim/workflow-persistence/subblocks' -import type { Loop, Parallel } from '@sim/workflow-types/workflow' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' -import { getEffectiveBlockOutputPaths } from '@/lib/workflows/blocks/block-outputs' -import { BlockPathCalculator } from '@/lib/workflows/blocks/block-path-calculator' -import { getBlockReferenceTags } from '@/lib/workflows/blocks/block-reference-tags' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { loadWorkflowFromNormalizedTables } from '@/lib/workflows/persistence/utils' import { resolveTriggerRunOptions, toPublicRunOption } from '@/lib/workflows/triggers/run-options' -import { hasTriggerCapability } from '@/lib/workflows/triggers/trigger-utils' -import { getBlock } from '@/blocks/registry' -import { isHumanInTheLoopBlock, normalizeName } from '@/executor/constants' - -const MAX_COPILOT_BLOCK_IDS = 100 interface CopilotWorkflowQueryInput { workflowId: string assertedWorkspaceId?: string } -interface WorkflowVariableReference { - id: string - name: string - type: string - tag: string -} - -interface AccessibleBlockEntry { - blockId: string - blockName: string - blockType: string - outputs: string[] - triggerMode?: boolean - accessContext?: 'inside' | 'outside' -} - -function resolveWorkflowContext({ - principal, - input, -}: { - principal: Principal - input: I -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - async function loadDraftWorkflow(workflowId: string) { const state = await loadWorkflowFromNormalizedTables(workflowId) if (!state) throw new OrchestrationError('not_found', 'Workflow has no saved state') return state } -function workflowVariables(value: unknown): WorkflowVariableReference[] { - const variablesRecord = (value as Record) || {} - return Object.values(variablesRecord) - .filter((variable): variable is Record => { - if (!variable || typeof variable !== 'object') return false - const record = variable as Record - return Boolean(record.name && String(record.name).trim()) - }) - .map((variable) => ({ - id: String(variable.id || ''), - name: String(variable.name || ''), - type: String(variable.type || 'plain'), - tag: `variable.${normalizeName(String(variable.name || ''))}`, - })) -} - -function subflowInsidePaths( - blockType: 'loop' | 'parallel', - blockId: string, - loops: Record, - parallels: Record -): string[] { - const paths = ['index'] - if (blockType === 'loop') { - if ((loops[blockId]?.loopType || 'for') === 'forEach') paths.push('currentItem', 'items') - } else if ((parallels[blockId]?.parallelType || 'count') === 'collection') { - paths.push('currentItem', 'items') - } - return paths -} - -function displayOutputs(paths: string[], blockName: string): string[] { - const normalizedName = normalizeName(blockName) - return paths.map((path) => `${normalizedName}.${path}`) -} - -function assertBlockIdBound(blockIds: string[]): void { - if (blockIds.length > MAX_COPILOT_BLOCK_IDS) { - throw new OrchestrationError( - 'validation', - `blockIds cannot contain more than ${MAX_COPILOT_BLOCK_IDS} entries` - ) - } -} - export interface ReadCopilotWorkflowRunOptionsInput extends CopilotWorkflowQueryInput {} export const readCopilotWorkflowRunOptions = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.readCopilotRunOptions, - resolveContext: resolveWorkflowContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ context }) { const state = await loadDraftWorkflow(context.workflowId) const merged = mergeSubblockStateWithValues(state.blocks) @@ -116,179 +31,3 @@ export const readCopilotWorkflowRunOptions = defineAuthorizedWorkflowUseCase({ } }, }) - -export interface ReadCopilotWorkflowBlockOutputsInput extends CopilotWorkflowQueryInput { - blockIds?: string[] -} - -export const readCopilotWorkflowBlockOutputs = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.readCopilotBlockOutputs, - resolveContext: resolveWorkflowContext, - async execute({ input, context }) { - if (input.blockIds) assertBlockIdBound(input.blockIds) - const state = await loadDraftWorkflow(context.workflowId) - const blocks = state.blocks || {} - const loops = (state.loops || {}) as Record - const parallels = (state.parallels || {}) as Record - const blockIds = input.blockIds?.length ? input.blockIds : Object.keys(blocks) - assertBlockIdBound(blockIds) - - const results = [] - for (const blockId of blockIds) { - const block = blocks[blockId] - if (!block?.type) continue - const blockName = block.name || block.type - if (block.type === 'loop' || block.type === 'parallel') { - const insidePaths = subflowInsidePaths(block.type, blockId, loops, parallels) - results.push({ - blockId, - blockName, - blockType: block.type, - outputs: [], - relativeOutputs: [], - insideSubflowOutputs: displayOutputs(insidePaths, blockName), - outsideSubflowOutputs: displayOutputs(['results'], blockName), - relativeInsideSubflowOutputs: insidePaths, - relativeOutsideSubflowOutputs: ['results'], - triggerMode: block.triggerMode, - }) - continue - } - - const blockConfig = getBlock(block.type) - const triggerMode = Boolean( - block.triggerMode && blockConfig && hasTriggerCapability(blockConfig) - ) - const outputs = getEffectiveBlockOutputPaths(block.type, block.subBlocks, { - triggerMode, - preferToolOutputs: !triggerMode, - }) - results.push({ - blockId, - blockName, - blockType: block.type, - outputs: displayOutputs(outputs, blockName), - relativeOutputs: outputs, - triggerMode: block.triggerMode, - }) - } - - return { blocks: results, variables: workflowVariables(context.workflow.variables) } - }, -}) - -export interface ReadCopilotWorkflowUpstreamReferencesInput extends CopilotWorkflowQueryInput { - blockIds: string[] -} - -export const readCopilotWorkflowUpstreamReferences = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.readCopilotUpstreamReferences, - resolveContext: resolveWorkflowContext, - async execute({ input, context }) { - assertBlockIdBound(input.blockIds) - const state = await loadDraftWorkflow(context.workflowId) - const blocks = state.blocks || {} - const loops = (state.loops || {}) as Record - const parallels = (state.parallels || {}) as Record - const graphEdges = (state.edges || []).map((edge) => ({ - source: edge.source, - target: edge.target, - })) - const variables = workflowVariables(context.workflow.variables) - const results = [] - - for (const blockId of input.blockIds) { - const targetBlock = blocks[blockId] - if (!targetBlock) continue - - const insideSubflows: Array<{ blockId: string; blockName: string; blockType: string }> = [] - const containingLoopIds = new Set() - const containingParallelIds = new Set() - - for (const loop of Object.values(loops)) { - if (!loop?.nodes?.includes(blockId)) continue - containingLoopIds.add(loop.id) - const loopBlock = blocks[loop.id] - if (loopBlock) { - insideSubflows.push({ - blockId: loop.id, - blockName: loopBlock.name || loopBlock.type, - blockType: 'loop', - }) - } - } - - for (const parallel of Object.values(parallels)) { - if (!parallel?.nodes?.includes(blockId)) continue - containingParallelIds.add(parallel.id) - const parallelBlock = blocks[parallel.id] - if (parallelBlock) { - insideSubflows.push({ - blockId: parallel.id, - blockName: parallelBlock.name || parallelBlock.type, - blockType: 'parallel', - }) - } - } - - const accessibleIds = new Set(BlockPathCalculator.findAllPathNodes(graphEdges, blockId)) - accessibleIds.add(blockId) - for (const loopId of containingLoopIds) accessibleIds.add(loopId) - for (const parallelId of containingParallelIds) accessibleIds.add(parallelId) - - const accessibleBlocks: AccessibleBlockEntry[] = [] - for (const accessibleBlockId of accessibleIds) { - const block = blocks[accessibleBlockId] - if (!block?.type) continue - const canSelfReference = block.type === 'approval' || isHumanInTheLoopBlock(block.type) - if (accessibleBlockId === blockId && !canSelfReference) continue - - const blockName = block.name || block.type - let accessContext: 'inside' | 'outside' | undefined - let outputs: string[] - if (block.type === 'loop' || block.type === 'parallel') { - const isInside = - (block.type === 'loop' && containingLoopIds.has(accessibleBlockId)) || - (block.type === 'parallel' && containingParallelIds.has(accessibleBlockId)) - accessContext = isInside ? 'inside' : 'outside' - outputs = displayOutputs( - isInside - ? subflowInsidePaths(block.type, accessibleBlockId, loops, parallels) - : ['results'], - blockName - ) - } else { - outputs = getBlockReferenceTags({ - block: { - id: accessibleBlockId, - type: block.type, - name: block.name, - triggerMode: block.triggerMode, - subBlocks: block.subBlocks, - }, - currentBlockId: blockId, - }) - } - accessibleBlocks.push({ - blockId: accessibleBlockId, - blockName, - blockType: block.type, - outputs, - ...(block.triggerMode ? { triggerMode: true } : {}), - ...(accessContext ? { accessContext } : {}), - }) - } - - results.push({ - blockId, - blockName: targetBlock.name || targetBlock.type, - blockType: targetBlock.type, - accessibleBlocks, - insideSubflows, - variables, - }) - } - - return { results } - }, -}) diff --git a/apps/sim/lib/workflows/application/read-workflow-definition.ts b/apps/sim/lib/workflows/application/read-workflow-definition.ts index 5c1557ff52d..d11e7b25a1c 100644 --- a/apps/sim/lib/workflows/application/read-workflow-definition.ts +++ b/apps/sim/lib/workflows/application/read-workflow-definition.ts @@ -1,10 +1,9 @@ -import type { Principal } from '@sim/auth/principal' import type { NormalizedWorkflowData } from '@sim/workflow-persistence/types' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' +import type { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { type DeployedWorkflowData, loadDeployedWorkflowState, @@ -35,17 +34,7 @@ async function loadDeployedDefinition(workflowId: string, workspaceId: string) { export const readWorkflowDefinition = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.read, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: ReadWorkflowDefinitionInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ input, context }): Promise { if (input.state === 'draft') { const snapshot = await loadWorkflowReadSnapshot(context.workflowId, context.workspaceId) diff --git a/apps/sim/lib/workflows/application/read-workflow-deployment-overview.test.ts b/apps/sim/lib/workflows/application/read-workflow-deployment-overview.test.ts deleted file mode 100644 index beb59ab301b..00000000000 --- a/apps/sim/lib/workflows/application/read-workflow-deployment-overview.test.ts +++ /dev/null @@ -1,182 +0,0 @@ -import { queueTableRows, resetDbChainMock, schemaMock } from '@sim/testing' -import { - workflowDeploymentStatusMock, - workflowDeploymentStatusMockFns, -} from '@sim/testing/mocks/workflow-deployment-status.mock' -import { - workflowsOrchestrationMock, - workflowsOrchestrationMockFns, -} from '@sim/testing/mocks/workflows-orchestration.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { - workspaceContextMock, - workspaceContextMockFns, -} from '@sim/testing/mocks/workspace-context.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -vi.mock('@/lib/workspaces/application/workspace-context', () => workspaceContextMock) - -vi.mock('@/lib/workflows/deployment-status', () => workflowDeploymentStatusMock) - -vi.mock('@/lib/workflows/orchestration', () => workflowsOrchestrationMock) - -import { - MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES, - MAX_WORKFLOW_MCP_STATUS_TOOLS, - MAX_WORKFLOW_MCP_STATUS_TOTAL_SCHEMA_BYTES, - readWorkflowDeploymentOverview, -} from '@/lib/workflows/application/read-workflow-deployment-overview' - -const mocks = { - deploymentSummary: workflowsOrchestrationMockFns.mockGetWorkflowDeploymentSummary, - redeployment: workflowDeploymentStatusMockFns.mockCheckNeedsRedeployment, -} - -const mockPermission = workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission -const mockLoadWorkspace = workspaceContextMockFns.mockLoadActiveWorkspaceApplicationContext - -const workflowRecord = { - id: 'workflow-1', - workspaceId: 'workspace-1', - name: 'Workflow', - archivedAt: null, -} -const principal = { - kind: 'delegated' as const, - serviceId: 'copilot' as const, - subjectUserId: 'user-1', - workspaceId: 'workspace-1', - delegationId: 'tool-call-1', - audience: 'sim:workflows', - issuedAt: new Date('2026-01-01T00:00:00Z'), - expiresAt: new Date('2099-01-01T00:00:00Z'), -} - -describe('readWorkflowDeploymentOverview', () => { - beforeEach(() => { - resetDbChainMock() - mockLoadWorkspace.mockResolvedValue({ - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mockPermission.mockResolvedValue('read') - mocks.deploymentSummary.mockResolvedValue({ - activeDeployment: null, - latestDeploymentAttempt: null, - warnings: [], - }) - mocks.redeployment.mockResolvedValue(false) - }) - - it('caps workflow MCP status rows and reports truncation', async () => { - queueTableRows(schemaMock.workflow, [ - { - workflowId: workflowRecord.id, - workflow: workflowRecord, - workspaceId: workflowRecord.workspaceId, - }, - ]) - queueTableRows(schemaMock.chat, []) - queueTableRows( - schemaMock.workflowMcpTool, - Array.from({ length: MAX_WORKFLOW_MCP_STATUS_TOOLS + 1 }, (_, index) => ({ - serverId: `server-${index}`, - serverName: `Server ${index}`, - toolName: `tool_${index}`, - toolDescription: null, - parameterSchema: {}, - parameterSchemaBytes: 2, - toolId: `tool-${index}`, - })) - ) - - const result = await readWorkflowDeploymentOverview.execute({ - principal, - input: { workflowId: workflowRecord.id }, - }) - - expect(result.mcpTools).toHaveLength(MAX_WORKFLOW_MCP_STATUS_TOOLS) - expect(result.mcpToolsTruncated).toBe(true) - }) - - it('truncates schema materialization at individual and aggregate byte budgets', async () => { - queueTableRows(schemaMock.workflow, [ - { - workflowId: workflowRecord.id, - workflow: workflowRecord, - workspaceId: workflowRecord.workspaceId, - }, - ]) - queueTableRows(schemaMock.chat, []) - const aggregateRows = Array.from( - { - length: MAX_WORKFLOW_MCP_STATUS_TOTAL_SCHEMA_BYTES / MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES, - }, - (_, index) => ({ - serverId: 'server-1', - serverName: 'Server 1', - toolName: `within-budget-${index}`, - toolDescription: null, - parameterSchema: { type: 'object' }, - parameterSchemaBytes: MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES, - toolId: `tool-${index + 2}`, - }) - ) - queueTableRows(schemaMock.workflowMcpTool, [ - { - serverId: 'server-1', - serverName: 'Server 1', - toolName: 'oversized', - toolDescription: null, - parameterSchema: null, - parameterSchemaBytes: MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES + 1, - toolId: 'tool-1', - }, - ...aggregateRows, - { - serverId: 'server-1', - serverName: 'Server 1', - toolName: 'past-budget', - toolDescription: null, - parameterSchema: { type: 'object' }, - parameterSchemaBytes: 1, - toolId: 'tool-last', - }, - ]) - - const result = await readWorkflowDeploymentOverview.execute({ - principal, - input: { workflowId: workflowRecord.id }, - }) - - expect(result.mcpTools).toHaveLength(1 + aggregateRows.length) - expect(result.mcpTools[0].parameterSchema).toEqual({ - truncated: true, - bytes: MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES + 1, - }) - expect(result.mcpToolsTruncated).toBe(true) - }) - - it('returns forbidden for a cross-workspace delegated principal before protected status loads', async () => { - queueTableRows(schemaMock.workflow, [ - { - workflowId: workflowRecord.id, - workflow: workflowRecord, - workspaceId: workflowRecord.workspaceId, - }, - ]) - - await expect( - readWorkflowDeploymentOverview.execute({ - principal: { ...principal, workspaceId: 'workspace-2' }, - input: { workflowId: workflowRecord.id }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.deploymentSummary).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/workflows/application/read-workflow-deployment-overview.ts b/apps/sim/lib/workflows/application/read-workflow-deployment-overview.ts deleted file mode 100644 index 832d741adc1..00000000000 --- a/apps/sim/lib/workflows/application/read-workflow-deployment-overview.ts +++ /dev/null @@ -1,130 +0,0 @@ -import type { Principal } from '@sim/auth/principal' -import { chat, db, workflowMcpServer, workflowMcpTool } from '@sim/db' -import { and, asc, eq, isNull, sql } from 'drizzle-orm' -import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' -import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' -import { checkNeedsRedeployment } from '@/lib/workflows/deployment-status' -import { getWorkflowDeploymentSummary } from '@/lib/workflows/orchestration' - -export const MAX_WORKFLOW_MCP_STATUS_TOOLS = 100 -export const MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES = 64 * 1024 -export const MAX_WORKFLOW_MCP_STATUS_TOTAL_SCHEMA_BYTES = 1024 * 1024 - -export interface ReadWorkflowDeploymentOverviewInput { - workflowId: string - assertedWorkspaceId?: string -} - -function resolveWorkflowContext({ - principal, - input, -}: { - principal: Principal - input: ReadWorkflowDeploymentOverviewInput -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - -export const readWorkflowDeploymentOverview = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.readDeploymentOverview, - resolveContext: resolveWorkflowContext, - async execute({ context }) { - const [deploymentSummary, chatDeploy, mcpRows] = await Promise.all([ - getWorkflowDeploymentSummary(context.workflowId), - db - .select({ - id: chat.id, - identifier: chat.identifier, - title: chat.title, - description: chat.description, - authType: chat.authType, - allowedEmails: chat.allowedEmails, - outputConfigs: chat.outputConfigs, - includeThinking: chat.includeThinking, - includeToolCalls: chat.includeToolCalls, - password: chat.password, - customizations: chat.customizations, - }) - .from(chat) - .where(and(eq(chat.workflowId, context.workflowId), isNull(chat.archivedAt))) - .limit(1), - db - .select({ - serverId: workflowMcpServer.id, - serverName: workflowMcpServer.name, - toolName: workflowMcpTool.toolName, - toolDescription: workflowMcpTool.toolDescription, - parameterSchema: sql`CASE - WHEN COALESCE(octet_length(${workflowMcpTool.parameterSchema}::text), 0) - <= ${MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES} - THEN ${workflowMcpTool.parameterSchema} - ELSE NULL - END`, - parameterSchemaBytes: - sql`COALESCE(octet_length(${workflowMcpTool.parameterSchema}::text), 0)`.mapWith( - Number - ), - toolId: workflowMcpTool.id, - }) - .from(workflowMcpTool) - .innerJoin(workflowMcpServer, eq(workflowMcpTool.serverId, workflowMcpServer.id)) - .where( - and( - eq(workflowMcpTool.workflowId, context.workflowId), - isNull(workflowMcpTool.archivedAt), - isNull(workflowMcpServer.deletedAt) - ) - ) - .orderBy(asc(workflowMcpServer.id), asc(workflowMcpTool.toolName)) - .limit(MAX_WORKFLOW_MCP_STATUS_TOOLS + 1), - ]) - - const isDeployed = deploymentSummary.activeDeployment !== null - const attemptStatus = deploymentSummary.latestDeploymentAttempt?.status - const needsRedeployment = - isDeployed && attemptStatus !== 'preparing' && attemptStatus !== 'activating' - ? await checkNeedsRedeployment(context.workflowId) - : false - let schemaBytes = 0 - let mcpToolsTruncated = mcpRows.length > MAX_WORKFLOW_MCP_STATUS_TOOLS - const mcpTools = [] - for (const row of mcpRows.slice(0, MAX_WORKFLOW_MCP_STATUS_TOOLS)) { - if (!Number.isFinite(row.parameterSchemaBytes) || row.parameterSchemaBytes < 0) { - throw new Error('Workflow MCP status query returned an invalid schema byte count') - } - const schemaOversized = row.parameterSchemaBytes > MAX_WORKFLOW_MCP_STATUS_SCHEMA_BYTES - if ( - !schemaOversized && - schemaBytes + row.parameterSchemaBytes > MAX_WORKFLOW_MCP_STATUS_TOTAL_SCHEMA_BYTES - ) { - mcpToolsTruncated = true - break - } - if (!schemaOversized) schemaBytes += row.parameterSchemaBytes - if (schemaOversized) mcpToolsTruncated = true - const { parameterSchemaBytes: _parameterSchemaBytes, ...tool } = row - mcpTools.push({ - ...tool, - parameterSchema: schemaOversized - ? { truncated: true, bytes: row.parameterSchemaBytes } - : row.parameterSchema, - }) - } - - return { - workflow: context.workflow, - workspaceId: context.workspaceId, - isDeployed, - needsRedeployment, - ...deploymentSummary, - chatDeployment: chatDeploy[0] ?? null, - mcpTools, - mcpToolsTruncated, - } - }, -}) diff --git a/apps/sim/lib/workflows/application/read-workflow-graph.ts b/apps/sim/lib/workflows/application/read-workflow-graph.ts index 03a849cdcae..205af682e75 100644 --- a/apps/sim/lib/workflows/application/read-workflow-graph.ts +++ b/apps/sim/lib/workflows/application/read-workflow-graph.ts @@ -1,10 +1,8 @@ -import type { Principal } from '@sim/auth/principal' import type { BlockState, Variable, WorkflowState } from '@sim/workflow-types/workflow' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { loadWorkflowReadSnapshot } from '@/lib/workflows/queries' import { parseWorkflowVariables } from '@/lib/workflows/variables/parse' @@ -42,11 +40,7 @@ export interface ReadWorkflowGraphResult { */ export const readWorkflowGraph = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.read, - resolveContext: ({ principal, input }: { principal: Principal; input: ReadWorkflowGraphInput }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ context }): Promise { return loadWorkflowGraph(context) }, diff --git a/apps/sim/lib/workflows/application/read-workflow-version.ts b/apps/sim/lib/workflows/application/read-workflow-version.ts index 4cab173c1a2..a605892fb6f 100644 --- a/apps/sim/lib/workflows/application/read-workflow-version.ts +++ b/apps/sim/lib/workflows/application/read-workflow-version.ts @@ -1,10 +1,8 @@ -import type { Principal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { sanitizeWorkflowForSharing } from '@/lib/workflows/credentials/credential-extractor' import { getWorkflowDeploymentVersion } from '@/lib/workflows/persistence/utils' import type { WorkflowState } from '@/stores/workflows/workflow/types' @@ -48,17 +46,7 @@ export interface ReadWorkflowVersionInput { export const readWorkflowVersion = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.readVersion, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: ReadWorkflowVersionInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { const version = await getWorkflowDeploymentVersion(context.workflowId, input.version) if (!version?.state) { diff --git a/apps/sim/lib/workflows/application/read-workflow.ts b/apps/sim/lib/workflows/application/read-workflow.ts index 8ac0d652a87..fd710f93831 100644 --- a/apps/sim/lib/workflows/application/read-workflow.ts +++ b/apps/sim/lib/workflows/application/read-workflow.ts @@ -1,12 +1,10 @@ -import type { Principal } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { OrchestrationError } from '@/lib/core/orchestration/types' import { MAX_FOLDERS_PER_WORKSPACE } from '@/lib/folders/constants' import { loadActiveFolderPathIndex } from '@/lib/folders/queries' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { workflowFolderPathForId } from '@/lib/workflows/application/workflow-folders' import { extractInputFieldsFromBlocks } from '@/lib/workflows/input-format' import { loadWorkflowReadSnapshot } from '@/lib/workflows/queries' @@ -18,19 +16,6 @@ export interface ReadWorkflowInput { assertedWorkspaceId?: string } -function resolveReadContext({ - principal, - input, -}: { - principal: Principal - input: ReadWorkflowInput -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - async function loadWorkflowFolderPath(workspaceId: string, folderId: string | null) { const index = await loadActiveFolderPathIndex(workspaceId, 'workflow', undefined, { maxRows: MAX_FOLDERS_PER_WORKSPACE, @@ -41,7 +26,7 @@ async function loadWorkflowFolderPath(workspaceId: string, folderId: string | nu /** Reads canonical workflow metadata and location without loading the workflow graph. */ export const readWorkflowMetadata = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.read, - resolveContext: resolveReadContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ context }) { return { workflow: context.workflow, @@ -52,7 +37,7 @@ export const readWorkflowMetadata = defineAuthorizedWorkflowUseCase({ export const readWorkflow = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.read, - resolveContext: resolveReadContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, context }) { const snapshot = await loadWorkflowReadSnapshot(context.workflowId, context.workspaceId) const workflow = snapshot.workflowRecord diff --git a/apps/sim/lib/workflows/application/replace-workflow-state.ts b/apps/sim/lib/workflows/application/replace-workflow-state.ts index 346afe660e2..322e33a0f35 100644 --- a/apps/sim/lib/workflows/application/replace-workflow-state.ts +++ b/apps/sim/lib/workflows/application/replace-workflow-state.ts @@ -12,9 +12,8 @@ import { principalAuditSource } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' import { notifyWorkflowUpdated } from '@/lib/realtime/notify' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { withWorkflowBlockScope } from '@/lib/workflows/application/workflow-block-scope' import { requireMutableWorkflow } from '@/lib/workflows/application/workflow-mutability' import { normalizeWorkflowVariables } from '@/lib/workflows/application/workflow-variables' @@ -102,17 +101,7 @@ export interface ReplaceWorkflowStateResult { */ export const replaceWorkflowState = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.replaceState, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: ReplaceWorkflowStateInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }): Promise { await requireMutableWorkflow(context.workflowId) diff --git a/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts b/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts index e5828aef3d2..9309695690c 100644 --- a/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts +++ b/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts @@ -8,9 +8,9 @@ import type { BillingAttributionSnapshot } from '@/lib/billing/core/billing-attr import { OrchestrationError } from '@/lib/core/orchestration/types' import { generateRequestId } from '@/lib/core/utils/request' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' +import type { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { prepareWorkflowExecutionAdmission } from '@/lib/workflows/execution-admission' import { executeWorkflow } from '@/lib/workflows/executor/execute-workflow' import { @@ -94,19 +94,6 @@ interface SnapshotCopilotRunInput extends BaseCopilotRunInput { export interface RunFromBlockFromCopilotInput extends SnapshotCopilotRunInput {} export interface RunBlockFromCopilotInput extends SnapshotCopilotRunInput {} -function resolveContext({ - principal, - input, -}: { - principal: Principal - input: I -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - async function loadDefinition(input: BaseCopilotRunInput, workspaceId: string) { if (input.useDraftState) return loadWorkflowFromNormalizedTables(input.workflowId) try { @@ -428,7 +415,7 @@ function defineTriggerRunUseCase, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { const prepared = await resolveTriggerExecution({ input, workspaceId: context.workspaceId }) return executeCopilotRun({ @@ -460,7 +447,7 @@ function defineSnapshotRunUseCase( ) { return defineAuthorizedWorkflowUseCase({ operation, - resolveContext: resolveContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { const state = await loadDefinition(input, context.workspaceId) if (!state?.blocks) { diff --git a/apps/sim/lib/workflows/application/update-workflow-content.ts b/apps/sim/lib/workflows/application/update-workflow-content.ts index bd7501c55f0..37db975c265 100644 --- a/apps/sim/lib/workflows/application/update-workflow-content.ts +++ b/apps/sim/lib/workflows/application/update-workflow-content.ts @@ -1,5 +1,5 @@ import { AuditAction, AuditResourceType } from '@sim/audit' -import { type Principal, resolvePrincipalAttribution } from '@sim/auth/principal' +import { resolvePrincipalAttribution } from '@sim/auth/principal' import { db } from '@sim/db' import { workflow } from '@sim/db/schema' import { generateId } from '@sim/utils/id' @@ -13,9 +13,8 @@ import { principalAuditSource } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' import { notifyWorkflowUpdated } from '@/lib/realtime/notify' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { requireMutableWorkflow } from '@/lib/workflows/application/workflow-mutability' import { coerceWorkflowVariableValue, @@ -46,19 +45,6 @@ interface WorkflowContentInput { assertedWorkspaceId?: string } -function resolveWorkflowContentContext({ - principal, - input, -}: { - principal: Principal - input: I -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - export interface WorkflowVariableOperation { name: string operation: 'add' | 'edit' | 'delete' @@ -105,7 +91,7 @@ function applyVariableOperations( export const applyWorkflowVariableOperations = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.applyVariableOperations, - resolveContext: resolveWorkflowContentContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ input, context }) { if (input.operations.length > MAX_WORKFLOW_VARIABLE_OPERATIONS) { throw new OrchestrationError( @@ -189,7 +175,7 @@ export interface SetWorkflowBlockEnabledInput extends WorkflowContentInput { */ export const setWorkflowBlockEnabled = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.setBlockEnabled, - resolveContext: resolveWorkflowContentContext, + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { await requireMutableWorkflow(context.workflowId) diff --git a/apps/sim/lib/workflows/application/update-workflow-deployment-settings.ts b/apps/sim/lib/workflows/application/update-workflow-deployment-settings.ts index cf7800553a4..cf4d6c3e8f8 100644 --- a/apps/sim/lib/workflows/application/update-workflow-deployment-settings.ts +++ b/apps/sim/lib/workflows/application/update-workflow-deployment-settings.ts @@ -1,5 +1,5 @@ import { AuditAction, AuditResourceType } from '@sim/audit' -import { type Principal, requirePrincipalSubjectUserId } from '@sim/auth/principal' +import { requirePrincipalSubjectUserId } from '@sim/auth/principal' import { db, workflow } from '@sim/db' import { assertWorkflowMutable, WorkflowLockedError } from '@sim/platform-authz/workflow' import { eq } from 'drizzle-orm' @@ -7,9 +7,8 @@ import { ForbiddenOperationError } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' import { notifyWorkflowUpdated } from '@/lib/realtime/notify' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { PublicApiNotAllowedError, validatePublicApiAllowed, @@ -23,17 +22,7 @@ export interface UpdateWorkflowPublicApiInput { export const updateWorkflowPublicApi = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.updatePublicApi, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: UpdateWorkflowPublicApiInput - }) => - resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }), + resolveContext: resolvePrincipalWorkflowContext, async execute({ principal, input, context }) { const actingUserId = requirePrincipalSubjectUserId(principal) try { diff --git a/apps/sim/lib/workflows/application/update-workflow.ts b/apps/sim/lib/workflows/application/update-workflow.ts index d56cd95445a..206fdaf3cd8 100644 --- a/apps/sim/lib/workflows/application/update-workflow.ts +++ b/apps/sim/lib/workflows/application/update-workflow.ts @@ -13,12 +13,9 @@ import { MAX_FOLDERS_PER_WORKSPACE } from '@/lib/folders/constants' import { loadActiveFolderPathIndex } from '@/lib/folders/queries' import { notifyWorkflowUpdated, notifyWorkspaceWorkflowsChanged } from '@/lib/realtime/notify' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { - type ActiveWorkflowApplicationContext, - resolveActiveWorkflowApplicationContext, -} from '@/lib/workflows/application/context' +import type { ActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' -import { assertedWorkflowWorkspaceId } from '@/lib/workflows/application/principal-scope' +import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { requireWorkflowTransition } from '@/lib/workflows/application/transition-result' import { resolveWorkflowFolderPath, @@ -64,19 +61,6 @@ interface WorkflowUpdateResult { } } -function resolveWorkflowUpdateContext({ - principal, - input, -}: { - principal: Principal - input: UpdateWorkflowInput -}) { - return resolveActiveWorkflowApplicationContext({ - workflowId: input.workflowId, - assertedWorkspaceId: assertedWorkflowWorkspaceId(principal, input.assertedWorkspaceId), - }) -} - async function requireMutableWorkflowUpdate( context: ActiveWorkflowApplicationContext, input: UpdateWorkflowPolicyInput, @@ -262,7 +246,7 @@ async function notifyAfterWorkflowUpdate(args: { export const updateWorkflow = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.update, - resolveContext: resolveWorkflowUpdateContext, + resolveContext: resolvePrincipalWorkflowContext, execute: executeWorkflowUpdate, projectAudit: projectWorkflowUpdateAudit, afterSuccess: notifyAfterWorkflowUpdate, @@ -270,7 +254,7 @@ export const updateWorkflow = defineAuthorizedWorkflowUseCase({ export const updateWorkflowPolicy = defineAuthorizedWorkflowUseCase({ operation: workflowOperations.updatePolicy, - resolveContext: resolveWorkflowUpdateContext, + resolveContext: resolvePrincipalWorkflowContext, execute: executeWorkflowUpdate, projectAudit: projectWorkflowUpdateAudit, afterSuccess: notifyAfterWorkflowUpdate, diff --git a/apps/sim/lib/workflows/application/workflow-vfs.test.ts b/apps/sim/lib/workflows/application/workflow-vfs.test.ts deleted file mode 100644 index 6101425f121..00000000000 --- a/apps/sim/lib/workflows/application/workflow-vfs.test.ts +++ /dev/null @@ -1,191 +0,0 @@ -import { queueTableRows, resetDbChainMock, schemaMock } from '@sim/testing' -import { auditMock, auditMockFns } from '@sim/testing/mocks/audit.mock' -import { folderQueriesMock, folderQueriesMockFns } from '@sim/testing/mocks/folder-queries.mock' -import { - foldersOrchestrationMock, - foldersOrchestrationMockFns, -} from '@sim/testing/mocks/folders-orchestration.mock' -import { getAllMockLoggers } from '@sim/testing/mocks/logger.mock' -import { realtimeNotifyMock, realtimeNotifyMockFns } from '@sim/testing/mocks/realtime-notify.mock' -import { workflowAuthzMockFns } from '@sim/testing/mocks/workflow-authz.mock' -import { - workflowsOrchestrationMock, - workflowsOrchestrationMockFns, -} from '@sim/testing/mocks/workflows-orchestration.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { - workspaceContextMock, - workspaceContextMockFns, -} from '@sim/testing/mocks/workspace-context.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const hoisted = vi.hoisted(() => ({ - duplicateWorkflow: vi.fn(), -})) - -vi.mock('@sim/audit', () => auditMock) - -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) - -vi.mock('@/lib/workspaces/application/workspace-context', () => workspaceContextMock) - -vi.mock('@/lib/folders/queries', () => folderQueriesMock) - -vi.mock('@/lib/folders/orchestration', () => foldersOrchestrationMock) - -vi.mock('@/lib/workflows/orchestration', () => workflowsOrchestrationMock) - -vi.mock('@/lib/workflows/persistence/duplicate', () => ({ - duplicateWorkflow: hoisted.duplicateWorkflow, -})) - -vi.mock('@/lib/realtime/notify', () => realtimeNotifyMock) - -import { moveWorkflowVfsItems } from '@/lib/workflows/application/workflow-vfs' - -const mocks = { - ...hoisted, - createFolder: foldersOrchestrationMockFns.mockCreateFolderAtPath, - deleteFolder: foldersOrchestrationMockFns.mockDeleteFolderByPath, - relocateFolder: foldersOrchestrationMockFns.mockRelocateFolderByPath, - deleteWorkflow: workflowsOrchestrationMockFns.mockDeleteWorkflowRecord, - updateWorkflow: workflowsOrchestrationMockFns.mockUpdateWorkflowRecord, - loadFolderIndex: folderQueriesMockFns.mockLoadActiveFolderPathIndex, -} - -const { mockAssertFolderMutable, mockAssertWorkflowMutable } = workflowAuthzMockFns - -const mockAudit = auditMockFns.mockRecordAudit -const mockPermission = workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission -const mockResolveContext = workspaceContextMockFns.mockResolveActiveWorkspaceApplicationContext -const mockNotifyFolder = realtimeNotifyMockFns.mockNotifyFolderResourceChanged -const mockNotifyWorkflow = realtimeNotifyMockFns.mockNotifyWorkflowUpdated - -const workspaceContext = { - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', -} -const principal = { - kind: 'delegated' as const, - serviceId: 'copilot' as const, - subjectUserId: 'user-1', - workspaceId: 'workspace-1', - delegationId: 'tool-call-1', - audience: 'sim:workflows', - issuedAt: new Date('2026-01-01T00:00:00Z'), - expiresAt: new Date('2099-01-01T00:00:00Z'), -} -const emptyIndex = { - rowById: new Map(), - pathById: new Map(), - idByPath: new Map(), -} - -describe('workflow VFS application commands', () => { - beforeEach(() => { - resetDbChainMock() - mockResolveContext.mockResolvedValue(workspaceContext) - mockPermission.mockResolvedValue('write') - mockAssertFolderMutable.mockResolvedValue(undefined) - mockAssertWorkflowMutable.mockResolvedValue(undefined) - mocks.loadFolderIndex.mockResolvedValue(emptyIndex) - }) - - it('rejects a forged cross-workspace delegation before loading the protected VFS index', async () => { - await expect( - moveWorkflowVfsItems.execute({ - principal: { ...principal, workspaceId: 'workspace-2' }, - input: { - workspaceId: 'workspace-1', - sources: [{ source: 'workflows/One', segments: ['One'] }], - destination: { segments: ['Archive'], trailingSlash: true }, - }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.loadFolderIndex).not.toHaveBeenCalled() - }) - - it('rechecks current permission before canonical index loading', async () => { - mockPermission.mockResolvedValueOnce(null) - - await expect( - moveWorkflowVfsItems.execute({ - principal, - input: { - workspaceId: 'workspace-1', - sources: [{ source: 'workflows/One', segments: ['One'] }], - destination: { segments: [], trailingSlash: false }, - }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.loadFolderIndex).not.toHaveBeenCalled() - }) - - it('keeps partial failures bounded while auditing and notifying only durable successes', async () => { - queueTableRows(schemaMock.workflow, [ - { id: 'workflow-1', name: 'One', folderId: null }, - { id: 'workflow-2', name: 'Two', folderId: null }, - ]) - queueTableRows(schemaMock.workflow, [{ id: 'workflow-1', name: 'One', folderId: null }]) - queueTableRows(schemaMock.workflow, [{ id: 'workflow-2', name: 'Two', folderId: null }]) - mocks.updateWorkflow - .mockResolvedValueOnce({ - success: true, - workflow: { id: 'workflow-1', name: 'One', folderId: null }, - }) - .mockResolvedValueOnce({ success: false, error: 'Workflow is locked', errorCode: 'locked' }) - - const result = await moveWorkflowVfsItems.execute({ - principal, - input: { - workspaceId: 'workspace-1', - sources: [ - { source: 'workflows/One', segments: ['One'] }, - { source: 'workflows/Two', segments: ['Two'] }, - ], - destination: { segments: [], trailingSlash: true }, - }, - }) - - for (const logger of getAllMockLoggers()) expect(logger.error).not.toHaveBeenCalled() - expect(result.outcomes).toEqual([ - expect.objectContaining({ source: 'workflows/One', resourceId: 'workflow-1' }), - expect.objectContaining({ source: 'workflows/Two', error: 'Workflow is locked' }), - ]) - expect(mockAudit).toHaveBeenCalledOnce() - expect(mockAudit).toHaveBeenCalledWith( - expect.objectContaining({ - action: 'workflow.updated', - resourceId: 'workflow-1', - metadata: expect.objectContaining({ operation: 'workflows.vfs.move' }), - }) - ) - expect(mockNotifyWorkflow).toHaveBeenCalledWith('workflow-1') - expect(mockNotifyWorkflow).not.toHaveBeenCalledWith('workflow-2') - }) - - it('propagates an unexpected mutation failure without projecting a partial outcome', async () => { - queueTableRows(schemaMock.workflow, [{ id: 'workflow-1', name: 'One', folderId: null }]) - queueTableRows(schemaMock.workflow, [{ id: 'workflow-1', name: 'One', folderId: null }]) - mocks.updateWorkflow.mockRejectedValueOnce(new Error('postgres password=secret')) - - await expect( - moveWorkflowVfsItems.execute({ - principal, - input: { - workspaceId: 'workspace-1', - sources: [{ source: 'workflows/One', segments: ['One'] }], - destination: { segments: [], trailingSlash: true }, - }, - }) - ).rejects.toThrow('postgres password=secret') - - expect(mockAudit).not.toHaveBeenCalled() - expect(mockNotifyWorkflow).not.toHaveBeenCalled() - expect(mockNotifyFolder).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/workflows/application/workflow-vfs.ts b/apps/sim/lib/workflows/application/workflow-vfs.ts deleted file mode 100644 index a313729984b..00000000000 --- a/apps/sim/lib/workflows/application/workflow-vfs.ts +++ /dev/null @@ -1,847 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { resolvePrincipalAttribution } from '@sim/auth/principal' -import { db } from '@sim/db' -import { workflow } from '@sim/db/schema' -import { - assertFolderMutable, - assertWorkflowMutable, - FolderLockedError, - WorkflowLockedError, -} from '@sim/platform-authz/workflow' -import { and, eq, isNull } from 'drizzle-orm' -import { - asOrchestrationError, - OrchestrationError, - type OrchestrationErrorCode, - throwOrchestrationFailure, -} from '@/lib/core/orchestration/types' -import { generateRequestId } from '@/lib/core/utils/request' -import { - createFolderAtPath, - deleteFolderByPath, - relocateFolderByPath, -} from '@/lib/folders/orchestration' -import { - buildFolderPath, - FolderPathError, - type FolderPathIndex, - parseFolderPath, -} from '@/lib/folders/paths' -import { loadActiveFolderPathIndex } from '@/lib/folders/queries' -import { - notifyFolderResourceChanged, - notifyWorkflowDeleted, - notifyWorkflowUpdated, -} from '@/lib/realtime/notify' -import { VfsPathLimitError, validateVfsPathSegments } from '@/lib/vfs/limits' -import { encodeVfsPathSegments } from '@/lib/vfs/path' -import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import { workflowOperations } from '@/lib/workflows/application/operations' -import { requireWorkflowTransition } from '@/lib/workflows/application/transition-result' -import { deleteWorkflowRecord, updateWorkflowRecord } from '@/lib/workflows/orchestration' -import { duplicateWorkflow as duplicateWorkflowRecord } from '@/lib/workflows/persistence/duplicate' -import type { ActiveWorkspaceApplicationContext } from '@/lib/workspaces/application/workspace-context' -import { resolveActiveWorkspaceApplicationContext } from '@/lib/workspaces/application/workspace-context' - -const MAX_WORKFLOW_VFS_ITEMS = 100 -const MAX_WORKFLOW_VFS_INDEX_ROWS = 10_000 -const MAX_WORKFLOW_NAME_LENGTH = 200 - -export interface WorkflowVfsPathReference { - source: string - segments: string[] -} - -export interface WorkflowVfsDestination { - segments: string[] - trailingSlash: boolean -} - -export interface WorkflowVfsOutcome { - source: string - targetSegments?: string[] - resourceType: 'workflow' | 'folder' - resourceId?: string - error?: string -} - -export interface CreateWorkflowVfsFoldersInput { - workspaceId: string - paths: WorkflowVfsPathReference[] -} - -export interface TransferWorkflowVfsItemsInput { - workspaceId: string - sources: WorkflowVfsPathReference[] - destination: WorkflowVfsDestination -} - -export interface DeleteWorkflowVfsItemsInput { - workspaceId: string - paths: WorkflowVfsPathReference[] -} - -interface WorkflowVfsRow { - id: string - name: string - folderId: string | null -} - -interface CreatedFolderChange { - id: string - name: string - path: string -} - -interface MovedWorkflowChange { - id: string - name: string - previousFolderId: string | null - folderId: string | null -} - -interface MovedFolderChange { - id: string - name: string - sourcePath: string - destinationPath: string -} - -interface DuplicatedWorkflowChange { - id: string - name: string - sourceWorkflowId: string -} - -interface DeletedWorkflowChange { - id: string - name: string -} - -interface DeletedFolderChange { - id: string - name: string - path: string - workflows: number - folders: number -} - -interface WorkflowVfsIndexState { - folderIndex: FolderPathIndex - workflows: WorkflowVfsRow[] - createdFolders: CreatedFolderChange[] -} - -interface ResolvedWorkflowSource { - source: string - workflow?: WorkflowVfsRow - folderId?: string - error?: string -} - -interface DestinationPlan { - dirMode: boolean - folderSegments: string[] - leafName?: string - ensureFolderId(): Promise -} - -function canonicalSegmentsKey(segments: readonly string[]): string { - return encodeVfsPathSegments([...segments]) -} - -function normalizeReferences( - references: readonly WorkflowVfsPathReference[] -): WorkflowVfsPathReference[] { - if (references.length > MAX_WORKFLOW_VFS_ITEMS) { - throw new OrchestrationError( - 'validation', - `Workflow VFS commands cannot exceed ${MAX_WORKFLOW_VFS_ITEMS} items` - ) - } - const byPath = new Map() - for (const reference of references) { - try { - validateVfsPathSegments(reference.segments) - } catch (error) { - if (error instanceof VfsPathLimitError) { - throw new OrchestrationError('validation', error.message) - } - throw error - } - const key = canonicalSegmentsKey(reference.segments) - if (!byPath.has(key)) byPath.set(key, reference) - } - if (byPath.size === 0) throw new OrchestrationError('validation', 'At least one path is required') - return [...byPath.values()] -} - -function validateDestination(destination: WorkflowVfsDestination): void { - try { - validateVfsPathSegments(destination.segments) - } catch (error) { - if (error instanceof VfsPathLimitError) { - throw new OrchestrationError('validation', error.message) - } - throw error - } -} - -async function loadWorkflowVfsIndex( - context: ActiveWorkspaceApplicationContext -): Promise { - const [folderIndex, workflows] = await Promise.all([ - loadActiveFolderPathIndex(context.workspaceId, 'workflow', db, { - maxRows: MAX_WORKFLOW_VFS_INDEX_ROWS, - }), - db - .select({ id: workflow.id, name: workflow.name, folderId: workflow.folderId }) - .from(workflow) - .where(and(eq(workflow.workspaceId, context.workspaceId), isNull(workflow.archivedAt))) - .limit(MAX_WORKFLOW_VFS_INDEX_ROWS + 1), - ]) - if (workflows.length > MAX_WORKFLOW_VFS_INDEX_ROWS) { - throw new Error(`Workflow VFS index exceeds the ${MAX_WORKFLOW_VFS_INDEX_ROWS} row limit`) - } - return { folderIndex, workflows, createdFolders: [] } -} - -function folderSegmentsForId(index: FolderPathIndex, folderId: string | null): string[] { - if (!folderId) return [] - const path = index.pathById.get(folderId) - if (!path) throw new Error('Workflow references an inactive or missing folder') - return parseFolderPath(path) -} - -function resolveWorkflowSources( - state: WorkflowVfsIndexState, - references: readonly WorkflowVfsPathReference[] -): ResolvedWorkflowSource[] { - const workflowsByPath = new Map() - for (const row of state.workflows) { - const path = canonicalSegmentsKey([ - ...folderSegmentsForId(state.folderIndex, row.folderId), - row.name, - ]) - if (!workflowsByPath.has(path)) workflowsByPath.set(path, row) - } - const foldersByPath = new Map() - for (const [folderId, path] of state.folderIndex.pathById) { - foldersByPath.set(canonicalSegmentsKey(parseFolderPath(path)), folderId) - } - - return references.map((reference) => { - if (reference.segments.length === 0) { - return { - source: reference.source, - error: 'Source must name a workflow or folder under workflows/', - } - } - const key = canonicalSegmentsKey(reference.segments) - const workflowRow = workflowsByPath.get(key) - if (workflowRow) return { source: reference.source, workflow: workflowRow } - const folderId = foldersByPath.get(key) - if (folderId) return { source: reference.source, folderId } - return { source: reference.source, error: `Not found: ${reference.source}` } - }) -} - -function throwFolderFailure(result: { error?: string; errorCode?: OrchestrationErrorCode }): never { - throwOrchestrationFailure(result, 'Workflow folder mutation failed') -} - -async function reloadFolderIndex(state: WorkflowVfsIndexState, workspaceId: string): Promise { - state.folderIndex = await loadActiveFolderPathIndex(workspaceId, 'workflow', db, { - maxRows: MAX_WORKFLOW_VFS_INDEX_ROWS, - }) -} - -async function ensureWorkflowFolderPath( - state: WorkflowVfsIndexState, - context: ActiveWorkspaceApplicationContext, - userId: string, - segments: readonly string[] -): Promise { - let folderId: string | null = null - for (let position = 0; position < segments.length; position += 1) { - const path = buildFolderPath(segments.slice(0, position + 1)) - const existing = state.folderIndex.idByPath.get(path) - if (existing) { - folderId = existing - continue - } - - const result = await createFolderAtPath({ - resourceType: 'workflow', - workspaceId: context.workspaceId, - userId, - path, - effects: false, - throwInfrastructure: true, - maxFolderRows: MAX_WORKFLOW_VFS_INDEX_ROWS, - }) - if (!result.success || !result.folder) { - if (result.errorCode === 'conflict') { - await reloadFolderIndex(state, context.workspaceId) - const concurrentlyCreated = state.folderIndex.idByPath.get(path) - if (concurrentlyCreated) { - folderId = concurrentlyCreated - continue - } - } - throwFolderFailure(result) - } - - state.createdFolders.push({ id: result.folder.id, name: result.folder.name, path }) - await reloadFolderIndex(state, context.workspaceId) - folderId = result.folder.id - } - return folderId -} - -function planDestination( - input: TransferWorkflowVfsItemsInput, - state: WorkflowVfsIndexState, - context: ActiveWorkspaceApplicationContext, - userId: string, - sourceCount: number -): DestinationPlan { - const segments = input.destination.segments - const plan = ( - dirMode: boolean, - folderSegments: string[], - leafName?: string, - knownFolderId?: string | null - ): DestinationPlan => { - let memo: Promise | undefined - return { - dirMode, - folderSegments, - leafName, - ensureFolderId: () => - (memo ??= - knownFolderId !== undefined - ? Promise.resolve(knownFolderId) - : folderSegments.length === 0 - ? Promise.resolve(null) - : ensureWorkflowFolderPath(state, context, userId, folderSegments)), - } - } - - if (segments.length === 0) return plan(true, [], undefined, null) - if (input.destination.trailingSlash) return plan(true, segments) - const existingFolderId = state.folderIndex.idByPath.get(buildFolderPath(segments)) - if (existingFolderId) return plan(true, segments, undefined, existingFolderId) - if (sourceCount > 1) { - throw new OrchestrationError( - 'validation', - `With multiple sources the destination must be a folder. "workflows/${canonicalSegmentsKey(segments)}" does not exist — end it with "/" to create it.` - ) - } - return plan(false, segments.slice(0, -1), segments.at(-1)) -} - -function expectedOutcomeMessage(error: unknown): string { - const classified = asOrchestrationError(error) - if (classified && classified.code !== 'internal') return classified.message - if ( - error instanceof WorkflowLockedError || - error instanceof FolderLockedError || - error instanceof FolderPathError - ) { - return error.message - } - throw error -} - -async function moveWorkflowRow(params: { - row: WorkflowVfsRow - targetName?: string - targetFolderId: string | null - context: ActiveWorkspaceApplicationContext - userId: string -}): Promise { - try { - await Promise.all([ - assertWorkflowMutable(params.row.id), - assertFolderMutable(params.targetFolderId), - ]) - } catch (error) { - if (error instanceof WorkflowLockedError || error instanceof FolderLockedError) { - throw new OrchestrationError('locked', error.message) - } - throw error - } - - return db.transaction(async (tx) => { - const [current] = await tx - .select({ id: workflow.id, name: workflow.name, folderId: workflow.folderId }) - .from(workflow) - .where( - and( - eq(workflow.id, params.row.id), - eq(workflow.workspaceId, params.context.workspaceId), - isNull(workflow.archivedAt) - ) - ) - .limit(1) - .for('update') - if (!current) throw new OrchestrationError('not_found', 'Workflow not found') - - const transition = await updateWorkflowRecord({ - workflowId: current.id, - userId: params.userId, - workspaceId: params.context.workspaceId, - currentName: current.name, - currentFolderId: current.folderId, - name: params.targetName, - folderId: params.targetFolderId, - tx, - }) - requireWorkflowTransition(transition, 'Workflow mutation failed') - if (!transition.workflow) throw new Error('Successful workflow move returned no workflow') - return { - id: transition.workflow.id, - name: transition.workflow.name, - previousFolderId: current.folderId, - folderId: transition.workflow.folderId, - } - }) -} - -function createdFolderAuditEntries(createdFolders: readonly CreatedFolderChange[]) { - return createdFolders.map((folder) => ({ - action: AuditAction.FOLDER_CREATED, - resourceType: AuditResourceType.FOLDER, - resourceId: folder.id, - resourceName: folder.name, - description: `Created workflow folder "${folder.path}"`, - metadata: { path: folder.path, folderResourceType: 'workflow' }, - })) -} - -export const createWorkflowVfsFolders = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.createVfsFolders, - resolveContext: ({ input }: { input: CreateWorkflowVfsFoldersInput }) => - resolveActiveWorkspaceApplicationContext(input.workspaceId), - async execute({ principal, input, context }) { - const paths = normalizeReferences(input.paths) - const state = await loadWorkflowVfsIndex(context) - const userId = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId - const outcomes: WorkflowVfsOutcome[] = [] - - for (const path of paths) { - if (path.segments.length === 0) { - outcomes.push({ - source: path.source, - resourceType: 'folder', - error: 'Path must include at least one folder segment', - }) - continue - } - try { - const folderId = await ensureWorkflowFolderPath(state, context, userId, path.segments) - outcomes.push({ - source: path.source, - targetSegments: path.segments, - resourceType: 'folder', - resourceId: folderId ?? undefined, - }) - } catch (error) { - outcomes.push({ - source: path.source, - resourceType: 'folder', - error: expectedOutcomeMessage(error), - }) - } - } - return { outcomes, createdFolders: state.createdFolders } - }, - projectAudit: ({ result }) => createdFolderAuditEntries(result.createdFolders), - afterSuccess: ({ context, result }) => - result.createdFolders.length > 0 - ? notifyFolderResourceChanged('workflow', context.workspaceId) - : undefined, -}) - -export const moveWorkflowVfsItems = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.moveVfsItems, - resolveContext: ({ input }: { input: TransferWorkflowVfsItemsInput }) => - resolveActiveWorkspaceApplicationContext(input.workspaceId), - async execute({ principal, input, context }) { - const sources = normalizeReferences(input.sources) - validateDestination(input.destination) - const state = await loadWorkflowVfsIndex(context) - const refs = resolveWorkflowSources(state, sources) - const userId = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId - const destination = planDestination(input, state, context, userId, sources.length) - if (!destination.dirMode && (destination.leafName?.length ?? 0) > MAX_WORKFLOW_NAME_LENGTH) { - throw new OrchestrationError( - 'validation', - `Workflow name must be ${MAX_WORKFLOW_NAME_LENGTH} characters or less` - ) - } - const outcomes: WorkflowVfsOutcome[] = [] - const movedWorkflows: MovedWorkflowChange[] = [] - const movedFolders: MovedFolderChange[] = [] - - for (const ref of refs) { - if (ref.error) { - outcomes.push({ source: ref.source, resourceType: 'workflow', error: ref.error }) - continue - } - if (ref.workflow) { - const targetName = destination.dirMode - ? ref.workflow.name - : (destination.leafName as string) - try { - const targetFolderId = await destination.ensureFolderId() - const change = await moveWorkflowRow({ - row: ref.workflow, - targetName: destination.dirMode ? undefined : targetName, - targetFolderId, - context, - userId, - }) - movedWorkflows.push(change) - outcomes.push({ - source: ref.source, - targetSegments: [...destination.folderSegments, change.name], - resourceType: 'workflow', - resourceId: change.id, - }) - } catch (error) { - outcomes.push({ - source: ref.source, - resourceType: 'workflow', - error: expectedOutcomeMessage(error), - }) - } - continue - } - - const folderId = ref.folderId as string - try { - const targetFolderId = await destination.ensureFolderId() - if (targetFolderId === folderId) { - outcomes.push({ - source: ref.source, - resourceType: 'folder', - error: 'Cannot move a folder into itself', - }) - continue - } - const sourcePath = state.folderIndex.pathById.get(folderId) - const sourceRow = state.folderIndex.rowById.get(folderId) - if (!sourcePath || !sourceRow) throw new Error('Workflow folder path index is incomplete') - const finalLeaf = destination.dirMode - ? (sources.find((source) => source.source === ref.source)?.segments.at(-1) ?? '') - : (destination.leafName as string) - const destinationPath = buildFolderPath([...destination.folderSegments, finalLeaf]) - const result = await relocateFolderByPath({ - resourceType: 'workflow', - workspaceId: context.workspaceId, - userId, - path: sourcePath, - destinationPath, - effects: false, - throwInfrastructure: true, - maxFolderRows: MAX_WORKFLOW_VFS_INDEX_ROWS, - }) - if (!result.success || !result.folder) throwFolderFailure(result) - movedFolders.push({ - id: result.folder.id, - name: result.folder.name, - sourcePath, - destinationPath, - }) - outcomes.push({ - source: ref.source, - targetSegments: [...destination.folderSegments, finalLeaf], - resourceType: 'folder', - resourceId: result.folder.id, - }) - } catch (error) { - outcomes.push({ - source: ref.source, - resourceType: 'folder', - error: expectedOutcomeMessage(error), - }) - } - } - - return { outcomes, createdFolders: state.createdFolders, movedWorkflows, movedFolders } - }, - projectAudit: ({ result }) => [ - ...createdFolderAuditEntries(result.createdFolders), - ...result.movedWorkflows.map((change) => ({ - action: AuditAction.WORKFLOW_UPDATED, - resourceType: AuditResourceType.WORKFLOW, - resourceId: change.id, - resourceName: change.name, - description: `Moved workflow "${change.name}"`, - metadata: { - previousFolderId: change.previousFolderId, - folderId: change.folderId, - }, - })), - ...result.movedFolders.map((change) => ({ - action: AuditAction.FOLDER_MOVED, - resourceType: AuditResourceType.FOLDER, - resourceId: change.id, - resourceName: change.name, - description: `Moved workflow folder to "${change.destinationPath}"`, - metadata: { - sourcePath: change.sourcePath, - destinationPath: change.destinationPath, - folderResourceType: 'workflow', - }, - })), - ], - afterSuccess: async ({ context, result }) => { - for (const change of result.movedWorkflows) { - await notifyWorkflowUpdated(change.id) - } - if (result.createdFolders.length > 0 || result.movedFolders.length > 0) { - await notifyFolderResourceChanged('workflow', context.workspaceId) - } - }, -}) - -export const copyWorkflowVfsItems = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.copyVfsItems, - resolveContext: ({ input }: { input: TransferWorkflowVfsItemsInput }) => - resolveActiveWorkspaceApplicationContext(input.workspaceId), - async execute({ principal, input, context }) { - const sources = normalizeReferences(input.sources) - validateDestination(input.destination) - const state = await loadWorkflowVfsIndex(context) - const refs = resolveWorkflowSources(state, sources) - const userId = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId - const destination = planDestination(input, state, context, userId, sources.length) - if (!destination.dirMode && (destination.leafName?.length ?? 0) > MAX_WORKFLOW_NAME_LENGTH) { - throw new OrchestrationError( - 'validation', - `Workflow name must be ${MAX_WORKFLOW_NAME_LENGTH} characters or less` - ) - } - const outcomes: WorkflowVfsOutcome[] = [] - const duplicatedWorkflows: DuplicatedWorkflowChange[] = [] - - for (const ref of refs) { - if (ref.error) { - outcomes.push({ source: ref.source, resourceType: 'workflow', error: ref.error }) - continue - } - if (!ref.workflow) { - outcomes.push({ - source: ref.source, - resourceType: 'folder', - error: 'Workflow folders cannot be copied.', - }) - continue - } - - try { - const targetFolderId = await destination.ensureFolderId() - const targetName = destination.dirMode - ? ref.workflow.name - : (destination.leafName as string) - const duplicated = await db.transaction(async (tx) => { - const [source] = await tx - .select({ id: workflow.id }) - .from(workflow) - .where( - and( - eq(workflow.id, ref.workflow?.id as string), - eq(workflow.workspaceId, context.workspaceId), - isNull(workflow.archivedAt) - ) - ) - .limit(1) - .for('update') - if (!source) throw new OrchestrationError('not_found', 'Workflow not found') - return duplicateWorkflowRecord({ - sourceWorkflowId: source.id, - userId, - workspaceId: context.workspaceId, - folderId: targetFolderId, - name: targetName, - requestId: generateRequestId(), - tx, - }) - }) - duplicatedWorkflows.push({ - id: duplicated.id, - name: duplicated.name, - sourceWorkflowId: ref.workflow.id, - }) - outcomes.push({ - source: ref.source, - targetSegments: [...destination.folderSegments, duplicated.name], - resourceType: 'workflow', - resourceId: duplicated.id, - }) - } catch (error) { - outcomes.push({ - source: ref.source, - resourceType: 'workflow', - error: expectedOutcomeMessage(error), - }) - } - } - - return { outcomes, createdFolders: state.createdFolders, duplicatedWorkflows } - }, - projectAudit: ({ context, result }) => [ - ...createdFolderAuditEntries(result.createdFolders), - ...result.duplicatedWorkflows.map((change) => ({ - action: AuditAction.WORKFLOW_DUPLICATED, - resourceType: AuditResourceType.WORKFLOW, - resourceId: change.id, - resourceName: change.name, - description: `Duplicated workflow as "${change.name}"`, - metadata: { - sourceWorkflowId: change.sourceWorkflowId, - workspaceId: context.workspaceId, - }, - })), - ], - afterSuccess: async ({ context, result }) => { - for (const change of result.duplicatedWorkflows) { - await notifyWorkflowUpdated(change.id) - } - if (result.createdFolders.length > 0) { - await notifyFolderResourceChanged('workflow', context.workspaceId) - } - }, -}) - -export const deleteWorkflowVfsItems = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.deleteVfsItems, - resolveContext: ({ input }: { input: DeleteWorkflowVfsItemsInput }) => - resolveActiveWorkspaceApplicationContext(input.workspaceId), - async execute({ principal, input, context }) { - const paths = normalizeReferences(input.paths) - const state = await loadWorkflowVfsIndex(context) - const refs = resolveWorkflowSources(state, paths) - const userId = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId - const outcomes: WorkflowVfsOutcome[] = [] - const deletedWorkflows: DeletedWorkflowChange[] = [] - const deletedFolders: DeletedFolderChange[] = [] - - for (const ref of refs) { - if (ref.error) { - outcomes.push({ source: ref.source, resourceType: 'workflow', error: ref.error }) - continue - } - if (ref.workflow) { - try { - await assertWorkflowMutable(ref.workflow.id) - const result = await deleteWorkflowRecord({ - workflowId: ref.workflow.id, - userId, - notifySocket: false, - }) - requireWorkflowTransition(result, 'Workflow deletion failed') - if (!result.workflow || !result.archived) { - throw new OrchestrationError('validation', 'Workflow is already deleted') - } - deletedWorkflows.push({ id: result.workflow.id, name: result.workflow.name }) - outcomes.push({ - source: ref.source, - resourceType: 'workflow', - resourceId: result.workflow.id, - }) - } catch (error) { - outcomes.push({ - source: ref.source, - resourceType: 'workflow', - error: expectedOutcomeMessage(error), - }) - } - continue - } - - const folderId = ref.folderId as string - const path = state.folderIndex.pathById.get(folderId) - if (!path) throw new Error('Workflow folder path index is incomplete') - try { - const result = await deleteFolderByPath({ - resourceType: 'workflow', - workspaceId: context.workspaceId, - userId, - path, - recursive: true, - effects: false, - throwInfrastructure: true, - maxFolderRows: MAX_WORKFLOW_VFS_INDEX_ROWS, - }) - if (!result.success || !result.folderId || !result.folderName || !result.deletedItems) { - throwFolderFailure(result) - } - deletedFolders.push({ - id: result.folderId, - name: result.folderName, - path, - workflows: result.deletedItems.workflows ?? 0, - folders: result.deletedItems.folders, - }) - outcomes.push({ - source: ref.source, - resourceType: 'folder', - resourceId: result.folderId, - }) - } catch (error) { - outcomes.push({ - source: ref.source, - resourceType: 'folder', - error: expectedOutcomeMessage(error), - }) - } - } - - return { outcomes, deletedWorkflows, deletedFolders } - }, - projectAudit: ({ result }) => [ - ...result.deletedWorkflows.map((change) => ({ - action: AuditAction.WORKFLOW_DELETED, - resourceType: AuditResourceType.WORKFLOW, - resourceId: change.id, - resourceName: change.name, - description: `Archived workflow "${change.name}"`, - metadata: { archived: true }, - })), - ...result.deletedFolders.map((change) => ({ - action: AuditAction.FOLDER_DELETED, - resourceType: AuditResourceType.FOLDER, - resourceId: change.id, - resourceName: change.name, - description: `Deleted workflow folder "${change.path}"`, - metadata: { - folderResourceType: 'workflow', - path: change.path, - affected: { - workflows: change.workflows, - subfolders: Math.max(change.folders - 1, 0), - }, - }, - })), - ], - afterSuccess: async ({ context, result }) => { - for (const workflow of result.deletedWorkflows) { - await notifyWorkflowDeleted(workflow.id) - } - if (result.deletedFolders.length > 0) { - await notifyFolderResourceChanged('workflow', context.workspaceId) - } - }, -}) diff --git a/apps/sim/lib/workflows/comparison/describe.ts b/apps/sim/lib/workflows/comparison/describe.ts index 4ef398720a0..6a98b331fda 100644 --- a/apps/sim/lib/workflows/comparison/describe.ts +++ b/apps/sim/lib/workflows/comparison/describe.ts @@ -1,7 +1,6 @@ import { createLogger } from '@sim/logger' import type { WorkflowDiffSummary } from '@/lib/workflows/comparison/compare' import { - formatValueForDisplay, resolveFieldLabel, resolveValueForDisplay, } from '@/lib/workflows/comparison/resolve-values' @@ -22,50 +21,6 @@ const MAX_EDGE_DETAILS = 3 const logger = createLogger('WorkflowDescribe') -/** - * Convert a WorkflowDiffSummary to a human-readable string for AI description generation - */ -export function formatDiffSummaryForDescription(summary: WorkflowDiffSummary): string { - if (!summary.hasChanges) { - return 'No structural changes detected (configuration may have changed)' - } - - const changes: string[] = [] - - for (const block of summary.addedBlocks) { - const name = block.name || block.type - changes.push(`Added block: ${name} (${block.type})`) - } - - for (const block of summary.removedBlocks) { - const name = block.name || block.type - changes.push(`Removed block: ${name} (${block.type})`) - } - - for (const block of summary.modifiedBlocks) { - const name = block.name || block.type - const meaningfulChanges = block.changes.filter((c) => !c.field.endsWith('.properties')) - for (const change of meaningfulChanges.slice(0, MAX_CHANGES_PER_BLOCK)) { - const fieldLabel = resolveFieldLabel(block.type, change.field) - const oldStr = formatValueForDisplay(change.oldValue) - const newStr = formatValueForDisplay(change.newValue) - changes.push(`Modified ${name}: ${fieldLabel} changed from "${oldStr}" to "${newStr}"`) - } - if (meaningfulChanges.length > MAX_CHANGES_PER_BLOCK) { - changes.push( - ` ...and ${meaningfulChanges.length - MAX_CHANGES_PER_BLOCK} more changes in ${name}` - ) - } - } - - formatEdgeChanges(summary, changes) - formatCountChanges(summary.loopChanges, 'loop', changes) - formatCountChanges(summary.parallelChanges, 'parallel group', changes) - formatVariableChanges(summary, changes) - - return changes.join('\n') -} - /** * Converts a WorkflowDiffSummary to a human-readable string with resolved display names. * Resolves IDs (credentials, channels, workflows, etc.) to human-readable names using diff --git a/apps/sim/lib/workflows/comparison/format-description.test.ts b/apps/sim/lib/workflows/comparison/format-description.test.ts index 5eb1153517e..0f5739f3ca5 100644 --- a/apps/sim/lib/workflows/comparison/format-description.test.ts +++ b/apps/sim/lib/workflows/comparison/format-description.test.ts @@ -34,10 +34,7 @@ vi.mock('@/lib/selectors/client/execute-selector', () => ({ import { WorkflowBuilder } from '@sim/testing' import type { WorkflowDiffSummary } from '@/lib/workflows/comparison/compare' -import { - formatDiffSummaryForDescription, - formatDiffSummaryForDescriptionAsync, -} from '@/lib/workflows/comparison/describe' +import { formatDiffSummaryForDescriptionAsync } from '@/lib/workflows/comparison/describe' import { resolveFieldLabel, resolveValueForDisplay, @@ -119,8 +116,10 @@ describe('resolveValueForDisplay', () => { }) }) -describe('formatDiffSummaryForDescription', () => { - it('uses human-readable field labels for modified blocks', () => { +describe('formatDiffSummaryForDescriptionAsync shared formatting', () => { + const state = { blocks: {} } as any + + it('uses human-readable field labels for modified blocks', async () => { mockGetBlock.mockReturnValue({ subBlocks: [ { id: 'systemPrompt', title: 'System Prompt' }, @@ -143,7 +142,7 @@ describe('formatDiffSummaryForDescription', () => { ], }) - const result = formatDiffSummaryForDescription(summary) + const result = await formatDiffSummaryForDescriptionAsync(summary, state, 'wf-1') expect(result).toContain( 'Modified My Agent: System Prompt changed from "You are helpful" to "You are an expert"' ) @@ -154,7 +153,7 @@ describe('formatDiffSummaryForDescription', () => { expect(result).not.toContain('model changed') }) - it('filters out .properties changes', () => { + it('filters out .properties changes', async () => { mockGetBlock.mockReturnValue({ subBlocks: [] }) const summary = emptyDiffSummary({ @@ -177,13 +176,13 @@ describe('formatDiffSummaryForDescription', () => { ], }) - const result = formatDiffSummaryForDescription(summary) + const result = await formatDiffSummaryForDescriptionAsync(summary, state, 'wf-1') expect(result).toContain('systemPrompt changed') expect(result).not.toContain('.properties') expect(result).not.toContain('model.properties') }) - it('respects MAX_CHANGES_PER_BLOCK limit of 6', () => { + it('respects MAX_CHANGES_PER_BLOCK limit of 6', async () => { mockGetBlock.mockReturnValue({ subBlocks: [] }) const changes = Array.from({ length: 8 }, (_, i) => ({ @@ -197,14 +196,14 @@ describe('formatDiffSummaryForDescription', () => { modifiedBlocks: [{ id: 'b1', type: 'agent', name: 'Agent', changes }], }) - const result = formatDiffSummaryForDescription(summary) + const result = await formatDiffSummaryForDescriptionAsync(summary, state, 'wf-1') const lines = result.split('\n') const modifiedLines = lines.filter((l) => l.startsWith('Modified')) expect(modifiedLines).toHaveLength(6) expect(result).toContain('...and 2 more changes in Agent') }) - it('shows edge changes with block names', () => { + it('shows edge changes with block names', async () => { const summary = emptyDiffSummary({ hasChanges: true, edgeChanges: { @@ -218,13 +217,13 @@ describe('formatDiffSummaryForDescription', () => { }, }) - const result = formatDiffSummaryForDescription(summary) + const result = await formatDiffSummaryForDescriptionAsync(summary, state, 'wf-1') expect(result).toContain('Added connection: My Agent -> Slack') expect(result).toContain('Added connection: Router -> Gmail') expect(result).toContain('Removed connection: Function -> Webhook') }) - it('truncates edge details beyond MAX_EDGE_DETAILS', () => { + it('truncates edge details beyond MAX_EDGE_DETAILS', async () => { const summary = emptyDiffSummary({ hasChanges: true, edgeChanges: { @@ -241,7 +240,7 @@ describe('formatDiffSummaryForDescription', () => { }, }) - const result = formatDiffSummaryForDescription(summary) + const result = await formatDiffSummaryForDescriptionAsync(summary, state, 'wf-1') const connectionLines = result.split('\n').filter((l) => l.startsWith('Added connection')) expect(connectionLines).toHaveLength(3) expect(result).toContain('...and 2 more added connection(s)') diff --git a/apps/sim/lib/workflows/editing/validation.ts b/apps/sim/lib/workflows/editing/validation.ts index ba5ac500457..2e9bab4f840 100644 --- a/apps/sim/lib/workflows/editing/validation.ts +++ b/apps/sim/lib/workflows/editing/validation.ts @@ -1384,67 +1384,6 @@ function collectSelectorFields( return fields } -/** - * Validates selector IDs in the workflow state exist in the database. - * Returns validation errors for any invalid selector IDs. - * - * `options.includeCredentials` controls whether oauth-input credential fields - * are validated (the edit path defaults to skipping them since they are - * pre-validated; the lint opts in to close that gap). - */ -export async function validateWorkflowSelectorIds( - workflowState: any, - context: { userId: string; workspaceId?: string }, - options: { includeCredentials?: boolean } = {} -): Promise { - const logger = createLogger('EditWorkflowSelectorValidation') - const errors: ValidationError[] = [] - - const selectorsToValidate = collectSelectorFields(workflowState, options) - - if (selectorsToValidate.length === 0) { - return errors - } - - logger.info('Validating selector IDs', { - selectorCount: selectorsToValidate.length, - userId: context.userId, - workspaceId: context.workspaceId, - }) - - // Validate each selector field - for (const selector of selectorsToValidate) { - const result = await validateSelectorIds(selector.selectorType, selector.value, context) - - if (result.invalid.length > 0) { - // Include warning info (like available credentials) in the error message for better LLM feedback - const warningInfo = result.warning ? `. ${result.warning}` : '' - errors.push({ - blockId: selector.blockId, - blockType: selector.blockType, - field: selector.fieldName, - value: selector.value, - error: `Invalid ${selector.selectorType} ID(s): ${result.invalid.join(', ')} — they do not exist in this workspace or you lack access. Discover valid ids first (glob/read the matching workspace resource, e.g. environment/credentials.json, knowledgebases/*/meta.json, tables/*/meta.json) instead of guessing${warningInfo}`, - }) - } else if (result.warning) { - // Log warnings that don't have errors (shouldn't happen for credentials but may for other selectors) - logger.warn(result.warning, { - blockId: selector.blockId, - fieldName: selector.fieldName, - }) - } - } - - if (errors.length > 0) { - logger.warn('Found invalid selector IDs', { - errorCount: errors.length, - errors: errors.map((e) => ({ blockId: e.blockId, field: e.field, error: e.error })), - }) - } - - return errors -} - /** * Lint-facing Tier-2 resolution: validate every ACTIVE credential/resource * member (including oauth-input) against the workspace and return the references diff --git a/apps/sim/lib/workflows/executor/execution-queries.test.ts b/apps/sim/lib/workflows/executor/execution-queries.test.ts deleted file mode 100644 index 23cf0da1c16..00000000000 --- a/apps/sim/lib/workflows/executor/execution-queries.test.ts +++ /dev/null @@ -1,91 +0,0 @@ -import { queueTableRows, resetDbChainMock, schemaMock } from '@sim/testing/mocks' -import { asyncJobsMock, asyncJobsMockFns } from '@sim/testing/mocks/async-jobs.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -vi.mock('@/lib/core/async-jobs', () => asyncJobsMock) - -import { resolveWorkflowExecutionOwnership } from '@/lib/workflows/executor/execution-queries' - -const mockGetJob = asyncJobsMockFns.mockJobQueue.getJob -const { mockGetJobQueue } = asyncJobsMockFns - -describe('resolveWorkflowExecutionOwnership', () => { - beforeEach(() => { - resetDbChainMock() - }) - - it('accepts a durable execution bound to the requested workflow', async () => { - queueTableRows(schemaMock.workflowExecutionLogs, [{ workflowId: 'workflow-1' }]) - - await expect( - resolveWorkflowExecutionOwnership('execution-1', 'workflow-1') - ).resolves.toMatchObject({ belongsToWorkflow: true, workflowGroupWorkspaceId: null }) - expect(mockGetJobQueue).not.toHaveBeenCalled() - }) - - it('rejects a durable execution bound to another workflow', async () => { - queueTableRows(schemaMock.workflowExecutionLogs, [{ workflowId: 'workflow-2' }]) - - await expect( - resolveWorkflowExecutionOwnership('execution-1', 'workflow-1') - ).resolves.toMatchObject({ belongsToWorkflow: false }) - expect(mockGetJobQueue).not.toHaveBeenCalled() - }) - - it('projects the workflow-group workspace from the same log row it already reads', async () => { - queueTableRows(schemaMock.workflowExecutionLogs, [ - { workflowId: 'workflow-1', workspaceId: 'workspace-1', executionOrigin: 'workflow_group' }, - ]) - - await expect( - resolveWorkflowExecutionOwnership('execution-1', 'workflow-1') - ).resolves.toMatchObject({ - belongsToWorkflow: true, - workflowGroupWorkspaceId: 'workspace-1', - }) - }) - - /** - * A run that paused before its log row landed — or whose log row is gone — is - * still durable, and the paused row's workflow id is the only thing standing - * between it and the queue fallback, which would answer `false` for a run the - * queue no longer holds. Nothing else here queues a `pausedExecutions` row, so - * dropping it from the ownership decision is otherwise invisible. - */ - it('resolves ownership from a paused-only execution', async () => { - queueTableRows(schemaMock.workflowExecutionLogs, []) - queueTableRows(schemaMock.pausedExecutions, [{ workflowId: 'workflow-1' }]) - - await expect( - resolveWorkflowExecutionOwnership('execution-1', 'workflow-1') - ).resolves.toMatchObject({ - belongsToWorkflow: true, - workflowGroupWorkspaceId: null, - priorStatus: null, - }) - expect(mockGetJobQueue).not.toHaveBeenCalled() - }) - - it('rejects a paused-only execution bound to another workflow', async () => { - queueTableRows(schemaMock.workflowExecutionLogs, []) - queueTableRows(schemaMock.pausedExecutions, [{ workflowId: 'workflow-2' }]) - - await expect( - resolveWorkflowExecutionOwnership('execution-1', 'workflow-1') - ).resolves.toMatchObject({ belongsToWorkflow: false }) - expect(mockGetJobQueue).not.toHaveBeenCalled() - }) - - it('checks deterministic queue metadata before the durable log exists', async () => { - mockGetJob.mockResolvedValue({ metadata: { workflowId: 'workflow-1' } }) - - await expect( - resolveWorkflowExecutionOwnership('execution-1', 'workflow-1') - ).resolves.toMatchObject({ - belongsToWorkflow: true, - workflowGroupWorkspaceId: null, - priorStatus: null, - }) - expect(mockGetJob).toHaveBeenCalledWith('workflow-execution:execution-1') - }) -}) diff --git a/apps/sim/lib/workflows/executor/execution-queries.ts b/apps/sim/lib/workflows/executor/execution-queries.ts index 6c150192eae..0835acd023f 100644 --- a/apps/sim/lib/workflows/executor/execution-queries.ts +++ b/apps/sim/lib/workflows/executor/execution-queries.ts @@ -1,9 +1,6 @@ import { db } from '@sim/db' import { pausedExecutions, workflowExecutionLogs } from '@sim/db/schema' import { and, asc, desc, eq, gt, gte, lt, lte, or, sql } from 'drizzle-orm' -import { getJobQueue } from '@/lib/core/async-jobs' -import { workflowExecutionOriginSql } from '@/lib/logs/execution-origin' -import { WORKFLOW_EXECUTION_JOB_ID_PREFIX } from '@/lib/workflows/executor/execution-job-ids' export type WorkflowExecutionStatus = | 'pending' @@ -94,78 +91,3 @@ export async function listWorkflowExecutions(input: ListWorkflowExecutionsInput) nextCursor: hasMore && last ? { startedAt: last.startedAt, rowId: last.rowId } : null, } } - -export interface WorkflowExecutionOwnership { - /** Whether the execution id really belongs to the asserted workflow. */ - belongsToWorkflow: boolean - /** - * Workspace of the durable log row when — and only when — the execution was - * produced by a workflow group. `null` for a standalone run, a queue-only run - * that has no log row yet, and a paused-only run. - */ - workflowGroupWorkspaceId: string | null - /** - * Status the durable log row already carried. `null` when there is no log row - * — a queue-only run, or a paused-only run. - */ - priorStatus: string | null -} - -/** - * Resolves the durable and queued execution records without trusting the - * workflow id supplied by an HTTP path. Mutating callers must use this before - * operating on an execution id because execution ids are globally unique, not - * nested DB keys under a workflow. - * - * The workflow-group origin and the row's current status both ride along on the - * same log row the ownership check already reads. A group run owns a table cell - * sidecar, so cancelling only the workflow log would leave the cell stuck as - * running; and a cancel has to tell a live run apart from one that had already - * finished. Resolving either from a second SELECT of the identical row would - * double the read on every cancel. - */ -export async function resolveWorkflowExecutionOwnership( - executionId: string, - workflowId: string -): Promise { - const [logRows, pausedRows] = await Promise.all([ - db - .select({ - workflowId: workflowExecutionLogs.workflowId, - workspaceId: workflowExecutionLogs.workspaceId, - status: workflowExecutionLogs.status, - executionOrigin: workflowExecutionOriginSql(), - }) - .from(workflowExecutionLogs) - .where(eq(workflowExecutionLogs.executionId, executionId)) - .limit(1), - db - .select({ workflowId: pausedExecutions.workflowId }) - .from(pausedExecutions) - .where(eq(pausedExecutions.executionId, executionId)) - .limit(1), - ]) - - const logRow = logRows[0] - const workflowGroupWorkspaceId = - logRow?.executionOrigin === 'workflow_group' && logRow.workspaceId ? logRow.workspaceId : null - - const durableWorkflowIds = [logRow?.workflowId, pausedRows[0]?.workflowId].filter( - (value): value is string => typeof value === 'string' - ) - if (durableWorkflowIds.length > 0) { - return { - belongsToWorkflow: durableWorkflowIds.every((value) => value === workflowId), - workflowGroupWorkspaceId, - priorStatus: logRow?.status ?? null, - } - } - - const queue = await getJobQueue() - const job = await queue.getJob(`${WORKFLOW_EXECUTION_JOB_ID_PREFIX}${executionId}`) - return { - belongsToWorkflow: job?.metadata.workflowId === workflowId, - workflowGroupWorkspaceId: null, - priorStatus: null, - } -} diff --git a/apps/sim/lib/workflows/utils.ts b/apps/sim/lib/workflows/utils.ts index d7d070372ae..badbecda842 100644 --- a/apps/sim/lib/workflows/utils.ts +++ b/apps/sim/lib/workflows/utils.ts @@ -459,24 +459,6 @@ export async function setWorkflowVariables(workflowId: string, variables: Record // ── Folder CRUD ── -export async function verifyFolderWorkspace( - folderId: string, - workspaceId: string -): Promise { - const [row] = await db - .select({ id: folderTable.id }) - .from(folderTable) - .where( - and( - eq(folderTable.id, folderId), - eq(folderTable.workspaceId, workspaceId), - eq(folderTable.resourceType, 'workflow') - ) - ) - .limit(1) - return Boolean(row) -} - export async function listFolders(workspaceId: string) { return db .select({ diff --git a/scripts/check-egress-boundary.ts b/scripts/check-egress-boundary.ts index 3909ddf245c..5997104f0fe 100644 --- a/scripts/check-egress-boundary.ts +++ b/scripts/check-egress-boundary.ts @@ -74,8 +74,6 @@ const ALLOWED = new Set([ 'apps/sim/lib/core/security/input-validation.server.ts', // Streaming MCP transport, built on the guard's pinned dispatcher. 'apps/sim/lib/mcp/pinned-fetch.ts', - // Builds a dispatcher to carry a caller's deadline; issues no request itself. - 'apps/sim/lib/core/utils/fetch-deadline.ts', ]) function walk(dir: string, out: string[] = []): string[] { From 552f6ca3531bd033bfa5c39d66543e2fd8dab7c4 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:15:46 -0700 Subject: [PATCH 11/30] chore(cleanup): remove unreferenced Copilot table commands, workflow mutation handlers, and param types chore(cleanup): drop unused billing client usage-pill and plan-view exports --- apps/sim/lib/billing/client/consts.ts | 23 -- apps/sim/lib/billing/client/index.ts | 6 +- apps/sim/lib/billing/client/plan-view.ts | 48 +--- apps/sim/lib/billing/client/utils.ts | 14 - .../mothership/application/table-commands.ts | 114 -------- .../mothership/tools/handlers/param-types.ts | 258 ------------------ .../tools/handlers/workflow/mutations.test.ts | 39 --- .../tools/handlers/workflow/mutations.ts | 217 --------------- 8 files changed, 3 insertions(+), 716 deletions(-) delete mode 100644 apps/sim/lib/billing/client/consts.ts diff --git a/apps/sim/lib/billing/client/consts.ts b/apps/sim/lib/billing/client/consts.ts deleted file mode 100644 index 0809e8e836d..00000000000 --- a/apps/sim/lib/billing/client/consts.ts +++ /dev/null @@ -1,23 +0,0 @@ -/** - * Usage percentage thresholds for visual states. - */ -export const USAGE_THRESHOLDS = { - /** Warning threshold (yellow/orange state) */ - WARNING: 75, - /** Critical threshold (red state) */ - CRITICAL: 90, -} as const - -/** - * Color values for usage pill states using CSS variables - */ -export const USAGE_PILL_COLORS = { - /** Unfilled pill color (gray) */ - UNFILLED: 'var(--surface-7)', - /** Normal filled pill color (blue) */ - FILLED: 'var(--brand-secondary)', - /** Warning state pill color (yellow/orange) */ - WARNING: 'var(--warning)', - /** Critical/limit reached pill color (red) */ - AT_LIMIT: 'var(--text-error)', -} as const diff --git a/apps/sim/lib/billing/client/index.ts b/apps/sim/lib/billing/client/index.ts index 61879295b7a..c3e7a12d3f0 100644 --- a/apps/sim/lib/billing/client/index.ts +++ b/apps/sim/lib/billing/client/index.ts @@ -1,14 +1,10 @@ -export { USAGE_PILL_COLORS, USAGE_THRESHOLDS } from './consts' export { type CtaIntent, type CtaVariant, - derivePlanView, getUpgradeCardCta, type PlanCardCta, type PlanTier, - type PlanView, - resolvePlanTier, type UpgradeCardId, } from './plan-view' export { useLimitUpgradeToast } from './use-limit-upgrade-toast' -export { getFilledPillColor, getSubscriptionAccessState } from './utils' +export { getSubscriptionAccessState } from './utils' diff --git a/apps/sim/lib/billing/client/plan-view.ts b/apps/sim/lib/billing/client/plan-view.ts index 23a30f8a263..d062e61fafa 100644 --- a/apps/sim/lib/billing/client/plan-view.ts +++ b/apps/sim/lib/billing/client/plan-view.ts @@ -1,16 +1,7 @@ -import { MAX_TIER_CREDITS } from '@/lib/billing/constants' -import { getPlanTierCredits, isEnterprise, isFree, isPaid } from '@/lib/billing/plan-helpers' - /** - * Canonical client-side plan abstraction. - * - * Single source of truth for the plan-derived UI decisions shared across the - * upgrade page, the home credits chip, the sidebar usage indicator, and the - * settings billing page: which tier the user is on, the per-card CTA on the - * upgrade page, whether the upgrade page is accessible, and whether credit - * balances should be shown. + * Client-side plan tiers and the per-card CTA on the upgrade page. * - * Pure and framework-free — see {@link usePlanView} for the React Query wrapper. + * Pure and framework-free. */ /** Credit-tier-resolved plan identity used to drive upgrade-page UI. */ @@ -34,31 +25,9 @@ export interface PlanCardCta { highlighted: boolean } -/** Plan-derived view consumed by every billing surface. */ -export interface PlanView { - tier: PlanTier - isFree: boolean - isPaid: boolean - isEnterprise: boolean - /** Enterprise manages billing out-of-band — it cannot use the self-serve upgrade page. */ - canAccessUpgrade: boolean - /** Credit balances are meaningless for enterprise (custom limits) and are hidden. */ - showCredits: boolean -} - /** Tier ordering used to derive upgrade/downgrade/highlight relationships. */ const PLAN_RANK: Record = { free: 0, pro: 1, max: 2, enterprise: 3 } -/** - * Resolve a plan name to its credit-tier identity. Paid pro/team plans split - * into `pro` / `max` by their credit allocation (>= 25k credits => Max). - */ -export function resolvePlanTier(plan: string | null | undefined): PlanTier { - if (isEnterprise(plan)) return 'enterprise' - if (isFree(plan)) return 'free' - return getPlanTierCredits(plan) >= MAX_TIER_CREDITS ? 'max' : 'pro' -} - /** * Derive the CTA for a single upgrade card given the current plan tier. * @@ -101,16 +70,3 @@ export function getUpgradeCardCta(current: PlanTier, card: UpgradeCardId): PlanC highlighted: isNextStepUp, } } - -/** Derive the shared plan view from a plan name. */ -export function derivePlanView(plan: string | null | undefined): PlanView { - const enterprise = isEnterprise(plan) - return { - tier: resolvePlanTier(plan), - isFree: isFree(plan), - isPaid: isPaid(plan), - isEnterprise: enterprise, - canAccessUpgrade: !enterprise, - showCredits: !enterprise, - } -} diff --git a/apps/sim/lib/billing/client/utils.ts b/apps/sim/lib/billing/client/utils.ts index b2e28b56ee3..a274729d178 100644 --- a/apps/sim/lib/billing/client/utils.ts +++ b/apps/sim/lib/billing/client/utils.ts @@ -6,7 +6,6 @@ import { DEFAULT_FREE_CREDITS } from '@/lib/billing/constants' import { isFree, isMaxTier, isPro } from '@/lib/billing/plan-helpers' import { hasUsableSubscriptionAccess } from '@/lib/billing/subscriptions/utils' -import { USAGE_PILL_COLORS } from './consts' import type { BillingStatus, SubscriptionData, UsageData } from './types' const defaultUsage: UsageData = { @@ -173,16 +172,3 @@ export function canUpgrade( const status = getSubscriptionStatus(subscriptionData) return isFree(status.plan) || isPro(status.plan) } - -/** - * Get the appropriate filled pill color based on usage thresholds. - * - * @param isCritical - Whether usage is at critical level (blocked or >= 90%) - * @param isWarning - Whether usage is at warning level (>= 75% but < critical) - * @returns CSS color value for filled pills - */ -export function getFilledPillColor(isCritical: boolean, isWarning: boolean): string { - if (isCritical) return USAGE_PILL_COLORS.AT_LIMIT - if (isWarning) return USAGE_PILL_COLORS.WARNING - return USAGE_PILL_COLORS.FILLED -} diff --git a/apps/sim/lib/mothership/application/table-commands.ts b/apps/sim/lib/mothership/application/table-commands.ts index e00c9c7aa6c..0c31bf0aa2f 100644 --- a/apps/sim/lib/mothership/application/table-commands.ts +++ b/apps/sim/lib/mothership/application/table-commands.ts @@ -1,29 +1,9 @@ import { executeCopilotTableUseCase } from '@/lib/mothership/application/execute-table-use-case' import type { CopilotTableDelegationContext } from '@/lib/mothership/auth/table-delegation' -import { - type DeleteCopilotTablesInput, - deleteCopilotTables, -} from '@/lib/table/application/copilot-table-lifecycle' -import { - type AddTableGroupOutputInput, - addWorkflowTableGroupOutput, - type CreateTableEnrichmentGroupInput, - type CreateWorkflowTableGroupInput, - createTableEnrichmentGroup, - createWorkflowTableGroup, - type UpdateWorkflowTableGroupInput, - updateWorkflowTableGroup, -} from '@/lib/table/application/groups' import { type ReplaceProjectedWireRowsInput, replaceProjectedWireRows, } from '@/lib/table/application/rows' -import { - type CreateTableFromWorkspaceFileInput, - createTableFromWorkspaceFile, - type ImportWorkspaceFileInput, - importWorkspaceFileIntoTable, -} from '@/lib/table/application/workspace-file-imports' const INHERITED_COPILOT_RATE_POLICY = { kind: 'inherited_copilot_request', @@ -35,18 +15,6 @@ const NO_DIRECT_PROVIDER_COST_POLICY = { reason: 'This command does not invoke a paid provider; table quota and storage limits apply.', } as const -export const copilotDeleteTablesPolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotDeleteTables( - context: CopilotTableDelegationContext | undefined, - input: DeleteCopilotTablesInput -) { - return executeCopilotTableUseCase(context, deleteCopilotTables, input) -} - export const copilotReplaceProjectedWireRowsPolicy = { rate: INHERITED_COPILOT_RATE_POLICY, cost: NO_DIRECT_PROVIDER_COST_POLICY, @@ -60,85 +28,3 @@ export function executeCopilotReplaceProjectedWireRows( tableId: input.tableId, }) } - -export const copilotCreateWorkflowTableGroupPolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotCreateWorkflowTableGroup( - context: CopilotTableDelegationContext | undefined, - input: CreateWorkflowTableGroupInput -) { - return executeCopilotTableUseCase(context, createWorkflowTableGroup, input, { - tableId: input.tableId, - }) -} - -export const copilotUpdateWorkflowTableGroupPolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotUpdateWorkflowTableGroup( - context: CopilotTableDelegationContext | undefined, - input: UpdateWorkflowTableGroupInput -) { - return executeCopilotTableUseCase(context, updateWorkflowTableGroup, input, { - tableId: input.tableId, - }) -} - -export const copilotAddWorkflowTableGroupOutputPolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotAddWorkflowTableGroupOutput( - context: CopilotTableDelegationContext | undefined, - input: AddTableGroupOutputInput -) { - return executeCopilotTableUseCase(context, addWorkflowTableGroupOutput, input, { - tableId: input.tableId, - }) -} - -export const copilotCreateTableEnrichmentGroupPolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotCreateTableEnrichmentGroup( - context: CopilotTableDelegationContext | undefined, - input: CreateTableEnrichmentGroupInput -) { - return executeCopilotTableUseCase(context, createTableEnrichmentGroup, input, { - tableId: input.tableId, - }) -} - -export const copilotCreateTableFromWorkspaceFilePolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotCreateTableFromWorkspaceFile( - context: CopilotTableDelegationContext | undefined, - input: CreateTableFromWorkspaceFileInput -) { - return executeCopilotTableUseCase(context, createTableFromWorkspaceFile, input) -} - -export const copilotImportWorkspaceFileIntoTablePolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - -export function executeCopilotImportWorkspaceFileIntoTable( - context: CopilotTableDelegationContext | undefined, - input: ImportWorkspaceFileInput -) { - return executeCopilotTableUseCase(context, importWorkspaceFileIntoTable, input, { - tableId: input.tableId, - }) -} diff --git a/apps/sim/lib/mothership/tools/handlers/param-types.ts b/apps/sim/lib/mothership/tools/handlers/param-types.ts index 5bcc622c416..c07fcb3b240 100644 --- a/apps/sim/lib/mothership/tools/handlers/param-types.ts +++ b/apps/sim/lib/mothership/tools/handlers/param-types.ts @@ -3,47 +3,6 @@ * Replaces Record with specific shapes based on actual property access. */ -import type { MothershipResourceType } from '@/lib/mothership/resources/types' - -// === Workflow Query Params === - -export interface GetWorkflowDataParams { - workflowId?: string - data_type?: string - dataType?: string -} - -export interface GetWorkflowRunOptionsParams { - workflowId?: string -} - -export interface GetBlockOutputsParams { - workflowId?: string - blockIds?: string[] -} - -export interface GetBlockUpstreamReferencesParams { - workflowId?: string - blockIds: string[] -} - -// === Workflow Mutation Params === - -export interface CreateWorkflowParams { - name?: string - workspaceId?: string - /** Canonical workflow-folder VFS path, for example `workflows/Dream`. */ - folderPath?: string - /** Legacy executor input. New tool calls use folderPath and resolve the ID internally. */ - folderId?: string -} - -export interface CreateFolderParams { - name?: string - workspaceId?: string - parentId?: string -} - export interface RunWorkflowParams { workflowId?: string workflow_input?: unknown @@ -115,224 +74,7 @@ export interface RunBlockParams { select?: string[] } -export interface GetDeployedWorkflowStateParams { - workflowId?: string -} - export interface GenerateApiKeyParams { name: string workspaceId?: string } - -export interface VariableOperation { - name: string - operation: 'add' | 'edit' | 'delete' - value?: unknown - type?: string -} - -export interface SetGlobalWorkflowVariablesParams { - workflowId?: string - operations?: VariableOperation[] -} - -export interface SetBlockEnabledParams { - workflowId?: string - blockId: string - enabled: boolean -} - -// === Deployment Params === - -export interface DeployApiParams { - workflowId?: string - action?: 'deploy' | 'undeploy' - /** Description of what changed in this deployment version. Required when action is 'deploy'. */ - versionDescription?: string - /** Short human-readable name/label for this deployment version. Required when action is 'deploy'. */ - versionName?: string -} - -export interface DeployChatParams { - workflowId?: string - action?: 'deploy' | 'undeploy' | 'update' - identifier?: string - title?: string - description?: string - /** Description of what changed in this deployment version (distinct from the chat-facing `description`). Required when action is 'deploy'. */ - versionDescription?: string - /** Short human-readable name/label for this deployment version. Required when action is 'deploy'. */ - versionName?: string - welcomeMessage?: string - customizations?: { - primaryColor?: string - secondaryColor?: string - welcomeMessage?: string - imageUrl?: string - /** @deprecated Prefer imageUrl for compatibility with chat deploy APIs. */ - iconUrl?: string - } - authType?: 'password' | 'public' | 'email' | 'sso' - password?: string - subdomain?: string - allowedEmails?: string[] - outputConfigs?: unknown[] - includeThinking?: boolean - includeToolCalls?: boolean -} - -export interface DeployMcpParams { - workflowId?: string - action?: 'deploy' | 'undeploy' - toolName?: string - toolDescription?: string - serverId?: string - /** - * Per-parameter descriptions as `[{ name, description }]`. Overlaid onto the - * workflow's input format before generating the tool schema — the same path - * the deploy modal uses. Parameter names/types/required come from the - * workflow's input trigger, not from this tool. - */ - parameterDescriptions?: Array<{ name: string; description: string }> -} - -export interface DeployCustomBlockParams { - workflowId?: string - action?: 'deploy' | 'undeploy' - /** Block display name (max 60 chars). Required on first publish. */ - name?: string - /** Block-picker description (max 280 chars). */ - description?: string - /** Icon image URL; omit for the organization's default icon. */ - iconUrl?: string - /** - * Per-input placeholder overrides keyed by the input trigger field's stable id. - * The field set itself is always derived from the workflow's deployment. - */ - inputs?: Array<{ id: string; placeholder?: string }> - /** Curated outputs; omit to expose the terminal block's whole result. */ - exposedOutputs?: Array<{ blockId: string; path: string; name: string }> -} - -export interface CheckDeploymentStatusParams { - workflowId?: string -} - -export interface UpdateDeploymentVersionParams { - workflowId?: string - version: number | string - /** New name/label for the version. Provide name and/or description. */ - name?: string - /** New description for the version. Provide name and/or description. */ - description?: string -} - -export interface GetDeploymentLogParams { - workflowId?: string -} - -export interface DiffWorkflowsParams { - workflowId?: string - /** Base/previous side: a version number, "live", or "draft". */ - ref1: number | string - /** Target/current side: a version number, "live", or "draft". */ - ref2: number | string -} - -export interface LoadDeploymentParams { - workflowId?: string - /** Version number to load, or "live" for the active deployment. */ - version: number | string -} - -export interface PromoteToLiveParams { - workflowId?: string - /** Version number to promote to live. */ - version: number -} - -export interface ListWorkspaceMcpServersParams { - workspaceId?: string -} - -export interface CreateWorkspaceMcpServerParams { - workspaceId?: string - name?: string - description?: string - isPublic?: boolean - workflowIds?: string[] -} - -// === Workflow Organization Params === - -export interface RenameWorkflowParams { - workflowId: string - name: string -} - -export interface DeleteWorkflowParams { - workflowIds: string[] -} - -export interface MoveWorkflowParams { - workflowIds: string[] - folderId: string | null -} - -export interface MoveFolderParams { - folderId: string - parentId: string | null -} - -export interface RenameFolderParams { - folderId: string - name: string -} - -export interface DeleteFolderParams { - folderIds: string[] -} - -export interface ManageFolderParams { - operation: string - path?: string - folderId?: string - name?: string - destinationPath?: string - parentId?: string | null -} - -export interface UpdateWorkspaceMcpServerParams { - serverId: string - name?: string - description?: string - isPublic?: boolean -} - -export interface DeleteWorkspaceMcpServerParams { - serverId: string -} - -export type OpenResourceType = MothershipResourceType - -export interface OpenResourceItem { - type?: OpenResourceType - id?: string - path?: string - /** Saved-view id or exact name to open a table pinned to (table type only). */ - view?: string -} - -export interface OpenResourceParams { - resources?: OpenResourceItem[] - type?: OpenResourceType - id?: string - path?: string -} - -export interface ValidOpenResourceParams { - type: OpenResourceType - id?: string - path?: string - view?: string -} diff --git a/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts b/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts index 76a66aaccd9..9136c27e3fa 100644 --- a/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts +++ b/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts @@ -63,7 +63,6 @@ vi.mock('@/lib/mothership/tools/server/router', () => ({ getRegisteredServerTool import { executeCancelWorkflowRun, - executeCreateWorkflow, executeRunBlock, executeRunWorkflow, } from '@/lib/mothership/tools/handlers/workflow/mutations' @@ -83,44 +82,6 @@ describe('workflow mutation Copilot adapters', () => { mocks.readAttemptedExecutionId.mockReturnValue(undefined) }) - it('maps encoded folder aliases into one create application command', async () => { - mocks.executeWorkflowUseCase.mockResolvedValue({ - workflow: { - id: 'workflow-new', - name: 'New Workflow', - workspaceId: 'workspace-1', - folderId: 'folder-1', - }, - normalizedState: { blocks: {}, edges: [], loops: {}, parallels: {} }, - }) - - const result = await executeCreateWorkflow( - { name: ' New Workflow ', folderPath: 'workflows/Launch%20Plans' }, - context - ) - - expect(result.success).toBe(true) - expect(mocks.executeWorkflowUseCase).toHaveBeenCalledWith( - context, - expect.objectContaining({ operation: expect.objectContaining({ id: 'workflows.create' }) }), - { - workspaceId: 'workspace-1', - name: 'New Workflow', - folderPath: '/Launch%20Plans', - } - ) - }) - - it('rejects a create-workflow workspaceId that names a different workspace', async () => { - const result = await executeCreateWorkflow( - { name: 'New Workflow', workspaceId: 'workspace-other' }, - context - ) - - expect(result.success).toBe(false) - expect(mocks.executeWorkflowUseCase).not.toHaveBeenCalled() - }) - it('projects one run command result without exposing binary payloads', async () => { mocks.executeWorkflowUseCase.mockResolvedValue({ success: true, diff --git a/apps/sim/lib/mothership/tools/handlers/workflow/mutations.ts b/apps/sim/lib/mothership/tools/handlers/workflow/mutations.ts index bc8bcc88bb9..6c2727d78f8 100644 --- a/apps/sim/lib/mothership/tools/handlers/workflow/mutations.ts +++ b/apps/sim/lib/mothership/tools/handlers/workflow/mutations.ts @@ -16,42 +16,26 @@ import { } from '@/lib/mothership/tool-executor/types' import type { CancelWorkflowRunParams, - CreateWorkflowParams, GenerateApiKeyParams, - MoveWorkflowParams, - RenameWorkflowParams, RunBlockParams, RunFromBlockParams, RunWorkflowParams, RunWorkflowUntilBlockParams, - SetBlockEnabledParams, - SetGlobalWorkflowVariablesParams, - VariableOperation, } from '@/lib/mothership/tools/handlers/param-types' import { requireCopilotWorkspace } from '@/lib/mothership/tools/server/workspace-scope' import { boundRunResultForModel, presentWorkflowLogsForModel, } from '@/lib/mothership/tools/workflow-output' -import { decodeVfsPathSegments, encodeVfsPathSegments } from '@/lib/mothership/vfs/path-utils' import { cancelWorkflowRun } from '@/lib/workflows/application/cancel-run' -import { createWorkflow } from '@/lib/workflows/application/create-workflow' -import { moveWorkflowsBulk } from '@/lib/workflows/application/move-workflows-bulk' import { runBlockFromCopilot, runFromBlockFromCopilot, runWorkflowFromCopilot, runWorkflowUntilBlockFromCopilot, } from '@/lib/workflows/application/run-workflow-from-copilot' -import { updateWorkflow } from '@/lib/workflows/application/update-workflow' -import { - applyWorkflowVariableOperations, - setWorkflowBlockEnabled, -} from '@/lib/workflows/application/update-workflow-content' -import { sanitizeForCopilot } from '@/lib/workflows/sanitization/json-sanitizer' import { hasExecutionResult, readAttemptedExecutionId } from '@/executor/utils/errors' import type { ResolvedSecretTraceRegistry } from '@/executor/utils/resolved-secret-trace-registry' -import type { WorkflowState } from '@/stores/workflows/workflow/types' const logger = createLogger('WorkflowMutations') @@ -277,63 +261,6 @@ function assertWorkflowMutationNotAborted( } } -export async function executeCreateWorkflow( - params: CreateWorkflowParams, - context: ExecutionContext -): Promise { - try { - const name = typeof params?.name === 'string' ? params.name.trim() : '' - if (!name) { - return { success: false, error: 'name is required' } - } - if (name.length > 200) { - return { success: false, error: 'Workflow name must be 200 characters or less' } - } - const workspaceId = requireCopilotWorkspace(context, params?.workspaceId) - - const folderPath = typeof params?.folderPath === 'string' ? params.folderPath.trim() : '' - const folderId = - typeof params?.folderId === 'string' && params.folderId.trim() ? params.folderId.trim() : null - let canonicalFolderPath: string | undefined - if (folderPath) { - const relativePath = workflowFolderRelativePath(folderPath) - canonicalFolderPath = relativePath - ? `/${encodeVfsPathSegments(decodeVfsPathSegments(relativePath))}` - : '/' - } - - assertWorkflowMutationNotAborted(context) - - const result = await executeCopilotWorkflowUseCase(context, createWorkflow, { - workspaceId, - name, - ...(canonicalFolderPath !== undefined ? { folderPath: canonicalFolderPath } : { folderId }), - }) - const copilotSanitizedWorkflowState = sanitizeForCopilot({ - blocks: result.normalizedState.blocks || {}, - edges: result.normalizedState.edges || [], - loops: result.normalizedState.loops || {}, - parallels: result.normalizedState.parallels || {}, - } as WorkflowState) - - return { - success: true, - output: { - workflowId: result.workflow.id, - workflowName: result.workflow.name, - workspaceId: result.workflow.workspaceId, - folderId: result.workflow.folderId, - ...(copilotSanitizedWorkflowState ? { copilotSanitizedWorkflowState } : {}), - }, - } - } catch (error) { - return { - success: false, - error: messageForCopilotWorkflowError(error, 'Failed to create workflow'), - } - } -} - export async function executeRunWorkflow( params: RunWorkflowParams, context: ExecutionContext @@ -409,94 +336,6 @@ export async function executeCancelWorkflowRun( } } -export async function executeSetGlobalWorkflowVariables( - params: SetGlobalWorkflowVariablesParams, - context: ExecutionContext -): Promise { - try { - const workflowId = params.workflowId || context.workflowId - if (!workflowId) { - return { success: false, error: 'workflowId is required' } - } - const operations: VariableOperation[] = Array.isArray(params.operations) - ? params.operations - : [] - - assertWorkflowMutationNotAborted(context) - const result = await executeCopilotWorkflowUseCase(context, applyWorkflowVariableOperations, { - workflowId, - assertedWorkspaceId: context.workspaceId, - operations, - }) - - return { success: true, output: { updated: result.updated } } - } catch (error) { - return { success: false, error: messageForCopilotWorkflowError(error) } - } -} - -export async function executeRenameWorkflow( - params: RenameWorkflowParams, - context: ExecutionContext -): Promise { - try { - const workflowId = params.workflowId - if (!workflowId) { - return { success: false, error: 'workflowId is required' } - } - const name = typeof params.name === 'string' ? params.name.trim() : '' - if (!name) { - return { success: false, error: 'name is required' } - } - if (name.length > 200) { - return { success: false, error: 'Workflow name must be 200 characters or less' } - } - - assertWorkflowMutationNotAborted(context) - await executeCopilotWorkflowUseCase(context, updateWorkflow, { - workflowId, - assertedWorkspaceId: context.workspaceId, - name, - }) - - return { success: true, output: { workflowId, name } } - } catch (error) { - return { - success: false, - error: messageForCopilotWorkflowError(error, 'Failed to rename workflow'), - } - } -} - -export async function executeMoveWorkflow( - params: MoveWorkflowParams, - context: ExecutionContext -): Promise { - try { - const workflowIds = params.workflowIds - if (!workflowIds || workflowIds.length === 0) { - return { success: false, error: 'workflowIds is required' } - } - if (!context.workspaceId) { - return { success: false, error: 'Workspace context is required' } - } - - assertWorkflowMutationNotAborted(context) - const result = await executeCopilotWorkflowUseCase(context, moveWorkflowsBulk, { - workspaceId: context.workspaceId, - workflowIds, - folderId: params.folderId || null, - }) - - return { - success: result.moved.length > 0, - output: { moved: result.moved, failed: result.failed, folderId: result.folderId }, - } - } catch (error) { - return { success: false, error: messageForCopilotWorkflowError(error) } - } -} - export async function executeRunWorkflowUntilBlock( params: RunWorkflowUntilBlockParams, context: ExecutionContext @@ -619,62 +458,6 @@ export async function executeRunFromBlock( } } -export async function executeSetBlockEnabled( - params: SetBlockEnabledParams, - context: ExecutionContext -): Promise { - try { - const workflowId = params.workflowId || context.workflowId - if (!workflowId) { - return { success: false, error: 'workflowId is required' } - } - if (!params.blockId) { - return { success: false, error: 'blockId is required' } - } - if (typeof params.enabled !== 'boolean') { - return { success: false, error: 'enabled must be a boolean' } - } - - assertWorkflowMutationNotAborted(context) - const result = await executeCopilotWorkflowUseCase(context, setWorkflowBlockEnabled, { - workflowId, - assertedWorkspaceId: context.workspaceId, - blockId: params.blockId, - enabled: params.enabled, - }) - - return { - success: true, - output: { - workflowId, - workflowName: result.workflowName, - blockId: params.blockId, - enabled: params.enabled, - affectedBlockIds: result.affectedBlockIds, - copilotSanitizedWorkflowState: sanitizeForCopilot(result.state), - ...(!result.changed - ? { - message: `Block ${params.blockId} is already ${params.enabled ? 'enabled' : 'disabled'}`, - } - : {}), - }, - } - } catch (error) { - return { success: false, error: messageForCopilotWorkflowError(error) } - } -} - -/** - * Strip the `workflows/` VFS prefix from a folder path, returning the - * folder-relative remainder. `workflows` (or an empty path) maps to the - * workspace root and yields an empty string. - */ -function workflowFolderRelativePath(rawPath: string): string { - const trimmed = rawPath.trim().replace(/^\/+|\/+$/g, '') - if (!trimmed || trimmed === 'workflows') return '' - return trimmed.startsWith('workflows/') ? trimmed.slice('workflows/'.length) : trimmed -} - export async function executeRunBlock( params: RunBlockParams, context: ExecutionContext From 9e779f451f190fba1eec7560f54f384ee6ce608c Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:15:48 -0700 Subject: [PATCH 12/30] chore(cleanup): share webhook provider config, notification URL, and credential token helpers --- apps/sim/app/api/webhooks/route.ts | 11 ++---- apps/sim/lib/webhooks/deploy.ts | 3 +- .../webhooks/provider-subscription-utils.ts | 19 +++++++++- .../lib/webhooks/provider-subscriptions.ts | 4 +- apps/sim/lib/webhooks/providers/airtable.ts | 35 +++++------------- apps/sim/lib/webhooks/providers/attio.ts | 34 ++++++----------- .../lib/webhooks/providers/azure-devops.ts | 3 +- .../lib/webhooks/providers/bitbucket.test.ts | 15 +++----- apps/sim/lib/webhooks/providers/bitbucket.ts | 8 +--- .../lib/webhooks/providers/clickup.test.ts | 13 +++---- apps/sim/lib/webhooks/providers/clickup.ts | 20 +++------- apps/sim/lib/webhooks/providers/confluence.ts | 3 +- apps/sim/lib/webhooks/providers/gmail.ts | 3 +- .../lib/webhooks/providers/google-forms.ts | 3 +- apps/sim/lib/webhooks/providers/imap.ts | 3 +- apps/sim/lib/webhooks/providers/jira.ts | 2 +- apps/sim/lib/webhooks/providers/jsm.ts | 3 +- .../lib/webhooks/providers/microsoft-teams.ts | 24 +++--------- apps/sim/lib/webhooks/providers/monday.ts | 26 +++---------- apps/sim/lib/webhooks/providers/outlook.ts | 3 +- apps/sim/lib/webhooks/providers/rss.ts | 3 +- apps/sim/lib/webhooks/providers/slack.ts | 2 +- apps/sim/lib/webhooks/providers/typeform.ts | 3 +- apps/sim/lib/webhooks/providers/webflow.ts | 37 ++++++------------- apps/sim/lib/webhooks/providers/whatsapp.ts | 4 +- apps/sim/lib/webhooks/providers/zoom.ts | 6 +-- apps/sim/lib/webhooks/slack-dispatch.ts | 3 +- .../workflows/sanitization/json-sanitizer.ts | 11 ++++-- 28 files changed, 118 insertions(+), 186 deletions(-) diff --git a/apps/sim/app/api/webhooks/route.ts b/apps/sim/app/api/webhooks/route.ts index eb3942c7262..c41fea88902 100644 --- a/apps/sim/app/api/webhooks/route.ts +++ b/apps/sim/app/api/webhooks/route.ts @@ -9,7 +9,7 @@ import { } from '@sim/platform-authz/workflow' import { getErrorMessage } from '@sim/utils/errors' import { generateId, generateShortId } from '@sim/utils/id' -import { omit } from '@sim/utils/object' +import { omit, toRecord } from '@sim/utils/object' import { and, desc, eq, inArray, isNull, or } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import { listWebhooksContract, upsertWebhookContract } from '@/lib/api/contracts/webhooks' @@ -454,8 +454,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => { shouldRecreateExternalWebhookSubscription({ previousProvider: existingWebhook.provider as string, nextProvider: provider, - previousConfig: ((existingWebhook.providerConfig as Record) || - {}) as Record, + previousConfig: toRecord(existingWebhook.providerConfig), nextConfig: resolvedProviderConfig, }) @@ -525,11 +524,7 @@ export const POST = withRouteHandler(async (request: NextRequest) => { ) } } else { - mergeNonUserFields( - configToSave, - (existingWebhook.providerConfig as Record) || {}, - userProvided - ) + mergeNonUserFields(configToSave, toRecord(existingWebhook.providerConfig), userProvided) } configToSave.userId = undefined diff --git a/apps/sim/lib/webhooks/deploy.ts b/apps/sim/lib/webhooks/deploy.ts index 4efd3d894d8..e3456e48eba 100644 --- a/apps/sim/lib/webhooks/deploy.ts +++ b/apps/sim/lib/webhooks/deploy.ts @@ -3,6 +3,7 @@ import { account, credential, webhook, workflowDeploymentVersion } from '@sim/db import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { generateShortId } from '@sim/utils/id' +import { toRecord } from '@sim/utils/object' import { and, asc, eq, inArray, isNull, ne, or } from 'drizzle-orm' import type { NextRequest } from 'next/server' import { isSlackExtendedScopesEnabled } from '@/lib/core/config/env-flags' @@ -956,7 +957,7 @@ export async function saveTriggerWebhooksForDeploy({ } // Check if config changed or if we're forcing recreation (e.g., activating old version) - const existingConfig = (existingWh.providerConfig as Record) || {} + const existingConfig = toRecord(existingWh.providerConfig) const needsRecreation = forceRecreateSubscriptions || existingWh.provider !== provider || diff --git a/apps/sim/lib/webhooks/provider-subscription-utils.ts b/apps/sim/lib/webhooks/provider-subscription-utils.ts index b6f02f880ff..c26047a90c6 100644 --- a/apps/sim/lib/webhooks/provider-subscription-utils.ts +++ b/apps/sim/lib/webhooks/provider-subscription-utils.ts @@ -1,15 +1,16 @@ import { db } from '@sim/db' import { account } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import { getBaseUrl } from '@/lib/core/utils/urls' -import { resolveOAuthAccountId } from '@/lib/oauth/credential-service' +import { refreshAccessTokenIfNeeded, resolveOAuthAccountId } from '@/lib/oauth/credential-service' const logger = createLogger('WebhookProviderSubscriptions') /** Safely read a webhook row's provider config as a plain object. */ export function getProviderConfig(webhook: Record): Record { - return (webhook.providerConfig as Record) || {} + return toRecord(webhook.providerConfig) } /** Build the public callback URL providers should deliver webhook events to. */ @@ -46,3 +47,17 @@ export async function getCredentialOwner( return { userId: credentialRecord.userId, accountId: resolved.accountId } } + +/** + * Resolve an OAuth-backed credential to a fresh access token for its owner. + * + * Returns null when the credential's owner cannot be resolved or no token is + * available, leaving the caller to decide whether that is fatal. + */ +export async function getCredentialAccessToken( + credentialId: string, + requestId: string +): Promise { + const owner = await getCredentialOwner(credentialId, requestId) + return owner ? refreshAccessTokenIfNeeded(owner.accountId, owner.userId, requestId) : null +} diff --git a/apps/sim/lib/webhooks/provider-subscriptions.ts b/apps/sim/lib/webhooks/provider-subscriptions.ts index ca4c0e6e5e4..91360774825 100644 --- a/apps/sim/lib/webhooks/provider-subscriptions.ts +++ b/apps/sim/lib/webhooks/provider-subscriptions.ts @@ -1,6 +1,6 @@ import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' -import { omit } from '@sim/utils/object' +import { omit, toRecord } from '@sim/utils/object' import type { NextRequest } from 'next/server' import { withResourceOutboundScope } from '@/lib/core/network/resource-scope.server' import { isSensitiveKey } from '@/lib/core/security/redaction' @@ -174,7 +174,7 @@ export async function createExternalWebhookSubscription( options: { signal?: AbortSignal } = {} ): Promise { const provider = webhookData.provider as string - const providerConfig = (webhookData.providerConfig as Record) || {} + const providerConfig = toRecord(webhookData.providerConfig) const handler = getProviderHandler(provider) if (!handler.createSubscription) { diff --git a/apps/sim/lib/webhooks/providers/airtable.ts b/apps/sim/lib/webhooks/providers/airtable.ts index 99b77074cf6..39463c56dc4 100644 --- a/apps/sim/lib/webhooks/providers/airtable.ts +++ b/apps/sim/lib/webhooks/providers/airtable.ts @@ -1,16 +1,16 @@ import { db } from '@sim/db' import { account, webhook } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import { validateAirtableId } from '@/lib/core/security/input-validation' -import { getBaseUrl } from '@/lib/core/utils/urls' import { getOAuthToken, refreshAccessTokenIfNeeded, resolveOAuthAccountId, } from '@/lib/oauth/credential-service' import { - getCredentialOwner, + getCredentialAccessToken, getNotificationUrl, getProviderConfig, } from '@/lib/webhooks/provider-subscription-utils' @@ -61,9 +61,9 @@ async function fetchAndProcessAirtablePayloads( const consolidatedChangesMap = new Map() // Capture raw payloads from Airtable for exposure to workflows const allPayloads = [] - const localProviderConfig = { - ...((webhookData.providerConfig as Record) || {}), - } as Record + const localProviderConfig: Record = { + ...toRecord(webhookData.providerConfig), + } try { const baseId = localProviderConfig.baseId @@ -445,8 +445,7 @@ export const airtableHandler: WebhookProviderHandler = { requestId, }: SubscriptionContext): Promise { try { - const { path, providerConfig } = webhookRecord as Record - const config = (providerConfig as Record) || {} + const config = getProviderConfig(webhookRecord) const { baseId, tableId, includeCellValuesInFieldIds, credentialId } = config as { baseId?: string tableId?: string @@ -473,17 +472,8 @@ export const airtableHandler: WebhookProviderHandler = { throw new Error(tableIdValidation.error) } - const credentialOwner = credentialId - ? await getCredentialOwner(credentialId, requestId) - : null const accessToken = credentialId - ? credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + ? await getCredentialAccessToken(credentialId, requestId) : await getOAuthToken(userId, 'airtable') if (!accessToken) { logger.warn( @@ -494,7 +484,7 @@ export const airtableHandler: WebhookProviderHandler = { ) } - const notificationUrl = `${getBaseUrl()}/api/webhooks/trigger/${path}` + const notificationUrl = getNotificationUrl(webhookRecord) const airtableApiUrl = `https://api.airtable.com/v0/bases/${baseId}/webhooks` @@ -608,14 +598,7 @@ export const airtableHandler: WebhookProviderHandler = { return } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { const message = `[${requestId}] Could not retrieve Airtable access token. Cannot delete webhook in Airtable.` logger.warn(message, { webhookId: webhookRecord.id }) diff --git a/apps/sim/lib/webhooks/providers/attio.ts b/apps/sim/lib/webhooks/providers/attio.ts index 693e7e1f5e6..f01f03748c5 100644 --- a/apps/sim/lib/webhooks/providers/attio.ts +++ b/apps/sim/lib/webhooks/providers/attio.ts @@ -2,10 +2,13 @@ import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { hmacSha256Hex } from '@sim/security/hmac' import { toError } from '@sim/utils/errors' +import { toRecord } from '@sim/utils/object' import { NextResponse } from 'next/server' -import { getBaseUrl } from '@/lib/core/utils/urls' -import { refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service' -import { getCredentialOwner, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils' +import { + getCredentialAccessToken, + getNotificationUrl, + getProviderConfig, +} from '@/lib/webhooks/provider-subscription-utils' import type { AuthContext, DeleteSubscriptionContext, @@ -104,8 +107,7 @@ export const attioHandler: WebhookProviderHandler = { requestId, }: SubscriptionContext): Promise { try { - const { path, providerConfig } = webhookRecord as Record - const config = (providerConfig as Record) || {} + const config = getProviderConfig(webhookRecord) const { triggerId, credentialId } = config as { triggerId?: string credentialId?: string @@ -120,14 +122,7 @@ export const attioHandler: WebhookProviderHandler = { ) } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { logger.warn( @@ -138,7 +133,7 @@ export const attioHandler: WebhookProviderHandler = { ) } - const notificationUrl = `${getBaseUrl()}/api/webhooks/trigger/${path}` + const notificationUrl = getNotificationUrl(webhookRecord) const { TRIGGER_EVENT_MAP } = await import('@/triggers/attio/utils') @@ -264,14 +259,7 @@ export const attioHandler: WebhookProviderHandler = { return } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { const message = `[${requestId}] Could not retrieve Attio access token. Cannot delete webhook.` @@ -319,7 +307,7 @@ export const attioHandler: WebhookProviderHandler = { } = await import('@/triggers/attio/utils') const b = body as Record - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined if (triggerId === 'attio_record_updated') { diff --git a/apps/sim/lib/webhooks/providers/azure-devops.ts b/apps/sim/lib/webhooks/providers/azure-devops.ts index 27f5b95403b..557443344c2 100644 --- a/apps/sim/lib/webhooks/providers/azure-devops.ts +++ b/apps/sim/lib/webhooks/providers/azure-devops.ts @@ -1,4 +1,5 @@ import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import type { EventMatchContext, FormatInputContext, @@ -60,7 +61,7 @@ export const azureDevOpsHandler: WebhookProviderHandler = { async formatInput({ body, webhook, requestId }: FormatInputContext): Promise { const b = body as Record - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined const eventType = b.eventType as string | undefined diff --git a/apps/sim/lib/webhooks/providers/bitbucket.test.ts b/apps/sim/lib/webhooks/providers/bitbucket.test.ts index 6ffc5ddf007..7da611d8c23 100644 --- a/apps/sim/lib/webhooks/providers/bitbucket.test.ts +++ b/apps/sim/lib/webhooks/providers/bitbucket.test.ts @@ -1,10 +1,10 @@ import { jsonResponse } from '@sim/testing/helpers/http' -import { authOAuthUtilsMock, authOAuthUtilsMockFns } from '@sim/testing/mocks/auth-oauth-utils.mock' +import { authOAuthUtilsMock } from '@sim/testing/mocks/auth-oauth-utils.mock' import { NextRequest } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockGetCredentialOwner } = vi.hoisted(() => ({ - mockGetCredentialOwner: vi.fn(), +const { mockGetCredentialAccessToken } = vi.hoisted(() => ({ + mockGetCredentialAccessToken: vi.fn(), })) vi.mock('@/lib/oauth/credential-service', () => authOAuthUtilsMock) @@ -13,7 +13,7 @@ vi.mock('@/lib/webhooks/provider-subscription-utils', () => ({ getProviderConfig: (webhook: { providerConfig?: Record }) => webhook.providerConfig || {}, getNotificationUrl: () => 'https://app.example.com/api/webhooks/trigger/bitbucket-path', - getCredentialOwner: mockGetCredentialOwner, + getCredentialAccessToken: mockGetCredentialAccessToken, })) import { IdempotencyService } from '@/lib/core/idempotency/service' @@ -24,8 +24,6 @@ import { buildBitbucketOutputs, } from '@/triggers/bitbucket/utils' -const mockRefreshAccessTokenIfNeeded = authOAuthUtilsMockFns.mockRefreshAccessTokenIfNeeded - const fetchMock = vi.fn() const CALLBACK_URL = 'https://app.example.com/api/webhooks/trigger/bitbucket-path' const CANDIDATE_DESCRIPTION = 'Sim workflow trigger (bitbucket_push) [sim:webhook-1]' @@ -103,8 +101,7 @@ describe('Bitbucket webhook provider', () => { beforeEach(() => { fetchMock.mockReset() vi.stubGlobal('fetch', fetchMock) - mockGetCredentialOwner.mockResolvedValue({ accountId: 'account-1', userId: 'user-1' }) - mockRefreshAccessTokenIfNeeded.mockResolvedValue('oauth-token') + mockGetCredentialAccessToken.mockResolvedValue('oauth-token') }) describe('verifyAuth', () => { @@ -333,7 +330,7 @@ describe('Bitbucket webhook provider', () => { ).rejects.toThrow(/webhook ID is required/i) expect(fetchMock).not.toHaveBeenCalled() - expect(mockRefreshAccessTokenIfNeeded).not.toHaveBeenCalled() + expect(mockGetCredentialAccessToken).not.toHaveBeenCalled() } ) diff --git a/apps/sim/lib/webhooks/providers/bitbucket.ts b/apps/sim/lib/webhooks/providers/bitbucket.ts index c2664f94544..2f61d11825c 100644 --- a/apps/sim/lib/webhooks/providers/bitbucket.ts +++ b/apps/sim/lib/webhooks/providers/bitbucket.ts @@ -6,9 +6,8 @@ import { getErrorMessage, toError } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' import { toRecord, toRecordOrNull } from '@sim/utils/object' import { truncate } from '@sim/utils/string' -import { refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service' import { - getCredentialOwner, + getCredentialAccessToken, getNotificationUrl, getProviderConfig, } from '@/lib/webhooks/provider-subscription-utils' @@ -95,10 +94,7 @@ async function resolveBitbucketAccessToken( ) } - const owner = await getCredentialOwner(credentialId, requestId) - const accessToken = owner - ? await refreshAccessTokenIfNeeded(owner.accountId, owner.userId, requestId) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { throw new Error( diff --git a/apps/sim/lib/webhooks/providers/clickup.test.ts b/apps/sim/lib/webhooks/providers/clickup.test.ts index db904025797..9e06561356a 100644 --- a/apps/sim/lib/webhooks/providers/clickup.test.ts +++ b/apps/sim/lib/webhooks/providers/clickup.test.ts @@ -1,26 +1,24 @@ import { hmacSha256Hex } from '@sim/security/hmac' import { jsonResponse } from '@sim/testing/helpers/http' -import { authOAuthUtilsMock, authOAuthUtilsMockFns } from '@sim/testing/mocks/auth-oauth-utils.mock' +import { authOAuthUtilsMock } from '@sim/testing/mocks/auth-oauth-utils.mock' import { NextRequest, NextResponse } from 'next/server' import { beforeEach, describe, expect, it, vi } from 'vitest' -const { mockGetCredentialOwner } = vi.hoisted(() => ({ - mockGetCredentialOwner: vi.fn(), +const { mockGetCredentialAccessToken } = vi.hoisted(() => ({ + mockGetCredentialAccessToken: vi.fn(), })) vi.mock('@/lib/webhooks/provider-subscription-utils', () => ({ getProviderConfig: (webhook: { providerConfig?: Record }) => webhook.providerConfig || {}, getNotificationUrl: () => 'https://app.example.com/api/webhooks/trigger/clickup-path', - getCredentialOwner: mockGetCredentialOwner, + getCredentialAccessToken: mockGetCredentialAccessToken, })) vi.mock('@/lib/oauth/credential-service', () => authOAuthUtilsMock) import { clickupHandler } from '@/lib/webhooks/providers/clickup' -const mockRefreshAccessTokenIfNeeded = authOAuthUtilsMockFns.mockRefreshAccessTokenIfNeeded - const fetchMock = vi.fn() function reqWithHeaders(headers: Record): NextRequest { @@ -39,8 +37,7 @@ function createContext(providerConfig: Record) { describe('ClickUp webhook provider', () => { beforeEach(() => { vi.stubGlobal('fetch', fetchMock) - mockGetCredentialOwner.mockResolvedValue({ userId: 'user-1', accountId: 'account-1' }) - mockRefreshAccessTokenIfNeeded.mockResolvedValue('oauth-token') + mockGetCredentialAccessToken.mockResolvedValue('oauth-token') }) describe('verifyAuth', () => { diff --git a/apps/sim/lib/webhooks/providers/clickup.ts b/apps/sim/lib/webhooks/providers/clickup.ts index 2ca160112a6..1f97de6ee3f 100644 --- a/apps/sim/lib/webhooks/providers/clickup.ts +++ b/apps/sim/lib/webhooks/providers/clickup.ts @@ -2,10 +2,10 @@ import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { hmacSha256Hex } from '@sim/security/hmac' import { toError } from '@sim/utils/errors' +import { toRecord } from '@sim/utils/object' import { NextResponse } from 'next/server' -import { refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service' import { - getCredentialOwner, + getCredentialAccessToken, getNotificationUrl, getProviderConfig, } from '@/lib/webhooks/provider-subscription-utils' @@ -51,10 +51,7 @@ async function resolveClickUpAccessToken( ) } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded(credentialOwner.accountId, credentialOwner.userId, requestId) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { throw new Error( @@ -309,14 +306,7 @@ export const clickupHandler: WebhookProviderHandler = { return } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { const message = `[${requestId}] Could not retrieve ClickUp access token. Cannot delete webhook.` @@ -354,7 +344,7 @@ export const clickupHandler: WebhookProviderHandler = { } = await import('@/triggers/clickup/utils') const b = body as Record - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined if (triggerId?.startsWith('clickup_task_')) { diff --git a/apps/sim/lib/webhooks/providers/confluence.ts b/apps/sim/lib/webhooks/providers/confluence.ts index cfb35bf920f..f828555efcd 100644 --- a/apps/sim/lib/webhooks/providers/confluence.ts +++ b/apps/sim/lib/webhooks/providers/confluence.ts @@ -1,4 +1,5 @@ import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { NextResponse } from 'next/server' import { validateJiraSignature } from '@/lib/webhooks/providers/jira' import type { @@ -30,7 +31,7 @@ export const confluenceHandler: WebhookProviderHandler = { extractPagePermissionsData, extractUserData, } = await import('@/triggers/confluence/utils') - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined if (triggerId?.startsWith('confluence_comment_')) { return { input: extractCommentData(body) } diff --git a/apps/sim/lib/webhooks/providers/gmail.ts b/apps/sim/lib/webhooks/providers/gmail.ts index 4abc2caa6fb..11cbc50e21b 100644 --- a/apps/sim/lib/webhooks/providers/gmail.ts +++ b/apps/sim/lib/webhooks/providers/gmail.ts @@ -1,6 +1,7 @@ import { db } from '@sim/db' import { account, webhook } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import { refreshAccessTokenIfNeeded, resolveOAuthAccountId } from '@/lib/oauth/credential-service' import type { @@ -29,7 +30,7 @@ export const gmailHandler: WebhookProviderHandler = { logger.info(`[${requestId}] Setting up Gmail polling for webhook ${webhookData.id}`) try { - const providerConfig = (webhookData.providerConfig as Record) || {} + const providerConfig = toRecord(webhookData.providerConfig) const credentialId = providerConfig.credentialId as string | undefined if (!credentialId) { diff --git a/apps/sim/lib/webhooks/providers/google-forms.ts b/apps/sim/lib/webhooks/providers/google-forms.ts index ef943e7d408..bbb6c13cefd 100644 --- a/apps/sim/lib/webhooks/providers/google-forms.ts +++ b/apps/sim/lib/webhooks/providers/google-forms.ts @@ -1,4 +1,5 @@ import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { NextResponse } from 'next/server' import type { AuthContext, @@ -13,7 +14,7 @@ const logger = createLogger('WebhookProvider:GoogleForms') export const googleFormsHandler: WebhookProviderHandler = { async formatInput({ body, webhook }: FormatInputContext): Promise { const b = body as Record - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const normalizeAnswers = (src: unknown): Record => { if (!src || typeof src !== 'object') return {} const out: Record = {} diff --git a/apps/sim/lib/webhooks/providers/imap.ts b/apps/sim/lib/webhooks/providers/imap.ts index c4f3e12a158..722bfa145b4 100644 --- a/apps/sim/lib/webhooks/providers/imap.ts +++ b/apps/sim/lib/webhooks/providers/imap.ts @@ -1,6 +1,7 @@ import { db } from '@sim/db' import { webhook, workflowDeploymentVersion } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import { createSecureImapClient, @@ -53,7 +54,7 @@ export const imapHandler: WebhookProviderHandler = { logger.info(`[${requestId}] Setting up IMAP polling for webhook ${webhookData.id}`) try { - const providerConfig = (webhookData.providerConfig as Record) || {} + const providerConfig = toRecord(webhookData.providerConfig) const now = new Date() if (!providerConfig.host || !providerConfig.username || !providerConfig.password) { diff --git a/apps/sim/lib/webhooks/providers/jira.ts b/apps/sim/lib/webhooks/providers/jira.ts index 4bcabf77652..fb9675f0d8e 100644 --- a/apps/sim/lib/webhooks/providers/jira.ts +++ b/apps/sim/lib/webhooks/providers/jira.ts @@ -79,7 +79,7 @@ export const jiraHandler: WebhookProviderHandler = { extractProjectData, extractVersionData, } = await import('@/triggers/jira/utils') - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined if ( diff --git a/apps/sim/lib/webhooks/providers/jsm.ts b/apps/sim/lib/webhooks/providers/jsm.ts index 9d1a785f904..c3cbc651525 100644 --- a/apps/sim/lib/webhooks/providers/jsm.ts +++ b/apps/sim/lib/webhooks/providers/jsm.ts @@ -1,4 +1,5 @@ import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { validateJiraSignature } from '@/lib/webhooks/providers/jira' import type { EventMatchContext, @@ -27,7 +28,7 @@ export const jsmHandler: WebhookProviderHandler = { async formatInput({ body, webhook }: FormatInputContext): Promise { const { extractRequestData, extractCommentData } = await import('@/triggers/jsm/utils') - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined if (triggerId === 'jsm_request_commented') { diff --git a/apps/sim/lib/webhooks/providers/microsoft-teams.ts b/apps/sim/lib/webhooks/providers/microsoft-teams.ts index 98dfe9830a2..5af7b2c95cb 100644 --- a/apps/sim/lib/webhooks/providers/microsoft-teams.ts +++ b/apps/sim/lib/webhooks/providers/microsoft-teams.ts @@ -4,7 +4,7 @@ import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { hmacSha256Base64 } from '@sim/security/hmac' import { getErrorMessage, toError } from '@sim/utils/errors' -import { isRecordLike } from '@sim/utils/object' +import { isRecordLike, toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import { isMicrosoftContentUrl } from '@/lib/core/security/input-validation' @@ -16,7 +16,7 @@ import { import { sanitizeUrlForLog } from '@/lib/core/utils/logging' import { refreshAccessTokenIfNeeded, resolveOAuthAccountId } from '@/lib/oauth/credential-service' import { - getCredentialOwner, + getCredentialAccessToken, getNotificationUrl, getProviderConfig, } from '@/lib/webhooks/provider-subscription-utils' @@ -197,7 +197,7 @@ async function formatTeamsGraphNotification( } const resolvedChatId = chatId as string const resolvedMessageId = messageId as string - const providerConfig = (foundWebhook?.providerConfig as Record) || {} + const providerConfig = toRecord(foundWebhook?.providerConfig) const credentialId = providerConfig.credentialId const includeAttachments = providerConfig.includeAttachments !== false @@ -621,14 +621,7 @@ export const microsoftTeamsHandler: WebhookProviderHandler = { ) } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { logger.error(`[${requestId}] Failed to get access token for Teams subscription ${webhook.id}`) throw new Error( @@ -763,14 +756,7 @@ export const microsoftTeamsHandler: WebhookProviderHandler = { return } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { logger.warn( `[${requestId}] Could not get access token to delete Teams subscription for webhook ${webhook.id}` diff --git a/apps/sim/lib/webhooks/providers/monday.ts b/apps/sim/lib/webhooks/providers/monday.ts index 64ba5bde7f4..3d9ebafc470 100644 --- a/apps/sim/lib/webhooks/providers/monday.ts +++ b/apps/sim/lib/webhooks/providers/monday.ts @@ -2,9 +2,9 @@ import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' import { NextResponse } from 'next/server' import { validateMondayNumericId } from '@/lib/core/security/input-validation' -import { getOAuthToken, refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service' +import { getOAuthToken } from '@/lib/oauth/credential-service' import { - getCredentialOwner, + getCredentialAccessToken, getNotificationUrl, getProviderConfig, } from '@/lib/webhooks/provider-subscription-utils' @@ -22,7 +22,7 @@ const logger = createLogger('WebhookProvider:Monday') /** * Resolves an OAuth access token from the webhook's credential configuration. - * Follows the Airtable pattern: credentialId → getCredentialOwner → refreshAccessTokenIfNeeded. + * Follows the Airtable pattern: credentialId → getCredentialAccessToken. */ async function resolveAccessToken( config: Record, @@ -32,15 +32,8 @@ async function resolveAccessToken( const credentialId = config.credentialId as string | undefined if (credentialId) { - const credentialOwner = await getCredentialOwner(credentialId, requestId) - if (credentialOwner) { - const token = await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - if (token) return token - } + const token = await getCredentialAccessToken(credentialId, requestId) + if (token) return token } const fallbackToken = await getOAuthToken(userId, 'monday') @@ -193,14 +186,7 @@ export const mondayHandler: WebhookProviderHandler = { try { const credentialId = config.credentialId as string | undefined if (credentialId) { - const credentialOwner = await getCredentialOwner(credentialId, ctx.requestId) - if (credentialOwner) { - accessToken = await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - ctx.requestId - ) - } + accessToken = await getCredentialAccessToken(credentialId, ctx.requestId) } } catch (error) { logger.warn( diff --git a/apps/sim/lib/webhooks/providers/outlook.ts b/apps/sim/lib/webhooks/providers/outlook.ts index a6f2fa3d80e..4ac0da014fb 100644 --- a/apps/sim/lib/webhooks/providers/outlook.ts +++ b/apps/sim/lib/webhooks/providers/outlook.ts @@ -1,6 +1,7 @@ import { db } from '@sim/db' import { account, webhook } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import { refreshAccessTokenIfNeeded, resolveOAuthAccountId } from '@/lib/oauth/credential-service' import type { @@ -29,7 +30,7 @@ export const outlookHandler: WebhookProviderHandler = { logger.info(`[${requestId}] Setting up Outlook polling for webhook ${webhookData.id}`) try { - const providerConfig = (webhookData.providerConfig as Record) || {} + const providerConfig = toRecord(webhookData.providerConfig) const credentialId = providerConfig.credentialId as string | undefined if (!credentialId) { diff --git a/apps/sim/lib/webhooks/providers/rss.ts b/apps/sim/lib/webhooks/providers/rss.ts index 26f140f65d6..f89720bd871 100644 --- a/apps/sim/lib/webhooks/providers/rss.ts +++ b/apps/sim/lib/webhooks/providers/rss.ts @@ -1,6 +1,7 @@ import { db } from '@sim/db' import { webhook } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' import type { FormatInputContext, @@ -37,7 +38,7 @@ export const rssHandler: WebhookProviderHandler = { logger.info(`[${requestId}] Setting up RSS polling for webhook ${webhookData.id}`) try { - const providerConfig = (webhookData.providerConfig as Record) || {} + const providerConfig = toRecord(webhookData.providerConfig) const now = new Date() const configuredProviderConfig = { diff --git a/apps/sim/lib/webhooks/providers/slack.ts b/apps/sim/lib/webhooks/providers/slack.ts index af61151108a..873dac4a32b 100644 --- a/apps/sim/lib/webhooks/providers/slack.ts +++ b/apps/sim/lib/webhooks/providers/slack.ts @@ -908,7 +908,7 @@ export const slackHandler: WebhookProviderHandler = { */ async formatInput({ body, webhook, requestId }: FormatInputContext): Promise { const b = toRecord(body) - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) let botToken = providerConfig.botToken as string | undefined // Reusable custom Slack bot credential: use its stored bot token directly. if (!botToken && typeof providerConfig.credentialId === 'string') { diff --git a/apps/sim/lib/webhooks/providers/typeform.ts b/apps/sim/lib/webhooks/providers/typeform.ts index e366366868a..aa4f2181c02 100644 --- a/apps/sim/lib/webhooks/providers/typeform.ts +++ b/apps/sim/lib/webhooks/providers/typeform.ts @@ -2,6 +2,7 @@ import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { hmacSha256Base64 } from '@sim/security/hmac' import { getErrorMessage } from '@sim/utils/errors' +import { toRecord } from '@sim/utils/object' import { getNotificationUrl, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils' import type { DeleteSubscriptionContext, @@ -36,7 +37,7 @@ export const typeformHandler: WebhookProviderHandler = { async formatInput({ body, webhook }: FormatInputContext): Promise { const b = body as Record const formResponse = (b?.form_response || {}) as Record - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const includeDefinition = providerConfig.includeDefinition === true return { input: { diff --git a/apps/sim/lib/webhooks/providers/webflow.ts b/apps/sim/lib/webhooks/providers/webflow.ts index 25fabfc36d4..dd25cb21cd9 100644 --- a/apps/sim/lib/webhooks/providers/webflow.ts +++ b/apps/sim/lib/webhooks/providers/webflow.ts @@ -1,8 +1,12 @@ import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { validateAlphanumericId } from '@/lib/core/security/input-validation' -import { getBaseUrl } from '@/lib/core/utils/urls' -import { getOAuthToken, refreshAccessTokenIfNeeded } from '@/lib/oauth/credential-service' -import { getCredentialOwner, getProviderConfig } from '@/lib/webhooks/provider-subscription-utils' +import { getOAuthToken } from '@/lib/oauth/credential-service' +import { + getCredentialAccessToken, + getNotificationUrl, + getProviderConfig, +} from '@/lib/webhooks/provider-subscription-utils' import type { DeleteSubscriptionContext, EventFilterContext, @@ -23,8 +27,7 @@ export const webflowHandler: WebhookProviderHandler = { requestId, }: SubscriptionContext): Promise { try { - const { path, providerConfig } = webhookRecord as Record - const config = (providerConfig as Record) || {} + const config = getProviderConfig(webhookRecord) const { siteId, triggerId, collectionId, formName, credentialId } = config as { siteId?: string triggerId?: string @@ -52,17 +55,8 @@ export const webflowHandler: WebhookProviderHandler = { throw new Error('Trigger type is required to create Webflow webhook') } - const credentialOwner = credentialId - ? await getCredentialOwner(credentialId, requestId) - : null const accessToken = credentialId - ? credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + ? await getCredentialAccessToken(credentialId, requestId) : await getOAuthToken(userId, 'webflow') if (!accessToken) { logger.warn( @@ -73,7 +67,7 @@ export const webflowHandler: WebhookProviderHandler = { ) } - const notificationUrl = `${getBaseUrl()}/api/webhooks/trigger/${path}` + const notificationUrl = getNotificationUrl(webhookRecord) const triggerTypeMap: Record = { webflow_collection_item_created: 'collection_item_created', @@ -202,14 +196,7 @@ export const webflowHandler: WebhookProviderHandler = { return } - const credentialOwner = await getCredentialOwner(credentialId, requestId) - const accessToken = credentialOwner - ? await refreshAccessTokenIfNeeded( - credentialOwner.accountId, - credentialOwner.userId, - requestId - ) - : null + const accessToken = await getCredentialAccessToken(credentialId, requestId) if (!accessToken) { const message = `[${requestId}] Could not retrieve Webflow access token. Cannot delete webhook.` logger.warn(message, { webhookId: webhookRecord.id }) @@ -247,7 +234,7 @@ export const webflowHandler: WebhookProviderHandler = { async formatInput({ body, webhook }: FormatInputContext): Promise { const b = body as Record - const providerConfig = (webhook.providerConfig as Record) || {} + const providerConfig = toRecord(webhook.providerConfig) const triggerId = providerConfig.triggerId as string | undefined if (triggerId === 'webflow_form_submission') { return { diff --git a/apps/sim/lib/webhooks/providers/whatsapp.ts b/apps/sim/lib/webhooks/providers/whatsapp.ts index 5ab60964565..9748fdbcf0c 100644 --- a/apps/sim/lib/webhooks/providers/whatsapp.ts +++ b/apps/sim/lib/webhooks/providers/whatsapp.ts @@ -4,7 +4,7 @@ import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { sha256Hex } from '@sim/security/hash' import { hmacSha256Hex } from '@sim/security/hmac' -import { isRecordLike } from '@sim/utils/object' +import { isRecordLike, toRecord } from '@sim/utils/object' import { and, eq, isNull, or } from 'drizzle-orm' import { type NextRequest, NextResponse } from 'next/server' import type { @@ -206,7 +206,7 @@ async function handleWhatsAppVerification( for (const row of webhooks) { const wh = row.webhook - const providerConfig = (wh.providerConfig as Record) || {} + const providerConfig = toRecord(wh.providerConfig) const verificationToken = providerConfig.verificationToken if (!verificationToken) { diff --git a/apps/sim/lib/webhooks/providers/zoom.ts b/apps/sim/lib/webhooks/providers/zoom.ts index 0052dbd39e5..85d2fe8cd40 100644 --- a/apps/sim/lib/webhooks/providers/zoom.ts +++ b/apps/sim/lib/webhooks/providers/zoom.ts @@ -3,7 +3,7 @@ import { createLogger } from '@sim/logger' import { safeCompare } from '@sim/security/compare' import { hmacSha256Hex } from '@sim/security/hmac' import { toError } from '@sim/utils/errors' -import { isRecordLike } from '@sim/utils/object' +import { toRecord } from '@sim/utils/object' import { and, eq } from 'drizzle-orm' import type { NextRequest } from 'next/server' import { NextResponse } from 'next/server' @@ -68,9 +68,7 @@ async function resolveZoomChallengeSecrets( const resolvedRows = await Promise.all( rows.map(async (row) => { - const rawConfig = isRecordLike(row.providerConfig) - ? (row.providerConfig as Record) - : {} + const rawConfig = toRecord(row.providerConfig) try { /** diff --git a/apps/sim/lib/webhooks/slack-dispatch.ts b/apps/sim/lib/webhooks/slack-dispatch.ts index 18c83b8430b..9aebf59bd8d 100644 --- a/apps/sim/lib/webhooks/slack-dispatch.ts +++ b/apps/sim/lib/webhooks/slack-dispatch.ts @@ -1,5 +1,6 @@ import type { ExternalUserSubject } from '@sim/auth/principal' import { createLogger } from '@sim/logger' +import { toRecord } from '@sim/utils/object' import { type NextRequest, NextResponse } from 'next/server' import { mapWithConcurrency } from '@/lib/core/utils/concurrency' import { @@ -125,7 +126,7 @@ export async function dispatchSlackWebhooks( if (result.outcome === 'ignored' && result.reason === 'filtered') { const rawEvent = payload.event as Record | undefined - const providerConfig = (foundWebhook.providerConfig as Record) || {} + const providerConfig = toRecord(foundWebhook.providerConfig) logger.info( `[${requestId}] Event skipped by trigger filter for webhook ${foundWebhook.id}`, { diff --git a/apps/sim/lib/workflows/sanitization/json-sanitizer.ts b/apps/sim/lib/workflows/sanitization/json-sanitizer.ts index 69878d43c27..51b94488781 100644 --- a/apps/sim/lib/workflows/sanitization/json-sanitizer.ts +++ b/apps/sim/lib/workflows/sanitization/json-sanitizer.ts @@ -5,7 +5,6 @@ import { } from '@sim/workflow-persistence/subflow-helpers' import { normalizeWorkflowEdgeSourceHandle } from '@sim/workflow-types/workflow' import type { Edge } from '@xyflow/react' -import { getBaseUrl } from '@/lib/core/utils/urls' import { sanitizeWorkflowForSharing } from '@/lib/workflows/credentials/credential-extractor' import { getBlock } from '@/blocks/registry' import type { @@ -16,7 +15,11 @@ import type { WorkflowState, } from '@/stores/workflows/workflow/types' import { TRIGGER_ROUTING_FIELD, TRIGGER_WEBHOOK_URL_FIELD } from '@/triggers/constants' -import { blockAdvertisesWebhookUrl, resolveBlockTriggerId } from '@/triggers/webhook-url' +import { + blockAdvertisesWebhookUrl, + buildWebhookTriggerUrl, + resolveBlockTriggerId, +} from '@/triggers/webhook-url' /** * Sanitized workflow state for copilot (removes all UI-specific data) @@ -363,9 +366,9 @@ function resolveTriggerWebhookUrl(blockId: string, block: BlockState): string | const triggerPath = block.subBlocks?.triggerPath?.value const path = typeof triggerPath === 'string' && triggerPath.length > 0 ? triggerPath : blockId try { - return `${getBaseUrl()}/api/webhooks/trigger/${path}` + return buildWebhookTriggerUrl(path) } catch { - // getBaseUrl throws when NEXT_PUBLIC_APP_URL is unset; omit the field rather + // The base URL lookup throws when NEXT_PUBLIC_APP_URL is unset; omit the field rather // than fail the whole state read. return null } From 745079b77ada2de5ab65d27a90b572ee407985f2 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:15:49 -0700 Subject: [PATCH 13/30] chore(cleanup): remove dead billing, credential, MCP, and API key exports --- apps/sim/lib/api-key/auth.test.ts | 16 +------------- apps/sim/lib/api-key/auth.ts | 9 -------- apps/sim/lib/api-key/service.ts | 16 -------------- apps/sim/lib/billing/credits/balance.ts | 20 +---------------- apps/sim/lib/billing/usage-sources.ts | 7 ------ apps/sim/lib/credential-groups/providers.ts | 6 ------ apps/sim/lib/credentials/managed-mcp.ts | 4 ---- apps/sim/lib/mcp/shared.ts | 24 +-------------------- 8 files changed, 3 insertions(+), 99 deletions(-) diff --git a/apps/sim/lib/api-key/auth.test.ts b/apps/sim/lib/api-key/auth.test.ts index fab35506a5c..46a08a46292 100644 --- a/apps/sim/lib/api-key/auth.test.ts +++ b/apps/sim/lib/api-key/auth.test.ts @@ -26,7 +26,7 @@ const cryptoMock = vi.hoisted(() => ({ vi.mock('@/lib/api-key/crypto', () => cryptoMock) -import { isEncryptedKey, isValidApiKeyFormat } from '@/lib/api-key/auth' +import { isEncryptedKey } from '@/lib/api-key/auth' describe('isEncryptedKey', () => { it('detects only the three-part iv:encrypted:authTag storage format', () => { @@ -36,17 +36,3 @@ describe('isEncryptedKey', () => { expect(isEncryptedKey('a:b:c:d')).toBe(false) }) }) - -describe('isValidApiKeyFormat', () => { - it('enforces the 11 to 199 character bounds', () => { - expect(isValidApiKeyFormat('a'.repeat(11))).toBe(true) - expect(isValidApiKeyFormat('a'.repeat(199))).toBe(true) - expect(isValidApiKeyFormat('a'.repeat(10))).toBe(false) - expect(isValidApiKeyFormat('a'.repeat(200))).toBe(false) - }) - - it('rejects non-string input', () => { - expect(isValidApiKeyFormat(null as unknown as string)).toBe(false) - expect(isValidApiKeyFormat(123 as unknown as string)).toBe(false) - }) -}) diff --git a/apps/sim/lib/api-key/auth.ts b/apps/sim/lib/api-key/auth.ts index 9afba959588..99009d27736 100644 --- a/apps/sim/lib/api-key/auth.ts +++ b/apps/sim/lib/api-key/auth.ts @@ -136,15 +136,6 @@ export function formatApiKeyForDisplay(apiKey: string): string { return `...${last4}` } -/** - * Validates API key format (basic validation) - * @param apiKey - The API key to validate - * @returns boolean - true if the format appears valid - */ -export function isValidApiKeyFormat(apiKeyValue: string): boolean { - return typeof apiKeyValue === 'string' && apiKeyValue.length > 10 && apiKeyValue.length < 200 -} - export async function createWorkspaceApiKey(params: { workspaceId: string userId: string diff --git a/apps/sim/lib/api-key/service.ts b/apps/sim/lib/api-key/service.ts index 79baf152f56..0ca2c46abb7 100644 --- a/apps/sim/lib/api-key/service.ts +++ b/apps/sim/lib/api-key/service.ts @@ -8,22 +8,6 @@ import { getWorkspaceBillingSettings, type WorkspaceBillingSettings } from '@/li const logger = createLogger('ApiKeyService') -export async function listApiKeys(workspaceId: string) { - return db - .select({ - id: apiKeyTable.id, - name: apiKeyTable.name, - type: apiKeyTable.type, - lastUsed: apiKeyTable.lastUsed, - createdAt: apiKeyTable.createdAt, - expiresAt: apiKeyTable.expiresAt, - createdBy: apiKeyTable.createdBy, - }) - .from(apiKeyTable) - .where(and(eq(apiKeyTable.workspaceId, workspaceId), eq(apiKeyTable.type, 'workspace'))) - .orderBy(apiKeyTable.createdAt) -} - export interface ApiKeyAuthOptions { userId?: string workspaceId?: string diff --git a/apps/sim/lib/billing/credits/balance.ts b/apps/sim/lib/billing/credits/balance.ts index d702056d809..5334bb4d4a7 100644 --- a/apps/sim/lib/billing/credits/balance.ts +++ b/apps/sim/lib/billing/credits/balance.ts @@ -2,13 +2,8 @@ import { db } from '@sim/db' import { organization, userStats } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { eq, sql } from 'drizzle-orm' -import { getEffectiveBillingStatus } from '@/lib/billing/core/access' import { getHighestPrioritySubscription } from '@/lib/billing/core/subscription' -import { isPro, isTeam } from '@/lib/billing/plan-helpers' -import { - hasUsableSubscriptionAccess, - isOrgScopedSubscription, -} from '@/lib/billing/subscriptions/utils' +import { isOrgScopedSubscription } from '@/lib/billing/subscriptions/utils' import { toDecimal, toFixedString, toNumber } from '@/lib/billing/utils/decimal' import type { DbClient } from '@/lib/db/types' @@ -150,16 +145,3 @@ async function atomicDeductOrgCredits(orgId: string, cost: number): Promise { - const subscription = await getHighestPrioritySubscription(userId) - if (!subscription) { - return false - } - const billingStatus = await getEffectiveBillingStatus(userId) - if (!hasUsableSubscriptionAccess(subscription.status, billingStatus.billingBlocked)) { - return false - } - // Enterprise users must contact support to purchase credits - return isPro(subscription.plan) || isTeam(subscription.plan) -} diff --git a/apps/sim/lib/billing/usage-sources.ts b/apps/sim/lib/billing/usage-sources.ts index 36a140d4a2d..0e7faeae394 100644 --- a/apps/sim/lib/billing/usage-sources.ts +++ b/apps/sim/lib/billing/usage-sources.ts @@ -14,13 +14,6 @@ export const INTERNAL_USAGE_LOG_SOURCES = [ export type InternalUsageLogSource = (typeof INTERNAL_USAGE_LOG_SOURCES)[number] -export const INTERNAL_CHAT_BILLING_SOURCES = [ - 'copilot', - 'workspace-chat', - 'mcp_copilot', - 'mothership_block', -] as const satisfies readonly InternalUsageLogSource[] - export const BILLING_USAGE_LOG_SOURCES = [ 'workflow', 'wand', diff --git a/apps/sim/lib/credential-groups/providers.ts b/apps/sim/lib/credential-groups/providers.ts index 27d36adae35..483818ab2f1 100644 --- a/apps/sim/lib/credential-groups/providers.ts +++ b/apps/sim/lib/credential-groups/providers.ts @@ -258,12 +258,6 @@ export function getCredentialGroupProviderService( return service } -export function getCredentialGroupProviderSupport( - provider: CredentialGroupProvider -): CredentialGroupProviderSupport { - return CREDENTIAL_GROUP_PROVIDER_SUPPORT[provider] -} - export function getCredentialGroupProviderId(provider: CredentialGroupProvider): string { return getCredentialGroupProviderService(provider).providerId } diff --git a/apps/sim/lib/credentials/managed-mcp.ts b/apps/sim/lib/credentials/managed-mcp.ts index 15a1c837e6e..54ca13c9c88 100644 --- a/apps/sim/lib/credentials/managed-mcp.ts +++ b/apps/sim/lib/credentials/managed-mcp.ts @@ -136,10 +136,6 @@ async function decryptManagedMcpEnvelope(encrypted: string): Promise { - return (await decryptManagedMcpEnvelope(encrypted)).tokens -} - export async function loadManagedMcpCredentialApplicationContext( credentialId: string, executingWorkspaceId?: string diff --git a/apps/sim/lib/mcp/shared.ts b/apps/sim/lib/mcp/shared.ts index 2df05010595..cf980e19b05 100644 --- a/apps/sim/lib/mcp/shared.ts +++ b/apps/sim/lib/mcp/shared.ts @@ -1,4 +1,4 @@ -import { type McpOperationPolicy, normalizeMcpOperationPolicy } from '@/lib/mcp/operation-policy' +import { normalizeMcpOperationPolicy } from '@/lib/mcp/operation-policy' /** * Shared MCP utilities - safe for both client and server. * No server-side dependencies (database, fs, etc.) should be imported here. @@ -8,28 +8,6 @@ import { isMcpTool, MCP } from '@/executor/constants' export const MCP_SERVER_ADVANCED_TOOL_TYPE = 'mcp-server-advanced' as const -export interface McpServerAdvancedToolBinding { - type: typeof MCP_SERVER_ADVANCED_TOOL_TYPE - params: { - serverId: string - } - operationPolicy?: McpOperationPolicy - usageControl?: 'auto' | 'force' | 'none' -} - -export function isMcpServerAdvancedToolBinding( - value: unknown -): value is McpServerAdvancedToolBinding { - if (!value || typeof value !== 'object' || Array.isArray(value)) return false - const binding = value as { type?: unknown; params?: unknown } - if (binding.type !== MCP_SERVER_ADVANCED_TOOL_TYPE) return false - if (!binding.params || typeof binding.params !== 'object' || Array.isArray(binding.params)) { - return false - } - const serverId = (binding.params as { serverId?: unknown }).serverId - return typeof serverId === 'string' && serverId.trim().length > 0 -} - /** Rejects ambiguous server-wide bindings while leaving legacy MCP entries untouched. */ export function assertValidMcpServerToolBindings(value: unknown): void { if (!Array.isArray(value)) return From 001026149412b3035d1fdd1ece245b7325e3c33a Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:06:13 -0700 Subject: [PATCH 14/30] chore(cleanup): remove dead executor helpers and redundant LoopConfigWithNodes type --- apps/sim/executor/orchestrators/loop.ts | 12 +++--- apps/sim/executor/types/loop.ts | 5 --- .../utils/code-secret-references.test.ts | 42 ------------------- .../executor/utils/code-secret-references.ts | 21 ---------- apps/sim/executor/utils/delegation.test.ts | 16 ------- apps/sim/executor/utils/delegation.ts | 21 ---------- .../function-execute-table-mounts.test.ts | 3 -- 7 files changed, 5 insertions(+), 115 deletions(-) delete mode 100644 apps/sim/executor/types/loop.ts delete mode 100644 apps/sim/executor/utils/code-secret-references.test.ts delete mode 100644 apps/sim/executor/utils/code-secret-references.ts delete mode 100644 apps/sim/executor/utils/delegation.test.ts delete mode 100644 apps/sim/executor/utils/delegation.ts diff --git a/apps/sim/executor/orchestrators/loop.ts b/apps/sim/executor/orchestrators/loop.ts index 946eb264b66..54a4972f255 100644 --- a/apps/sim/executor/orchestrators/loop.ts +++ b/apps/sim/executor/orchestrators/loop.ts @@ -16,7 +16,6 @@ import type { EdgeManager } from '@/executor/execution/edge-manager' import type { LoopScope } from '@/executor/execution/state' import type { BlockStateController, ContextExtensions } from '@/executor/execution/types' import type { ExecutionContext, NormalizedBlockOutput } from '@/executor/types' -import type { LoopConfigWithNodes } from '@/executor/types/loop' import { createReferencePattern } from '@/executor/utils/reference-validation' import { projectResolvedSecretDiagnosticError } from '@/executor/utils/resolved-secret-content-projection' import { mergeSubflowSecretProvenance } from '@/executor/utils/subflow-secret-provenance' @@ -33,7 +32,6 @@ import { } from '@/executor/utils/subflow-utils' import { resolveArrayInputAsync } from '@/executor/utils/subflow-utils.server' import type { VariableResolver } from '@/executor/variables/resolver' -import type { SerializedLoop } from '@/serializer/types' const logger = createLogger('LoopOrchestrator') @@ -76,7 +74,7 @@ export class LoopOrchestrator { ) {} async initializeLoopScope(ctx: ExecutionContext, loopId: string): Promise { - const loopConfig = this.dag.loopConfigs.get(loopId) as SerializedLoop | undefined + const loopConfig = this.dag.loopConfigs.get(loopId) if (!loopConfig) { throw new Error(`Loop config not found: ${loopId}`) } @@ -416,7 +414,7 @@ export class LoopOrchestrator { * on the next outer iteration. */ private resetNestedLoopScopes(loopId: string, ctx: ExecutionContext): void { - const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + const loopConfig = this.dag.loopConfigs.get(loopId) if (!loopConfig) return for (const nodeId of loopConfig.nodes) { @@ -445,7 +443,7 @@ export class LoopOrchestrator { * next outer loop iteration. */ private resetNestedParallelScopes(loopId: string, ctx: ExecutionContext): void { - const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + const loopConfig = this.dag.loopConfigs.get(loopId) if (!loopConfig) return for (const nodeId of loopConfig.nodes) { @@ -507,7 +505,7 @@ export class LoopOrchestrator { if (visited.has(loopId)) return new Set() visited.add(loopId) - const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + const loopConfig = this.dag.loopConfigs.get(loopId) if (!loopConfig) return new Set() const sentinelStartId = buildSentinelStartId(loopId) @@ -609,7 +607,7 @@ export class LoopOrchestrator { } restoreLoopEdges(loopId: string): void { - const loopConfig = this.dag.loopConfigs.get(loopId) as LoopConfigWithNodes | undefined + const loopConfig = this.dag.loopConfigs.get(loopId) if (!loopConfig) { logger.warn('Loop config not found for edge restoration', { loopId }) return diff --git a/apps/sim/executor/types/loop.ts b/apps/sim/executor/types/loop.ts deleted file mode 100644 index e2be8cca95a..00000000000 --- a/apps/sim/executor/types/loop.ts +++ /dev/null @@ -1,5 +0,0 @@ -import type { SerializedLoop } from '@/serializer/types' - -export interface LoopConfigWithNodes extends SerializedLoop { - nodes: string[] -} diff --git a/apps/sim/executor/utils/code-secret-references.test.ts b/apps/sim/executor/utils/code-secret-references.test.ts deleted file mode 100644 index c041ab83f01..00000000000 --- a/apps/sim/executor/utils/code-secret-references.test.ts +++ /dev/null @@ -1,42 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { extractCodeSecretNames } from '@/executor/utils/code-secret-references' - -describe('Copilot code secret declarations', () => { - it.each(['javascript', 'python'])( - 'matches trimmed and embedded references for %s', - async (language) => { - expect( - await extractCodeSecretNames( - 'const first = "prefix-{{ API_KEY }}"\nreturn "{{TOKEN}}/{{API_KEY}}"', - language - ) - ).toEqual(['API_KEY', 'TOKEN']) - } - ) - - it('matches shell references supported by the shared compiler', async () => { - expect( - await extractCodeSecretNames( - 'echo {{API_KEY}} {{ API_KEY }} {{9INVALID}} {{WITH-DASH}} {{_TOKEN}}', - 'shell' - ) - ).toEqual(['API_KEY', '_TOKEN']) - }) - - it('ignores direct environment access, shell variables, literals, and malformed references', async () => { - expect( - await extractCodeSecretNames( - 'return environmentVariables.API_KEY + "$TOKEN" + "literal" + "{{}}" + "{{MISSING"', - 'javascript' - ) - ).toEqual([]) - }) - - it.each([ - ['javascript', '// {{COMMENT_ONLY}}\nreturn {{USED}}'], - ['python', '# {{COMMENT_ONLY}}\nreturn {{USED}}'], - ['shell', '# {{COMMENT_ONLY}}\necho {{USED}}'], - ])('ignores placeholders in %s comments', async (language, code) => { - await expect(extractCodeSecretNames(code, language)).resolves.toEqual(['USED']) - }) -}) diff --git a/apps/sim/executor/utils/code-secret-references.ts b/apps/sim/executor/utils/code-secret-references.ts deleted file mode 100644 index df033ad58c9..00000000000 --- a/apps/sim/executor/utils/code-secret-references.ts +++ /dev/null @@ -1,21 +0,0 @@ -import { analyzeCodePlaceholders } from '@/lib/execution/code-placeholders' -import { - type CodeLanguage, - DEFAULT_CODE_LANGUAGE, - isValidCodeLanguage, -} from '@/lib/execution/languages' - -function resolveCodeLanguage(language: unknown): CodeLanguage { - return typeof language === 'string' && isValidCodeLanguage(language) - ? language - : DEFAULT_CODE_LANGUAGE -} - -/** - * Extracts only environment references the Function runtime can resolve for the selected language. - * The returned order follows the code, with duplicate names removed after their first occurrence. - */ -export async function extractCodeSecretNames(code: unknown, language?: unknown): Promise { - if (typeof code !== 'string') return [] - return analyzeCodePlaceholders(code, resolveCodeLanguage(language)) -} diff --git a/apps/sim/executor/utils/delegation.test.ts b/apps/sim/executor/utils/delegation.test.ts deleted file mode 100644 index ab8a8b45b66..00000000000 --- a/apps/sim/executor/utils/delegation.test.ts +++ /dev/null @@ -1,16 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { executionScopeForTarget } from '@/executor/utils/delegation' - -describe('executionScopeForTarget', () => { - it('binds the execution when the target is the running workflow', () => { - expect( - executionScopeForTarget({ workflowId: 'workflow-1', executionId: 'run-1' }, 'workflow-1') - ).toEqual({ executionId: 'run-1' }) - }) - - it('omits the execution for a child workflow, which binds on its own id', () => { - expect( - executionScopeForTarget({ workflowId: 'parent', executionId: 'run-1' }, 'child') - ).toEqual({}) - }) -}) diff --git a/apps/sim/executor/utils/delegation.ts b/apps/sim/executor/utils/delegation.ts deleted file mode 100644 index 6bead347656..00000000000 --- a/apps/sim/executor/utils/delegation.ts +++ /dev/null @@ -1,21 +0,0 @@ -import type { GenerateInternalDelegationTokenInput } from '@/lib/auth/internal' - -/** - * Binds the running execution to a delegation only when it targets the workflow that - * is actually running. - * - * A child workflow is a separate resource and binds on its own id, so forwarding the - * parent's `executionId` would assert a run that does not cover the target and the - * delegation would fail to bind. Callers spread the result into their delegation input. - * - * Kept free of runtime imports so client-reachable modules can read it without pulling - * in the executor graph. - */ -export function executionScopeForTarget( - context: { workflowId?: string; executionId?: string }, - targetWorkflowId: string -): Pick { - return context.workflowId === targetWorkflowId && context.executionId - ? { executionId: context.executionId } - : {} -} diff --git a/apps/sim/lib/mothership/tools/handlers/function-execute-table-mounts.test.ts b/apps/sim/lib/mothership/tools/handlers/function-execute-table-mounts.test.ts index e75341161ab..97f3ae629c0 100644 --- a/apps/sim/lib/mothership/tools/handlers/function-execute-table-mounts.test.ts +++ b/apps/sim/lib/mothership/tools/handlers/function-execute-table-mounts.test.ts @@ -36,9 +36,6 @@ vi.mock('@/lib/table/snapshot-cache', () => ({ })) vi.mock('@/lib/table/rows/secret-provenance', () => tableRowsSecretProvenanceMock) vi.mock('@/lib/uploads/core/storage-service', () => storageServiceMock) -vi.mock('@/executor/utils/code-secret-references', () => ({ - extractCodeSecretNames: vi.fn().mockResolvedValue([]), -})) vi.mock('@/lib/secrets/usage/record', () => ({ recordSecretUsage: vi.fn() })) vi.mock('@/lib/realtime/notify', () => realtimeNotifyMock) From 5887e5942a0dd1d67e77799b7ffe3343f0fcce49 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:09:23 -0700 Subject: [PATCH 15/30] refactor(executor): export subflow node-id codec functions directly and drop forwarders --- .../utils/workflow-execution-utils.ts | 2 +- apps/sim/executor/dag/builder.test.ts | 2 +- apps/sim/executor/dag/builder.ts | 6 +- .../executor/dag/construction/edges.test.ts | 2 +- apps/sim/executor/dag/construction/edges.ts | 34 ++--- apps/sim/executor/dag/construction/loops.ts | 9 +- apps/sim/executor/dag/construction/nodes.ts | 2 +- .../executor/dag/construction/parallels.ts | 2 +- apps/sim/executor/dag/construction/paths.ts | 2 +- apps/sim/executor/execution/block-executor.ts | 2 +- .../executor/execution/edge-manager.test.ts | 10 +- apps/sim/executor/execution/executor.test.ts | 2 +- apps/sim/executor/execution/executor.ts | 2 +- apps/sim/executor/execution/state.ts | 17 +-- .../condition/condition-handler.test.ts | 10 +- .../handlers/condition/condition-handler.ts | 2 +- apps/sim/executor/human-in-the-loop/utils.ts | 4 +- apps/sim/executor/orchestrators/loop.ts | 21 ++- apps/sim/executor/orchestrators/node.ts | 2 +- .../executor/orchestrators/parallel.test.ts | 10 +- apps/sim/executor/orchestrators/parallel.ts | 15 +- apps/sim/executor/utils/block-data.ts | 2 +- apps/sim/executor/utils/iteration-context.ts | 2 +- .../executor/utils/parallel-expansion.test.ts | 2 +- apps/sim/executor/utils/parallel-expansion.ts | 26 ++-- apps/sim/executor/utils/run-from-block.ts | 20 +-- .../utils/subflow-node-id-codec.test.ts | 115 ++++++++------- .../executor/utils/subflow-node-id-codec.ts | 73 +++------- apps/sim/executor/utils/subflow-utils.test.ts | 2 +- apps/sim/executor/utils/subflow-utils.ts | 131 ------------------ .../sim/executor/variables/resolvers/block.ts | 7 +- apps/sim/executor/variables/resolvers/loop.ts | 5 +- .../executor/variables/resolvers/parallel.ts | 5 +- .../trace-spans/iteration-grouping.ts | 2 +- .../lib/workflows/executor/execution-core.ts | 7 +- 35 files changed, 200 insertions(+), 357 deletions(-) diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts index bc0c9d32afe..90e6db26335 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/workflow-execution-utils.ts @@ -13,7 +13,7 @@ import type { BlockStartedData, } from '@/lib/workflows/executor/execution-events' import type { BlockLog, BlockState, ExecutionResult, StreamingExecution } from '@/executor/types' -import { stripCloneSuffixes } from '@/executor/utils/subflow-utils' +import { stripCloneSuffixes } from '@/executor/utils/subflow-node-id-codec' import { ExecutionStreamHttpError, processSSEStream, diff --git a/apps/sim/executor/dag/builder.test.ts b/apps/sim/executor/dag/builder.test.ts index 9dee64615b3..3732568e302 100644 --- a/apps/sim/executor/dag/builder.test.ts +++ b/apps/sim/executor/dag/builder.test.ts @@ -5,7 +5,7 @@ import { buildBranchNodeId, buildParallelSentinelEndId, buildParallelSentinelStartId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' function createBlock(id: string, metadataId: string): SerializedBlock { diff --git a/apps/sim/executor/dag/builder.ts b/apps/sim/executor/dag/builder.ts index 6d8ee819ea7..3ebc9bd691b 100644 --- a/apps/sim/executor/dag/builder.ts +++ b/apps/sim/executor/dag/builder.ts @@ -6,10 +6,10 @@ import { ParallelConstructor } from '@/executor/dag/construction/parallels' import { PathConstructor } from '@/executor/dag/construction/paths' import type { DAGEdge, NodeMetadata } from '@/executor/dag/types' import { + buildLoopSentinelStartId, buildParallelSentinelStartId, - buildSentinelStartId, normalizeNodeId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedLoop, @@ -150,7 +150,7 @@ export class DAGBuilder { type: 'Loop' | 'Parallel' ): void { const sentinelStartId = - type === 'Loop' ? buildSentinelStartId(id) : buildParallelSentinelStartId(id) + type === 'Loop' ? buildLoopSentinelStartId(id) : buildParallelSentinelStartId(id) const sentinelStartNode = dag.nodes.get(sentinelStartId) if (!sentinelStartNode) return diff --git a/apps/sim/executor/dag/construction/edges.test.ts b/apps/sim/executor/dag/construction/edges.test.ts index 9158e6b6d94..2309a63eeb2 100644 --- a/apps/sim/executor/dag/construction/edges.test.ts +++ b/apps/sim/executor/dag/construction/edges.test.ts @@ -1,6 +1,6 @@ import { beforeEach, describe, expect, it } from 'vitest' import type { DAG, DAGNode } from '@/executor/dag/builder' -import { buildBranchNodeId } from '@/executor/utils/subflow-utils' +import { buildBranchNodeId } from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedLoop, SerializedWorkflow } from '@/serializer/types' import { EdgeConstructor } from './edges' diff --git a/apps/sim/executor/dag/construction/edges.ts b/apps/sim/executor/dag/construction/edges.ts index afe4aac5671..e587e184787 100644 --- a/apps/sim/executor/dag/construction/edges.ts +++ b/apps/sim/executor/dag/construction/edges.ts @@ -10,12 +10,12 @@ import { import type { DAG, DAGNode } from '@/executor/dag/builder' import { buildBranchNodeId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, normalizeNodeId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedWorkflow } from '@/serializer/types' const logger = createLogger('EdgeConstructor') @@ -239,8 +239,8 @@ export class EdgeConstructor { } if (sourceIsLoopBlock) { - const sentinelEndId = buildSentinelEndId(originalSource) - const loopSentinelStartId = buildSentinelStartId(originalSource) + const sentinelEndId = buildLoopSentinelEndId(originalSource) + const loopSentinelStartId = buildLoopSentinelStartId(originalSource) if (!dag.nodes.has(sentinelEndId) || !dag.nodes.has(loopSentinelStartId)) { continue } @@ -250,7 +250,7 @@ export class EdgeConstructor { } if (targetIsLoopBlock) { - const sentinelStartId = buildSentinelStartId(target) + const sentinelStartId = buildLoopSentinelStartId(target) if (!dag.nodes.has(sentinelStartId)) { continue } @@ -315,8 +315,8 @@ export class EdgeConstructor { if (nodes.length === 0) continue - const sentinelStartId = buildSentinelStartId(loopId) - const sentinelEndId = buildSentinelEndId(loopId) + const sentinelStartId = buildLoopSentinelStartId(loopId) + const sentinelEndId = buildLoopSentinelEndId(loopId) if (!dag.nodes.has(sentinelStartId) || !dag.nodes.has(sentinelEndId)) { continue @@ -405,7 +405,7 @@ export class EdgeConstructor { return buildParallelSentinelStartId(nodeId) } if (dag.loopConfigs.has(nodeId)) { - return buildSentinelStartId(nodeId) + return buildLoopSentinelStartId(nodeId) } return buildBranchNodeId(nodeId, 0) } @@ -420,7 +420,7 @@ export class EdgeConstructor { return buildParallelSentinelEndId(nodeId) } if (dag.loopConfigs.has(nodeId)) { - return buildSentinelEndId(nodeId) + return buildLoopSentinelEndId(nodeId) } return buildBranchNodeId(nodeId, 0) } @@ -518,7 +518,7 @@ export class EdgeConstructor { ): { resolvedId: string; node: DAGNode | undefined } { if (dag.loopConfigs.has(nodeId)) { const resolvedId = - sentinel === 'start' ? buildSentinelStartId(nodeId) : buildSentinelEndId(nodeId) + sentinel === 'start' ? buildLoopSentinelStartId(nodeId) : buildLoopSentinelEndId(nodeId) return { resolvedId, node: dag.nodes.get(resolvedId) } } if (dag.parallelConfigs.has(nodeId)) { @@ -547,8 +547,8 @@ export class EdgeConstructor { const effective = new Set() for (const nodeId of nodes) { if (dag.loopConfigs.has(nodeId)) { - effective.add(buildSentinelStartId(nodeId)) - effective.add(buildSentinelEndId(nodeId)) + effective.add(buildLoopSentinelStartId(nodeId)) + effective.add(buildLoopSentinelEndId(nodeId)) } else if (dag.parallelConfigs.has(nodeId)) { effective.add(buildParallelSentinelStartId(nodeId)) effective.add(buildParallelSentinelEndId(nodeId)) @@ -679,8 +679,8 @@ export class EdgeConstructor { } if (dag.loopConfigs.has(nodeId)) { return { - startNode: dag.nodes.get(buildSentinelStartId(nodeId)), - endNode: dag.nodes.get(buildSentinelEndId(nodeId)), + startNode: dag.nodes.get(buildLoopSentinelStartId(nodeId)), + endNode: dag.nodes.get(buildLoopSentinelEndId(nodeId)), } } // Regular block — use branch template node for both @@ -744,8 +744,8 @@ export class EdgeConstructor { } if (dag.loopConfigs.has(subflowId)) { - const sourceId = buildSentinelStartId(subflowId) - const targetId = buildSentinelEndId(subflowId) + const sourceId = buildLoopSentinelStartId(subflowId) + const targetId = buildLoopSentinelEndId(subflowId) if (dag.nodes.has(sourceId) && dag.nodes.has(targetId)) { this.addEdge(dag, sourceId, targetId, EDGE.LOOP_EXIT, undefined, { registerIncoming: false, diff --git a/apps/sim/executor/dag/construction/loops.ts b/apps/sim/executor/dag/construction/loops.ts index 87388b34519..1d72d69a86f 100644 --- a/apps/sim/executor/dag/construction/loops.ts +++ b/apps/sim/executor/dag/construction/loops.ts @@ -1,7 +1,10 @@ import { BlockType, LOOP } from '@/executor/constants' import type { DAG } from '@/executor/dag/builder' import { createSubflowSentinelNode } from '@/executor/dag/construction/sentinels' -import { buildSentinelEndId, buildSentinelStartId } from '@/executor/utils/subflow-utils' +import { + buildLoopSentinelEndId, + buildLoopSentinelStartId, +} from '@/executor/utils/subflow-node-id-codec' export class LoopConstructor { execute(dag: DAG, reachableBlocks: Set): void { @@ -22,8 +25,8 @@ export class LoopConstructor { } private createSentinelPair(dag: DAG, loopId: string): void { - const startId = buildSentinelStartId(loopId) - const endId = buildSentinelEndId(loopId) + const startId = buildLoopSentinelStartId(loopId) + const endId = buildLoopSentinelEndId(loopId) dag.nodes.set( startId, diff --git a/apps/sim/executor/dag/construction/nodes.ts b/apps/sim/executor/dag/construction/nodes.ts index 51e66acc5a2..57d77ae94ff 100644 --- a/apps/sim/executor/dag/construction/nodes.ts +++ b/apps/sim/executor/dag/construction/nodes.ts @@ -1,6 +1,6 @@ import { isHumanInTheLoopBlock, isMetadataOnlyBlockType } from '@/executor/constants' import type { DAG } from '@/executor/dag/builder' -import { buildBranchNodeId } from '@/executor/utils/subflow-utils' +import { buildBranchNodeId } from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' export class NodeConstructor { diff --git a/apps/sim/executor/dag/construction/parallels.ts b/apps/sim/executor/dag/construction/parallels.ts index 0f8bc9918f4..3ecd729974d 100644 --- a/apps/sim/executor/dag/construction/parallels.ts +++ b/apps/sim/executor/dag/construction/parallels.ts @@ -4,7 +4,7 @@ import { createSubflowSentinelNode } from '@/executor/dag/construction/sentinels import { buildParallelSentinelEndId, buildParallelSentinelStartId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' export class ParallelConstructor { execute(dag: DAG, reachableBlocks: Set): void { diff --git a/apps/sim/executor/dag/construction/paths.ts b/apps/sim/executor/dag/construction/paths.ts index 2242a6d4399..301eb1563a6 100644 --- a/apps/sim/executor/dag/construction/paths.ts +++ b/apps/sim/executor/dag/construction/paths.ts @@ -1,6 +1,6 @@ import { createLogger } from '@sim/logger' import { isMetadataOnlyBlockType, isTriggerBlockType } from '@/executor/constants' -import { extractBaseBlockId } from '@/executor/utils/subflow-utils' +import { extractBaseBlockId } from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' const logger = createLogger('PathConstructor') diff --git a/apps/sim/executor/execution/block-executor.ts b/apps/sim/executor/execution/block-executor.ts index e893f5dde92..c76fd5def2e 100644 --- a/apps/sim/executor/execution/block-executor.ts +++ b/apps/sim/executor/execution/block-executor.ts @@ -74,7 +74,7 @@ import { buildBranchNodeId, buildOuterBranchScopedId, extractOuterBranchIndex, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import { FUNCTION_BLOCK_CONTEXT_VARS_KEY, FUNCTION_BLOCK_DISPLAY_CODE_KEY, diff --git a/apps/sim/executor/execution/edge-manager.test.ts b/apps/sim/executor/execution/edge-manager.test.ts index 33ab2219892..34ca54e22e0 100644 --- a/apps/sim/executor/execution/edge-manager.test.ts +++ b/apps/sim/executor/execution/edge-manager.test.ts @@ -6,11 +6,11 @@ import { EdgeManager } from '@/executor/execution/edge-manager' import type { NormalizedBlockOutput } from '@/executor/types' import { buildBranchNodeId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' function createMockBlock(id: string): SerializedBlock { @@ -106,11 +106,11 @@ describe('EdgeManager', () => { const edgeManager = new EdgeManager(dag) const sentinelStartId = subflowType === 'loop' - ? buildSentinelStartId('subflow') + ? buildLoopSentinelStartId('subflow') : buildParallelSentinelStartId('subflow') const sentinelEndId = subflowType === 'loop' - ? buildSentinelEndId('subflow') + ? buildLoopSentinelEndId('subflow') : buildParallelSentinelEndId('subflow') const conditionId = subflowType === 'loop' ? 'condition' : buildBranchNodeId('condition', 0) diff --git a/apps/sim/executor/execution/executor.test.ts b/apps/sim/executor/execution/executor.test.ts index f3dd282ba15..c307535850b 100644 --- a/apps/sim/executor/execution/executor.test.ts +++ b/apps/sim/executor/execution/executor.test.ts @@ -8,7 +8,7 @@ import { DAGExecutor } from '@/executor/execution/executor' import type { SerializableExecutionState } from '@/executor/execution/types' import type { ExecutionContext, ExecutionResult } from '@/executor/types' import { RunFromBlockValidationError } from '@/executor/utils/run-from-block' -import { stripCloneSuffixes } from '@/executor/utils/subflow-utils' +import { stripCloneSuffixes } from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' const { executed, gateChoice } = vi.hoisted(() => ({ diff --git a/apps/sim/executor/execution/executor.ts b/apps/sim/executor/execution/executor.ts index 4329b883762..6fcd09f611d 100644 --- a/apps/sim/executor/execution/executor.ts +++ b/apps/sim/executor/execution/executor.ts @@ -41,7 +41,7 @@ import { extractParallelIdFromSentinel, stripCloneSuffixes, stripOuterBranchSuffix, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import { VariableResolver } from '@/executor/variables/resolver' import { navigatePathAsync } from '@/executor/variables/resolvers/reference-async.server' import type { SerializedWorkflow } from '@/serializer/types' diff --git a/apps/sim/executor/execution/state.ts b/apps/sim/executor/execution/state.ts index d9999257cab..fb3b0b9b777 100644 --- a/apps/sim/executor/execution/state.ts +++ b/apps/sim/executor/execution/state.ts @@ -1,24 +1,15 @@ import type { BlockStateController } from '@/executor/execution/types' import type { BlockState, NormalizedBlockOutput } from '@/executor/types' import type { ResolvedSecretTraceProvenanceV1 } from '@/executor/utils/resolved-secret-trace-registry' -import { SubflowNodeIdCodec } from '@/executor/utils/subflow-node-id-codec' import { buildOuterBranchScopedId, + extractBranchSuffix, + extractLoopSuffix, extractOuterBranchIndex, + normalizeLookupId, stripCloneSuffixes, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' -function normalizeLookupId(id: string): string { - return SubflowNodeIdCodec.normalizeLookupId(id) -} - -function extractBranchSuffix(id: string): string { - return SubflowNodeIdCodec.extractBranchSuffix(id) -} - -function extractLoopSuffix(id: string): string { - return SubflowNodeIdCodec.extractLoopSuffix(id) -} export interface LoopScope { iteration: number currentIterationOutputs: Map diff --git a/apps/sim/executor/handlers/condition/condition-handler.test.ts b/apps/sim/executor/handlers/condition/condition-handler.test.ts index 4b347469487..5b94100eaec 100644 --- a/apps/sim/executor/handlers/condition/condition-handler.test.ts +++ b/apps/sim/executor/handlers/condition/condition-handler.test.ts @@ -8,11 +8,11 @@ import { ConditionBlockHandler } from '@/executor/handlers/condition/condition-h import type { BlockState, ExecutionContext, NormalizedBlockOutput } from '@/executor/types' import { buildBranchNodeId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' vi.mock('@/tools', () => toolsMock) @@ -466,11 +466,11 @@ describe('ConditionBlockHandler', () => { subflowType === 'loop' ? mockBlock.id : buildBranchNodeId(mockBlock.id, 0) const sentinelStartId = subflowType === 'loop' - ? buildSentinelStartId(subflowId) + ? buildLoopSentinelStartId(subflowId) : buildParallelSentinelStartId(subflowId) const sentinelEndId = subflowType === 'loop' - ? buildSentinelEndId(subflowId) + ? buildLoopSentinelEndId(subflowId) : buildParallelSentinelEndId(subflowId) const conditionNode = dag.nodes.get(conditionNodeId)! mockContext.currentVirtualBlockId = conditionNodeId diff --git a/apps/sim/executor/handlers/condition/condition-handler.ts b/apps/sim/executor/handlers/condition/condition-handler.ts index 5086cdd81a6..44824f6c894 100644 --- a/apps/sim/executor/handlers/condition/condition-handler.ts +++ b/apps/sim/executor/handlers/condition/condition-handler.ts @@ -16,7 +16,7 @@ import { extractBaseBlockId, extractBranchIndex, isBranchNodeId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import { CONDITION_READS_ENVIRONMENT_KEY } from '@/executor/variables/resolver' import type { SerializedBlock } from '@/serializer/types' import { executeTool } from '@/tools' diff --git a/apps/sim/executor/human-in-the-loop/utils.ts b/apps/sim/executor/human-in-the-loop/utils.ts index 0b2c5467cae..ef9c8501c7a 100644 --- a/apps/sim/executor/human-in-the-loop/utils.ts +++ b/apps/sim/executor/human-in-the-loop/utils.ts @@ -1,5 +1,5 @@ -import { PARALLEL } from '@/executor/constants' import type { ExecutionContext, LoopPauseScope, ParallelPauseScope } from '@/executor/types' +import { buildBranchNodeId } from '@/executor/utils/subflow-node-id-codec' interface NodeMetadataLike { nodeId: string @@ -17,7 +17,7 @@ export function generatePauseContextId( let contextId = baseBlockId if (typeof nodeMetadata.branchIndex === 'number') { - contextId = `${contextId}${PARALLEL.BRANCH.PREFIX}${nodeMetadata.branchIndex}${PARALLEL.BRANCH.SUFFIX}` + contextId = buildBranchNodeId(contextId, nodeMetadata.branchIndex) } if (loopScope) { diff --git a/apps/sim/executor/orchestrators/loop.ts b/apps/sim/executor/orchestrators/loop.ts index 54a4972f255..25e8fdcf7d3 100644 --- a/apps/sim/executor/orchestrators/loop.ts +++ b/apps/sim/executor/orchestrators/loop.ts @@ -18,18 +18,17 @@ import type { BlockStateController, ContextExtensions } from '@/executor/executi import type { ExecutionContext, NormalizedBlockOutput } from '@/executor/types' import { createReferencePattern } from '@/executor/utils/reference-validation' import { projectResolvedSecretDiagnosticError } from '@/executor/utils/resolved-secret-content-projection' -import { mergeSubflowSecretProvenance } from '@/executor/utils/subflow-secret-provenance' import { - addSubflowErrorLog, + buildLoopSentinelEndId, + buildLoopSentinelStartId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, - emitSubflowSuccessEvents, extractBaseBlockId, extractLoopIdFromSentinel, extractParallelIdFromSentinel, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' +import { mergeSubflowSecretProvenance } from '@/executor/utils/subflow-secret-provenance' +import { addSubflowErrorLog, emitSubflowSuccessEvents } from '@/executor/utils/subflow-utils' import { resolveArrayInputAsync } from '@/executor/utils/subflow-utils.server' import type { VariableResolver } from '@/executor/variables/resolver' @@ -149,7 +148,7 @@ export class LoopOrchestrator { resolutionCtx, loopConfig.forEachItems, this.resolver, - buildSentinelStartId(loopId) + buildLoopSentinelStartId(loopId) ) } catch (error) { const errorMessage = `ForEach loop resolution failed: ${toError(error).message}` @@ -508,8 +507,8 @@ export class LoopOrchestrator { const loopConfig = this.dag.loopConfigs.get(loopId) if (!loopConfig) return new Set() - const sentinelStartId = buildSentinelStartId(loopId) - const sentinelEndId = buildSentinelEndId(loopId) + const sentinelStartId = buildLoopSentinelStartId(loopId) + const sentinelEndId = buildLoopSentinelEndId(loopId) const result = new Set([sentinelStartId, sentinelEndId]) for (const nodeId of loopConfig.nodes) { @@ -650,8 +649,8 @@ export class LoopOrchestrator { const loopId = extractLoopIdFromSentinel(sourceId) return ( !!loopId && - sourceId === buildSentinelStartId(loopId) && - targetId === buildSentinelEndId(loopId) + sourceId === buildLoopSentinelStartId(loopId) && + targetId === buildLoopSentinelEndId(loopId) ) } diff --git a/apps/sim/executor/orchestrators/node.ts b/apps/sim/executor/orchestrators/node.ts index 56229908445..1c0467afa58 100644 --- a/apps/sim/executor/orchestrators/node.ts +++ b/apps/sim/executor/orchestrators/node.ts @@ -11,7 +11,7 @@ import { buildOuterBranchScopedId, extractBaseBlockId, extractOuterBranchIndex, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' const logger = createLogger('NodeExecutionOrchestrator') diff --git a/apps/sim/executor/orchestrators/parallel.test.ts b/apps/sim/executor/orchestrators/parallel.test.ts index ed90d662366..25a047a9be4 100644 --- a/apps/sim/executor/orchestrators/parallel.test.ts +++ b/apps/sim/executor/orchestrators/parallel.test.ts @@ -5,11 +5,11 @@ import { ParallelOrchestrator } from '@/executor/orchestrators/parallel' import type { ExecutionContext } from '@/executor/types' import { buildBranchNodeId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' const { mockCompactSubflowResults } = vi.hoisted(() => ({ mockCompactSubflowResults: vi.fn(async (results: unknown) => results), @@ -331,8 +331,8 @@ describe('ParallelOrchestrator', () => { const taskId = 'task-1' const parallelStartId = buildParallelSentinelStartId(parallelId) const parallelEndId = buildParallelSentinelEndId(parallelId) - const loopStartId = buildSentinelStartId(loopId) - const loopEndId = buildSentinelEndId(loopId) + const loopStartId = buildLoopSentinelStartId(loopId) + const loopEndId = buildLoopSentinelEndId(loopId) dag.parallelConfigs.set(parallelId, { id: parallelId, diff --git a/apps/sim/executor/orchestrators/parallel.ts b/apps/sim/executor/orchestrators/parallel.ts index 901b26cf5cc..d4336726a68 100644 --- a/apps/sim/executor/orchestrators/parallel.ts +++ b/apps/sim/executor/orchestrators/parallel.ts @@ -8,18 +8,17 @@ import type { ParallelScope } from '@/executor/execution/state' import type { BlockStateController, ContextExtensions } from '@/executor/execution/types' import type { ExecutionContext, NormalizedBlockOutput } from '@/executor/types' import { type ClonedSubflowInfo, ParallelExpander } from '@/executor/utils/parallel-expansion' -import { mergeSubflowSecretProvenance } from '@/executor/utils/subflow-secret-provenance' import { - addSubflowErrorLog, buildBranchNodeId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, - emitSubflowSuccessEvents, extractBaseBlockId, extractBranchIndex, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' +import { mergeSubflowSecretProvenance } from '@/executor/utils/subflow-secret-provenance' +import { addSubflowErrorLog, emitSubflowSuccessEvents } from '@/executor/utils/subflow-utils' import { resolveArrayInputAsync } from '@/executor/utils/subflow-utils.server' import type { VariableResolver } from '@/executor/variables/resolver' import type { SerializedParallel } from '@/serializer/types' @@ -384,8 +383,8 @@ export class ParallelOrchestrator { } if (this.dag.loopConfigs.has(subflowId)) { - nodeIds.add(buildSentinelStartId(subflowId)) - nodeIds.add(buildSentinelEndId(subflowId)) + nodeIds.add(buildLoopSentinelStartId(subflowId)) + nodeIds.add(buildLoopSentinelEndId(subflowId)) for (const childId of this.dag.loopConfigs.get(subflowId)?.nodes ?? []) { if (this.dag.parallelConfigs.has(childId) || this.dag.loopConfigs.has(childId)) { this.collectSubflowNodeIds(childId, nodeIds, visited) diff --git a/apps/sim/executor/utils/block-data.ts b/apps/sim/executor/utils/block-data.ts index 3fe339b45aa..7304a005af9 100644 --- a/apps/sim/executor/utils/block-data.ts +++ b/apps/sim/executor/utils/block-data.ts @@ -8,7 +8,7 @@ import { extractBaseBlockId, extractBranchIndex, isBranchNodeId, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock } from '@/serializer/types' export interface BlockDataCollection { diff --git a/apps/sim/executor/utils/iteration-context.ts b/apps/sim/executor/utils/iteration-context.ts index 17532cec57e..924854d2a86 100644 --- a/apps/sim/executor/utils/iteration-context.ts +++ b/apps/sim/executor/utils/iteration-context.ts @@ -2,7 +2,7 @@ import { DEFAULTS } from '@/executor/constants' import type { NodeMetadata } from '@/executor/dag/types' import type { IterationContext, ParentIteration } from '@/executor/execution/types' import type { ExecutionContext } from '@/executor/types' -import { findEffectiveContainerId } from '@/executor/utils/subflow-utils' +import { findEffectiveContainerId } from '@/executor/utils/subflow-node-id-codec' /** Maximum ancestor depth to prevent runaway traversal in deeply nested subflows. */ const MAX_PARENT_DEPTH = DEFAULTS.MAX_NESTING_DEPTH diff --git a/apps/sim/executor/utils/parallel-expansion.test.ts b/apps/sim/executor/utils/parallel-expansion.test.ts index 1783c2504fa..ba422513afe 100644 --- a/apps/sim/executor/utils/parallel-expansion.test.ts +++ b/apps/sim/executor/utils/parallel-expansion.test.ts @@ -8,7 +8,7 @@ import { buildParallelSentinelEndId, buildParallelSentinelStartId, stripCloneSuffixes, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock, SerializedWorkflow } from '@/serializer/types' function createBlock(id: string, metadataId: string): SerializedBlock { diff --git a/apps/sim/executor/utils/parallel-expansion.ts b/apps/sim/executor/utils/parallel-expansion.ts index e4f7182aad7..e0833f5257f 100644 --- a/apps/sim/executor/utils/parallel-expansion.ts +++ b/apps/sim/executor/utils/parallel-expansion.ts @@ -4,17 +4,17 @@ import { CONTROL_BACK_EDGE_HANDLES, EDGE } from '@/executor/constants' import type { DAG, DAGNode } from '@/executor/dag/builder' import { buildBranchNodeId, - buildClonedSubflowId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, + buildOuterBranchScopedId, buildParallelSentinelEndId, buildParallelSentinelStartId, - buildSentinelEndId, - buildSentinelStartId, extractBaseBlockId, isLoopSentinelNodeId, isParallelSentinelNodeId, normalizeNodeId, stripOuterBranchSuffix, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedBlock } from '@/serializer/types' const logger = createLogger('ParallelExpansion') @@ -344,7 +344,7 @@ export class ParallelExpander { : buildParallelSentinelEndId(blockId) } if (dag.loopConfigs.has(blockId)) { - return side === 'start' ? buildSentinelStartId(blockId) : buildSentinelEndId(blockId) + return side === 'start' ? buildLoopSentinelStartId(blockId) : buildLoopSentinelEndId(blockId) } return buildBranchNodeId(blockId, 0) } @@ -361,15 +361,15 @@ export class ParallelExpander { } const effectiveSubflowId = - globalBranchIndex === 0 ? blockId : buildClonedSubflowId(blockId, globalBranchIndex) + globalBranchIndex === 0 ? blockId : buildOuterBranchScopedId(blockId, globalBranchIndex) if (dag.parallelConfigs.has(blockId)) { return side === 'start' ? buildParallelSentinelStartId(effectiveSubflowId) : buildParallelSentinelEndId(effectiveSubflowId) } return side === 'start' - ? buildSentinelStartId(effectiveSubflowId) - : buildSentinelEndId(effectiveSubflowId) + ? buildLoopSentinelStartId(effectiveSubflowId) + : buildLoopSentinelEndId(effectiveSubflowId) } /** @@ -405,7 +405,7 @@ export class ParallelExpander { outerBranchIndex: number, clonedSubflows: ClonedSubflowInfo[] ): { startId: string; endId: string; clonedId: string; idMap: Map } { - const clonedId = buildClonedSubflowId(subflowId, outerBranchIndex) + const clonedId = buildOuterBranchScopedId(subflowId, outerBranchIndex) const { startId, endId, idMap } = this.cloneSubflowGraph( dag, subflowId, @@ -438,16 +438,16 @@ export class ParallelExpander { // Map sentinel nodes const origStartId = isParallel ? buildParallelSentinelStartId(originalId) - : buildSentinelStartId(originalId) + : buildLoopSentinelStartId(originalId) const origEndId = isParallel ? buildParallelSentinelEndId(originalId) - : buildSentinelEndId(originalId) + : buildLoopSentinelEndId(originalId) const clonedStartId = isParallel ? buildParallelSentinelStartId(clonedId) - : buildSentinelStartId(clonedId) + : buildLoopSentinelStartId(clonedId) const clonedEndId = isParallel ? buildParallelSentinelEndId(clonedId) - : buildSentinelEndId(clonedId) + : buildLoopSentinelEndId(clonedId) idMap.set(origStartId, clonedStartId) idMap.set(origEndId, clonedEndId) diff --git a/apps/sim/executor/utils/run-from-block.ts b/apps/sim/executor/utils/run-from-block.ts index a95885e6caf..08300e3e481 100644 --- a/apps/sim/executor/utils/run-from-block.ts +++ b/apps/sim/executor/utils/run-from-block.ts @@ -1,24 +1,14 @@ import { isPlainRecord } from '@sim/utils/object' -import { LOOP, normalizeName, PARALLEL } from '@/executor/constants' +import { normalizeName } from '@/executor/constants' import type { DAG } from '@/executor/dag/builder' import type { SerializableExecutionState } from '@/executor/execution/types' import type { NormalizedBlockOutput } from '@/executor/types' +import { + buildLoopSentinelStartId, + buildParallelSentinelStartId, +} from '@/executor/utils/subflow-node-id-codec' import type { SerializedWorkflow } from '@/serializer/types' -/** - * Builds the sentinel-start node ID for a loop. - */ -function buildLoopSentinelStartId(loopId: string): string { - return `${LOOP.SENTINEL.PREFIX}${loopId}${LOOP.SENTINEL.START_SUFFIX}` -} - -/** - * Builds the sentinel-start node ID for a parallel. - */ -function buildParallelSentinelStartId(parallelId: string): string { - return `${PARALLEL.SENTINEL.PREFIX}${parallelId}${PARALLEL.SENTINEL.START_SUFFIX}` -} - /** * Checks if a block ID is a loop or parallel container and returns the sentinel-start ID if so. * Returns null if the block is not a container. diff --git a/apps/sim/executor/utils/subflow-node-id-codec.test.ts b/apps/sim/executor/utils/subflow-node-id-codec.test.ts index b9fa8d4fda2..e867b180e74 100644 --- a/apps/sim/executor/utils/subflow-node-id-codec.test.ts +++ b/apps/sim/executor/utils/subflow-node-id-codec.test.ts @@ -1,96 +1,115 @@ import { describe, expect, it } from 'vitest' -import { SubflowNodeIdCodec } from '@/executor/utils/subflow-node-id-codec' +import { + buildBranchNodeId, + buildLoopSentinelEndId, + buildLoopSentinelStartId, + buildOuterBranchScopedId, + buildParallelSentinelEndId, + buildParallelSentinelStartId, + extractBaseBlockId, + extractBranchIndex, + extractBranchSuffix, + extractInnermostOuterBranchIndex, + extractLoopIdFromSentinel, + extractLoopSuffix, + extractOuterBranchIndex, + extractParallelIdFromSentinel, + findEffectiveContainerId, + isBranchNodeId, + isLoopSentinelNodeId, + isParallelSentinelNodeId, + normalizeLookupId, + normalizeNodeId, + stripCloneSuffixes, + stripOuterBranchSuffix, +} from '@/executor/utils/subflow-node-id-codec' -describe('SubflowNodeIdCodec', () => { +describe('subflow node id codec', () => { describe('branch subscripts', () => { it('builds and round-trips branch node IDs', () => { - const id = SubflowNodeIdCodec.buildBranchNodeId('block-1', 2) + const id = buildBranchNodeId('block-1', 2) expect(id).toBe('block-1₍2₎') - expect(SubflowNodeIdCodec.isBranchNodeId(id)).toBe(true) - expect(SubflowNodeIdCodec.extractBaseBlockId(id)).toBe('block-1') - expect(SubflowNodeIdCodec.extractBranchIndex(id)).toBe(2) + expect(isBranchNodeId(id)).toBe(true) + expect(extractBaseBlockId(id)).toBe('block-1') + expect(extractBranchIndex(id)).toBe(2) }) it('only strips a trailing branch subscript', () => { - expect(SubflowNodeIdCodec.extractBaseBlockId('a₍1₎b')).toBe('a₍1₎b') - expect(SubflowNodeIdCodec.extractBaseBlockId('a₍1₎b₍2₎')).toBe('a₍1₎b') + expect(extractBaseBlockId('a₍1₎b')).toBe('a₍1₎b') + expect(extractBaseBlockId('a₍1₎b₍2₎')).toBe('a₍1₎b') }) }) describe('loop sentinels', () => { it('builds and parses loop sentinel IDs', () => { - const start = SubflowNodeIdCodec.buildLoopSentinelStartId('loop-1') - const end = SubflowNodeIdCodec.buildLoopSentinelEndId('loop-1') + const start = buildLoopSentinelStartId('loop-1') + const end = buildLoopSentinelEndId('loop-1') expect(start).toBe('loop-loop-1-sentinel-start') expect(end).toBe('loop-loop-1-sentinel-end') - expect(SubflowNodeIdCodec.isLoopSentinelNodeId(start)).toBe(true) - expect(SubflowNodeIdCodec.isLoopSentinelNodeId(end)).toBe(true) - expect(SubflowNodeIdCodec.extractLoopIdFromSentinel(start)).toBe('loop-1') - expect(SubflowNodeIdCodec.extractLoopIdFromSentinel(end)).toBe('loop-1') + expect(isLoopSentinelNodeId(start)).toBe(true) + expect(isLoopSentinelNodeId(end)).toBe(true) + expect(extractLoopIdFromSentinel(start)).toBe('loop-1') + expect(extractLoopIdFromSentinel(end)).toBe('loop-1') }) }) describe('parallel sentinels', () => { it('builds and parses parallel sentinel IDs', () => { - const start = SubflowNodeIdCodec.buildParallelSentinelStartId('p-1') - const end = SubflowNodeIdCodec.buildParallelSentinelEndId('p-1') + const start = buildParallelSentinelStartId('p-1') + const end = buildParallelSentinelEndId('p-1') expect(start).toBe('parallel-p-1-sentinel-start') expect(end).toBe('parallel-p-1-sentinel-end') - expect(SubflowNodeIdCodec.isParallelSentinelNodeId(start)).toBe(true) - expect(SubflowNodeIdCodec.isParallelSentinelNodeId(end)).toBe(true) - expect(SubflowNodeIdCodec.extractParallelIdFromSentinel(start)).toBe('p-1') - expect(SubflowNodeIdCodec.extractParallelIdFromSentinel(end)).toBe('p-1') + expect(isParallelSentinelNodeId(start)).toBe(true) + expect(isParallelSentinelNodeId(end)).toBe(true) + expect(extractParallelIdFromSentinel(start)).toBe('p-1') + expect(extractParallelIdFromSentinel(end)).toBe('p-1') }) }) describe('outer-branch clone scoping', () => { it('builds and extracts outer branch index', () => { - const id = SubflowNodeIdCodec.buildOuterBranchScopedId('loop-1', 3) + const id = buildOuterBranchScopedId('loop-1', 3) expect(id).toBe('loop-1__obranch-3') - expect(SubflowNodeIdCodec.extractOuterBranchIndex(id)).toBe(3) + expect(extractOuterBranchIndex(id)).toBe(3) }) it('extracts the innermost outer branch index for nested clones', () => { const id = 'loop-1__obranch-2__obranch-5' - expect(SubflowNodeIdCodec.extractOuterBranchIndex(id)).toBe(2) - expect(SubflowNodeIdCodec.extractInnermostOuterBranchIndex(id)).toBe(5) + expect(extractOuterBranchIndex(id)).toBe(2) + expect(extractInnermostOuterBranchIndex(id)).toBe(5) }) it('strips outer-branch and clone-digest suffixes', () => { - expect(SubflowNodeIdCodec.stripOuterBranchSuffix('loop-1__obranch-2')).toBe('loop-1') - expect(SubflowNodeIdCodec.stripOuterBranchSuffix('loop-1__cloneABCDEF__obranch-2')).toBe( - 'loop-1' - ) + expect(stripOuterBranchSuffix('loop-1__obranch-2')).toBe('loop-1') + expect(stripOuterBranchSuffix('loop-1__cloneABCDEF__obranch-2')).toBe('loop-1') }) it('strips all clone suffixes and branch subscripts to the base block ID', () => { - expect(SubflowNodeIdCodec.stripCloneSuffixes('block-1__obranch-2₍3₎')).toBe('block-1') - expect(SubflowNodeIdCodec.stripCloneSuffixes('block-1__clone0a1f__obranch-2₍0₎')).toBe( - 'block-1' - ) + expect(stripCloneSuffixes('block-1__obranch-2₍3₎')).toBe('block-1') + expect(stripCloneSuffixes('block-1__clone0a1f__obranch-2₍0₎')).toBe('block-1') }) }) describe('normalizeNodeId', () => { it('normalizes branch, loop sentinel, and parallel sentinel IDs', () => { - expect(SubflowNodeIdCodec.normalizeNodeId('block-1₍2₎')).toBe('block-1') - expect(SubflowNodeIdCodec.normalizeNodeId('loop-loop-1-sentinel-start')).toBe('loop-1') - expect(SubflowNodeIdCodec.normalizeNodeId('parallel-p-1-sentinel-end')).toBe('p-1') - expect(SubflowNodeIdCodec.normalizeNodeId('block-1')).toBe('block-1') + expect(normalizeNodeId('block-1₍2₎')).toBe('block-1') + expect(normalizeNodeId('loop-loop-1-sentinel-start')).toBe('loop-1') + expect(normalizeNodeId('parallel-p-1-sentinel-end')).toBe('p-1') + expect(normalizeNodeId('block-1')).toBe('block-1') }) }) describe('loop digest lookup helpers', () => { it('strips branch subscripts and loop digests for lookup keys', () => { - expect(SubflowNodeIdCodec.normalizeLookupId('block-1₍2₎_loop3')).toBe('block-1') - expect(SubflowNodeIdCodec.normalizeLookupId('block-1')).toBe('block-1') + expect(normalizeLookupId('block-1₍2₎_loop3')).toBe('block-1') + expect(normalizeLookupId('block-1')).toBe('block-1') }) it('extracts the leading branch suffix and loop digest segments', () => { - expect(SubflowNodeIdCodec.extractBranchSuffix('block-1₍2₎_loop3')).toBe('₍2₎') - expect(SubflowNodeIdCodec.extractBranchSuffix('block-1')).toBe('') - expect(SubflowNodeIdCodec.extractLoopSuffix('block-1₍2₎_loop3')).toBe('_loop3') - expect(SubflowNodeIdCodec.extractLoopSuffix('block-1')).toBe('') + expect(extractBranchSuffix('block-1₍2₎_loop3')).toBe('₍2₎') + expect(extractBranchSuffix('block-1')).toBe('') + expect(extractLoopSuffix('block-1₍2₎_loop3')).toBe('_loop3') + expect(extractLoopSuffix('block-1')).toBe('') }) }) @@ -100,9 +119,7 @@ describe('SubflowNodeIdCodec', () => { ['loop-1', {}], ['loop-1__obranch-2', {}], ]) - expect(SubflowNodeIdCodec.findEffectiveContainerId('loop-1', 'block-1', map, 2)).toBe( - 'loop-1__obranch-2' - ) + expect(findEffectiveContainerId('loop-1', 'block-1', map, 2)).toBe('loop-1__obranch-2') }) it('resolves the cloned scope from the current node ID suffix', () => { @@ -110,7 +127,7 @@ describe('SubflowNodeIdCodec', () => { ['loop-1', {}], ['loop-1__obranch-3', {}], ]) - expect(SubflowNodeIdCodec.findEffectiveContainerId('loop-1', 'block-1__obranch-3', map)).toBe( + expect(findEffectiveContainerId('loop-1', 'block-1__obranch-3', map)).toBe( 'loop-1__obranch-3' ) }) @@ -120,9 +137,9 @@ describe('SubflowNodeIdCodec', () => { ['loop-1__obranch-2', {}], ['loop-1__cloneabc__obranch-2', {}], ]) - expect( - SubflowNodeIdCodec.findEffectiveContainerId('loop-1', 'block-1__cloneabc__obranch-2', map) - ).toBe('loop-1__cloneabc__obranch-2') + expect(findEffectiveContainerId('loop-1', 'block-1__cloneabc__obranch-2', map)).toBe( + 'loop-1__cloneabc__obranch-2' + ) }) }) }) diff --git a/apps/sim/executor/utils/subflow-node-id-codec.ts b/apps/sim/executor/utils/subflow-node-id-codec.ts index 8b7aa2ecdff..7705cac9df0 100644 --- a/apps/sim/executor/utils/subflow-node-id-codec.ts +++ b/apps/sim/executor/utils/subflow-node-id-codec.ts @@ -47,39 +47,39 @@ const LOOP_DIGEST = { /** * Builds the loop sentinel-start node ID for a container. */ -function buildLoopSentinelStartId(loopId: string): string { +export function buildLoopSentinelStartId(loopId: string): string { return `${LOOP.SENTINEL.PREFIX}${loopId}${LOOP.SENTINEL.START_SUFFIX}` } /** * Builds the loop sentinel-end node ID for a container. */ -function buildLoopSentinelEndId(loopId: string): string { +export function buildLoopSentinelEndId(loopId: string): string { return `${LOOP.SENTINEL.PREFIX}${loopId}${LOOP.SENTINEL.END_SUFFIX}` } /** * Builds the parallel sentinel-start node ID for a container. */ -function buildParallelSentinelStartId(parallelId: string): string { +export function buildParallelSentinelStartId(parallelId: string): string { return `${PARALLEL.SENTINEL.PREFIX}${parallelId}${PARALLEL.SENTINEL.START_SUFFIX}` } /** * Builds the parallel sentinel-end node ID for a container. */ -function buildParallelSentinelEndId(parallelId: string): string { +export function buildParallelSentinelEndId(parallelId: string): string { return `${PARALLEL.SENTINEL.PREFIX}${parallelId}${PARALLEL.SENTINEL.END_SUFFIX}` } -function isLoopSentinelNodeId(nodeId: string): boolean { +export function isLoopSentinelNodeId(nodeId: string): boolean { return ( nodeId.startsWith(LOOP.SENTINEL.PREFIX) && (nodeId.endsWith(LOOP.SENTINEL.START_SUFFIX) || nodeId.endsWith(LOOP.SENTINEL.END_SUFFIX)) ) } -function isParallelSentinelNodeId(nodeId: string): boolean { +export function isParallelSentinelNodeId(nodeId: string): boolean { return ( nodeId.startsWith(PARALLEL.SENTINEL.PREFIX) && (nodeId.endsWith(PARALLEL.SENTINEL.START_SUFFIX) || @@ -87,7 +87,7 @@ function isParallelSentinelNodeId(nodeId: string): boolean { ) } -function extractLoopIdFromSentinel(sentinelId: string): string | null { +export function extractLoopIdFromSentinel(sentinelId: string): string | null { const startMatch = sentinelId.match(SENTINEL.LOOP_START) if (startMatch) return startMatch[1] const endMatch = sentinelId.match(SENTINEL.LOOP_END) @@ -95,7 +95,7 @@ function extractLoopIdFromSentinel(sentinelId: string): string | null { return null } -function extractParallelIdFromSentinel(sentinelId: string): string | null { +export function extractParallelIdFromSentinel(sentinelId: string): string | null { const startMatch = sentinelId.match(SENTINEL.PARALLEL_START) if (startMatch) return startMatch[1] const endMatch = sentinelId.match(SENTINEL.PARALLEL_END) @@ -103,43 +103,43 @@ function extractParallelIdFromSentinel(sentinelId: string): string | null { return null } -function buildBranchNodeId(baseId: string, branchIndex: number): string { +export function buildBranchNodeId(baseId: string, branchIndex: number): string { return `${baseId}${PARALLEL.BRANCH.PREFIX}${branchIndex}${PARALLEL.BRANCH.SUFFIX}` } -function extractBaseBlockId(branchNodeId: string): string { +export function extractBaseBlockId(branchNodeId: string): string { return branchNodeId.replace(BRANCH.MATCH, '') } -function extractBranchIndex(branchNodeId: string): number | null { +export function extractBranchIndex(branchNodeId: string): number | null { const match = branchNodeId.match(BRANCH.INDEX) return match ? Number.parseInt(match[1], 10) : null } -function isBranchNodeId(nodeId: string): boolean { +export function isBranchNodeId(nodeId: string): boolean { return BRANCH.MATCH.test(nodeId) } -function extractOuterBranchIndex(clonedId: string): number | undefined { +export function extractOuterBranchIndex(clonedId: string): number | undefined { const match = clonedId.match(OUTER_BRANCH.MATCH) return match ? Number.parseInt(match[1], 10) : undefined } -function extractInnermostOuterBranchIndex(clonedId: string): number | undefined { +export function extractInnermostOuterBranchIndex(clonedId: string): number | undefined { const matches = Array.from(clonedId.matchAll(OUTER_BRANCH.MATCH_GLOBAL)) const lastMatch = matches.at(-1) return lastMatch ? Number.parseInt(lastMatch[1], 10) : undefined } -function stripCloneSuffixes(nodeId: string): string { +export function stripCloneSuffixes(nodeId: string): string { return extractBaseBlockId(nodeId.replace(OUTER_BRANCH.STRIP, '').replace(CLONE.DIGEST_STRIP, '')) } -function buildOuterBranchScopedId(originalId: string, branchIndex: number): string { +export function buildOuterBranchScopedId(originalId: string, branchIndex: number): string { return `${originalId}__obranch-${branchIndex}` } -function stripOuterBranchSuffix(id: string): string { +export function stripOuterBranchSuffix(id: string): string { return id.replace(OUTER_BRANCH.STRIP, '').replace(CLONE.DIGEST_STRIP, '') } @@ -147,7 +147,7 @@ function hasCloneMarker(id: string): boolean { return id.includes(CLONE.MARKER) } -function normalizeNodeId(nodeId: string): string { +export function normalizeNodeId(nodeId: string): string { if (isBranchNodeId(nodeId)) { return extractBaseBlockId(nodeId) } @@ -160,7 +160,7 @@ function normalizeNodeId(nodeId: string): string { return nodeId } -function findEffectiveContainerId( +export function findEffectiveContainerId( originalId: string, currentNodeId: string, executionMap: Map, @@ -215,49 +215,20 @@ function findEffectiveContainerId( * Strips branch subscripts (`₍N₎`) and loop digests (`_loopN`) from a node ID, * yielding the lookup key used by execution-state block-output resolution. */ -function normalizeLookupId(id: string): string { +export function normalizeLookupId(id: string): string { return id.replace(BRANCH.SUFFIX_GLOBAL, '').replace(LOOP_DIGEST.STRIP, '') } /** * Returns the leading branch subscript (`₍N₎`) of a node ID, or '' when absent. */ -function extractBranchSuffix(id: string): string { +export function extractBranchSuffix(id: string): string { return id.match(BRANCH.SUFFIX)?.[0] ?? '' } /** * Returns the loop digest segment (`_loopN`) of a node ID, or '' when absent. */ -function extractLoopSuffix(id: string): string { +export function extractLoopSuffix(id: string): string { return id.match(LOOP_DIGEST.MATCH)?.[0] ?? '' } - -/** - * Codec exposing all subflow node-ID parsing/building operations as a single, - * pattern-free interface. Implementation owns every regex and string template. - */ -export const SubflowNodeIdCodec = { - buildLoopSentinelStartId, - buildLoopSentinelEndId, - buildParallelSentinelStartId, - buildParallelSentinelEndId, - isLoopSentinelNodeId, - isParallelSentinelNodeId, - extractLoopIdFromSentinel, - extractParallelIdFromSentinel, - buildBranchNodeId, - extractBaseBlockId, - extractBranchIndex, - isBranchNodeId, - extractOuterBranchIndex, - extractInnermostOuterBranchIndex, - stripCloneSuffixes, - buildOuterBranchScopedId, - stripOuterBranchSuffix, - normalizeNodeId, - findEffectiveContainerId, - normalizeLookupId, - extractBranchSuffix, - extractLoopSuffix, -} as const diff --git a/apps/sim/executor/utils/subflow-utils.test.ts b/apps/sim/executor/utils/subflow-utils.test.ts index 251293aa47b..55b3ba6329e 100644 --- a/apps/sim/executor/utils/subflow-utils.test.ts +++ b/apps/sim/executor/utils/subflow-utils.test.ts @@ -6,7 +6,7 @@ import { } from '@/lib/execution/payloads/large-array-manifest-metadata' import { LARGE_VALUE_REF_MARKER } from '@/lib/execution/payloads/large-value-ref' import type { ExecutionContext } from '@/executor/types' -import { findEffectiveContainerId } from '@/executor/utils/subflow-utils' +import { findEffectiveContainerId } from '@/executor/utils/subflow-node-id-codec' import { resolveArrayInputAsync } from '@/executor/utils/subflow-utils.server' import type { VariableResolver } from '@/executor/variables/resolver' diff --git a/apps/sim/executor/utils/subflow-utils.ts b/apps/sim/executor/utils/subflow-utils.ts index 0dc00d93466..a7821747655 100644 --- a/apps/sim/executor/utils/subflow-utils.ts +++ b/apps/sim/executor/utils/subflow-utils.ts @@ -4,141 +4,10 @@ import { DEFAULTS } from '@/executor/constants' import type { ContextExtensions } from '@/executor/execution/types' import { type BlockLog, type ExecutionContext, getNextExecutionOrder } from '@/executor/types' import { buildContainerIterationContext } from '@/executor/utils/iteration-context' -import { SubflowNodeIdCodec } from '@/executor/utils/subflow-node-id-codec' import type { SerializedWorkflow } from '@/serializer/types' const logger = createLogger('SubflowUtils') -/** - * Builds the loop sentinel-start node ID for a container. - */ -export function buildSentinelStartId(loopId: string): string { - return SubflowNodeIdCodec.buildLoopSentinelStartId(loopId) -} - -/** - * Builds the loop sentinel-end node ID for a container. - */ -export function buildSentinelEndId(loopId: string): string { - return SubflowNodeIdCodec.buildLoopSentinelEndId(loopId) -} - -export function buildParallelSentinelStartId(parallelId: string): string { - return SubflowNodeIdCodec.buildParallelSentinelStartId(parallelId) -} - -export function buildParallelSentinelEndId(parallelId: string): string { - return SubflowNodeIdCodec.buildParallelSentinelEndId(parallelId) -} - -export function isLoopSentinelNodeId(nodeId: string): boolean { - return SubflowNodeIdCodec.isLoopSentinelNodeId(nodeId) -} - -export function isParallelSentinelNodeId(nodeId: string): boolean { - return SubflowNodeIdCodec.isParallelSentinelNodeId(nodeId) -} - -export function extractLoopIdFromSentinel(sentinelId: string): string | null { - return SubflowNodeIdCodec.extractLoopIdFromSentinel(sentinelId) -} - -export function extractParallelIdFromSentinel(sentinelId: string): string | null { - return SubflowNodeIdCodec.extractParallelIdFromSentinel(sentinelId) -} - -/** - * Build branch node ID with subscript notation - * Example: ("blockId", 2) → "blockId₍2₎" - */ -export function buildBranchNodeId(baseId: string, branchIndex: number): string { - return SubflowNodeIdCodec.buildBranchNodeId(baseId, branchIndex) -} - -export function extractBaseBlockId(branchNodeId: string): string { - return SubflowNodeIdCodec.extractBaseBlockId(branchNodeId) -} - -export function extractBranchIndex(branchNodeId: string): number | null { - return SubflowNodeIdCodec.extractBranchIndex(branchNodeId) -} - -export function isBranchNodeId(nodeId: string): boolean { - return SubflowNodeIdCodec.isBranchNodeId(nodeId) -} - -/** - * Extracts the outer branch index from a cloned subflow ID. - * Cloned IDs follow the pattern `{originalId}__obranch-{index}`. - * Returns undefined if the ID is not a clone. - */ -export function extractOuterBranchIndex(clonedId: string): number | undefined { - return SubflowNodeIdCodec.extractOuterBranchIndex(clonedId) -} - -export function extractInnermostOuterBranchIndex(clonedId: string): number | undefined { - return SubflowNodeIdCodec.extractInnermostOuterBranchIndex(clonedId) -} - -/** - * Strips all clone suffixes (`__obranch-N`) and branch subscripts (`₍N₎`) - * from a node ID, returning the original workflow-level block ID. - */ -export function stripCloneSuffixes(nodeId: string): string { - return SubflowNodeIdCodec.stripCloneSuffixes(nodeId) -} - -/** - * Builds a stable ID for an output scoped to a global outer parallel branch. - */ -export function buildOuterBranchScopedId(originalId: string, branchIndex: number): string { - return SubflowNodeIdCodec.buildOuterBranchScopedId(originalId, branchIndex) -} - -/** - * Builds a cloned subflow ID from an original ID and outer branch index. - */ -export function buildClonedSubflowId(originalId: string, branchIndex: number): string { - return SubflowNodeIdCodec.buildOuterBranchScopedId(originalId, branchIndex) -} - -/** - * Strips outer-branch clone suffixes (`__obranch-N`) from an ID, - * returning the original workflow-level subflow ID. - */ -export function stripOuterBranchSuffix(id: string): string { - return SubflowNodeIdCodec.stripOuterBranchSuffix(id) -} - -/** - * Finds the effective (possibly cloned) container ID for a subflow, - * given the current node's ID and an execution map (loopExecutions or parallelExecutions). - * - * When inside a cloned subflow (e.g., loop-1__obranch-2), the execution scope is - * stored under the cloned ID, not the original. This function extracts the `__obranch-N` - * suffix from the current node ID, constructs the candidate cloned container ID, and - * checks if it exists in the execution map. - * - * Returns the effective ID (cloned or original) that exists in the map. - */ -export function findEffectiveContainerId( - originalId: string, - currentNodeId: string, - executionMap: Map, - mappedBranchIndex?: number -): string { - return SubflowNodeIdCodec.findEffectiveContainerId( - originalId, - currentNodeId, - executionMap, - mappedBranchIndex - ) -} - -export function normalizeNodeId(nodeId: string): string { - return SubflowNodeIdCodec.normalizeNodeId(nodeId) -} - type SubflowContainerType = 'loop' | 'parallel' function getSubflowNodes( diff --git a/apps/sim/executor/variables/resolvers/block.ts b/apps/sim/executor/variables/resolvers/block.ts index 2c3f634e08b..bf08d14fa02 100644 --- a/apps/sim/executor/variables/resolvers/block.ts +++ b/apps/sim/executor/variables/resolvers/block.ts @@ -14,7 +14,10 @@ import { resolveBlockReferenceAsync, } from '@/executor/utils/block-reference' import { formatInertStringLiteral, formatLiteralForCode } from '@/executor/utils/code-formatting' -import { buildClonedSubflowId, extractOuterBranchIndex } from '@/executor/utils/subflow-utils' +import { + buildOuterBranchScopedId, + extractOuterBranchIndex, +} from '@/executor/utils/subflow-node-id-codec' import { type AsyncPathNavigator, navigatePath, @@ -355,7 +358,7 @@ export class BlockResolver implements Resolver { if (shouldResolveClonedSubflowOutput) { const clonedState = context.executionState.getBlockState( - buildClonedSubflowId(blockId, mappedBranchIndex) + buildOuterBranchScopedId(blockId, mappedBranchIndex) ) if (clonedState !== undefined) { return clonedState diff --git a/apps/sim/executor/variables/resolvers/loop.ts b/apps/sim/executor/variables/resolvers/loop.ts index 523af3cfc9f..0295e89abd5 100644 --- a/apps/sim/executor/variables/resolvers/loop.ts +++ b/apps/sim/executor/variables/resolvers/loop.ts @@ -7,11 +7,10 @@ import { extractInnermostOuterBranchIndex, extractOuterBranchIndex, findEffectiveContainerId, - isSubflowNestedInside, stripCloneSuffixes, stripOuterBranchSuffix, - subflowContainsBlock, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' +import { isSubflowNestedInside, subflowContainsBlock } from '@/executor/utils/subflow-utils' import { type AsyncPathNavigator, navigatePath, diff --git a/apps/sim/executor/variables/resolvers/parallel.ts b/apps/sim/executor/variables/resolvers/parallel.ts index 12c95cf9ca8..f8d8be6d3c7 100644 --- a/apps/sim/executor/variables/resolvers/parallel.ts +++ b/apps/sim/executor/variables/resolvers/parallel.ts @@ -8,11 +8,10 @@ import { extractInnermostOuterBranchIndex, extractOuterBranchIndex, findEffectiveContainerId, - isSubflowNestedInside, stripCloneSuffixes, stripOuterBranchSuffix, - subflowContainsBlock, -} from '@/executor/utils/subflow-utils' +} from '@/executor/utils/subflow-node-id-codec' +import { isSubflowNestedInside, subflowContainsBlock } from '@/executor/utils/subflow-utils' import { type AsyncPathNavigator, navigatePath, diff --git a/apps/sim/lib/logs/execution/trace-spans/iteration-grouping.ts b/apps/sim/lib/logs/execution/trace-spans/iteration-grouping.ts index 832b5eeeca7..54bcff496f4 100644 --- a/apps/sim/lib/logs/execution/trace-spans/iteration-grouping.ts +++ b/apps/sim/lib/logs/execution/trace-spans/iteration-grouping.ts @@ -1,6 +1,6 @@ import { createLogger } from '@sim/logger' import type { TraceSpan } from '@/lib/logs/types' -import { stripCloneSuffixes } from '@/executor/utils/subflow-utils' +import { stripCloneSuffixes } from '@/executor/utils/subflow-node-id-codec' const logger = createLogger('IterationGrouping') diff --git a/apps/sim/lib/workflows/executor/execution-core.ts b/apps/sim/lib/workflows/executor/execution-core.ts index 12cbbe5c8f0..37d9759f793 100644 --- a/apps/sim/lib/workflows/executor/execution-core.ts +++ b/apps/sim/lib/workflows/executor/execution-core.ts @@ -67,7 +67,10 @@ import { type ResolvedSecretTraceRegistry, } from '@/executor/utils/resolved-secret-trace-registry' import { isRunMetadataEnabled } from '@/executor/utils/start-block' -import { buildParallelSentinelEndId, buildSentinelEndId } from '@/executor/utils/subflow-utils' +import { + buildLoopSentinelEndId, + buildParallelSentinelEndId, +} from '@/executor/utils/subflow-node-id-codec' import { Serializer } from '@/serializer' const logger = createLogger('ExecutionCore') @@ -901,7 +904,7 @@ async function executeWorkflowCoreImpl( let resolvedStopAfterBlockId = stopAfterBlockId if (stopAfterBlockId) { if (serializedWorkflow.loops?.[stopAfterBlockId]) { - resolvedStopAfterBlockId = buildSentinelEndId(stopAfterBlockId) + resolvedStopAfterBlockId = buildLoopSentinelEndId(stopAfterBlockId) } else if (serializedWorkflow.parallels?.[stopAfterBlockId]) { resolvedStopAfterBlockId = buildParallelSentinelEndId(stopAfterBlockId) } From 472a0d0ca77827d726eef3d397d78f3833d4d29d Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:16:09 -0700 Subject: [PATCH 16/30] refactor(providers): drop pass-through model wrappers and share the json_schema response format builder --- .../components/tool-input/tool-input.tsx | 4 +- .../sim/lib/permission-groups/model-access.ts | 2 +- apps/sim/providers/azure-openai/index.ts | 10 +- apps/sim/providers/baseten/index.ts | 11 +- apps/sim/providers/cerebras/index.ts | 10 +- apps/sim/providers/fireworks/index.ts | 11 +- apps/sim/providers/groq/index.ts | 10 +- apps/sim/providers/index.ts | 3 +- apps/sim/providers/meta/index.ts | 10 +- apps/sim/providers/mistral/index.ts | 10 +- apps/sim/providers/models.ts | 10 +- apps/sim/providers/nvidia/index.ts | 10 +- apps/sim/providers/openrouter/index.ts | 11 +- apps/sim/providers/sakana/index.ts | 10 +- apps/sim/providers/utils.test.ts | 10 +- apps/sim/providers/utils.ts | 134 ++++++------------ apps/sim/providers/vllm/index.ts | 10 +- apps/sim/tools/params-resolver.test.ts | 42 ------ .../testing/src/mocks/providers-utils.mock.ts | 39 +++-- 19 files changed, 107 insertions(+), 250 deletions(-) delete mode 100644 apps/sim/tools/params-resolver.test.ts diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx index 241f0c1f0bf..a9bada878cb 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx @@ -90,8 +90,8 @@ import { useCollaborativeWorkflow } from '@/hooks/use-collaborative-workflow' import { useOperationAccess } from '@/hooks/use-operation-access' import { usePermissionConfig } from '@/hooks/use-permission-config' import { useSettingsNavigation } from '@/hooks/use-settings-navigation' -import { supportsForcedToolUse } from '@/providers/models' -import { getProviderFromModel, supportsToolUsageControl } from '@/providers/utils' +import { supportsForcedToolUse, supportsToolUsageControl } from '@/providers/models' +import { getProviderFromModel } from '@/providers/utils' import type { ActiveSearchTarget } from '@/stores/panel/editor/store' import { useSubBlockStore } from '@/stores/workflows/subblock/store' import { useWorkflowStore } from '@/stores/workflows/workflow/store' diff --git a/apps/sim/lib/permission-groups/model-access.ts b/apps/sim/lib/permission-groups/model-access.ts index b78f98fd4e2..9f834e727aa 100644 --- a/apps/sim/lib/permission-groups/model-access.ts +++ b/apps/sim/lib/permission-groups/model-access.ts @@ -1,5 +1,5 @@ import type { PermissionGroupConfig } from '@/lib/permission-groups/fields' -import { findProviderFromModel } from '@/providers/utils' +import { findProviderFromModel } from '@/providers/models' /** Decides whether the caller's permission group allows a concrete model id. */ export type IsModelUsable = (model: string) => boolean diff --git a/apps/sim/providers/azure-openai/index.ts b/apps/sim/providers/azure-openai/index.ts index 4d922b52237..9b84b384641 100644 --- a/apps/sim/providers/azure-openai/index.ts +++ b/apps/sim/providers/azure-openai/index.ts @@ -57,6 +57,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, checkForForcedToolUsageOpenAI, isFunctionToolCall, @@ -176,14 +177,7 @@ async function executeChatCompletionsRequest( payload.verbosity = request.verbosity as ChatCompletionVerbosity if (request.responseFormat) { - payload.response_format = { - type: 'json_schema', - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + payload.response_format = buildJsonSchemaResponseFormat(request.responseFormat) logger.info('Added JSON schema response format to Azure OpenAI request') } diff --git a/apps/sim/providers/baseten/index.ts b/apps/sim/providers/baseten/index.ts index bebbf0e4975..7fd51d58611 100644 --- a/apps/sim/providers/baseten/index.ts +++ b/apps/sim/providers/baseten/index.ts @@ -9,6 +9,7 @@ import { } from '@/providers/openai-compat/chat-completions' import { openAICompatTransport } from '@/providers/transport' import type { ProviderConfig, ProviderRequest, ProviderResponse } from '@/providers/types' +import { buildJsonSchemaResponseFormat } from '@/providers/utils' const logger = createLogger('BasetenProvider') @@ -20,13 +21,9 @@ async function applyResponseFormat( model: string ): Promise { logger.info('Using native structured outputs for Baseten model', { model }) - targetPayload.response_format = { - type: 'json_schema', - json_schema: { - name: responseFormat.name || 'response_schema', - schema: responseFormat.schema || responseFormat, - }, - } + targetPayload.response_format = buildJsonSchemaResponseFormat(responseFormat, { + includeStrict: false, + }) return messages } diff --git a/apps/sim/providers/cerebras/index.ts b/apps/sim/providers/cerebras/index.ts index 17de3ce5578..317cacfb489 100644 --- a/apps/sim/providers/cerebras/index.ts +++ b/apps/sim/providers/cerebras/index.ts @@ -33,6 +33,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, @@ -99,14 +100,7 @@ export const cerebrasProvider: ProviderConfig = { payload.reasoning_effort = request.reasoningEffort } if (request.responseFormat) { - payload.response_format = { - type: 'json_schema', - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + payload.response_format = buildJsonSchemaResponseFormat(request.responseFormat) } let originalToolChoice: any diff --git a/apps/sim/providers/fireworks/index.ts b/apps/sim/providers/fireworks/index.ts index 866b9e7640c..28e31fb77bd 100644 --- a/apps/sim/providers/fireworks/index.ts +++ b/apps/sim/providers/fireworks/index.ts @@ -10,6 +10,7 @@ import { } from '@/providers/openai-compat/chat-completions' import { openAICompatTransport } from '@/providers/transport' import type { ProviderConfig, ProviderRequest, ProviderResponse } from '@/providers/types' +import { buildJsonSchemaResponseFormat } from '@/providers/utils' const logger = createLogger('FireworksProvider') @@ -21,13 +22,9 @@ async function applyResponseFormat( model: string ): Promise { logger.info('Using native structured outputs for Fireworks model', { model }) - targetPayload.response_format = { - type: 'json_schema', - json_schema: { - name: responseFormat.name || 'response_schema', - schema: responseFormat.schema || responseFormat, - }, - } + targetPayload.response_format = buildJsonSchemaResponseFormat(responseFormat, { + includeStrict: false, + }) return messages } diff --git a/apps/sim/providers/groq/index.ts b/apps/sim/providers/groq/index.ts index 4d27d293896..638291cb196 100644 --- a/apps/sim/providers/groq/index.ts +++ b/apps/sim/providers/groq/index.ts @@ -37,6 +37,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, @@ -121,14 +122,7 @@ export const groqProvider: ProviderConfig = { } if (request.responseFormat) { - payload.response_format = { - type: 'json_schema', - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + payload.response_format = buildJsonSchemaResponseFormat(request.responseFormat) } let originalToolChoice: any diff --git a/apps/sim/providers/index.ts b/apps/sim/providers/index.ts index c616100d449..90810114ab6 100644 --- a/apps/sim/providers/index.ts +++ b/apps/sim/providers/index.ts @@ -35,7 +35,7 @@ import { attachLargeFileRemoteUrls, uploadLargeFilesToProvider, } from '@/providers/file-attachments.server' -import { isEvaluationModel, isKnownModelId } from '@/providers/models' +import { isEvaluationModel, isKnownModelId, supportsTemperature } from '@/providers/models' import { getProviderExecutor } from '@/providers/registry' import { type ProviderRuntimeContext, @@ -54,7 +54,6 @@ import { sumToolCosts, supportsPromptCaching, supportsReasoningEffort, - supportsTemperature, supportsThinking, supportsVerbosity, } from '@/providers/utils' diff --git a/apps/sim/providers/meta/index.ts b/apps/sim/providers/meta/index.ts index 6f1f496e648..dfa5ff07061 100644 --- a/apps/sim/providers/meta/index.ts +++ b/apps/sim/providers/meta/index.ts @@ -32,6 +32,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, @@ -105,14 +106,7 @@ export const metaProvider: ProviderConfig = { } const responseFormatPayload = request.responseFormat - ? { - type: 'json_schema' as const, - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + ? buildJsonSchemaResponseFormat(request.responseFormat) : undefined let preparedTools: ReturnType | null = null diff --git a/apps/sim/providers/mistral/index.ts b/apps/sim/providers/mistral/index.ts index f1fdf44eaa0..66a901e398b 100644 --- a/apps/sim/providers/mistral/index.ts +++ b/apps/sim/providers/mistral/index.ts @@ -32,6 +32,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, @@ -110,14 +111,7 @@ export const mistralProvider: ProviderConfig = { if (request.maxTokens != null) payload.max_tokens = request.maxTokens if (request.responseFormat) { - payload.response_format = { - type: 'json_schema', - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + payload.response_format = buildJsonSchemaResponseFormat(request.responseFormat) } let preparedTools: ReturnType | null = null diff --git a/apps/sim/providers/models.ts b/apps/sim/providers/models.ts index dfd4a70511a..7baa6de5817 100644 --- a/apps/sim/providers/models.ts +++ b/apps/sim/providers/models.ts @@ -5762,7 +5762,15 @@ export function getBaseModelProviders(): Record { ) } -/** Resolves catalog entries and provider patterns without guessing a fallback provider. */ +/** + * The provider that declares `model`, or `null` when none does. Resolves catalog entries and + * provider patterns without guessing a fallback provider. + * + * The non-guessing half of `getProviderFromModel` in `@/providers/utils`. A caller that *gates* + * on the answer needs "unknown" to stay distinct from "ollama": this registry holds chat models + * only, so every embedding, speech, image and video model id would otherwise read as an Ollama + * model and be judged against an allowlist that was never about it. + */ export function findProviderFromModel(model: string): ProviderId | null { const normalizedModel = model.toLowerCase() diff --git a/apps/sim/providers/nvidia/index.ts b/apps/sim/providers/nvidia/index.ts index a5d384b94d7..234f637c92c 100644 --- a/apps/sim/providers/nvidia/index.ts +++ b/apps/sim/providers/nvidia/index.ts @@ -37,6 +37,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, generateSchemaInstructions, isFunctionToolCall, @@ -125,14 +126,7 @@ export const nvidiaProvider: ProviderConfig = { const responseFormatPayload = request.responseFormat ? useJsonMode ? { type: 'json_object' as const } - : { - type: 'json_schema' as const, - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + : buildJsonSchemaResponseFormat(request.responseFormat) : undefined if (useJsonMode) payload.chat_template_kwargs = { enable_thinking: false } diff --git a/apps/sim/providers/openrouter/index.ts b/apps/sim/providers/openrouter/index.ts index c5d47cbb383..3d69a0155a3 100644 --- a/apps/sim/providers/openrouter/index.ts +++ b/apps/sim/providers/openrouter/index.ts @@ -15,7 +15,7 @@ import { } from '@/providers/openrouter/utils' import { openAICompatTransport } from '@/providers/transport' import type { ProviderConfig, ProviderRequest, ProviderResponse } from '@/providers/types' -import { generateSchemaInstructions } from '@/providers/utils' +import { buildJsonSchemaResponseFormat, generateSchemaInstructions } from '@/providers/utils' const logger = createLogger('OpenRouterProvider') @@ -33,14 +33,7 @@ async function applyResponseFormat( if (useNative) { logger.info('Using native structured outputs for OpenRouter model', { model }) - targetPayload.response_format = { - type: 'json_schema', - json_schema: { - name: responseFormat.name || 'response_schema', - schema: responseFormat.schema || responseFormat, - strict: responseFormat.strict !== false, - }, - } + targetPayload.response_format = buildJsonSchemaResponseFormat(responseFormat) targetPayload.provider = { ...targetPayload.provider, require_parameters: true } return messages } diff --git a/apps/sim/providers/sakana/index.ts b/apps/sim/providers/sakana/index.ts index 78b1aeaca6d..98bb49c7b8e 100644 --- a/apps/sim/providers/sakana/index.ts +++ b/apps/sim/providers/sakana/index.ts @@ -33,6 +33,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, @@ -104,14 +105,7 @@ export const sakanaProvider: ProviderConfig = { if (request.maxTokens != null) payload.max_completion_tokens = request.maxTokens const responseFormatPayload = request.responseFormat - ? { - type: 'json_schema' as const, - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + ? buildJsonSchemaResponseFormat(request.responseFormat) : undefined let preparedTools: ReturnType | null = null diff --git a/apps/sim/providers/utils.test.ts b/apps/sim/providers/utils.test.ts index d1b0b3a1923..2810d1b8765 100644 --- a/apps/sim/providers/utils.test.ts +++ b/apps/sim/providers/utils.test.ts @@ -14,6 +14,12 @@ vi.mock('@/lib/internal/workflows/read-tool-enrichment', () => ({ import { RevenueCatBlock } from '@/blocks/blocks/revenuecat' import { VideoGeneratorV3Block } from '@/blocks/blocks/video_generator' import { normalizeFileInput } from '@/blocks/utils' +import { + findProviderFromModel, + getMaxOutputTokensForModel, + getMaxTemperature, + supportsTemperature, +} from '@/providers/models' import { assignProviderToolIdentities } from '@/providers/tool-identity' import type { ProviderToolConfig } from '@/providers/types' import { @@ -21,20 +27,16 @@ import { calculateCost, describeModelLevel, extractAndParseJSON, - findProviderFromModel, formatCost, generateStructuredOutputInstructions, getApiKey, getBaseModelProviders, - getMaxOutputTokensForModel, - getMaxTemperature, getProvider, getProviderFromModel, isGemini3Model, prepareToolExecution, prepareToolsWithUsageControl, shouldBillModelUsage, - supportsTemperature, transformBlockTool, } from '@/providers/utils' import { useProvidersStore } from '@/stores/providers/store' diff --git a/apps/sim/providers/utils.ts b/apps/sim/providers/utils.ts index fe5d6baab85..ef9640a3a67 100644 --- a/apps/sim/providers/utils.ts +++ b/apps/sim/providers/utils.ts @@ -26,11 +26,9 @@ import { assembleCustomBlockInputMapping, isCustomBlockType } from '@/blocks/cus import type { SubBlockConfig } from '@/blocks/types' import { isCustomTool } from '@/executor/constants' import { - findProviderFromModel as findProviderFromDefinitions, + findProviderFromModel, getComputerUseModels, getHostedModels as getHostedModelsFromDefinitions, - getMaxOutputTokensForModel as getMaxOutputTokensForModelFromDefinitions, - getMaxTemperature as getMaxTempFromDefinitions, getModelsWithDeepResearch, getModelsWithoutMemory, getModelsWithPromptCaching, @@ -38,14 +36,9 @@ import { getModelsWithThinking, getModelsWithVerbosity, getProviderDefaultModel as getProviderDefaultModelFromDefinitions, - getProviderModels as getProviderModelsFromDefinitions, - getReasoningEffortValuesForModel as getReasoningEffortValuesForModelFromDefinitions, - getThinkingLevelsForModel as getThinkingLevelsForModelFromDefinitions, - getVerbosityValuesForModel as getVerbosityValuesForModelFromDefinitions, + getProviderModels, isKnownModelLevelValue, PROVIDER_DEFINITIONS, - supportsTemperature as supportsTemperatureFromDefinitions, - supportsToolUsageControl as supportsToolUsageControlFromDefinitions, updateOllamaModels as updateOllamaModelsInDefinitions, } from '@/providers/models' import { @@ -58,7 +51,12 @@ import { getProviderToolModelInputRegistry, registerPreparedProviderToolInputProvenance, } from '@/providers/tool-input-provenance' -import type { ModelPricing, ProviderId, ProviderToolConfig } from '@/providers/types' +import type { + ModelPricing, + ProviderId, + ProviderRequest, + ProviderToolConfig, +} from '@/providers/types' import { mergeToolParameters } from '@/tools/merge-params' import { buildToolParamShapes, decodeToolParams } from '@/tools/param-shape' import type { WorkflowToolExecutionContext } from '@/tools/types' @@ -128,7 +126,7 @@ function buildProviderMetadata(providerId: ProviderId): ProviderMetadata { name: def?.name || providerId, description: def?.description || '', version: '1.0.0', - models: getProviderModelsFromDefinitions(providerId), + models: getProviderModels(providerId), defaultModel: getProviderDefaultModelFromDefinitions(providerId), modelPatterns: def?.modelPatterns, } @@ -146,7 +144,7 @@ export const providers: Record = { anthropic: { ...buildProviderMetadata('anthropic'), computerUseModels: getComputerUseModels().filter((model) => - getProviderModelsFromDefinitions('anthropic').includes(model) + getProviderModels('anthropic').includes(model) ), }, google: buildProviderMetadata('google'), @@ -174,49 +172,49 @@ export const providers: Record = { export function updateOllamaProviderModels(models: string[]): void { updateOllamaModelsInDefinitions(models) - providers.ollama.models = getProviderModelsFromDefinitions('ollama') + providers.ollama.models = getProviderModels('ollama') } export function updateVLLMProviderModels(models: string[]): void { const { updateVLLMModels } = require('@/providers/models') updateVLLMModels(models) - providers.vllm.models = getProviderModelsFromDefinitions('vllm') + providers.vllm.models = getProviderModels('vllm') } export function updateLiteLLMProviderModels(models: string[]): void { const { updateLiteLLMModels } = require('@/providers/models') updateLiteLLMModels(models) - providers.litellm.models = getProviderModelsFromDefinitions('litellm') + providers.litellm.models = getProviderModels('litellm') } export async function updateOpenRouterProviderModels(models: string[]): Promise { const { updateOpenRouterModels } = await import('@/providers/models') updateOpenRouterModels(models) - providers.openrouter.models = getProviderModelsFromDefinitions('openrouter') + providers.openrouter.models = getProviderModels('openrouter') } export async function updateFireworksProviderModels(models: string[]): Promise { const { updateFireworksModels } = await import('@/providers/models') updateFireworksModels(models) - providers.fireworks.models = getProviderModelsFromDefinitions('fireworks') + providers.fireworks.models = getProviderModels('fireworks') } export async function updateOllamaCloudProviderModels(models: string[]): Promise { const { updateOllamaCloudModels } = await import('@/providers/models') updateOllamaCloudModels(models) - providers['ollama-cloud'].models = getProviderModelsFromDefinitions('ollama-cloud') + providers['ollama-cloud'].models = getProviderModels('ollama-cloud') } export async function updateTogetherProviderModels(models: string[]): Promise { const { updateTogetherModels } = await import('@/providers/models') updateTogetherModels(models) - providers.together.models = getProviderModelsFromDefinitions('together') + providers.together.models = getProviderModels('together') } export async function updateBasetenProviderModels(models: string[]): Promise { const { updateBasetenModels } = await import('@/providers/models') updateBasetenModels(models) - providers.baseten.models = getProviderModelsFromDefinitions('baseten') + providers.baseten.models = getProviderModels('baseten') } export function getBaseModelProviders(): Record { @@ -260,19 +258,6 @@ function filterBlacklistedModelsFromProviderMap( return filtered } -/** - * The provider that declares `model`, or `null` when none does. - * - * The non-guessing half of {@link getProviderFromModel}. A caller that *gates* - * on the answer needs "unknown" to stay distinct from "ollama": this registry - * holds chat models only, so every embedding, speech, image and video model id - * would otherwise read as an Ollama model and be judged against an allowlist - * that was never about it. - */ -export function findProviderFromModel(model: string): ProviderId | null { - return findProviderFromDefinitions(model) -} - export function getProviderFromModel(model: string): ProviderId { const normalizedModel = model.toLowerCase() @@ -303,10 +288,6 @@ export function getAllProviderIds(): ProviderId[] { return Object.keys(providers) as ProviderId[] } -export function getProviderModels(providerId: ProviderId): string[] { - return getProviderModelsFromDefinitions(providerId) -} - export function isProviderBlacklisted(providerId: string): boolean { return getBlacklistedProvidersFromEnv().includes(providerId.toLowerCase()) } @@ -346,9 +327,33 @@ export function filterBlacklistedModels(models: string[]): string[] { return models.filter((model) => !isModelBlacklisted(model)) } -export function getProviderIcon(model: string): React.ComponentType<{ className?: string }> | null { - const providerId = getProviderFromModel(model) - return PROVIDER_DEFINITIONS[providerId]?.icon || null +/** OpenAI-compatible `json_schema` response format sent to chat-completions providers. */ +export interface JsonSchemaResponseFormat { + type: 'json_schema' + json_schema: { + name: string + schema: Record + strict?: boolean + } +} + +/** + * Builds the OpenAI-compatible `json_schema` response format from a request's `responseFormat`. + * Strict mode is on unless the caller set `strict: false`; `includeStrict: false` leaves the + * `strict` flag out of the payload entirely. + */ +export function buildJsonSchemaResponseFormat( + responseFormat: NonNullable, + { includeStrict = true }: { includeStrict?: boolean } = {} +): JsonSchemaResponseFormat { + return { + type: 'json_schema', + json_schema: { + name: responseFormat.name || 'response_schema', + schema: responseFormat.schema || responseFormat, + ...(includeStrict ? { strict: responseFormat.strict !== false } : {}), + }, + } } /** @@ -1482,10 +1487,6 @@ export const MODELS_WITH_PROMPT_CACHING = getModelsWithPromptCaching() export const MODELS_WITH_DEEP_RESEARCH = getModelsWithDeepResearch() export const MODELS_WITHOUT_MEMORY = getModelsWithoutMemory() -export function supportsTemperature(model: string): boolean { - return supportsTemperatureFromDefinitions(model) -} - /** * Levels the pickers offer on top of what a model declares. `auto` means "say nothing" and * `none` means "explicitly off"; provider adapters special-case both, so neither is an @@ -1542,51 +1543,6 @@ export function isGemini3Model(model: string): boolean { return normalized.startsWith('gemini-3') } -/** - * Get the maximum temperature value for a model - * @returns Maximum temperature value (1 or 2) or undefined if temperature not supported - */ -export function getMaxTemperature(model: string): number | undefined { - return getMaxTempFromDefinitions(model) -} - -export function supportsToolUsageControl(provider: string): boolean { - return supportsToolUsageControlFromDefinitions(provider) -} - -/** - * Get reasoning effort values for a specific model - * Returns the valid options for that model, or null if the model doesn't support reasoning effort - */ -export function getReasoningEffortValuesForModel(model: string): string[] | null { - return getReasoningEffortValuesForModelFromDefinitions(model) -} - -/** - * Get verbosity values for a specific model - * Returns the valid options for that model, or null if the model doesn't support verbosity - */ -export function getVerbosityValuesForModel(model: string): string[] | null { - return getVerbosityValuesForModelFromDefinitions(model) -} - -/** - * Get thinking levels for a specific model - * Returns the valid levels for that model, or null if the model doesn't support thinking - */ -export function getThinkingLevelsForModel(model: string): string[] | null { - return getThinkingLevelsForModelFromDefinitions(model) -} - -/** - * Get max output tokens for a specific model. - * - * @param model - The model ID - */ -export function getMaxOutputTokensForModel(model: string): number { - return getMaxOutputTokensForModelFromDefinitions(model) -} - /** * Prepare tool execution parameters, separating tool parameters from system parameters */ diff --git a/apps/sim/providers/vllm/index.ts b/apps/sim/providers/vllm/index.ts index a861d8017dd..d53be59a7a0 100644 --- a/apps/sim/providers/vllm/index.ts +++ b/apps/sim/providers/vllm/index.ts @@ -39,6 +39,7 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { + buildJsonSchemaResponseFormat, calculateCost, checkForForcedToolUsageOpenAI, isFunctionToolCall, @@ -200,14 +201,7 @@ export const vllmProvider: ProviderConfig = { if (request.maxTokens != null) payload.max_tokens = request.maxTokens if (request.responseFormat) { - payload.response_format = { - type: 'json_schema', - json_schema: { - name: request.responseFormat.name || 'response_schema', - schema: request.responseFormat.schema || request.responseFormat, - strict: request.responseFormat.strict !== false, - }, - } + payload.response_format = buildJsonSchemaResponseFormat(request.responseFormat) logger.info('Added JSON schema response format to vLLM request') } diff --git a/apps/sim/tools/params-resolver.test.ts b/apps/sim/tools/params-resolver.test.ts deleted file mode 100644 index 054d5e86211..00000000000 --- a/apps/sim/tools/params-resolver.test.ts +++ /dev/null @@ -1,42 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { buildCanonicalIndex, buildPreviewContextValues } from '@/tools/params-resolver' - -const canonicalIndex = buildCanonicalIndex([ - { - id: 'knowledgeBaseSelector', - type: 'knowledge-base-selector', - canonicalParamId: 'knowledgeBaseId', - mode: 'basic', - }, - { - id: 'manualKnowledgeBaseId', - type: 'short-input', - canonicalParamId: 'knowledgeBaseId', - mode: 'advanced', - }, -] as Parameters[0]) - -const values = { knowledgeBaseSelector: 'kb-basic', manualKnowledgeBaseId: 'kb-advanced' } - -describe('buildPreviewContextValues', () => { - it('honors an explicit advanced override so the preview matches execution', () => { - const result = buildPreviewContextValues(values, { - blockType: 'knowledge', - subBlocks: [], - canonicalIndex, - values, - overrides: { knowledgeBaseId: 'advanced' }, - }) - expect(result.knowledgeBaseId).toBe('kb-advanced') - }) - - it('falls back to the value heuristic (basic when present) without an override', () => { - const result = buildPreviewContextValues(values, { - blockType: 'knowledge', - subBlocks: [], - canonicalIndex, - values, - }) - expect(result.knowledgeBaseId).toBe('kb-basic') - }) -}) diff --git a/packages/testing/src/mocks/providers-utils.mock.ts b/packages/testing/src/mocks/providers-utils.mock.ts index cf5f324de7b..5fb73fecd3d 100644 --- a/packages/testing/src/mocks/providers-utils.mock.ts +++ b/packages/testing/src/mocks/providers-utils.mock.ts @@ -40,8 +40,9 @@ interface MockForcedToolUsage { * - `trackForcedToolUsage` / `checkForForcedToolUsageOpenAI` → nothing forced, used list unchanged; * - `enforceStrictSchema` and `filterBlacklistedModels` are identity; `isProviderBlacklisted`, * `shouldBillModelUsage` and every `supports*`/`is*Model` capability check → `false`; - * - `getProviderFromModel` → `'openai'`, `findProviderFromModel` → `null`; - * - `generateSchemaInstructions` → `'SCHEMA_INSTRUCTIONS'`; + * - `getProviderFromModel` → `'openai'`; + * - `generateSchemaInstructions` → `'SCHEMA_INSTRUCTIONS'`, `buildJsonSchemaResponseFormat` is the + * real `json_schema` payload builder; * - `getApiKey` returns the user key, else the real `PROVIDER_PLACEHOLDER_KEY`; * - the `MODELS_WITH_*` catalog constants are empty arrays and `providers` is `{}`. * @@ -63,14 +64,24 @@ export const providersUtilsMockFns = { mockUpdateTogetherProviderModels: vi.fn(async (_models: string[]): Promise => {}), mockUpdateBasetenProviderModels: vi.fn(async (_models: string[]): Promise => {}), mockGetBaseModelProviders: vi.fn((): Record => ({})), - mockFindProviderFromModel: vi.fn((_model: string): string | null => null), mockGetProviderFromModel: vi.fn((_model: string): string => 'openai'), mockGetProvider: vi.fn((_id: string): unknown => undefined), mockGetAllProviderIds: vi.fn((): string[] => []), - mockGetProviderModels: vi.fn((_providerId: string): string[] => []), mockIsProviderBlacklisted: vi.fn((_providerId: string): boolean => false), mockFilterBlacklistedModels: vi.fn((models: string[]): string[] => models), - mockGetProviderIcon: vi.fn((_model: string): unknown => null), + mockBuildJsonSchemaResponseFormat: vi.fn( + ( + responseFormat: { name?: string; schema?: unknown; strict?: boolean }, + { includeStrict = true }: { includeStrict?: boolean } = {} + ) => ({ + type: 'json_schema' as const, + json_schema: { + name: responseFormat.name || 'response_schema', + schema: responseFormat.schema || responseFormat, + ...(includeStrict ? { strict: responseFormat.strict !== false } : {}), + }, + }) + ), mockGenerateSchemaInstructions: vi.fn( (_schema: unknown, _schemaName?: string): string => 'SCHEMA_INSTRUCTIONS' ), @@ -124,7 +135,6 @@ export const providersUtilsMockFns = { usedForcedTools: string[] = [] ): MockForcedToolUsage => ({ hasUsedForcedTool: false, usedForcedTools }) ), - mockSupportsTemperature: vi.fn((_model: string): boolean => false), mockDescribeModelLevel: vi.fn((value: string | undefined): string => value || '(unset)'), mockSupportsReasoningEffort: vi.fn((_model: string): boolean => false), mockSupportsVerbosity: vi.fn((_model: string): boolean => false), @@ -132,12 +142,6 @@ export const providersUtilsMockFns = { mockSupportsPromptCaching: vi.fn((_model: string): boolean => false), mockIsDeepResearchModel: vi.fn((_model: string): boolean => false), mockIsGemini3Model: vi.fn((_model: string): boolean => false), - mockGetMaxTemperature: vi.fn((_model: string): number | undefined => undefined), - mockSupportsToolUsageControl: vi.fn((_provider: string): boolean => false), - mockGetReasoningEffortValuesForModel: vi.fn((_model: string): string[] | null => null), - mockGetVerbosityValuesForModel: vi.fn((_model: string): string[] | null => null), - mockGetThinkingLevelsForModel: vi.fn((_model: string): string[] | null => null), - mockGetMaxOutputTokensForModel: vi.fn((_model: string): number => 4096), mockPrepareToolExecution: vi.fn( ( _tool: unknown, @@ -186,14 +190,12 @@ export const providersUtilsMock = { updateTogetherProviderModels: providersUtilsMockFns.mockUpdateTogetherProviderModels, updateBasetenProviderModels: providersUtilsMockFns.mockUpdateBasetenProviderModels, getBaseModelProviders: providersUtilsMockFns.mockGetBaseModelProviders, - findProviderFromModel: providersUtilsMockFns.mockFindProviderFromModel, getProviderFromModel: providersUtilsMockFns.mockGetProviderFromModel, getProvider: providersUtilsMockFns.mockGetProvider, getAllProviderIds: providersUtilsMockFns.mockGetAllProviderIds, - getProviderModels: providersUtilsMockFns.mockGetProviderModels, isProviderBlacklisted: providersUtilsMockFns.mockIsProviderBlacklisted, filterBlacklistedModels: providersUtilsMockFns.mockFilterBlacklistedModels, - getProviderIcon: providersUtilsMockFns.mockGetProviderIcon, + buildJsonSchemaResponseFormat: providersUtilsMockFns.mockBuildJsonSchemaResponseFormat, generateSchemaInstructions: providersUtilsMockFns.mockGenerateSchemaInstructions, generateStructuredOutputInstructions: providersUtilsMockFns.mockGenerateStructuredOutputInstructions, @@ -211,7 +213,6 @@ export const providersUtilsMock = { prepareToolsWithUsageControl: providersUtilsMockFns.mockPrepareToolsWithUsageControl, isFunctionToolCall: providersUtilsMockFns.mockIsFunctionToolCall, trackForcedToolUsage: providersUtilsMockFns.mockTrackForcedToolUsage, - supportsTemperature: providersUtilsMockFns.mockSupportsTemperature, describeModelLevel: providersUtilsMockFns.mockDescribeModelLevel, supportsReasoningEffort: providersUtilsMockFns.mockSupportsReasoningEffort, supportsVerbosity: providersUtilsMockFns.mockSupportsVerbosity, @@ -219,12 +220,6 @@ export const providersUtilsMock = { supportsPromptCaching: providersUtilsMockFns.mockSupportsPromptCaching, isDeepResearchModel: providersUtilsMockFns.mockIsDeepResearchModel, isGemini3Model: providersUtilsMockFns.mockIsGemini3Model, - getMaxTemperature: providersUtilsMockFns.mockGetMaxTemperature, - supportsToolUsageControl: providersUtilsMockFns.mockSupportsToolUsageControl, - getReasoningEffortValuesForModel: providersUtilsMockFns.mockGetReasoningEffortValuesForModel, - getVerbosityValuesForModel: providersUtilsMockFns.mockGetVerbosityValuesForModel, - getThinkingLevelsForModel: providersUtilsMockFns.mockGetThinkingLevelsForModel, - getMaxOutputTokensForModel: providersUtilsMockFns.mockGetMaxOutputTokensForModel, prepareToolExecution: providersUtilsMockFns.mockPrepareToolExecution, checkForForcedToolUsageOpenAI: providersUtilsMockFns.mockCheckForForcedToolUsageOpenAI, } From 185eab2f9f8a5f06194926c2d36e15e414e78a10 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:16:10 -0700 Subject: [PATCH 17/30] chore(cleanup): remove dead executor, tools, and block registry exports --- apps/sim/blocks/registry.ts | 19 ---------- apps/sim/executor/constants.ts | 5 --- .../handlers/shared/response-format.ts | 27 -------------- apps/sim/executor/types.ts | 9 ----- apps/sim/executor/utils.ts | 3 +- .../executor/utils/reference-validation.ts | 11 ------ .../lib/function-execution/execute-request.ts | 8 ----- .../lib/workflows/diff/diff-engine.test.ts | 3 -- apps/sim/tools/params-resolver.ts | 36 ------------------- apps/sim/tools/params.test.ts | 28 --------------- apps/sim/tools/params.ts | 29 --------------- 11 files changed, 1 insertion(+), 177 deletions(-) diff --git a/apps/sim/blocks/registry.ts b/apps/sim/blocks/registry.ts index dfc104d7516..4708c8fdf3e 100644 --- a/apps/sim/blocks/registry.ts +++ b/apps/sim/blocks/registry.ts @@ -220,11 +220,6 @@ export function getLatestBlock(baseType: string): BlockConfig | undefined { return resolveLatest(baseType)?.config } -/** All blocks in a given category. */ -export function getBlocksByCategory(category: BlockCategory): BlockConfig[] { - return Object.values(BLOCK_REGISTRY).filter((block) => block.category === category) -} - /** * The canonical "latest-version, toolbar-visible" set of blocks for a * category. This is the single source of truth shared by every surface that @@ -243,20 +238,6 @@ export function getCanonicalBlocksByCategory(category: BlockCategory): BlockConf return visibilityInert(vis) ? blocks : blocks.map((block) => projectBlock(block, vis)) } -/** All registered block type identifiers. */ -export function getAllBlockTypes(): string[] { - return Object.keys(BLOCK_REGISTRY) -} - -/** Whether the given string is a registered block type. Accepts hyphens as a dash-form alias. */ -export function isValidBlockType(type: string): type is string { - return ( - type in BLOCK_REGISTRY || - normalizeType(type) in BLOCK_REGISTRY || - Boolean(resolveOverlayBlock(type)) - ) -} - /** * Get the presentation/catalog meta for a block type, resolving through the * version suffix the same way {@link getTemplatesForBlock} does. Metas are diff --git a/apps/sim/executor/constants.ts b/apps/sim/executor/constants.ts index 00d7c9c20db..1f0e8c657fc 100644 --- a/apps/sim/executor/constants.ts +++ b/apps/sim/executor/constants.ts @@ -427,7 +427,6 @@ export function parseReferencePath(reference: string): string[] { export const PATTERNS = { UUID: /^[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}$/i, - UUID_V4: /^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i, UUID_PREFIX: /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/i, ENV_VAR_NAME: /^[A-Za-z_][A-Za-z0-9_]*$/, } as const @@ -436,10 +435,6 @@ export function isUuid(value: string): boolean { return PATTERNS.UUID.test(value) } -export function isUuidV4(value: string): boolean { - return PATTERNS.UUID_V4.test(value) -} - export function startsWithUuid(value: string): boolean { return PATTERNS.UUID_PREFIX.test(value) } diff --git a/apps/sim/executor/handlers/shared/response-format.ts b/apps/sim/executor/handlers/shared/response-format.ts index 9f1ad68a5ef..6ded4f6b9a0 100644 --- a/apps/sim/executor/handlers/shared/response-format.ts +++ b/apps/sim/executor/handlers/shared/response-format.ts @@ -1,5 +1,4 @@ import { createLogger } from '@sim/logger' -import type { BlockOutput } from '@/blocks/types' import { REFERENCE } from '@/executor/constants' const logger = createLogger('SharedResponseFormat') @@ -48,29 +47,3 @@ export function parseResponseFormat(responseFormat?: string | object): any { return undefined } - -/** - * Try to parse the LLM response content as structured JSON and spread - * the fields into the block output. Falls back to returning raw content. - */ -export function processStructuredResponse( - result: { content?: string; model?: string; tokens?: any }, - defaultModel: string -): BlockOutput { - const content = result.content ?? '' - try { - const parsed = JSON.parse(content.trim()) - return { - ...parsed, - model: result.model || defaultModel, - tokens: result.tokens || {}, - } - } catch { - logger.warn('Failed to parse structured response, returning raw content') - return { - content, - model: result.model || defaultModel, - tokens: result.tokens || {}, - } - } -} diff --git a/apps/sim/executor/types.ts b/apps/sim/executor/types.ts index b272ee17cf2..5c987d03c79 100644 --- a/apps/sim/executor/types.ts +++ b/apps/sim/executor/types.ts @@ -836,12 +836,3 @@ interface Tool

> { interface ToolRegistry { [key: string]: Tool } - -export interface ResponseFormatStreamProcessor { - processStream( - originalStream: ReadableStream, - blockId: string, - selectedOutputs: string[], - responseFormat?: any - ): ReadableStream -} diff --git a/apps/sim/executor/utils.ts b/apps/sim/executor/utils.ts index c509e4121ca..11deda3f9a0 100644 --- a/apps/sim/executor/utils.ts +++ b/apps/sim/executor/utils.ts @@ -1,5 +1,4 @@ import { createLogger } from '@sim/logger' -import type { ResponseFormatStreamProcessor } from '@/executor/types' const logger = createLogger('ExecutorUtils') @@ -7,7 +6,7 @@ const logger = createLogger('ExecutorUtils') * Processes a streaming response to extract only the selected response format fields * instead of streaming the full JSON wrapper. */ -export class StreamingResponseFormatProcessor implements ResponseFormatStreamProcessor { +export class StreamingResponseFormatProcessor { processStream( originalStream: ReadableStream, blockId: string, diff --git a/apps/sim/executor/utils/reference-validation.ts b/apps/sim/executor/utils/reference-validation.ts index 6f724b26030..d0248289064 100644 --- a/apps/sim/executor/utils/reference-validation.ts +++ b/apps/sim/executor/utils/reference-validation.ts @@ -144,17 +144,6 @@ export function createWorkflowVariablePattern(): RegExp { ) } -/** - * Combined pattern matching both and {{env_var}} - */ -export function createCombinedPattern(): RegExp { - return new RegExp( - `${REFERENCE.START}[^${REFERENCE.START}${REFERENCE.END}]+${REFERENCE.END}|` + - `\\${REFERENCE.ENV_VAR_START}${ENV_VAR_BODY}\\${REFERENCE.ENV_VAR_END}`, - 'g' - ) -} - /** * Collects every string leaf in a nested value — the shared walk for reference/env-token * audits over block inputs (lint, deps, and the agent-cli mirrors each carried a copy). diff --git a/apps/sim/lib/function-execution/execute-request.ts b/apps/sim/lib/function-execution/execute-request.ts index 1e969a16ed8..143e14f642b 100644 --- a/apps/sim/lib/function-execution/execute-request.ts +++ b/apps/sim/lib/function-execution/execute-request.ts @@ -1485,14 +1485,6 @@ export interface FunctionExecutionRequestContext { signal: AbortSignal } -export function projectFunctionValidationResponse( - req: Pick, - response: NextResponse -): Promise { - const metadataType = getRequestedResolvedSecretNamesMetadataType(req.headers) - return appendPrivateResolvedSecretNames(response, metadataType ? [] : null, metadataType) -} - /** * Compares an about-to-be-exported buffer against the overwrite target's * current content. `identical: true` means the export is a byte-for-byte no-op: diff --git a/apps/sim/lib/workflows/diff/diff-engine.test.ts b/apps/sim/lib/workflows/diff/diff-engine.test.ts index 7d6c5dc1954..e6afd481aeb 100644 --- a/apps/sim/lib/workflows/diff/diff-engine.test.ts +++ b/apps/sim/lib/workflows/diff/diff-engine.test.ts @@ -43,10 +43,7 @@ vi.mock('@sim/workflow-persistence/subflow-helpers', () => ({ vi.mock('@/blocks', () => ({ getBlock: () => null, getAllBlocks: () => ({}), - getAllBlockTypes: () => [], getBlockByToolName: () => null, - getBlocksByCategory: () => [], - isValidBlockType: () => false, registry: {}, })) diff --git a/apps/sim/tools/params-resolver.ts b/apps/sim/tools/params-resolver.ts index d0bd4e83097..49f241e7aa5 100644 --- a/apps/sim/tools/params-resolver.ts +++ b/apps/sim/tools/params-resolver.ts @@ -3,7 +3,6 @@ import { type CanonicalIndex, type CanonicalModeOverrides, evaluateSubBlockCondition, - getCanonicalValues, isCanonicalPair, reindexToolCanonicalModes, resolveCanonicalMode, @@ -11,7 +10,6 @@ import { type SubBlockCondition, scopeCanonicalModesForTool, } from '@/lib/workflows/subblocks/visibility' -import type { SubBlockConfig as BlockSubBlockConfig } from '@/blocks/types' export { buildCanonicalIndex, @@ -25,37 +23,3 @@ export { scopeCanonicalModesForTool, type SubBlockCondition, } - -export interface ToolParamContext { - blockType: string - subBlocks: BlockSubBlockConfig[] - canonicalIndex: CanonicalIndex - values: Record - /** - * Canonical-id-keyed mode overrides (the tool-scoped `canonicalModes`) so the preview honors an - * explicit basic/advanced toggle, matching execution. Omitted -> the value heuristic. - */ - overrides?: CanonicalModeOverrides -} - -/** - * Build preview context values for selectors that need dependency resolution. - * Resolves canonical values so selectors get the correct credential/dependency values. - */ -export function buildPreviewContextValues( - params: Record, - context: ToolParamContext -): Record { - const result: Record = { ...params } - - for (const [canonicalId, group] of Object.entries(context.canonicalIndex.groupsById)) { - if (isCanonicalPair(group)) { - const mode = resolveCanonicalMode(group, context.values, context.overrides) - const { basicValue, advancedValue } = getCanonicalValues(group, context.values) - result[canonicalId] = - mode === 'advanced' ? (advancedValue ?? basicValue) : (basicValue ?? advancedValue) - } - } - - return result -} diff --git a/apps/sim/tools/params.test.ts b/apps/sim/tools/params.test.ts index 18f9258f72b..443106f4aa6 100644 --- a/apps/sim/tools/params.test.ts +++ b/apps/sim/tools/params.test.ts @@ -10,7 +10,6 @@ import { isPasswordParameter, type ToolSchema, ToolSchemaEnrichmentError, - validateToolParameters, } from '@/tools/params' import type { HttpMethod, ParameterVisibility } from '@/tools/types' @@ -562,33 +561,6 @@ describe('Tool Parameters Utils', () => { }) }) - describe('validateToolParameters', () => { - it.concurrent('should validate successfully with all required parameters', () => { - const finalParams = { - apiKey: 'test-key', - message: 'Hello world', - channel: '#general', - } - - const result = validateToolParameters(mockToolConfig, finalParams) - - expect(result.valid).toBe(true) - expect(result.missingParams).toHaveLength(0) - }) - - it.concurrent('should fail validation with missing required parameters', () => { - const finalParams = { - channel: '#general', - } - - const result = validateToolParameters(mockToolConfig, finalParams) - - expect(result.valid).toBe(false) - expect(result.missingParams).toContain('apiKey') - expect(result.missingParams).toContain('message') - }) - }) - describe('filterSchemaForLLM', () => { it.concurrent('should filter out user-provided parameters from schema', () => { const originalSchema: ToolSchema = { diff --git a/apps/sim/tools/params.ts b/apps/sim/tools/params.ts index f1cce808250..1729248b130 100644 --- a/apps/sim/tools/params.ts +++ b/apps/sim/tools/params.ts @@ -91,11 +91,6 @@ export class ToolSchemaEnrichmentError extends Error { } } -export interface ValidationResult { - valid: boolean - missingParams: string[] -} - let blockConfigCache: Record | null = null function getBlockConfigurations(): Record { @@ -523,30 +518,6 @@ export function filterSchemaForLLM( }) } -/** - * Validates that all required parameters are provided - */ -export function validateToolParameters( - toolConfig: ExecutableToolConfig, - finalParams: Record -): ValidationResult { - const requiredParams = Object.entries(toolConfig.params) - .filter(([_, param]) => param.required) - .map(([paramId]) => paramId) - - const missingParams = requiredParams.filter( - (paramId) => - finalParams[paramId] === undefined || - finalParams[paramId] === null || - finalParams[paramId] === '' - ) - - return { - valid: missingParams.length === 0, - missingParams, - } -} - /** * A tool param's effective visibility. * From 9854071d87c34861b63a267123c4b176b9089443 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:23:25 -0700 Subject: [PATCH 18/30] chore(cleanup): remove unreachable workspace-file and table use cases --- .../sim/lib/table/application/columns.test.ts | 96 +--- apps/sim/lib/table/application/columns.ts | 64 --- apps/sim/lib/table/application/groups.test.ts | 3 - apps/sim/lib/table/application/groups.ts | 44 -- apps/sim/lib/table/columns/service.ts | 107 ---- apps/sim/lib/table/workflow-groups/service.ts | 83 --- .../write-workspace-file-by-path.ts | 92 +--- .../orchestration/create.test.ts | 127 ----- .../workspace-files/orchestration/create.ts | 125 ----- .../file-folder-lifecycle.test.ts | 120 ----- .../orchestration/file-folder-lifecycle.ts | 505 +----------------- .../workspace-files/orchestration/index.ts | 28 - 12 files changed, 9 insertions(+), 1385 deletions(-) delete mode 100644 apps/sim/lib/workspace-files/orchestration/create.test.ts delete mode 100644 apps/sim/lib/workspace-files/orchestration/create.ts delete mode 100644 apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.test.ts diff --git a/apps/sim/lib/table/application/columns.test.ts b/apps/sim/lib/table/application/columns.test.ts index 5a8f7a103a4..4f9b9777233 100644 --- a/apps/sim/lib/table/application/columns.test.ts +++ b/apps/sim/lib/table/application/columns.test.ts @@ -1,6 +1,6 @@ import { createDelegatedPrincipal } from '@sim/testing/factories/principal.factory' import { auditMock, auditMockFns } from '@sim/testing/mocks/audit.mock' -import { tableMock, tableMockFns } from '@sim/testing/mocks/table.mock' +import { tableMock } from '@sim/testing/mocks/table.mock' import { tableApplicationContextMock, tableApplicationContextMockFns, @@ -17,10 +17,7 @@ const hoisted = vi.hoisted(() => ({ vi.mock('@sim/audit', () => auditMock) vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) -vi.mock('@/lib/table', () => ({ - ...tableMock, - TABLE_LIMITS: { ...tableMock.TABLE_LIMITS, MAX_COLUMNS_PER_TABLE: 3 }, -})) +vi.mock('@/lib/table', () => tableMock) vi.mock('@/lib/table/application/context', () => tableApplicationContextMock) vi.mock('@/lib/table/columns/workflow-references', () => ({ findUnmigratedTableBlockReferences: hoisted.findUnmigrated, @@ -28,14 +25,10 @@ vi.mock('@/lib/table/columns/workflow-references', () => ({ vi.mock('@/lib/table/events', () => tableEventsMock) vi.mock('@/lib/table/orchestration', () => ({ performUpdateTableColumn: hoisted.performUpdate })) -import { - deleteTableColumnsUseCase, - updateTableColumnUseCase, -} from '@/lib/table/application/columns' +import { updateTableColumnUseCase } from '@/lib/table/application/columns' const mocks = { ...hoisted, - deleteColumns: tableMockFns.mockDeleteColumns, resolveContext: tableApplicationContextMockFns.mockResolveActiveTableContext, audit: auditMockFns.mockRecordAudit, resolvePermission: workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission, @@ -68,89 +61,6 @@ const principal = createDelegatedPrincipal({ resourceScope: { tableId: 'table-1' }, }) -const tableAfterDelete: TableDefinition = { - ...table, - schema: { columns: [{ id: 'column-name', name: 'name', type: 'string' }] }, - updatedAt: new Date('2026-08-02T00:00:00.000Z'), -} - -describe('multi-column delete application use case', () => { - beforeEach(() => { - mocks.resolvePermission.mockResolvedValue('write') - mocks.resolveContext.mockResolvedValue({ - tableId: table.id, - table, - workspaceId: table.workspaceId, - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mocks.deleteColumns.mockResolvedValue(tableAfterDelete) - }) - - it('derives aliases and duplicate references from the authoritative schema delta', async () => { - mocks.deleteColumns.mockResolvedValue({ - ...table, - schema: { - columns: [ - { id: 'column-name', name: 'name', type: 'string' }, - { id: 'column-last', name: 'last', type: 'string' }, - ], - }, - }) - - const result = await deleteTableColumnsUseCase.execute({ - principal, - input: { - tableId: 'table-1', - workspaceId: 'workspace-1', - columnNames: ['first', 'column-first', 'FIRST'], - }, - }) - - expect(result.deletedColumns).toEqual([{ id: 'column-first', name: 'first' }]) - expect(mocks.audit).toHaveBeenCalledWith( - expect.objectContaining({ - description: 'Deleted 1 column from table "People"', - metadata: expect.objectContaining({ columnNames: ['first'] }), - }) - ) - }) - - it('rejects an oversized request before mutation', async () => { - await expect( - deleteTableColumnsUseCase.execute({ - principal, - input: { - tableId: 'table-1', - workspaceId: 'workspace-1', - columnNames: ['first', 'last', 'name', 'extra'], - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - - expect(mocks.deleteColumns).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - expect(mocks.signal).not.toHaveBeenCalled() - }) - - it('rejects admission before mutation when delegated scope is stale', async () => { - mocks.resolvePermission.mockResolvedValueOnce('read') - - await expect( - deleteTableColumnsUseCase.execute({ - principal, - input: { - tableId: 'table-1', - workspaceId: 'workspace-1', - columnNames: ['first', 'last'], - }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - expect(mocks.deleteColumns).not.toHaveBeenCalled() - }) -}) - /** * A rename leaves workflow Table blocks pointing at the old name — nothing * rewrites workflow state, lint stays clean, and the next run fails inside an diff --git a/apps/sim/lib/table/application/columns.ts b/apps/sim/lib/table/application/columns.ts index 4752ac1c613..b81f754d40d 100644 --- a/apps/sim/lib/table/application/columns.ts +++ b/apps/sim/lib/table/application/columns.ts @@ -2,17 +2,12 @@ import { AuditAction, AuditResourceType } from '@sim/audit' import { resolvePrincipalAttribution } from '@sim/auth/principal' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' -import { OrchestrationError } from '@/lib/core/orchestration/types' import { generateRequestId } from '@/lib/core/utils/request' import { addTableColumn, - type ColumnDefinition, type ColumnType, deleteColumn, - deleteColumns, - getColumnId, type SelectOption, - TABLE_LIMITS, type TableDefinition, } from '@/lib/table' import { defineAuthorizedTableUseCase } from '@/lib/table/application/authorized-table-use-case' @@ -202,62 +197,3 @@ export const deleteTableColumnUseCase = defineAuthorizedTableUseCase({ signalTableSchemaChanged(context.table.id) }, }) - -export interface DeleteTableColumnsInput extends TableColumnInput { - columnNames: string[] -} - -interface DeletedTableColumn { - id: string - name: string -} - -export const deleteTableColumnsUseCase = defineAuthorizedTableUseCase({ - operation: tableOperations.deleteColumn, - resolveContext: ({ input }: { input: DeleteTableColumnsInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.workspaceId, - }), - async execute({ input, context }): Promise<{ - table: TableDefinition - deletedColumns: DeletedTableColumn[] - }> { - if (input.columnNames.length < 1) { - throw new OrchestrationError('validation', 'At least one column name is required') - } - if (input.columnNames.length > TABLE_LIMITS.MAX_COLUMNS_PER_TABLE) { - throw new OrchestrationError( - 'validation', - `Cannot delete more than ${TABLE_LIMITS.MAX_COLUMNS_PER_TABLE} columns` - ) - } - const table = await deleteColumns( - { tableId: context.table.id, columnNames: input.columnNames }, - generateRequestId(), - { expectedWorkspaceId: context.workspaceId } - ) - const remainingColumnIds = new Set(table.schema.columns.map(getColumnId)) - const deletedColumns = context.table.schema.columns - .filter((column) => !remainingColumnIds.has(getColumnId(column))) - .map((column) => ({ id: getColumnId(column), name: column.name })) - return { table, deletedColumns } - }, - projectAudit({ context, result }) { - if (result.deletedColumns.length === 0) return [] - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Deleted ${result.deletedColumns.length} ${result.deletedColumns.length === 1 ? 'column' : 'columns'} from table "${context.table.name}"`, - metadata: { columnNames: result.deletedColumns.map((column) => column.name) }, - } - }, - afterSuccess({ context, result }) { - if (result.deletedColumns.length > 0) signalTableSchemaChanged(context.table.id) - }, -}) - -export type TableColumnApplicationResult = { table: TableDefinition } -export type TableColumnDefinition = ColumnDefinition diff --git a/apps/sim/lib/table/application/groups.test.ts b/apps/sim/lib/table/application/groups.test.ts index e89d433e918..12707306a3c 100644 --- a/apps/sim/lib/table/application/groups.test.ts +++ b/apps/sim/lib/table/application/groups.test.ts @@ -24,7 +24,6 @@ import type { TableDefinition, WorkflowGroup } from '@/lib/table/types' const hoisted = vi.hoisted(() => ({ addGroup: vi.fn(), addOutput: vi.fn(), - deleteOutput: vi.fn(), getEnrichment: vi.fn(), loadWorkflowOutputs: vi.fn(), updateGroup: vi.fn(), @@ -51,7 +50,6 @@ vi.mock('@/lib/table/workflow-groups/service', () => ({ addWorkflowGroup: hoisted.addGroup, addWorkflowGroupOutput: hoisted.addOutput, deleteWorkflowGroup: vi.fn(), - deleteWorkflowGroupOutput: hoisted.deleteOutput, updateWorkflowGroup: hoisted.updateGroup, })) vi.mock('@/lib/workflows/application/context', () => workflowContextMock) @@ -222,7 +220,6 @@ describe('workflow and enrichment Table application commands', () => { tableWithGroup(nextGroup, [...table.schema.columns, ...outputColumns]) ) mocks.addOutput.mockResolvedValue(table) - mocks.deleteOutput.mockResolvedValue(table) mocks.updateGroup.mockImplementation(async (input) => tableWithGroup({ ...group, diff --git a/apps/sim/lib/table/application/groups.ts b/apps/sim/lib/table/application/groups.ts index 34df08a1187..32aabda8e44 100644 --- a/apps/sim/lib/table/application/groups.ts +++ b/apps/sim/lib/table/application/groups.ts @@ -32,7 +32,6 @@ import { addWorkflowGroup, addWorkflowGroupOutput, deleteWorkflowGroup, - deleteWorkflowGroupOutput, updateWorkflowGroup, } from '@/lib/table/workflow-groups/service' import { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' @@ -1202,46 +1201,3 @@ export const addWorkflowTableGroupOutput = defineAuthorizedTableUseCase({ signalTableSchemaChanged(context.tableId) }, }) - -export interface DeleteTableGroupOutputInput extends TableGroupInput { - groupId: string - columnName: string -} - -export const deleteTableGroupOutputUseCase = defineAuthorizedTableUseCase({ - operation: tableOperations.updateGroup, - resolveContext: ({ input }: { input: DeleteTableGroupOutputInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.workspaceId, - }), - async execute({ input, context }) { - const table = await deleteWorkflowGroupOutput( - { - tableId: context.tableId, - workspaceId: context.workspaceId, - groupId: input.groupId, - columnName: input.columnName, - }, - generateRequestId() - ) - return { table, groupId: input.groupId, columnName: input.columnName } - }, - projectAudit({ result }) { - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Deleted an output from workflow group "${result.groupId}"`, - metadata: { - op: 'delete_group_output', - groupId: result.groupId, - columnName: result.columnName, - }, - } - }, - afterSuccess({ context }) { - signalTableSchemaChanged(context.tableId) - }, -}) diff --git a/apps/sim/lib/table/columns/service.ts b/apps/sim/lib/table/columns/service.ts index 24469f7cf96..433c57f4fdd 100644 --- a/apps/sim/lib/table/columns/service.ts +++ b/apps/sim/lib/table/columns/service.ts @@ -508,113 +508,6 @@ export async function deleteColumn( return def } -/** - * Deletes multiple columns from a table in a single transaction. - * Avoids the race condition of calling deleteColumn multiple times in parallel. - */ -export async function deleteColumns( - data: { tableId: string; columnNames: string[] }, - requestId: string, - options?: ColumnMutationOptions -): Promise { - const { def, stripKeys } = await withLockedTable( - data.tableId, - async (table, trx) => { - assertColumnDestructive(table) - const schema = table.schema - const namesToDelete = new Set() - const idsToDelete = new Set() - const notFound: string[] = [] - - for (const name of data.columnNames) { - const col = schema.columns.find((c) => columnMatchesRef(c, name)) - if (!col) { - notFound.push(name) - } else { - namesToDelete.add(col.name) - idsToDelete.add(getColumnId(col)) - } - } - - if (notFound.length > 0) { - throw new OrchestrationError('not_found', `Columns not found: ${notFound.join(', ')}`) - } - - const remaining = schema.columns.filter((c) => !namesToDelete.has(c.name)) - if (remaining.length === 0) { - throw new OrchestrationError('validation', 'Cannot delete all columns from a table') - } - - // For each group, drop outputs whose column (by id) is being deleted. Groups - // that end up with zero outputs are removed entirely (they'd be invalid). - // Then any remaining group's dependencies referencing a removed column are - // cleaned up. - const removedGroupIds = new Set() - let updatedGroups = (schema.workflowGroups ?? []).map((group) => { - const remainingOutputs = group.outputs.filter((o) => !idsToDelete.has(o.columnName)) - if (remainingOutputs.length === 0) { - removedGroupIds.add(group.id) - } - return remainingOutputs.length === group.outputs.length - ? group - : { ...group, outputs: remainingOutputs } - }) - updatedGroups = updatedGroups - .filter((g) => !removedGroupIds.has(g.id)) - .map((group) => stripGroupDeps(group, idsToDelete)) - const updatedSchema: TableSchema = { - ...schema, - columns: remaining, - ...(updatedGroups.length > 0 ? { workflowGroups: updatedGroups } : {}), - } - const updatedMetadata = stripColumnIdsFromMetadata( - table.metadata as TableMetadata | null, - idsToDelete - ) - assertValidSchema(updatedSchema, updatedMetadata?.columnOrder) - - const now = new Date() - - // Schema/metadata commit now; row storage for the deleted columns is - // reclaimed in the background (fire-and-forget). - await trx - .update(userTableDefinitions) - .set({ schema: updatedSchema, metadata: updatedMetadata, updatedAt: now }) - .where( - and( - eq(userTableDefinitions.id, data.tableId), - eq(userTableDefinitions.workspaceId, table.workspaceId) - ) - ) - - await stripGroupExecutions(trx, data.tableId, removedGroupIds, { - expectedWorkspaceId: table.workspaceId, - }) - - logger.info( - `[${requestId}] Deleted columns [${[...namesToDelete].join(', ')}] from table ${data.tableId}` - ) - - return { - def: { ...table, schema: updatedSchema, metadata: updatedMetadata, updatedAt: now }, - stripKeys: Array.from(idsToDelete), - } - }, - { expectedWorkspaceId: options?.expectedWorkspaceId } - ) - - if (stripKeys.length > 0) { - stripColumnDataInBackground( - data.tableId, - def.workspaceId, - stripKeys, - def.rowCount ?? 0, - requestId - ) - } - return def -} - /** * Validates a constraint change against the column's stored data, and returns * the column with those constraints applied. diff --git a/apps/sim/lib/table/workflow-groups/service.ts b/apps/sim/lib/table/workflow-groups/service.ts index 93d86938fe3..40098f12783 100644 --- a/apps/sim/lib/table/workflow-groups/service.ts +++ b/apps/sim/lib/table/workflow-groups/service.ts @@ -948,89 +948,6 @@ export async function addWorkflowGroupOutput( return updatedTable } -/** - * Removes a single output from a workflow group. Drops the bound column and - * strips the value from every row's `data` JSONB. If the output is the - * group's last, the empty group is left in place — drop it explicitly with - * `deleteWorkflowGroup` if needed. - */ -export async function deleteWorkflowGroupOutput( - data: { tableId: string; workspaceId?: string; groupId: string; columnName: string }, - requestId: string -): Promise { - return withLockedTable( - data.tableId, - async (table, trx) => { - assertColumnDestructive(table) - const schema = table.schema - const groups = schema.workflowGroups ?? [] - const groupIndex = groups.findIndex((g) => g.id === data.groupId) - if (groupIndex === -1) { - throw new OrchestrationError('not_found', `Workflow group "${data.groupId}" not found`) - } - const group = groups[groupIndex] - // `data.columnName` may be a column id (first-party) or display name - // (mothership/legacy); resolve to the stable id used everywhere below. - const targetColumn = schema.columns.find((c) => columnMatchesRef(c, data.columnName)) - const columnId = targetColumn ? getColumnId(targetColumn) : data.columnName - if (!group.outputs.some((o) => o.columnName === columnId)) { - throw new OrchestrationError( - 'not_found', - `Workflow group "${data.groupId}" has no output bound to column "${data.columnName}"` - ) - } - - const updatedGroup: WorkflowGroup = { - ...group, - outputs: group.outputs.filter((o) => o.columnName !== columnId), - } - const nextGroups = groups.map((g, i) => (i === groupIndex ? updatedGroup : g)) - const nextColumns = schema.columns.filter((c) => getColumnId(c) !== columnId) - const updatedSchema: TableSchema = { - ...schema, - columns: nextColumns, - workflowGroups: nextGroups, - } - - const updatedColumnOrder = table.metadata?.columnOrder?.filter((id) => id !== columnId) - assertValidSchema(updatedSchema, updatedColumnOrder) - - const updatedMetadata: TableMetadata | null = - updatedColumnOrder && table.metadata - ? { ...table.metadata, columnOrder: updatedColumnOrder } - : table.metadata - ? { ...table.metadata } - : null - - const now = new Date() - await setTableTxTimeouts(trx, { statementMs: 60_000 }) - await trx - .update(userTableDefinitions) - .set({ schema: updatedSchema, metadata: updatedMetadata, updatedAt: now }) - .where( - and( - eq(userTableDefinitions.id, data.tableId), - eq(userTableDefinitions.workspaceId, table.workspaceId) - ) - ) - await updateTableRowsWithDerivedSecretProvenance(trx, { - rowWhere: and( - eq(userTableRows.tableId, data.tableId), - eq(userTableRows.workspaceId, table.workspaceId) - )!, - transformation: { mode: 'remove-columns', columnIds: [columnId] }, - }) - - logger.info( - `[${requestId}] Removed output "${data.columnName}" from workflow group "${data.groupId}" in table ${data.tableId}` - ) - - return { ...table, schema: updatedSchema, metadata: updatedMetadata, updatedAt: now } - }, - { expectedWorkspaceId: data.workspaceId } - ) -} - /** * Removes a workflow group plus all its output columns. Also strips the * group's `executions[groupId]` entry from every row. diff --git a/apps/sim/lib/workspace-files/application/write-workspace-file-by-path.ts b/apps/sim/lib/workspace-files/application/write-workspace-file-by-path.ts index d38371d29cb..7298e522dfa 100644 --- a/apps/sim/lib/workspace-files/application/write-workspace-file-by-path.ts +++ b/apps/sim/lib/workspace-files/application/write-workspace-file-by-path.ts @@ -6,23 +6,18 @@ import type { WorkspaceFileSecretProvenance } from '@/lib/uploads/contexts/works import { encodeVfsPathSegments, encodeVfsSegment } from '@/lib/vfs/path' import { admitCreateWorkspaceFile, - createWorkspaceFile, createWorkspaceFileFromBuffer, } from '@/lib/workspace-files/application/create-workspace-file' import { fileOperations } from '@/lib/workspace-files/application/operations' import { resolveWorkspaceFileReference } from '@/lib/workspace-files/application/resolve-workspace-file-reference' -import { - updateWorkspaceFileContent, - updateWorkspaceFileContentFromBuffer, -} from '@/lib/workspace-files/application/update-workspace-file-content' +import { updateWorkspaceFileContentFromBuffer } from '@/lib/workspace-files/application/update-workspace-file-content' import { parseWorkspaceFileFolderDisplayPath } from '@/lib/workspace-files/folder-display-path' import { parseWorkspaceFileCreatePath } from '@/lib/workspace-files/workspace-file-path' -export interface WriteWorkspaceFileByPathInput { +export interface WriteWorkspaceFileBufferByPathInput { workspaceId: string path: string - content: string - encoding: 'utf-8' | 'base64' + content: Buffer contentType: string mode: 'create' | 'overwrite' exactName?: boolean @@ -30,11 +25,6 @@ export interface WriteWorkspaceFileByPathInput { secretProvenance?: WorkspaceFileSecretProvenance } -export interface WriteWorkspaceFileBufferByPathInput - extends Omit { - content: Buffer -} - export interface WriteWorkspaceFileByPathResult { id: string name: string @@ -42,7 +32,7 @@ export interface WriteWorkspaceFileByPathResult { contentType: string downloadUrl?: string vfsPath: string - mode: WriteWorkspaceFileByPathInput['mode'] + mode: WriteWorkspaceFileBufferByPathInput['mode'] } function toResult( @@ -54,7 +44,7 @@ function toResult( url?: string folderPath?: string | null }, - mode: WriteWorkspaceFileByPathInput['mode'] + mode: WriteWorkspaceFileBufferByPathInput['mode'] ): WriteWorkspaceFileByPathResult { const folderPath = file.folderPath ?? '' const encodedFolderPath = folderPath @@ -71,68 +61,6 @@ function toResult( } } -async function executeCreate({ - principal, - input, -}: { - principal: Principal - input: WriteWorkspaceFileByPathInput -}): Promise { - const parsed = parseWorkspaceFileCreatePath(input.path) - await admitCreateWorkspaceFile(principal, input.workspaceId) - - const folderUserId = await resolveFolderAttributionUserId(principal, input.workspaceId) - - const { folderId } = await ensureWorkspaceFileFolderPath({ - workspaceId: input.workspaceId, - userId: folderUserId, - pathSegments: parsed.folderSegments, - }) - const result = await createWorkspaceFile.execute({ - principal, - input: { - workspaceId: input.workspaceId, - name: parsed.fileName, - contentType: input.contentType, - content: input.content, - encoding: input.encoding, - folderId, - exactName: input.exactName ?? true, - secretProvenance: input.secretProvenance, - }, - }) - return toResult(result.file, 'create') -} - -async function executeOverwrite({ - principal, - input, -}: { - principal: Principal - input: WriteWorkspaceFileByPathInput -}): Promise { - const existing = await resolveWorkspaceFileReference({ - principal, - operation: fileOperations.updateContent, - workspaceId: input.workspaceId, - reference: input.path, - }) - const result = await updateWorkspaceFileContent.execute({ - principal, - input: { - fileId: existing.id, - assertedWorkspaceId: input.workspaceId, - content: input.content, - encoding: input.encoding, - contentType: input.contentType, - provenanceMode: 'replace_empty', - syncLiveDoc: input.syncLiveDoc, - secretProvenance: input.secretProvenance, - }, - }) - return toResult(result.file, 'overwrite') -} - async function executeCreateBuffer({ principal, input, @@ -204,16 +132,6 @@ async function resolveFolderAttributionUserId( return resolvePrincipalAttribution(principal, { workspaceBillingOwnerUserId }).attributedUserId } -export const createWorkspaceFileByPath = { - operation: fileOperations.create, - execute: executeCreate, -} as const - -export const updateWorkspaceFileContentByPath = { - operation: fileOperations.updateContent, - execute: executeOverwrite, -} as const - export const createWorkspaceFileBufferByPath = { operation: fileOperations.create, execute: executeCreateBuffer, diff --git a/apps/sim/lib/workspace-files/orchestration/create.test.ts b/apps/sim/lib/workspace-files/orchestration/create.test.ts deleted file mode 100644 index 97e7157b236..00000000000 --- a/apps/sim/lib/workspace-files/orchestration/create.test.ts +++ /dev/null @@ -1,127 +0,0 @@ -import { auditMock, auditMockFns } from '@sim/testing/mocks/audit.mock' -import { posthogServerMock, posthogServerMockFns } from '@sim/testing/mocks/posthog-server.mock' -import { - workspaceUploadsMock, - workspaceUploadsMockFns, -} from '@sim/testing/mocks/workspace-uploads.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -vi.mock('@sim/audit', () => auditMock) - -vi.mock('@/lib/uploads/contexts/workspace', () => workspaceUploadsMock) - -vi.mock('@/lib/posthog/server', () => posthogServerMock) - -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { FileConflictError } from '@/lib/uploads/contexts/workspace' -import { - MAX_WORKSPACE_FILE_CONTENT_BYTES, - performCreateWorkspaceFile, -} from '@/lib/workspace-files/orchestration' - -const mockUploadWorkspaceFile = workspaceUploadsMockFns.mockUploadWorkspaceFile - -const mockCaptureServerEvent = posthogServerMockFns.mockCaptureServerEvent - -const mockRecordAudit = auditMockFns.mockRecordAudit - -const WORKSPACE_ID = 'workspace-1' -const USER_ID = 'user-1' -const CREATED_FILE = { - id: 'wf_created', - workspaceId: WORKSPACE_ID, - name: 'untitled.md', - key: 'workspace/workspace-1/untitled.md', - path: '/api/files/serve/untitled.md', - url: '/api/files/serve/untitled.md', - size: 0, - type: 'text/markdown', - uploadedBy: USER_ID, - folderId: null, - folderPath: null, - deletedAt: null, - uploadedAt: new Date('2026-08-04T00:00:00.000Z'), - updatedAt: new Date('2026-08-04T00:00:00.000Z'), - context: 'workspace' as const, -} - -describe('performCreateWorkspaceFile', () => { - beforeEach(() => { - mockUploadWorkspaceFile.mockResolvedValue(CREATED_FILE) - }) - - it('rejects decoded content above the content-update limit before storage I/O', async () => { - const result = await performCreateWorkspaceFile({ - workspaceId: WORKSPACE_ID, - userId: USER_ID, - name: 'too-large.md', - contentType: 'text/markdown', - content: Buffer.alloc(MAX_WORKSPACE_FILE_CONTENT_BYTES + 1), - }) - - expect(result).toEqual({ - success: false, - error: 'File size exceeds 50MB limit', - errorCode: 'payload_too_large', - }) - expect(mockUploadWorkspaceFile).not.toHaveBeenCalled() - expect(mockRecordAudit).not.toHaveBeenCalled() - expect(mockCaptureServerEvent).not.toHaveBeenCalled() - }) - - it('classifies an exact-name collision as conflict and records no audit', async () => { - mockUploadWorkspaceFile.mockRejectedValue(new FileConflictError('untitled.md')) - - const result = await performCreateWorkspaceFile({ - workspaceId: WORKSPACE_ID, - userId: USER_ID, - name: 'untitled.md', - contentType: 'text/markdown', - }) - - expect(result).toEqual({ - success: false, - error: 'A file named "untitled.md" already exists in this workspace', - errorCode: 'conflict', - }) - expect(mockRecordAudit).not.toHaveBeenCalled() - expect(mockCaptureServerEvent).not.toHaveBeenCalled() - }) - - it('preserves classified folder failures and keeps unexpected faults internal', async () => { - mockUploadWorkspaceFile.mockRejectedValueOnce( - new OrchestrationError('not_found', 'Target folder not found') - ) - - const missingFolder = await performCreateWorkspaceFile({ - workspaceId: WORKSPACE_ID, - userId: USER_ID, - name: 'untitled.md', - contentType: 'text/markdown', - folderId: 'missing', - }) - - expect(missingFolder).toEqual({ - success: false, - error: 'Target folder not found', - errorCode: 'not_found', - }) - - mockUploadWorkspaceFile.mockRejectedValueOnce(new Error('connection terminated')) - - const unexpected = await performCreateWorkspaceFile({ - workspaceId: WORKSPACE_ID, - userId: USER_ID, - name: 'untitled.md', - contentType: 'text/markdown', - }) - - expect(unexpected).toEqual({ - success: false, - error: 'connection terminated', - errorCode: 'internal', - }) - expect(mockRecordAudit).not.toHaveBeenCalled() - expect(mockCaptureServerEvent).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/workspace-files/orchestration/create.ts b/apps/sim/lib/workspace-files/orchestration/create.ts deleted file mode 100644 index f7356ab0c55..00000000000 --- a/apps/sim/lib/workspace-files/orchestration/create.ts +++ /dev/null @@ -1,125 +0,0 @@ -import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' -import { createLogger } from '@sim/logger' -import { getPostgresErrorCode, toError } from '@sim/utils/errors' -import { - asOrchestrationError, - type OrchestrationErrorCode, - type OrchestrationRequestContext, -} from '@/lib/core/orchestration/types' -import { captureServerEvent } from '@/lib/posthog/server' -import { - FileConflictError, - uploadWorkspaceFile, - type WorkspaceFileRecord, -} from '@/lib/uploads/contexts/workspace' -import { EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' -import { MAX_WORKSPACE_FILE_CONTENT_BYTES } from '@/lib/workspace-files/orchestration/content' - -const logger = createLogger('WorkspaceFileCreateOrchestration') - -export interface PerformCreateWorkspaceFileParams { - workspaceId: string - userId: string - name: string - contentType: string - folderId?: string | null - folderPath?: string - content?: Buffer - exactName?: boolean - actorName?: string - actorEmail?: string - request?: OrchestrationRequestContext -} - -export interface PerformCreateWorkspaceFileResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - file?: WorkspaceFileRecord -} - -/** - * Creates a workspace file from server-held bytes and returns its canonical record. - * - * Exact-name mode keeps this operation suitable for public create surfaces: a - * live sibling with the requested name is a conflict instead of silently - * producing a suffixed copy. Uploads remain the storage and accounting - * primitive, including for an empty buffer. - */ -export async function performCreateWorkspaceFile( - params: PerformCreateWorkspaceFileParams -): Promise { - const { - workspaceId, - userId, - name, - contentType, - folderId, - folderPath, - content = Buffer.alloc(0), - exactName = true, - actorName, - actorEmail, - request, - } = params - - if (content.length > MAX_WORKSPACE_FILE_CONTENT_BYTES) { - return { - success: false, - error: `File size exceeds ${MAX_WORKSPACE_FILE_CONTENT_BYTES / 1024 / 1024}MB limit`, - errorCode: 'payload_too_large', - } - } - - try { - const file = await uploadWorkspaceFile(workspaceId, userId, content, name, contentType, { - folderId, - folderPath, - exactName, - secretProvenance: EXACT_EMPTY_WORKSPACE_FILE_SECRET_PROVENANCE, - }) - - logger.info('Created workspace file', { - workspaceId, - fileId: file.id, - folderId: file.folderId, - size: file.size, - }) - - recordAudit({ - workspaceId, - actorId: userId, - actorName, - actorEmail, - action: AuditAction.FILE_UPLOADED, - resourceType: AuditResourceType.FILE, - resourceId: file.id, - resourceName: file.name, - description: `Uploaded file "${file.name}"`, - metadata: { fileSize: file.size, fileType: file.type }, - request, - }) - - captureServerEvent( - userId, - 'file_uploaded', - { workspace_id: workspaceId, file_type: file.type }, - { groups: { workspace: workspaceId } } - ) - - return { success: true, file } - } catch (error) { - logger.error('Failed to create workspace file', { error, workspaceId, folderId, folderPath }) - - if (error instanceof FileConflictError || getPostgresErrorCode(error) === '23505') { - return { success: false, error: toError(error).message, errorCode: 'conflict' } - } - - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} diff --git a/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.test.ts b/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.test.ts deleted file mode 100644 index b0d9f0f5893..00000000000 --- a/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.test.ts +++ /dev/null @@ -1,120 +0,0 @@ -/** - * Failure classification. Every `perform*` here is consumed by a public v2 route - * that maps `errorCode` straight to an HTTP status, so a manager failure that - * arrives unclassified silently becomes a 500 for what is really a caller-fixable - * 400 or 404. These pin the mapping rather than the happy paths. - */ - -import { auditMock } from '@sim/testing/mocks/audit.mock' -import { realtimeNotifyMock } from '@sim/testing/mocks/realtime-notify.mock' -import { - workspaceUploadsMock, - workspaceUploadsMockFns, -} from '@sim/testing/mocks/workspace-uploads.mock' -import { describe, expect, it, vi } from 'vitest' - -vi.mock('@/lib/uploads/contexts/workspace', () => workspaceUploadsMock) - -vi.mock('@/lib/realtime/notify', () => realtimeNotifyMock) - -vi.mock('@sim/audit', () => auditMock) - -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { - performDeleteWorkspaceFileFolderByPath, - performMoveWorkspaceFileItems, - performRenameWorkspaceFile, - performUpdateWorkspaceFileFolder, -} from '@/lib/workspace-files/orchestration' - -const mockMoveWorkspaceFileItems = workspaceUploadsMockFns.mockMoveWorkspaceFileItems -const mockUpdateWorkspaceFileFolder = workspaceUploadsMockFns.mockUpdateWorkspaceFileFolder -const mockRenameWorkspaceFile = workspaceUploadsMockFns.mockRenameWorkspaceFile -const mockDeleteWorkspaceFileFolderByPath = - workspaceUploadsMockFns.mockDeleteWorkspaceFileFolderByPath - -const WS = 'workspace-1' -const USER = 'user-1' - -describe('workspace file orchestration error classification', () => { - it('maps a missing move target to not_found, not internal', async () => { - mockMoveWorkspaceFileItems.mockRejectedValue( - new OrchestrationError('not_found', 'Target folder not found') - ) - - const result = await performMoveWorkspaceFileItems({ - workspaceId: WS, - userId: USER, - fileIds: ['wf_1'], - targetFolderId: 'fold_missing', - }) - - expect(result.success).toBe(false) - expect(result.errorCode).toBe('not_found') - expect(result.error).toBe('Target folder not found') - }) - - it('maps a self-descendant move to validation, not internal', async () => { - mockMoveWorkspaceFileItems.mockRejectedValue( - new OrchestrationError('validation', 'Cannot move a folder into one of its descendants') - ) - - const result = await performMoveWorkspaceFileItems({ - workspaceId: WS, - userId: USER, - folderIds: ['fold_1'], - targetFolderId: 'fold_child', - }) - - expect(result.errorCode).toBe('validation') - }) - - it('classifies through a wrapper error chain, as drizzle produces inside a transaction', async () => { - const wrapped = new Error('update "folder" set ... failed', { - cause: new OrchestrationError('validation', 'Folder cannot be its own parent'), - }) - mockUpdateWorkspaceFileFolder.mockRejectedValue(wrapped) - - const result = await performUpdateWorkspaceFileFolder({ - workspaceId: WS, - folderId: 'fold_1', - userId: USER, - parentId: 'fold_1', - }) - - expect(result.errorCode).toBe('validation') - expect(result.error).toBe('Folder cannot be its own parent') - }) - - it('leaves a genuinely unexpected fault as internal', async () => { - mockRenameWorkspaceFile.mockRejectedValue(new Error('connection terminated unexpectedly')) - - const result = await performRenameWorkspaceFile({ - workspaceId: WS, - fileId: 'wf_1', - name: 'renamed.csv', - userId: USER, - }) - - expect(result.errorCode).toBe('internal') - }) - - it('classifies a non-empty non-recursive folder delete as a conflict', async () => { - mockDeleteWorkspaceFileFolderByPath.mockRejectedValue( - new OrchestrationError('conflict', 'Folder is not empty') - ) - - const result = await performDeleteWorkspaceFileFolderByPath({ - workspaceId: WS, - userId: USER, - path: '/Reports', - recursive: false, - }) - - expect(result).toEqual({ - success: false, - error: 'Folder is not empty', - errorCode: 'conflict', - }) - }) -}) diff --git a/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.ts b/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.ts index 58d5148adaf..c7c39e623c5 100644 --- a/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.ts +++ b/apps/sim/lib/workspace-files/orchestration/file-folder-lifecycle.ts @@ -1,27 +1,11 @@ import { AuditAction, AuditResourceType, recordAudit } from '@sim/audit' import { createLogger } from '@sim/logger' -import { getErrorMessage, getPostgresErrorCode, toError } from '@sim/utils/errors' +import { toError } from '@sim/utils/errors' import { asOrchestrationError, type OrchestrationErrorCode } from '@/lib/core/orchestration/types' -import { FolderPathError } from '@/lib/folders/paths' import { notifyWorkspaceFilesChanged } from '@/lib/realtime/notify' import { bulkArchiveWorkspaceFileItems, - createWorkspaceFileFolder, - createWorkspaceFileFolderAtPath, - deleteWorkspaceFileFolderByPath, - FileConflictError, - moveWorkspaceFileItems, - relocateWorkspaceFileFolderByPath, - renameWorkspaceFile, - restoreWorkspaceFile, - restoreWorkspaceFileFolder, - updateWorkspaceFileFolder, type WorkspaceFileArchiveResult, - WorkspaceFileFolderConflictError, - type WorkspaceFileFolderRecord, - WorkspaceFileItemsNotFoundError, - WorkspaceFileMoveConflictError, - type WorkspaceFileRecord, } from '@/lib/uploads/contexts/workspace' const logger = createLogger('WorkspaceFileFolderLifecycle') @@ -46,198 +30,6 @@ export interface PerformDeleteWorkspaceFileItemsResult { deletedItems?: WorkspaceFileArchiveResult } -export interface PerformMoveWorkspaceFileItemsParams { - workspaceId: string - userId: string - fileIds?: string[] - folderIds?: string[] - targetFolderId?: string | null - targetFolderPath?: string -} - -export interface PerformMoveWorkspaceFileItemsResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - movedItems?: { files: number; folders: number } -} - -export interface PerformRenameWorkspaceFileParams { - workspaceId: string - fileId: string - name: string - userId: string -} - -export interface PerformRenameWorkspaceFileResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - file?: WorkspaceFileRecord -} - -export interface PerformRestoreWorkspaceFileParams { - workspaceId: string - fileId: string - userId: string -} - -export interface PerformRestoreWorkspaceFileResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode -} - -export interface PerformCreateWorkspaceFileFolderParams { - workspaceId: string - userId: string - name: string - parentId?: string | null -} - -export interface PerformCreateWorkspaceFileFolderResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - folder?: WorkspaceFileFolderRecord -} - -export interface PerformUpdateWorkspaceFileFolderParams { - workspaceId: string - folderId: string - userId: string - name?: string - parentId?: string | null - sortOrder?: number -} - -export interface PerformUpdateWorkspaceFileFolderResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - folder?: WorkspaceFileFolderRecord -} - -export interface PerformRestoreWorkspaceFileFolderParams { - workspaceId: string - folderId: string - userId: string -} - -export interface PerformRestoreWorkspaceFileFolderResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - folder?: WorkspaceFileFolderRecord - restoredItems?: WorkspaceFileArchiveResult -} - -export interface PerformFileFolderPathMutationResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - folder?: WorkspaceFileFolderRecord - path?: string -} - -export interface PerformDeleteFileFolderByPathResult { - success: boolean - error?: string - errorCode?: OrchestrationErrorCode - deletedItems?: WorkspaceFileArchiveResult -} - -function fileFolderPathError(error: unknown): { - error: string - errorCode: OrchestrationErrorCode -} { - if (error instanceof FolderPathError) { - return { error: error.message, errorCode: 'validation' } - } - if ( - error instanceof WorkspaceFileFolderConflictError || - getPostgresErrorCode(error) === '23505' - ) { - return { error: toError(error).message, errorCode: 'conflict' } - } - const classified = asOrchestrationError(error) - if (classified) return { error: classified.message, errorCode: classified.code } - return { error: toError(error).message, errorCode: 'internal' } -} - -export async function performCreateWorkspaceFileFolderAtPath(params: { - workspaceId: string - userId: string - path: string -}): Promise { - try { - const result = await createWorkspaceFileFolderAtPath(params) - recordAudit({ - workspaceId: params.workspaceId, - actorId: params.userId, - action: AuditAction.FOLDER_CREATED, - resourceType: AuditResourceType.FOLDER, - resourceName: result.folder.name, - description: `Created file folder "${result.folder.name}"`, - metadata: { path: result.path }, - }) - await notifyWorkspaceFilesChanged(params.workspaceId) - return { success: true, folder: { ...result.folder, path: result.path }, path: result.path } - } catch (error) { - logger.error('Failed to create workspace file folder by path', { error }) - return { success: false, ...fileFolderPathError(error) } - } -} - -export async function performRelocateWorkspaceFileFolderByPath(params: { - workspaceId: string - userId: string - path: string - destinationPath: string -}): Promise { - try { - const result = await relocateWorkspaceFileFolderByPath(params) - recordAudit({ - workspaceId: params.workspaceId, - actorId: params.userId, - action: AuditAction.FOLDER_MOVED, - resourceType: AuditResourceType.FOLDER, - resourceName: result.folder.name, - description: `Moved file folder to "${result.path}"`, - metadata: { sourcePath: params.path, destinationPath: result.path }, - }) - await notifyWorkspaceFilesChanged(params.workspaceId) - return { success: true, folder: { ...result.folder, path: result.path }, path: result.path } - } catch (error) { - logger.error('Failed to relocate workspace file folder by path', { error }) - return { success: false, ...fileFolderPathError(error) } - } -} - -export async function performDeleteWorkspaceFileFolderByPath(params: { - workspaceId: string - userId: string - path: string - recursive: boolean -}): Promise { - try { - const deletedItems = await deleteWorkspaceFileFolderByPath(params) - recordAudit({ - workspaceId: params.workspaceId, - actorId: params.userId, - action: AuditAction.FOLDER_DELETED, - resourceType: AuditResourceType.FOLDER, - description: `Deleted file folder "${params.path}"`, - metadata: { path: params.path, affected: deletedItems }, - }) - await notifyWorkspaceFilesChanged(params.workspaceId) - return { success: true, deletedItems } - } catch (error) { - logger.error('Failed to delete workspace file folder by path', { error }) - return { success: false, ...fileFolderPathError(error) } - } -} - export async function performDeleteWorkspaceFileItems( params: PerformDeleteWorkspaceFileItemsParams ): Promise { @@ -310,298 +102,3 @@ export async function performDeleteWorkspaceFileItems( return { success: false, error: toError(error).message, errorCode: 'internal' } } } - -export async function performMoveWorkspaceFileItems( - params: PerformMoveWorkspaceFileItemsParams -): Promise { - const { - workspaceId, - userId, - fileIds = [], - folderIds = [], - targetFolderId, - targetFolderPath, - } = params - - if (fileIds.length === 0 && folderIds.length === 0) { - return { - success: false, - error: 'At least one file or folder must be selected', - errorCode: 'validation', - } - } - - try { - const moved = await moveWorkspaceFileItems({ - workspaceId, - fileIds, - folderIds, - targetFolderId, - targetFolderPath, - }) - const movedItems = { files: moved.movedFiles, folders: moved.movedFolders } - - logger.info('Moved workspace file items', { - workspaceId, - fileIds, - folderIds, - targetFolderId, - targetFolderPath, - movedItems, - }) - - if (fileIds.length > 0) { - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FILE_MOVED, - resourceType: AuditResourceType.FILE, - description: `Moved ${fileIds.length} file${fileIds.length === 1 ? '' : 's'}${targetFolderId || (targetFolderPath && targetFolderPath !== '/') ? ' to folder' : ' to root'}`, - metadata: { fileIds, targetFolderId, targetFolderPath }, - }) - } - - if (folderIds.length > 0) { - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FOLDER_MOVED, - resourceType: AuditResourceType.FOLDER, - resourceId: folderIds.length === 1 ? folderIds[0] : undefined, - description: `Moved ${folderIds.length} file folder${folderIds.length === 1 ? '' : 's'}${targetFolderId || (targetFolderPath && targetFolderPath !== '/') ? ' to folder' : ' to root'}`, - metadata: { folderIds, targetFolderId, targetFolderPath }, - }) - } - - await notifyWorkspaceFilesChanged(workspaceId) - return { success: true, movedItems } - } catch (error) { - logger.error('Failed to move workspace file items', { error }) - if ( - error instanceof WorkspaceFileMoveConflictError || - error instanceof WorkspaceFileFolderConflictError || - getPostgresErrorCode(error) === '23505' - ) { - return { - success: false, - error: getErrorMessage( - error, - 'A file or folder with this name already exists in the destination folder' - ), - errorCode: 'conflict', - } - } - if (error instanceof WorkspaceFileItemsNotFoundError) { - return { success: false, error: error.message, errorCode: 'not_found' } - } - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} - -export async function performRenameWorkspaceFile( - params: PerformRenameWorkspaceFileParams -): Promise { - const { workspaceId, fileId, name, userId } = params - - try { - const file = await renameWorkspaceFile(workspaceId, fileId, name) - - logger.info('Renamed workspace file', { workspaceId, fileId, name: file.name }) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FILE_UPDATED, - resourceType: AuditResourceType.FILE, - resourceId: fileId, - resourceName: file.name, - description: `Renamed file to "${file.name}"`, - }) - - await notifyWorkspaceFilesChanged(workspaceId) - return { success: true, file } - } catch (error) { - logger.error('Failed to rename workspace file', { error }) - if (error instanceof FileConflictError || getPostgresErrorCode(error) === '23505') { - return { success: false, error: toError(error).message, errorCode: 'conflict' } - } - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} - -export async function performRestoreWorkspaceFile( - params: PerformRestoreWorkspaceFileParams -): Promise { - const { workspaceId, fileId, userId } = params - - try { - await restoreWorkspaceFile(workspaceId, fileId) - - logger.info('Restored workspace file', { workspaceId, fileId }) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FILE_RESTORED, - resourceType: AuditResourceType.FILE, - resourceId: fileId, - resourceName: fileId, - description: `Restored workspace file ${fileId}`, - }) - - await notifyWorkspaceFilesChanged(workspaceId) - return { success: true } - } catch (error) { - logger.error('Failed to restore workspace file', { error }) - if (error instanceof FileConflictError || getPostgresErrorCode(error) === '23505') { - return { success: false, error: toError(error).message, errorCode: 'conflict' } - } - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} - -export async function performCreateWorkspaceFileFolder( - params: PerformCreateWorkspaceFileFolderParams -): Promise { - const { workspaceId, userId, name, parentId } = params - - try { - const folder = await createWorkspaceFileFolder({ workspaceId, userId, name, parentId }) - - logger.info('Created workspace file folder', { workspaceId, folderId: folder.id }) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FOLDER_CREATED, - resourceType: AuditResourceType.FOLDER, - resourceId: folder.id, - resourceName: folder.name, - description: `Created file folder "${folder.name}"`, - }) - - await notifyWorkspaceFilesChanged(workspaceId) - return { success: true, folder } - } catch (error) { - logger.error('Failed to create workspace file folder', { error }) - if ( - error instanceof WorkspaceFileFolderConflictError || - getPostgresErrorCode(error) === '23505' - ) { - return { success: false, error: toError(error).message, errorCode: 'conflict' } - } - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} - -export async function performUpdateWorkspaceFileFolder( - params: PerformUpdateWorkspaceFileFolderParams -): Promise { - const { workspaceId, folderId, userId, name, parentId, sortOrder } = params - - try { - const folder = await updateWorkspaceFileFolder({ - workspaceId, - folderId, - name, - parentId, - sortOrder, - }) - - logger.info('Updated workspace file folder', { workspaceId, folderId }) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FOLDER_UPDATED, - resourceType: AuditResourceType.FOLDER, - resourceId: folderId, - resourceName: folder.name, - description: `Updated file folder "${folder.name}"`, - }) - - await notifyWorkspaceFilesChanged(workspaceId) - return { success: true, folder } - } catch (error) { - logger.error('Failed to update workspace file folder', { error }) - if ( - error instanceof WorkspaceFileFolderConflictError || - getPostgresErrorCode(error) === '23505' - ) { - return { - success: false, - error: - getPostgresErrorCode(error) === '23505' - ? 'A folder with this name already exists in this location' - : toError(error).message, - errorCode: 'conflict', - } - } - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} - -export async function performRestoreWorkspaceFileFolder( - params: PerformRestoreWorkspaceFileFolderParams -): Promise { - const { workspaceId, folderId, userId } = params - - try { - const { folder, restoredItems } = await restoreWorkspaceFileFolder(workspaceId, folderId) - - logger.info('Restored workspace file folder', { workspaceId, folderId, restoredItems }) - - recordAudit({ - workspaceId, - actorId: userId, - action: AuditAction.FOLDER_RESTORED, - resourceType: AuditResourceType.FOLDER, - resourceId: folderId, - resourceName: folder.name, - description: `Restored file folder "${folder.name}"`, - metadata: { - affected: { - files: restoredItems.files, - subfolders: Math.max(0, restoredItems.folders - 1), - }, - }, - }) - - await notifyWorkspaceFilesChanged(workspaceId) - return { success: true, folder, restoredItems } - } catch (error) { - logger.error('Failed to restore workspace file folder', { error }) - if (getPostgresErrorCode(error) === '23505') { - return { - success: false, - error: 'A folder with this name already exists in this location', - errorCode: 'conflict', - } - } - const classified = asOrchestrationError(error) - if (classified) { - return { success: false, error: classified.message, errorCode: classified.code } - } - return { success: false, error: toError(error).message, errorCode: 'internal' } - } -} diff --git a/apps/sim/lib/workspace-files/orchestration/index.ts b/apps/sim/lib/workspace-files/orchestration/index.ts index ac07eebc424..df0db7d25ab 100644 --- a/apps/sim/lib/workspace-files/orchestration/index.ts +++ b/apps/sim/lib/workspace-files/orchestration/index.ts @@ -3,35 +3,7 @@ export { MAX_WORKSPACE_FILE_INLINE_BODY_BYTES, } from './content' export { - type PerformCreateWorkspaceFileParams, - type PerformCreateWorkspaceFileResult, - performCreateWorkspaceFile, -} from './create' -export { - type PerformCreateWorkspaceFileFolderParams, - type PerformCreateWorkspaceFileFolderResult, - type PerformDeleteFileFolderByPathResult, type PerformDeleteWorkspaceFileItemsParams, type PerformDeleteWorkspaceFileItemsResult, - type PerformFileFolderPathMutationResult, - type PerformMoveWorkspaceFileItemsParams, - type PerformMoveWorkspaceFileItemsResult, - type PerformRenameWorkspaceFileParams, - type PerformRenameWorkspaceFileResult, - type PerformRestoreWorkspaceFileFolderParams, - type PerformRestoreWorkspaceFileFolderResult, - type PerformRestoreWorkspaceFileParams, - type PerformRestoreWorkspaceFileResult, - type PerformUpdateWorkspaceFileFolderParams, - type PerformUpdateWorkspaceFileFolderResult, - performCreateWorkspaceFileFolder, - performCreateWorkspaceFileFolderAtPath, - performDeleteWorkspaceFileFolderByPath, performDeleteWorkspaceFileItems, - performMoveWorkspaceFileItems, - performRelocateWorkspaceFileFolderByPath, - performRenameWorkspaceFile, - performRestoreWorkspaceFile, - performRestoreWorkspaceFileFolder, - performUpdateWorkspaceFileFolder, } from './file-folder-lifecycle' From c484e435b6b6a35f6f6f4eb6e97aecaf93222933 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:23:27 -0700 Subject: [PATCH 19/30] chore(cleanup): reuse knowledge tag slot constants in the tags service --- apps/sim/lib/knowledge/tags/service.ts | 46 ++++++-------------------- 1 file changed, 10 insertions(+), 36 deletions(-) diff --git a/apps/sim/lib/knowledge/tags/service.ts b/apps/sim/lib/knowledge/tags/service.ts index 9a958d32ca4..6e7fb2381cb 100644 --- a/apps/sim/lib/knowledge/tags/service.ts +++ b/apps/sim/lib/knowledge/tags/service.ts @@ -13,6 +13,9 @@ import { and, eq, inArray, isNotNull, isNull, or, sql } from 'drizzle-orm' import { OrchestrationError } from '@/lib/core/orchestration/types' import type { DbOrTx, DbTransaction } from '@/lib/db/types' import { + ALL_TAG_SLOTS, + type AllTagSlot, + getFieldTypeForSlot, getSlotsForFieldType, isValidSlotForFieldType, SUPPORTED_FIELD_TYPES, @@ -27,25 +30,7 @@ import type { const logger = createLogger('TagsService') -/** Text tag slots */ -const VALID_TEXT_SLOTS = ['tag1', 'tag2', 'tag3', 'tag4', 'tag5', 'tag6', 'tag7'] as const - -const VALID_NUMBER_SLOTS = ['number1', 'number2', 'number3', 'number4', 'number5'] as const -/** Date tag slots (reduced to 2 for write performance) */ -const VALID_DATE_SLOTS = ['date1', 'date2'] as const -/** Boolean tag slots */ -const VALID_BOOLEAN_SLOTS = ['boolean1', 'boolean2', 'boolean3'] as const - -/** All valid tag slots combined */ -const VALID_TAG_SLOTS = [ - ...VALID_TEXT_SLOTS, - ...VALID_NUMBER_SLOTS, - ...VALID_DATE_SLOTS, - ...VALID_BOOLEAN_SLOTS, -] as const - -type ValidTagSlot = (typeof VALID_TAG_SLOTS)[number] -type ClearedTagValues = Partial> +type ClearedTagValues = Partial> const TAG_MUTATION_STATEMENT_TIMEOUT_MS = 120_000 const TAG_MUTATION_LOCK_TIMEOUT_MS = 5_000 const TAG_MUTATION_IDLE_TIMEOUT_MS = 30_000 @@ -63,9 +48,9 @@ export class KnowledgeTagProvenanceConflictError extends OrchestrationError { /** * Validates that a tag slot is a valid slot name */ -function validateTagSlot(tagSlot: string): asserts tagSlot is ValidTagSlot { - if (!VALID_TAG_SLOTS.includes(tagSlot as ValidTagSlot)) { - throw new Error(`Invalid tag slot: ${tagSlot}. Must be one of: ${VALID_TAG_SLOTS.join(', ')}`) +function validateTagSlot(tagSlot: string): asserts tagSlot is AllTagSlot { + if (!ALL_TAG_SLOTS.includes(tagSlot as AllTagSlot)) { + throw new Error(`Invalid tag slot: ${tagSlot}. Must be one of: ${ALL_TAG_SLOTS.join(', ')}`) } } @@ -120,7 +105,7 @@ async function assertKnowledgeBaseTagsCanBeClearedInTx( async function clearTagSlotsInTx( tx: DbTransaction, knowledgeBaseId: string, - tagSlots: readonly ValidTagSlot[] + tagSlots: readonly AllTagSlot[] ): Promise { if (tagSlots.length === 0) return @@ -150,17 +135,6 @@ async function clearTagSlotsInTx( ) } -/** - * Get the field type for a tag slot - */ -function getFieldTypeForSlot(tagSlot: string): string | null { - if ((VALID_TEXT_SLOTS as readonly string[]).includes(tagSlot)) return 'text' - if ((VALID_NUMBER_SLOTS as readonly string[]).includes(tagSlot)) return 'number' - if ((VALID_DATE_SLOTS as readonly string[]).includes(tagSlot)) return 'date' - if ((VALID_BOOLEAN_SLOTS as readonly string[]).includes(tagSlot)) return 'boolean' - return null -} - /** * Get the next available slot for a knowledge base and field type */ @@ -499,7 +473,7 @@ export async function createOrUpdateTagDefinitionsBulk( const newDefinition = { id: generateId(), knowledgeBaseId, - tagSlot: finalTagSlot as ValidTagSlot, + tagSlot: finalTagSlot as AllTagSlot, displayName, fieldType, createdAt: new Date(), @@ -746,7 +720,7 @@ async function insertTagDefinition( const newDefinition = { id: tagDefinitionId, knowledgeBaseId: data.knowledgeBaseId, - tagSlot: data.tagSlot as ValidTagSlot, + tagSlot: data.tagSlot as AllTagSlot, displayName: data.displayName, fieldType: data.fieldType, createdAt: now, From 2a805e18c0d44a9e63eeedec011c99199cb5ef95 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:23:27 -0700 Subject: [PATCH 20/30] chore(cleanup): share escapeLikePattern from @sim/utils/string --- apps/sim/ee/access-requests/lib/repository.ts | 2 +- apps/sim/lib/api/list-query.ts | 4 ---- apps/sim/lib/knowledge/application/connectors.ts | 2 +- apps/sim/lib/knowledge/documents/tag-filter.ts | 7 ++----- apps/sim/lib/knowledge/search/tag-filters.ts | 2 +- apps/sim/lib/knowledge/tags/utils.ts | 9 --------- apps/sim/lib/mothership/docs/docs-search.ts | 2 +- apps/sim/lib/table/rows/service.ts | 2 +- apps/sim/lib/table/sql.ts | 11 ++--------- apps/sim/lib/webhooks/quickbooks-credentials.ts | 2 +- apps/sim/lib/workflows/custom-blocks/operations.ts | 3 ++- apps/sim/lib/workspace-files/search/pattern.test.ts | 4 +--- apps/sim/lib/workspace-files/search/pattern.ts | 12 ++++-------- packages/utils/src/string.ts | 12 ++++++++++++ 14 files changed, 29 insertions(+), 45 deletions(-) diff --git a/apps/sim/ee/access-requests/lib/repository.ts b/apps/sim/ee/access-requests/lib/repository.ts index 0c09a648719..21605698497 100644 --- a/apps/sim/ee/access-requests/lib/repository.ts +++ b/apps/sim/ee/access-requests/lib/repository.ts @@ -1,7 +1,7 @@ import { permissionAccessRequest, user } from '@sim/db/schema' +import { escapeLikePattern } from '@sim/utils/string' import { and, count, desc, eq, ilike, or, type SQL } from 'drizzle-orm' import { - escapeLikePattern, keysetColumns, keysetPage, listOrderBy, diff --git a/apps/sim/lib/api/list-query.ts b/apps/sim/lib/api/list-query.ts index cc06165d61b..2fecf05fb32 100644 --- a/apps/sim/lib/api/list-query.ts +++ b/apps/sim/lib/api/list-query.ts @@ -32,10 +32,6 @@ export type ListSortOrder = (typeof LIST_SORT_ORDERS)[number] * Escapes LIKE/ILIKE wildcards so `%`, `_`, and `\` in a caller's term match * themselves. Postgres treats `\` as the default LIKE escape character, so no * explicit `ESCAPE` clause is needed. - * - * `lib/table/sql.ts` carries its own copy for the JSONB predicate engine; the - * two are worth folding together, but that module is table-specific and pulls - * the whole column-type registry with it. */ export function escapeLikePattern(value: string): string { return value.replace(/[\\%_]/g, '\\$&') diff --git a/apps/sim/lib/knowledge/application/connectors.ts b/apps/sim/lib/knowledge/application/connectors.ts index 45994904047..70af6e05231 100644 --- a/apps/sim/lib/knowledge/application/connectors.ts +++ b/apps/sim/lib/knowledge/application/connectors.ts @@ -10,6 +10,7 @@ import { knowledgeConnectorSyncLog, } from '@sim/db/schema' import { toError } from '@sim/utils/errors' +import { escapeLikePattern } from '@sim/utils/string' import { and, asc, desc, eq, inArray, isNull, lt, or, sql } from 'drizzle-orm' import type { ConnectorDocumentFilter } from '@/lib/api/contracts/knowledge/connectors' import type { BillingAttributionSnapshot } from '@/lib/billing/core/billing-attribution' @@ -105,7 +106,6 @@ import type { KnowledgeOperationSource, KnowledgeOrchestrationResult, } from '@/lib/knowledge/orchestration/shared' -import { escapeLikePattern } from '@/lib/knowledge/tags/utils' import { credentialProviderMatchesService, type ServiceProviderIdentity } from '@/lib/oauth' import { ServiceAccountTokenError } from '@/lib/oauth/credential-service' import { CAPABILITY_RULES, refuseCapability } from '@/lib/permission-groups/capabilities' diff --git a/apps/sim/lib/knowledge/documents/tag-filter.ts b/apps/sim/lib/knowledge/documents/tag-filter.ts index ca44a6d2dee..2e8c4ef6ac7 100644 --- a/apps/sim/lib/knowledge/documents/tag-filter.ts +++ b/apps/sim/lib/knowledge/documents/tag-filter.ts @@ -1,10 +1,7 @@ import { document } from '@sim/db/schema' +import { escapeLikePattern } from '@sim/utils/string' import { and, eq, gt, gte, lt, lte, ne, type SQL, sql } from 'drizzle-orm' -import { - buildDateTagCondition, - coerceTagFilterValue, - escapeLikePattern, -} from '@/lib/knowledge/tags/utils' +import { buildDateTagCondition, coerceTagFilterValue } from '@/lib/knowledge/tags/utils' /** * A single tag filter applied to a document list query. diff --git a/apps/sim/lib/knowledge/search/tag-filters.ts b/apps/sim/lib/knowledge/search/tag-filters.ts index eaf43248699..3368190877e 100644 --- a/apps/sim/lib/knowledge/search/tag-filters.ts +++ b/apps/sim/lib/knowledge/search/tag-filters.ts @@ -1,4 +1,5 @@ import { document, embedding } from '@sim/db/schema' +import { escapeLikePattern } from '@sim/utils/string' import { and, eq, inArray, type SQL, sql } from 'drizzle-orm' import { knowledgeAccessCondition } from '@/lib/knowledge/access/predicate' import { runSearchQuery } from '@/lib/knowledge/search/budget' @@ -15,7 +16,6 @@ import { import { buildDateTagCondition, coerceTagFilterValue, - escapeLikePattern, uncompilableTagFilterError, } from '@/lib/knowledge/tags/utils' import type { StructuredFilter } from '@/lib/knowledge/types' diff --git a/apps/sim/lib/knowledge/tags/utils.ts b/apps/sim/lib/knowledge/tags/utils.ts index 940ef784427..3bb57e5e80a 100644 --- a/apps/sim/lib/knowledge/tags/utils.ts +++ b/apps/sim/lib/knowledge/tags/utils.ts @@ -61,15 +61,6 @@ export function coerceTagFilterValue( } } -/** - * Escapes the LIKE metacharacters in a tag filter value so a `%` or `_` a - * caller typed matches itself instead of acting as a wildcard. Both filter - * builders pair this with `ESCAPE '\'`. - */ -export function escapeLikePattern(value: string): string { - return value.replace(/\\/g, '\\\\').replace(/%/g, '\\%').replace(/_/g, '\\_') -} - /** * Compiles a date tag filter as a half-open range on the raw column, which a * btree on the slot serves; a `column::date` comparison never can. Date slots diff --git a/apps/sim/lib/mothership/docs/docs-search.ts b/apps/sim/lib/mothership/docs/docs-search.ts index d84b7e64930..4fad48ebbc1 100644 --- a/apps/sim/lib/mothership/docs/docs-search.ts +++ b/apps/sim/lib/mothership/docs/docs-search.ts @@ -1,8 +1,8 @@ import { db } from '@sim/db' import { docsEmbeddings } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { escapeLikePattern } from '@sim/utils/string' import { and, eq, like, ne, notLike, or, sql } from 'drizzle-orm' -import { escapeLikePattern } from '@/lib/api/list-query' import { DOCS_EMBEDDING_DIMENSIONS } from '@/lib/chunkers/constants' import { OrchestrationError } from '@/lib/core/orchestration/types' import { DEFAULT_EMBEDDING_MODEL } from '@/lib/knowledge/embedding-models' diff --git a/apps/sim/lib/table/rows/service.ts b/apps/sim/lib/table/rows/service.ts index ae42156c51a..233812fa612 100644 --- a/apps/sim/lib/table/rows/service.ts +++ b/apps/sim/lib/table/rows/service.ts @@ -1,3 +1,4 @@ +import { escapeLikePattern } from '@sim/utils/string' /** * Row CRUD + query operations for the table service layer. * @@ -77,7 +78,6 @@ import { buildFilterClause, buildPredicateClause, buildSortClause, - escapeLikePattern, uniqueValuePredicate, } from '@/lib/table/sql' import { fireTableTrigger } from '@/lib/table/trigger' diff --git a/apps/sim/lib/table/sql.ts b/apps/sim/lib/table/sql.ts index 0f287ead4cb..e5a8b739515 100644 --- a/apps/sim/lib/table/sql.ts +++ b/apps/sim/lib/table/sql.ts @@ -6,7 +6,7 @@ */ import { isRecordLike } from '@sim/utils/object' -import { truncate } from '@sim/utils/string' +import { escapeLikePattern, truncate } from '@sim/utils/string' import type { SQL } from 'drizzle-orm' import { sql } from 'drizzle-orm' import { getColumnId } from '@/lib/table/column-keys' @@ -959,11 +959,6 @@ function buildComparisonClause( : sql`${cell} ${sql.raw(operator)} ${value}` } -/** Escapes LIKE/ILIKE wildcard characters so they match literally */ -export function escapeLikePattern(value: string): string { - return value.replace(/[\\%_]/g, '\\$&') -} - /** * General LIKE/ILIKE pattern match (the `like`/`ilike` ops). The caller's `*` * is the only wildcard — it maps to SQL `%`; any literal `%`/`_`/`\` in the @@ -981,9 +976,7 @@ function buildPatternClause( options: { caseInsensitive: boolean; negate?: boolean } ): SQL { const escapedField = field.replace(/'/g, "''") - const pattern = String(value) - .replace(/[\\%_]/g, '\\$&') - .replace(/\*/g, '%') + const pattern = escapeLikePattern(String(value)).replace(/\*/g, '%') const cell = sql.raw(`${tableName}.data->>'${escapedField}'`) const match = options.caseInsensitive ? sql`${cell} ILIKE ${pattern}` diff --git a/apps/sim/lib/webhooks/quickbooks-credentials.ts b/apps/sim/lib/webhooks/quickbooks-credentials.ts index b7d1b3b539d..baed664cbf6 100644 --- a/apps/sim/lib/webhooks/quickbooks-credentials.ts +++ b/apps/sim/lib/webhooks/quickbooks-credentials.ts @@ -1,7 +1,7 @@ import { db } from '@sim/db' import { account, credential } from '@sim/db/schema' +import { escapeLikePattern } from '@sim/utils/string' import { and, eq, like } from 'drizzle-orm' -import { escapeLikePattern } from '@/lib/api/list-query' import { normalizeQuickBooksRealmId, parseQuickBooksAccountId, diff --git a/apps/sim/lib/workflows/custom-blocks/operations.ts b/apps/sim/lib/workflows/custom-blocks/operations.ts index ffafec56483..3d901e70aa9 100644 --- a/apps/sim/lib/workflows/custom-blocks/operations.ts +++ b/apps/sim/lib/workflows/custom-blocks/operations.ts @@ -9,6 +9,7 @@ import { import { createLogger } from '@sim/logger' import { generateId, generateShortId } from '@sim/utils/id' import { omit } from '@sim/utils/object' +import { escapeLikePattern } from '@sim/utils/string' import { and, eq, isNull, ne, sql } from 'drizzle-orm' import { isOrganizationFeatureEntitled } from '@/lib/billing/core/subscription' import { acquireOrganizationMutationLock } from '@/lib/billing/organizations/membership' @@ -679,7 +680,7 @@ export async function getCustomBlockUsageCounts( ) // Escape LIKE wildcards — the `_`s in `custom_block_` would otherwise match // any character and let unrelated states through to the jsonb parse. - const likePattern = `%${blockType.replace(/[\\%_]/g, '\\$&')}%` + const likePattern = `%${escapeLikePattern(blockType)}%` const [liveRows, deployedRows] = await Promise.all([ db diff --git a/apps/sim/lib/workspace-files/search/pattern.test.ts b/apps/sim/lib/workspace-files/search/pattern.test.ts index c98bf5639f9..cdd6ea42b75 100644 --- a/apps/sim/lib/workspace-files/search/pattern.test.ts +++ b/apps/sim/lib/workspace-files/search/pattern.test.ts @@ -1,7 +1,6 @@ import { describe, expect, it } from 'vitest' import { compileFileSearchPattern, - escapeFileSearchLikePattern, FileSearchPatternError, isFileSearchCaseSensitive, } from '@/lib/workspace-files/search/pattern' @@ -23,11 +22,10 @@ describe('compileFileSearchPattern', () => { }) describe('exact mode', () => { - it('implements Unicode smart-case and escapes LIKE metacharacters', () => { + it('implements Unicode smart-case', () => { expect(isFileSearchCaseSensitive('résumé')).toBe(false) expect(isFileSearchCaseSensitive('Résumé')).toBe(true) expect(isFileSearchCaseSensitive('東京A')).toBe(true) - expect(escapeFileSearchLikePattern('100%_done\\')).toBe('100\\%\\_done\\\\') }) it('wraps the escaped query for LIKE and keeps the raw text for ranking', () => { diff --git a/apps/sim/lib/workspace-files/search/pattern.ts b/apps/sim/lib/workspace-files/search/pattern.ts index 644db8c2910..e7e99a3b3c9 100644 --- a/apps/sim/lib/workspace-files/search/pattern.ts +++ b/apps/sim/lib/workspace-files/search/pattern.ts @@ -1,4 +1,5 @@ import { getErrorMessage } from '@sim/utils/errors' +import { escapeLikePattern } from '@sim/utils/string' import { FILE_SEARCH_CANDIDATE_LITERAL_CHARS, FILE_SEARCH_MAX_QUERY_LENGTH, @@ -51,10 +52,6 @@ export function isFileSearchCaseSensitive(text: string): boolean { return /\p{Lu}/u.test(text) } -export function escapeFileSearchLikePattern(query: string): string { - return query.replace(/[\\%_]/g, '\\$&') -} - /** * Maps a match found in the case-folded line back onto the original one. * Folding is not length-preserving — `İ` lowercases to two code units — so the @@ -108,9 +105,9 @@ function compileExactPattern(query: string): CompiledFileSearchPattern { return { mode: 'exact', caseSensitive, - sqlPattern: `%${escapeFileSearchLikePattern(query)}%`, + sqlPattern: `%${escapeLikePattern(query)}%`, candidatePatterns: [ - `%${escapeFileSearchLikePattern([...query].slice(0, FILE_SEARCH_CANDIDATE_LITERAL_CHARS).join(''))}%`, + `%${escapeLikePattern([...query].slice(0, FILE_SEARCH_CANDIDATE_LITERAL_CHARS).join(''))}%`, ], literalText: query, findMatchRange: (segment) => findLiteralMatchRange(segment, query, caseSensitive), @@ -146,8 +143,7 @@ function compileRegexPattern(query: string): CompiledFileSearchPattern { caseSensitive, sqlPattern: analysis.postgresSource, candidatePatterns: - analysis.candidateLiterals?.map((literal) => `%${escapeFileSearchLikePattern(literal)}%`) ?? - null, + analysis.candidateLiterals?.map((literal) => `%${escapeLikePattern(literal)}%`) ?? null, literalText: null, findMatchRange: () => null, } diff --git a/packages/utils/src/string.ts b/packages/utils/src/string.ts index 8cf8dbb5c33..83343a07d46 100644 --- a/packages/utils/src/string.ts +++ b/packages/utils/src/string.ts @@ -371,6 +371,18 @@ export function escapeRegExp(value: string): string { return value.replace(REGEX_METACHARACTERS, '\\$&') } +/** + * Escapes the SQL LIKE/ILIKE metacharacters `%`, `_`, and `\` in `value` so + * each matches itself. Postgres uses `\` as the default LIKE escape character, + * so the result needs no explicit `ESCAPE` clause. + * + * @example + * escapeLikePattern('100%_done') // '100\\%\\_done' + */ +export function escapeLikePattern(value: string): string { + return value.replace(/[\\%_]/g, '\\$&') +} + /** Reports whether `value` carries a character {@link escapeRegExp} would escape. */ export function hasRegexMetacharacter(value: string): boolean { return REGEX_METACHARACTER.test(value) From 6594698ff04f36f39edebd701facef832f78398c Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:23:28 -0700 Subject: [PATCH 21/30] chore(cleanup): reuse zip-guard EOCD parsing in the file sniffer --- apps/sim/lib/file-parsers/sniff.ts | 58 ++++++-------------------- apps/sim/lib/file-parsers/zip-guard.ts | 16 +++---- 2 files changed, 20 insertions(+), 54 deletions(-) diff --git a/apps/sim/lib/file-parsers/sniff.ts b/apps/sim/lib/file-parsers/sniff.ts index a976b97e22b..0002c98be5f 100644 --- a/apps/sim/lib/file-parsers/sniff.ts +++ b/apps/sim/lib/file-parsers/sniff.ts @@ -2,7 +2,16 @@ import { FileParserError } from '@/lib/file-parsers/errors' import { isEncryptedOoxmlContainer } from '@/lib/file-parsers/ooxml-encryption' import type { FileParseOptions } from '@/lib/file-parsers/types' import { decodeTextBuffer, detectBomlessUtf16 } from '@/lib/file-parsers/utils' -import { isZipShaped } from '@/lib/file-parsers/zip-guard' +import { + CENTRAL_DIRECTORY_HEADER_MIN_SIZE, + CENTRAL_DIRECTORY_HEADER_SIGNATURE, + COMPRESSION_METHOD_STORED, + EOCD_MIN_SIZE, + findEocdOffset, + isZipShaped, + LOCAL_FILE_HEADER_MIN_SIZE, + locateCentralDirectory, +} from '@/lib/file-parsers/zip-guard' /** * What the bytes of a buffer look like, independent of the caller-supplied @@ -34,18 +43,6 @@ const OLE2_SIGNATURE = Buffer.from([0xd0, 0xcf, 0x11, 0xe0, 0xa1, 0xb1, 0x1a, 0x /** An RTF file is a group opening with the `rtf` control word; nothing may precede it. */ const RTF_SIGNATURE = Buffer.from('{\\rtf', 'latin1') -const EOCD_SIGNATURE = 0x06054b50 -const EOCD_MIN_SIZE = 22 -const MAX_EOCD_COMMENT_SIZE = 0xffff -const ZIP64_EOCD_LOCATOR_SIGNATURE = 0x07064b50 -const ZIP64_EOCD_LOCATOR_SIZE = 20 -const ZIP64_EOCD_SIGNATURE = 0x06064b50 -const CENTRAL_DIRECTORY_HEADER_SIGNATURE = 0x02014b50 -const CENTRAL_DIRECTORY_HEADER_MIN_SIZE = 46 -const LOCAL_FILE_HEADER_MIN_SIZE = 30 -const COMPRESSION_METHOD_STORED = 0 -const UINT16_SENTINEL = 0xffff -const UINT32_SENTINEL = 0xffffffff /** Enough to reach the first `word/`, `xl/` or `ppt/` part in any real package. */ const MAX_INSPECTED_ENTRIES = 256 const MAX_MIMETYPE_BYTES = 128 @@ -63,37 +60,6 @@ interface ZipEntry { localHeaderOffset: number } -/** Same EOCD anchoring as the zip guard: only a record whose comment ends the buffer counts. */ -function findEocdOffset(buffer: Buffer): number { - const minStart = Math.max(0, buffer.length - EOCD_MIN_SIZE - MAX_EOCD_COMMENT_SIZE) - for (let offset = buffer.length - EOCD_MIN_SIZE; offset >= minStart; offset--) { - if (buffer.readUInt32LE(offset) !== EOCD_SIGNATURE) continue - const commentLength = buffer.readUInt16LE(offset + 20) - if (offset + EOCD_MIN_SIZE + commentLength === buffer.length) return offset - } - return -1 -} - -function locateCentralDirectory(buffer: Buffer, eocdOffset: number): number | null { - const entryCount = buffer.readUInt16LE(eocdOffset + 10) - const directoryOffset = buffer.readUInt32LE(eocdOffset + 16) - if (entryCount !== UINT16_SENTINEL && directoryOffset !== UINT32_SENTINEL) { - return directoryOffset - } - - const locatorOffset = eocdOffset - ZIP64_EOCD_LOCATOR_SIZE - if (locatorOffset < 0 || buffer.readUInt32LE(locatorOffset) !== ZIP64_EOCD_LOCATOR_SIGNATURE) { - return null - } - const zip64Eocd = buffer.readBigUInt64LE(locatorOffset + 8) - if (zip64Eocd > BigInt(buffer.length - 56)) return null - const zip64EocdOffset = Number(zip64Eocd) - if (buffer.readUInt32LE(zip64EocdOffset) !== ZIP64_EOCD_SIGNATURE) return null - const zip64DirectoryOffset = buffer.readBigUInt64LE(zip64EocdOffset + 48) - if (zip64DirectoryOffset > BigInt(buffer.length)) return null - return Number(zip64DirectoryOffset) -} - /** * Reads central-directory entry names without decompressing anything. Returns * `null` for a buffer whose directory cannot be located. Bounded to the first @@ -104,8 +70,8 @@ function readZipEntries(buffer: Buffer): ZipEntry[] | null { if (buffer.length < EOCD_MIN_SIZE) return null const eocdOffset = findEocdOffset(buffer) if (eocdOffset < 0) return null - const directoryOffset = locateCentralDirectory(buffer, eocdOffset) - if (directoryOffset === null) return null + const directoryOffset = locateCentralDirectory(buffer, eocdOffset)?.offset + if (directoryOffset === undefined) return null const entries: ZipEntry[] = [] let cursor = directoryOffset diff --git a/apps/sim/lib/file-parsers/zip-guard.ts b/apps/sim/lib/file-parsers/zip-guard.ts index 096d0308a9c..b77ff6034bf 100644 --- a/apps/sim/lib/file-parsers/zip-guard.ts +++ b/apps/sim/lib/file-parsers/zip-guard.ts @@ -29,18 +29,18 @@ const LOCAL_FILE_HEADER_SIGNATURE = 0x04034b50 const EOCD_SIGNATURE = 0x06054b50 const ZIP64_EOCD_LOCATOR_SIGNATURE = 0x07064b50 const ZIP64_EOCD_SIGNATURE = 0x06064b50 -const CENTRAL_DIRECTORY_HEADER_SIGNATURE = 0x02014b50 +export const CENTRAL_DIRECTORY_HEADER_SIGNATURE = 0x02014b50 const ZIP64_EXTRA_FIELD_ID = 0x0001 -const EOCD_MIN_SIZE = 22 +export const EOCD_MIN_SIZE = 22 const ZIP64_EOCD_LOCATOR_SIZE = 20 -const CENTRAL_DIRECTORY_HEADER_MIN_SIZE = 46 -const LOCAL_FILE_HEADER_MIN_SIZE = 30 +export const CENTRAL_DIRECTORY_HEADER_MIN_SIZE = 46 +export const LOCAL_FILE_HEADER_MIN_SIZE = 30 const MAX_EOCD_COMMENT_SIZE = 0xffff const UINT32_SENTINEL = 0xffffffff const UINT16_SENTINEL = 0xffff -const COMPRESSION_METHOD_STORED = 0 +export const COMPRESSION_METHOD_STORED = 0 const COMPRESSION_METHOD_DEFLATE = 8 /** General-purpose bit 3: sizes live in a trailing data descriptor, not the local header. */ @@ -99,7 +99,7 @@ export function isZipShaped(buffer: Buffer): boolean { * at the buffer tail, so a decoy EOCD signature planted in the comment region * cannot redirect the guard to a smaller, attacker-chosen central directory. */ -function findEocdOffset(buffer: Buffer): number { +export function findEocdOffset(buffer: Buffer): number { const minStart = Math.max(0, buffer.length - EOCD_MIN_SIZE - MAX_EOCD_COMMENT_SIZE) for (let offset = buffer.length - EOCD_MIN_SIZE; offset >= minStart; offset--) { if (buffer.readUInt32LE(offset) !== EOCD_SIGNATURE) { @@ -113,7 +113,7 @@ function findEocdOffset(buffer: Buffer): number { return -1 } -interface CentralDirectoryLocation { +export interface CentralDirectoryLocation { offset: number entryCount: number } @@ -122,7 +122,7 @@ interface CentralDirectoryLocation { * Resolve the central directory offset and entry count, following the ZIP64 * end-of-central-directory chain when the 32-bit fields are saturated. */ -function locateCentralDirectory( +export function locateCentralDirectory( buffer: Buffer, eocdOffset: number ): CentralDirectoryLocation | null { From 9993fc2401b70cb3d79f531af56d8a0d89042995 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:23:29 -0700 Subject: [PATCH 22/30] chore(cleanup): use isRecordLike for inline record guards --- .../knowledge/documents/processing-outbox-handler.ts | 3 ++- apps/sim/lib/knowledge/secret-provenance-selection.ts | 3 ++- apps/sim/lib/table/columns/workflow-references.ts | 11 ++++------- apps/sim/lib/table/import.ts | 3 ++- apps/sim/lib/table/orchestration/import-resource.ts | 7 ++++--- apps/sim/lib/table/query-builder/converters.ts | 2 +- apps/sim/lib/table/query-builder/field-names.ts | 11 ++++------- apps/sim/lib/table/rows/cursor.ts | 3 ++- apps/sim/lib/table/rows/run-state.ts | 3 ++- apps/sim/lib/table/rows/secret-provenance.ts | 3 ++- apps/sim/lib/table/workflow-group-cancellation.ts | 5 +++-- .../workspace/workspace-file-live-doc-outbox.ts | 3 ++- .../workspace-file-storage-cleanup-outbox.ts | 3 ++- 13 files changed, 32 insertions(+), 28 deletions(-) diff --git a/apps/sim/lib/knowledge/documents/processing-outbox-handler.ts b/apps/sim/lib/knowledge/documents/processing-outbox-handler.ts index 71581bea323..478c7babca6 100644 --- a/apps/sim/lib/knowledge/documents/processing-outbox-handler.ts +++ b/apps/sim/lib/knowledge/documents/processing-outbox-handler.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' import { assertBillingAttributionSnapshot } from '@/lib/billing/core/billing-attribution' import { env, envNumber } from '@/lib/core/config/env' import { isTriggerAvailable } from '@/lib/core/config/trigger-availability' @@ -67,7 +68,7 @@ import { } from '@/lib/knowledge/documents/storage-cleanup' function requirePayloadRecord(payload: unknown): Record { - if (!payload || typeof payload !== 'object' || Array.isArray(payload)) { + if (!isRecordLike(payload)) { throw new Error('Knowledge document processing outbox payload must be an object') } return payload as Record diff --git a/apps/sim/lib/knowledge/secret-provenance-selection.ts b/apps/sim/lib/knowledge/secret-provenance-selection.ts index e249c80c2bd..838ecb4ec3b 100644 --- a/apps/sim/lib/knowledge/secret-provenance-selection.ts +++ b/apps/sim/lib/knowledge/secret-provenance-selection.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' export interface KnowledgeDocumentTagProvenanceTarget { tagName: string value: unknown @@ -16,7 +17,7 @@ export function parseKnowledgeDocumentTagProvenanceTargets( const parsed: unknown = JSON.parse(documentTagsData) if (!Array.isArray(parsed)) return [] return parsed.flatMap((candidate) => { - if (!candidate || typeof candidate !== 'object' || Array.isArray(candidate)) return [] + if (!isRecordLike(candidate)) return [] const record = candidate as Record const tagName = typeof record.tagName === 'string' ? record.tagName.trim() : '' if (!tagName || record.value === undefined || record.value === null || record.value === '') { diff --git a/apps/sim/lib/table/columns/workflow-references.ts b/apps/sim/lib/table/columns/workflow-references.ts index 3d43e4abdc7..6ae2d563436 100644 --- a/apps/sim/lib/table/columns/workflow-references.ts +++ b/apps/sim/lib/table/columns/workflow-references.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' /** * Finds workflow Table blocks whose saved configuration still names a column by * its old name after a rename. @@ -39,13 +40,9 @@ const TABLE_ID_SUB_BLOCKS = ['manualTableId', 'tableSelector', 'tableId'] as con const TABLE_BLOCK_TYPE = 'table_v2' -function isRecord(value: unknown): value is Record { - return typeof value === 'object' && value !== null && !Array.isArray(value) -} - function subBlockValue(subBlocks: SubBlockValues, id: string): unknown { const entry = subBlocks[id] - return isRecord(entry) ? entry.value : undefined + return isRecordLike(entry) ? entry.value : undefined } /** The raw text of a sub-block, or `undefined` when it holds nothing. */ @@ -70,7 +67,7 @@ export function isTableBlockBoundTo(subBlocks: SubBlockValues, tableId: string): /** Every column a row payload names: the keys of the `{ column: value }` object. */ function collectDataFieldNames(root: unknown): string[] { - return isRecord(root) ? Object.keys(root) : [] + return isRecordLike(root) ? Object.keys(root) : [] } const FIELD_COLLECTORS: Record string[]> = { @@ -147,7 +144,7 @@ export async function findUnmigratedTableBlockReferences(input: { const unmigrated: UnmigratedTableBlockReference[] = [] for (const row of rows) { - if (!isRecord(row.subBlocks)) continue + if (!isRecordLike(row.subBlocks)) continue const subBlocks = row.subBlocks as SubBlockValues if (!isTableBlockBoundTo(subBlocks, input.tableId)) continue const fields = collectTableBlockColumnReferences(subBlocks, input.columnName) diff --git a/apps/sim/lib/table/import.ts b/apps/sim/lib/table/import.ts index 9e71e66d345..ac8802acec3 100644 --- a/apps/sim/lib/table/import.ts +++ b/apps/sim/lib/table/import.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' /** * Shared CSV import helpers for user-defined tables. * @@ -760,7 +761,7 @@ export function parseJsonRows(buffer: Buffer | string): { } const headerSet = new Set() for (const row of parsed) { - if (typeof row !== 'object' || row === null || Array.isArray(row)) { + if (!isRecordLike(row)) { throw new OrchestrationError( 'validation', 'Each element in the JSON array must be a plain object' diff --git a/apps/sim/lib/table/orchestration/import-resource.ts b/apps/sim/lib/table/orchestration/import-resource.ts index 9759073f121..ac7468efb7a 100644 --- a/apps/sim/lib/table/orchestration/import-resource.ts +++ b/apps/sim/lib/table/orchestration/import-resource.ts @@ -4,6 +4,7 @@ import { tableJobs } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { getErrorMessage } from '@sim/utils/errors' import { generateId } from '@sim/utils/id' +import { isRecordLike } from '@sim/utils/object' import { and, eq } from 'drizzle-orm' import { type V2CreateTableImportBody, @@ -527,7 +528,7 @@ function parseRejectionSummary(payload: unknown): TableImportRejectionSummary { cellsRejected: 0, rejectedSamples: [], } - if (!payload || typeof payload !== 'object' || Array.isArray(payload)) return empty + if (!isRecordLike(payload)) return empty const candidate = payload as Partial const samples = Array.isArray(candidate.rejectedSamples) ? candidate.rejectedSamples : [] return { @@ -548,7 +549,7 @@ function parseRejectionSummary(payload: unknown): TableImportRejectionSummary { * handed unchanged to a `.strict()` schema and turn a read of the import into a 500. */ function parseRejectedSample(value: unknown): CsvSkippedRecord | null { - if (!value || typeof value !== 'object' || Array.isArray(value)) return null + if (!isRecordLike(value)) return null const candidate = value as Partial if (typeof candidate.code !== 'string' || typeof candidate.message !== 'string') return null const line = @@ -570,7 +571,7 @@ function parseRejectionCount(value: unknown): number { * an ordinary "not this resource" answer rather than an error condition. */ function parseImportJobPayload(payload: unknown): ParsedTableImportPayload | null { - if (!payload || typeof payload !== 'object' || Array.isArray(payload)) return null + if (!isRecordLike(payload)) return null const candidate = payload as Partial if ( candidate.kind !== 'table_import' || diff --git a/apps/sim/lib/table/query-builder/converters.ts b/apps/sim/lib/table/query-builder/converters.ts index 4063bf53b19..9a226c287f9 100644 --- a/apps/sim/lib/table/query-builder/converters.ts +++ b/apps/sim/lib/table/query-builder/converters.ts @@ -251,7 +251,7 @@ function parseScalar(value: string): JsonValue { } function parseFilterGroup(group: Filter): FilterRule[] { - if (!group || typeof group !== 'object' || Array.isArray(group)) return [] + if (!isRecordLike(group)) return [] const rules: FilterRule[] = [] diff --git a/apps/sim/lib/table/query-builder/field-names.ts b/apps/sim/lib/table/query-builder/field-names.ts index d657ef99b29..be9fed8176c 100644 --- a/apps/sim/lib/table/query-builder/field-names.ts +++ b/apps/sim/lib/table/query-builder/field-names.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' /** * Column names an authored filter or sort names, read from untrusted JSON. * @@ -8,10 +9,6 @@ * these are advisory readers of persisted block state. */ -function isRecord(value: unknown): value is Record { - return typeof value === 'object' && value !== null && !Array.isArray(value) -} - /** * Every `field` a predicate tree names, in document order: a bare * `{ field, op, value }` condition, or nested `{ all | any: [...] }` groups. @@ -29,7 +26,7 @@ export function collectPredicateFieldNames(root: unknown): string[] { visitLater(node) continue } - if (!isRecord(node)) continue + if (!isRecordLike(node)) continue if (typeof node.field === 'string') names.push(node.field) if (Array.isArray(node.any)) visitLater(node.any) if (Array.isArray(node.all)) visitLater(node.all) @@ -44,8 +41,8 @@ export function collectPredicateFieldNames(root: unknown): string[] { export function collectSortFieldNames(root: unknown): string[] { if (Array.isArray(root)) { return root.flatMap((entry) => - isRecord(entry) && typeof entry.field === 'string' ? [entry.field] : [] + isRecordLike(entry) && typeof entry.field === 'string' ? [entry.field] : [] ) } - return isRecord(root) ? Object.keys(root) : [] + return isRecordLike(root) ? Object.keys(root) : [] } diff --git a/apps/sim/lib/table/rows/cursor.ts b/apps/sim/lib/table/rows/cursor.ts index 3ceea330316..c0e47669ef9 100644 --- a/apps/sim/lib/table/rows/cursor.ts +++ b/apps/sim/lib/table/rows/cursor.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' /** * Opaque pagination cursor for the v2 table-query surface. * @@ -235,7 +236,7 @@ export function decodeCursor(token: string): { } catch { invalidCursor() } - if (typeof payload !== 'object' || payload === null || Array.isArray(payload)) { + if (!isRecordLike(payload)) { invalidCursor() } diff --git a/apps/sim/lib/table/rows/run-state.ts b/apps/sim/lib/table/rows/run-state.ts index b751a19041b..14d646275b7 100644 --- a/apps/sim/lib/table/rows/run-state.ts +++ b/apps/sim/lib/table/rows/run-state.ts @@ -1,3 +1,4 @@ +import { isRecordLike } from '@sim/utils/object' /** * Shared normalizers for the `tableRowExecutions` sidecar columns that are * stored looser than every consumer declares them. @@ -17,7 +18,7 @@ * rather than publish an empty map. */ export function normalizeBlockErrors(value: unknown): Record | undefined { - if (!value || typeof value !== 'object' || Array.isArray(value)) return undefined + if (!isRecordLike(value)) return undefined const blockErrors: Record = {} for (const [blockId, error] of Object.entries(value)) { diff --git a/apps/sim/lib/table/rows/secret-provenance.ts b/apps/sim/lib/table/rows/secret-provenance.ts index b36583d0546..41f08c64ef5 100644 --- a/apps/sim/lib/table/rows/secret-provenance.ts +++ b/apps/sim/lib/table/rows/secret-provenance.ts @@ -6,6 +6,7 @@ import { userTableRows, } from '@sim/db/schema' import { createLogger } from '@sim/logger' +import { isRecordLike } from '@sim/utils/object' import { compareStrings } from '@sim/utils/string' import { and, asc, eq, gt, inArray, type SQL, sql } from 'drizzle-orm' import { SecretProvenanceBudget } from '@/lib/execution/provenance-budget' @@ -127,7 +128,7 @@ function serializedBytes(value: unknown): number { } function isStoredEntry(value: unknown): value is StoredTableRowSecretProvenanceEntry { - if (!value || typeof value !== 'object' || Array.isArray(value)) return false + if (!isRecordLike(value)) return false const record = value as Record if ( Reflect.ownKeys(record).some((key) => typeof key !== 'string' || !STORED_ENTRY_KEYS.has(key)) diff --git a/apps/sim/lib/table/workflow-group-cancellation.ts b/apps/sim/lib/table/workflow-group-cancellation.ts index bf7f638e51b..2b8563cb47e 100644 --- a/apps/sim/lib/table/workflow-group-cancellation.ts +++ b/apps/sim/lib/table/workflow-group-cancellation.ts @@ -2,6 +2,7 @@ import { db } from '@sim/db' import { tableRowExecutions, userTableDefinitions, workflowExecutionLogs } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' +import { isRecordLike } from '@sim/utils/object' import { and, eq, inArray } from 'drizzle-orm' import { cancelledExecutionLogFields } from '@/lib/logs/execution/cancellation' import { appendTableEvent } from '@/lib/table/events' @@ -68,10 +69,10 @@ interface WorkflowGroupExecutionTarget { } function getExecutionCorrelationSource(value: unknown): string | null { - if (!value || typeof value !== 'object' || Array.isArray(value)) return null + if (!isRecordLike(value)) return null const executionData = value as Record const correlation = executionData.correlation - if (!correlation || typeof correlation !== 'object' || Array.isArray(correlation)) return null + if (!isRecordLike(correlation)) return null const source = (correlation as Record).source return typeof source === 'string' ? source : null } diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-live-doc-outbox.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-live-doc-outbox.ts index b7dfe968f71..5043a58f974 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-live-doc-outbox.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-live-doc-outbox.ts @@ -1,5 +1,6 @@ import { db } from '@sim/db' import { workspaceFiles } from '@sim/db/schema' +import { isRecordLike } from '@sim/utils/object' import { PASTE_LIMITS } from '@sim/utils/paste' import { and, eq, isNull } from 'drizzle-orm' import { @@ -22,7 +23,7 @@ interface WorkspaceFileLiveDocPayload { } function parsePayload(payload: unknown): WorkspaceFileLiveDocPayload { - if (!payload || typeof payload !== 'object' || Array.isArray(payload)) { + if (!isRecordLike(payload)) { throw new Error('Workspace file live-document outbox payload must be an object') } const candidate = payload as Partial diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-storage-cleanup-outbox.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-storage-cleanup-outbox.ts index 1cc6cbe5e9b..e97dd72278e 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-storage-cleanup-outbox.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-storage-cleanup-outbox.ts @@ -2,6 +2,7 @@ import type { db } from '@sim/db' import { createLogger } from '@sim/logger' import { describeError } from '@sim/utils/errors' import { chunkArray } from '@sim/utils/helpers' +import { isRecordLike } from '@sim/utils/object' import { enqueueOutboxEvents, MAX_BULK_ENQUEUE_EVENTS, @@ -20,7 +21,7 @@ interface WorkspaceFileStorageCleanupPayload { } function parsePayload(payload: unknown): WorkspaceFileStorageCleanupPayload { - if (!payload || typeof payload !== 'object' || Array.isArray(payload)) { + if (!isRecordLike(payload)) { throw new Error('Workspace file storage cleanup outbox payload must be an object') } const key = (payload as Record).key From 934f384ddefab65263d231e63f2b6c1a870c7806 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:23:30 -0700 Subject: [PATCH 23/30] chore(cleanup): remove dead uploads, tokenization, table, knowledge, and search exports --- apps/sim/app/api/files/delete/route.test.ts | 1 - apps/sim/lib/file-parsers/yaml-parser.ts | 8 -- apps/sim/lib/internal/file/parser.test.ts | 18 +--- .../application/search-integrations.test.ts | 8 -- apps/sim/lib/knowledge/constants.ts | 1 - apps/sim/lib/knowledge/documents/service.ts | 9 -- .../workspace-source-provenance.test.ts | 56 +---------- apps/sim/lib/knowledge/service.test.ts | 27 ------ apps/sim/lib/knowledge/service.ts | 16 ---- apps/sim/lib/sim-search/connectors.ts | 17 +--- apps/sim/lib/table/jobs/service.ts | 18 ---- apps/sim/lib/table/validation.test.ts | 65 ------------- apps/sim/lib/table/validation.ts | 37 -------- apps/sim/lib/table/views/service.test.ts | 49 ---------- apps/sim/lib/table/views/service.ts | 82 ---------------- apps/sim/lib/tokenization/calculators.ts | 40 +------- apps/sim/lib/tokenization/constants.ts | 1 - apps/sim/lib/tokenization/index.ts | 7 +- apps/sim/lib/tokenization/utils.ts | 14 +-- .../workspace/workspace-file-manager.test.ts | 9 -- .../workspace/workspace-file-manager.ts | 80 ---------------- apps/sim/lib/uploads/core/storage-client.ts | 10 -- apps/sim/lib/uploads/core/storage-service.ts | 24 ----- apps/sim/lib/uploads/index.ts | 6 +- apps/sim/lib/uploads/providers/blob/client.ts | 31 ------ .../providers/google-cloud-storage/client.ts | 10 -- apps/sim/lib/uploads/providers/s3/client.ts | 15 --- apps/sim/lib/uploads/server/metadata.test.ts | 50 ---------- apps/sim/lib/uploads/server/metadata.ts | 95 ------------------- apps/sim/lib/uploads/utils/file-utils.test.ts | 21 ---- apps/sim/lib/uploads/utils/file-utils.ts | 18 ---- apps/sim/lib/uploads/utils/validation.ts | 19 ---- .../resolve-workspace-file-reference.ts | 48 ---------- packages/testing/src/mocks/file-utils.mock.ts | 13 --- .../mocks/knowledge-documents-service.mock.ts | 2 - .../src/mocks/knowledge-service.mock.ts | 3 - .../src/mocks/sim-search-connectors.mock.ts | 9 -- .../testing/src/mocks/storage-service.mock.ts | 2 - .../src/mocks/table-jobs-service.mock.ts | 2 - packages/testing/src/mocks/table.mock.ts | 14 --- .../src/mocks/uploads-metadata.mock.ts | 2 - packages/testing/src/mocks/uploads.mock.ts | 5 +- .../src/mocks/workspace-file-manager.mock.ts | 13 +-- .../mocks/workspace-file-reference.mock.ts | 2 - 44 files changed, 14 insertions(+), 963 deletions(-) diff --git a/apps/sim/app/api/files/delete/route.test.ts b/apps/sim/app/api/files/delete/route.test.ts index b7596b12b51..6c331f9cd06 100644 --- a/apps/sim/app/api/files/delete/route.test.ts +++ b/apps/sim/app/api/files/delete/route.test.ts @@ -54,7 +54,6 @@ describe('File Delete API Route', () => { filesAuthorizationMockFns.mockVerifyFileAccess.mockResolvedValue(true) storageServiceMockFns.mockDeleteFile.mockResolvedValue(undefined) storageServiceMockFns.mockHasCloudStorage.mockReturnValue(true) - uploadsMockFns.mockGetStorageProvider.mockReturnValue('s3') uploadsMockFns.mockIsUsingCloudStorage.mockReturnValue(true) }) diff --git a/apps/sim/lib/file-parsers/yaml-parser.ts b/apps/sim/lib/file-parsers/yaml-parser.ts index 97b89d166a9..c4f393b41da 100644 --- a/apps/sim/lib/file-parsers/yaml-parser.ts +++ b/apps/sim/lib/file-parsers/yaml-parser.ts @@ -29,14 +29,6 @@ export class YamlComplexityError extends FileParserError { } } -/** - * Type guard for {@link YamlComplexityError}. Callers use this to fail closed on - * a complexity-limit rejection instead of falling back to a generic parse. - */ -export function isYamlComplexityError(error: unknown): error is YamlComplexityError { - return error instanceof YamlComplexityError -} - /** * Validate that a parsed YAML value stays within the file parser's expansion * limits, returning the document depth. diff --git a/apps/sim/lib/internal/file/parser.test.ts b/apps/sim/lib/internal/file/parser.test.ts index 8ad033d6d27..67f78334c46 100644 --- a/apps/sim/lib/internal/file/parser.test.ts +++ b/apps/sim/lib/internal/file/parser.test.ts @@ -177,7 +177,7 @@ vi.mock('fs/promises', () => ({ writeFile: mockFsWriteFile, })) -const { mockGetStorageProvider, mockIsUsingCloudStorage } = uploadsMockFns +const { mockIsUsingCloudStorage } = uploadsMockFns const { mockGetBoundWorkspaceFileSecretProvenance } = workspaceFileSecretProvenanceMockFns import { fileParseBodySchema } from '@/lib/api/contracts/storage-transfer' @@ -237,14 +237,8 @@ async function POST(request: NextRequest): Promise { }) } -function setupFileApiMocks( - options: { - authenticated?: boolean - storageProvider?: 's3' | 'blob' | 'local' - cloudEnabled?: boolean - } = {} -) { - const { authenticated = true, storageProvider = 's3', cloudEnabled = true } = options +function setupFileApiMocks(options: { authenticated?: boolean; cloudEnabled?: boolean } = {}) { + const { authenticated = true, cloudEnabled = true } = options if (authenticated) { authMockFns.mockGetSession.mockResolvedValue({ @@ -272,7 +266,6 @@ function setupFileApiMocks( error: authenticated ? undefined : 'Unauthorized', }) - mockGetStorageProvider.mockReturnValue(storageProvider) mockIsUsingCloudStorage.mockReturnValue(cloudEnabled) } @@ -475,7 +468,6 @@ describe('file parser operation', () => { it('should keep known binary extensions as binary even when the bytes are valid UTF-8', async () => { setupFileApiMocks({ cloudEnabled: true, - storageProvider: 's3', authenticated: true, }) mockIsSupportedFileType.mockReturnValue(false) @@ -496,7 +488,6 @@ describe('file parser operation', () => { it('should parse unknown extensions as text when the bytes look like UTF-8 text', async () => { setupFileApiMocks({ cloudEnabled: true, - storageProvider: 's3', authenticated: true, }) mockIsSupportedFileType.mockReturnValue(false) @@ -521,7 +512,6 @@ describe('file parser operation', () => { it('reports degraded parser output as a failure instead of returning it as content', async () => { setupFileApiMocks({ cloudEnabled: false, - storageProvider: 'local', authenticated: true, }) mockParseBuffer.mockResolvedValue({ @@ -546,7 +536,6 @@ describe('file parser operation', () => { it('should reject parser complexity limits instead of returning raw text', async () => { setupFileApiMocks({ cloudEnabled: true, - storageProvider: 's3', authenticated: true, }) storageServiceMockFns.mockDownloadFile.mockResolvedValue(Buffer.from('{"value":true}')) @@ -743,7 +732,6 @@ describe('file parser operation', () => { it('should reject oversized local files before materializing them', async () => { setupFileApiMocks({ cloudEnabled: false, - storageProvider: 'local', authenticated: true, }) mockFsStat.mockResolvedValue({ isFile: () => true, size: 104857601 }) diff --git a/apps/sim/lib/knowledge/application/search-integrations.test.ts b/apps/sim/lib/knowledge/application/search-integrations.test.ts index e353084631e..b67f2c9c8ed 100644 --- a/apps/sim/lib/knowledge/application/search-integrations.test.ts +++ b/apps/sim/lib/knowledge/application/search-integrations.test.ts @@ -43,14 +43,6 @@ vi.mock('@/lib/sim-search/connectors', () => ({ ['github', { name: 'GitHub' }], ['jira', { name: 'Jira' }], ], - searchMemberAccountProvider: (type: string) => - type === 'google_drive' - ? 'google-drive' - : ['gmail', 'jira', 'github'].includes(type) - ? type === 'github' - ? 'github-repositories' - : type - : null, })) import { CredentialGroupProviderConfigurationError } from '@/lib/credential-groups/provider-adapter' diff --git a/apps/sim/lib/knowledge/constants.ts b/apps/sim/lib/knowledge/constants.ts index 3f9093870d8..08444d02a23 100644 --- a/apps/sim/lib/knowledge/constants.ts +++ b/apps/sim/lib/knowledge/constants.ts @@ -32,7 +32,6 @@ export const MAX_KNOWLEDGE_CONNECTOR_DOCUMENT_SEARCH_LENGTH = 200 /** Bound viewer-specific source resolution and document counts to a single page. */ export const SEARCH_SOURCE_PAGE_SIZE = 25 export const SEARCH_SOURCE_CANDIDATE_PAGE_SIZE = 100 -export const MAX_SEARCH_SOURCE_PROVIDER_TYPES = 100 /** * Chunking a knowledge base gets when its creator names no configuration. diff --git a/apps/sim/lib/knowledge/documents/service.ts b/apps/sim/lib/knowledge/documents/service.ts index c767cd0b3ca..1db5a4b1a42 100644 --- a/apps/sim/lib/knowledge/documents/service.ts +++ b/apps/sim/lib/knowledge/documents/service.ts @@ -145,7 +145,6 @@ import { enqueueKnowledgeStorageCleanup, getKnowledgeBaseStorageKey, isKnowledgeBaseOwnedStorageKey, - type KnowledgeStorageCleanupDocument, } from '@/lib/knowledge/documents/storage-cleanup' import { claimKnowledgeUploadForAttachment } from '@/lib/knowledge/documents/storage-upload' import { @@ -4067,14 +4066,6 @@ export async function updateDocument( } } -/** Persists standalone cleanup intents; document mutations supply their own transaction. */ -export async function deleteDocumentStorageFiles( - documentsToDelete: readonly KnowledgeStorageCleanupDocument[], - requestId: string -): Promise { - await enqueueKnowledgeStorageCleanup(db, documentsToDelete, requestId) -} - async function excludeConnectorDocuments( documentIds: string[], requestId: string diff --git a/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts b/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts index c55f2d7fb7e..b8721ba01dd 100644 --- a/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts +++ b/apps/sim/lib/knowledge/documents/workspace-source-provenance.test.ts @@ -1,6 +1,6 @@ import { dbChainMockFns, resetDbChainMock } from '@sim/testing' import { billingStorageMock, billingStorageMockFns } from '@sim/testing/mocks/billing-storage.mock' -import { storageServiceMock, storageServiceMockFns } from '@sim/testing/mocks/storage-service.mock' +import { storageServiceMock } from '@sim/testing/mocks/storage-service.mock' import { uploadsMetadataMock, uploadsMetadataMockFns, @@ -22,11 +22,7 @@ vi.mock('@/lib/uploads/core/storage-service', () => storageServiceMock) vi.mock('@/lib/uploads/server/metadata', () => uploadsMetadataMock) -import { - createDocumentRecords, - createSingleDocument, - deleteDocumentStorageFiles, -} from '@/lib/knowledge/documents/service' +import { createDocumentRecords, createSingleDocument } from '@/lib/knowledge/documents/service' const mockCheckStorageQuotaForBillingContext = billingStorageMockFns.mockCheckStorageQuotaForBillingContext @@ -36,8 +32,6 @@ const mockMaybeNotifyStorageLimitForBillingContext = billingStorageMockFns.mockMaybeNotifyStorageLimitForBillingContext const mockResolveStorageBillingContext = billingStorageMockFns.mockResolveStorageBillingContext -const mockDeleteFile = storageServiceMockFns.mockDeleteFile -const mockDeleteFileMetadataByIdentity = uploadsMetadataMockFns.mockDeleteFileMetadataByIdentity const mockGetFileMetadataByKeys = uploadsMetadataMockFns.mockGetFileMetadataByKeys const mockGetBoundWorkspaceFileSecretProvenanceByMetadata = workspaceFileSecretProvenanceMockFns.mockGetBoundWorkspaceFileSecretProvenanceByMetadata @@ -228,50 +222,4 @@ describe('knowledge workspace source provenance', () => { }) } }) - - it('never deletes a referenced workspace source as knowledge-base storage', async () => { - await deleteDocumentStorageFiles( - [{ id: 'document-1', fileUrl: SOURCE_URL, workspaceId: WORKSPACE_ID }], - 'request-1' - ) - - expect(mockGetFileMetadataByKeys).not.toHaveBeenCalled() - expect(mockDeleteFile).not.toHaveBeenCalled() - expect(mockDeleteFileMetadataByIdentity).not.toHaveBeenCalled() - }) - - it.each(['org-1', 'org-2', null])( - 'only queues an organization cache for its exact owner: %s', - async (organizationId) => { - const storageKey = 'kb/org-source.pdf' - mockGetFileMetadataByKeys.mockResolvedValue([ - { - ...SOURCE_BINDING, - key: storageKey, - context: 'knowledge-base', - workspaceId: null, - organizationId: 'org-1', - }, - ]) - const cleanup = deleteDocumentStorageFiles( - [ - { - id: 'org-doc', - fileUrl: `/api/files/serve/${encodeURIComponent(storageKey)}`, - workspaceId: null, - organizationId, - }, - ], - 'request-1' - ) - if (organizationId === 'org-1') { - await expect(cleanup).resolves.toBeUndefined() - expect(dbChainMockFns.values).toHaveBeenCalledOnce() - } else { - await expect(cleanup).rejects.toThrow() - expect(dbChainMockFns.values).not.toHaveBeenCalled() - } - expect(mockDeleteFile).not.toHaveBeenCalled() - } - ) }) diff --git a/apps/sim/lib/knowledge/service.test.ts b/apps/sim/lib/knowledge/service.test.ts index 8e02cb151d2..ee1d4308f64 100644 --- a/apps/sim/lib/knowledge/service.test.ts +++ b/apps/sim/lib/knowledge/service.test.ts @@ -1,6 +1,5 @@ import { dbChainMockFns, - hasMockCondition, permissionsMock, permissionsMockFns, queueTableRows, @@ -18,7 +17,6 @@ vi.mock('@/lib/billing/storage', () => billingStorageMock) import { attachKnowledgeBaseConnectors, - findActiveKnowledgeBasesByExactName, getWorkspaceKnowledgeBases, KnowledgeBasePermissionError, updateKnowledgeBase, @@ -59,31 +57,6 @@ describe('getWorkspaceKnowledgeBases — paging', () => { }) }) -/** - * A VFS path names one knowledge base exactly. Resolving it by reading every base whose name - * merely CONTAINS the term, then filtering in JS, makes a single-row lookup scale with the - * workspace — the sibling `findActiveTablesByExactName` is the shape to match. - */ -describe('findActiveKnowledgeBasesByExactName', () => { - beforeEach(() => { - resetDbChainMock() - }) - - it('matches the name exactly and reads at most two rows', async () => { - await findActiveKnowledgeBasesByExactName('ws-1', 'Docs') - - const [condition] = dbChainMockFns.where.mock.calls[0] ?? [] - expect( - hasMockCondition( - condition, - (node) => - node.type === 'eq' && node.left === schemaMock.knowledgeBase.name && node.right === 'Docs' - ) - ).toBe(true) - expect(dbChainMockFns.limit).toHaveBeenCalledWith(2) - }) -}) - /** * These tests guard the workspace mass-assignment fix: * a user with write/admin on the *source* workspace must not be able to move a diff --git a/apps/sim/lib/knowledge/service.ts b/apps/sim/lib/knowledge/service.ts index 2da15a1863d..f6e43d4bc4f 100644 --- a/apps/sim/lib/knowledge/service.ts +++ b/apps/sim/lib/knowledge/service.ts @@ -424,22 +424,6 @@ export async function getWorkspaceKnowledgeBases( } } -/** Loads at most two active exact-name matches so a caller can fail on corrupt ambiguity. */ -export async function findActiveKnowledgeBasesByExactName( - workspaceId: string, - name: string -): Promise { - return readKnowledgeBaseRows( - and( - eq(knowledgeBase.workspaceId, workspaceId), - eq(knowledgeBase.name, name), - isNull(knowledgeBase.deletedAt) - ), - listOrderBy(keysetColumns(KNOWLEDGE_BASE_SORTS.createdAt), 'asc'), - 2 - ) -} - /** * Create a new knowledge base */ diff --git a/apps/sim/lib/sim-search/connectors.ts b/apps/sim/lib/sim-search/connectors.ts index 0e7da56295a..8bff18561db 100644 --- a/apps/sim/lib/sim-search/connectors.ts +++ b/apps/sim/lib/sim-search/connectors.ts @@ -1,9 +1,6 @@ import type { ComponentType } from 'react' import type { IntegrationAvailabilityResponse } from '@/lib/api/contracts/common' -import { - findCredentialGroupProviderFromProviderId, - isCredentialGroupStandardOAuthProvider, -} from '@/lib/credential-groups/providers' +import { findCredentialGroupProviderFromProviderId } from '@/lib/credential-groups/providers' import { getIntegrationsForCredentialProvider } from '@/lib/integrations/credential-display' import { getCanonicalScopesForProvider, @@ -83,13 +80,6 @@ export const SEARCH_CONNECTORS: readonly SearchConnector[] = Object.entries(CONN }) .sort((a, b) => a.meta.name.localeCompare(b.meta.name)) -/** Standard member sign-in configured by an explicit Search source addition. */ -export function searchMemberAccountProvider(connectorType: string) { - const connector = SEARCH_CONNECTORS.find((candidate) => candidate.type === connectorType) - const provider = connector && findCredentialGroupProviderFromProviderId(connector.providerId) - return provider && isCredentialGroupStandardOAuthProvider(provider) ? provider : null -} - /** * Every source an admin may set up for Sim Search, alphabetical by name: the * connectors that either mirror their source's permissions or connect per person. @@ -125,11 +115,6 @@ export function personalSetupFields(meta: ConnectorMeta): ConnectorConfigField[] ) } -/** Personal sources use defaults even when they also support central indexing. Slack needs a custom app first. */ -export function canConnectWithDefaults(meta: ConnectorMeta): boolean { - return canConnectPersonally(meta) && meta.id !== 'slack' && personalSetupFields(meta).length === 0 -} - /** The name a connector shows, from its registry entry. */ export function connectorDisplayName(connectorType: string): string { return CONNECTOR_META_REGISTRY[connectorType]?.name ?? connectorType diff --git a/apps/sim/lib/table/jobs/service.ts b/apps/sim/lib/table/jobs/service.ts index d33ffb755ef..0716f6a610c 100644 --- a/apps/sim/lib/table/jobs/service.ts +++ b/apps/sim/lib/table/jobs/service.ts @@ -462,24 +462,6 @@ export async function listWorkspaceExportJobs(workspaceId: string): Promise { - const [job] = await db - .select({ - id: tableJobs.id, - type: tableJobs.type, - status: tableJobs.status, - payload: tableJobs.payload, - }) - .from(tableJobs) - .where(and(eq(tableJobs.id, jobId), eq(tableJobs.tableId, tableId))) - .limit(1) - return job ?? null -} - /** Stamps an export result only while the canonical workspace-scoped job is active. */ export async function setJobResultKeyInWorkspace( tableId: string, diff --git a/apps/sim/lib/table/validation.test.ts b/apps/sim/lib/table/validation.test.ts index 7b28a9163d7..7f8e4ed882f 100644 --- a/apps/sim/lib/table/validation.test.ts +++ b/apps/sim/lib/table/validation.test.ts @@ -9,7 +9,6 @@ import { validateRowSize, validateTableName, validateTableSchema, - validateUniqueConstraints, } from '@/lib/table/validation' const selectColumn: ColumnDefinition = { @@ -556,68 +555,4 @@ describe('Validation', () => { }) }) }) - - describe('validateUniqueConstraints', () => { - const schema: TableSchema = { - columns: [ - { name: 'id', type: 'string', unique: true }, - { name: 'email', type: 'string', unique: true }, - { name: 'name', type: 'string' }, - ], - } - - const existingRows = [ - { id: 'row1', data: { id: 'abc123', email: 'john@example.com', name: 'John' } }, - { id: 'row2', data: { id: 'def456', email: 'jane@example.com', name: 'Jane' } }, - ] - - it('should reject duplicate unique value', () => { - const data = { id: 'abc123', email: 'new@example.com', name: 'New User' } - const result = validateUniqueConstraints(data, schema, existingRows) - expect(result.valid).toBe(false) - expect(result.errors[0]).toContain('must be unique') - expect(result.errors[0]).toContain('abc123') - }) - - it('should be case-sensitive for string comparisons', () => { - // U333 vs u333: differing case is a DISTINCT value (matches the DB - // containment leaf). This is the v2 contract that fixes the upsert wedge. - const data = { id: 'ABC123', email: 'new@example.com', name: 'New User' } - const result = validateUniqueConstraints(data, schema, existingRows) - expect(result.valid).toBe(true) - }) - - it('compares expiration uniqueness by instant while retaining microseconds', () => { - const expirationSchema: TableSchema = { - columns: [{ name: 'expires', type: 'ttl', unique: true }], - } - const rows = [{ id: 'existing', data: { expires: '2026-09-07T07:30:00.000001-07:00' } }] - for (const value of [ - '2026-09-07T14:30:00.000001Z', - '2026-09-07T20:15:00.000001+05:45', - '2026-09-07T14:30:00.000001-00:00', - ]) { - expect(validateUniqueConstraints({ expires: value }, expirationSchema, rows).valid).toBe( - false - ) - expect( - validateUniqueConstraints({ expires: value }, expirationSchema, rows, 'existing').valid - ).toBe(true) - } - expect( - validateUniqueConstraints( - { expires: '2026-09-07T14:30:00.000002-00:00' }, - expirationSchema, - rows - ).valid - ).toBe(true) - }) - - it('should report multiple violations', () => { - const data = { id: 'abc123', email: 'john@example.com', name: 'New User' } - const result = validateUniqueConstraints(data, schema, existingRows) - expect(result.valid).toBe(false) - expect(result.errors).toHaveLength(2) - }) - }) }) diff --git a/apps/sim/lib/table/validation.ts b/apps/sim/lib/table/validation.ts index 29bd9157078..ac8b5a3bd5d 100644 --- a/apps/sim/lib/table/validation.ts +++ b/apps/sim/lib/table/validation.ts @@ -432,43 +432,6 @@ export function uniqueValueKey(value: JsonValue, column: ColumnDefinition): stri return canonicalJson(columnValueForEquality(value, column)) } -/** Validates unique constraints against existing rows (in-memory version for batch validation within a batch). */ -export function validateUniqueConstraints( - data: RowData, - schema: TableSchema, - existingRows: { id: string; data: RowData; position?: number }[], - excludeRowId?: string -): ValidationResult { - const errors: string[] = [] - const uniqueColumns = getUniqueColumns(schema) - - for (const column of uniqueColumns) { - const key = getColumnId(column) - const value = cellOf(data, key) - if (value === null || value === undefined) continue - - const duplicate = existingRows.find((row) => { - if (excludeRowId && row.id === excludeRowId) return false - // Case-sensitive, matching the DB unique-check leaf (`fieldPredicate` eq). - const existing = cellOf(row.data, key) - return ( - existing !== undefined && - columnValueForEquality(value, column) === columnValueForEquality(existing, column) - ) - }) - - if (duplicate) { - const rowLabel = - typeof duplicate.position === 'number' ? `row ${duplicate.position + 1}` : duplicate.id - errors.push( - `Column "${column.name}" must be unique. Value "${value}" already exists in ${rowLabel}` - ) - } - } - - return { valid: errors.length === 0, errors } -} - /** * Checks unique constraints using targeted database queries. * Only queries for specific conflicting values instead of loading all rows. diff --git a/apps/sim/lib/table/views/service.test.ts b/apps/sim/lib/table/views/service.test.ts index 6ec689266be..c0c666ac46c 100644 --- a/apps/sim/lib/table/views/service.test.ts +++ b/apps/sim/lib/table/views/service.test.ts @@ -169,55 +169,6 @@ describe('getTableView', () => { }) }) -describe('view config name/id translation', () => { - const columns = [ - { id: 'col_a', name: 'status', type: 'string' }, - { id: 'col_b', name: 'due', type: 'date' }, - ] as never[] - - it('round-trips a config between id and name domains', async () => { - const { viewConfigIdsToNames, viewConfigNamesToIds } = await import('@/lib/table/views/service') - const stored = { - filter: { - any: [ - { field: 'col_a', op: 'eq', value: 'Open' }, - { all: [{ field: 'col_b', op: 'isNotNull' }] }, - ], - }, - sort: [{ field: 'col_b', direction: 'desc' }], - hiddenColumns: ['col_a'], - } as never - const named = viewConfigIdsToNames(stored, columns as never) - expect(named.filter).toEqual({ - any: [ - { field: 'status', op: 'eq', value: 'Open' }, - { all: [{ field: 'due', op: 'isNotNull' }] }, - ], - }) - expect(named.sort).toEqual([{ field: 'due', direction: 'desc' }]) - expect(named.hiddenColumns).toEqual(['status']) - expect(viewConfigNamesToIds(named, columns as never)).toEqual(stored) - }) - - it('passes stale ids through on read but rejects unknown names on write', async () => { - const { viewConfigIdsToNames, viewConfigNamesToIds } = await import('@/lib/table/views/service') - const withStale = { filter: { all: [{ field: 'col_gone', op: 'isNull' }] } } as never - expect( - ( - viewConfigIdsToNames(withStale, columns as never).filter as never as { - all: { field: string }[] - } - ).all[0].field - ).toBe('col_gone') - expect(() => - viewConfigNamesToIds( - { filter: { all: [{ field: 'nope', op: 'isNull' }] } } as never, - columns as never - ) - ).toThrow(/Unknown column/) - }) -}) - describe('saved-view ceiling', () => { beforeEach(() => { resetDbChainMock() diff --git a/apps/sim/lib/table/views/service.ts b/apps/sim/lib/table/views/service.ts index 550cb617713..ae05b60f578 100644 --- a/apps/sim/lib/table/views/service.ts +++ b/apps/sim/lib/table/views/service.ts @@ -695,85 +695,3 @@ export async function deleteTableView( } return deleted } - -/** - * All of a workspace's views in one query, keyed by tableId — the snapshot - * materializer's shape (per-table listTableViews would be N queries). Configs - * are returned RAW (id-domain, unpruned); callers translate/prune with each - * table's own columns. - */ -export async function listTableViewsByWorkspace( - workspaceId: string -): Promise>> { - const rows = await db - .select() - .from(tableViews) - .where(eq(tableViews.workspaceId, workspaceId)) - .orderBy(asc(tableViews.createdAt), asc(tableViews.id)) - const byTable = new Map>() - for (const row of rows) { - const list = byTable.get(row.tableId) ?? [] - list.push(row) - byTable.set(row.tableId, list) - } - return byTable -} - -function mapPredicateFields( - node: PredicateNode, - mapField: (field: string) => string -): PredicateNode { - if ('all' in node) return { all: node.all.map((child) => mapPredicateFields(child, mapField)) } - if ('any' in node) return { any: node.any.map((child) => mapPredicateFields(child, mapField)) } - const leaf = node as Predicate - return { ...leaf, field: mapField(leaf.field) } -} - -/** - * Stored (id-domain) view config → the column-NAME domain agents speak. - * Unknown ids pass through unchanged, mirroring pruneViewConfig's philosophy - * for filters: surfacing a stale reference beats silently widening the view. - */ -export function viewConfigIdsToNames( - config: TableViewConfig, - columns: ColumnDefinition[] -): TableViewConfig { - const nameById = new Map(columns.map((col) => [getColumnId(col), col.name])) - const toName = (field: string) => nameById.get(field) ?? field - const out: TableViewConfig = { ...config } - if (config.filter) out.filter = mapPredicateFields(config.filter, toName) as typeof config.filter - if (config.sort) out.sort = config.sort.map((s) => ({ ...s, field: toName(s.field) })) - if (config.hiddenColumns) out.hiddenColumns = config.hiddenColumns.map(toName) - return out -} - -/** - * Agent-supplied (name-domain) view config → the id-domain stored shape. - * Unknown column names are an error — a saved view with a dangling reference - * is exactly the artifact this translation exists to prevent. - */ -export function viewConfigNamesToIds( - config: TableViewConfig, - columns: ColumnDefinition[] -): TableViewConfig { - const idByName = new Map(columns.map((col) => [col.name, getColumnId(col)])) - const unknown = new Set() - const toId = (field: string) => { - const id = idByName.get(field) - if (!id) { - unknown.add(field) - return field - } - return id - } - const out: TableViewConfig = { ...config } - if (config.filter) out.filter = mapPredicateFields(config.filter, toId) as typeof config.filter - if (config.sort) out.sort = config.sort.map((s) => ({ ...s, field: toId(s.field) })) - if (config.hiddenColumns) out.hiddenColumns = config.hiddenColumns.map(toId) - if (unknown.size > 0) { - throw new TableViewValidationError( - `Unknown column(s): ${[...unknown].join(', ')}. Use exact column names from get_schema.` - ) - } - return out -} diff --git a/apps/sim/lib/tokenization/calculators.ts b/apps/sim/lib/tokenization/calculators.ts index 3a1a34a2657..ed4edf6d747 100644 --- a/apps/sim/lib/tokenization/calculators.ts +++ b/apps/sim/lib/tokenization/calculators.ts @@ -10,12 +10,7 @@ import { estimateOutputTokens, estimateTokenCount, } from '@/lib/tokenization/estimators' -import type { - CostBreakdown, - StreamingCostResult, - TokenizationInput, - TokenUsage, -} from '@/lib/tokenization/types' +import type { CostBreakdown, StreamingCostResult, TokenUsage } from '@/lib/tokenization/types' import { getProviderForTokenization, logTokenizationDetails, @@ -110,36 +105,3 @@ export function calculateStreamingCost( ) } } - -/** - * Calculates cost for tokenization input object - */ -export function calculateTokenizationCost(input: TokenizationInput): StreamingCostResult { - return calculateStreamingCost( - input.model, - input.inputText, - input.outputText, - input.systemPrompt, - input.context, - input.messages - ) -} - -/** - * Creates a streaming cost result from existing provider response data - */ -export function createCostResultFromProviderData( - model: string, - providerTokens: TokenUsage, - providerCost: CostBreakdown -): StreamingCostResult { - const providerId = getProviderForTokenization(model) - - return { - tokens: providerTokens, - cost: providerCost, - model, - provider: providerId, - method: 'provider_response', - } -} diff --git a/apps/sim/lib/tokenization/constants.ts b/apps/sim/lib/tokenization/constants.ts index 3e78ebddc01..d12c5b094f7 100644 --- a/apps/sim/lib/tokenization/constants.ts +++ b/apps/sim/lib/tokenization/constants.ts @@ -108,4 +108,3 @@ export const TOKENIZATION_CONFIG = { export const LLM_BLOCK_TYPES = ['agent', 'router', 'evaluator'] as const export const MIN_TEXT_LENGTH_FOR_ESTIMATION = 1 -export const MAX_PREVIEW_LENGTH = 100 diff --git a/apps/sim/lib/tokenization/index.ts b/apps/sim/lib/tokenization/index.ts index 32750225ad5..78983d94638 100644 --- a/apps/sim/lib/tokenization/index.ts +++ b/apps/sim/lib/tokenization/index.ts @@ -1,8 +1,4 @@ -export { - calculateStreamingCost, - calculateTokenizationCost, - createCostResultFromProviderData, -} from '@/lib/tokenization/calculators' +export { calculateStreamingCost } from '@/lib/tokenization/calculators' export { LLM_BLOCK_TYPES, TOKENIZATION_CONFIG } from '@/lib/tokenization/constants' export { createTokenizationError, TokenizationError } from '@/lib/tokenization/errors' /** @@ -18,7 +14,6 @@ export { } from '@/lib/tokenization/estimators' export { processStreamingBlockLog, processStreamingBlockLogs } from '@/lib/tokenization/streaming' export { - createTextPreview, extractTextContent, formatTokenCount, getProviderConfig, diff --git a/apps/sim/lib/tokenization/utils.ts b/apps/sim/lib/tokenization/utils.ts index 3a4c8cadee8..7430aad8c2f 100644 --- a/apps/sim/lib/tokenization/utils.ts +++ b/apps/sim/lib/tokenization/utils.ts @@ -4,12 +4,7 @@ import { createLogger } from '@sim/logger' import { toError } from '@sim/utils/errors' -import { truncate } from '@sim/utils/string' -import { - LLM_BLOCK_TYPES, - MAX_PREVIEW_LENGTH, - TOKENIZATION_CONFIG, -} from '@/lib/tokenization/constants' +import { LLM_BLOCK_TYPES, TOKENIZATION_CONFIG } from '@/lib/tokenization/constants' import { createTokenizationError } from '@/lib/tokenization/errors' import type { ProviderTokenizationConfig, TokenUsage } from '@/lib/tokenization/types' import type { BlockTokens } from '@/executor/types' @@ -100,13 +95,6 @@ export function extractTextContent(input: unknown): string { return String(input || '') } -/** - * Creates a preview of text for logging (truncated) - */ -export function createTextPreview(text: string): string { - return truncate(text, MAX_PREVIEW_LENGTH) -} - /** * Validates tokenization input */ diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.test.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.test.ts index d2cf049632a..aa66a5b9b95 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.test.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.test.ts @@ -3,7 +3,6 @@ import { LOCAL_UPLOAD_METADATA_SUFFIX } from '@/lib/uploads/core/storage-key' import { findWorkspaceFileRecord, generateWorkspaceFileKey, - normalizeWorkspaceFileReference, type WorkspaceFileRecord, } from './workspace-file-manager' @@ -27,14 +26,6 @@ function makeFileRecord(): WorkspaceFileRecord { } describe('workspace file reference normalization', () => { - it('normalizes canonical VFS paths to their sanitized display path', () => { - expect(normalizeWorkspaceFileReference('files/Reports/q1.csv/content')).toBe('Reports/q1.csv') - expect(normalizeWorkspaceFileReference('files/Reports/q1.csv/meta.json')).toBe('Reports/q1.csv') - expect(normalizeWorkspaceFileReference('recently-deleted/files/data.csv/content')).toBe( - 'data.csv' - ) - }) - it('still resolves a raw file id passed directly', () => { const files = [makeFileRecord()] diff --git a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts index cae7057d2ed..71af2ae2c9e 100644 --- a/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts +++ b/apps/sim/lib/uploads/contexts/workspace/workspace-file-manager.ts @@ -1519,15 +1519,6 @@ export async function queryWorkspaceFiles( return { files, nextKeys: hasMore && last ? encodeKeyset(keys, last) : null } } -/** - * Normalize a workspace file reference to either a display name or canonical file ID. - * Supports raw IDs, `files/{name}`, `files/{name}/content`, and `files/{name}/meta.json`. - * Files are addressed by their sanitized canonical path; id-based VFS paths are not supported. - */ -export function normalizeWorkspaceFileReference(fileReference: string): string { - return normalizeWorkspaceFileReferenceSegments(fileReference).join('/') -} - function normalizeWorkspaceFileReferenceSegments(fileReference: string): string[] { const trimmed = fileReference.trim().replace(/^\/+/, '') const withoutDeletedPrefix = trimmed.startsWith('recently-deleted/') @@ -2365,77 +2356,6 @@ export async function renameWorkspaceFile( } } -/** - * Move and/or rename a workspace file in one atomic row update. Either side - * may be a no-op (same folder = pure rename, same name = pure move); when - * both are unchanged the record is returned untouched. Conflicts at the - * destination throw {@link FileConflictError}. The `renamed`/`moved` flags - * report what actually changed, computed from the same read the update uses. - */ -export async function moveRenameWorkspaceFile(params: { - workspaceId: string - fileId: string - targetFolderId: string | null - newName: string -}): Promise<{ file: WorkspaceFileRecord; renamed: boolean; moved: boolean }> { - const normalizedName = normalizeWorkspaceFileItemName(params.newName.trim(), 'File') - - const fileRecord = await getWorkspaceFile(params.workspaceId, params.fileId) - if (!fileRecord) { - throw new OrchestrationError('not_found', 'File not found') - } - - const targetFolderId = await assertWorkspaceFileFolderTarget( - params.workspaceId, - params.targetFolderId - ) - const currentFolderId = fileRecord.folderId ?? null - const renamed = fileRecord.name !== normalizedName - const moved = currentFolderId !== targetFolderId - if (!renamed && !moved) { - return { file: fileRecord, renamed, moved } - } - - const exists = await fileExistsInWorkspace(params.workspaceId, normalizedName, targetFolderId) - if (exists) { - throw new FileConflictError(normalizedName) - } - - let updated: { id: string }[] - try { - updated = await db - .update(workspaceFiles) - .set({ originalName: normalizedName, folderId: targetFolderId, updatedAt: new Date() }) - .where( - and( - eq(workspaceFiles.id, params.fileId), - eq(workspaceFiles.workspaceId, params.workspaceId), - eq(workspaceFiles.context, 'workspace') - ) - ) - .returning({ id: workspaceFiles.id }) - } catch (error: unknown) { - if (getPostgresErrorCode(error) === '23505') { - throw new FileConflictError(normalizedName) - } - throw error - } - - if (updated.length === 0) { - throw new OrchestrationError('not_found', 'File not found or could not be moved') - } - - return { - file: { - ...fileRecord, - name: normalizedName, - folderId: targetFolderId, - }, - renamed, - moved, - } -} - /** * Soft delete a workspace file. */ diff --git a/apps/sim/lib/uploads/core/storage-client.ts b/apps/sim/lib/uploads/core/storage-client.ts index ff5795d8be0..8d00a1559ef 100644 --- a/apps/sim/lib/uploads/core/storage-client.ts +++ b/apps/sim/lib/uploads/core/storage-client.ts @@ -3,16 +3,6 @@ import type { StorageConfig } from '@/lib/uploads/shared/types' export type { StorageConfig } from '@/lib/uploads/shared/types' -/** - * Get the current storage provider name - */ -export function getStorageProvider(): 'blob' | 's3' | 'gcs' | 'local' { - if (USE_BLOB_STORAGE) return 'blob' - if (USE_S3_STORAGE) return 's3' - if (USE_GCS_STORAGE) return 'gcs' - return 'local' -} - /** * Get the serve path prefix (unified across all storage providers) */ diff --git a/apps/sim/lib/uploads/core/storage-service.ts b/apps/sim/lib/uploads/core/storage-service.ts index d40de3572c8..12720253838 100644 --- a/apps/sim/lib/uploads/core/storage-service.ts +++ b/apps/sim/lib/uploads/core/storage-service.ts @@ -848,27 +848,3 @@ export async function generatePresignedDownloadUrl( export function hasCloudStorage(): boolean { return USE_BLOB_STORAGE || USE_S3_STORAGE || USE_GCS_STORAGE } - -/** - * Get S3 bucket and key information for a storage key - * Useful for services that need direct S3 access (e.g., AWS Textract async) - */ -export function getS3InfoForKey( - key: string, - context: StorageContext -): { bucket: string; key: string } { - if (!USE_S3_STORAGE) { - throw new Error('S3 storage is not configured. Cannot retrieve S3 info for key.') - } - - const config = getStorageConfig(context) - - if (!config.bucket) { - throw new Error(`S3 bucket not configured for context: ${context}`) - } - - return { - bucket: config.bucket, - key, - } -} diff --git a/apps/sim/lib/uploads/index.ts b/apps/sim/lib/uploads/index.ts index fbaa9c3fe6b..3c026bae8f0 100644 --- a/apps/sim/lib/uploads/index.ts +++ b/apps/sim/lib/uploads/index.ts @@ -5,9 +5,5 @@ export { } from '@/lib/uploads/config' export * as ChatFiles from '@/lib/uploads/contexts/chat' export * as CopilotFiles from '@/lib/uploads/contexts/copilot' -export { - getFileMetadata, - getServePathPrefix, - getStorageProvider, -} from '@/lib/uploads/core/storage-client' +export { getFileMetadata, getServePathPrefix } from '@/lib/uploads/core/storage-client' export * as StorageService from '@/lib/uploads/core/storage-service' diff --git a/apps/sim/lib/uploads/providers/blob/client.ts b/apps/sim/lib/uploads/providers/blob/client.ts index 7452dbd8b4e..3a42ccbf43d 100644 --- a/apps/sim/lib/uploads/providers/blob/client.ts +++ b/apps/sim/lib/uploads/providers/blob/client.ts @@ -189,37 +189,6 @@ export async function uploadToBlob( } } -/** - * Generate a presigned URL for direct file access - * @param key Blob name - * @param expiresIn Time in seconds until URL expires - * @returns Presigned URL - */ -export async function getPresignedUrl(key: string, expiresIn = 3600) { - const { BlobSASPermissions, generateBlobSASQueryParameters, StorageSharedKeyCredential } = - await import('@azure/storage-blob') - const blobServiceClient = await getBlobServiceClient() - const containerClient = blobServiceClient.getContainerClient(BLOB_CONFIG.containerName) - const blockBlobClient = containerClient.getBlockBlobClient(key) - - const { accountName, accountKey } = getAccountCredentials() - - const sasOptions = { - containerName: BLOB_CONFIG.containerName, - blobName: key, - permissions: BlobSASPermissions.parse('r'), // Read permission - startsOn: new Date(), - expiresOn: new Date(Date.now() + expiresIn * 1000), - } - - const sasToken = generateBlobSASQueryParameters( - sasOptions, - new StorageSharedKeyCredential(accountName, accountKey) - ).toString() - - return `${blockBlobClient.url}?${sasToken}` -} - /** * Generate a presigned URL for direct file access with custom container * @param key Blob name diff --git a/apps/sim/lib/uploads/providers/google-cloud-storage/client.ts b/apps/sim/lib/uploads/providers/google-cloud-storage/client.ts index 7f3c38da153..3a2061d15d1 100644 --- a/apps/sim/lib/uploads/providers/google-cloud-storage/client.ts +++ b/apps/sim/lib/uploads/providers/google-cloud-storage/client.ts @@ -226,16 +226,6 @@ export async function uploadToGcs( } } -/** - * Generate a presigned URL for direct file access - * @param key GCS object key - * @param expiresIn Time in seconds until URL expires - * @returns Presigned URL - */ -export async function getPresignedUrl(key: string, expiresIn = 3600) { - return getPresignedUrlWithConfig(key, { bucket: GCS_CONFIG.bucket }, expiresIn) -} - /** * Generate a presigned URL for direct file access with custom bucket * @param key GCS object key diff --git a/apps/sim/lib/uploads/providers/s3/client.ts b/apps/sim/lib/uploads/providers/s3/client.ts index 599187e57f6..ca5ad061dab 100644 --- a/apps/sim/lib/uploads/providers/s3/client.ts +++ b/apps/sim/lib/uploads/providers/s3/client.ts @@ -145,21 +145,6 @@ export async function uploadToS3( } } -/** - * Generate a presigned URL for direct file access - * @param key S3 object key - * @param expiresIn Time in seconds until URL expires - * @returns Presigned URL - */ -export async function getPresignedUrl(key: string, expiresIn = 3600) { - const command = new GetObjectCommand({ - Bucket: S3_CONFIG.bucket, - Key: key, - }) - - return getSignedUrl(getS3Client(), command, { expiresIn }) -} - /** * Generate a presigned URL for direct file access with custom bucket * @param key S3 object key diff --git a/apps/sim/lib/uploads/server/metadata.test.ts b/apps/sim/lib/uploads/server/metadata.test.ts index 80ed7af358f..0d9960bdaa1 100644 --- a/apps/sim/lib/uploads/server/metadata.test.ts +++ b/apps/sim/lib/uploads/server/metadata.test.ts @@ -12,7 +12,6 @@ import { ActiveFileMetadataKeyConflictError, deleteFileMetadataByIdentity, insertFileMetadata, - insertFileMetadataMany, insertImmutableFileMetadata, recordKnowledgeBaseFileOwnership, resolveStoredFileContext, @@ -183,55 +182,6 @@ describe('insertFileMetadata content versions', () => { }) }) -describe('insertFileMetadataMany active-key idempotence', () => { - beforeEach(() => { - resetDbChainMock() - }) - - const row = { - key: 'knowledge-base/workspace-1/document.pdf', - userId: 'user-1', - workspaceId: 'workspace-1', - folderId: null, - context: 'knowledge-base' as const, - originalName: 'document.pdf', - contentType: 'application/pdf', - size: 12, - } - - it('accepts an exact retry after a concurrent insert', async () => { - dbChainMockFns.returning.mockResolvedValueOnce([]) - queueTableRows(workspaceFiles, [{ id: 'file-1', ...row, sizeBytes: row.size, deletedAt: null }]) - - await expect(insertFileMetadataMany([row])).resolves.toBeUndefined() - }) - - it('rejects a conflicting active row instead of silently adopting it', async () => { - dbChainMockFns.returning.mockResolvedValueOnce([]) - queueTableRows(workspaceFiles, [ - { - id: 'file-1', - ...row, - sizeBytes: row.size, - userId: 'different-user', - deletedAt: null, - }, - ]) - - await expect(insertFileMetadataMany([row])).rejects.toBeInstanceOf( - ActiveFileMetadataKeyConflictError - ) - }) - - it('rejects mismatched same-batch rows before writing either identity', async () => { - await expect( - insertFileMetadataMany([row, { ...row, userId: 'different-user' }]) - ).rejects.toBeInstanceOf(ActiveFileMetadataKeyConflictError) - - expect(dbChainMockFns.insert).not.toHaveBeenCalled() - }) -}) - describe('resolveStoredFileContext', () => { beforeEach(() => { resetDbChainMock() diff --git a/apps/sim/lib/uploads/server/metadata.ts b/apps/sim/lib/uploads/server/metadata.ts index c80b1228d0d..68ead271692 100644 --- a/apps/sim/lib/uploads/server/metadata.ts +++ b/apps/sim/lib/uploads/server/metadata.ts @@ -71,24 +71,6 @@ function isSameFileMetadataInsert( ) } -function isSameFileMetadataRequest( - left: FileMetadataInsertOptions, - right: FileMetadataInsertOptions -): boolean { - return ( - left.key === right.key && - left.userId === right.userId && - (left.workspaceId ?? null) === (right.workspaceId ?? null) && - (left.organizationId ?? null) === (right.organizationId ?? null) && - (left.folderId ?? null) === (right.folderId ?? null) && - left.context === right.context && - left.originalName === right.originalName && - left.contentType === right.contentType && - left.size === right.size && - (left.id ?? null) === (right.id ?? null) - ) -} - async function findActiveFileMetadataByKey( executor: DbOrTx, key: string @@ -286,83 +268,6 @@ export async function insertImmutableFileMetadata( return insertFileMetadataWithExecutor(db, options, true) } -/** - * Bulk-insert file metadata rows in a single statement. - * - * Intended for batch upload flows that create many fresh keys at once (e.g. the - * presigned batch route), replacing a fan-out of individual `insertFileMetadata` - * calls. Uses `ON CONFLICT DO NOTHING` on the active-key unique index, so it is - * safe against a concurrent single insert. Already-present active keys are - * accepted only when every ownership and file-identity field matches; any - * mismatch is rejected. Unlike {@link insertFileMetadata} it does NOT restore - * soft-deleted rows — callers use this only for newly generated keys. - */ -export async function insertFileMetadataMany( - rows: Array & { id?: string }> -): Promise { - if (rows.length === 0) { - return - } - - const uniqueRowsByKey = new Map() - for (const row of rows) { - assertFileMetadataOrganizationOwner(row) - const existing = uniqueRowsByKey.get(row.key) - if (existing && !isSameFileMetadataRequest(existing, row)) { - throw new ActiveFileMetadataKeyConflictError(row.key) - } - uniqueRowsByKey.set(row.key, existing ?? row) - } - const uniqueRows = [...uniqueRowsByKey.values()] - - const inserted = await db - .insert(workspaceFiles) - .values( - uniqueRows.map((row) => ({ - id: row.id || generateId(), - key: row.key, - userId: row.userId, - workspaceId: row.workspaceId || null, - organizationId: row.organizationId || null, - folderId: row.folderId ?? null, - context: row.context, - originalName: row.originalName, - displayName: row.originalName, - contentType: row.contentType, - sizeBytes: row.size, - deletedAt: null, - uploadedAt: new Date(), - })) - ) - .onConflictDoNothing() - .returning() - - const insertedKeys = new Set(inserted.map((record) => record.key)) - const conflictingRows = uniqueRows.filter((row) => !insertedKeys.has(row.key)) - if (conflictingRows.length > 0) { - const activeRows = await db - .select() - .from(workspaceFiles) - .where( - and( - inArray( - workspaceFiles.key, - conflictingRows.map((row) => row.key) - ), - isNull(workspaceFiles.deletedAt) - ) - ) - const activeByKey = new Map(activeRows.map((record) => [record.key, record])) - for (const row of conflictingRows) { - const active = activeByKey.get(row.key) - if (!active) { - throw new ActiveFileMetadataKeyConflictError(row.key) - } - resolveExistingFileMetadata(active, row) - } - } -} - /** * Get file metadata by key with optional context filter */ diff --git a/apps/sim/lib/uploads/utils/file-utils.test.ts b/apps/sim/lib/uploads/utils/file-utils.test.ts index 338566c9335..ffcf4bb9cf2 100644 --- a/apps/sim/lib/uploads/utils/file-utils.test.ts +++ b/apps/sim/lib/uploads/utils/file-utils.test.ts @@ -5,7 +5,6 @@ import { extractWorkspaceIdFromStorageKey, inferContextFromKey, isInternalFileUrl, - isNetworkError, processSingleFileToUserFile, resolveEffectiveMimeType, resolveFileType, @@ -122,26 +121,6 @@ describe('resolveTrustedFileContext', () => { }) }) -describe('isNetworkError', () => { - it.each([ - 'fetch failed', - 'Network request failed', - 'connection reset', - 'request timeout', - 'operation timed out', - 'ECONNRESET while reading body', - ])('matches transient message %s', (msg) => { - expect(isNetworkError(new Error(msg))).toBe(true) - }) - - it('does not match deterministic errors', () => { - expect(isNetworkError(new Error('Forbidden'))).toBe(false) - expect(isNetworkError(new Error('Validation failed: name is required'))).toBe(false) - expect(isNetworkError('not an error')).toBe(false) - expect(isNetworkError(null)).toBe(false) - }) -}) - describe('processSingleFileToUserFile', () => { it('strips server-only provider file handles from untrusted input', () => { const result = processSingleFileToUserFile( diff --git a/apps/sim/lib/uploads/utils/file-utils.ts b/apps/sim/lib/uploads/utils/file-utils.ts index 76c17f9d162..2e7382d267a 100644 --- a/apps/sim/lib/uploads/utils/file-utils.ts +++ b/apps/sim/lib/uploads/utils/file-utils.ts @@ -524,24 +524,6 @@ export function isAbortError(error: unknown): boolean { ) } -/** - * Heuristic: whether `error` is a transient network/connection failure that's - * worth retrying (vs. a deterministic 4xx/auth/validation error). Sniffs the - * message because browsers and servers report these without standardized codes. - */ -export function isNetworkError(error: unknown): boolean { - if (!(error instanceof Error)) return false - const message = error.message.toLowerCase() - return ( - message.includes('network') || - message.includes('fetch') || - message.includes('connection') || - message.includes('timeout') || - message.includes('timed out') || - message.includes('econnreset') - ) -} - const MIME_TO_EXTENSION: Record = { // Images 'image/jpeg': 'jpg', diff --git a/apps/sim/lib/uploads/utils/validation.ts b/apps/sim/lib/uploads/utils/validation.ts index 86ec94fdf0b..fce28fa6ac1 100644 --- a/apps/sim/lib/uploads/utils/validation.ts +++ b/apps/sim/lib/uploads/utils/validation.ts @@ -166,25 +166,6 @@ export const SUPPORTED_MIME_TYPES: Record yml: ['text/yaml', 'text/x-yaml', 'application/yaml', 'application/x-yaml'], } -export const SUPPORTED_AUDIO_MIME_TYPES: Record = { - mp3: ['audio/mpeg', 'audio/mp3'], - m4a: ['audio/mp4', 'audio/x-m4a', 'audio/m4a'], - wav: ['audio/wav', 'audio/wave', 'audio/x-wav'], - webm: ['audio/webm'], - ogg: ['audio/ogg', 'audio/vorbis'], - flac: ['audio/flac', 'audio/x-flac'], - aac: ['audio/aac', 'audio/x-aac'], - opus: ['audio/opus'], -} - -export const SUPPORTED_VIDEO_MIME_TYPES: Record = { - mp4: ['video/mp4', 'video/mpeg'], - mov: ['video/quicktime', 'video/x-quicktime'], - avi: ['video/x-msvideo', 'video/avi'], - mkv: ['video/x-matroska'], - webm: ['video/webm'], -} - export const ACCEPTED_FILE_TYPES = Object.values(SUPPORTED_MIME_TYPES).flat() export const ACCEPTED_FILE_EXTENSIONS = SUPPORTED_DOCUMENT_EXTENSIONS.map((ext) => `.${ext}`) diff --git a/apps/sim/lib/workspace-files/application/resolve-workspace-file-reference.ts b/apps/sim/lib/workspace-files/application/resolve-workspace-file-reference.ts index 7f1c862af94..06d517477fa 100644 --- a/apps/sim/lib/workspace-files/application/resolve-workspace-file-reference.ts +++ b/apps/sim/lib/workspace-files/application/resolve-workspace-file-reference.ts @@ -3,7 +3,6 @@ import type { OperationUseCase, WorkspaceOperation } from '@/lib/core/applicatio import { OrchestrationError } from '@/lib/core/orchestration/types' import { type ActiveWorkspaceFileContext, - fetchWorkspaceFileBuffer, getWorkspaceFileByName, loadActiveWorkspaceFileContext, resolveWorkspaceFileReference as resolveStoredWorkspaceFileReference, @@ -36,10 +35,6 @@ interface WorkspaceFileReferenceResult { file: WorkspaceFileRecord } -interface WorkspaceFileReferenceReadInput extends WorkspaceFileReferenceInput { - maxBytes: number -} - /** Canonical file context plus the record the reference resolved to. */ export interface ReferencedWorkspaceFileContext extends ActiveWorkspaceFileContext { file: WorkspaceFileRecord @@ -154,46 +149,3 @@ export async function resolveWorkspaceFileReference({ }) return result.file } - -export interface ReadWorkspaceFileReferenceInput - extends Omit { - maxBytes: number -} - -const readWorkspaceFileReferenceUseCase = defineAuthorizedWorkspaceFileUseCase({ - operation: fileOperations.readContent, - resolveContext: ({ - principal, - input, - }: { - principal: Principal - input: WorkspaceFileReferenceReadInput - }) => resolveReferencedWorkspaceFileContext(principal, input, CHAT_UPLOAD_LOOKUP), - async execute({ input, context }): Promise<{ file: WorkspaceFileRecord; content: Buffer }> { - return { - file: context.file, - content: await fetchWorkspaceFileBuffer(context.file, { maxBytes: input.maxBytes }), - } - }, -}) - -/** Resolve one trusted workspace-file reference and read it under the shared file policy. */ -export async function readWorkspaceFileReference({ - principal, - workspaceId, - reference, - folderId, - maxBytes, - chatId, -}: ReadWorkspaceFileReferenceInput): Promise<{ file: WorkspaceFileRecord; content: Buffer }> { - return readWorkspaceFileReferenceUseCase.execute({ - principal, - input: { - workspaceId, - reference, - maxBytes, - ...(chatId === undefined ? {} : { chatId }), - ...(folderId === undefined ? {} : { folderId }), - }, - }) -} diff --git a/packages/testing/src/mocks/file-utils.mock.ts b/packages/testing/src/mocks/file-utils.mock.ts index a5f4a4240ac..2185fad7f22 100644 --- a/packages/testing/src/mocks/file-utils.mock.ts +++ b/packages/testing/src/mocks/file-utils.mock.ts @@ -545,18 +545,6 @@ export const fileUtilsMockFns = { 'name' in error && String((error as { name?: unknown }).name) === 'AbortError' ), - mockIsNetworkError: vi.fn((error: unknown) => { - if (!(error instanceof Error)) return false - const message = error.message.toLowerCase() - return ( - message.includes('network') || - message.includes('fetch') || - message.includes('connection') || - message.includes('timeout') || - message.includes('timed out') || - message.includes('econnreset') - ) - }), mockGetExtensionFromMimeType: vi.fn(getExtensionFromMimeType), mockEnsureFileNameExtension: vi.fn((fileName: string, contentType: string | null | undefined) => { if (!contentType || /^[a-z0-9]+$/.test(getFileExtension(fileName))) return fileName @@ -738,7 +726,6 @@ export const fileUtilsMock = { resolveFileType: fileUtilsMockFns.mockResolveFileType, getFileContentType: fileUtilsMockFns.mockGetFileContentType, isAbortError: fileUtilsMockFns.mockIsAbortError, - isNetworkError: fileUtilsMockFns.mockIsNetworkError, getExtensionFromMimeType: fileUtilsMockFns.mockGetExtensionFromMimeType, ensureFileNameExtension: fileUtilsMockFns.mockEnsureFileNameExtension, formatFileSize: fileUtilsMockFns.mockFormatFileSize, diff --git a/packages/testing/src/mocks/knowledge-documents-service.mock.ts b/packages/testing/src/mocks/knowledge-documents-service.mock.ts index 445345e3975..dbfa82db7a6 100644 --- a/packages/testing/src/mocks/knowledge-documents-service.mock.ts +++ b/packages/testing/src/mocks/knowledge-documents-service.mock.ts @@ -55,7 +55,6 @@ export const knowledgeDocumentsServiceMockFns = { mockMarkDocumentAsFailedTimeout: vi.fn(), mockRetryDocumentProcessing: vi.fn(), mockUpdateDocument: vi.fn(), - mockDeleteDocumentStorageFiles: vi.fn(), mockHardDeleteDocuments: vi.fn(), mockDeleteDocument: vi.fn(), mockDeleteKnowledgeDocumentInKnowledgeBase: vi.fn(), @@ -87,7 +86,6 @@ export const knowledgeDocumentsServiceMock = { markDocumentAsFailedTimeout: knowledgeDocumentsServiceMockFns.mockMarkDocumentAsFailedTimeout, retryDocumentProcessing: knowledgeDocumentsServiceMockFns.mockRetryDocumentProcessing, updateDocument: knowledgeDocumentsServiceMockFns.mockUpdateDocument, - deleteDocumentStorageFiles: knowledgeDocumentsServiceMockFns.mockDeleteDocumentStorageFiles, hardDeleteDocuments: knowledgeDocumentsServiceMockFns.mockHardDeleteDocuments, deleteDocument: knowledgeDocumentsServiceMockFns.mockDeleteDocument, deleteKnowledgeDocumentInKnowledgeBase: diff --git a/packages/testing/src/mocks/knowledge-service.mock.ts b/packages/testing/src/mocks/knowledge-service.mock.ts index 8e62b188ae6..233f19870dd 100644 --- a/packages/testing/src/mocks/knowledge-service.mock.ts +++ b/packages/testing/src/mocks/knowledge-service.mock.ts @@ -54,7 +54,6 @@ export class MockKnowledgeBaseNotFoundError extends Error { */ export const knowledgeServiceMockFns = { mockGetWorkspaceKnowledgeBases: vi.fn(), - mockFindActiveKnowledgeBasesByExactName: vi.fn(), mockCreateKnowledgeBase: vi.fn(), mockCreateAuthorizedKnowledgeBase: vi.fn(), mockUpdateKnowledgeBase: vi.fn(), @@ -81,8 +80,6 @@ export const knowledgeServiceMock = { KnowledgeBaseFolderError: MockKnowledgeBaseFolderError, KnowledgeBaseNotFoundError: MockKnowledgeBaseNotFoundError, getWorkspaceKnowledgeBases: knowledgeServiceMockFns.mockGetWorkspaceKnowledgeBases, - findActiveKnowledgeBasesByExactName: - knowledgeServiceMockFns.mockFindActiveKnowledgeBasesByExactName, createKnowledgeBase: knowledgeServiceMockFns.mockCreateKnowledgeBase, createAuthorizedKnowledgeBase: knowledgeServiceMockFns.mockCreateAuthorizedKnowledgeBase, updateKnowledgeBase: knowledgeServiceMockFns.mockUpdateKnowledgeBase, diff --git a/packages/testing/src/mocks/sim-search-connectors.mock.ts b/packages/testing/src/mocks/sim-search-connectors.mock.ts index f5870f53c72..0f36182fcb8 100644 --- a/packages/testing/src/mocks/sim-search-connectors.mock.ts +++ b/packages/testing/src/mocks/sim-search-connectors.mock.ts @@ -36,14 +36,12 @@ function personalSetupFields(meta: MockConnectorMeta): MockConnectorConfigField[ * Defaults (faithful ports of the real pure logic, reading only the `meta` passed in): * - `mockCanConnectPersonally`: `search === true`, OAuth auth, and a `permissionScopedListing`. * - `mockPersonalSetupFields`: required, non-selector config fields that are not listing caps. - * - `mockCanConnectWithDefaults`: connects personally, is not `slack`, and has no setup fields. * - `mockPersonalSourceConfigFieldIds`: setup-field ids plus `searchDefaultSourceConfig` keys. * - `mockWithSearchSourceDefaults`: defaults overlaid by supplied values; a blank value keeps its default. * - `mockMissingSetupFields`: setup fields whose `sourceConfig` value is missing or blank. * * Registry-backed defaults mirror the mock's EMPTY connector registry: * - `mockConnectorDisplayName` returns the connector type unchanged. - * - `mockSearchMemberAccountProvider` returns `null`. * * `mockGetConnectorAccessAvailability` and `mockIsSearchConnectorAvailable` are bare. * @@ -58,13 +56,8 @@ function personalSetupFields(meta: MockConnectorMeta): MockConnectorConfigField[ * ``` */ export const simSearchConnectorsMockFns = { - mockSearchMemberAccountProvider: vi.fn((_connectorType: string): unknown => null), mockCanConnectPersonally: vi.fn(canConnectPersonally), mockPersonalSetupFields: vi.fn(personalSetupFields), - mockCanConnectWithDefaults: vi.fn( - (meta: MockConnectorMeta): boolean => - canConnectPersonally(meta) && meta.id !== 'slack' && personalSetupFields(meta).length === 0 - ), mockPersonalSourceConfigFieldIds: vi.fn( (meta: MockConnectorMeta): Set => new Set([ @@ -112,10 +105,8 @@ export const simSearchConnectorsMock = { SIM_SEARCH_KNOWLEDGE_BASE_NAME: 'Sim Search', SEARCH_CONNECTORS: [] as readonly unknown[], SEARCH_SOURCE_TYPES: [] as readonly (readonly [string, unknown])[], - searchMemberAccountProvider: simSearchConnectorsMockFns.mockSearchMemberAccountProvider, canConnectPersonally: simSearchConnectorsMockFns.mockCanConnectPersonally, personalSetupFields: simSearchConnectorsMockFns.mockPersonalSetupFields, - canConnectWithDefaults: simSearchConnectorsMockFns.mockCanConnectWithDefaults, personalSourceConfigFieldIds: simSearchConnectorsMockFns.mockPersonalSourceConfigFieldIds, withSearchSourceDefaults: simSearchConnectorsMockFns.mockWithSearchSourceDefaults, missingSetupFields: simSearchConnectorsMockFns.mockMissingSetupFields, diff --git a/packages/testing/src/mocks/storage-service.mock.ts b/packages/testing/src/mocks/storage-service.mock.ts index d33ef648fc4..e6ad81d17be 100644 --- a/packages/testing/src/mocks/storage-service.mock.ts +++ b/packages/testing/src/mocks/storage-service.mock.ts @@ -35,7 +35,6 @@ export const storageServiceMockFns = { mockHeadObject: vi.fn(), mockGeneratePresignedDownloadUrl: vi.fn(), mockHasCloudStorage: vi.fn(() => false), - mockGetS3InfoForKey: vi.fn(), mockCreateBlobConfig: vi.fn((config: MockStorageConfig) => { if (!config.containerName) { throw new Error('Blob configuration missing required property: containerName') @@ -86,7 +85,6 @@ export const storageServiceMock = { headObject: storageServiceMockFns.mockHeadObject, generatePresignedDownloadUrl: storageServiceMockFns.mockGeneratePresignedDownloadUrl, hasCloudStorage: storageServiceMockFns.mockHasCloudStorage, - getS3InfoForKey: storageServiceMockFns.mockGetS3InfoForKey, createBlobConfig: storageServiceMockFns.mockCreateBlobConfig, createS3Config: storageServiceMockFns.mockCreateS3Config, createGcsConfig: storageServiceMockFns.mockCreateGcsConfig, diff --git a/packages/testing/src/mocks/table-jobs-service.mock.ts b/packages/testing/src/mocks/table-jobs-service.mock.ts index e7ff6408f32..cd90da0b991 100644 --- a/packages/testing/src/mocks/table-jobs-service.mock.ts +++ b/packages/testing/src/mocks/table-jobs-service.mock.ts @@ -60,7 +60,6 @@ export const tableJobsServiceMockFns = { mockGetJobProgress: vi.fn(), mockSelectExportRowPage: vi.fn(), mockListWorkspaceExportJobs: vi.fn(), - mockGetTableJob: vi.fn(), mockSetJobResultKeyInWorkspace: vi.fn(), mockMarkJobReady: vi.fn(), mockMarkJobReadyInWorkspace: vi.fn(), @@ -92,7 +91,6 @@ export const tableJobsServiceMock = { getJobProgress: tableJobsServiceMockFns.mockGetJobProgress, selectExportRowPage: tableJobsServiceMockFns.mockSelectExportRowPage, listWorkspaceExportJobs: tableJobsServiceMockFns.mockListWorkspaceExportJobs, - getTableJob: tableJobsServiceMockFns.mockGetTableJob, setJobResultKeyInWorkspace: tableJobsServiceMockFns.mockSetJobResultKeyInWorkspace, markJobReady: tableJobsServiceMockFns.mockMarkJobReady, markJobReadyInWorkspace: tableJobsServiceMockFns.mockMarkJobReadyInWorkspace, diff --git a/packages/testing/src/mocks/table.mock.ts b/packages/testing/src/mocks/table.mock.ts index cd2533a960f..98b1611de8f 100644 --- a/packages/testing/src/mocks/table.mock.ts +++ b/packages/testing/src/mocks/table.mock.ts @@ -163,7 +163,6 @@ export const tableMockFns = { mockSelectValueForConversion: vi.fn(), mockAddTableColumn: vi.fn(), mockDeleteColumn: vi.fn(), - mockDeleteColumns: vi.fn(), mockRenameColumn: vi.fn(), mockUpdateColumnConstraints: vi.fn(), mockUpdateColumnCurrency: vi.fn(), @@ -234,7 +233,6 @@ export const tableMockFns = { mockBuildFilterClause: vi.fn(), mockBuildPredicateClause: vi.fn(), mockBuildSortClause: vi.fn(), - mockEscapeLikePattern: vi.fn(), mockFieldPredicate: vi.fn(), mockCoerceRowToSchema: vi.fn(), mockCoerceRowValues: vi.fn(), @@ -244,7 +242,6 @@ export const tableMockFns = { mockValidateRowSize: vi.fn(), mockValidateTableName: vi.fn(), mockValidateTableSchema: vi.fn(), - mockValidateUniqueConstraints: vi.fn(), mockCheckBatchUniqueConstraintsDb: vi.fn(), mockCheckUniqueConstraintsDb: vi.fn(), mockValidateBatchRows: vi.fn(), @@ -254,18 +251,14 @@ export const tableMockFns = { mockNormalizeStoredViewConfig: vi.fn(), mockNormalizeViewConfigForStorage: vi.fn(), mockPruneViewConfig: vi.fn(), - mockViewConfigIdsToNames: vi.fn(), - mockViewConfigNamesToIds: vi.fn(), mockCreateTableView: vi.fn(), mockDeleteTableView: vi.fn(), mockGetTableView: vi.fn(), mockListTableViews: vi.fn(), - mockListTableViewsByWorkspace: vi.fn(), mockUpdateTableView: vi.fn(), mockAddWorkflowGroup: vi.fn(), mockAddWorkflowGroupOutput: vi.fn(), mockDeleteWorkflowGroup: vi.fn(), - mockDeleteWorkflowGroupOutput: vi.fn(), mockPruneStaleWorkflowGroupOutputs: vi.fn(), mockUpdateWorkflowGroup: vi.fn(), } @@ -416,7 +409,6 @@ export const tableMock = { selectValueForConversion: fns.mockSelectValueForConversion, addTableColumn: fns.mockAddTableColumn, deleteColumn: fns.mockDeleteColumn, - deleteColumns: fns.mockDeleteColumns, renameColumn: fns.mockRenameColumn, updateColumnConstraints: fns.mockUpdateColumnConstraints, updateColumnCurrency: fns.mockUpdateColumnCurrency, @@ -477,7 +469,6 @@ export const tableMock = { buildFilterClause: fns.mockBuildFilterClause, buildPredicateClause: fns.mockBuildPredicateClause, buildSortClause: fns.mockBuildSortClause, - escapeLikePattern: fns.mockEscapeLikePattern, fieldPredicate: fns.mockFieldPredicate, coerceRowToSchema: fns.mockCoerceRowToSchema, coerceRowValues: fns.mockCoerceRowValues, @@ -487,7 +478,6 @@ export const tableMock = { validateRowSize: fns.mockValidateRowSize, validateTableName: fns.mockValidateTableName, validateTableSchema: fns.mockValidateTableSchema, - validateUniqueConstraints: fns.mockValidateUniqueConstraints, checkBatchUniqueConstraintsDb: fns.mockCheckBatchUniqueConstraintsDb, checkUniqueConstraintsDb: fns.mockCheckUniqueConstraintsDb, validateBatchRows: fns.mockValidateBatchRows, @@ -497,18 +487,14 @@ export const tableMock = { normalizeStoredViewConfig: fns.mockNormalizeStoredViewConfig, normalizeViewConfigForStorage: fns.mockNormalizeViewConfigForStorage, pruneViewConfig: fns.mockPruneViewConfig, - viewConfigIdsToNames: fns.mockViewConfigIdsToNames, - viewConfigNamesToIds: fns.mockViewConfigNamesToIds, createTableView: fns.mockCreateTableView, deleteTableView: fns.mockDeleteTableView, getTableView: fns.mockGetTableView, listTableViews: fns.mockListTableViews, - listTableViewsByWorkspace: fns.mockListTableViewsByWorkspace, updateTableView: fns.mockUpdateTableView, addWorkflowGroup: fns.mockAddWorkflowGroup, addWorkflowGroupOutput: fns.mockAddWorkflowGroupOutput, deleteWorkflowGroup: fns.mockDeleteWorkflowGroup, - deleteWorkflowGroupOutput: fns.mockDeleteWorkflowGroupOutput, pruneStaleWorkflowGroupOutputs: fns.mockPruneStaleWorkflowGroupOutputs, updateWorkflowGroup: fns.mockUpdateWorkflowGroup, } diff --git a/packages/testing/src/mocks/uploads-metadata.mock.ts b/packages/testing/src/mocks/uploads-metadata.mock.ts index 98400f311d9..fb35700b322 100644 --- a/packages/testing/src/mocks/uploads-metadata.mock.ts +++ b/packages/testing/src/mocks/uploads-metadata.mock.ts @@ -29,7 +29,6 @@ export class MockActiveFileMetadataKeyConflictError extends Error { export const uploadsMetadataMockFns = { mockInsertFileMetadata: vi.fn(), mockInsertImmutableFileMetadata: vi.fn(), - mockInsertFileMetadataMany: vi.fn(), mockGetFileMetadataByKey: vi.fn(async (..._args: unknown[]): Promise => null), mockResolveStoredFileContext: vi.fn(async (_key: string): Promise => 'workspace'), mockGetFileMetadataByKeys: vi.fn(async (..._args: unknown[]): Promise => []), @@ -51,7 +50,6 @@ export const uploadsMetadataMock = { ActiveFileMetadataKeyConflictError: MockActiveFileMetadataKeyConflictError, insertFileMetadata: uploadsMetadataMockFns.mockInsertFileMetadata, insertImmutableFileMetadata: uploadsMetadataMockFns.mockInsertImmutableFileMetadata, - insertFileMetadataMany: uploadsMetadataMockFns.mockInsertFileMetadataMany, getFileMetadataByKey: uploadsMetadataMockFns.mockGetFileMetadataByKey, resolveStoredFileContext: uploadsMetadataMockFns.mockResolveStoredFileContext, getFileMetadataByKeys: uploadsMetadataMockFns.mockGetFileMetadataByKeys, diff --git a/packages/testing/src/mocks/uploads.mock.ts b/packages/testing/src/mocks/uploads.mock.ts index 1dd979950fd..70fd18b3ad7 100644 --- a/packages/testing/src/mocks/uploads.mock.ts +++ b/packages/testing/src/mocks/uploads.mock.ts @@ -5,8 +5,7 @@ import { storageServiceMock } from './storage-service.mock' * Controllable mock functions for the `@/lib/uploads` barrel. * * Defaults describe local storage: `isUsingCloudStorage` → `false`, - * `getStorageProvider` → `'local'`, `getServePathPrefix` → `'/api/files/serve/'` (the real - * constant). The `StorageService` namespace is `storageServiceMock`, so drive storage I/O + * `getServePathPrefix` → `'/api/files/serve/'` (the real constant). The `StorageService` namespace is `storageServiceMock`, so drive storage I/O * through `storageServiceMockFns` from `@sim/testing/mocks/storage-service.mock`. * * @example @@ -23,7 +22,6 @@ export const uploadsMockFns = { mockIsUsingCloudStorage: vi.fn(() => false), mockGetFileMetadata: vi.fn(), mockGetServePathPrefix: vi.fn(() => '/api/files/serve/'), - mockGetStorageProvider: vi.fn((): 'blob' | 's3' | 'gcs' | 'local' => 'local'), mockProcessChatFiles: vi.fn(), mockDownloadCopilotFile: vi.fn(), mockUploadCopilotFile: vi.fn(), @@ -42,7 +40,6 @@ export const uploadsMock = { isUsingCloudStorage: uploadsMockFns.mockIsUsingCloudStorage, getFileMetadata: uploadsMockFns.mockGetFileMetadata, getServePathPrefix: uploadsMockFns.mockGetServePathPrefix, - getStorageProvider: uploadsMockFns.mockGetStorageProvider, ChatFiles: { processChatFiles: uploadsMockFns.mockProcessChatFiles, }, diff --git a/packages/testing/src/mocks/workspace-file-manager.mock.ts b/packages/testing/src/mocks/workspace-file-manager.mock.ts index ece4895031c..81a8336fb36 100644 --- a/packages/testing/src/mocks/workspace-file-manager.mock.ts +++ b/packages/testing/src/mocks/workspace-file-manager.mock.ts @@ -96,10 +96,9 @@ function vfsPath(file: MockWorkspaceFilePathFields): string { * * I/O functions are bare `vi.fn()`s. Pure helpers default to ports of the real logic: * `matchesWorkspaceFilePattern`, `parseWorkspaceFileKey`, `suffixedName` are faithful; - * the VFS helpers (`normalizeWorkspaceFileReference`, `workspaceFileVfsPath`, - * `getSandboxWorkspaceFilePath`, `parseChatUploadReference`, `findWorkspaceFileRecord`) - * percent-encode with `encodeURIComponent` and split folder paths on `/` (no escaped-slash - * folder names). `generateWorkspaceFileKey` is deterministic — + * the VFS helpers (`workspaceFileVfsPath`, `getSandboxWorkspaceFilePath`, + * `parseChatUploadReference`, `findWorkspaceFileRecord`) percent-encode with + * `encodeURIComponent` and split folder paths on `/` (no escaped-slash folder names). `generateWorkspaceFileKey` is deterministic — * `workspace/{workspaceId}/generated-{fileName}` — which `parseWorkspaceFileKey` does * not recognize (no timestamp/random segment). * @@ -130,9 +129,6 @@ export const workspaceFileManagerMockFns = { mockGetWorkspaceFileByName: vi.fn(), mockListWorkspaceFiles: vi.fn(), mockQueryWorkspaceFiles: vi.fn(), - mockNormalizeWorkspaceFileReference: vi.fn((fileReference: string) => - referenceSegments(fileReference).join('/') - ), mockWorkspaceFileVfsPath: vi.fn(vfsPath), mockGetSandboxWorkspaceFilePath: vi.fn( (file: MockWorkspaceFilePathFields) => `/home/user/${vfsPath(file)}` @@ -171,7 +167,6 @@ export const workspaceFileManagerMockFns = { mockUpdateWorkspaceFileContent: vi.fn(), mockDeleteWorkspaceFileVersion: vi.fn(), mockRenameWorkspaceFile: vi.fn(), - mockMoveRenameWorkspaceFile: vi.fn(), mockDeleteWorkspaceFile: vi.fn(), mockPurgeCreatedWorkspaceFile: vi.fn(), mockRestoreWorkspaceFile: vi.fn(), @@ -205,7 +200,6 @@ export const workspaceFileManagerMock = { getWorkspaceFileByName: fns.mockGetWorkspaceFileByName, listWorkspaceFiles: fns.mockListWorkspaceFiles, queryWorkspaceFiles: fns.mockQueryWorkspaceFiles, - normalizeWorkspaceFileReference: fns.mockNormalizeWorkspaceFileReference, workspaceFileVfsPath: fns.mockWorkspaceFileVfsPath, getSandboxWorkspaceFilePath: fns.mockGetSandboxWorkspaceFilePath, parseChatUploadReference: fns.mockParseChatUploadReference, @@ -222,7 +216,6 @@ export const workspaceFileManagerMock = { updateWorkspaceFileContent: fns.mockUpdateWorkspaceFileContent, deleteWorkspaceFileVersion: fns.mockDeleteWorkspaceFileVersion, renameWorkspaceFile: fns.mockRenameWorkspaceFile, - moveRenameWorkspaceFile: fns.mockMoveRenameWorkspaceFile, deleteWorkspaceFile: fns.mockDeleteWorkspaceFile, purgeCreatedWorkspaceFile: fns.mockPurgeCreatedWorkspaceFile, restoreWorkspaceFile: fns.mockRestoreWorkspaceFile, diff --git a/packages/testing/src/mocks/workspace-file-reference.mock.ts b/packages/testing/src/mocks/workspace-file-reference.mock.ts index 2a6db6c7028..690be7b7714 100644 --- a/packages/testing/src/mocks/workspace-file-reference.mock.ts +++ b/packages/testing/src/mocks/workspace-file-reference.mock.ts @@ -15,7 +15,6 @@ import { vi } from 'vitest' export const workspaceFileReferenceMockFns = { mockResolveReferencedWorkspaceFileContext: vi.fn(), mockResolveWorkspaceFileReference: vi.fn(), - mockReadWorkspaceFileReference: vi.fn(), } /** @@ -33,5 +32,4 @@ export const workspaceFileReferenceMock = { resolveReferencedWorkspaceFileContext: workspaceFileReferenceMockFns.mockResolveReferencedWorkspaceFileContext, resolveWorkspaceFileReference: workspaceFileReferenceMockFns.mockResolveWorkspaceFileReference, - readWorkspaceFileReference: workspaceFileReferenceMockFns.mockReadWorkspaceFileReference, } From c3aceb6005422c44f093c8ee31f0434437e7a1db Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 15:24:38 -0700 Subject: [PATCH 24/30] chore(cleanup): use shared escapeLikePattern in list-query and logs filters --- apps/sim/lib/api/list-query.test.ts | 9 --------- apps/sim/lib/api/list-query.ts | 10 +--------- apps/sim/lib/logs/public-filters.ts | 2 +- 3 files changed, 2 insertions(+), 19 deletions(-) diff --git a/apps/sim/lib/api/list-query.test.ts b/apps/sim/lib/api/list-query.test.ts index 80dddfd1cdb..c4a8ff110a9 100644 --- a/apps/sim/lib/api/list-query.test.ts +++ b/apps/sim/lib/api/list-query.test.ts @@ -11,7 +11,6 @@ vi.unmock('drizzle-orm') import { decimal, integer, PgDialect, pgTable, text, timestamp } from 'drizzle-orm/pg-core' import { decimalKey, - escapeLikePattern, keysetAfter, listOrderBy, numberKey, @@ -34,14 +33,6 @@ function render(fragment: Parameters[0]) { return dialect.sqlToQuery(fragment) } -describe('escapeLikePattern', () => { - it('neutralizes the LIKE wildcards so a caller cannot widen its own match', () => { - expect(escapeLikePattern('100%')).toBe('100\\%') - expect(escapeLikePattern('a_b')).toBe('a\\_b') - expect(escapeLikePattern('back\\slash')).toBe('back\\\\slash') - }) -}) - describe('searchFilter', () => { it('binds the caller term as a parameter instead of inlining it into the SQL', () => { const { sql, params } = render(searchFilter(thing.name, "o'brien; drop table thing --")!) diff --git a/apps/sim/lib/api/list-query.ts b/apps/sim/lib/api/list-query.ts index 2fecf05fb32..91a35c8e1ec 100644 --- a/apps/sim/lib/api/list-query.ts +++ b/apps/sim/lib/api/list-query.ts @@ -1,3 +1,4 @@ +import { escapeLikePattern } from '@sim/utils/string' import { and, asc, @@ -28,15 +29,6 @@ export type ListSortOrder = (typeof LIST_SORT_ORDERS)[number] * cursor's values are type-checked against their key before they are bound. */ -/** - * Escapes LIKE/ILIKE wildcards so `%`, `_`, and `\` in a caller's term match - * themselves. Postgres treats `\` as the default LIKE escape character, so no - * explicit `ESCAPE` clause is needed. - */ -export function escapeLikePattern(value: string): string { - return value.replace(/[\\%_]/g, '\\$&') -} - /** * Case-insensitive substring predicate for a v2 `search` term, or `undefined` * when the caller did not search (which drops out of an `and(...)`). diff --git a/apps/sim/lib/logs/public-filters.ts b/apps/sim/lib/logs/public-filters.ts index 231842b9e85..cebdc436ccb 100644 --- a/apps/sim/lib/logs/public-filters.ts +++ b/apps/sim/lib/logs/public-filters.ts @@ -1,6 +1,6 @@ import { jobExecutionLogs, workflow, workflowExecutionLogs } from '@sim/db/schema' +import { escapeLikePattern } from '@sim/utils/string' import { and, eq, gte, inArray, lte, or, type SQL, sql } from 'drizzle-orm' -import { escapeLikePattern } from '@/lib/api/list-query' import { handledErrorSpanCondition } from '@/lib/logs/handled-errors' import type { PersistedWorkflowExecutionStatus } from '@/lib/logs/types' From 28894522fa4c6205ce7d9b4fe3e4016d8a03da5e Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 16:11:41 -0700 Subject: [PATCH 25/30] docs(skills): align codebase-design testing guidance with repo rules and add its license --- .agents/skills/codebase-design/DEEPENING.md | 19 ++++++++++--------- .agents/skills/codebase-design/LICENSE | 21 +++++++++++++++++++++ .agents/skills/codebase-design/SKILL.md | 10 +++++----- 3 files changed, 36 insertions(+), 14 deletions(-) create mode 100644 .agents/skills/codebase-design/LICENSE diff --git a/.agents/skills/codebase-design/DEEPENING.md b/.agents/skills/codebase-design/DEEPENING.md index cd94075cfd7..2a3ef47c4bd 100644 --- a/.agents/skills/codebase-design/DEEPENING.md +++ b/.agents/skills/codebase-design/DEEPENING.md @@ -12,26 +12,27 @@ Pure computation, in-memory state, no I/O. Always deepenable: merge the modules ### 2. Local-substitutable -Dependencies that have local test stand-ins (PGLite for Postgres, in-memory filesystem). Deepenable if the stand-in exists. The deepened module is tested with the stand-in running in the test suite. The seam is internal; no port at the module's external interface. +Dependencies you can run for real in a test: Postgres, Redis, the filesystem. Deepenable. The seam is internal; no port at the module's external interface. Verify the deepened module against the real dependency (a `*.integration.ts` suite on real Postgres/Redis) or end to end, never against a hand-built stand-in. ### 3. Remote but owned (Ports & Adapters) -Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Tests use an in-memory adapter. Production uses an HTTP/gRPC/queue adapter. +Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Production uses an HTTP/gRPC/queue adapter. Prove the module end to end over the real transport (`apps/sim/scripts/test-*-e2e.ts`) rather than through an in-memory adapter written only for tests. -Recommendation shape: *"Define a port at the seam, implement an HTTP adapter for production and an in-memory adapter for testing, so the logic sits in one deep module even though it's deployed across a network."* +Recommendation shape: *"Define a port at the seam and implement the production adapter there, so the logic sits in one deep module even though it's deployed across a network."* ### 4. True external (Mock) -Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port; tests provide a mock adapter. +Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port. Where an isolated test is unavoidable, use the shared mocks in `@sim/testing` and `apps/sim/vitest.setup.ts` rather than hand-rolling one. ## Seam discipline -- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified (typically production + test). A single-adapter seam is just indirection. +- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified. A single-adapter seam is just indirection. - **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them. ## Testing strategy: replace, don't layer -- Old unit tests on shallow modules become waste once tests at the deepened module's interface exist; delete them. -- Write new tests at the deepened module's interface. The **interface is the test surface**. -- Tests assert on observable outcomes through the interface, not internal state. -- Tests should survive internal refactors, since they describe behaviour, not implementation. If a test has to change when the implementation changes, it's testing past the interface. +Testing follows the repo's rules in `CLAUDE.md` ("Testing") and the `test-audit` skill; this section only says how they apply to deepening. + +- Old unit tests on shallow modules become waste once the deepened module exists; delete them. Deleting a test that only restated a pass-through is a win, not lost coverage. +- Don't write new unit tests at the deepened interface after the fact. Prove the change through the real boundary: a `*.integration.ts` suite on real Postgres/Redis, or an end-to-end run. +- Whatever tests remain assert on observable outcomes through the interface, not internal state. If a test has to change when the implementation changes, it's testing past the interface. diff --git a/.agents/skills/codebase-design/LICENSE b/.agents/skills/codebase-design/LICENSE new file mode 100644 index 00000000000..f1dd2c09108 --- /dev/null +++ b/.agents/skills/codebase-design/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Matt Pocock + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/.agents/skills/codebase-design/SKILL.md b/.agents/skills/codebase-design/SKILL.md index 8499f52ab60..8f87fc40b5f 100644 --- a/.agents/skills/codebase-design/SKILL.md +++ b/.agents/skills/codebase-design/SKILL.md @@ -75,10 +75,10 @@ Good interfaces make testing natural: ```typescript // Testable - function processOrder(order, paymentGateway) {} + function processOrder(order: Order, paymentGateway: PaymentGateway): Promise {} // Hard to test - function processOrder(order) { + function processOrder(order: Order): Promise { const gateway = new StripeGateway(); } ``` @@ -87,10 +87,10 @@ Good interfaces make testing natural: ```typescript // Testable - function calculateDiscount(cart): Discount {} + function calculateDiscount(cart: Cart): Discount {} // Hard to test - function applyDiscount(cart): void { + function applyDiscount(cart: Cart): void { cart.total -= discount; } ``` @@ -113,5 +113,5 @@ Good interfaces make testing natural: ## Going deeper -- **Deepening a cluster given its dependencies**, see [DEEPENING.md](DEEPENING.md): dependency categories, seam discipline, and replace-don't-layer testing. +- **Deepening a cluster given its dependencies**, see [DEEPENING.md](DEEPENING.md): dependency categories, seam discipline, and replace-don't-layer testing (which defers to `CLAUDE.md` "Testing" and the `test-audit` skill). - **Exploring alternative interfaces**, see [DESIGN-IT-TWICE.md](DESIGN-IT-TWICE.md): spin up parallel sub-agents to design the interface several radically different ways, then compare on depth, locality, and seam placement. From 841c34d3740ed3192189a40f3d99aa22604089fc Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 16:20:00 -0700 Subject: [PATCH 26/30] chore(cleanup): cascade-remove use cases, helpers, and mocks orphaned by deleted Copilot commands --- apps/sim/app/api/v1/middleware.ts | 4 +- .../lib/api/server/routes/internal-request.ts | 22 - .../api/server/routes/v2-json-route.test.ts | 16 - .../lib/api/server/routes/v2-json-route.ts | 12 +- .../core/account-billing-snapshot.test.ts | 63 -- .../billing/core/account-billing-snapshot.ts | 58 -- apps/sim/lib/billing/core/usage.ts | 4 +- apps/sim/lib/core/security/csp.ts | 4 +- .../credential-groups/application/context.ts | 34 +- apps/sim/lib/credential-groups/credentials.ts | 7 - apps/sim/lib/credential-groups/providers.ts | 2 +- .../custom-tools/application/operations.ts | 24 - apps/sim/lib/folders/scope.ts | 4 +- ...xecution-archive-provenance.integration.ts | 71 +- .../mothership/application/table-commands.ts | 19 +- .../tools/handlers/workflow/mutations.test.ts | 4 - .../tools/sandbox-resource-transport.test.ts | 3 - .../copilot-table-lifecycle.test.ts | 143 ---- .../application/copilot-table-lifecycle.ts | 85 --- apps/sim/lib/table/application/groups.test.ts | 165 +---- apps/sim/lib/table/application/groups.ts | 591 +---------------- apps/sim/lib/table/application/operations.ts | 19 - .../workspace-file-imports.test.ts | 369 ----------- .../application/workspace-file-imports.ts | 615 ------------------ apps/sim/lib/table/constants.ts | 7 - apps/sim/lib/table/import.test.ts | 9 - apps/sim/lib/table/import.ts | 105 +-- apps/sim/lib/table/jobs/service.ts | 20 - apps/sim/lib/table/types.ts | 18 - .../lib/table/workflow-groups/service.test.ts | 24 +- apps/sim/lib/table/workflow-groups/service.ts | 270 +------- .../lib/workflows/application/operations.ts | 8 - .../update-workflow-content.test.ts | 72 +- .../application/update-workflow-content.ts | 163 ----- .../lib/workflows/custom-tools/operations.ts | 58 -- .../lib/workflows/editing/block-enablement.ts | 6 +- apps/sim/lib/workspaces/organization/types.ts | 8 - apps/sim/lib/workspaces/utils.ts | 2 +- .../testing/src/mocks/billing-usage.mock.ts | 2 - .../credential-groups-credentials.mock.ts | 3 - .../testing/src/mocks/table-constants.mock.ts | 1 - .../src/mocks/table-jobs-service.mock.ts | 2 - packages/testing/src/mocks/table.mock.ts | 8 - .../src/mocks/workspaces-utils.mock.ts | 2 - 44 files changed, 31 insertions(+), 3095 deletions(-) delete mode 100644 apps/sim/lib/api/server/routes/internal-request.ts delete mode 100644 apps/sim/lib/billing/core/account-billing-snapshot.test.ts delete mode 100644 apps/sim/lib/billing/core/account-billing-snapshot.ts delete mode 100644 apps/sim/lib/table/application/copilot-table-lifecycle.test.ts delete mode 100644 apps/sim/lib/table/application/copilot-table-lifecycle.ts delete mode 100644 apps/sim/lib/table/application/workspace-file-imports.test.ts delete mode 100644 apps/sim/lib/table/application/workspace-file-imports.ts diff --git a/apps/sim/app/api/v1/middleware.ts b/apps/sim/app/api/v1/middleware.ts index 9c9aa5110b3..414aaf2efa6 100644 --- a/apps/sim/app/api/v1/middleware.ts +++ b/apps/sim/app/api/v1/middleware.ts @@ -40,7 +40,7 @@ const rateLimiter = new RateLimiter() * `defineV2JsonRoute` and rate-limited through `v2RateLimits`. Add a member only * when a route actually passes it to `checkRateLimit` / `authenticateRequest`. */ -export type ApiEndpoint = +type ApiEndpoint = | 'logs' | 'logs-detail' | 'workflows' @@ -84,7 +84,7 @@ export interface RateLimitResult { error?: string } -export interface AuthorizedRequest { +interface AuthorizedRequest { requestId: string userId: string rateLimit: RateLimitResult diff --git a/apps/sim/lib/api/server/routes/internal-request.ts b/apps/sim/lib/api/server/routes/internal-request.ts deleted file mode 100644 index 7dcbfc97e77..00000000000 --- a/apps/sim/lib/api/server/routes/internal-request.ts +++ /dev/null @@ -1,22 +0,0 @@ -/** - * Requests the server makes to itself. - * - * The embedded CLI and the agent-cli engines dispatch to the v2 route handlers - * in-process (see `in-process-transport.ts`). Those requests are marked here so - * admission can skip the abuse controls that exist for callers arriving over the - * network: the pre-auth IP bucket and the per-key rate limits. Authentication is - * not skipped — an internal request still carries the caller's key and resolves - * to the same principal the network path would. - * - * A WeakSet keyed by the Request object, not a header: any client on the wire - * can set a header; nothing outside this process can reach the set. - */ -const INTERNAL_REQUESTS = new WeakSet() - -export function markInternalRequest(request: Request): void { - INTERNAL_REQUESTS.add(request) -} - -export function isInternalRequest(request: Request): boolean { - return INTERNAL_REQUESTS.has(request) -} diff --git a/apps/sim/lib/api/server/routes/v2-json-route.test.ts b/apps/sim/lib/api/server/routes/v2-json-route.test.ts index 4e294259ca4..c287431dfc5 100644 --- a/apps/sim/lib/api/server/routes/v2-json-route.test.ts +++ b/apps/sim/lib/api/server/routes/v2-json-route.test.ts @@ -25,7 +25,6 @@ class TestLockedError extends HttpError { vi.mock('@/lib/api/server/routes/v2-api-key-auth', () => v2ApiKeyAuthModuleMock) vi.mock('@/lib/core/rate-limiter', () => v2RateLimiterModuleMock) -import { markInternalRequest } from '@/lib/api/server/routes/internal-request' import type { V2ApiKeyAuthContext } from '@/lib/api/server/routes/v2-api-key-auth' import { admitOptionalV2Request, @@ -207,21 +206,6 @@ describe('defineV2JsonRoute', () => { expect(response.headers.get('Cache-Control')).toBe('private, no-store') }) - it('authenticates but never rate-limits a request the server marked as its own', async () => { - // The embedded CLI and the agent-cli engines dispatch to these handlers in-process; - // the IP bucket and the per-key limits exist for callers on the wire, and a chat - // turn's tool calls all land on one key (dev 2026-09-03: grep hit the limit). - const internal = request() - markInternalRequest(internal) - - const response = await createHandler()(internal) - - expect(response.status).toBe(201) - expect(v2RouteMocks.authenticate).toHaveBeenCalledTimes(1) - expect(v2RouteMocks.preauthRate).not.toHaveBeenCalled() - expect(v2RouteMocks.operationRate).not.toHaveBeenCalled() - }) - it('fails closed before authentication when the IP bucket cannot admit the request', async () => { v2RouteMocks.preauthRate.mockResolvedValueOnce({ allowed: false, diff --git a/apps/sim/lib/api/server/routes/v2-json-route.ts b/apps/sim/lib/api/server/routes/v2-json-route.ts index 48b1393f73b..153b52c81c4 100644 --- a/apps/sim/lib/api/server/routes/v2-json-route.ts +++ b/apps/sim/lib/api/server/routes/v2-json-route.ts @@ -9,7 +9,6 @@ import { methodMatchesContract, requireJsonRouteDefinition, } from '@/lib/api/server/routes/definition' -import { isInternalRequest } from '@/lib/api/server/routes/internal-request' import type { JsonApiRouteContract, JsonNextRouteHandler, @@ -358,12 +357,7 @@ async function admitAuthenticatedV2Request( throw error } - // The server's own requests (embedded CLI, agent-cli engines) are authenticated like - // any other but never rate limited: the buckets exist for callers on the wire, and a - // chat turn's tool calls all land on one key. See `internal-request.ts`. - const limited = isInternalRequest(request) - ? null - : await rateLimitPolicy.enforce(request, auth, operation) + const limited = await rateLimitPolicy.enforce(request, auth, operation) return limited ? { success: false, response: limited } : { success: true, auth } } @@ -397,7 +391,7 @@ async function admitRateLimitedV2Request( setRequestAuth(describePrincipalAuth(principal)) return { success: true, auth: { principal } } } - const preAuthResponse = isInternalRequest(request) ? null : await enforceV2PreAuthIpLimit(request) + const preAuthResponse = await enforceV2PreAuthIpLimit(request) if (preAuthResponse) return { success: false, response: preAuthResponse } return admitAuthenticatedV2Request(request, operation, authPolicy, rateLimitPolicy) } @@ -437,7 +431,7 @@ export async function admitOptionalV2Request( ): Promise<{ success: true; auth?: V2AdmissionAuth } | { success: false; response: NextResponse }> { if (isCopilotRequest(request)) return admitRateLimitedV2Request(request, operation, authPolicy, rateLimitPolicy, useCase) - const preAuthResponse = isInternalRequest(request) ? null : await enforceV2PreAuthIpLimit(request) + const preAuthResponse = await enforceV2PreAuthIpLimit(request) if (preAuthResponse) return { success: false, response: preAuthResponse } if (!hasV2Credential(request.headers)) return { success: true } return admitAuthenticatedV2Request(request, operation, authPolicy, rateLimitPolicy) diff --git a/apps/sim/lib/billing/core/account-billing-snapshot.test.ts b/apps/sim/lib/billing/core/account-billing-snapshot.test.ts deleted file mode 100644 index a538e42fa8d..00000000000 --- a/apps/sim/lib/billing/core/account-billing-snapshot.test.ts +++ /dev/null @@ -1,63 +0,0 @@ -import { - billingSubscriptionUtilsMock, - billingSubscriptionUtilsMockFns, -} from '@sim/testing/mocks/billing-subscription-utils.mock' -import { billingUsageMock, billingUsageMockFns } from '@sim/testing/mocks/billing-usage.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' - -const hoisted = vi.hoisted(() => ({ - events: [] as string[], - getCreditBalanceForEntity: vi.fn(), -})) - -vi.mock('@/lib/billing/core/usage', () => billingUsageMock) - -vi.mock('@/lib/billing/credits/balance', () => ({ - getCreditBalanceForEntity: hoisted.getCreditBalanceForEntity, -})) - -vi.mock('@/lib/billing/subscriptions/utils', () => billingSubscriptionUtilsMock) - -import { getAccountBillingSnapshot } from '@/lib/billing/core/account-billing-snapshot' - -const mocks = { - ...hoisted, - getResolvedUserUsageData: billingUsageMockFns.mockGetResolvedUserUsageData, - isOrgScopedSubscription: billingSubscriptionUtilsMockFns.mockIsOrgScopedSubscription, -} - -const usage = { - currentUsage: 18.5, - limit: 40, - percentUsed: 46.25, - isWarning: false, - isExceeded: false, - billingPeriodStart: new Date('2026-08-01T00:00:00Z'), - billingPeriodEnd: new Date('2026-09-01T00:00:00Z'), - lastPeriodCost: 31, -} - -describe('getAccountBillingSnapshot', () => { - beforeEach(() => { - mocks.events.length = 0 - }) - - it('preserves personal scope and clamps negative remaining usage to zero', async () => { - mocks.getResolvedUserUsageData.mockResolvedValue({ - usage: { ...usage, currentUsage: 45, isExceeded: true }, - subscription: { plan: 'pro', referenceId: 'user-1' }, - personalCreditBalance: 0, - }) - mocks.isOrgScopedSubscription.mockReturnValue(false) - mocks.getCreditBalanceForEntity.mockResolvedValue(0) - - await expect(getAccountBillingSnapshot('user-1')).resolves.toMatchObject({ - plan: 'pro', - billingScope: 'user', - organizationId: null, - usage: { remaining: 0, isExceeded: true }, - credits: { balance: 0, scope: 'user' }, - }) - expect(mocks.getCreditBalanceForEntity).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/billing/core/account-billing-snapshot.ts b/apps/sim/lib/billing/core/account-billing-snapshot.ts deleted file mode 100644 index a7591357aca..00000000000 --- a/apps/sim/lib/billing/core/account-billing-snapshot.ts +++ /dev/null @@ -1,58 +0,0 @@ -import { db } from '@sim/db' -import { getResolvedUserUsageData } from '@/lib/billing/core/usage' -import { getCreditBalanceForEntity } from '@/lib/billing/credits/balance' -import { isOrgScopedSubscription } from '@/lib/billing/subscriptions/utils' -import type { DbClient } from '@/lib/db/types' - -export interface AccountBillingSnapshot { - plan: string - billingScope: 'user' | 'organization' - organizationId: string | null - usage: { - currentPeriodCost: number - limit: number - remaining: number - percentUsed: number - isExceeded: boolean - billingPeriodEnd: Date | null - } - credits: { - balance: number - scope: 'user' | 'organization' - } -} - -/** Resolves one coherent subscription, usage, limit, and credit snapshot for an account. */ -export async function getAccountBillingSnapshot( - userId: string, - executor: DbClient = db -): Promise { - const { usage, subscription, personalCreditBalance } = await getResolvedUserUsageData( - userId, - executor - ) - const organizationScoped = isOrgScopedSubscription(subscription, userId) && subscription !== null - const billingScope = organizationScoped ? 'organization' : 'user' - const billingEntityId = organizationScoped ? subscription.referenceId : userId - const creditBalance = organizationScoped - ? await getCreditBalanceForEntity('organization', billingEntityId, executor) - : personalCreditBalance - - return { - plan: subscription?.plan || 'free', - billingScope, - organizationId: organizationScoped ? subscription.referenceId : null, - usage: { - currentPeriodCost: usage.currentUsage, - limit: usage.limit, - remaining: Math.max(0, usage.limit - usage.currentUsage), - percentUsed: usage.percentUsed, - isExceeded: usage.isExceeded, - billingPeriodEnd: usage.billingPeriodEnd, - }, - credits: { - balance: creditBalance, - scope: billingScope, - }, - } -} diff --git a/apps/sim/lib/billing/core/usage.ts b/apps/sim/lib/billing/core/usage.ts index 8052798a855..3bad7ae2c86 100644 --- a/apps/sim/lib/billing/core/usage.ts +++ b/apps/sim/lib/billing/core/usage.ts @@ -220,7 +220,7 @@ export async function ensureUserStatsExists(userId: string): Promise { .onConflictDoNothing({ target: userStats.userId }) } -export interface ResolvedUserUsageData { +interface ResolvedUserUsageData { usage: UsageData subscription: HighestPrioritySubscription /** The personal balance from the same user-stats row used to calculate usage. */ @@ -228,7 +228,7 @@ export interface ResolvedUserUsageData { } /** Resolves comprehensive usage and the subscription that determined its billing scope. */ -export async function getResolvedUserUsageData( +async function getResolvedUserUsageData( userId: string, executor: DbClient = db ): Promise { diff --git a/apps/sim/lib/core/security/csp.ts b/apps/sim/lib/core/security/csp.ts index 6e3b1b5408b..f2d3c558573 100644 --- a/apps/sim/lib/core/security/csp.ts +++ b/apps/sim/lib/core/security/csp.ts @@ -72,7 +72,7 @@ function getS3EndpointSources( return [origin, `${url.protocol}//*.${url.host}`] } -export interface CSPDirectives { +interface CSPDirectives { 'default-src'?: string[] 'script-src'?: string[] 'style-src'?: string[] @@ -208,7 +208,7 @@ const STATIC_FRAME_SRC = [ ] as const // Build-time CSP directives (for next.config.ts) -export const buildTimeCSPDirectives: CSPDirectives = { +const buildTimeCSPDirectives: CSPDirectives = { 'default-src': ["'self'"], 'script-src': [...STATIC_SCRIPT_SRC], 'style-src': ["'self'", "'unsafe-inline'", 'https://fonts.googleapis.com'], diff --git a/apps/sim/lib/credential-groups/application/context.ts b/apps/sim/lib/credential-groups/application/context.ts index a5f8a97a262..ef23a367375 100644 --- a/apps/sim/lib/credential-groups/application/context.ts +++ b/apps/sim/lib/credential-groups/application/context.ts @@ -1,31 +1,10 @@ import { getWorkspaceOwnerSubscriptionAccess } from '@/lib/billing/core/workspace-access' import { OrchestrationError } from '@/lib/core/orchestration/types' import type { CredentialGroupApplicationContext } from '@/lib/credential-groups/application/authorization' -import { - isCredentialGroupsAvailable, - resolveCredentialGroupsAvailability, -} from '@/lib/credential-groups/availability' -import { - loadCredentialGroupCredentialListContext, - loadWorkspaceAccountsCredentialListContext, -} from '@/lib/credential-groups/credentials' +import { isCredentialGroupsAvailable } from '@/lib/credential-groups/availability' +import { loadCredentialGroupCredentialListContext } from '@/lib/credential-groups/credentials' import { loadActiveWorkspaceApplicationContext } from '@/lib/workspaces/application/workspace-context' -export async function requireCredentialGroupsAvailable(workspaceId: string): Promise { - const ownerBilling = await getWorkspaceOwnerSubscriptionAccess(workspaceId) - const availability = await resolveCredentialGroupsAvailability({ - organizationId: ownerBilling.organizationId, - ownerBilling, - }) - if (!availability.available) { - const message = - availability.reason === 'enterprise_plan_required' - ? 'Credential Groups are not available. Enterprise plan required.' - : 'Credential Groups are not available' - throw new OrchestrationError('forbidden', message) - } -} - export async function requireCredentialGroupSettingsAvailable(workspaceId: string): Promise { const ownerBilling = await getWorkspaceOwnerSubscriptionAccess(workspaceId) if ( @@ -52,15 +31,6 @@ export async function resolveCredentialGroupContext( return { ...(await resolveCredentialGroupWorkspaceContext(group.workspaceId)), ...group } } -export async function resolveWorkspaceAccountsContext( - workspaceId: string -): Promise { - const workspace = await resolveCredentialGroupWorkspaceContext(workspaceId) - const group = await loadWorkspaceAccountsCredentialListContext(workspaceId) - if (!group) throw new OrchestrationError('not_found', 'Connected accounts are not configured') - return { ...workspace, ...group } -} - export async function resolveCredentialGroupSettingsContext( credentialGroupId: string, assertedWorkspaceId: string diff --git a/apps/sim/lib/credential-groups/credentials.ts b/apps/sim/lib/credential-groups/credentials.ts index fb7d0d4af20..bbe6a0762ff 100644 --- a/apps/sim/lib/credential-groups/credentials.ts +++ b/apps/sim/lib/credential-groups/credentials.ts @@ -191,13 +191,6 @@ export async function loadCredentialGroupCredentialListContext( return loadCredentialGroupContext(eq(credentialGroup.id, credentialGroupId)) } -/** Loads the workspace's single accounts container without decrypting provider settings. */ -export async function loadWorkspaceAccountsCredentialListContext( - workspaceId: string -): Promise { - return loadCredentialGroupContext(eq(credentialGroup.workspaceId, workspaceId)) -} - async function loadCredentialGroupContext( scope: SQL ): Promise { diff --git a/apps/sim/lib/credential-groups/providers.ts b/apps/sim/lib/credential-groups/providers.ts index 483818ab2f1..e7d0fb36cdd 100644 --- a/apps/sim/lib/credential-groups/providers.ts +++ b/apps/sim/lib/credential-groups/providers.ts @@ -48,7 +48,7 @@ export const CREDENTIAL_GROUP_PROVIDER_IDS = [ export type CredentialGroupProvider = (typeof CREDENTIAL_GROUP_PROVIDER_IDS)[number] -export interface CredentialGroupProviderSupport { +interface CredentialGroupProviderSupport { serviceId: string description: string configuration: 'oauth' | 'slack_custom_bot' diff --git a/apps/sim/lib/custom-tools/application/operations.ts b/apps/sim/lib/custom-tools/application/operations.ts index 374fde84a86..d00ab6124bc 100644 --- a/apps/sim/lib/custom-tools/application/operations.ts +++ b/apps/sim/lib/custom-tools/application/operations.ts @@ -62,14 +62,6 @@ export const customToolOperations = { capability: 'custom_tools.use', ...ALL_PRINCIPAL_POLICY, }), - save: defineWorkspaceOperation({ - id: 'custom_tools.save', - oauthScope: 'api:write', - minimumRole: 'write', - workspaceApiKey: 'allow', - capability: 'custom_tools.use', - ...ALL_PRINCIPAL_POLICY, - }), update: defineWorkspaceOperation({ id: 'custom_tools.update', oauthScope: 'api:write', @@ -78,14 +70,6 @@ export const customToolOperations = { capability: 'custom_tools.use', ...ALL_PRINCIPAL_POLICY, }), - updateAvailable: defineWorkspaceOperation({ - id: 'custom_tools.update_available', - oauthScope: 'api:write', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'custom_tools.use', - ...HUMAN_PRINCIPAL_POLICY, - }), delete: defineWorkspaceOperation({ id: 'custom_tools.delete', oauthScope: 'api:write', @@ -94,14 +78,6 @@ export const customToolOperations = { capability: 'custom_tools.use', ...ALL_PRINCIPAL_POLICY, }), - deleteAvailable: defineWorkspaceOperation({ - id: 'custom_tools.delete_available', - oauthScope: 'api:write', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'custom_tools.use', - ...HUMAN_PRINCIPAL_POLICY, - }), } as const export type CustomToolOperation = (typeof customToolOperations)[keyof typeof customToolOperations] diff --git a/apps/sim/lib/folders/scope.ts b/apps/sim/lib/folders/scope.ts index db4a5aa1780..049ce9212a2 100644 --- a/apps/sim/lib/folders/scope.ts +++ b/apps/sim/lib/folders/scope.ts @@ -39,8 +39,8 @@ export function isWithinFolderScope( * * `isWithinFolderScope` answers "is this folder inside that scope" one folder * at a time. This is the same question asked of many items at once, - * after the paths have been walked to ids: a listing filters against it in - * memory, and a query pushes it down into SQL. + * after the paths have been walked to ids, so a query can push it down into + * SQL. * * The root is carried as its own flag rather than as an entry in `folderIds`, * because an item at the root has no folder id to match. A sentinel string diff --git a/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts b/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts index e7f4b4afc69..f2e85e1983d 100644 --- a/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts +++ b/apps/sim/lib/knowledge/__integration__/execution-archive-provenance.integration.ts @@ -3,7 +3,6 @@ import { mkdtempSync } from 'node:fs' import { rm } from 'node:fs/promises' import { tmpdir } from 'node:os' import path from 'node:path' -import type { DelegatedPrincipal } from '@sim/auth/principal' import { db } from '@sim/db' import { document, @@ -12,8 +11,6 @@ import { organization, outboxEvent, user, - userTableRowSecretProvenance, - userTableRows, workspace, workspaceFiles, } from '@sim/db/schema' @@ -57,7 +54,6 @@ import { KNOWLEDGE_DOCUMENT_PROCESSING_OUTBOX_EVENT } from '@/lib/knowledge/docu import { knowledgeDocumentProcessingOutboxHandlers } from '@/lib/knowledge/documents/processing-outbox-handler' import { createSingleDocument } from '@/lib/knowledge/documents/service' import { loadKnowledgeDocumentSecretRegistry } from '@/lib/knowledge/secret-provenance' -import { createTableFromWorkspaceFile } from '@/lib/table/application/workspace-file-imports' import { uploadExecutionFile } from '@/lib/uploads/contexts/execution/execution-file-manager' import { deleteWorkspaceFile, @@ -94,20 +90,6 @@ function sessionPrincipal(ids: Fixture) { return { kind: 'session', userId: ids.aliceId, sessionId: 'fixture-session' } as const } -function tablePrincipal(ids: Fixture): DelegatedPrincipal { - const issuedAt = new Date() - return { - kind: 'delegated', - serviceId: 'copilot', - subjectUserId: ids.aliceId, - workspaceId: ids.workspaceId, - delegationId: generateId(), - audience: 'sim:tables', - issuedAt, - expiresAt: new Date(issuedAt.getTime() + 5 * 60_000), - } -} - async function uploadArchive( ids: Fixture, provenance?: WorkspaceFileSecretProvenance, @@ -180,12 +162,6 @@ async function assertBlockedConsumers(ids: Fixture, source: Awaited { @@ -206,7 +182,7 @@ afterAll(async () => { }) describe('execution archive durable provenance', () => { - it('carries exact-empty lineage through extraction, table rows, and delayed KB indexing/search', async () => { + it('carries exact-empty lineage through extraction and delayed KB indexing/search', async () => { const ids = await seed() const archive = await uploadArchive(ids, { status: 'exact', entries: [] }) const [storedArchive] = await db @@ -228,44 +204,6 @@ describe('execution archive durable provenance', () => { REPORT_CSV ) - const table = await createTableFromWorkspaceFile.execute({ - principal: tablePrincipal(ids), - input: { workspaceId: ids.workspaceId, fileReference: source.child.id }, - }) - expect(table.kind).toBe('inline') - if (table.kind !== 'inline') throw new Error('Small CSV did not use the inline import path') - expect(table.insertedCount).toBe(1) - const rows = await db - .select({ - data: userTableRows.data, - updatedAt: userTableRows.updatedAt, - version: userTableRows.secretProvenanceVersion, - contentUpdatedAt: userTableRowSecretProvenance.contentUpdatedAt, - status: userTableRowSecretProvenance.status, - entries: userTableRowSecretProvenance.entries, - }) - .from(userTableRows) - .leftJoin( - userTableRowSecretProvenance, - eq(userTableRowSecretProvenance.rowId, userTableRows.id) - ) - .where(eq(userTableRows.tableId, table.table.id)) - expect(rows).toHaveLength(1) - const nameColumn = table.table.schema.columns.find((column) => column.name === 'name') - const descriptionColumn = table.table.schema.columns.find( - (column) => column.name === 'description' - ) - if (!nameColumn?.id || !descriptionColumn?.id) { - throw new Error('Imported table lost its canonical source columns') - } - expect(rows[0]).toMatchObject({ - data: { [nameColumn.id]: 'Orion', [descriptionColumn.id]: REPORT_TEXT }, - version: 1, - status: 'exact', - entries: [], - }) - expect(rows[0].contentUpdatedAt).toEqual(rows[0].updatedAt) - const imported = await addWorkspaceFilesToKnowledgeBase.execute({ principal: sessionPrincipal(ids), input: { knowledgeBaseId: ids.knowledgeBaseId, fileReferences: [source.child.id] }, @@ -309,7 +247,7 @@ describe('execution archive durable provenance', () => { expect(search.results.map((entry) => entry.documentId)).toContain(documentId) }) - it('keeps an explicitly unknown execution source unavailable to model, KB, and table consumers', async () => { + it('keeps an explicitly unknown execution source unavailable to model and KB consumers', async () => { const ids = await seed() const archive = await uploadArchive(ids, { status: 'unknown' }) const source = await extract(ids, archive) @@ -360,11 +298,6 @@ describe('execution archive durable provenance', () => { expect(storedArchive.secretProvenanceVersion).toBeNull() const source = await extract(ids, archive) expect(await isOpaqueWorkspaceFileEgressSafe(ids.workspaceId, source.identity)).toBe(true) - const imported = await createTableFromWorkspaceFile.execute({ - principal: tablePrincipal(ids), - input: { workspaceId: ids.workspaceId, fileReference: source.child.id }, - }) - expect(imported.kind).toBe('inline') }) it.each([false, true])( diff --git a/apps/sim/lib/mothership/application/table-commands.ts b/apps/sim/lib/mothership/application/table-commands.ts index 0c31bf0aa2f..fb552edf439 100644 --- a/apps/sim/lib/mothership/application/table-commands.ts +++ b/apps/sim/lib/mothership/application/table-commands.ts @@ -5,21 +5,10 @@ import { replaceProjectedWireRows, } from '@/lib/table/application/rows' -const INHERITED_COPILOT_RATE_POLICY = { - kind: 'inherited_copilot_request', - reason: 'The authenticated Copilot request owns request-rate admission.', -} as const - -const NO_DIRECT_PROVIDER_COST_POLICY = { - kind: 'none', - reason: 'This command does not invoke a paid provider; table quota and storage limits apply.', -} as const - -export const copilotReplaceProjectedWireRowsPolicy = { - rate: INHERITED_COPILOT_RATE_POLICY, - cost: NO_DIRECT_PROVIDER_COST_POLICY, -} as const - +/** + * Request-rate admission is inherited from the authenticated Copilot request, and + * no paid provider is invoked, so only table quota and storage limits apply. + */ export function executeCopilotReplaceProjectedWireRows( context: CopilotTableDelegationContext | undefined, input: ReplaceProjectedWireRowsInput diff --git a/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts b/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts index 9136c27e3fa..658e3900d07 100644 --- a/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts +++ b/apps/sim/lib/mothership/tools/handlers/workflow/mutations.test.ts @@ -30,10 +30,6 @@ vi.mock('@/lib/mothership/application/execute-credential-use-case', () => ({ executeCopilotCredentialUseCase: mocks.credential, })) -vi.mock('@/lib/workflows/sanitization/json-sanitizer', () => ({ - sanitizeForCopilot: vi.fn((state) => state), -})) - /** * The use cases these handlers dispatch are only passed through to the mocked * use-case executor above, so their execution-side leaves — the workflow diff --git a/apps/sim/lib/mothership/tools/sandbox-resource-transport.test.ts b/apps/sim/lib/mothership/tools/sandbox-resource-transport.test.ts index 2f0659341dc..39ea3ef05b8 100644 --- a/apps/sim/lib/mothership/tools/sandbox-resource-transport.test.ts +++ b/apps/sim/lib/mothership/tools/sandbox-resource-transport.test.ts @@ -24,7 +24,6 @@ vi.mock('@/lib/mothership/tools/sandbox-resources', () => ({ recordSandboxResourceEffects: recordEffects, })) -import { isInternalRequest } from '@/lib/api/server/routes/internal-request' import { proxySandboxResourceRequest } from '@/lib/mothership/tools/sandbox-resource-transport' const target = mothershipWorkspaceTargetMockFns.mockResolveInvocationWorkspace @@ -72,7 +71,6 @@ describe('private sandbox v2 resource transport', () => { fetcher.mockImplementation(async (input: Request) => { expect(input.url).toBe('http://internal-sim/api/v2/tables/table/rows?workspaceId=workspace') expect(input.method).toBe('POST') - expect(isInternalRequest(input)).toBe(false) expect(input.redirect).toBe('manual') expect(input.headers.get('x-api-key')).toBeNull() expect(isCopilotRequest(input)).toBe(true) @@ -280,7 +278,6 @@ it.each([ load: async () => ({ GET: fetcher }), }) fetcher.mockImplementation(async (input: Request) => { - expect(isInternalRequest(input)).toBe(false) await reportWorkspaceFileDelivery({ ...provenance, ...('entries' in provenance ? { entries: [...provenance.entries] } : {}), diff --git a/apps/sim/lib/table/application/copilot-table-lifecycle.test.ts b/apps/sim/lib/table/application/copilot-table-lifecycle.test.ts deleted file mode 100644 index 0e218c31024..00000000000 --- a/apps/sim/lib/table/application/copilot-table-lifecycle.test.ts +++ /dev/null @@ -1,143 +0,0 @@ -import { createDelegatedPrincipal } from '@sim/testing/factories/principal.factory' -import { auditMock, auditMockFns } from '@sim/testing/mocks/audit.mock' -import { requestUtilsMockFns } from '@sim/testing/mocks/request.mock' -import { tableMock, tableMockFns } from '@sim/testing/mocks/table.mock' -import { - tableApplicationContextMock, - tableApplicationContextMockFns, -} from '@sim/testing/mocks/table-application-context.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' -import { OrchestrationError } from '@/lib/core/orchestration/types' - -vi.mock('@sim/audit', () => auditMock) -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) -vi.mock('@/lib/table', () => tableMock) -vi.mock('@/lib/table/application/context', () => tableApplicationContextMock) - -import { deleteCopilotTables } from '@/lib/table/application/copilot-table-lifecycle' - -const mocks = { - deleteTable: tableMockFns.mockDeleteTable, - resolveActiveTableContext: tableApplicationContextMockFns.mockResolveActiveTableContext, - resolveWorkspaceContext: tableApplicationContextMockFns.mockResolveTableWorkspaceContext, - audit: auditMockFns.mockRecordAudit, - resolvePermission: workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission, -} - -const principal = createDelegatedPrincipal({ - delegationId: 'copilot-tool:tool-1', - audience: 'sim:tables', - resourceScope: { chatId: 'chat-1' }, -}) - -describe('deleteCopilotTables', () => { - beforeEach(() => { - requestUtilsMockFns.mockGenerateRequestId.mockReturnValue('request-1') - mocks.resolvePermission.mockResolvedValue('write') - mocks.resolveWorkspaceContext.mockResolvedValue({ - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mocks.resolveActiveTableContext.mockImplementation( - async ({ tableId }: { tableId: string }) => ({ - tableId, - workspaceId: 'workspace-1', - }) - ) - mocks.deleteTable.mockImplementation(async (tableId: string) => ({ - archived: { name: `Table ${tableId}`, workspaceId: 'workspace-1' }, - })) - }) - - it('conceals a cross-workspace table as a best-effort miss', async () => { - mocks.resolveActiveTableContext.mockRejectedValueOnce( - new OrchestrationError('not_found', 'Table not found') - ) - - await expect( - deleteCopilotTables.execute({ - principal, - input: { - workspaceId: 'workspace-1', - tableIds: ['table-other'], - assertNotAborted: vi.fn(), - }, - }) - ).resolves.toEqual({ deleted: [], failed: ['table-other'] }) - - expect(mocks.deleteTable).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - }) - - it('rejects admission before canonical loads or mutation', async () => { - mocks.resolvePermission.mockResolvedValueOnce(null) - - await expect( - deleteCopilotTables.execute({ - principal, - input: { - workspaceId: 'workspace-1', - tableIds: ['table-1'], - assertNotAborted: vi.fn(), - }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.resolveActiveTableContext).not.toHaveBeenCalled() - expect(mocks.deleteTable).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - }) - - it('does not project partial audit when the compound command fails', async () => { - const failure = new Error('delete storage unavailable') - mocks.deleteTable - .mockResolvedValueOnce({ - archived: { name: 'Table table-1', workspaceId: 'workspace-1' }, - }) - .mockRejectedValueOnce(failure) - - await expect( - deleteCopilotTables.execute({ - principal, - input: { - workspaceId: 'workspace-1', - tableIds: ['table-1', 'table-2'], - assertNotAborted: vi.fn(), - }, - }) - ).rejects.toBe(failure) - - expect(mocks.audit).not.toHaveBeenCalled() - }) - - it('checks cancellation immediately before each archive and stops partial progress', async () => { - const canceled = new Error('Request aborted before tool mutation could be applied') - const assertNotAborted = vi - .fn() - .mockImplementationOnce(() => undefined) - .mockImplementationOnce(() => { - throw canceled - }) - - await expect( - deleteCopilotTables.execute({ - principal, - input: { - workspaceId: 'workspace-1', - tableIds: ['table-1', 'table-2'], - assertNotAborted, - }, - }) - ).rejects.toBe(canceled) - - expect(mocks.resolveActiveTableContext).toHaveBeenCalledTimes(2) - expect(mocks.deleteTable).toHaveBeenCalledTimes(1) - expect(mocks.deleteTable).toHaveBeenCalledWith('table-1', 'request-1', { - expectedWorkspaceId: 'workspace-1', - }) - expect(mocks.audit).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/table/application/copilot-table-lifecycle.ts b/apps/sim/lib/table/application/copilot-table-lifecycle.ts deleted file mode 100644 index 57369da7510..00000000000 --- a/apps/sim/lib/table/application/copilot-table-lifecycle.ts +++ /dev/null @@ -1,85 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { asOrchestrationError, OrchestrationError } from '@/lib/core/orchestration/types' -import { generateRequestId } from '@/lib/core/utils/request' -import { deleteTable, TABLE_LIMITS } from '@/lib/table' -import { defineAuthorizedTableUseCase } from '@/lib/table/application/authorized-table-use-case' -import { - resolveActiveTableContext, - resolveTableWorkspaceContext, -} from '@/lib/table/application/context' -import { tableOperations } from '@/lib/table/application/operations' - -export interface DeleteCopilotTablesInput { - workspaceId: string - tableIds: string[] - assertNotAborted: () => void -} - -export interface ArchivedCopilotTable { - id: string - name: string -} - -export interface DeleteCopilotTablesResult { - deleted: ArchivedCopilotTable[] - failed: string[] -} - -/** Owns Copilot's ordered, best-effort multi-table archive operation. */ -export const deleteCopilotTables = defineAuthorizedTableUseCase({ - operation: tableOperations.delete, - resolveContext: ({ input }: { input: DeleteCopilotTablesInput }) => - resolveTableWorkspaceContext(input.workspaceId), - async execute({ input, context }): Promise { - if ( - input.tableIds.length < 1 || - input.tableIds.length > TABLE_LIMITS.MAX_TABLES_PER_WORKSPACE - ) { - throw new OrchestrationError( - 'validation', - `Table ID count must be between 1 and ${TABLE_LIMITS.MAX_TABLES_PER_WORKSPACE}` - ) - } - if (input.tableIds.some((tableId) => typeof tableId !== 'string' || !tableId.trim())) { - throw new OrchestrationError('validation', 'Each table ID must be a non-empty string') - } - - const deleted: ArchivedCopilotTable[] = [] - const failed: string[] = [] - - for (const tableId of input.tableIds) { - try { - const tableContext = await resolveActiveTableContext({ - tableId, - assertedWorkspaceId: context.workspaceId, - }) - input.assertNotAborted() - const { archived } = await deleteTable(tableContext.tableId, generateRequestId(), { - expectedWorkspaceId: context.workspaceId, - }) - if (!archived) { - failed.push(tableId) - continue - } - - deleted.push({ id: tableId, name: archived.name }) - } catch (error) { - if (asOrchestrationError(error)?.code === 'not_found') { - failed.push(tableId) - continue - } - throw error - } - } - - return { deleted, failed } - }, - projectAudit: ({ result }) => - result.deleted.map((table) => ({ - action: AuditAction.TABLE_DELETED, - resourceType: AuditResourceType.TABLE, - resourceId: table.id, - resourceName: table.name, - description: `Archived table "${table.name}"`, - })), -}) diff --git a/apps/sim/lib/table/application/groups.test.ts b/apps/sim/lib/table/application/groups.test.ts index 12707306a3c..1d6034b932d 100644 --- a/apps/sim/lib/table/application/groups.test.ts +++ b/apps/sim/lib/table/application/groups.test.ts @@ -18,12 +18,10 @@ import { } from '@sim/testing/mocks/workflow-context.mock' import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' import { beforeEach, describe, expect, it, vi } from 'vitest' -import { OrchestrationError } from '@/lib/core/orchestration/types' import type { TableDefinition, WorkflowGroup } from '@/lib/table/types' const hoisted = vi.hoisted(() => ({ addGroup: vi.fn(), - addOutput: vi.fn(), getEnrichment: vi.fn(), loadWorkflowOutputs: vi.fn(), updateGroup: vi.fn(), @@ -38,17 +36,11 @@ vi.mock('@/lib/table/application/context', () => tableApplicationContextMock) vi.mock('@/lib/table/column-naming', () => ({ columnTypeForLeaf: (leafType: string | undefined) => leafType === 'number' ? 'number' : 'string', - deriveOutputColumnName: (path: string, taken: Set) => { - const base = path.replace(/[^a-zA-Z0-9_]/g, '_').toLowerCase() - if (!taken.has(base)) return base - return `${base}_0` - }, })) vi.mock('@/lib/table/events', () => tableEventsMock) vi.mock('@/lib/table/workflow-columns', () => tableWorkflowColumnsMock) vi.mock('@/lib/table/workflow-groups/service', () => ({ addWorkflowGroup: hoisted.addGroup, - addWorkflowGroupOutput: hoisted.addOutput, deleteWorkflowGroup: vi.fn(), updateWorkflowGroup: hoisted.updateGroup, })) @@ -57,13 +49,7 @@ vi.mock('@/lib/workflows/application/resolve-workflow-outputs', () => ({ loadResolvedDeployedWorkflowOutputs: hoisted.loadWorkflowOutputs, })) -import { - addWorkflowTableGroupOutput, - createTableEnrichmentGroup, - createTableGroupUseCase, - createWorkflowTableGroup, - updateTableGroupUseCase, -} from '@/lib/table/application/groups' +import { createTableGroupUseCase, updateTableGroupUseCase } from '@/lib/table/application/groups' const mocks = { ...hoisted, @@ -219,7 +205,6 @@ describe('workflow and enrichment Table application commands', () => { mocks.addGroup.mockImplementation(async ({ group: nextGroup, outputColumns }) => tableWithGroup(nextGroup, [...table.schema.columns, ...outputColumns]) ) - mocks.addOutput.mockResolvedValue(table) mocks.updateGroup.mockImplementation(async (input) => tableWithGroup({ ...group, @@ -237,55 +222,6 @@ describe('workflow and enrichment Table application commands', () => { }) }) - /** - * Adding an output backfills it from saved runs, and a backfilled cell can - * satisfy a downstream group's deps and start it. That cascade is gated on - * the acting person, which is not the billing attribution beside it. - */ - it('names the acting person, not the billing actor, as the backfill cascade subject', async () => { - await addWorkflowTableGroupOutput.execute({ - principal, - input: { - tableId: table.id, - workspaceId: table.workspaceId, - groupId: group.id, - blockId: 'block-2', - path: 'score', - }, - }) - - expect(mocks.addOutput).toHaveBeenCalledWith( - expect.objectContaining({ capabilityGovernedUserId: 'user-1' }), - 'request-1' - ) - }) - - it('conceals a cross-workspace workflow before group mutation or effects', async () => { - mocks.resolveWorkflowContext.mockRejectedValueOnce( - new OrchestrationError('not_found', 'Workflow not found') - ) - - await expect( - createWorkflowTableGroup.execute({ - principal, - input: { - tableId: table.id, - workspaceId: table.workspaceId, - workflowId: 'workflow-other', - outputs: [{ blockId: 'block-2', path: 'score' }], - }, - }) - ).rejects.toMatchObject({ code: 'not_found' }) - - expect(mocks.resolveWorkflowContext).toHaveBeenCalledWith({ - workflowId: 'workflow-other', - assertedWorkspaceId: table.workspaceId, - }) - expect(mocks.addGroup).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - expect(mocks.signal).not.toHaveBeenCalled() - }) - it('refuses an output whose column is neither declared nor existing', async () => { await expect( createTableGroupUseCase.execute({ @@ -465,103 +401,4 @@ describe('workflow and enrichment Table application commands', () => { expect(mocks.runDetached).not.toHaveBeenCalled() expect(mocks.runWorkflowColumn).not.toHaveBeenCalled() }) - - it('rejects oversized workflow output construction before resolution or mutation', async () => { - await expect( - createWorkflowTableGroup.execute({ - principal, - input: { - tableId: table.id, - workspaceId: table.workspaceId, - workflowId: 'workflow-1', - outputs: Array.from({ length: 1001 }, (_, index) => ({ - blockId: `block-${index}`, - path: 'content', - })), - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - - expect(mocks.resolveWorkflowContext).not.toHaveBeenCalled() - expect(mocks.addGroup).not.toHaveBeenCalled() - }) - - it('rejects adding a workflow output to an enrichment group before resolution or mutation', async () => { - mocks.resolveContext.mockResolvedValueOnce({ - tableId: table.id, - table: tableWithGroup({ - id: 'enrichment-group-1', - type: 'enrichment', - workflowId: '', - enrichmentId: 'company-domain', - outputs: [], - }), - workspaceId: table.workspaceId, - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - - await expect( - addWorkflowTableGroupOutput.execute({ - principal, - input: { - tableId: table.id, - workspaceId: table.workspaceId, - groupId: 'enrichment-group-1', - blockId: 'block-2', - path: 'score', - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - - expect(mocks.resolveWorkflowContext).not.toHaveBeenCalled() - expect(mocks.loadWorkflowOutputs).not.toHaveBeenCalled() - expect(mocks.addOutput).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - expect(mocks.signal).not.toHaveBeenCalled() - }) - - it('validates enrichment mappings before constructing the group', async () => { - await expect( - createTableEnrichmentGroup.execute({ - principal, - input: { - tableId: table.id, - workspaceId: table.workspaceId, - enrichmentId: 'company-domain', - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - expect(mocks.addGroup).not.toHaveBeenCalled() - - const result = await createTableEnrichmentGroup.execute({ - principal, - input: { - tableId: table.id, - workspaceId: table.workspaceId, - enrichmentId: 'company-domain', - inputMappings: [{ inputName: 'company', columnName: 'name' }], - }, - }) - - expect(mocks.addGroup).toHaveBeenCalledWith( - expect.objectContaining({ - group: expect.objectContaining({ - id: 'generated-id', - enrichmentId: 'company-domain', - inputMappings: [{ inputName: 'company', columnName: 'name' }], - dependencies: { columns: ['name'] }, - outputs: [{ blockId: '', path: '', outputId: 'domain', columnName: 'domain' }], - }), - outputColumns: [ - expect.objectContaining({ name: 'domain', workflowGroupId: 'generated-id' }), - ], - }), - 'request-1' - ) - expect(result.group.enrichmentId).toBe('company-domain') - expect(mocks.audit).toHaveBeenCalledTimes(1) - expect(mocks.signal).toHaveBeenCalledWith(table.id) - }) }) diff --git a/apps/sim/lib/table/application/groups.ts b/apps/sim/lib/table/application/groups.ts index 32aabda8e44..69b5847d0f1 100644 --- a/apps/sim/lib/table/application/groups.ts +++ b/apps/sim/lib/table/application/groups.ts @@ -8,29 +8,22 @@ import { asOrchestrationError, OrchestrationError } from '@/lib/core/orchestrati import { runDetached } from '@/lib/core/utils/background' import { generateRequestId } from '@/lib/core/utils/request' import { - type ColumnDefinition, columnMatchesRef, type DeleteWorkflowGroupData, - getColumnId, TABLE_LIMITS, type TableDefinition, type TableSchema, type UpdateWorkflowGroupData, type WorkflowGroup, - type WorkflowGroupDependencies, - type WorkflowGroupDeploymentMode, - type WorkflowGroupInputMapping, - type WorkflowGroupOutput, } from '@/lib/table' import { defineAuthorizedTableUseCase } from '@/lib/table/application/authorized-table-use-case' import { resolveActiveTableContext } from '@/lib/table/application/context' import { tableOperations } from '@/lib/table/application/operations' -import { columnTypeForLeaf, deriveOutputColumnName } from '@/lib/table/column-naming' +import { columnTypeForLeaf } from '@/lib/table/column-naming' import { signalTableSchemaChanged } from '@/lib/table/events' import { runWorkflowColumn } from '@/lib/table/workflow-columns' import { addWorkflowGroup, - addWorkflowGroupOutput, deleteWorkflowGroup, updateWorkflowGroup, } from '@/lib/table/workflow-groups/service' @@ -154,21 +147,6 @@ function requireKnownEnrichmentOutputIds( } } -function workflowOutputColumnType( - requestedType: string | undefined, - resolvedLeafType: string | undefined -): ColumnDefinition['type'] { - if (requestedType === undefined) return columnTypeForLeaf(resolvedLeafType) - const type = columnTypeForLeaf(requestedType) - if (type !== requestedType) { - throw new OrchestrationError( - 'validation', - `Invalid workflow output column type "${requestedType}"` - ) - } - return type -} - function attributedUserId( principal: Parameters[0], billedAccountUserId: string @@ -366,283 +344,6 @@ export const createTableGroupUseCase = defineAuthorizedTableUseCase({ }, }) -export interface CreateWorkflowTableGroupInput extends TableGroupInput { - workflowId: string - outputs: Array<{ - blockId: string - path: string - columnName?: string - columnType?: string - }> - name?: string - dependencies?: WorkflowGroupDependencies - deploymentMode?: WorkflowGroupDeploymentMode - autoRun?: boolean -} - -/** Creates a workflow-backed group from requested workflow output coordinates. */ -export const createWorkflowTableGroup = defineAuthorizedTableUseCase({ - operation: tableOperations.createGroup, - resolveContext: ({ input }: { input: CreateWorkflowTableGroupInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.workspaceId, - }), - async execute({ principal, input, context }) { - requireBoundedGroupItems(input.outputs, 'Workflow group outputs') - if (input.outputs.length === 0) { - throw new OrchestrationError('validation', 'At least one workflow output is required') - } - if (input.outputs.some((output) => !output.blockId || !output.path)) { - throw new OrchestrationError( - 'validation', - 'Each output entry must include both blockId and path' - ) - } - - const resolvedWorkflow = await resolveWorkflowForAuthorizedTableCommand( - input.workflowId, - context.workspaceId - ) - const canonicalOutputs = validateRequestedOutputs( - input.outputs, - resolvedWorkflow, - input.workflowId - ) - const leafTypeByKey = new Map( - canonicalOutputs.map((output) => [`${output.blockId}::${output.path}`, output.leafType]) - ) - const taken = new Set(context.table.schema.columns.map((column) => column.name)) - const groupId = generateId() - const outputs: WorkflowGroupOutput[] = [] - const outputColumns: ColumnDefinition[] = [] - for (const requested of input.outputs) { - const columnName = requested.columnName ?? deriveOutputColumnName(requested.path, taken) - taken.add(columnName) - outputs.push({ - blockId: requested.blockId, - path: requested.path, - columnName, - }) - outputColumns.push({ - name: columnName, - type: workflowOutputColumnType( - requested.columnType, - leafTypeByKey.get(`${requested.blockId}::${requested.path}`) - ), - required: false, - unique: false, - workflowGroupId: groupId, - }) - } - - const group: WorkflowGroup = { - id: groupId, - workflowId: input.workflowId, - ...(input.name ? { name: input.name } : {}), - ...(input.dependencies ? { dependencies: input.dependencies } : {}), - ...(input.deploymentMode ? { deploymentMode: input.deploymentMode } : {}), - autoRun: input.autoRun ?? false, - outputs, - } - const actorUserId = attributedUserId(principal, context.billedAccountUserId) - const capabilityGovernedUserId = capabilityGovernedPrincipalUserId(principal) - const table = await addWorkflowGroup( - { - tableId: context.tableId, - workspaceId: context.workspaceId, - group, - outputColumns, - autoRun: input.autoRun ?? false, - suppressAutoRunDispatch: true, - actorUserId, - capabilityGovernedUserId, - }, - generateRequestId() - ) - return { - table, - group: groupFromTable(table, groupId), - actorUserId, - capabilityGovernedUserId, - } - }, - projectAudit({ result }) { - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Added workflow group "${result.group.id}" to table "${result.table.name}"`, - metadata: { op: 'add_workflow_group', groupId: result.group.id }, - } - }, - afterSuccess({ input, context, result }) { - signalTableSchemaChanged(context.tableId) - if (input.autoRun === true) { - dispatchGroupAutoRun({ - tableId: context.tableId, - workspaceId: context.workspaceId, - groupId: result.group.id, - actorUserId: result.actorUserId, - capabilityGovernedUserId: result.capabilityGovernedUserId, - label: 'table-workflow-group-create-auto-run', - }) - } - }, -}) - -export interface CreateTableEnrichmentGroupInput extends TableGroupInput { - enrichmentId: string - inputMappings?: Array<{ inputName: string; columnName: string }> - outputColumnNames?: Record - dependencies?: WorkflowGroupDependencies - name?: string - autoRun?: boolean -} - -/** Creates an enrichment group from the code-defined enrichment registry. */ -export const createTableEnrichmentGroup = defineAuthorizedTableUseCase({ - operation: tableOperations.createGroup, - resolveContext: ({ input }: { input: CreateTableEnrichmentGroupInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.workspaceId, - }), - async execute({ principal, input, context }) { - requireBoundedGroupItems(input.inputMappings, 'Enrichment input mappings') - if (Object.keys(input.outputColumnNames ?? {}).length > TABLE_LIMITS.MAX_COLUMNS_PER_TABLE) { - throw new OrchestrationError( - 'validation', - `Enrichment output names cannot exceed ${TABLE_LIMITS.MAX_COLUMNS_PER_TABLE} entries` - ) - } - const enrichment = requireEnrichment(input.enrichmentId) - - const enrichmentInputIds = new Set( - enrichment.inputs.map((enrichmentInput) => enrichmentInput.id) - ) - const mappingByInput = new Map() - for (const mapping of input.inputMappings ?? []) { - if (!enrichmentInputIds.has(mapping.inputName)) { - throw new OrchestrationError( - 'validation', - `Enrichment "${enrichment.name}" has no input "${mapping.inputName}"` - ) - } - if (mappingByInput.has(mapping.inputName)) { - throw new OrchestrationError( - 'validation', - `Enrichment input "${mapping.inputName}" cannot be mapped more than once` - ) - } - mappingByInput.set(mapping.inputName, mapping.columnName) - } - requireKnownEnrichmentOutputIds(enrichment, Object.keys(input.outputColumnNames ?? {})) - const existingColumns = new Set(context.table.schema.columns.map((column) => column.name)) - for (const enrichmentInput of enrichment.inputs) { - const mapped = mappingByInput.get(enrichmentInput.id) - if (enrichmentInput.required && !mapped) { - throw new OrchestrationError( - 'validation', - `Enrichment "${enrichment.name}" requires input "${enrichmentInput.id}" to be mapped to a column` - ) - } - if (mapped && !existingColumns.has(mapped)) { - throw new OrchestrationError( - 'validation', - `Mapped column "${mapped}" for input "${enrichmentInput.id}" does not exist on table ${context.tableId}` - ) - } - } - - const inputMappings: WorkflowGroupInputMapping[] = enrichment.inputs - .filter((enrichmentInput) => mappingByInput.has(enrichmentInput.id)) - .map((enrichmentInput) => ({ - inputName: enrichmentInput.id, - columnName: mappingByInput.get(enrichmentInput.id) as string, - })) - const taken = new Set(context.table.schema.columns.map((column) => column.name)) - const groupId = generateId() - const outputs: WorkflowGroupOutput[] = [] - const outputColumns: ColumnDefinition[] = [] - for (const output of enrichment.outputs) { - const desired = (input.outputColumnNames?.[output.id] ?? '').trim() || output.name - const columnName = deriveOutputColumnName(desired, taken) - taken.add(columnName) - outputs.push({ blockId: '', path: '', outputId: output.id, columnName }) - outputColumns.push({ - name: columnName, - type: output.type, - required: false, - unique: false, - workflowGroupId: groupId, - }) - } - - const name = input.name ?? enrichment.name - const group: WorkflowGroup = { - id: groupId, - workflowId: '', - enrichmentId: input.enrichmentId, - name, - type: 'enrichment', - dependencies: input.dependencies ?? { columns: inputMappings.map((item) => item.columnName) }, - outputs, - inputMappings, - autoRun: input.autoRun ?? false, - } - const actorUserId = attributedUserId(principal, context.billedAccountUserId) - const capabilityGovernedUserId = capabilityGovernedPrincipalUserId(principal) - const table = await addWorkflowGroup( - { - tableId: context.tableId, - workspaceId: context.workspaceId, - group, - outputColumns, - autoRun: input.autoRun ?? false, - suppressAutoRunDispatch: true, - actorUserId, - capabilityGovernedUserId, - }, - generateRequestId() - ) - return { - table, - group: groupFromTable(table, groupId), - actorUserId, - capabilityGovernedUserId, - } - }, - projectAudit({ result }) { - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Added enrichment "${result.group.name ?? result.group.id}" to table "${result.table.name}"`, - metadata: { - op: 'add_enrichment', - groupId: result.group.id, - enrichmentId: result.group.enrichmentId, - }, - } - }, - afterSuccess({ input, context, result }) { - signalTableSchemaChanged(context.tableId) - if (input.autoRun === true) { - dispatchGroupAutoRun({ - tableId: context.tableId, - workspaceId: context.workspaceId, - groupId: result.group.id, - actorUserId: result.actorUserId, - capabilityGovernedUserId: result.capabilityGovernedUserId, - label: 'table-enrichment-group-create-auto-run', - }) - } - }, -}) - export interface UpdateTableGroupInput extends TableGroupInput, Omit< @@ -678,10 +379,8 @@ export const updateTableGroupUseCase = defineAuthorizedTableUseCase({ * cannot be talked back out of. * * `manual` → `enrichment` leaves `enrichmentId` undefined, which is the - * exact shape `refineGroupSource` rejects on create, and it bricks the - * group for output editing: `addWorkflowTableGroupOutput` and - * `updateWorkflowTableGroup` both refuse a group whose `type` reads - * `enrichment`. `enrichment` → `manual` is worse — it keeps `enrichmentId` + * exact shape `refineGroupSource` rejects on create. `enrichment` → + * `manual` is worse — it keeps `enrichmentId` * but steers the runner off the enrichment branch and onto the workflow * one, where the group's `workflowId` is `''` and every cell run fails. * @@ -881,199 +580,6 @@ export const updateTableGroupUseCase = defineAuthorizedTableUseCase({ }, }) -export interface UpdateWorkflowTableGroupInput extends TableGroupInput { - groupId: string - workflowId?: string - name?: string - dependencies?: WorkflowGroupDependencies - outputs?: Array<{ - blockId: string - path: string - columnName?: string - columnType?: string - }> - mappingUpdates?: Array<{ columnName: string; blockId: string; path: string }> - deploymentMode?: WorkflowGroupDeploymentMode - autoRun?: boolean -} - -/** Updates a workflow-backed group from output coordinates rather than caller-built columns. */ -export const updateWorkflowTableGroup = defineAuthorizedTableUseCase({ - operation: tableOperations.updateGroup, - resolveContext: ({ input }: { input: UpdateWorkflowTableGroupInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.workspaceId, - }), - async execute({ principal, input, context }) { - requireBoundedGroupItems(input.outputs, 'Workflow group outputs') - requireBoundedGroupItems(input.mappingUpdates, 'Workflow group mapping updates') - const previousGroup = context.table.schema.workflowGroups?.find( - (candidate) => candidate.id === input.groupId - ) - if (!previousGroup) { - throw new OrchestrationError('not_found', `Workflow group "${input.groupId}" not found`) - } - if (previousGroup.type === 'enrichment' || !previousGroup.workflowId) { - throw new OrchestrationError( - 'validation', - `Workflow group "${input.groupId}" is not backed by a workflow` - ) - } - - const targetWorkflowId = input.workflowId ?? previousGroup.workflowId - const workflowMetadataRequired = - input.workflowId !== undefined || - input.outputs !== undefined || - (input.mappingUpdates?.length ?? 0) > 0 - const resolvedWorkflow = workflowMetadataRequired - ? await resolveWorkflowForAuthorizedTableCommand(targetWorkflowId, context.workspaceId) - : undefined - if (input.outputs && resolvedWorkflow) { - validateRequestedOutputs(input.outputs, resolvedWorkflow, targetWorkflowId) - } else if (input.workflowId && resolvedWorkflow) { - validateRequestedOutputs(previousGroup.outputs, resolvedWorkflow, targetWorkflowId) - } - - let outputs: WorkflowGroupOutput[] | undefined - let newOutputColumns: ColumnDefinition[] | undefined - if (input.outputs) { - if (!resolvedWorkflow) { - throw new Error('Workflow metadata is required to restructure workflow outputs') - } - const canonicalOutputs = requireWorkflowOutputs(resolvedWorkflow, targetWorkflowId) - const leafTypeByKey = new Map( - canonicalOutputs.map((output) => [`${output.blockId}::${output.path}`, output.leafType]) - ) - const existingByKey = new Map( - previousGroup.outputs.map((output) => [`${output.blockId}::${output.path}`, output]) - ) - const requestedKeys = new Set( - input.outputs.map((output) => `${output.blockId}::${output.path}`) - ) - const releasedColumnIds = new Set( - previousGroup.outputs - .filter((output) => !requestedKeys.has(`${output.blockId}::${output.path}`)) - .map((output) => output.columnName) - ) - const taken = new Set( - context.table.schema.columns - .filter((column) => !releasedColumnIds.has(getColumnId(column))) - .map((column) => column.name) - ) - outputs = [] - newOutputColumns = [] - for (const requested of input.outputs) { - const key = `${requested.blockId}::${requested.path}` - const existing = existingByKey.get(key) - if (existing) { - outputs.push(existing) - continue - } - const requestedName = requested.columnName?.trim() - const columnName = requestedName || deriveOutputColumnName(requested.path, taken) - if (taken.has(columnName)) { - throw new OrchestrationError('validation', `Column "${columnName}" already exists`) - } - taken.add(columnName) - outputs.push({ - blockId: requested.blockId, - path: requested.path, - columnName, - }) - newOutputColumns.push({ - name: columnName, - type: workflowOutputColumnType(requested.columnType, leafTypeByKey.get(key)), - required: false, - unique: false, - workflowGroupId: input.groupId, - }) - } - } - - const resolvedMappingTypes = - input.mappingUpdates && input.mappingUpdates.length > 0 && resolvedWorkflow - ? { - workflowId: resolvedWorkflow.workflowId, - columns: input.mappingUpdates.map((mapping) => { - const output = resolvedWorkflow.outputs?.find( - (candidate) => - candidate.blockId === mapping.blockId && candidate.path === mapping.path - ) - if (!output) { - throw new OrchestrationError( - 'validation', - `Output ${mapping.blockId}::${mapping.path} is not a valid pickable output on workflow ${targetWorkflowId}` - ) - } - return { columnName: mapping.columnName, type: columnTypeForLeaf(output.leafType) } - }), - } - : undefined - if (input.mappingUpdates?.length && !resolvedMappingTypes) { - throw new Error('Workflow metadata is required for workflow group mapping updates') - } - - const actorUserId = attributedUserId(principal, context.billedAccountUserId) - const capabilityGovernedUserId = capabilityGovernedPrincipalUserId(principal) - const table = await updateWorkflowGroup( - { - tableId: context.tableId, - workspaceId: context.workspaceId, - groupId: input.groupId, - actorUserId, - capabilityGovernedUserId, - suppressAutoRunDispatch: true, - ...(input.workflowId !== undefined ? { workflowId: input.workflowId } : {}), - ...(input.name !== undefined ? { name: input.name } : {}), - ...(input.dependencies !== undefined ? { dependencies: input.dependencies } : {}), - ...(outputs !== undefined ? { outputs } : {}), - ...(newOutputColumns !== undefined ? { newOutputColumns } : {}), - ...(input.mappingUpdates !== undefined ? { mappingUpdates: input.mappingUpdates } : {}), - ...(resolvedMappingTypes ? { resolvedMappingTypes } : {}), - ...(input.deploymentMode !== undefined ? { deploymentMode: input.deploymentMode } : {}), - ...(input.autoRun !== undefined ? { autoRun: input.autoRun } : {}), - }, - generateRequestId() - ) - const group = groupFromTable(table, input.groupId) - return { - table, - group, - changed: - JSON.stringify(context.table.schema) !== JSON.stringify(table.schema) || - JSON.stringify(context.table.metadata) !== JSON.stringify(table.metadata), - startAutoRun: previousGroup.autoRun === false && input.autoRun === true, - actorUserId, - capabilityGovernedUserId, - } - }, - projectAudit({ result }) { - if (!result.changed) return [] - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Updated workflow group "${result.group.id}" in table "${result.table.name}"`, - metadata: { op: 'update_workflow_group', groupId: result.group.id }, - } - }, - afterSuccess({ context, result }) { - if (result.changed) signalTableSchemaChanged(context.tableId) - if (result.startAutoRun) { - dispatchGroupAutoRun({ - tableId: context.tableId, - workspaceId: context.workspaceId, - groupId: result.group.id, - actorUserId: result.actorUserId, - capabilityGovernedUserId: result.capabilityGovernedUserId, - label: 'table-workflow-group-update-auto-run', - }) - } - }, -}) - export interface DeleteTableGroupInput extends TableGroupInput, Omit {} @@ -1110,94 +616,3 @@ export const deleteTableGroupUseCase = defineAuthorizedTableUseCase({ signalTableSchemaChanged(context.table.id) }, }) - -export interface AddTableGroupOutputInput extends TableGroupInput { - groupId: string - blockId: string - path: string - columnName?: string -} - -export const addWorkflowTableGroupOutput = defineAuthorizedTableUseCase({ - operation: tableOperations.updateGroup, - resolveContext: ({ input }: { input: AddTableGroupOutputInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.workspaceId, - }), - async execute({ principal, input, context }) { - const group = context.table.schema.workflowGroups?.find( - (candidate) => candidate.id === input.groupId - ) - if (!group) - throw new OrchestrationError('not_found', `Workflow group "${input.groupId}" not found`) - if (group.type === 'enrichment' || !group.workflowId) { - throw new OrchestrationError( - 'validation', - `Workflow group "${input.groupId}" is not backed by a workflow` - ) - } - const resolvedWorkflow = await resolveWorkflowForAuthorizedTableCommand( - group.workflowId, - context.workspaceId - ) - const outputs = requireWorkflowOutputs(resolvedWorkflow, group.workflowId) - validateRequestedOutputs( - [...group.outputs, { blockId: input.blockId, path: input.path }], - resolvedWorkflow, - group.workflowId - ) - const output = outputs.find( - (candidate) => candidate.blockId === input.blockId && candidate.path === input.path - ) - if (!output) { - throw new OrchestrationError( - 'validation', - `Output ${input.blockId}::${input.path} is not a valid pickable output on workflow ${group.workflowId}` - ) - } - const table = await addWorkflowGroupOutput( - { - tableId: context.tableId, - workspaceId: context.workspaceId, - groupId: input.groupId, - blockId: input.blockId, - path: input.path, - columnName: input.columnName, - actorUserId: resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId, - capabilityGovernedUserId: capabilityGovernedPrincipalUserId(principal), - resolvedOutput: { - workflowId: resolvedWorkflow.workflowId, - columnType: columnTypeForLeaf(output.leafType), - order: outputs.map((candidate, discoveryIndex) => { - const distance = resolvedWorkflow.executionOrderByBlockId[candidate.blockId] - return { - blockId: candidate.blockId, - path: candidate.path, - executionDistance: - distance === undefined || distance < 0 ? Number.POSITIVE_INFINITY : distance, - discoveryIndex, - } - }), - }, - }, - generateRequestId() - ) - return { table, groupId: input.groupId } - }, - projectAudit({ result }) { - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Added an output to workflow group "${result.groupId}"`, - metadata: { op: 'add_group_output', groupId: result.groupId }, - } - }, - afterSuccess({ context }) { - signalTableSchemaChanged(context.tableId) - }, -}) diff --git a/apps/sim/lib/table/application/operations.ts b/apps/sim/lib/table/application/operations.ts index aadd168947c..103f0e4f31b 100644 --- a/apps/sim/lib/table/application/operations.ts +++ b/apps/sim/lib/table/application/operations.ts @@ -109,20 +109,6 @@ function stagedWriteOperation(id: Id) { }) } -function delegatedWriteOperation( - id: Id, - capability: OperationDeclarableCapability -) { - return defineWorkspaceOperation({ - id, - minimumRole: 'write', - workspaceApiKey: 'deny', - capability, - principalKinds: ['delegated'], - delegatedServices: ['copilot'], - }) -} - export const tableOperations = { list: toolReadOperation('tables.list'), read: toolReadOperation('tables.read'), @@ -179,11 +165,6 @@ export const tableOperations = { readRun: readOperation('tables.runs.read'), cancelRuns: writeOperation('tables.runs.cancel'), createImport: stagedWriteOperation('tables.imports.create'), - createFromWorkspaceFile: delegatedWriteOperation( - 'tables.imports.create_from_workspace_file', - 'tables.create' - ), - importWorkspaceFile: delegatedWriteOperation('tables.imports.workspace_file', 'tables.use'), readImport: stagedReadOperation('tables.imports.read', 'tables.use', 'api:read'), createImportParts: stagedWriteOperation('tables.imports.create_parts'), completeImport: stagedWriteOperation('tables.imports.complete'), diff --git a/apps/sim/lib/table/application/workspace-file-imports.test.ts b/apps/sim/lib/table/application/workspace-file-imports.test.ts deleted file mode 100644 index 50ef0d2fd7a..00000000000 --- a/apps/sim/lib/table/application/workspace-file-imports.test.ts +++ /dev/null @@ -1,369 +0,0 @@ -import { - createDelegatedPrincipal, - createWorkspaceApiKeyPrincipal, -} from '@sim/testing/factories/principal.factory' -import { auditMock, auditMockFns } from '@sim/testing/mocks/audit.mock' -import { backgroundTaskMock, backgroundTaskMockFns } from '@sim/testing/mocks/background-task.mock' -import { idMock, idMockFns } from '@sim/testing/mocks/id.mock' -import { tableMock, tableMockFns } from '@sim/testing/mocks/table.mock' -import { - tableApplicationContextMock, - tableApplicationContextMockFns, -} from '@sim/testing/mocks/table-application-context.mock' -import { tableEventsMock, tableEventsMockFns } from '@sim/testing/mocks/table-events.mock' -import { - tableJobsServiceMock, - tableJobsServiceMockFns, -} from '@sim/testing/mocks/table-jobs-service.mock' -import { - tableRowsSecretProvenanceMock, - tableRowsSecretProvenanceMockFns, -} from '@sim/testing/mocks/table-rows-secret-provenance.mock' -import { tableServiceMock, tableServiceMockFns } from '@sim/testing/mocks/table-service.mock' -import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' -import { - workspaceFileManagerMock, - workspaceFileManagerMockFns, -} from '@sim/testing/mocks/workspace-file-manager.mock' -import { - workspaceFileSecretProvenanceMock, - workspaceFileSecretProvenanceMockFns, -} from '@sim/testing/mocks/workspace-file-secret-provenance.mock' -import { beforeEach, describe, expect, it, vi } from 'vitest' -import type { TableDefinition } from '@/lib/table/types' - -vi.mock('@sim/audit', () => auditMock) -vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) -vi.mock('@sim/utils/id', () => idMock) -vi.mock('@/lib/core/utils/background', () => backgroundTaskMock) -vi.mock('@/lib/table', () => ({ ...tableMock, CSV_MAX_BATCH_SIZE: 1000 })) -vi.mock('@/lib/table/application/context', () => tableApplicationContextMock) -vi.mock('@/lib/table/events', () => tableEventsMock) -vi.mock('@/lib/table/import-runner', () => ({ runTableImport: vi.fn() })) -vi.mock('@/lib/table/jobs/service', () => tableJobsServiceMock) -vi.mock('@/lib/table/rows/secret-provenance', () => tableRowsSecretProvenanceMock) -vi.mock('@/lib/table/service', () => tableServiceMock) -vi.mock('@/lib/uploads/contexts/workspace/workspace-file-manager', () => workspaceFileManagerMock) -vi.mock( - '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance', - () => workspaceFileSecretProvenanceMock -) - -import { - createTableFromWorkspaceFile, - importWorkspaceFileIntoTable, -} from '@/lib/table/application/workspace-file-imports' - -const mocks = { - batchInsert: tableMockFns.mockBatchInsertRows, - inferSchema: tableMockFns.mockInferSchemaFromCsv, - markJob: tableJobsServiceMockFns.mockMarkTableJobRunningInWorkspace, - parseRows: tableMockFns.mockParseFileRows, - releaseJob: tableJobsServiceMockFns.mockReleaseJobClaimInWorkspace, - replaceRows: tableMockFns.mockReplaceTableRows, - resolveTableContext: tableApplicationContextMockFns.mockResolveActiveTableContext, - resolveWorkspaceContext: tableApplicationContextMockFns.mockResolveTableWorkspaceContext, - runDetached: backgroundTaskMockFns.mockRunDetached, - validateMapping: tableMockFns.mockValidateMapping, - coerceRows: tableMockFns.mockCoerceRowsForTable, - audit: auditMockFns.mockRecordAudit, - createTable: tableServiceMockFns.mockCreateTable, - deleteTable: tableServiceMockFns.mockDeleteTable, - fetchFile: workspaceFileManagerMockFns.mockFetchWorkspaceFileBuffer, - loadFileContext: workspaceFileManagerMockFns.mockLoadActiveWorkspaceFileContext, - provenance: workspaceFileSecretProvenanceMockFns.mockGetBoundWorkspaceFileSecretProvenance, - resolveFile: workspaceFileManagerMockFns.mockResolveWorkspaceFileReference, - resolvePermission: workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission, - signal: tableEventsMockFns.mockSignalTableRowsChanged, -} - -tableMockFns.mockBuildAutoMapping.mockReturnValue({ name: 'name' }) -tableMockFns.mockGetWorkspaceTableLimits.mockReturnValue({ maxRowsPerTable: 100, maxTables: 5 }) -tableMockFns.mockSanitizeName.mockImplementation((value: string) => value) -tableRowsSecretProvenanceMockFns.mockCreateExactEmptyTableRowSecretProvenance.mockReturnValue({ - complete: true, - columns: {}, -}) - -idMockFns.mockGenerateId.mockReturnValue('request-id-1234') - -const table: TableDefinition = { - id: 'table-1', - name: 'People', - description: 'Imported', - schema: { columns: [{ id: 'column-name', name: 'name', type: 'string' }] }, - metadata: null, - rowCount: 0, - maxRows: 100, - workspaceId: 'workspace-1', - createdBy: 'user-1', - archivedAt: null, - createdAt: new Date('2026-08-01T00:00:00.000Z'), - updatedAt: new Date('2026-08-01T00:00:00.000Z'), -} -const rejectedSample = { code: 'CSV_QUOTE_NOT_CLOSED', line: 3, message: 'Quote Not Closed' } - -const sourceFile = { - id: 'file-1', - workspaceId: 'workspace-1', - key: 'workspace/workspace-1/people.csv', - name: 'people.csv', - type: 'text/csv', - size: 128, -} -const principal = createDelegatedPrincipal({ - delegationId: 'copilot-tool:tool-1', - audience: 'sim:tables', -}) -const tablePrincipal = { ...principal, resourceScope: { tableId: 'table-1' } } - -describe('workspace-file Table application commands', () => { - beforeEach(() => { - mocks.resolvePermission.mockResolvedValue('write') - mocks.resolveWorkspaceContext.mockResolvedValue({ - workspaceId: 'workspace-1', - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mocks.resolveTableContext.mockResolvedValue({ - tableId: table.id, - table, - workspaceId: table.workspaceId, - workspaceOrganizationId: null, - allowPersonalApiKeys: true, - billedAccountUserId: 'billing-owner-1', - }) - mocks.resolveFile.mockResolvedValue(sourceFile) - mocks.loadFileContext.mockResolvedValue(sourceFile) - mocks.provenance.mockResolvedValue({ status: 'exact', entries: [] }) - mocks.fetchFile.mockResolvedValue(Buffer.from('name\nAda')) - mocks.parseRows.mockResolvedValue({ - headers: ['name'], - rows: [{ name: 'Ada' }], - rejections: { rowsRejected: 0, rejectedSamples: [] }, - }) - mocks.inferSchema.mockReturnValue({ - columns: [{ name: 'name', type: 'string' }], - headerToColumn: new Map([['name', 'name']]), - }) - mocks.createTable.mockResolvedValue(table) - mocks.deleteTable.mockResolvedValue(undefined) - mocks.batchInsert.mockImplementation(async ({ rows }: { rows: unknown[] }) => - rows.map((_, index) => ({ id: `row-${index}` })) - ) - mocks.replaceRows.mockResolvedValue({ insertedCount: 1, deletedCount: 2 }) - mocks.markJob.mockResolvedValue(true) - mocks.releaseJob.mockResolvedValue(true) - mocks.coerceRows.mockImplementation((rows: unknown[]) => rows) - mocks.validateMapping.mockReturnValue({ - effectiveMap: new Map([['name', 'name']]), - mappedHeaders: ['name'], - skippedHeaders: [], - }) - }) - - /** - * The parse dropped these records silently, so an inline import used to finish - * with a smaller table and nothing distinguishing it from a clean one. - */ - it('surfaces the records the parse dropped', async () => { - mocks.parseRows.mockResolvedValueOnce({ - headers: ['name'], - rows: [{ name: 'Ada' }], - rejections: { rowsRejected: 2, rejectedSamples: [rejectedSample] }, - }) - - const result = await createTableFromWorkspaceFile.execute({ - principal, - input: { workspaceId: 'workspace-1', fileReference: 'files/people.csv' }, - }) - - expect(result).toMatchObject({ - kind: 'inline', - rejections: { rowsRejected: 2, cellsRejected: 0, rejectedSamples: [rejectedSample] }, - }) - }) - - it('conceals cross-workspace files before parsing or table mutation', async () => { - mocks.resolveFile.mockResolvedValueOnce({ ...sourceFile, workspaceId: 'workspace-other' }) - - await expect( - createTableFromWorkspaceFile.execute({ - principal, - input: { workspaceId: 'workspace-1', fileReference: 'files/people.csv' }, - }) - ).rejects.toMatchObject({ code: 'not_found' }) - - expect(mocks.fetchFile).not.toHaveBeenCalled() - expect(mocks.createTable).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - }) - - it('rejects non-delegated upload identities before canonical workspace or file loading', async () => { - await expect( - createTableFromWorkspaceFile.execute({ - principal: createWorkspaceApiKeyPrincipal({ keyId: 'workspace-key-1' }) as never, - input: { workspaceId: 'workspace-1', fileReference: 'files/people.csv' }, - }) - ).rejects.toMatchObject({ code: 'forbidden' }) - - expect(mocks.resolveWorkspaceContext).not.toHaveBeenCalled() - expect(mocks.resolveFile).not.toHaveBeenCalled() - }) - - it('rolls back a partially-created table and emits no audit or effect on insertion failure', async () => { - const failure = new Error('database unavailable') - mocks.batchInsert.mockRejectedValueOnce(failure) - - await expect( - createTableFromWorkspaceFile.execute({ - principal, - input: { workspaceId: 'workspace-1', fileReference: 'files/people.csv' }, - }) - ).rejects.toBe(failure) - - expect(mocks.deleteTable).toHaveBeenCalledWith(table.id, 'request-') - expect(mocks.audit).not.toHaveBeenCalled() - expect(mocks.signal).not.toHaveBeenCalled() - }) - - it('holds the concurrency claim across file loading and inline mutation', async () => { - const events: string[] = [] - mocks.markJob.mockImplementationOnce(async () => { - events.push('claim') - return true - }) - mocks.fetchFile.mockImplementationOnce(async () => { - events.push('load') - return Buffer.from('name\nAda') - }) - mocks.batchInsert.mockImplementationOnce(async () => { - events.push('mutate') - return [{ id: 'row-1' }] - }) - mocks.releaseJob.mockImplementationOnce(async () => { - events.push('release') - return true - }) - - await importWorkspaceFileIntoTable.execute({ - principal: tablePrincipal, - input: { - tableId: table.id, - assertedWorkspaceId: table.workspaceId, - fileReference: 'files/people.csv', - mode: 'append', - }, - }) - - expect(events).toEqual(['claim', 'load', 'mutate', 'release']) - }) - - it('surfaces dropped records and uncoercible cells on an inline append', async () => { - mocks.parseRows.mockResolvedValueOnce({ - headers: ['name'], - rows: [{ name: 'Ada' }], - rejections: { rowsRejected: 1, rejectedSamples: [rejectedSample] }, - }) - mocks.coerceRows.mockImplementationOnce( - ( - rows: unknown[], - _schema: unknown, - _map: unknown, - _options: unknown, - onValueRejected?: (columnName: string) => void - ) => { - onValueRejected?.('name') - return rows - } - ) - - const result = await importWorkspaceFileIntoTable.execute({ - principal: tablePrincipal, - input: { - tableId: table.id, - assertedWorkspaceId: table.workspaceId, - fileReference: 'files/people.csv', - mode: 'append', - }, - }) - - expect(result).toMatchObject({ - kind: 'inline', - rejections: { rowsRejected: 1, cellsRejected: 1, rejectedSamples: [rejectedSample] }, - }) - }) - - it('rejects a concurrent import claim before buffering or mutating rows', async () => { - mocks.markJob.mockResolvedValueOnce(false) - - await expect( - importWorkspaceFileIntoTable.execute({ - principal: tablePrincipal, - input: { - tableId: table.id, - assertedWorkspaceId: table.workspaceId, - fileReference: 'files/people.csv', - mode: 'append', - }, - }) - ).rejects.toMatchObject({ code: 'conflict' }) - - expect(mocks.fetchFile).not.toHaveBeenCalled() - expect(mocks.batchInsert).not.toHaveBeenCalled() - expect(mocks.audit).not.toHaveBeenCalled() - }) - - it('preserves append partial-failure semantics and releases the claim on abort', async () => { - mocks.parseRows.mockResolvedValueOnce({ - headers: ['name'], - rows: Array.from({ length: 1001 }, (_, index) => ({ name: `Person ${index}` })), - rejections: { rowsRejected: 0, rejectedSamples: [] }, - }) - const stopped = new Error('stopped') - let checks = 0 - const assertNotAborted = vi.fn(() => { - checks += 1 - if (checks === 3) throw stopped - }) - - await expect( - importWorkspaceFileIntoTable.execute({ - principal: tablePrincipal, - input: { - tableId: table.id, - assertedWorkspaceId: table.workspaceId, - fileReference: 'files/people.csv', - mode: 'append', - assertNotAborted, - }, - }) - ).rejects.toBe(stopped) - - expect(mocks.batchInsert).toHaveBeenCalledTimes(1) - expect(mocks.releaseJob).toHaveBeenCalledWith(table.id, table.workspaceId, 'request-id-1234') - expect(mocks.audit).not.toHaveBeenCalled() - expect(mocks.signal).not.toHaveBeenCalled() - }) - - it('rejects non-empty secret provenance before parsing or mutation', async () => { - mocks.provenance.mockResolvedValueOnce({ status: 'exact', entries: [{ name: 'SECRET' }] }) - - await expect( - importWorkspaceFileIntoTable.execute({ - principal: tablePrincipal, - input: { - tableId: table.id, - assertedWorkspaceId: table.workspaceId, - fileReference: 'files/people.csv', - mode: 'append', - }, - }) - ).rejects.toMatchObject({ code: 'validation' }) - - expect(mocks.fetchFile).not.toHaveBeenCalled() - expect(mocks.markJob).not.toHaveBeenCalled() - expect(mocks.batchInsert).not.toHaveBeenCalled() - }) -}) diff --git a/apps/sim/lib/table/application/workspace-file-imports.ts b/apps/sim/lib/table/application/workspace-file-imports.ts deleted file mode 100644 index 78e2bc0a2b1..00000000000 --- a/apps/sim/lib/table/application/workspace-file-imports.ts +++ /dev/null @@ -1,615 +0,0 @@ -import { AuditAction, AuditResourceType } from '@sim/audit' -import { resolvePrincipalAttribution } from '@sim/auth/principal' -import { createLogger } from '@sim/logger' -import { getErrorMessage } from '@sim/utils/errors' -import { generateId } from '@sim/utils/id' -import { capabilityGovernedPrincipalUserId } from '@/lib/core/application' -import { isTriggerDevEnabled } from '@/lib/core/config/env-flags' -import { OrchestrationError } from '@/lib/core/orchestration/types' -import { runDetached } from '@/lib/core/utils/background' -import { - batchInsertRows, - buildAutoMapping, - type ColumnDefinition, - CSV_ASYNC_IMPORT_THRESHOLD_BYTES, - CSV_MAX_BATCH_SIZE, - type CsvHeaderMapping, - CsvImportValidationError, - type CsvRejectionSummary, - coerceRowsForTable, - getWorkspaceTableLimits, - inferSchemaFromCsv, - parseFileRows, - type RowData, - replaceTableRows, - sanitizeName, - TABLE_LIMITS, - type TableDefinition, - validateMapping, -} from '@/lib/table' -import { defineAuthorizedTableUseCase } from '@/lib/table/application/authorized-table-use-case' -import { - resolveActiveTableContext, - resolveTableWorkspaceContext, -} from '@/lib/table/application/context' -import { tableOperations } from '@/lib/table/application/operations' -import { signalTableRowsChanged } from '@/lib/table/events' -import { runTableImport, type TableImportPayload } from '@/lib/table/import-runner' -import { - markTableJobRunningInWorkspace, - releaseJobClaimInWorkspace, - type TableImportRejectionSummary, -} from '@/lib/table/jobs/service' -import { createExactEmptyTableRowSecretProvenance } from '@/lib/table/rows/secret-provenance' -import { createTable, deleteTable } from '@/lib/table/service' -import { - fetchWorkspaceFileBuffer, - loadActiveWorkspaceFileContext, - resolveWorkspaceFileReference, - type WorkspaceFileRecord, -} from '@/lib/uploads/contexts/workspace/workspace-file-manager' -import { getBoundWorkspaceFileSecretProvenance } from '@/lib/uploads/contexts/workspace/workspace-file-secret-provenance' - -const logger = createLogger('TableWorkspaceFileImportApplication') - -export interface TableWorkspaceFileSource { - id: string - workspaceId: string - key: string - name: string - type: string - size: number -} - -const MAX_INLINE_FILE_BYTES = 50 * 1024 * 1024 - -export interface CreateTableFromWorkspaceFileInput { - workspaceId: string - fileReference: string - name?: string - description?: string - assertNotAborted?: () => void -} - -export type CreateTableFromWorkspaceFileResult = - | { - kind: 'empty' - sourceFile: TableWorkspaceFileSource - } - | { - kind: 'background' - table: TableDefinition - jobId: string - sourceFile: TableWorkspaceFileSource - } - | { - kind: 'inline' - table: TableDefinition - columns: ColumnDefinition[] - insertedCount: number - droppedRows: number - maxRowsPerTable: number - sourceFile: TableWorkspaceFileSource - /** Omitted when the file imported cleanly. See {@link summarizeRejections}. */ - rejections?: TableImportRejectionSummary - } - -export interface ImportWorkspaceFileInput { - tableId: string - assertedWorkspaceId: string - fileReference: string - mode: 'append' | 'replace' - mapping?: CsvHeaderMapping - assertNotAborted?: () => void -} - -export type ImportWorkspaceFileResult = - | { - kind: 'background' - table: TableDefinition - jobId: string - mode: 'append' | 'replace' - sourceFileName: string - } - | { - kind: 'empty' - table: TableDefinition - mode: 'append' | 'replace' - } - | { - kind: 'inline' - table: TableDefinition - mode: 'append' - matchedColumns: string[] - skippedColumns: string[] - insertedCount: number - sourceFileName: string - /** Omitted when the file imported cleanly. See {@link summarizeRejections}. */ - rejections?: TableImportRejectionSummary - } - | { - kind: 'inline' - table: TableDefinition - mode: 'replace' - matchedColumns: string[] - skippedColumns: string[] - insertedCount: number - deletedCount: number - sourceFileName: string - /** Omitted when the file imported cleanly. See {@link summarizeRejections}. */ - rejections?: TableImportRejectionSummary - } - -function requestId(): string { - return generateId().slice(0, 8) -} - -async function resolveSafeSourceFile( - workspaceId: string, - reference: string -): Promise { - const file = await resolveWorkspaceFileReference(workspaceId, reference, { - includeChatUploads: true, - }) - if (!file) { - if (reference.replace(/^\/+/, '').startsWith('uploads/')) { - throw new OrchestrationError( - 'not_found', - `Cannot import "${reference}": no chat upload by that name in this workspace. Use the exact uploads/ path from the upload notice.` - ) - } - throw new OrchestrationError( - 'not_found', - `File not found: "${reference}". Use \`files ls\` or \`files list --search \` to find the path.` - ) - } - const canonical = await loadActiveWorkspaceFileContext(file.id, { includeChatUploads: true }) - if (!canonical || canonical.workspaceId !== workspaceId || file.workspaceId !== workspaceId) { - throw new OrchestrationError('not_found', 'Workspace file not found') - } - const provenance = await getBoundWorkspaceFileSecretProvenance(workspaceId, { - fileId: file.id, - key: file.key, - context: 'workspace', - }) - if (provenance.status !== 'exact' || provenance.entries.length > 0) { - throw new OrchestrationError( - 'validation', - `Cannot import "${reference}": the file cannot be verified as free of resolved secrets.` - ) - } - return file -} - -function shouldImportInBackground(file: TableWorkspaceFileSource): boolean { - const extension = file.name.split('.').pop()?.toLowerCase() - return ( - (extension === 'csv' || extension === 'tsv') && file.size >= CSV_ASYNC_IMPORT_THRESHOLD_BYTES - ) -} - -async function loadInlineRows(file: WorkspaceFileRecord) { - const content = await fetchWorkspaceFileBuffer(file, { maxBytes: MAX_INLINE_FILE_BYTES }) - return parseFileRows(content, file.name, file.type) -} - -/** - * What an inline (buffered) import reports about the data it lost, so a - * partially imported file is not presented as a clean one — the same accounting - * the streaming runner folds into the import record, returned inline because - * these imports never create one. - * - * `undefined` when nothing was lost, leaving a clean import's result exactly - * what it has always been. `rowsRejected` stays a FLOOR: one parser failure can - * discard many records and is reported once. Any loss is also warned, so a - * caller that discards the summary still leaves a trace. - */ -function summarizeRejections( - rejections: CsvRejectionSummary, - cellsRejected: number, - file: TableWorkspaceFileSource -): TableImportRejectionSummary | undefined { - if (rejections.rowsRejected === 0 && cellsRejected === 0) return undefined - const summary = { - rowsRejected: rejections.rowsRejected, - cellsRejected, - rejectedSamples: rejections.rejectedSamples, - } - logger.warn('Inline table import lost source data', { - workspaceId: file.workspaceId, - fileId: file.id, - fileName: file.name, - ...summary, - }) - return summary -} - -async function batchInsertAll(params: { - table: TableDefinition - rows: RowData[] - workspaceId: string - userId: string - /** The gate's subject for enrichment the landed rows auto-fire; see - * {@link BatchInsertData.capabilityGovernedUserId}. */ - capabilityGovernedUserId: string | null - assertNotAborted?: () => void -}): Promise { - let inserted = 0 - for (let index = 0; index < params.rows.length; index += CSV_MAX_BATCH_SIZE) { - params.assertNotAborted?.() - const batch = params.rows.slice(index, index + CSV_MAX_BATCH_SIZE) - const result = await batchInsertRows( - { - tableId: params.table.id, - rows: batch, - workspaceId: params.workspaceId, - userId: params.userId, - capabilityGovernedUserId: params.capabilityGovernedUserId, - secretProvenance: batch.map(createExactEmptyTableRowSecretProvenance), - }, - { ...params.table, rowCount: params.table.rowCount + inserted }, - requestId() - ) - inserted += result.length - } - return inserted -} - -async function dispatchImportJob(payload: TableImportPayload): Promise { - if (isTriggerDevEnabled) { - try { - const [{ tableImportTask }, { tasks }, { resolveTriggerRegion }] = await Promise.all([ - import('@/background/table-import'), - import('@trigger.dev/sdk'), - import('@/lib/core/async-jobs/region'), - ]) - await tasks.trigger('table-import', payload, { - tags: [`tableId:${payload.tableId}`, `jobId:${payload.importId}`], - region: await resolveTriggerRegion(), - }) - } catch (error) { - try { - const released = await releaseJobClaimInWorkspace( - payload.tableId, - payload.workspaceId, - payload.importId - ) - if (!released) throw new Error('Table import claim was no longer active') - } catch (cleanupError) { - logger.error('Failed to release table import claim after dispatch failure', { - tableId: payload.tableId, - jobId: payload.importId, - error: getErrorMessage(cleanupError), - }) - } - throw error - } - return - } - runDetached('table-import', () => runTableImport(payload)) -} - -async function withReleasedTableJobClaim( - tableId: string, - workspaceId: string, - jobId: string, - run: () => Promise -): Promise { - let result: T - try { - result = await run() - } catch (error) { - try { - const released = await releaseJobClaimInWorkspace(tableId, workspaceId, jobId) - if (!released) throw new Error('Table import claim was no longer active') - } catch (cleanupError) { - logger.error('Failed to release table import claim after operation failure', { - tableId, - workspaceId, - jobId, - error: getErrorMessage(cleanupError), - }) - } - throw error - } - const released = await releaseJobClaimInWorkspace(tableId, workspaceId, jobId) - if (!released) throw new Error('Table import claim was no longer active') - return result -} - -export const createTableFromWorkspaceFile = defineAuthorizedTableUseCase({ - operation: tableOperations.createFromWorkspaceFile, - resolveContext: ({ input }: { input: CreateTableFromWorkspaceFileInput }) => - resolveTableWorkspaceContext(input.workspaceId), - async execute({ principal, input, context }): Promise { - const sourceFile = await resolveSafeSourceFile(context.workspaceId, input.fileReference) - const userId = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId - const limits = await getWorkspaceTableLimits(context.workspaceId) - const name = - input.name ?? - sanitizeName(sourceFile.name.replace(/\.[^.]+$/, ''), 'imported_table').slice( - 0, - TABLE_LIMITS.MAX_TABLE_NAME_LENGTH - ) - const description = input.description ?? `Imported from ${sourceFile.name}` - - if (shouldImportInBackground(sourceFile)) { - input.assertNotAborted?.() - const jobId = generateId() - const table = await createTable( - { - name, - description, - schema: { columns: [{ name: 'column_1', type: 'string' }] }, - workspaceId: context.workspaceId, - userId, - maxRows: limits.maxRowsPerTable, - maxTables: limits.maxTables, - jobStatus: 'running', - jobType: 'import', - jobId, - }, - requestId() - ) - try { - await dispatchImportJob({ - importId: jobId, - tableId: table.id, - workspaceId: context.workspaceId, - userId, - fileKey: sourceFile.key, - fileName: sourceFile.name, - delimiter: sourceFile.name.toLowerCase().endsWith('.tsv') ? '\t' : ',', - mode: 'create', - deleteSourceFile: false, - }) - } catch (error) { - try { - await deleteTable(table.id, requestId()) - } catch (cleanupError) { - logger.error('Failed to remove placeholder table after import dispatch failure', { - tableId: table.id, - error: getErrorMessage(cleanupError), - }) - } - throw error - } - return { kind: 'background', table, jobId, sourceFile } - } - - const { headers, rows: sourceRows, rejections } = await loadInlineRows(sourceFile) - if (sourceRows.length === 0) { - summarizeRejections(rejections, 0, sourceFile) - return { kind: 'empty', sourceFile } - } - const { columns, headerToColumn } = inferSchemaFromCsv(headers, sourceRows) - input.assertNotAborted?.() - const droppedRows = Math.max(0, sourceRows.length - limits.maxRowsPerTable) - const rows = droppedRows > 0 ? sourceRows.slice(0, limits.maxRowsPerTable) : sourceRows - const table = await createTable( - { - name, - description, - schema: { columns }, - workspaceId: context.workspaceId, - userId, - maxTables: limits.maxTables, - }, - requestId() - ) - let cellsRejected = 0 - try { - const insertedCount = await batchInsertAll({ - table, - rows: coerceRowsForTable(rows, table.schema, headerToColumn, undefined, () => { - cellsRejected++ - }), - workspaceId: context.workspaceId, - userId, - capabilityGovernedUserId: capabilityGovernedPrincipalUserId(principal), - assertNotAborted: input.assertNotAborted, - }) - const summary = summarizeRejections(rejections, cellsRejected, sourceFile) - return { - kind: 'inline', - table, - columns, - insertedCount, - droppedRows, - maxRowsPerTable: limits.maxRowsPerTable, - sourceFile, - ...(summary ? { rejections: summary } : {}), - } - } catch (error) { - try { - await deleteTable(table.id, requestId()) - } catch (cleanupError) { - logger.error('Failed to roll back table after import failure', { - tableId: table.id, - error: getErrorMessage(cleanupError), - }) - } - throw error - } - }, - projectAudit({ result }) { - if (result.kind === 'empty') return [] - return { - action: AuditAction.TABLE_CREATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Created table "${result.table.name}" from workspace file`, - metadata: { sourceFileId: result.sourceFile.id, importMode: result.kind }, - } - }, - afterSuccess({ result }) { - if (result.kind === 'inline' && result.insertedCount > 0) { - signalTableRowsChanged(result.table.id) - } - }, -}) - -export const importWorkspaceFileIntoTable = defineAuthorizedTableUseCase({ - operation: tableOperations.importWorkspaceFile, - resolveContext: ({ input }: { input: ImportWorkspaceFileInput }) => - resolveActiveTableContext({ - tableId: input.tableId, - assertedWorkspaceId: input.assertedWorkspaceId, - }), - async execute({ principal, input, context }): Promise { - const sourceFile = await resolveSafeSourceFile(context.workspaceId, input.fileReference) - const userId = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }).attributedUserId - - if (shouldImportInBackground(sourceFile)) { - input.assertNotAborted?.() - const jobId = generateId() - const claimed = await markTableJobRunningInWorkspace( - context.table.id, - context.workspaceId, - jobId, - 'import' - ) - if (!claimed) - throw new OrchestrationError('conflict', 'A job is already in progress for this table') - await dispatchImportJob({ - importId: jobId, - tableId: context.table.id, - workspaceId: context.workspaceId, - userId, - fileKey: sourceFile.key, - fileName: sourceFile.name, - delimiter: sourceFile.name.toLowerCase().endsWith('.tsv') ? '\t' : ',', - mode: input.mode, - mapping: input.mapping, - deleteSourceFile: false, - }) - return { - kind: 'background', - table: context.table, - jobId, - mode: input.mode, - sourceFileName: sourceFile.name, - } - } - - const jobId = generateId() - const claimed = await markTableJobRunningInWorkspace( - context.table.id, - context.workspaceId, - jobId, - 'import' - ) - if (!claimed) - throw new OrchestrationError('conflict', 'A job is already in progress for this table') - return withReleasedTableJobClaim(context.table.id, context.workspaceId, jobId, async () => { - const { headers, rows: sourceRows, rejections } = await loadInlineRows(sourceFile) - input.assertNotAborted?.() - if (sourceRows.length === 0) { - summarizeRejections(rejections, 0, sourceFile) - return { kind: 'empty', table: context.table, mode: input.mode } - } - const mapping = input.mapping ?? buildAutoMapping(headers, context.table.schema) - let validation: ReturnType - try { - validation = validateMapping({ - csvHeaders: headers, - mapping, - tableSchema: context.table.schema, - }) - } catch (error) { - if (!(error instanceof CsvImportValidationError)) throw error - throw new OrchestrationError('validation', error.message) - } - if (validation.mappedHeaders.length === 0) { - throw new OrchestrationError( - 'validation', - `No matching columns between file (${headers.join(', ')}) and table (${context.table.schema.columns.map((column) => column.name).join(', ')})` - ) - } - let cellsRejected = 0 - const rows = coerceRowsForTable( - sourceRows, - context.table.schema, - validation.effectiveMap, - undefined, - () => { - cellsRejected++ - } - ) - const summary = summarizeRejections(rejections, cellsRejected, sourceFile) - if (input.mode === 'replace') { - const result = await replaceTableRows( - { - tableId: context.table.id, - rows, - workspaceId: context.workspaceId, - userId, - secretProvenance: rows.map(createExactEmptyTableRowSecretProvenance), - }, - context.table, - requestId() - ) - return { - kind: 'inline', - table: context.table, - mode: input.mode, - matchedColumns: validation.mappedHeaders, - skippedColumns: validation.skippedHeaders, - insertedCount: result.insertedCount, - deletedCount: result.deletedCount, - sourceFileName: sourceFile.name, - ...(summary ? { rejections: summary } : {}), - } - } - const insertedCount = await batchInsertAll({ - table: context.table, - rows, - workspaceId: context.workspaceId, - userId, - capabilityGovernedUserId: capabilityGovernedPrincipalUserId(principal), - assertNotAborted: input.assertNotAborted, - }) - return { - kind: 'inline', - table: context.table, - mode: input.mode, - matchedColumns: validation.mappedHeaders, - skippedColumns: validation.skippedHeaders, - insertedCount, - sourceFileName: sourceFile.name, - ...(summary ? { rejections: summary } : {}), - } - }) - }, - projectAudit({ result }) { - if (result.kind !== 'inline') return [] - const affected = result.insertedCount + (result.mode === 'replace' ? result.deletedCount : 0) - if (affected === 0) return [] - return { - action: AuditAction.TABLE_UPDATED, - resourceType: AuditResourceType.TABLE, - resourceId: result.table.id, - resourceName: result.table.name, - description: `Imported workspace file into table "${result.table.name}"`, - metadata: { - op: 'workspace_file_import', - mode: result.mode, - rowsInserted: result.insertedCount, - ...(result.mode === 'replace' ? { rowsDeleted: result.deletedCount } : {}), - }, - } - }, - afterSuccess({ result }) { - if ( - result.kind === 'inline' && - (result.insertedCount > 0 || (result.mode === 'replace' && result.deletedCount > 0)) - ) { - signalTableRowsChanged(result.table.id) - } - }, -}) diff --git a/apps/sim/lib/table/constants.ts b/apps/sim/lib/table/constants.ts index 56e1f25c05b..5560877e0e5 100644 --- a/apps/sim/lib/table/constants.ts +++ b/apps/sim/lib/table/constants.ts @@ -301,13 +301,6 @@ export const NAME_PATTERN = /^[A-Za-z_][A-Za-z0-9_]*$/ export const USER_TABLE_ROWS_SQL_NAME = 'user_table_rows' -/** - * CSV/TSV uploads at or above this size import in the background (direct-to-storage - * upload + async worker) instead of being POSTed through the server. Kept safely under - * the Next.js proxy request-body cap (10MB) so a synchronous upload is never truncated. - */ -export const CSV_ASYNC_IMPORT_THRESHOLD_BYTES = 8 * 1024 * 1024 - const TABLE_NAME_ADJECTIVES = [ 'Radiant', 'Luminous', diff --git a/apps/sim/lib/table/import.test.ts b/apps/sim/lib/table/import.test.ts index 2747c347f9f..fb95491bd0d 100644 --- a/apps/sim/lib/table/import.test.ts +++ b/apps/sim/lib/table/import.test.ts @@ -14,7 +14,6 @@ import { inferSchemaFromCsv, MAX_REJECTED_SAMPLES, parseCsvBuffer, - parseFileRows, validateMapping, } from '@/lib/table/import' import { createCsvParser } from '@/lib/table/import-stream' @@ -254,14 +253,6 @@ describe('import', () => { expect(rejections.rejectedSamples[0]).toMatchObject({ code: 'CSV_QUOTE_NOT_CLOSED' }) }) - it('threads the summary through parseFileRows for CSV', async () => { - const { rejections } = await parseFileRows( - Buffer.from('name\nOk\nBroken,"unterminated\nAnother\n'), - 'rows.csv' - ) - expect(rejections.rowsRejected).toBeGreaterThan(0) - }) - /** * The count is a floor and the samples are capped, so a systematically broken * million-row file cannot accumulate an entry per lost record. diff --git a/apps/sim/lib/table/import.ts b/apps/sim/lib/table/import.ts index ac8802acec3..6fac1af2913 100644 --- a/apps/sim/lib/table/import.ts +++ b/apps/sim/lib/table/import.ts @@ -1,13 +1,11 @@ -import { isRecordLike } from '@sim/utils/object' /** * Shared CSV import helpers for user-defined tables. * * Used by: * - `POST /api/table/import-csv` (create new table from CSV — streams via {@link createCsvParser}) * - `POST /api/table/[tableId]/import` (append/replace into existing table) - * - Copilot `user-table` tool (`create_from_file`, `import_file` — buffers via {@link parseCsvBuffer}) * - * Keeping a single implementation avoids drift between HTTP and agent code paths. + * Keeping a single implementation avoids drift between import paths. * Both the buffered ({@link parseCsvBuffer}) and streaming ({@link createCsvParser}) * parsers share {@link csvParseOptions} so their behavior can't drift. */ @@ -706,104 +704,3 @@ export function coerceRowsForTable( return coerced }) } - -/** - * Sanitizes raw JSON keys so they conform to the same column-name rules as CSV - * headers, letting `inferSchemaFromCsv` and `coerceRowsForTable` be reused for - * JSON imports. Collisions after sanitization are disambiguated with a trailing - * underscore. Returns the headers and rows untouched when no key needs renaming. - */ -export function sanitizeJsonHeaders( - headers: string[], - rows: Record[] -): { headers: string[]; rows: Record[] } { - const renamed = new Map() - const seen = new Set() - - for (const raw of headers) { - let safe = sanitizeName(raw) - while (seen.has(safe)) safe = `${safe}_` - seen.add(safe) - renamed.set(raw, safe) - } - - const noChange = headers.every((h) => renamed.get(h) === h) - if (noChange) return { headers, rows } - - return { - headers: headers.map((h) => renamed.get(h)!), - rows: rows.map((row) => { - const out: Record = {} - for (const [raw, safe] of renamed) { - if (raw in row) out[safe] = row[raw] - } - return out - }), - } -} - -/** - * Parses a JSON payload that must be an array of plain objects into the same - * `{ headers, rows }` shape produced by `parseCsvBuffer`. The header set is the - * union of all object keys, sanitized via {@link sanitizeJsonHeaders}. - */ -export function parseJsonRows(buffer: Buffer | string): { - headers: string[] - rows: Record[] -} { - const text = typeof buffer === 'string' ? buffer : buffer.toString('utf-8') - const parsed = JSON.parse(text) - if (!Array.isArray(parsed)) { - throw new OrchestrationError('validation', 'JSON file must contain an array of objects') - } - if (parsed.length === 0) { - throw new OrchestrationError('validation', 'JSON file contains an empty array') - } - const headerSet = new Set() - for (const row of parsed) { - if (!isRecordLike(row)) { - throw new OrchestrationError( - 'validation', - 'Each element in the JSON array must be a plain object' - ) - } - for (const key of Object.keys(row)) headerSet.add(key) - } - return sanitizeJsonHeaders([...headerSet], parsed) -} - -/** - * Parses a tabular upload (CSV, TSV, or JSON array-of-objects) into a uniform - * `{ headers, rows, rejections }` shape, dispatching on file extension and falling - * back to the MIME content type. Throws on unsupported formats so callers fail fast. - * - * `rejections` is what a caller publishes so a partially imported file is not - * reported as a clean import. The JSON path never drops records — a malformed - * element throws — so it always reports none. - */ -export async function parseFileRows( - buffer: Buffer, - fileName: string, - contentType?: string -): Promise<{ - headers: string[] - rows: Record[] - rejections: CsvRejectionSummary -}> { - const ext = fileName.split('.').pop()?.toLowerCase() - if (ext === 'json' || contentType === 'application/json') { - return { ...parseJsonRows(buffer), rejections: { rowsRejected: 0, rejectedSamples: [] } } - } - if (ext === 'csv' || ext === 'tsv' || contentType === 'text/csv') { - const delimiter = await detectCsvDelimiter( - buffer.subarray(0, CSV_DELIMITER_SNIFF_BYTES), - ext === 'tsv' ? '\t' : ',', - { complete: buffer.length <= CSV_DELIMITER_SNIFF_BYTES } - ) - return parseCsvBuffer(buffer, delimiter) - } - throw new OrchestrationError( - 'validation', - `Unsupported file format: "${ext ?? fileName}". Supported: csv, tsv, json` - ) -} diff --git a/apps/sim/lib/table/jobs/service.ts b/apps/sim/lib/table/jobs/service.ts index 0716f6a610c..d9519ce0c16 100644 --- a/apps/sim/lib/table/jobs/service.ts +++ b/apps/sim/lib/table/jobs/service.ts @@ -244,26 +244,6 @@ export async function releaseJobClaim(tableId: string, jobId: string): Promise { - const released = await db - .delete(tableJobs) - .where( - and( - eq(tableJobs.id, jobId), - eq(tableJobs.tableId, tableId), - eq(tableJobs.workspaceId, workspaceId), - eq(tableJobs.status, 'running') - ) - ) - .returning({ id: tableJobs.id }) - return released.length > 0 -} - /** * Records job progress (rows processed so far) and bumps `updated_at` so the stale-job janitor * (`cleanup-stale-executions`) sees a live heartbeat. diff --git a/apps/sim/lib/table/types.ts b/apps/sim/lib/table/types.ts index 1980e763e13..e75f6c7021e 100644 --- a/apps/sim/lib/table/types.ts +++ b/apps/sim/lib/table/types.ts @@ -347,24 +347,6 @@ export interface TableDeleteJobPayload { maxRows?: number } -/** - * Persisted scope of a running bulk-update job (`table_jobs.payload`): the same `data` patch is - * merged into every row matching `filter` with `created_at <= cutoff` (so mid-job inserts are - * spared, matching the delete job's snapshot semantics). `affectedCount` is the kickoff estimate, - * display-only. Unlike delete, reads are not masked — updated rows still exist, so a background - * update is eventually consistent (readers may see a mix of patched/unpatched rows mid-job). - */ -export interface TableUpdateJobPayload { - filter: Filter - /** Column-id-keyed partial patch applied to every matched row (JSONB merge). */ - data: RowData - /** ISO timestamp; rows created after it are not patched. */ - cutoff: string - affectedCount?: number - /** Stop after updating this many rows (an explicit caller-supplied limit). Omitted = every match. */ - maxRows?: number -} - export type TableExportFormat = 'csv' | 'json' /** diff --git a/apps/sim/lib/table/workflow-groups/service.test.ts b/apps/sim/lib/table/workflow-groups/service.test.ts index c451bc5eb99..54a8a861918 100644 --- a/apps/sim/lib/table/workflow-groups/service.test.ts +++ b/apps/sim/lib/table/workflow-groups/service.test.ts @@ -26,16 +26,11 @@ vi.mock('@/lib/table/schema-invariants', () => ({ })) import { TABLE_LIMITS } from '@/lib/table/constants' -import { - addWorkflowGroup, - addWorkflowGroupOutput, - updateWorkflowGroup, -} from '@/lib/table/workflow-groups/service' +import { addWorkflowGroup, updateWorkflowGroup } from '@/lib/table/workflow-groups/service' const mockAssertTableRowTtlEnabled = tableTtlAvailabilityMockFns.mockAssertTableRowTtlEnabled const mockWithLockedTable = tableServiceMockFns.mockWithLockedTable -const mockGetTableById = tableServiceMockFns.mockGetTableById function groupAt(index: number): WorkflowGroup { return { @@ -85,7 +80,6 @@ describe('addWorkflowGroup group ceiling', () => { execute: () => Promise.resolve(), }) ) - mockGetTableById.mockResolvedValue(table) return addWorkflowGroup( { tableId: 'table-1', @@ -138,22 +132,6 @@ describe('workflow group TTL availability', () => { 'request-1' ), ], - [ - 'single output addition', - () => - addWorkflowGroupOutput( - { - tableId: 'table-1', - workspaceId: 'workspace-1', - groupId: 'group-1', - blockId: 'block-1', - path: 'expiresAt', - capabilityGovernedUserId: null, - resolvedOutput: { workflowId: 'workflow-1', columnType: 'ttl', order: [] }, - }, - 'request-1' - ), - ], ])('rejects TTL introduction through %s while disabled', async (_label, introduceTtl) => { await expect(introduceTtl()).rejects.toThrow('Expiration columns are not enabled') expect(mockWithLockedTable).not.toHaveBeenCalled() diff --git a/apps/sim/lib/table/workflow-groups/service.ts b/apps/sim/lib/table/workflow-groups/service.ts index 40098f12783..5bd853a6cb7 100644 --- a/apps/sim/lib/table/workflow-groups/service.ts +++ b/apps/sim/lib/table/workflow-groups/service.ts @@ -19,13 +19,12 @@ import { getColumnId, remapGroupColumnRefs, } from '@/lib/table/column-keys' -import { deriveOutputColumnName } from '@/lib/table/column-naming' import { NAME_PATTERN, TABLE_LIMITS } from '@/lib/table/constants' import { assertColumnDestructive, assertSchemaMutable } from '@/lib/table/mutation-locks' import { stripGroupExecutions } from '@/lib/table/rows/executions' import { updateTableRowsWithDerivedSecretProvenance } from '@/lib/table/rows/secret-provenance' import { assertValidSchema } from '@/lib/table/schema-invariants' -import { getTableById, withLockedTable } from '@/lib/table/service' +import { withLockedTable } from '@/lib/table/service' import { assertTableRowTtlEnabled } from '@/lib/table/ttl-availability' import { setTableTxTimeouts } from '@/lib/table/tx' import type { @@ -681,273 +680,6 @@ export async function updateWorkflowGroup( return updatedTable } -/** - * Adds a single output to an existing workflow group. Mirrors `addTableColumn` - * for plain columns: one canonical op, one column created, type inferred from - * the workflow's flattened outputs (`leafType` for `(blockId, path)`). The - * column is spliced into the group's contiguous run so the table renders the - * new output next to its siblings. - */ -export async function addWorkflowGroupOutput( - data: { - tableId: string - /** Canonical workspace derived by the authorized caller. */ - workspaceId?: string - groupId: string - blockId: string - path: string - /** Optional override; defaults to a slug derived from `path`. */ - columnName?: string - /** The member adding the output — the billing attribution for the backfill's - * row writes. Not the gate: see `capabilityGovernedUserId`. */ - actorUserId?: string | null - /** Person whose permission group gates any cell the backfill's writes - * cascade into; `null` when the change has no acting person. Required; see - * {@link InsertRowData.capabilityGovernedUserId} in `@/lib/table/types`. */ - capabilityGovernedUserId: string | null - resolvedOutput: { - workflowId: string - columnType: ColumnDefinition['type'] - order: Array<{ - blockId: string - path: string - executionDistance: number - discoveryIndex: number - }> - } - }, - requestId: string -): Promise { - if (data.resolvedOutput.columnType === 'ttl') await assertTableRowTtlEnabled() - - // Phase 1 (no lock): validate the authorized workflow metadata against the - // group's current workflow. Phase 2 re-validates the same binding under the - // table lock before applying the mutation. - const preTable = await getTableById(data.tableId) - if (!preTable || (data.workspaceId && preTable.workspaceId !== data.workspaceId)) { - throw new OrchestrationError('not_found', 'Table not found') - } - const preGroup = (preTable.schema.workflowGroups ?? []).find((g) => g.id === data.groupId) - if (!preGroup) { - throw new OrchestrationError('not_found', `Workflow group "${data.groupId}" not found`) - } - const workflowId = preGroup.workflowId - if (data.resolvedOutput.workflowId !== workflowId) { - throw new OrchestrationError('not_found', 'Workflow not found') - } - const newColumnType = data.resolvedOutput.columnType - const resolvedOrder = new Map( - data.resolvedOutput.order.map((output) => [ - `${output.blockId}::${output.path}`, - [output.executionDistance, output.discoveryIndex] as const, - ]) - ) - - // Phase 2 (locked): re-read fresh, validate against the current schema, and - // write. The critical section holds no I/O — just the in-memory splice + the - // schema UPDATE — so concurrent adders queue behind it quickly. - const { updatedTable, newOutput } = await withLockedTable( - data.tableId, - async (table, trx) => { - assertSchemaMutable(table) - const schema = table.schema - const groups = schema.workflowGroups ?? [] - const groupIndex = groups.findIndex((g) => g.id === data.groupId) - if (groupIndex === -1) { - throw new OrchestrationError('not_found', `Workflow group "${data.groupId}" not found`) - } - const group = groups[groupIndex] - if (group.workflowId !== workflowId) { - throw new OrchestrationError( - 'conflict', - `Workflow group "${data.groupId}" was remapped to a different workflow concurrently; retry the add.` - ) - } - - if (group.outputs.some((o) => o.blockId === data.blockId && o.path === data.path)) { - throw new OrchestrationError( - 'validation', - `Workflow group "${data.groupId}" already has an output at ${data.blockId}::${data.path}` - ) - } - - const taken = new Set(schema.columns.map((c) => c.name)) - const columnName = data.columnName ?? deriveOutputColumnName(data.path, taken) - if (!NAME_PATTERN.test(columnName)) { - throw new OrchestrationError( - 'validation', - `Invalid column name "${columnName}". Must satisfy ${NAME_PATTERN.source}.` - ) - } - if (taken.has(columnName)) { - throw new OrchestrationError('validation', `Column "${columnName}" already exists`) - } - if (schema.columns.length + 1 > TABLE_LIMITS.MAX_COLUMNS_PER_TABLE) { - throw new OrchestrationError( - 'validation', - `Adding a column would exceed the maximum (${TABLE_LIMITS.MAX_COLUMNS_PER_TABLE}).` - ) - } - - const newColDef: ColumnDefinition = { - id: generateColumnId(), - name: columnName, - type: newColumnType, - required: false, - unique: false, - workflowGroupId: data.groupId, - } - const newColumnId = getColumnId(newColDef) - const newOutput: WorkflowGroupOutput = { - blockId: data.blockId, - path: data.path, - columnName: newColumnId, - } - - // Sort all of the group's outputs (existing + new) in workflow execution - // order: BFS distance from the start block ASC, with discovery order as - // tiebreak. This matches what the column-sidebar does at create time, so - // columns from the same workflow always read in the order their blocks run - // — regardless of whether they were added at create time or one-by-one. - const groupColIdsBefore = new Set(group.outputs.map((o) => o.columnName)) - const orderKey = (o: { blockId: string; path: string }) => { - return ( - resolvedOrder.get(`${o.blockId}::${o.path}`) ?? - ([Number.POSITIVE_INFINITY, Number.POSITIVE_INFINITY] as const) - ) - } - const allGroupOutputs = [...group.outputs, newOutput].sort((a, b) => { - const [da, ia] = orderKey(a) - const [db, ib] = orderKey(b) - return da !== db ? da - db : ia - ib - }) - const invalidOutput = allGroupOutputs.find( - (output) => !resolvedOrder.has(`${output.blockId}::${output.path}`) - ) - if (invalidOutput) { - throw new OrchestrationError( - 'conflict', - `Workflow group "${data.groupId}" mappings changed concurrently; retry the add.` - ) - } - const orderedGroupColIds = allGroupOutputs.map((o) => o.columnName) - const updatedGroup: WorkflowGroup = { - ...group, - outputs: allGroupOutputs, - } - const nextGroups = groups.map((g, i) => (i === groupIndex ? updatedGroup : g)) - - // Splice the new column run into nextColumns: keep the columns outside the - // group where they were, replace the group's contiguous run with the - // BFS-ordered list. Anchor at the position of the first existing sibling - // (or append if the group was empty). - const colById = new Map(schema.columns.map((c) => [getColumnId(c), c])) - const orderedGroupCols: ColumnDefinition[] = orderedGroupColIds.map((id) => { - if (id === newColumnId) return newColDef - const existing = colById.get(id) - if (!existing) { - throw new Error(`Internal: column "${id}" missing while splicing group outputs`) - } - return existing - }) - const remainingCols = schema.columns.filter((c) => !groupColIdsBefore.has(getColumnId(c))) - const firstGroupIdx = schema.columns.findIndex((c) => groupColIdsBefore.has(getColumnId(c))) - const colAnchor = firstGroupIdx === -1 ? remainingCols.length : firstGroupIdx - const nextColumns = [ - ...remainingCols.slice(0, colAnchor), - ...orderedGroupCols, - ...remainingCols.slice(colAnchor), - ] - - const updatedSchema: TableSchema = { - ...schema, - columns: nextColumns, - workflowGroups: nextGroups, - } - - const updatedColumnOrder = table.metadata?.columnOrder - ? (() => { - const orderWithoutGroup = table.metadata!.columnOrder!.filter( - (id) => !groupColIdsBefore.has(id) - ) - const firstGroupOrderIdx = table.metadata!.columnOrder!.findIndex((id) => - groupColIdsBefore.has(id) - ) - const orderAnchor = - firstGroupOrderIdx === -1 ? orderWithoutGroup.length : firstGroupOrderIdx - return [ - ...orderWithoutGroup.slice(0, orderAnchor), - ...orderedGroupColIds, - ...orderWithoutGroup.slice(orderAnchor), - ] - })() - : undefined - - assertValidSchema(updatedSchema, updatedColumnOrder) - - const updatedMetadata: TableMetadata | null = - updatedColumnOrder && table.metadata - ? { ...table.metadata, columnOrder: updatedColumnOrder } - : table.metadata - ? { ...table.metadata } - : null - - const now = new Date() - await trx - .update(userTableDefinitions) - .set({ schema: updatedSchema, metadata: updatedMetadata, updatedAt: now }) - .where( - and( - eq(userTableDefinitions.id, data.tableId), - eq(userTableDefinitions.workspaceId, table.workspaceId) - ) - ) - - logger.info( - `[${requestId}] Added output "${columnName}" (${newColDef.type}) to workflow group "${data.groupId}" in table ${data.tableId}` - ) - - const updatedTable: TableDefinition = { - ...table, - schema: updatedSchema, - metadata: updatedMetadata, - updatedAt: now, - } - return { updatedTable, newOutput } - }, - { expectedWorkspaceId: data.workspaceId } - ) - - // Backfill from saved execution logs — same flow `updateWorkflowGroup` - // uses for added outputs. Reads each row's saved trace spans for the - // group's executionId and writes the new output's value back. Existing - // rows that have hand-edited values are left alone (overwrite: false). - // Cheap compared to re-running the workflow on every row, which is what - // an earlier version of this code did — that mistakenly fanned out N - // workflow-group-cell jobs and burned compute the user didn't ask for. - // Small tables backfill inline; large ones run as a background job. - // Lazy import: backfill-runner closes a cycle back to this module. - try { - const { maybeBackfillGroupOutputs } = await import('@/lib/table/backfill-runner') - await maybeBackfillGroupOutputs({ - table: updatedTable, - groupId: data.groupId, - outputs: [newOutput], - overwrite: false, - requestId, - actorUserId: data.actorUserId, - capabilityGovernedUserId: data.capabilityGovernedUserId, - }) - } catch (err) { - logger.warn( - `[${requestId}] Backfill from execution logs failed for ${data.tableId} group ${data.groupId} after adding output "${newOutput.columnName}":`, - err - ) - } - - return updatedTable -} - /** * Removes a workflow group plus all its output columns. Also strips the * group's `executions[groupId]` entry from every row. diff --git a/apps/sim/lib/workflows/application/operations.ts b/apps/sim/lib/workflows/application/operations.ts index 9ad2572b420..93a6ab692e7 100644 --- a/apps/sim/lib/workflows/application/operations.ts +++ b/apps/sim/lib/workflows/application/operations.ts @@ -178,14 +178,6 @@ export const workflowOperations = { capability: 'none', ...ALL_WORKFLOW_PRINCIPAL_POLICY, }), - // permission-group-exempt: toggling a block edits workflow content; which integrations a member may use is allowedIntegrations, enforced against the block type rather than the operation - setBlockEnabled: defineWorkspaceOperation({ - id: 'workflows.blocks.set_enabled', - minimumRole: 'write', - workspaceApiKey: 'deny', - capability: 'none', - ...COPILOT_WORKFLOW_PRINCIPAL_POLICY, - }), // permission-group-exempt: moving workflows between folders is placement, governed by workspace role moveBulk: defineWorkspaceOperation({ id: 'workflows.bulk.move', diff --git a/apps/sim/lib/workflows/application/update-workflow-content.test.ts b/apps/sim/lib/workflows/application/update-workflow-content.test.ts index 9a8b15e5c6d..95413a3ced9 100644 --- a/apps/sim/lib/workflows/application/update-workflow-content.test.ts +++ b/apps/sim/lib/workflows/application/update-workflow-content.test.ts @@ -5,15 +5,10 @@ import { workflowContextMock, workflowContextMockFns, } from '@sim/testing/mocks/workflow-context.mock' -import { - workflowsPersistenceUtilsMock, - workflowsPersistenceUtilsMockFns, -} from '@sim/testing/mocks/workflows-persistence-utils.mock' import { workspaceAuthzMock, workspaceAuthzMockFns } from '@sim/testing/mocks/workspace-authz.mock' import { beforeEach, describe, expect, it, vi } from 'vitest' const mocks = vi.hoisted(() => ({ - replace: vi.fn(), requireMutable: vi.fn(), })) @@ -24,20 +19,11 @@ vi.mock('@sim/platform-authz/workspace', () => workspaceAuthzMock) vi.mock('@/lib/workflows/application/context', () => workflowContextMock) vi.mock('@/lib/realtime/notify', () => realtimeNotifyMock) -vi.mock('@/lib/workflows/persistence/utils', () => workflowsPersistenceUtilsMock) -vi.mock('@/lib/workflows/persistence/replace-normalized-state', () => ({ - replaceWorkflowNormalizedState: mocks.replace, -})) vi.mock('@/lib/workflows/application/workflow-mutability', () => ({ requireMutableWorkflow: mocks.requireMutable, })) -import { - applyWorkflowVariableOperations, - setWorkflowBlockEnabled, -} from '@/lib/workflows/application/update-workflow-content' - -const mockLoadNormalized = workflowsPersistenceUtilsMockFns.mockLoadWorkflowFromNormalizedTables +import { applyWorkflowVariableOperations } from '@/lib/workflows/application/update-workflow-content' const mockRecordAudit = auditMockFns.mockRecordAudit const mockResolvePermission = workspaceAuthzMockFns.mockResolveEffectiveWorkspacePermission @@ -156,59 +142,3 @@ describe('applyWorkflowVariableOperations', () => { expect(mockResolveContext).not.toHaveBeenCalled() }) }) - -describe('setWorkflowBlockEnabled', () => { - const BLOCK = { - id: 'block-1', - type: 'agent', - name: 'Triage', - position: { x: 0, y: 0 }, - subBlocks: {}, - outputs: {}, - enabled: true, - data: {}, - } - - beforeEach(() => { - resetDbChainMock() - mockResolveContext.mockResolvedValue(context) - mockResolvePermission.mockResolvedValue('write') - mocks.requireMutable.mockResolvedValue(undefined) - mockLoadNormalized.mockResolvedValue({ - blocks: { 'block-1': BLOCK }, - edges: [], - loops: {}, - parallels: {}, - }) - mocks.replace.mockResolvedValue({ - warnings: [], - state: { blocks: { 'block-1': { ...BLOCK, enabled: false } }, edges: [] }, - }) - }) - - /** - * The graph is produced inside the primitive's transaction, not handed to it - * pre-read: the editor's own save takes the same row lock, so a graph read - * before the lock can be a stale copy that this write — a whole graph, not a - * delta — would persist over a concurrent autosave. - */ - it('re-reads and re-decides inside the write transaction', async () => { - await setWorkflowBlockEnabled.execute({ - principal, - input: { workflowId: 'workflow-1', blockId: 'block-1', enabled: false }, - }) - - const { state } = mocks.replace.mock.calls[0]![0] - expect(typeof state).toBe('function') - - mockLoadNormalized.mockClear() - const tx = Symbol('tx') - await expect(state(tx)).resolves.toEqual({ - blocks: { 'block-1': { ...BLOCK, enabled: false } }, - edges: [], - }) - expect(mockLoadNormalized).toHaveBeenCalledWith('workflow-1', tx) - }) - - /** The returned state is what was persisted, not what was proposed. */ -}) diff --git a/apps/sim/lib/workflows/application/update-workflow-content.ts b/apps/sim/lib/workflows/application/update-workflow-content.ts index 37db975c265..68a9278e718 100644 --- a/apps/sim/lib/workflows/application/update-workflow-content.ts +++ b/apps/sim/lib/workflows/application/update-workflow-content.ts @@ -1,13 +1,7 @@ import { AuditAction, AuditResourceType } from '@sim/audit' -import { resolvePrincipalAttribution } from '@sim/auth/principal' import { db } from '@sim/db' import { workflow } from '@sim/db/schema' import { generateId } from '@sim/utils/id' -import { - generateLoopBlocks, - generateParallelBlocks, -} from '@sim/workflow-persistence/subflow-helpers' -import type { BlockState, WorkflowState } from '@sim/workflow-types/workflow' import { and, eq, isNull } from 'drizzle-orm' import { principalAuditSource } from '@/lib/core/application' import { OrchestrationError } from '@/lib/core/orchestration/types' @@ -21,25 +15,9 @@ import { normalizeWorkflowVariables, type WorkflowVariable, } from '@/lib/workflows/application/workflow-variables' -import { - type BlockEnablementRefusal, - decideBlockEnablement, -} from '@/lib/workflows/editing/block-enablement' -import { replaceWorkflowNormalizedState } from '@/lib/workflows/persistence/replace-normalized-state' -import { loadWorkflowFromNormalizedTables } from '@/lib/workflows/persistence/utils' const MAX_WORKFLOW_VARIABLE_OPERATIONS = 100 -/** How each protection refusal is classified when a single block toggle is the whole request. */ -const BLOCK_ENABLEMENT_REFUSAL_CODES: Record< - BlockEnablementRefusal['reason'], - 'not_found' | 'locked' | 'validation' -> = { - not_found: 'not_found', - locked: 'locked', - disabled_ancestor: 'validation', -} - interface WorkflowContentInput { workflowId: string assertedWorkspaceId?: string @@ -157,144 +135,3 @@ export const applyWorkflowVariableOperations = defineAuthorizedWorkflowUseCase({ afterSuccess: ({ context, result }) => result.changed ? notifyWorkflowUpdated(context.workflowId) : undefined, }) - -export interface SetWorkflowBlockEnabledInput extends WorkflowContentInput { - blockId: string - enabled: boolean -} - -/** - * Toggles one block, or a container and its unlocked descendants. - * - * The write goes through {@link replaceWorkflowNormalizedState}, the same door - * `replaceWorkflowState` and `applyWorkflowOperations` use, so this toggle - * cannot acquire different persistence behavior by being a different entry - * point: it gets the same state preparation, the same row-locked replace - * transaction, the same `lastSynced` stamp, and the same custom-tool - * extraction. Writing the graph here directly was how those diverged. - */ -export const setWorkflowBlockEnabled = defineAuthorizedWorkflowUseCase({ - operation: workflowOperations.setBlockEnabled, - resolveContext: resolvePrincipalWorkflowContext, - async execute({ principal, input, context }) { - await requireMutableWorkflow(context.workflowId) - - const normalized = await loadWorkflowFromNormalizedTables(context.workflowId) - if (!normalized) { - throw new OrchestrationError( - 'validation', - `Workflow ${context.workflowId} has no normalized state` - ) - } - const currentState: WorkflowState = { - blocks: normalized.blocks as Record, - edges: normalized.edges || [], - loops: normalized.loops || {}, - parallels: normalized.parallels || {}, - lastSaved: Date.now(), - } - const decision = decideBlockEnablement(currentState.blocks, input.blockId, input.enabled) - if (decision.outcome === 'refused') { - throw new OrchestrationError( - BLOCK_ENABLEMENT_REFUSAL_CODES[decision.refusal.reason], - decision.refusal.reason === 'not_found' - ? `Block ${input.blockId} not found in workflow ${context.workflowId}` - : decision.refusal.message - ) - } - if (decision.outcome === 'unchanged') { - return { - changed: false, - workflowName: context.workflow.name, - affectedBlockIds: decision.affectedBlockIds, - state: currentState, - } - } - - const attribution = resolvePrincipalAttribution(principal, { - workspaceBillingOwnerUserId: context.billedAccountUserId, - }) - /** - * The graph is re-read and the toggle re-decided inside the row lock. - * - * The read above is advisory: it answers "is this a refusal or a no-op" - * cheaply, but a graph read outside the lock cannot be written back safely. - * The editor's own save takes the same lock, so between that read and this - * write a canvas autosave can commit — and this operation writes a whole - * graph, not a delta, so persisting the stale copy would discard it wholly. - */ - const persisted = await replaceWorkflowNormalizedState({ - workflowId: context.workflowId, - workspaceId: context.workspaceId, - attributedUserId: attribution.attributedUserId, - /** - * Actorless on purpose. This operation writes back the graph it just read - * under the row lock with one block's `enabled` flipped — the caller - * supplies no blocks, so there is no caller-chosen block type for an - * allowlist to judge. Governing it would only mean refusing a member the - * ability to *disable* a block their group withholds. - * - * `attribution.attributedUserId` is deliberately not reused: it answers a - * workspace API key with the workspace's billing owner, which is right for - * custom-tool ownership and wrong for anything reading a person's grants. - */ - subjectUserId: null, - state: async (tx) => { - const locked = await loadWorkflowFromNormalizedTables(context.workflowId, tx) - if (!locked) { - throw new OrchestrationError( - 'validation', - `Workflow ${context.workflowId} has no normalized state` - ) - } - const lockedBlocks = locked.blocks as Record - const lockedDecision = decideBlockEnablement(lockedBlocks, input.blockId, input.enabled) - if (lockedDecision.outcome === 'refused') { - throw new OrchestrationError( - BLOCK_ENABLEMENT_REFUSAL_CODES[lockedDecision.refusal.reason], - lockedDecision.refusal.reason === 'not_found' - ? `Block ${input.blockId} not found in workflow ${context.workflowId}` - : lockedDecision.refusal.message - ) - } - return { - blocks: lockedDecision.outcome === 'unchanged' ? lockedBlocks : lockedDecision.blocks, - edges: locked.edges || [], - } - }, - }) - - const blocks = persisted.state.blocks as Record - return { - changed: true, - workflowName: context.workflow.name, - affectedBlockIds: decision.affectedBlockIds, - state: { - blocks, - edges: persisted.state.edges, - loops: generateLoopBlocks(blocks), - parallels: generateParallelBlocks(blocks), - lastSaved: Date.now(), - } satisfies WorkflowState, - } - }, - projectAudit: ({ principal, input, context, result }) => - result.changed - ? { - action: AuditAction.WORKFLOW_UPDATED, - resourceType: AuditResourceType.WORKFLOW, - resourceId: context.workflowId, - resourceName: result.workflowName, - description: `${input.enabled ? 'Enabled' : 'Disabled'} workflow block "${input.blockId}"`, - metadata: { - op: 'set_block_enabled', - blockId: input.blockId, - enabled: input.enabled, - affectedBlockIds: result.affectedBlockIds, - source: principalAuditSource(principal), - }, - } - : [], - afterSuccess: ({ context, result }) => - result.changed ? notifyWorkflowUpdated(context.workflowId) : undefined, -}) diff --git a/apps/sim/lib/workflows/custom-tools/operations.ts b/apps/sim/lib/workflows/custom-tools/operations.ts index 565e49735c1..7ebd763cbfd 100644 --- a/apps/sim/lib/workflows/custom-tools/operations.ts +++ b/apps/sim/lib/workflows/custom-tools/operations.ts @@ -337,61 +337,3 @@ export async function getCustomToolById(params: { lookup: 'id', }) } - -export async function updateCustomTool(params: { - toolId: string - userId: string - workspaceId: string - title: string - schema: unknown - code: string -}) { - const workspaceTool = await updateWorkspaceCustomTool(params) - if (workspaceTool) return workspaceTool - - const [legacyTool] = await db - .update(customTools) - .set({ - title: params.title, - schema: params.schema, - code: params.code, - updatedAt: new Date(), - }) - .where( - and( - eq(customTools.id, params.toolId), - isNull(customTools.workspaceId), - eq(customTools.userId, params.userId) - ) - ) - .returning() - return legacyTool ?? null -} - -export async function deleteCustomTool(params: { - toolId: string - userId: string - workspaceId?: string -}): Promise { - const { toolId, userId, workspaceId } = params - - if (workspaceId) { - const workspaceDelete = await db - .delete(customTools) - .where(and(eq(customTools.id, toolId), eq(customTools.workspaceId, workspaceId))) - .returning({ id: customTools.id }) - if (workspaceDelete.length > 0) return true - } - - const legacyDelete = await db - .delete(customTools) - .where( - and( - eq(customTools.id, toolId), - isNull(customTools.workspaceId), - eq(customTools.userId, userId) - ) - ) - .returning({ id: customTools.id }) - return legacyDelete.length > 0 -} diff --git a/apps/sim/lib/workflows/editing/block-enablement.ts b/apps/sim/lib/workflows/editing/block-enablement.ts index d1b8a02a748..6a5abe978b7 100644 --- a/apps/sim/lib/workflows/editing/block-enablement.ts +++ b/apps/sim/lib/workflows/editing/block-enablement.ts @@ -55,10 +55,8 @@ export type BlockEnablementDecision = * Pure, and the single source of truth for the three protection rules — a * locked block or locked container cannot be toggled, a block cannot be enabled * while a container above it is disabled, and toggling a loop or parallel - * cascades to its unlocked descendants. Both the dedicated - * `workflows.blocks.set_enabled` use case and the `setBlockEnabled` slice of a - * `workflows.operations.apply` batch call it, so the two cannot drift into - * disagreeing about what is protected. + * cascades to its unlocked descendants. The `setBlockEnabled` slice of a + * `workflows.operations.apply` batch calls it. */ export function decideBlockEnablement( blocks: Record, diff --git a/apps/sim/lib/workspaces/organization/types.ts b/apps/sim/lib/workspaces/organization/types.ts index fcb091fea8e..ce817a5d5c9 100644 --- a/apps/sim/lib/workspaces/organization/types.ts +++ b/apps/sim/lib/workspaces/organization/types.ts @@ -11,20 +11,12 @@ export interface Member { user?: User } -interface Invitation { - id: string - email: string - status: string - membershipIntent?: 'internal' | 'external' -} - export interface Organization { id: string name: string slug: string logo?: string | null members?: Member[] - invitations?: Invitation[] createdAt: string | Date [key: string]: unknown } diff --git a/apps/sim/lib/workspaces/utils.ts b/apps/sim/lib/workspaces/utils.ts index 39c3ba85d0a..40dcd050f33 100644 --- a/apps/sim/lib/workspaces/utils.ts +++ b/apps/sim/lib/workspaces/utils.ts @@ -68,7 +68,7 @@ export async function getWorkspaceOrganizationId(workspaceId: string): Promise { let name = raw .trim() @@ -210,7 +208,6 @@ export const tableMockFns = { mockValidateMapping: vi.fn(), mockDetectCsvDelimiter: vi.fn(), mockParseCsvBuffer: vi.fn(), - mockParseFileRows: vi.fn(), mockEnrichTableToolDescription: vi.fn(), mockEnrichTableToolParameters: vi.fn(), mockFilterRulesToFilter: vi.fn(), @@ -257,7 +254,6 @@ export const tableMockFns = { mockListTableViews: vi.fn(), mockUpdateTableView: vi.fn(), mockAddWorkflowGroup: vi.fn(), - mockAddWorkflowGroupOutput: vi.fn(), mockDeleteWorkflowGroup: vi.fn(), mockPruneStaleWorkflowGroupOutputs: vi.fn(), mockUpdateWorkflowGroup: vi.fn(), @@ -440,13 +436,10 @@ export const tableMock = { dedupeHeaders: fns.mockDedupeHeaders, inferColumnType: fns.mockInferColumnType, inferSchemaFromCsv: fns.mockInferSchemaFromCsv, - parseJsonRows: fns.mockParseJsonRows, - sanitizeJsonHeaders: fns.mockSanitizeJsonHeaders, sanitizeName: fns.mockSanitizeName, validateMapping: fns.mockValidateMapping, detectCsvDelimiter: fns.mockDetectCsvDelimiter, parseCsvBuffer: fns.mockParseCsvBuffer, - parseFileRows: fns.mockParseFileRows, enrichTableToolDescription: fns.mockEnrichTableToolDescription, enrichTableToolParameters: fns.mockEnrichTableToolParameters, filterRulesToFilter: fns.mockFilterRulesToFilter, @@ -493,7 +486,6 @@ export const tableMock = { listTableViews: fns.mockListTableViews, updateTableView: fns.mockUpdateTableView, addWorkflowGroup: fns.mockAddWorkflowGroup, - addWorkflowGroupOutput: fns.mockAddWorkflowGroupOutput, deleteWorkflowGroup: fns.mockDeleteWorkflowGroup, pruneStaleWorkflowGroupOutputs: fns.mockPruneStaleWorkflowGroupOutputs, updateWorkflowGroup: fns.mockUpdateWorkflowGroup, diff --git a/packages/testing/src/mocks/workspaces-utils.mock.ts b/packages/testing/src/mocks/workspaces-utils.mock.ts index ec42e42e9e1..4c2da675994 100644 --- a/packages/testing/src/mocks/workspaces-utils.mock.ts +++ b/packages/testing/src/mocks/workspaces-utils.mock.ts @@ -34,7 +34,6 @@ export const workspacesUtilsMockFns = { mockGetWorkspaceBillingSettings: vi.fn(), mockGetWorkspaceBilledAccountUserId: vi.fn(), mockGetWorkspaceOrganizationId: vi.fn(), - mockGetOrgAdminWorkspaceRows: vi.fn(), mockListAccessibleWorkspaceRowsForUser: vi.fn(), mockTransferWorkspaceOwnershipToBilledAccountForMemberRemovalTx: vi.fn(), mockReassignWorkflowOwnershipForWorkspaceMemberRemovalTx: vi.fn(), @@ -74,7 +73,6 @@ export const workspacesUtilsMock = { getWorkspaceBillingSettings: workspacesUtilsMockFns.mockGetWorkspaceBillingSettings, getWorkspaceBilledAccountUserId: workspacesUtilsMockFns.mockGetWorkspaceBilledAccountUserId, getWorkspaceOrganizationId: workspacesUtilsMockFns.mockGetWorkspaceOrganizationId, - getOrgAdminWorkspaceRows: workspacesUtilsMockFns.mockGetOrgAdminWorkspaceRows, listAccessibleWorkspaceRowsForUser: workspacesUtilsMockFns.mockListAccessibleWorkspaceRowsForUser, transferWorkspaceOwnershipToBilledAccountForMemberRemovalTx: workspacesUtilsMockFns.mockTransferWorkspaceOwnershipToBilledAccountForMemberRemovalTx, From 81d5f82aabd942f9618842833758a881076f62f5 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 16:27:20 -0700 Subject: [PATCH 27/30] chore(cleanup): consolidate shared helpers and restore TSDoc on moved codec and subflow helpers --- apps/docs/components/ui/block-info-card.tsx | 28 +---------- apps/sim/app/api/v1/auth.ts | 7 ++- .../components/note-block/note-block.tsx | 4 +- .../components/tool-input/tool-input.tsx | 20 ++++---- .../panel/components/editor/editor.tsx | 19 ++++---- .../utils/block-protection-utils.ts | 8 ++-- .../[workspaceId]/w/[workflowId]/workflow.tsx | 6 +-- apps/sim/blocks/icon-color.ts | 9 ++-- apps/sim/blocks/pi-api-key-condition.test.ts | 2 +- .../fork-sync-detail-view.tsx | 2 +- .../components/fork-sync/fork-sync-view.tsx | 2 +- .../human-in-the-loop-handler.ts | 12 ++--- .../executor/utils/subflow-node-id-codec.ts | 28 +++++++++++ .../hooks/queries/workflow-search-replace.ts | 22 ++++----- apps/sim/hooks/use-collaborative-workflow.ts | 16 ++++--- .../lib/api/server/routes/v2-api-key-auth.ts | 10 ++-- apps/sim/lib/audit-logs/query.ts | 4 +- apps/sim/lib/mcp/shared.ts | 2 +- .../lib/table/columns/workflow-references.ts | 2 +- .../lib/table/query-builder/field-names.ts | 2 +- apps/sim/lib/table/rows/cursor.ts | 2 +- apps/sim/lib/table/rows/run-state.ts | 2 +- apps/sim/lib/table/rows/service.ts | 2 +- apps/sim/lib/webhooks/deployed-urls.test.ts | 4 +- apps/sim/lib/webhooks/deployed-urls.ts | 3 +- .../webhooks/provider-subscription-utils.ts | 4 +- apps/sim/lib/webhooks/trigger-url.ts | 11 +++++ .../application/read-workflow-definition.ts | 4 +- .../application/run-workflow-from-copilot.ts | 4 +- .../comparison/format-description.test.ts | 3 +- .../lib/workflows/editing/validation.test.ts | 8 ++-- apps/sim/lib/workflows/editing/validation.ts | 2 +- .../workflows/sanitization/json-sanitizer.ts | 7 +-- apps/sim/providers/azure-openai/index.ts | 2 +- apps/sim/providers/baseten/index.ts | 2 +- apps/sim/providers/cerebras/index.ts | 2 +- apps/sim/providers/fireworks/index.ts | 2 +- apps/sim/providers/groq/index.ts | 2 +- apps/sim/providers/meta/index.ts | 2 +- apps/sim/providers/mistral/index.ts | 2 +- apps/sim/providers/nvidia/index.ts | 2 +- apps/sim/providers/openrouter/index.ts | 3 +- apps/sim/providers/response-format.ts | 30 ++++++++++++ apps/sim/providers/sakana/index.ts | 2 +- apps/sim/providers/utils.ts | 46 +------------------ apps/sim/providers/vllm/index.ts | 2 +- apps/sim/stores/workflows/workflow/store.ts | 14 ++++-- .../stores/workflows/workflow/utils.test.ts | 33 ++++--------- apps/sim/stores/workflows/workflow/utils.ts | 28 ----------- apps/sim/tools/params-resolver.ts | 25 ---------- apps/sim/triggers/webhook-url.ts | 5 -- .../testing/src/mocks/providers-utils.mock.ts | 19 +------- packages/testing/src/mocks/uploads.mock.ts | 5 +- .../src/mocks/workspace-file-manager.mock.ts | 6 +-- .../src/subflow-helpers.test.ts | 25 ++++++++++ .../src/subflow-helpers.ts | 34 ++++++++++++++ 56 files changed, 265 insertions(+), 289 deletions(-) create mode 100644 apps/sim/lib/webhooks/trigger-url.ts create mode 100644 apps/sim/providers/response-format.ts delete mode 100644 apps/sim/tools/params-resolver.ts create mode 100644 packages/workflow-persistence/src/subflow-helpers.test.ts diff --git a/apps/docs/components/ui/block-info-card.tsx b/apps/docs/components/ui/block-info-card.tsx index d20380c3d93..5bd1bc7e802 100644 --- a/apps/docs/components/ui/block-info-card.tsx +++ b/apps/docs/components/ui/block-info-card.tsx @@ -1,6 +1,7 @@ 'use client' import type * as React from 'react' +import { isLightTileColor } from '@sim/workflow-renderer/tile-icon-color' import { blockTypeToIconMap } from '@/components/ui/icon-mapping' interface BlockInfoCardProps { @@ -9,33 +10,6 @@ interface BlockInfoCardProps { icon?: React.ComponentType<{ className?: string }> } -/** - * Brightness above which a tile background is "clearly light" and a white - * foreground icon would wash out. Mirrors apps/sim's LIGHT_TILE_THRESHOLD - * (blocks/icon-color.ts) so monochrome `currentColor` icons (e.g. Daytona, - * Notion) stay legible on white/pale tiles instead of white-on-white. - */ -const LIGHT_TILE_THRESHOLD = 0.75 - -function isLightTileColor(color: string): boolean { - const hex = color.trim().replace('#', '').toLowerCase() - let r: number - let g: number - let b: number - if (/^[0-9a-f]{3}$/.test(hex)) { - r = Number.parseInt(hex[0] + hex[0], 16) - g = Number.parseInt(hex[1] + hex[1], 16) - b = Number.parseInt(hex[2] + hex[2], 16) - } else if (/^[0-9a-f]{6}$/.test(hex)) { - r = Number.parseInt(hex.slice(0, 2), 16) - g = Number.parseInt(hex.slice(2, 4), 16) - b = Number.parseInt(hex.slice(4, 6), 16) - } else { - return false - } - return (0.299 * r + 0.587 * g + 0.114 * b) / 255 > LIGHT_TILE_THRESHOLD -} - export function BlockInfoCard({ type, color, diff --git a/apps/sim/app/api/v1/auth.ts b/apps/sim/app/api/v1/auth.ts index c75f00cd1ec..bb94b6952ea 100644 --- a/apps/sim/app/api/v1/auth.ts +++ b/apps/sim/app/api/v1/auth.ts @@ -1,5 +1,8 @@ -import type { PersonalApiKeyPrincipal, WorkspaceApiKeyPrincipal } from '@sim/auth/principal' -import { ANONYMOUS_USER_ID } from '@sim/auth/principal' +import { + ANONYMOUS_USER_ID, + type PersonalApiKeyPrincipal, + type WorkspaceApiKeyPrincipal, +} from '@sim/auth/principal' import { createLogger } from '@sim/logger' import type { NextRequest } from 'next/server' import { authenticateApiKeyFromHeader, updateApiKeyLastUsed } from '@/lib/api-key/service' diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/note-block/note-block.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/note-block/note-block.tsx index 8127f54e481..2a1593298bb 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/note-block/note-block.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/note-block/note-block.tsx @@ -13,6 +13,7 @@ import { type NoteSearchHighlight, type NoteSearchRange, } from '@sim/workflow-renderer' +import { isWorkflowBlockProtected } from '@sim/workflow-types/workflow' import { type Node, type NodeProps, useReactFlow } from '@xyflow/react' import dynamic from 'next/dynamic' import { useShallow } from 'zustand/react/shallow' @@ -28,7 +29,6 @@ import { useNoteImageUpload } from '@/app/workspace/[workspaceId]/w/[workflowId] import type { WorkflowBlockProps } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/workflow-block/types' import { useBlockVisual } from '@/app/workspace/[workspaceId]/w/[workflowId]/hooks' import { useBlockDimensions } from '@/app/workspace/[workspaceId]/w/[workflowId]/hooks/use-block-dimensions' -import { isBlockProtected } from '@/app/workspace/[workspaceId]/w/[workflowId]/utils' import { useCollaborativeWorkflow } from '@/hooks/use-collaborative-workflow' import { useIsCurrentWorkflowExecuting } from '@/stores/execution' import { usePanelEditorSearchStore, usePanelEditorStore } from '@/stores/panel' @@ -109,7 +109,7 @@ export const NoteBlock = memo(function NoteBlock({ id, data, selected }: NodePro const isWorkflowRunning = useIsCurrentWorkflowExecuting() const canEditWorkflow = userPermissions.canEdit && !data.isWorkflowLocked const isProtected = useWorkflowStore( - useCallback((state) => isBlockProtected(id, state.blocks), [id]) + useCallback((state) => isWorkflowBlockProtected(id, state.blocks), [id]) ) const clearCurrentBlock = usePanelEditorStore((state) => state.clearCurrentBlock) /* Flattened to primitives under a shallow compare, never held as the target diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx index a9bada878cb..bb82ec4e2d5 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/components/sub-block/components/tool-input/tool-input.tsx @@ -34,6 +34,16 @@ import { OPERATION_SUBBLOCK_ID, } from '@/lib/permission-groups/operation-access' import { resolveStoredToolName } from '@/lib/workflows/subblocks/display' +import { + buildCanonicalIndex, + type CanonicalIndex, + type CanonicalModeOverrides, + isCanonicalPair, + reindexToolCanonicalModes, + resolveCanonicalMode, + resolveDependencyValue, + scopeCanonicalModesForTool, +} from '@/lib/workflows/subblocks/visibility' import { buildToolSubBlockId } from '@/lib/workflows/tool-input/synthetic-subblocks' import { buildAgentToolUsageControlCanonicalKey, @@ -104,16 +114,6 @@ import { isUserFacingToolParam, type SubBlocksForToolInput, } from '@/tools/params' -import { - buildCanonicalIndex, - type CanonicalIndex, - type CanonicalModeOverrides, - isCanonicalPair, - reindexToolCanonicalModes, - resolveCanonicalMode, - resolveDependencyValue, - scopeCanonicalModesForTool, -} from '@/tools/params-resolver' const logger = createLogger('ToolInput') diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/editor.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/editor.tsx index 8edfbb54b46..ca5a2c44052 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/editor.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/components/panel/components/editor/editor.tsx @@ -12,7 +12,11 @@ import { SquareArrowUpRight, Unlock, } from '@sim/emcn/icons' -import type { BlockRetryConfig } from '@sim/workflow-types/workflow' +import { + type BlockRetryConfig, + isWorkflowBlockAncestorLocked, + isWorkflowBlockProtected, +} from '@sim/workflow-types/workflow' import { isEqual } from 'es-toolkit' import { useParams } from 'next/navigation' import { usePostHog } from 'posthog-js/react' @@ -50,10 +54,6 @@ import { LoopTool } from '@/app/workspace/[workspaceId]/w/[workflowId]/component import { ParallelTool } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/subflows/parallel/parallel-config' import { getSubBlockStableKey } from '@/app/workspace/[workspaceId]/w/[workflowId]/components/workflow-block/utils' import { useCurrentWorkflow } from '@/app/workspace/[workspaceId]/w/[workflowId]/hooks' -import { - isAncestorProtected, - isBlockProtected, -} from '@/app/workspace/[workspaceId]/w/[workflowId]/utils/block-protection-utils' import { PreviewWorkflow } from '@/app/workspace/[workspaceId]/w/components/preview' import { BlockTile } from '@/blocks/block-tile' import { getBlock } from '@/blocks/registry' @@ -138,8 +138,10 @@ export function Editor() { // Check if block is locked (or inside a locked ancestor) and compute edit permission // Locked blocks cannot be edited by anyone (admins can only lock/unlock) const blocks = useWorkflowStore((state) => state.blocks) - const isLocked = currentBlockId ? isBlockProtected(currentBlockId, blocks) : false - const isAncestorLocked = currentBlockId ? isAncestorProtected(currentBlockId, blocks) : false + const isLocked = currentBlockId ? isWorkflowBlockProtected(currentBlockId, blocks) : false + const isAncestorLocked = currentBlockId + ? isWorkflowBlockAncestorLocked(currentBlockId, blocks) + : false const canEditBlock = userPermissions.canEdit && !workflowLocked && !isLocked const { advancedMode, triggerMode } = useEditorBlockProperties( @@ -332,7 +334,8 @@ export function Editor() { const block = blocks[blockId] if (!block) return - if (!userPermissions.canEdit || workflowLocked || isBlockProtected(blockId, blocks)) return + if (!userPermissions.canEdit || workflowLocked || isWorkflowBlockProtected(blockId, blocks)) + return renamingBlockIdRef.current = blockId setEditedName(block.name || '') diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/block-protection-utils.ts b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/block-protection-utils.ts index ddf80d336b6..35d759b6e15 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/block-protection-utils.ts +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/utils/block-protection-utils.ts @@ -1,7 +1,5 @@ +import { isWorkflowBlockProtected } from '@sim/workflow-types/workflow' import type { BlockState } from '@/stores/workflows/workflow/types' -import { isAncestorProtected, isBlockProtected } from '@/stores/workflows/workflow/utils' - -export { isAncestorProtected, isBlockProtected } /** * Result of filtering protected blocks from a deletion operation @@ -28,7 +26,7 @@ export function isEdgeProtected( edge: { source: string; target: string }, blocks: Record ): boolean { - return isBlockProtected(edge.target, blocks) + return isWorkflowBlockProtected(edge.target, blocks) } /** @@ -43,7 +41,7 @@ export function filterProtectedBlocks( blockIds: string[], blocks: Record ): FilterProtectedBlocksResult { - const protectedIds = blockIds.filter((id) => isBlockProtected(id, blocks)) + const protectedIds = blockIds.filter((id) => isWorkflowBlockProtected(id, blocks)) const deletableIds = blockIds.filter((id) => !protectedIds.includes(id)) return { diff --git a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx index 8dbddd716de..ac508e892e2 100644 --- a/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx +++ b/apps/sim/app/workspace/[workspaceId]/w/[workflowId]/workflow.tsx @@ -1,6 +1,7 @@ 'use client' import React, { lazy, Suspense, useCallback, useEffect, useMemo, useRef, useState } from 'react' +import { isWorkflowBlockProtected } from '@sim/workflow-types/workflow' import { applyNodeChanges, ConnectionLineType, @@ -102,7 +103,6 @@ import { getNodeSelectionContextId, getRunFromBlockDependencyState, getWorkflowLockToggleIds, - isBlockProtected, isEdgeProtected, isInEditableElement, isPositionalTriggerBlock, @@ -2890,7 +2890,7 @@ const WorkflowContent = React.memo( className: block.data?.parentId ? SUBFLOW_CHILD_NODE_CLASS : undefined, extent: block.data?.extent || undefined, dragHandle: '.workflow-drag-handle', - draggable: !workflowReadOnly && !isBlockProtected(block.id, blocks), + draggable: !workflowReadOnly && !isWorkflowBlockProtected(block.id, blocks), zIndex: depth, data: { ...block.data, @@ -2937,7 +2937,7 @@ const WorkflowContent = React.memo( parentId, className: parentId ? SUBFLOW_CHILD_NODE_CLASS : undefined, dragHandle, - draggable: !workflowReadOnly && !isBlockProtected(block.id, blocks), + draggable: !workflowReadOnly && !isWorkflowBlockProtected(block.id, blocks), zIndex: cardZIndex, extent: (() => { // Clamp children to subflow body (exclude header) diff --git a/apps/sim/blocks/icon-color.ts b/apps/sim/blocks/icon-color.ts index 6f85b343c65..37f7e7afd2b 100644 --- a/apps/sim/blocks/icon-color.ts +++ b/apps/sim/blocks/icon-color.ts @@ -1,8 +1,9 @@ /** - * Contrast helpers for brand tiles. Pure colour maths — deliberately free of any - * `@/blocks/registry` import so the public landing `/integrations` page can use - * these without pulling 282 block configs and the tool registry into its bundle. - * Registry-backed icon styling lives in `@/blocks/brand-icon`. + * Tailwind classes for brand-tile icons, built on the shared + * `isLightTileColor` predicate. Deliberately free of any `@/blocks/registry` + * import so the public landing `/integrations` page can use it without pulling + * every block config and the tool registry into its bundle. Registry-backed + * icon styling lives in `@/blocks/brand-icon`. */ import { isLightTileColor } from '@sim/workflow-renderer/tile-icon-color' diff --git a/apps/sim/blocks/pi-api-key-condition.test.ts b/apps/sim/blocks/pi-api-key-condition.test.ts index 706ecbb1b57..8f7dcb6dc4c 100644 --- a/apps/sim/blocks/pi-api-key-condition.test.ts +++ b/apps/sim/blocks/pi-api-key-condition.test.ts @@ -2,7 +2,7 @@ import { resetEnvFlagsMock, setEnvFlags } from '@sim/testing' import { afterAll, afterEach, beforeEach, describe, expect, it } from 'vitest' import { evaluateSubBlockCondition } from '@/lib/workflows/subblocks/visibility' import { PiBlock } from '@/blocks/blocks/pi' -import { getHostedModels } from '@/providers/utils' +import { getHostedModels } from '@/providers/models' const apiKeySubBlock = PiBlock.subBlocks.find((subBlock) => subBlock.id === 'apiKey') const hostedModel = getHostedModels()[0] diff --git a/apps/sim/ee/workspace-forking/components/fork-sync-detail-view/fork-sync-detail-view.tsx b/apps/sim/ee/workspace-forking/components/fork-sync-detail-view/fork-sync-detail-view.tsx index 8afda8d40c0..6cea66844c2 100644 --- a/apps/sim/ee/workspace-forking/components/fork-sync-detail-view/fork-sync-detail-view.tsx +++ b/apps/sim/ee/workspace-forking/components/fork-sync-detail-view/fork-sync-detail-view.tsx @@ -6,6 +6,7 @@ import { ArrowLeft } from '@sim/emcn/icons' import { useQueryState } from 'nuqs' import { saveDiscardActions } from '@/components/settings/save-discard-actions' import type { SettingsAction } from '@/components/settings/settings-header' +import { buildWebhookTriggerUrl } from '@/lib/webhooks/trigger-url' import { UnsavedChangesModal } from '@/app/workspace/[workspaceId]/components/credential-detail' import { forkSyncDirectionParam, @@ -19,7 +20,6 @@ import { useForkSync, } from '@/ee/workspace-forking/components/fork-sync/use-fork-sync' import type { ForkDirection } from '@/ee/workspace-forking/hooks/workspace-fork' -import { buildWebhookTriggerUrl } from '@/triggers/webhook-url' interface ForkSyncDetailViewProps { title: string diff --git a/apps/sim/ee/workspace-forking/components/fork-sync/fork-sync-view.tsx b/apps/sim/ee/workspace-forking/components/fork-sync/fork-sync-view.tsx index df4cf34dc98..fe2d8c765cc 100644 --- a/apps/sim/ee/workspace-forking/components/fork-sync/fork-sync-view.tsx +++ b/apps/sim/ee/workspace-forking/components/fork-sync/fork-sync-view.tsx @@ -24,6 +24,7 @@ import type { ForkTriggerMapping, } from '@/lib/api/contracts/workspace-fork' import type { SelectorKey } from '@/lib/selectors/manifest' +import { buildWebhookTriggerUrl } from '@/lib/webhooks/trigger-url' import { SettingsEmptyState } from '@/app/workspace/[workspaceId]/settings/components/settings-empty-state' import { SettingsSection } from '@/app/workspace/[workspaceId]/settings/components/settings-section/settings-section' import { @@ -59,7 +60,6 @@ import type { } from '@/ee/workspace-forking/components/fork-sync/use-fork-sync' import type { ForkDirection } from '@/ee/workspace-forking/hooks/workspace-fork' import { forkSyncBlockerReasonFor } from '@/ee/workspace-forking/lib/promote/sync-blockers' -import { buildWebhookTriggerUrl } from '@/triggers/webhook-url' /** * Copyable kinds as expandable rows in the "Copy resources" section, ordered + labeled to match diff --git a/apps/sim/executor/handlers/human-in-the-loop/human-in-the-loop-handler.ts b/apps/sim/executor/handlers/human-in-the-loop/human-in-the-loop-handler.ts index 7cb94a0290a..1ac7e5d5a8f 100644 --- a/apps/sim/executor/handlers/human-in-the-loop/human-in-the-loop-handler.ts +++ b/apps/sim/executor/handlers/human-in-the-loop/human-in-the-loop-handler.ts @@ -1,7 +1,12 @@ import { createLogger } from '@sim/logger' import { isRecordLike } from '@sim/utils/object' import { getBaseUrl } from '@/lib/core/utils/urls' -import type { CanonicalGroup } from '@/lib/workflows/subblocks/visibility' +import { + buildCanonicalIndex, + type CanonicalGroup, + isCanonicalPair, + scopeCanonicalModesForTool, +} from '@/lib/workflows/subblocks/visibility' import { getBlock } from '@/blocks/registry' import type { BlockOutput } from '@/blocks/types' import { @@ -25,11 +30,6 @@ import { parseObjectStrings } from '@/executor/utils/json' import { buildBlockToolParamsTransform } from '@/providers/utils' import type { SerializedBlock } from '@/serializer/types' import { executeTool } from '@/tools' -import { - buildCanonicalIndex, - isCanonicalPair, - scopeCanonicalModesForTool, -} from '@/tools/params-resolver' import { getTool } from '@/tools/utils' const logger = createLogger('HumanInTheLoopBlockHandler') diff --git a/apps/sim/executor/utils/subflow-node-id-codec.ts b/apps/sim/executor/utils/subflow-node-id-codec.ts index 7705cac9df0..4a62d3d89d3 100644 --- a/apps/sim/executor/utils/subflow-node-id-codec.ts +++ b/apps/sim/executor/utils/subflow-node-id-codec.ts @@ -103,6 +103,9 @@ export function extractParallelIdFromSentinel(sentinelId: string): string | null return null } +/** + * Builds a branch node ID with subscript notation, e.g. `("blockId", 2)` → `"blockId₍2₎"`. + */ export function buildBranchNodeId(baseId: string, branchIndex: number): string { return `${baseId}${PARALLEL.BRANCH.PREFIX}${branchIndex}${PARALLEL.BRANCH.SUFFIX}` } @@ -120,6 +123,10 @@ export function isBranchNodeId(nodeId: string): boolean { return BRANCH.MATCH.test(nodeId) } +/** + * Extracts the outer branch index from a cloned subflow ID (`{originalId}__obranch-{index}`). + * Returns undefined when the ID is not a clone. + */ export function extractOuterBranchIndex(clonedId: string): number | undefined { const match = clonedId.match(OUTER_BRANCH.MATCH) return match ? Number.parseInt(match[1], 10) : undefined @@ -131,14 +138,25 @@ export function extractInnermostOuterBranchIndex(clonedId: string): number | und return lastMatch ? Number.parseInt(lastMatch[1], 10) : undefined } +/** + * Strips outer-branch suffixes (`__obranch-N`), clone digests (`__clone{hex}`), and the + * trailing branch subscript (`₍N₎`) from a node ID, returning the workflow-level block ID. + */ export function stripCloneSuffixes(nodeId: string): string { return extractBaseBlockId(nodeId.replace(OUTER_BRANCH.STRIP, '').replace(CLONE.DIGEST_STRIP, '')) } +/** + * Builds a stable ID scoped to a global outer parallel branch; also the ID of a cloned subflow. + */ export function buildOuterBranchScopedId(originalId: string, branchIndex: number): string { return `${originalId}__obranch-${branchIndex}` } +/** + * Strips outer-branch suffixes (`__obranch-N`) and clone digests (`__clone{hex}`) from an ID, + * returning the workflow-level subflow ID. + */ export function stripOuterBranchSuffix(id: string): string { return id.replace(OUTER_BRANCH.STRIP, '').replace(CLONE.DIGEST_STRIP, '') } @@ -160,6 +178,16 @@ export function normalizeNodeId(nodeId: string): string { return nodeId } +/** + * Finds the effective (possibly cloned) container ID for a subflow, given the current node's ID + * and an execution map (`loopExecutions` or `parallelExecutions`). + * + * Inside a cloned subflow (e.g. `loop-1__obranch-2`) the execution scope is stored under the + * cloned ID, not the original. This reads the `__obranch-N` suffix from `mappedBranchIndex` or the + * current node ID, builds the candidate cloned container ID, and checks the map for it. + * + * @returns The effective ID (cloned or original) that exists in the map, else `originalId`. + */ export function findEffectiveContainerId( originalId: string, currentNodeId: string, diff --git a/apps/sim/hooks/queries/workflow-search-replace.ts b/apps/sim/hooks/queries/workflow-search-replace.ts index a559778a0ff..f24b4428d98 100644 --- a/apps/sim/hooks/queries/workflow-search-replace.ts +++ b/apps/sim/hooks/queries/workflow-search-replace.ts @@ -419,9 +419,7 @@ export function useWorkflowSearchMcpServerDetails( const serversQuery = useQuery({ queryKey: workflowSearchReplaceKeys.mcpServerListDetails(workspaceId), - queryFn: async ({ signal }: { signal: AbortSignal }) => { - return fetchWorkspaceMcpServers(workspaceId as string, signal) - }, + queryFn: ({ signal }) => fetchWorkspaceMcpServers(workspaceId as string, signal), enabled: Boolean(workspaceId && serverMatches.length > 0), staleTime: WORKFLOW_SEARCH_MCP_SERVER_LIST_STALE_TIME, }) @@ -454,9 +452,7 @@ export function useWorkflowSearchMcpToolDetails( const toolsQuery = useQuery({ queryKey: workflowSearchReplaceKeys.mcpToolListDetails(workspaceId), - queryFn: async ({ signal }: { signal: AbortSignal }) => { - return fetchWorkspaceMcpTools(workspaceId as string, signal) - }, + queryFn: ({ signal }) => fetchWorkspaceMcpTools(workspaceId as string, signal), enabled: Boolean(workspaceId && toolMatches.length > 0), staleTime: WORKFLOW_SEARCH_MCP_TOOL_LIST_STALE_TIME, }) @@ -721,13 +717,12 @@ export function useWorkflowSearchMcpServerReplacementOptions( queries: [ { queryKey: workflowSearchReplaceKeys.mcpServerReplacementOptions(workspaceId), - queryFn: async ({ + queryFn: ({ signal, }: { signal: AbortSignal - }): Promise => { - return fetchWorkspaceMcpServers(workspaceId as string, signal) - }, + }): Promise => + fetchWorkspaceMcpServers(workspaceId as string, signal), enabled: Boolean(workspaceId && serverGroups.length > 0), staleTime: WORKFLOW_SEARCH_MCP_SERVER_REPLACEMENT_STALE_TIME, select: ( @@ -773,13 +768,12 @@ export function useWorkflowSearchMcpToolReplacementOptions( queries: [ { queryKey: workflowSearchReplaceKeys.mcpToolReplacementOptions(workspaceId), - queryFn: async ({ + queryFn: ({ signal, }: { signal: AbortSignal - }): Promise => { - return fetchWorkspaceMcpTools(workspaceId as string, signal) - }, + }): Promise => + fetchWorkspaceMcpTools(workspaceId as string, signal), enabled: Boolean(workspaceId && toolGroups.length > 0), staleTime: WORKFLOW_SEARCH_MCP_TOOL_REPLACEMENT_STALE_TIME, select: ( diff --git a/apps/sim/hooks/use-collaborative-workflow.ts b/apps/sim/hooks/use-collaborative-workflow.ts index 0c3f528fa26..90b8a978f97 100644 --- a/apps/sim/hooks/use-collaborative-workflow.ts +++ b/apps/sim/hooks/use-collaborative-workflow.ts @@ -13,7 +13,11 @@ import { } from '@sim/realtime-protocol/constants' import { generateId } from '@sim/utils/id' import type { BlockRetryConfig } from '@sim/workflow-types/workflow' -import { filterAcyclicEdges, getWorkflowBlockNameConflict } from '@sim/workflow-types/workflow' +import { + filterAcyclicEdges, + getWorkflowBlockNameConflict, + isWorkflowBlockProtected, +} from '@sim/workflow-types/workflow' import { useQueryClient } from '@tanstack/react-query' import type { Edge } from '@xyflow/react' import { isEqual } from 'es-toolkit' @@ -56,7 +60,7 @@ import type { Position, WorkflowState, } from '@/stores/workflows/workflow/types' -import { findAllDescendantNodes, isBlockProtected } from '@/stores/workflows/workflow/utils' +import { findAllDescendantNodes } from '@/stores/workflows/workflow/utils' const logger = createLogger('CollaborativeWorkflow') @@ -1102,7 +1106,7 @@ export function useCollaborativeWorkflow() { const block = blocks[id] if (block) { - if (isBlockProtected(id, blocks)) { + if (isWorkflowBlockProtected(id, blocks)) { logger.error('Cannot rename locked block') toast({ message: 'Cannot rename locked blocks' }) return { success: false, error: 'Block is locked' } @@ -1199,14 +1203,14 @@ export function useCollaborativeWorkflow() { if (!block) continue // Skip protected blocks (locked or inside a locked ancestor) - if (isBlockProtected(id, currentBlocks)) continue + if (isWorkflowBlockProtected(id, currentBlocks)) continue validIds.push(id) previousStates[id] = block.enabled // If it's a loop or parallel, also capture descendants' previous states for undo/redo if (block.type === 'loop' || block.type === 'parallel') { findAllDescendantNodes(id, currentBlocks).forEach((descId) => { - if (!isBlockProtected(descId, currentBlocks)) { + if (!isWorkflowBlockProtected(descId, currentBlocks)) { previousStates[descId] = currentBlocks[descId]?.enabled ?? true } }) @@ -1402,7 +1406,7 @@ export function useCollaborativeWorkflow() { for (const id of ids) { const block = blocks[id] - if (block && !isBlockProtected(id, blocks)) { + if (block && !isWorkflowBlockProtected(id, blocks)) { previousStates[id] = block.horizontalHandles ?? false validIds.push(id) } diff --git a/apps/sim/lib/api/server/routes/v2-api-key-auth.ts b/apps/sim/lib/api/server/routes/v2-api-key-auth.ts index 7e7b5f79a1f..e84b81a73df 100644 --- a/apps/sim/lib/api/server/routes/v2-api-key-auth.ts +++ b/apps/sim/lib/api/server/routes/v2-api-key-auth.ts @@ -1,9 +1,9 @@ -import type { - OAuthAccessTokenPrincipal, - PersonalApiKeyPrincipal, - WorkspaceApiKeyPrincipal, +import { + ANONYMOUS_USER_ID, + type OAuthAccessTokenPrincipal, + type PersonalApiKeyPrincipal, + type WorkspaceApiKeyPrincipal, } from '@sim/auth/principal' -import { ANONYMOUS_USER_ID } from '@sim/auth/principal' import { db } from '@sim/db' import { apiKey, user } from '@sim/db/schema' import { createLogger } from '@sim/logger' diff --git a/apps/sim/lib/audit-logs/query.ts b/apps/sim/lib/audit-logs/query.ts index b55605982d5..9c84d67751a 100644 --- a/apps/sim/lib/audit-logs/query.ts +++ b/apps/sim/lib/audit-logs/query.ts @@ -1,6 +1,7 @@ import { AuditResourceType } from '@sim/audit' import { db, dbReplica } from '@sim/db' import { auditLog, workspace } from '@sim/db/schema' +import { escapeLikePattern } from '@sim/utils/string' import type { InferSelectModel } from 'drizzle-orm' import { and, desc, eq, gte, ilike, inArray, isNull, lt, lte, or, type SQL, sql } from 'drizzle-orm' import { parseUnorderedList } from '@/lib/api/cursor-binding' @@ -70,8 +71,7 @@ export function buildFilterConditions(params: AuditLogFilterParams): SQL` shape the domain type and the published contract diff --git a/apps/sim/lib/table/rows/service.ts b/apps/sim/lib/table/rows/service.ts index 233812fa612..51bb937de90 100644 --- a/apps/sim/lib/table/rows/service.ts +++ b/apps/sim/lib/table/rows/service.ts @@ -1,4 +1,3 @@ -import { escapeLikePattern } from '@sim/utils/string' /** * Row CRUD + query operations for the table service layer. * @@ -15,6 +14,7 @@ import { db } from '@sim/db' import { userTableRows } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { generateId } from '@sim/utils/id' +import { escapeLikePattern } from '@sim/utils/string' import { and, asc, count, eq, inArray, type SQL, sql } from 'drizzle-orm' import { OrchestrationError } from '@/lib/core/orchestration/types' import { diff --git a/apps/sim/lib/webhooks/deployed-urls.test.ts b/apps/sim/lib/webhooks/deployed-urls.test.ts index c636694eff2..f1d7faf1976 100644 --- a/apps/sim/lib/webhooks/deployed-urls.test.ts +++ b/apps/sim/lib/webhooks/deployed-urls.test.ts @@ -2,8 +2,10 @@ import { webhook } from '@sim/db/schema' import { dbChainMockFns, queueTableRows, resetDbChainMock } from '@sim/testing' import { afterAll, beforeEach, describe, expect, it, vi } from 'vitest' -vi.mock('@/triggers/webhook-url', () => ({ +vi.mock('@/lib/webhooks/trigger-url', () => ({ buildWebhookTriggerUrl: (path: string) => `https://sim.test/api/webhooks/trigger/${path}`, +})) +vi.mock('@/triggers/webhook-url', () => ({ buildSlackCustomBotRequestUrl: (credentialId: string) => `https://sim.test/api/webhooks/slack/custom/${credentialId}`, })) diff --git a/apps/sim/lib/webhooks/deployed-urls.ts b/apps/sim/lib/webhooks/deployed-urls.ts index 0d3f5137e53..04321a13033 100644 --- a/apps/sim/lib/webhooks/deployed-urls.ts +++ b/apps/sim/lib/webhooks/deployed-urls.ts @@ -4,7 +4,8 @@ import { isRecordLike } from '@sim/utils/object' import { and, eq, isNull } from 'drizzle-orm' import type { DbOrTx } from '@/lib/db/types' import { LEGACY_SLACK_CUSTOM_BOT_INGRESS_MODE } from '@/lib/webhooks/slack-custom-ingress-constants' -import { buildSlackCustomBotRequestUrl, buildWebhookTriggerUrl } from '@/triggers/webhook-url' +import { buildWebhookTriggerUrl } from '@/lib/webhooks/trigger-url' +import { buildSlackCustomBotRequestUrl } from '@/triggers/webhook-url' /** The public URL one live webhook registration receives events on, and the block it feeds. */ export interface DeployedWebhookUrl { diff --git a/apps/sim/lib/webhooks/provider-subscription-utils.ts b/apps/sim/lib/webhooks/provider-subscription-utils.ts index c26047a90c6..71b84f2c9f7 100644 --- a/apps/sim/lib/webhooks/provider-subscription-utils.ts +++ b/apps/sim/lib/webhooks/provider-subscription-utils.ts @@ -3,8 +3,8 @@ import { account } from '@sim/db/schema' import { createLogger } from '@sim/logger' import { toRecord } from '@sim/utils/object' import { eq } from 'drizzle-orm' -import { getBaseUrl } from '@/lib/core/utils/urls' import { refreshAccessTokenIfNeeded, resolveOAuthAccountId } from '@/lib/oauth/credential-service' +import { buildWebhookTriggerUrl } from '@/lib/webhooks/trigger-url' const logger = createLogger('WebhookProviderSubscriptions') @@ -15,7 +15,7 @@ export function getProviderConfig(webhook: Record): Record): string { - return `${getBaseUrl()}/api/webhooks/trigger/${webhook.path}` + return buildWebhookTriggerUrl(String(webhook.path)) } /** diff --git a/apps/sim/lib/webhooks/trigger-url.ts b/apps/sim/lib/webhooks/trigger-url.ts new file mode 100644 index 00000000000..044dccc337d --- /dev/null +++ b/apps/sim/lib/webhooks/trigger-url.ts @@ -0,0 +1,11 @@ +import { getBaseUrl } from '@/lib/core/utils/urls' + +/** + * The public URL an external system POSTs to for a given webhook path. + * + * A leaf module on purpose: provider subscription handlers and client views both need it, and + * `@/triggers/webhook-url` pulls in the block and trigger registries. + */ +export function buildWebhookTriggerUrl(path: string): string { + return `${getBaseUrl()}/api/webhooks/trigger/${path}` +} diff --git a/apps/sim/lib/workflows/application/read-workflow-definition.ts b/apps/sim/lib/workflows/application/read-workflow-definition.ts index d11e7b25a1c..3389eae9f80 100644 --- a/apps/sim/lib/workflows/application/read-workflow-definition.ts +++ b/apps/sim/lib/workflows/application/read-workflow-definition.ts @@ -1,7 +1,7 @@ import type { NormalizedWorkflowData } from '@sim/workflow-persistence/types' import { OrchestrationError } from '@/lib/core/orchestration/types' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import type { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' +import type { ActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { @@ -18,7 +18,7 @@ export interface ReadWorkflowDefinitionInput { } export interface ReadWorkflowDefinitionResult { - workflow: Awaited>['workflow'] + workflow: ActiveWorkflowApplicationContext['workflow'] workspaceId: string state: NormalizedWorkflowData | DeployedWorkflowData | null } diff --git a/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts b/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts index 9309695690c..acbec79fafa 100644 --- a/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts +++ b/apps/sim/lib/workflows/application/run-workflow-from-copilot.ts @@ -8,7 +8,7 @@ import type { BillingAttributionSnapshot } from '@/lib/billing/core/billing-attr import { OrchestrationError } from '@/lib/core/orchestration/types' import { generateRequestId } from '@/lib/core/utils/request' import { defineAuthorizedWorkflowUseCase } from '@/lib/workflows/application/authorized-workflow-use-case' -import type { resolveActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' +import type { ActiveWorkflowApplicationContext } from '@/lib/workflows/application/context' import { workflowOperations } from '@/lib/workflows/application/operations' import { resolvePrincipalWorkflowContext } from '@/lib/workflows/application/principal-scope' import { prepareWorkflowExecutionAdmission } from '@/lib/workflows/execution-admission' @@ -227,7 +227,7 @@ async function resolveSourceSnapshot(input: SnapshotCopilotRunInput): Promise<{ async function executeCopilotRun(params: { principal: Principal input: BaseCopilotRunInput - context: Awaited> + context: ActiveWorkflowApplicationContext executionInput: unknown triggerBlockId?: string stopAfterBlockId?: string diff --git a/apps/sim/lib/workflows/comparison/format-description.test.ts b/apps/sim/lib/workflows/comparison/format-description.test.ts index 0f5739f3ca5..fd84f5b606b 100644 --- a/apps/sim/lib/workflows/comparison/format-description.test.ts +++ b/apps/sim/lib/workflows/comparison/format-description.test.ts @@ -33,6 +33,7 @@ vi.mock('@/lib/selectors/client/execute-selector', () => ({ })) import { WorkflowBuilder } from '@sim/testing' +import type { WorkflowState } from '@sim/workflow-types/workflow' import type { WorkflowDiffSummary } from '@/lib/workflows/comparison/compare' import { formatDiffSummaryForDescriptionAsync } from '@/lib/workflows/comparison/describe' import { @@ -117,7 +118,7 @@ describe('resolveValueForDisplay', () => { }) describe('formatDiffSummaryForDescriptionAsync shared formatting', () => { - const state = { blocks: {} } as any + const state: WorkflowState = { blocks: {}, edges: [], loops: {}, parallels: {} } it('uses human-readable field labels for modified blocks', async () => { mockGetBlock.mockReturnValue({ diff --git a/apps/sim/lib/workflows/editing/validation.test.ts b/apps/sim/lib/workflows/editing/validation.test.ts index 410c753ecc0..f83de885a0d 100644 --- a/apps/sim/lib/workflows/editing/validation.test.ts +++ b/apps/sim/lib/workflows/editing/validation.test.ts @@ -4,7 +4,7 @@ import { integrationsAvailabilityMock, integrationsAvailabilityMockFns, } from '@sim/testing/mocks/integrations-availability.mock' -import { providersUtilsMock, providersUtilsMockFns } from '@sim/testing/mocks/providers-utils.mock' +import { providersUtilsMock } from '@sim/testing/mocks/providers-utils.mock' import { tableServiceMock, tableServiceMockFns } from '@sim/testing/mocks/table-service.mock' import type { WorkflowState } from '@sim/workflow-types/workflow' import type { Mock } from 'vitest' @@ -279,6 +279,7 @@ vi.mock('@/providers/utils', () => providersUtilsMock) vi.mock('@/lib/integrations/availability.server', () => integrationsAvailabilityMock) import { buildWorkflowLintReport } from '@/lib/workflows/editing/lint-report' +import * as providerModels from '@/providers/models' import { collectUnresolvedAgentToolReferences, collectUnresolvedReferences, @@ -297,8 +298,6 @@ tableServiceMockFns.mockGetTableById.mockResolvedValue(null) const mockGetBlock = getBlock as Mock mockGetBlock.mockImplementation((type: string) => blockConfigsByType[type]) -const mockGetHostedModels = providersUtilsMockFns.mockGetHostedModels - const CTX = { userId: 'user-1', workspaceId: 'workspace-1' } afterAll(resetEnvFlagsMock) @@ -836,12 +835,11 @@ describe('preValidateCredentialInputs (hosted-tool blocks)', () => { describe('preValidateCredentialInputs (hosted models)', () => { beforeEach(() => { mockValidateSelectorIds.mockResolvedValue({ valid: [], invalid: [] }) - mockGetHostedModels.mockReturnValue(['claude-sonnet-4-6']) + vi.spyOn(providerModels, 'getHostedModels').mockReturnValue(['claude-sonnet-4-6']) setEnvFlags({ isHosted: true }) }) afterEach(() => { - mockGetHostedModels.mockReset() setEnvFlags({ isHosted: false }) }) diff --git a/apps/sim/lib/workflows/editing/validation.ts b/apps/sim/lib/workflows/editing/validation.ts index 2e9bab4f840..2e38c7dd9e6 100644 --- a/apps/sim/lib/workflows/editing/validation.ts +++ b/apps/sim/lib/workflows/editing/validation.ts @@ -41,6 +41,7 @@ import { getModelOptions } from '@/blocks/utils' import { overlayVisibility } from '@/blocks/visibility/context' import { BlockType, EDGE, normalizeName } from '@/executor/constants' import { + getHostedModels, isAutoModel, isCustomModelId, isKnownModelId, @@ -1570,7 +1571,6 @@ export async function preValidateCredentialInputs( workflowState?: Record ): Promise<{ filteredOperations: EditWorkflowOperation[]; errors: ValidationError[] }> { const { isHosted } = await import('@/lib/core/config/env-flags') - const { getHostedModels } = await import('@/providers/utils') const logger = createLogger('PreValidateCredentials') const errors: ValidationError[] = [] diff --git a/apps/sim/lib/workflows/sanitization/json-sanitizer.ts b/apps/sim/lib/workflows/sanitization/json-sanitizer.ts index 51b94488781..3025cf9f878 100644 --- a/apps/sim/lib/workflows/sanitization/json-sanitizer.ts +++ b/apps/sim/lib/workflows/sanitization/json-sanitizer.ts @@ -5,6 +5,7 @@ import { } from '@sim/workflow-persistence/subflow-helpers' import { normalizeWorkflowEdgeSourceHandle } from '@sim/workflow-types/workflow' import type { Edge } from '@xyflow/react' +import { buildWebhookTriggerUrl } from '@/lib/webhooks/trigger-url' import { sanitizeWorkflowForSharing } from '@/lib/workflows/credentials/credential-extractor' import { getBlock } from '@/blocks/registry' import type { @@ -15,11 +16,7 @@ import type { WorkflowState, } from '@/stores/workflows/workflow/types' import { TRIGGER_ROUTING_FIELD, TRIGGER_WEBHOOK_URL_FIELD } from '@/triggers/constants' -import { - blockAdvertisesWebhookUrl, - buildWebhookTriggerUrl, - resolveBlockTriggerId, -} from '@/triggers/webhook-url' +import { blockAdvertisesWebhookUrl, resolveBlockTriggerId } from '@/triggers/webhook-url' /** * Sanitized workflow state for copilot (removes all UI-specific data) diff --git a/apps/sim/providers/azure-openai/index.ts b/apps/sim/providers/azure-openai/index.ts index 9b84b384641..8d78341ca36 100644 --- a/apps/sim/providers/azure-openai/index.ts +++ b/apps/sim/providers/azure-openai/index.ts @@ -41,6 +41,7 @@ import { getProviderDefaultModel, getProviderModels } from '@/providers/models' import { executeResponsesProviderRequest } from '@/providers/openai/core' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -57,7 +58,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, checkForForcedToolUsageOpenAI, isFunctionToolCall, diff --git a/apps/sim/providers/baseten/index.ts b/apps/sim/providers/baseten/index.ts index 7fd51d58611..3d6ffacfe78 100644 --- a/apps/sim/providers/baseten/index.ts +++ b/apps/sim/providers/baseten/index.ts @@ -7,9 +7,9 @@ import { type ChatCompletionPayload, executeChatCompletionRequest, } from '@/providers/openai-compat/chat-completions' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { openAICompatTransport } from '@/providers/transport' import type { ProviderConfig, ProviderRequest, ProviderResponse } from '@/providers/types' -import { buildJsonSchemaResponseFormat } from '@/providers/utils' const logger = createLogger('BasetenProvider') diff --git a/apps/sim/providers/cerebras/index.ts b/apps/sim/providers/cerebras/index.ts index 317cacfb489..2a957482b07 100644 --- a/apps/sim/providers/cerebras/index.ts +++ b/apps/sim/providers/cerebras/index.ts @@ -18,6 +18,7 @@ import { getProviderDefaultModel, getProviderModels } from '@/providers/models' import { createOpenAICompatAssistantHistory } from '@/providers/openai-compat/assistant-history' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -33,7 +34,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, diff --git a/apps/sim/providers/fireworks/index.ts b/apps/sim/providers/fireworks/index.ts index 28e31fb77bd..53c340ce485 100644 --- a/apps/sim/providers/fireworks/index.ts +++ b/apps/sim/providers/fireworks/index.ts @@ -8,9 +8,9 @@ import { type ChatCompletionPayload, executeChatCompletionRequest, } from '@/providers/openai-compat/chat-completions' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { openAICompatTransport } from '@/providers/transport' import type { ProviderConfig, ProviderRequest, ProviderResponse } from '@/providers/types' -import { buildJsonSchemaResponseFormat } from '@/providers/utils' const logger = createLogger('FireworksProvider') diff --git a/apps/sim/providers/groq/index.ts b/apps/sim/providers/groq/index.ts index 638291cb196..e5cd22d9e26 100644 --- a/apps/sim/providers/groq/index.ts +++ b/apps/sim/providers/groq/index.ts @@ -23,6 +23,7 @@ import { getProviderDefaultModel, getProviderModels } from '@/providers/models' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' import { createOpenAICompatStreamingToolLoopStream } from '@/providers/openai-compat/streaming-tool-loop' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createStreamingExecution } from '@/providers/streaming-execution' import { isAbortError, parseToolArguments } from '@/providers/streaming-tool-loop-shared' @@ -37,7 +38,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, diff --git a/apps/sim/providers/meta/index.ts b/apps/sim/providers/meta/index.ts index dfa5ff07061..d3366bf794c 100644 --- a/apps/sim/providers/meta/index.ts +++ b/apps/sim/providers/meta/index.ts @@ -17,6 +17,7 @@ import { import { getProviderDefaultModel, getProviderModels } from '@/providers/models' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -32,7 +33,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, diff --git a/apps/sim/providers/mistral/index.ts b/apps/sim/providers/mistral/index.ts index 66a901e398b..32e9f424e64 100644 --- a/apps/sim/providers/mistral/index.ts +++ b/apps/sim/providers/mistral/index.ts @@ -17,6 +17,7 @@ import { import { getProviderDefaultModel, getProviderModels } from '@/providers/models' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -32,7 +33,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, diff --git a/apps/sim/providers/nvidia/index.ts b/apps/sim/providers/nvidia/index.ts index 234f637c92c..eae48b4506c 100644 --- a/apps/sim/providers/nvidia/index.ts +++ b/apps/sim/providers/nvidia/index.ts @@ -22,6 +22,7 @@ import { import { createOpenAICompatAssistantHistory } from '@/providers/openai-compat/assistant-history' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -37,7 +38,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, generateSchemaInstructions, isFunctionToolCall, diff --git a/apps/sim/providers/openrouter/index.ts b/apps/sim/providers/openrouter/index.ts index 3d69a0155a3..75fdaa11f69 100644 --- a/apps/sim/providers/openrouter/index.ts +++ b/apps/sim/providers/openrouter/index.ts @@ -13,9 +13,10 @@ import { getOpenRouterModelCapabilities, supportsNativeStructuredOutputs, } from '@/providers/openrouter/utils' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { openAICompatTransport } from '@/providers/transport' import type { ProviderConfig, ProviderRequest, ProviderResponse } from '@/providers/types' -import { buildJsonSchemaResponseFormat, generateSchemaInstructions } from '@/providers/utils' +import { generateSchemaInstructions } from '@/providers/utils' const logger = createLogger('OpenRouterProvider') diff --git a/apps/sim/providers/response-format.ts b/apps/sim/providers/response-format.ts new file mode 100644 index 00000000000..672df7bb468 --- /dev/null +++ b/apps/sim/providers/response-format.ts @@ -0,0 +1,30 @@ +import type { ProviderRequest } from '@/providers/types' + +/** OpenAI-compatible `json_schema` response format sent to chat-completions providers. */ +export interface JsonSchemaResponseFormat { + type: 'json_schema' + json_schema: { + name: string + schema: Record + strict?: boolean + } +} + +/** + * Builds the OpenAI-compatible `json_schema` response format from a request's `responseFormat`. + * Strict mode is on unless the caller set `strict: false`; `includeStrict: false` leaves the + * `strict` flag out of the payload entirely. + */ +export function buildJsonSchemaResponseFormat( + responseFormat: NonNullable, + { includeStrict = true }: { includeStrict?: boolean } = {} +): JsonSchemaResponseFormat { + return { + type: 'json_schema', + json_schema: { + name: responseFormat.name || 'response_schema', + schema: responseFormat.schema || responseFormat, + ...(includeStrict ? { strict: responseFormat.strict !== false } : {}), + }, + } +} diff --git a/apps/sim/providers/sakana/index.ts b/apps/sim/providers/sakana/index.ts index 98bb49c7b8e..d88521715ac 100644 --- a/apps/sim/providers/sakana/index.ts +++ b/apps/sim/providers/sakana/index.ts @@ -18,6 +18,7 @@ import { getProviderDefaultModel, getProviderModels } from '@/providers/models' import { createOpenAICompatAssistantHistory } from '@/providers/openai-compat/assistant-history' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -33,7 +34,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, isFunctionToolCall, prepareToolExecution, diff --git a/apps/sim/providers/utils.ts b/apps/sim/providers/utils.ts index ef9640a3a67..1ede85f3a4f 100644 --- a/apps/sim/providers/utils.ts +++ b/apps/sim/providers/utils.ts @@ -28,7 +28,7 @@ import { isCustomTool } from '@/executor/constants' import { findProviderFromModel, getComputerUseModels, - getHostedModels as getHostedModelsFromDefinitions, + getHostedModels, getModelsWithDeepResearch, getModelsWithoutMemory, getModelsWithPromptCaching, @@ -51,12 +51,7 @@ import { getProviderToolModelInputRegistry, registerPreparedProviderToolInputProvenance, } from '@/providers/tool-input-provenance' -import type { - ModelPricing, - ProviderId, - ProviderRequest, - ProviderToolConfig, -} from '@/providers/types' +import type { ModelPricing, ProviderId, ProviderToolConfig } from '@/providers/types' import { mergeToolParameters } from '@/tools/merge-params' import { buildToolParamShapes, decodeToolParams } from '@/tools/param-shape' import type { WorkflowToolExecutionContext } from '@/tools/types' @@ -327,35 +322,6 @@ export function filterBlacklistedModels(models: string[]): string[] { return models.filter((model) => !isModelBlacklisted(model)) } -/** OpenAI-compatible `json_schema` response format sent to chat-completions providers. */ -export interface JsonSchemaResponseFormat { - type: 'json_schema' - json_schema: { - name: string - schema: Record - strict?: boolean - } -} - -/** - * Builds the OpenAI-compatible `json_schema` response format from a request's `responseFormat`. - * Strict mode is on unless the caller set `strict: false`; `includeStrict: false` leaves the - * `strict` flag out of the payload entirely. - */ -export function buildJsonSchemaResponseFormat( - responseFormat: NonNullable, - { includeStrict = true }: { includeStrict?: boolean } = {} -): JsonSchemaResponseFormat { - return { - type: 'json_schema', - json_schema: { - name: responseFormat.name || 'response_schema', - schema: responseFormat.schema || responseFormat, - ...(includeStrict ? { strict: responseFormat.strict !== false } : {}), - }, - } -} - /** * Generates prompt instructions for structured JSON output from a JSON schema. * Used as a fallback when native structured outputs are not supported. @@ -1103,14 +1069,6 @@ export function formatCost(cost: number): string { return formatCreditCost(cost) ?? '—' } -/** - * Get the list of models that are hosted by the platform (don't require user API keys) - * These are the models for which we hide the API key field in the hosted environment - */ -export function getHostedModels(): string[] { - return getHostedModelsFromDefinitions() -} - /** * Determine if model usage should be billed to the user * diff --git a/apps/sim/providers/vllm/index.ts b/apps/sim/providers/vllm/index.ts index d53be59a7a0..881813b5b09 100644 --- a/apps/sim/providers/vllm/index.ts +++ b/apps/sim/providers/vllm/index.ts @@ -23,6 +23,7 @@ import { createOpenAICompatAssistantHistory } from '@/providers/openai-compat/as import { getOpenAICompatibleApiBaseUrl } from '@/providers/openai-compat/base-url' import { getChatCompletionConversationUsage } from '@/providers/openai-compat/conversation-usage' import { createOpenAICompatibleAgentEventStream } from '@/providers/openai-compat/stream-events' +import { buildJsonSchemaResponseFormat } from '@/providers/response-format' import { executeProviderTool } from '@/providers/runtime-context' import { createSettledAgentEventStream } from '@/providers/stream-events' import { createStreamingExecution } from '@/providers/streaming-execution' @@ -39,7 +40,6 @@ import type { } from '@/providers/types' import { ProviderError } from '@/providers/types' import { - buildJsonSchemaResponseFormat, calculateCost, checkForForcedToolUsageOpenAI, isFunctionToolCall, diff --git a/apps/sim/stores/workflows/workflow/store.ts b/apps/sim/stores/workflows/workflow/store.ts index 6305b9d354e..716ade55cd5 100644 --- a/apps/sim/stores/workflows/workflow/store.ts +++ b/apps/sim/stores/workflows/workflow/store.ts @@ -6,7 +6,11 @@ import { generateParallelBlocks, } from '@sim/workflow-persistence/subflow-helpers' import type { BlockRetryConfig } from '@sim/workflow-types/workflow' -import { filterAcyclicEdges, getWorkflowBlockNameConflict } from '@sim/workflow-types/workflow' +import { + filterAcyclicEdges, + getWorkflowBlockNameConflict, + isWorkflowBlockProtected, +} from '@sim/workflow-types/workflow' import type { Edge } from '@xyflow/react' import { create } from 'zustand' import { devtools } from 'zustand/middleware' @@ -31,7 +35,7 @@ import type { WorkflowState, WorkflowStore, } from '@/stores/workflows/workflow/types' -import { findAllDescendantNodes, isBlockProtected } from '@/stores/workflows/workflow/utils' +import { findAllDescendantNodes } from '@/stores/workflows/workflow/utils' import { normalizeWorkflowState } from '@/stores/workflows/workflow/validation' const logger = createLogger('WorkflowStore') @@ -353,14 +357,14 @@ export const useWorkflowStore = create()( if (!block) continue // Skip protected blocks entirely (locked or inside a locked ancestor) - if (isBlockProtected(id, currentBlocks)) continue + if (isWorkflowBlockProtected(id, currentBlocks)) continue blocksToToggle.add(id) // If it's a loop or parallel, also include non-locked descendants if (block.type === 'loop' || block.type === 'parallel') { findAllDescendantNodes(id, currentBlocks).forEach((descId) => { - if (!isBlockProtected(descId, currentBlocks)) { + if (!isWorkflowBlockProtected(descId, currentBlocks)) { blocksToToggle.add(descId) } }) @@ -389,7 +393,7 @@ export const useWorkflowStore = create()( const newBlocks = { ...currentBlocks } for (const id of ids) { - if (!newBlocks[id] || isBlockProtected(id, currentBlocks)) continue + if (!newBlocks[id] || isWorkflowBlockProtected(id, currentBlocks)) continue newBlocks[id] = { ...newBlocks[id], horizontalHandles: !newBlocks[id].horizontalHandles, diff --git a/apps/sim/stores/workflows/workflow/utils.test.ts b/apps/sim/stores/workflows/workflow/utils.test.ts index e73f3ed4d6d..c92f1b3f078 100644 --- a/apps/sim/stores/workflows/workflow/utils.test.ts +++ b/apps/sim/stores/workflows/workflow/utils.test.ts @@ -1,27 +1,10 @@ import { createAgentBlock, createLoopBlock } from '@sim/testing' -import { convertLoopBlockToLoop } from '@sim/workflow-persistence/subflow-helpers' +import { + isWorkflowBlockAncestorLocked, + isWorkflowBlockProtected, +} from '@sim/workflow-types/workflow' import { describe, expect, it } from 'vitest' import type { BlockState } from '@/stores/workflows/workflow/types' -import { isAncestorProtected, isBlockProtected } from '@/stores/workflows/workflow/utils' - -describe('convertLoopBlockToLoop', () => { - it.concurrent('should keep string as-is if not valid JSON', () => { - const blocks: Record = { - loop1: createLoopBlock({ - id: 'loop1', - name: 'Test Loop', - loopType: 'forEach', - count: 5, - data: { collection: '' }, - }), - } - - const result = convertLoopBlockToLoop('loop1', blocks) - - expect(result).toBeDefined() - expect(result?.forEachItems).toBe('') - }) -}) describe('block lock protection', () => { it.concurrent('treats deeply nested blocks inside locked containers as protected', () => { @@ -43,8 +26,8 @@ describe('block lock protection', () => { }), } - expect(isAncestorProtected('child', blocks)).toBe(true) - expect(isBlockProtected('child', blocks)).toBe(true) + expect(isWorkflowBlockAncestorLocked('child', blocks)).toBe(true) + expect(isWorkflowBlockProtected('child', blocks)).toBe(true) }) it.concurrent( @@ -63,8 +46,8 @@ describe('block lock protection', () => { }), } - expect(isAncestorProtected('first', blocks)).toBe(false) - expect(isBlockProtected('first', blocks)).toBe(false) + expect(isWorkflowBlockAncestorLocked('first', blocks)).toBe(false) + expect(isWorkflowBlockProtected('first', blocks)).toBe(false) } ) }) diff --git a/apps/sim/stores/workflows/workflow/utils.ts b/apps/sim/stores/workflows/workflow/utils.ts index 37d466b61e5..15894a651f4 100644 --- a/apps/sim/stores/workflows/workflow/utils.ts +++ b/apps/sim/stores/workflows/workflow/utils.ts @@ -1,7 +1,3 @@ -import { - isWorkflowBlockAncestorLocked, - isWorkflowBlockProtected, -} from '@sim/workflow-types/workflow' import type { BlockState } from '@/stores/workflows/workflow/types' /** @@ -31,27 +27,3 @@ export function findAllDescendantNodes( } return descendants } - -/** - * Checks if any ancestor container of a block is locked. - * Unlike {@link isBlockProtected}, this ignores the block's own locked state. - * - * @param blockId - The ID of the block to check - * @param blocks - Record of all blocks in the workflow - * @returns True if any ancestor is locked - */ -export function isAncestorProtected(blockId: string, blocks: Record): boolean { - return isWorkflowBlockAncestorLocked(blockId, blocks) -} - -/** - * Checks if a block is protected from editing/deletion. - * A block is protected if it is locked or if any ancestor container is locked. - * - * @param blockId - The ID of the block to check - * @param blocks - Record of all blocks in the workflow - * @returns True if the block is protected - */ -export function isBlockProtected(blockId: string, blocks: Record): boolean { - return isWorkflowBlockProtected(blockId, blocks) -} diff --git a/apps/sim/tools/params-resolver.ts b/apps/sim/tools/params-resolver.ts deleted file mode 100644 index 49f241e7aa5..00000000000 --- a/apps/sim/tools/params-resolver.ts +++ /dev/null @@ -1,25 +0,0 @@ -import { - buildCanonicalIndex, - type CanonicalIndex, - type CanonicalModeOverrides, - evaluateSubBlockCondition, - isCanonicalPair, - reindexToolCanonicalModes, - resolveCanonicalMode, - resolveDependencyValue, - type SubBlockCondition, - scopeCanonicalModesForTool, -} from '@/lib/workflows/subblocks/visibility' - -export { - buildCanonicalIndex, - type CanonicalIndex, - type CanonicalModeOverrides, - evaluateSubBlockCondition, - isCanonicalPair, - reindexToolCanonicalModes, - resolveCanonicalMode, - resolveDependencyValue, - scopeCanonicalModesForTool, - type SubBlockCondition, -} diff --git a/apps/sim/triggers/webhook-url.ts b/apps/sim/triggers/webhook-url.ts index 6f28289f084..4397adbdac6 100644 --- a/apps/sim/triggers/webhook-url.ts +++ b/apps/sim/triggers/webhook-url.ts @@ -7,11 +7,6 @@ import { getBlock } from '@/blocks/registry' import type { BlockState } from '@/stores/workflows/workflow/types' import { getTrigger, isTriggerValid } from '@/triggers' -/** The public URL an external system POSTs to for a given webhook path. */ -export function buildWebhookTriggerUrl(path: string): string { - return `${getBaseUrl()}/api/webhooks/trigger/${path}` -} - /** * The Request URL a Slack custom-bot app posts events to. One URL per * credential (not per workflow): the endpoint verifies with the credential's diff --git a/packages/testing/src/mocks/providers-utils.mock.ts b/packages/testing/src/mocks/providers-utils.mock.ts index 5fb73fecd3d..d0309338198 100644 --- a/packages/testing/src/mocks/providers-utils.mock.ts +++ b/packages/testing/src/mocks/providers-utils.mock.ts @@ -41,8 +41,7 @@ interface MockForcedToolUsage { * - `enforceStrictSchema` and `filterBlacklistedModels` are identity; `isProviderBlacklisted`, * `shouldBillModelUsage` and every `supports*`/`is*Model` capability check → `false`; * - `getProviderFromModel` → `'openai'`; - * - `generateSchemaInstructions` → `'SCHEMA_INSTRUCTIONS'`, `buildJsonSchemaResponseFormat` is the - * real `json_schema` payload builder; + * - `generateSchemaInstructions` → `'SCHEMA_INSTRUCTIONS'`; * - `getApiKey` returns the user key, else the real `PROVIDER_PLACEHOLDER_KEY`; * - the `MODELS_WITH_*` catalog constants are empty arrays and `providers` is `{}`. * @@ -69,19 +68,6 @@ export const providersUtilsMockFns = { mockGetAllProviderIds: vi.fn((): string[] => []), mockIsProviderBlacklisted: vi.fn((_providerId: string): boolean => false), mockFilterBlacklistedModels: vi.fn((models: string[]): string[] => models), - mockBuildJsonSchemaResponseFormat: vi.fn( - ( - responseFormat: { name?: string; schema?: unknown; strict?: boolean }, - { includeStrict = true }: { includeStrict?: boolean } = {} - ) => ({ - type: 'json_schema' as const, - json_schema: { - name: responseFormat.name || 'response_schema', - schema: responseFormat.schema || responseFormat, - ...(includeStrict ? { strict: responseFormat.strict !== false } : {}), - }, - }) - ), mockGenerateSchemaInstructions: vi.fn( (_schema: unknown, _schemaName?: string): string => 'SCHEMA_INSTRUCTIONS' ), @@ -104,7 +90,6 @@ export const providersUtilsMockFns = { }), mockGetModelPricing: vi.fn((_modelId: string): unknown => null), mockFormatCost: vi.fn((_cost: number): string => '—'), - mockGetHostedModels: vi.fn((): string[] => []), mockShouldBillModelUsage: vi.fn((_model: string): boolean => false), mockGetApiKey: vi.fn( (_provider: string, _model: string, userProvidedKey?: string): string => @@ -195,7 +180,6 @@ export const providersUtilsMock = { getAllProviderIds: providersUtilsMockFns.mockGetAllProviderIds, isProviderBlacklisted: providersUtilsMockFns.mockIsProviderBlacklisted, filterBlacklistedModels: providersUtilsMockFns.mockFilterBlacklistedModels, - buildJsonSchemaResponseFormat: providersUtilsMockFns.mockBuildJsonSchemaResponseFormat, generateSchemaInstructions: providersUtilsMockFns.mockGenerateSchemaInstructions, generateStructuredOutputInstructions: providersUtilsMockFns.mockGenerateStructuredOutputInstructions, @@ -207,7 +191,6 @@ export const providersUtilsMock = { sumToolCosts: providersUtilsMockFns.mockSumToolCosts, getModelPricing: providersUtilsMockFns.mockGetModelPricing, formatCost: providersUtilsMockFns.mockFormatCost, - getHostedModels: providersUtilsMockFns.mockGetHostedModels, shouldBillModelUsage: providersUtilsMockFns.mockShouldBillModelUsage, getApiKey: providersUtilsMockFns.mockGetApiKey, prepareToolsWithUsageControl: providersUtilsMockFns.mockPrepareToolsWithUsageControl, diff --git a/packages/testing/src/mocks/uploads.mock.ts b/packages/testing/src/mocks/uploads.mock.ts index 70fd18b3ad7..4d43558fd9a 100644 --- a/packages/testing/src/mocks/uploads.mock.ts +++ b/packages/testing/src/mocks/uploads.mock.ts @@ -5,8 +5,9 @@ import { storageServiceMock } from './storage-service.mock' * Controllable mock functions for the `@/lib/uploads` barrel. * * Defaults describe local storage: `isUsingCloudStorage` → `false`, - * `getServePathPrefix` → `'/api/files/serve/'` (the real constant). The `StorageService` namespace is `storageServiceMock`, so drive storage I/O - * through `storageServiceMockFns` from `@sim/testing/mocks/storage-service.mock`. + * `getServePathPrefix` → `'/api/files/serve/'` (the real constant). The `StorageService` + * namespace is `storageServiceMock`, so drive storage I/O through `storageServiceMockFns` + * from `@sim/testing/mocks/storage-service.mock`. * * @example * ```ts diff --git a/packages/testing/src/mocks/workspace-file-manager.mock.ts b/packages/testing/src/mocks/workspace-file-manager.mock.ts index 81a8336fb36..2dd10d8b6e0 100644 --- a/packages/testing/src/mocks/workspace-file-manager.mock.ts +++ b/packages/testing/src/mocks/workspace-file-manager.mock.ts @@ -98,9 +98,9 @@ function vfsPath(file: MockWorkspaceFilePathFields): string { * `matchesWorkspaceFilePattern`, `parseWorkspaceFileKey`, `suffixedName` are faithful; * the VFS helpers (`workspaceFileVfsPath`, `getSandboxWorkspaceFilePath`, * `parseChatUploadReference`, `findWorkspaceFileRecord`) percent-encode with - * `encodeURIComponent` and split folder paths on `/` (no escaped-slash folder names). `generateWorkspaceFileKey` is deterministic — - * `workspace/{workspaceId}/generated-{fileName}` — which `parseWorkspaceFileKey` does - * not recognize (no timestamp/random segment). + * `encodeURIComponent` and split folder paths on `/` (no escaped-slash folder names). + * `generateWorkspaceFileKey` is deterministic — `workspace/{workspaceId}/generated-{fileName}` + * — which `parseWorkspaceFileKey` does not recognize (no timestamp/random segment). * * @example * ```ts diff --git a/packages/workflow-persistence/src/subflow-helpers.test.ts b/packages/workflow-persistence/src/subflow-helpers.test.ts new file mode 100644 index 00000000000..1ee7777c46d --- /dev/null +++ b/packages/workflow-persistence/src/subflow-helpers.test.ts @@ -0,0 +1,25 @@ +import type { BlockState } from '@sim/workflow-types/workflow' +import { describe, expect, it } from 'vitest' +import { convertLoopBlockToLoop } from './subflow-helpers' + +describe('convertLoopBlockToLoop', () => { + it.concurrent('should keep string as-is if not valid JSON', () => { + const blocks: Record = { + loop1: { + id: 'loop1', + type: 'loop', + name: 'Test Loop', + position: { x: 0, y: 0 }, + subBlocks: {}, + outputs: {}, + enabled: true, + data: { loopType: 'forEach', count: 5, collection: '' }, + }, + } + + const result = convertLoopBlockToLoop('loop1', blocks) + + expect(result).toBeDefined() + expect(result?.forEachItems).toBe('') + }) +}) diff --git a/packages/workflow-persistence/src/subflow-helpers.ts b/packages/workflow-persistence/src/subflow-helpers.ts index 18b1a42320c..fb44c8c12d2 100644 --- a/packages/workflow-persistence/src/subflow-helpers.ts +++ b/packages/workflow-persistence/src/subflow-helpers.ts @@ -4,6 +4,7 @@ const DEFAULT_LOOP_ITERATIONS = 5 const DEFAULT_PARALLEL_BATCH_SIZE = 20 const MAX_PARALLEL_BATCH_SIZE = 20 +/** Clamps a parallel block's batch size to 1..20, defaulting to 20 when it is not a number. */ export function clampParallelBatchSize(batchSize: unknown): number { const parsed = typeof batchSize === 'number' ? batchSize : Number.parseInt(String(batchSize), 10) if (Number.isNaN(parsed)) { @@ -12,12 +13,26 @@ export function clampParallelBatchSize(batchSize: unknown): number { return Math.max(1, Math.min(MAX_PARALLEL_BATCH_SIZE, parsed)) } +/** + * Finds the direct children of a loop or parallel container. + * + * @param containerId - ID of the container to find children for + * @param blocks - Record of all blocks in the workflow + * @returns IDs of the blocks whose parent is this container + */ export function findChildNodes(containerId: string, blocks: Record): string[] { return Object.values(blocks) .filter((block) => block.data?.parentId === containerId) .map((block) => block.id) } +/** + * Converts a loop block into the executor's {@link Loop} format. + * + * @param loopBlockId - ID of the loop block to convert + * @param blocks - Record of all blocks in the workflow + * @returns The loop, or undefined when the block is missing or not a loop + */ export function convertLoopBlockToLoop( loopBlockId: string, blocks: Record @@ -42,6 +57,13 @@ export function convertLoopBlockToLoop( return loop } +/** + * Converts a parallel block into the executor's {@link Parallel} format. + * + * @param parallelBlockId - ID of the parallel block to convert + * @param blocks - Record of all blocks in the workflow + * @returns The parallel, or undefined when the block is missing or not a parallel + */ export function convertParallelBlockToParallel( parallelBlockId: string, blocks: Record @@ -73,6 +95,12 @@ export function convertParallelBlockToParallel( } } +/** + * Builds every loop in a workflow from its loop blocks. + * + * @param blocks - Record of all blocks in the workflow + * @returns Loops keyed by block ID + */ export function generateLoopBlocks(blocks: Record): Record { const loops: Record = {} @@ -88,6 +116,12 @@ export function generateLoopBlocks(blocks: Record): Record ): Record { From 8e326cf214bc55ae2c2cb2ccaaa35187e88025f4 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 16:28:22 -0700 Subject: [PATCH 28/30] test(cleanup): port storage-cleanup ownership tests and drop stale mock keys --- .../documents/storage-cleanup.test.ts | 67 +++++++++++++++++++ apps/sim/lib/logs/log-views.test.ts | 2 +- .../lib/table/bulk-update-concurrency.test.ts | 1 - apps/sim/lib/table/find-row-matches.test.ts | 1 - .../rows/bulk-update-patch-validation.test.ts | 1 - 5 files changed, 68 insertions(+), 4 deletions(-) diff --git a/apps/sim/lib/knowledge/documents/storage-cleanup.test.ts b/apps/sim/lib/knowledge/documents/storage-cleanup.test.ts index e0e98d6b0e8..711ee6e0f43 100644 --- a/apps/sim/lib/knowledge/documents/storage-cleanup.test.ts +++ b/apps/sim/lib/knowledge/documents/storage-cleanup.test.ts @@ -41,6 +41,26 @@ const payload = { workspaceId: binding.workspaceId, organizationId: null, } +const SOURCE_WORKSPACE_ID = 'workspace-1' +const SOURCE_KEY = `workspace/${SOURCE_WORKSPACE_ID}/source.pdf` +const SOURCE_URL = `/api/files/serve/${encodeURIComponent(SOURCE_KEY)}?context=workspace` +const SOURCE_UPDATED_AT = new Date('2026-08-05T12:00:00.000Z') +const SOURCE_BINDING = { + id: 'source-file-1', + key: SOURCE_KEY, + userId: 'source-user', + workspaceId: SOURCE_WORKSPACE_ID, + context: 'workspace', + originalName: 'source.pdf', + displayName: 'source.pdf', + contentType: 'application/pdf', + size: 512, + folderId: null, + uploadedAt: SOURCE_UPDATED_AT, + contentUpdatedAt: SOURCE_UPDATED_AT, + deletedAt: null, + secretProvenanceVersion: 1, +} function context(): OutboxEventContext { return { eventId: 'cleanup-1', @@ -179,4 +199,51 @@ describe('durable knowledge storage cleanup', () => { expect(dbChainMockFns.transaction).not.toHaveBeenCalled() expect(mockDeleteFile).not.toHaveBeenCalled() }) + it('never deletes a referenced workspace source as knowledge-base storage', async () => { + await enqueueKnowledgeStorageCleanup( + db, + [{ id: 'document-1', fileUrl: SOURCE_URL, workspaceId: SOURCE_WORKSPACE_ID }], + 'request-1' + ) + + expect(mockGetBindings).not.toHaveBeenCalled() + expect(mockDeleteFile).not.toHaveBeenCalled() + expect(mockDeleteMetadata).not.toHaveBeenCalled() + }) + + it.each(['org-1', 'org-2', null])( + 'only queues an organization cache for its exact owner: %s', + async (organizationId) => { + const storageKey = 'kb/org-source.pdf' + mockGetBindings.mockResolvedValue([ + { + ...SOURCE_BINDING, + key: storageKey, + context: 'knowledge-base', + workspaceId: null, + organizationId: 'org-1', + }, + ]) + const cleanup = enqueueKnowledgeStorageCleanup( + db, + [ + { + id: 'org-doc', + fileUrl: `/api/files/serve/${encodeURIComponent(storageKey)}`, + workspaceId: null, + organizationId, + }, + ], + 'request-1' + ) + if (organizationId === 'org-1') { + await cleanup + expect(dbChainMockFns.values).toHaveBeenCalledOnce() + } else { + await expect(cleanup).rejects.toThrow() + expect(dbChainMockFns.values).not.toHaveBeenCalled() + } + expect(mockDeleteFile).not.toHaveBeenCalled() + } + ) }) diff --git a/apps/sim/lib/logs/log-views.test.ts b/apps/sim/lib/logs/log-views.test.ts index 50771da77f2..27509339a91 100644 --- a/apps/sim/lib/logs/log-views.test.ts +++ b/apps/sim/lib/logs/log-views.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from 'vitest' +import { toOverview } from '@/lib/logs/log-views' import type { TraceSpan } from '@/lib/logs/types' -import { toOverview } from './log-views' const ref = (preview: unknown) => ({ __sim: 'ref', preview, size: 100 }) diff --git a/apps/sim/lib/table/bulk-update-concurrency.test.ts b/apps/sim/lib/table/bulk-update-concurrency.test.ts index ecfa636c891..97b0d326b52 100644 --- a/apps/sim/lib/table/bulk-update-concurrency.test.ts +++ b/apps/sim/lib/table/bulk-update-concurrency.test.ts @@ -29,7 +29,6 @@ vi.mock('@/lib/table/sql', () => ({ buildFilterClause: vi.fn(() => sql`true`), buildPredicateClause: vi.fn(() => sql`true`), buildSortClause: vi.fn(() => sql`true`), - escapeLikePattern: vi.fn((value: string) => value), uniqueValuePredicate: vi.fn(() => sql`true`), })) diff --git a/apps/sim/lib/table/find-row-matches.test.ts b/apps/sim/lib/table/find-row-matches.test.ts index 4e44ec956eb..df46ef8e3cc 100644 --- a/apps/sim/lib/table/find-row-matches.test.ts +++ b/apps/sim/lib/table/find-row-matches.test.ts @@ -14,7 +14,6 @@ import type { ColumnDefinition, TableDefinition } from '@/lib/table/types' vi.mock('@/lib/table/sql', () => ({ buildFilterClause: vi.fn(() => sql`true`), buildSortClause: vi.fn(() => sql`true`), - escapeLikePattern: vi.fn((s: string) => s), })) vi.mock('@/lib/table/trigger', () => tableTriggerMock) diff --git a/apps/sim/lib/table/rows/bulk-update-patch-validation.test.ts b/apps/sim/lib/table/rows/bulk-update-patch-validation.test.ts index 61f18fa16a3..c8209c7283b 100644 --- a/apps/sim/lib/table/rows/bulk-update-patch-validation.test.ts +++ b/apps/sim/lib/table/rows/bulk-update-patch-validation.test.ts @@ -31,7 +31,6 @@ vi.mock('@/lib/table/sql', () => ({ buildFilterClause: vi.fn(() => sql`true`), buildPredicateClause: vi.fn(() => sql`true`), buildSortClause: vi.fn(() => sql`true`), - escapeLikePattern: vi.fn((value: string) => value), uniqueValuePredicate: vi.fn(() => sql`true`), })) From cd2bd67d5e239c960ba59a140f5ffbe0d7060baf Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 16:54:49 -0700 Subject: [PATCH 29/30] test(scripts): follow model-access import move in application-graph fixture --- scripts/check-application-graph.test.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/check-application-graph.test.ts b/scripts/check-application-graph.test.ts index 7a434f24c98..faf495973d8 100644 --- a/scripts/check-application-graph.test.ts +++ b/scripts/check-application-graph.test.ts @@ -33,11 +33,11 @@ describe('the guarded roots', () => { forbidden: FORBIDDEN_PREFIXES, }) expect(violations).toHaveLength(1) - expect(violations[0].forbidden).toBe('providers/utils.ts') + expect(violations[0].forbidden).toBe('providers/models.ts') expect(violations[0].reason).toBe(FORBIDDEN_PREFIXES['providers/']) expect(violations[0].path).toEqual([ 'lib/permission-groups/model-access.ts', - 'providers/utils.ts', + 'providers/models.ts', ]) }) }) From 9abbf176a16486d27f54fabe3a81131abbe98daa Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Thu, 1 Oct 2026 16:55:30 -0700 Subject: [PATCH 30/30] chore(skills): drop vendored codebase-design skill from the repo --- .agents/skills/codebase-design/DEEPENING.md | 38 ------ .../skills/codebase-design/DESIGN-IT-TWICE.md | 44 ------- .agents/skills/codebase-design/LICENSE | 21 ---- .agents/skills/codebase-design/SKILL.md | 117 ------------------ .claude/skills/codebase-design | 1 - 5 files changed, 221 deletions(-) delete mode 100644 .agents/skills/codebase-design/DEEPENING.md delete mode 100644 .agents/skills/codebase-design/DESIGN-IT-TWICE.md delete mode 100644 .agents/skills/codebase-design/LICENSE delete mode 100644 .agents/skills/codebase-design/SKILL.md delete mode 120000 .claude/skills/codebase-design diff --git a/.agents/skills/codebase-design/DEEPENING.md b/.agents/skills/codebase-design/DEEPENING.md deleted file mode 100644 index 2a3ef47c4bd..00000000000 --- a/.agents/skills/codebase-design/DEEPENING.md +++ /dev/null @@ -1,38 +0,0 @@ -# Deepening - -How to deepen a cluster of shallow modules safely, given its dependencies. Assumes the vocabulary in [SKILL.md](SKILL.md): **module**, **interface**, **seam**, **adapter**. - -## Dependency categories - -When assessing a candidate for deepening, classify its dependencies. The category determines how the deepened module is tested across its seam. - -### 1. In-process - -Pure computation, in-memory state, no I/O. Always deepenable: merge the modules and test through the new interface directly. No adapter needed. - -### 2. Local-substitutable - -Dependencies you can run for real in a test: Postgres, Redis, the filesystem. Deepenable. The seam is internal; no port at the module's external interface. Verify the deepened module against the real dependency (a `*.integration.ts` suite on real Postgres/Redis) or end to end, never against a hand-built stand-in. - -### 3. Remote but owned (Ports & Adapters) - -Your own services across a network boundary (microservices, internal APIs). Define a **port** (interface) at the seam. The deep module owns the logic; the transport is injected as an **adapter**. Production uses an HTTP/gRPC/queue adapter. Prove the module end to end over the real transport (`apps/sim/scripts/test-*-e2e.ts`) rather than through an in-memory adapter written only for tests. - -Recommendation shape: *"Define a port at the seam and implement the production adapter there, so the logic sits in one deep module even though it's deployed across a network."* - -### 4. True external (Mock) - -Third-party services (Stripe, Twilio, etc.) you don't control. The deepened module takes the external dependency as an injected port. Where an isolated test is unavoidable, use the shared mocks in `@sim/testing` and `apps/sim/vitest.setup.ts` rather than hand-rolling one. - -## Seam discipline - -- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a port unless at least two adapters are justified. A single-adapter seam is just indirection. -- **Internal seams vs external seams.** A deep module can have internal seams (private to its implementation, used by its own tests) as well as the external seam at its interface. Don't expose internal seams through the interface just because tests use them. - -## Testing strategy: replace, don't layer - -Testing follows the repo's rules in `CLAUDE.md` ("Testing") and the `test-audit` skill; this section only says how they apply to deepening. - -- Old unit tests on shallow modules become waste once the deepened module exists; delete them. Deleting a test that only restated a pass-through is a win, not lost coverage. -- Don't write new unit tests at the deepened interface after the fact. Prove the change through the real boundary: a `*.integration.ts` suite on real Postgres/Redis, or an end-to-end run. -- Whatever tests remain assert on observable outcomes through the interface, not internal state. If a test has to change when the implementation changes, it's testing past the interface. diff --git a/.agents/skills/codebase-design/DESIGN-IT-TWICE.md b/.agents/skills/codebase-design/DESIGN-IT-TWICE.md deleted file mode 100644 index af03b2724c6..00000000000 --- a/.agents/skills/codebase-design/DESIGN-IT-TWICE.md +++ /dev/null @@ -1,44 +0,0 @@ -# Design It Twice - -When the user wants to explore alternative interfaces for a chosen deepening candidate, use this parallel sub-agent pattern. Based on "Design It Twice" (Ousterhout): your first idea is unlikely to be the best. - -Uses the vocabulary in [SKILL.md](SKILL.md): **module**, **interface**, **seam**, **adapter**, **leverage**. - -## Process - -### 1. Frame the problem space - -Before spawning sub-agents, write a user-facing explanation of the problem space for the chosen candidate: - -- The constraints any new interface would need to satisfy -- The dependencies it would rely on, and which category they fall into (see [DEEPENING.md](DEEPENING.md)) -- A rough illustrative code sketch to ground the constraints, not a proposal, just a way to make the constraints concrete - -Show this to the user, then immediately proceed to Step 2. The user reads and thinks while the sub-agents work in parallel. - -### 2. Spawn sub-agents - -Spawn 3+ sub-agents in parallel. Each must produce a **radically different** interface for the deepened module. - -Prompt each sub-agent with a separate technical brief (file paths, coupling details, dependency category from [DEEPENING.md](DEEPENING.md), what sits behind the seam). The brief is independent of the user-facing problem-space explanation in Step 1. Give each agent a different design constraint: - -- Agent 1: "Minimize the interface: aim for 1–3 entry points max. Maximise leverage per entry point." -- Agent 2: "Maximise flexibility: support many use cases and extension." -- Agent 3: "Optimise for the most common caller: make the default case trivial." -- Agent 4 (if applicable): "Design around ports & adapters for cross-seam dependencies." - -Include both [SKILL.md](SKILL.md) vocabulary and the repo's domain language (root `AGENTS.md` and the relevant `.claude/rules/*.md`) in the brief so each sub-agent names things consistently with the architecture language and the project's domain language. - -Each sub-agent outputs: - -1. Interface (types, methods, params, plus invariants, ordering, error modes) -2. Usage example showing how callers use it -3. What the implementation hides behind the seam -4. Dependency strategy and adapters (see [DEEPENING.md](DEEPENING.md)) -5. Trade-offs: where leverage is high, where it's thin - -### 3. Present and compare - -Present designs sequentially so the user can absorb each one, then compare them in prose. Contrast by **depth** (leverage at the interface), **locality** (where change concentrates), and **seam placement**. - -After comparing, give your own recommendation: which design you think is strongest and why. If elements from different designs would combine well, propose a hybrid. Be opinionated: the user wants a strong read, not a menu. diff --git a/.agents/skills/codebase-design/LICENSE b/.agents/skills/codebase-design/LICENSE deleted file mode 100644 index f1dd2c09108..00000000000 --- a/.agents/skills/codebase-design/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2026 Matt Pocock - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/.agents/skills/codebase-design/SKILL.md b/.agents/skills/codebase-design/SKILL.md deleted file mode 100644 index 8f87fc40b5f..00000000000 --- a/.agents/skills/codebase-design/SKILL.md +++ /dev/null @@ -1,117 +0,0 @@ ---- -name: codebase-design -description: Shared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a seam goes, make code more testable or AI-navigable, or when another skill needs the deep-module vocabulary. -# Vendored from github.com/mattpocock/skills (MIT, Copyright (c) 2026 Matt Pocock). No agents/openai.yaml by design: this is a vocabulary reference, not a service-integration builder. ---- - -# Codebase Design - -This is a reference, not a process: it supplies vocabulary and principles and stops. It has no loop, artifact, or checkpoint, so never treat it as a driver that starts reading files and proposing refactors on its own — pair it with a task that names the code being designed. - -Design **deep modules**: a lot of behaviour behind a small interface, placed at a clean seam, testable through that interface. Use this language and these principles wherever code is being designed or restructured. The aim is leverage for callers, locality for maintainers, and testability for everyone. - -## Glossary - -Use these terms exactly: don't substitute "component," "service," "API," or "boundary." Consistent language is the whole point. - -**Module**: anything with an interface and an implementation. Deliberately scale-agnostic: a function, class, package, or tier-spanning slice. _Avoid_: unit, component, service. - -**Interface**: everything a caller must know to use the module correctly: the type signature, but also invariants, ordering constraints, error modes, required configuration, and performance characteristics. _Avoid_: API, signature (too narrow, they refer only to the type-level surface). - -**Implementation**: what's inside a module, its body of code. Distinct from **Adapter**: a thing can be a small adapter with a large implementation (a Postgres repo) or a large adapter with a small implementation (an in-memory fake). Reach for "adapter" when the seam is the topic; "implementation" otherwise. - -**Depth**: leverage at the interface. The amount of behaviour a caller (or test) can exercise per unit of interface they have to learn. A module is **deep** when a large amount of behaviour sits behind a small interface, **shallow** when the interface is nearly as complex as the implementation. - -**Seam** _(Michael Feathers)_: a place where you can alter behaviour without editing in that place; the *location* at which a module's interface lives. Where to put the seam is its own design decision, distinct from what goes behind it. _Avoid_: boundary (overloaded with DDD's bounded context). - -**Adapter**: a concrete thing that satisfies an interface at a seam. Describes *role* (what slot it fills), not substance (what's inside). - -**Leverage**: what callers get from depth. More capability per unit of interface they learn. One implementation pays back across N call sites and M tests. - -**Locality**: what maintainers get from depth. Change, bugs, knowledge, and verification concentrate in one place rather than spreading across callers. Fix once, fixed everywhere. - -## Deep vs shallow - -**Deep module** = small interface + lots of implementation: - -``` -┌─────────────────────┐ -│ Small Interface │ ← Few methods, simple params -├─────────────────────┤ -│ │ -│ Deep Implementation│ ← Complex logic hidden -│ │ -└─────────────────────┘ -``` - -**Shallow module** = large interface + little implementation (avoid): - -``` -┌─────────────────────────────────┐ -│ Large Interface │ ← Many methods, complex params -├─────────────────────────────────┤ -│ Thin Implementation │ ← Just passes through -└─────────────────────────────────┘ -``` - -When designing an interface, ask: - -- Can I reduce the number of methods? -- Can I simplify the parameters? -- Can I hide more complexity inside? - -## Principles - -- **Depth is a property of the interface, not the implementation.** A deep module can be internally composed of small, mockable, swappable parts; they just aren't part of the interface. A module can have **internal seams** (private to its implementation, used by its own tests) as well as the **external seam** at its interface. -- **The deletion test.** Imagine deleting the module. If complexity vanishes, it was a pass-through. If complexity reappears across N callers, it was earning its keep. -- **The interface is the test surface.** Callers and tests cross the same seam. If you want to test *past* the interface, the module is probably the wrong shape. -- **One adapter means a hypothetical seam. Two adapters means a real one.** Don't introduce a seam unless something actually varies across it. - -## Designing for testability - -Good interfaces make testing natural: - -1. **Accept dependencies, don't create them.** - - ```typescript - // Testable - function processOrder(order: Order, paymentGateway: PaymentGateway): Promise {} - - // Hard to test - function processOrder(order: Order): Promise { - const gateway = new StripeGateway(); - } - ``` - -2. **Return results, don't produce side effects.** - - ```typescript - // Testable - function calculateDiscount(cart: Cart): Discount {} - - // Hard to test - function applyDiscount(cart: Cart): void { - cart.total -= discount; - } - ``` - -3. **Small surface area.** Fewer methods = fewer tests needed. Fewer params = simpler test setup. - -## Relationships - -- A **Module** has exactly one **Interface** (the surface it presents to callers and tests). -- **Depth** is a property of a **Module**, measured against its **Interface**. -- A **Seam** is where a **Module**'s **Interface** lives. -- An **Adapter** sits at a **Seam** and satisfies the **Interface**. -- **Depth** produces **Leverage** for callers and **Locality** for maintainers. - -## Rejected framings - -- **Depth as ratio of implementation-lines to interface-lines** (Ousterhout): rewards padding the implementation. We use depth-as-leverage instead. -- **"Interface" as the TypeScript `interface` keyword or a class's public methods**: too narrow: interface here includes every fact a caller must know. -- **"Boundary"**: overloaded with DDD's bounded context. Say **seam** or **interface**. - -## Going deeper - -- **Deepening a cluster given its dependencies**, see [DEEPENING.md](DEEPENING.md): dependency categories, seam discipline, and replace-don't-layer testing (which defers to `CLAUDE.md` "Testing" and the `test-audit` skill). -- **Exploring alternative interfaces**, see [DESIGN-IT-TWICE.md](DESIGN-IT-TWICE.md): spin up parallel sub-agents to design the interface several radically different ways, then compare on depth, locality, and seam placement. diff --git a/.claude/skills/codebase-design b/.claude/skills/codebase-design deleted file mode 120000 index 08b466ebb0a..00000000000 --- a/.claude/skills/codebase-design +++ /dev/null @@ -1 +0,0 @@ -../../.agents/skills/codebase-design \ No newline at end of file