Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions docs/design/tool-search-design-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,8 @@ SearchExtraToolsTool 支持四种查询模式:

| 模式 | 语法 | 行为 | 返回 |
|------|------|------|------|
| **Select** | `select:CronCreate,Snip` | 按名称直接获取,逗号分隔多选 | 精确匹配列表 |
| **Discover** | `discover:schedule cron job` | 纯发现模式,返回描述+schema | 工具信息文本 |
| **Select** | `select:CronCreate,Snip` | 按名称直接获取,逗号分隔多选 | 精确匹配列表 + 每个 deferred 工具的参数 JSON Schema |
| **Discover** | `discover:schedule cron job` | 纯发现模式,返回描述+schema | 工具信息文本 + 参数 JSON Schema |
| **Keyword** | `notebook jupyter` | 关键词搜索 | 按相关性排序 |
| **Required** | `+slack send` | `+` 前缀强制包含 | 包含必选词的结果 |

Expand All @@ -105,6 +105,16 @@ mcp__slack__send_message → parts: ["slack", "send", "message"]
CamelCase → parts: ["cron", "create"]
```

**参数 schema 下发(关键)**:

Deferred 工具不进 API tools 数组,模型只能通过 SearchExtraTools 的 tool_result 文本拿到参数 schema。因此 `select:` 与 `discover:` 都必须把命中工具的 `inputJSONSchema`(MCP 工具)或由 zod 转换得到的 JSON Schema(内置 deferred 工具)渲染进 tool_result。若缺失,模型只能凭工具名猜参数名(属性名可能被改写、翻译或臆造),而错误参数会被透传给 MCP 服务端导致空结果或报错。

作为第二道防线,`ExecuteExtraTool` 在委托前对声明了 `inputJSONSchema` 的工具用 Ajv 校验入参:参数名/类型错误会返回包含「期望属性名」的明确报错,而不是静默透传。inter-turn 预取生成的 `tool_discovery` attachment 同样会带上 schema 并提示使用精确属性名。

**未声明键兜底(Ajv 之外)**:JSON Schema 默认允许未声明属性,因此当 schema 未声明 `required`(或仅声明可选属性)时,改名的键不会触发任何 Ajv 错误,错误键名仍会被透传——原缺陷会静默复现。所以校验通过后仍会比对「模型实际传入的键」与「schema 声明的 properties」:只要 schema 声明了至少一个属性且未显式设置 `additionalProperties: true`,命中未声明键即按参数不匹配拒绝,报错同时列出期望属性名与实际错传键名。该判定集中在 `src/utils/toolSchemaRender.ts:findUndeclaredKeys`。

schema 渲染统一收口到 `src/utils/toolSchemaRender.ts`(`normalizeSchemaForModel` / `renderSchemaForModel`):剥离 `$schema`、限制单份 schema 体积、以 fenced block 包裹并标注仅作数据。SearchExtraTools 的 tool_result 与 `tool_discovery` 预取附件共用同一实现,避免两条路径的加固程度不一致。

### 3.4 执行管道

```
Expand Down
159 changes: 157 additions & 2 deletions packages/builtin-tools/src/tools/ExecuteTool/ExecuteTool.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import { Ajv, type ErrorObject } from 'ajv'
import { z } from 'zod/v4'
import {
buildTool,
Expand All @@ -9,8 +10,10 @@ import {
type Tools,
} from 'src/Tool.js'
import { lazySchema } from 'src/utils/lazySchema.js'
import { logForDebugging } from 'src/utils/debug.js'
import { createUserMessage } from 'src/utils/messages.js'
import { formatZodValidationError } from 'src/utils/toolErrors.js'
import { findUndeclaredKeys } from 'src/utils/toolSchemaRender.js'
import {
extractDiscoveredToolNames,
isSearchExtraToolsEnabledOptimistic,
Expand Down Expand Up @@ -44,6 +47,113 @@ type OutputSchema = ReturnType<typeof outputSchema>

export type Output = z.infer<OutputSchema>

type JsonSchemaObject = Record<string, unknown>
type JsonValidator = ((data: unknown) => boolean) & {
errors?: ErrorObject[] | null
}

// MCP tool schemas are stable for the lifetime of a session, so compile each
// one once and reuse it. Keyed by schema identity so the Ajv JIT codegen runs
// at most once per tool rather than on every ExecuteExtraTool call.
const jsonValidatorCache = new WeakMap<object, JsonValidator | null>()

function compileJsonSchemaValidator(schema: object): JsonValidator | null {
const candidates: object[] = [schema]
if ('$schema' in schema) {
// Ajv 8 bundles draft-07 only; an explicit 2019-09/2020-12 $schema would
// throw "no schema with key or ref". Strip it and fall back to draft-07
// semantics, which cover the constructs MCP tool schemas actually use.
const clone = { ...(schema as JsonSchemaObject) }
delete clone.$schema
candidates.push(clone)
}
let lastError: unknown
for (const candidate of candidates) {
try {
const ajv = new Ajv({ allErrors: true, strict: false })
return ajv.compile(candidate as never) as JsonValidator
} catch (err) {
// Try the next candidate; on exhaustion fall through to null (skip
// validation rather than blocking execution on an unusable schema).
lastError = err
}
}
// Log rather than swallow: a schema we cannot compile means this tool skips
// client-side validation entirely and the original bug silently returns.
logForDebugging(
`ExecuteExtraTool: could not compile JSON Schema validator (validation skipped): ${String(lastError)}`,
)
return null
}

function getJsonSchemaValidator(schema: object): JsonValidator | null {
const cached = jsonValidatorCache.get(schema)
if (cached !== undefined) return cached
const validator = compileJsonSchemaValidator(schema)
jsonValidatorCache.set(schema, validator)
return validator
}

/**
* Build a model-readable error that names the offending fields AND the
* expected property names, so the model can correct the call instead of
* blindly retrying. MCP schemas rarely set additionalProperties:false, so a
* renamed key surfaces as a missing required property rather than as an
* unknown key — spelling out both the expected names AND the keys the model
* actually sent is what lets it see the rename instead of guessing again.
*
* `undeclaredKeys` is passed in rather than recomputed because the caller
* needs it to decide whether to reject at all: a schema whose properties are
* all optional accepts a renamed key without emitting any JSON Schema error.
*/
function formatJsonSchemaMismatchError(
toolName: string,
schema: JsonSchemaObject,
params: Record<string, unknown>,
errors: readonly ErrorObject[],
undeclaredKeys: readonly string[],
): string {
const problems =
errors
.map(e => `${e.instancePath || '(params)'} ${e.message ?? 'is invalid'}`)
.join('; ') || 'the parameter names do not match the tool schema'

const properties = schema.properties as JsonSchemaObject | undefined
const required = new Set(
Array.isArray(schema.required) ? (schema.required as string[]) : [],
)
let expected = ''
if (properties) {
expected =
' Expected parameters: ' +
Object.entries(properties)
.map(([key, def]) => {
const type =
def && typeof def === 'object' && 'type' in def
? String((def as { type?: unknown }).type)
: 'any'
return `${key} (${type}${required.has(key) ? ', required' : ''})`
})
.join(', ') +
'.'
}

// Name the keys the model actually sent that the schema does not declare.
// This is the direct signal for a renamed-parameter bug: the
// required-property error alone never mentions the stray key, and when the
// schema declares no required property there is no error to begin with.
const unexpected =
undeclaredKeys.length > 0
? ` You sent undeclared key(s): ${undeclaredKeys.join(', ')}.`
: ''

return (
`Invalid parameters for tool "${toolName}": ${problems}.${expected}${unexpected}` +
` The keys in "params" must exactly match these property names.` +
` Call SearchExtraTools({"query": "select:${toolName}"}) to see the full schema, then retry.`
)
}

export const ExecuteTool = buildTool({
name: EXECUTE_TOOL_NAME,
searchHint: 'execute run invoke call a deferred tool by name with parameters',
Expand Down Expand Up @@ -130,8 +240,14 @@ export const ExecuteTool = buildTool({
// .trim()/.length/.split() crashes with "undefined is not an object".
// CronCreateTool's .trim() crash was the reported symptom; centralizing
// the check here covers every deferred tool without relying on each one
// to defensively guard its own validateInput. Duck-typed so MCP tools
// (whose schema is inputJSONSchema, not zod) skip this branch.
// to defensively guard its own validateInput. Tools whose schema is zod
// are handled here; tools exposing a raw JSON Schema (MCP tools,
// StructuredOutput) are handled by the inputJSONSchema branch below.
//
// Ordering matters: MCP tools expose a passthrough zod schema, so this
// branch succeeds and copies params through verbatim. It must NOT start
// stripping unknown keys — the JSON Schema branch below needs the keys the
// model actually sent in order to name the rename in its error.
const targetSchema = targetTool.inputSchema as
| { safeParse?: (data: unknown) => unknown }
| undefined
Expand All @@ -158,6 +274,45 @@ export const ExecuteTool = buildTool({
input.params = parsed.data
}

// MCP tools (and any tool declaring a raw JSON Schema) carry their real
// schema in inputJSONSchema, not zod. Validate params against it so a
// renamed or invented field is rejected here with a precise error instead
// of being forwarded to the MCP server, which would fail opaquely or
// silently return empty results.
const inputJSONSchema = (targetTool as { inputJSONSchema?: object })
.inputJSONSchema
if (inputJSONSchema && typeof inputJSONSchema === 'object') {
const validate = getJsonSchemaValidator(inputJSONSchema)
const errors =
validate && !validate(input.params) ? (validate.errors ?? []) : []
// JSON Schema permits undeclared properties by default, so a schema that
// declares no `required` (or only optional ones) accepts a renamed key
// without emitting any error — the wrong name would then be forwarded
// and the original bug would come back silently. Reject undeclared keys
// whenever the schema declares properties and does not explicitly allow
// extras; findUndeclaredKeys owns that decision.
const undeclaredKeys = findUndeclaredKeys(inputJSONSchema, input.params)
if (errors.length > 0 || undeclaredKeys.length > 0) {
return {
data: {
result: null,
tool_name: input.tool_name,
},
newMessages: [
createUserMessage({
content: formatJsonSchemaMismatchError(
input.tool_name,
inputJSONSchema as JsonSchemaObject,
input.params,
errors,
undeclaredKeys,
),
}),
],
}
}
}

// Validate input before delegating — prevents crashes when the model
// omits required params (e.g. TeamCreate without team_name →
// sanitizeName(undefined).replace() TypeError).
Expand Down
Loading