Skip to content

fix(tools): carry element types and constraints into tool schemas - #1694

Open
planetf1 wants to merge 5 commits into
generative-computing:mainfrom
planetf1:issue-1693
Open

planetf1 wants to merge 5 commits into
generative-computing:mainfrom
planetf1:issue-1693

Conversation

@planetf1

@planetf1 planetf1 commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request

Issue

Fixes #1693

Description

Tools taking lists, dicts or constrained values now get their arguments as declared, on every backend.

Before

  • A list[int] or list[float] argument reached the tool as strings ([1, 2] became ['1', '2']), so sum(nums) raised a TypeError inside the user's own code.
  • list[bool], list[list[int]] and lists of Pydantic models (discriminated unions included) failed validation on every call, and were passed to the tool unchecked with a warning.
  • The model was never told what goes inside a list or dict. list[int], list[Point] and dict[str, int] went out as a bare array or object, and Field constraints were dropped.
  • An explicit null for an optional parameter (limit: int | None = None) failed validation. The whole call then fell back to the raw arguments, so "count": "3" was no longer coerced to 3. (Found while working on fix(backends): parse Granite XML tool calls on the local HF path #1692.)
  • A tool schema containing a boolean sub-schema (valid JSON Schema, e.g. properties: {"a": true} from an MCP server) made validation raise even in lenient mode. On HF the generation failed; on OpenAI-compatible backends the tool call was dropped or the error escaped.

After

  • Element types arrive intact, and model or union elements are validated against their fields.
  • The model sees items, additionalProperties and constraints such as minimum, maxLength and minItems. On Ollama only items gets through, because the ollama client drops the other keywords.
  • Optional parameters accept null. A parameter with a real default (page_size: int = 10) still rejects it.
  • A schema the validator can't build falls back to the model's arguments with a warning, as lenient mode documents. Strict mode still raises.

How: convert_function_to_ollama_tool() rebuilt simple parameters from four keys and discarded the rest of Pydantic's schema. It now carries a fixed set of JSON Schema keywords (_CARRIED_SCHEMA_KEYWORDS) and flattens any discriminated unions that exposes. validate_tool_arguments() reads an items schema with no type as list[Any] rather than list[str], treats a non-required field with no non-null default as nullable, and builds its model inside the existing try.

Not covered

  • Unions with several non-null branches (int | list[int]) and tuple element types still reach the model without element types. Their validation is unaffected.
  • limit: int | None = 5 still rejects null. The schema doesn't record nullability, so it is inferred from the default.
  • The validator still doesn't enforce numeric or length bounds, or dict value types (unchanged).
  • On Ollama, a Pydantic-model parameter still reaches the model without its fields. The ollama client strips properties, required and anyOf before the request (upstream Add self-referencing properties field to Tool.Property ollama/ollama-python#724, feat: citation requirement #725), so this needs an upstream fix and a dependency bump.

Testing

  • Tests added to the respective file if code was changed

  • New code has 100% coverage if code was added

  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

  • New test/backends/test_tool_collection_params_unit.py: 41 tests covering each case above through schema generation and validation. 35 of them fail on main. Validation tests use strict=True, so a pass can't come from the lenient fallback.

  • test/backends/test_tool_validation_integration.py: test_optional_param_none now runs strict, plus three new null tests (three of the four fail on main) and six boolean sub-schema tests (the three lenient ones fail on main).

  • uv run pytest test/ -m "not qualitative": 4739 passed, 21 skipped. ruff and mypy are clean.

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

The schema rebuild for simple tool parameters kept only description,
type, enum and default, dropping items, additionalProperties and Field
constraints. The model was never told a list's or dict's element type,
and validate_tool_arguments, which builds its validator from the same
schema, fell back to string elements: list[int] arguments reached the
tool as strings, and list[bool], nested lists and lists of models failed
validation and were passed through unchecked.

Copy items, additionalProperties and the standard JSON Schema
constraint keywords onto the rebuilt property, taking them from the
non-null anyOf branch for Optional parameters. In the validator, treat
an array with a missing or empty items schema as list[Any] rather than
list[str], so list[Any], tuples and external schemas without items are
no longer coerced to strings.

Fixes generative-computing#1693

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@github-actions github-actions Bot added the bug Something isn't working label Sep 29, 2026
convert_function_to_ollama_tool drops null from a simple Optional type,
so `limit: int | None = None` reaches validate_tool_arguments as a
non-required integer. The validator built that field as `int` with a
None default, so an explicit None failed: strict mode raised, and
lenient mode (used by every backend) fell back to the unvalidated
arguments, losing every other coercion in the call.

Type non-required validator fields as `T | None`. The existing
explicit-null tests passed only through the lenient fallback; they now
run with strict=True, and a new test checks the other arguments are
still coerced alongside the null.

Refs generative-computing#1693

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Flatten discriminated unions exposed by the carried keywords. Copying
items and additionalProperties sent list[Pet], dict[str, Pet] and
list[Owner] (where a field is a discriminated union) out with raw oneOf
and discriminator, which tool APIs reject and the rest of the pipeline
strips. Re-run _recursively_flatten_in_properties after the final ref
inlining, which also covers a discriminated union in a nested model's
field.

Only treat a non-required validator field as nullable when its schema
has no non-null default. `limit: int | None = None` still accepts an
explicit None; `page_size: int = 10` rejects it again in strict mode.

Treat any items schema that names no type (annotation-only, or a
non-object form such as `true`) as list[Any], not just a missing or
empty one.

Run the correct-input validation tests with strict=True so they cannot
pass through the lenient fallback.

Refs generative-computing#1693

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Comment-only change; no behaviour change.

Refs generative-computing#1693

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
validate_tool_arguments built its validator model outside the try, so a
schema it could not build (boolean sub-schemas such as
properties: {"a": true}, valid JSON Schema from MCP servers) raised
TypeError or AttributeError even with strict=False. The HF path does not
catch it, so generation failed; the OpenAI-compatible path dropped the
tool call as malformed on TypeError and failed on AttributeError.

Build the model inside the existing try, so lenient mode logs and
returns the original arguments and strict mode still raises. Also trim
test docstrings to one line.

Refs generative-computing#1693

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(tools): list[int] tool arguments arrive as strings, and list/dict element types never reach the model

1 participant