[azure-ai-projects] builtin tool_input_accuracy returns status "skipped" / score null although tool calls and tool definitions are present #49923

• Package Name: azure-ai-projects (cloud built-in evaluators, run through the Foundry evaluations API)
• Package Version: 2.9.0 (submitted via agent-framework-foundry 1.16.1 and agent-framework-core 1.25.0)
• Operating System: Ubuntu (GitHub-hosted runner)
• Python Version: 3.13

Describe the bug
The built-in agent evaluator tool_input_accuracy intermittently returns status: "skipped" and score: null for rows where the conversation does contain tool calls and the tool definitions are supplied. The judge's own reason acknowledges that tool calls and definitions are present, and then skips anyway. In a smaller number of rows it returns score: 0.0 while its reason describes the parameters as grounded and correctly typed.

The evaluator's documented skip conditions (empty tool calls, missing tool definitions, missing conversation) are not met in these rows. Because the affected row changes on every run of the same dataset, the null results cannot be tied to a specific input.

Observed over 12 evaluation runs (4 CI runs x 3 datasets, 80 tool_input_accuracy results, agent responses replayed against fixed tool fixtures, judge model gpt-5.4-mini): 17 results were skipped although tool calls were present, and 2 were 0.0 with a reason that describes valid parameters.

Excerpts from the judge reason of skipped rows (they vary between runs for the same row):

Evaluation skipped because the conversation history/query and tool call details were provided, but the Tool Calls Made section contains a final assistant answer rather than only tool-call parameters to evaluate.

Evaluation was skipped because the conversation includes tool calls and tool definitions, but there is no actual agent tool-call payload to validate in isolation beyond the already shown narrative/result context.

Skipped evaluation because the Tool Calls Made section is not empty, and tool definitions and conversation context are provided, so a full evaluation would normally be possible.

Excerpt from a 0.0 row:

The call parameters appear grounded and correctly typed: [skill load] was called with a valid skill name. However, the evaluation request is to judge the correctness of parameters passed to tool calls made by the agent. [...]

The skip conditions in the public tool_input_accuracy.prompty of azure-ai-evaluation (no tool calls, no tool definitions, no conversation context) do not match what the judge reports. I could not confirm whether the hosted evaluator uses the same prompt.

To Reproduce
Steps to reproduce the behavior:

1. Create an agent (function tools, one of them a skill-loading tool, plus MCP tools) and a dataset of single-turn queries that make it call 1 to 6 tools.
2. Evaluate it with FoundryEvals(evaluators=[TOOL_SELECTION, TOOL_INPUT_ACCURACY, TOOL_CALL_SUCCESS, TOOL_OUTPUT_UTILIZATION], model="gpt-5.4-mini").evaluate_agent(...), or create the same evaluation with the azure_ai_evaluator testing criteria (builtin.tool_input_accuracy).
3. Read the output items of the run with openai_client.evals.runs.output_items.list(run_id, eval_id=eval_id).
4. Repeat 3 to 4 times with the same dataset. Up to 4 of the 8 or 9 tool_input_accuracy results per run come back with status: "skipped", and the affected rows are different each time.

Expected behavior
When the tool calls, tool definitions and conversation are all present, the evaluator returns a 0 or 1 score based on the parameters. skipped is returned only for the documented conditions, and a reason that describes the parameters as valid does not produce a 0.0.

Screenshots
Not applicable.

Additional context

• I don't know whether the cause is the way the tool calls are serialized into the judge input (several reasons mention that the "Tool Calls Made" section contains the final assistant answer) or the judge prompt itself. If the payload is built by agent-framework, please point me to the right repository.
• A related earlier problem, where tool_selection and tool_input_accuracy were null for most rows because tool definitions were not sent, was reported and fixed in Python: [Bug]: evaluate_agent() omits ContextProvider/SkillsProvider tools from tool_definitions, breaking tool_selection/tool_input_accuracy microsoft/agent-framework#9160. The behavior described here appears after that fix.
• Microsoft Learn lists tool_input_accuracy as requiring (query, response, tool_definitions) or (messages, tool_definitions), and MCP tools as supported.
• I can provide more information if needed (raw output items, run ids, or the exact payload).