Skip to content

Add FinishReasonCategory for provider-agnostic finish reason normalization - #5190

Open
QuarkOS wants to merge 2 commits into
spring-projects:mainfrom
QuarkOS:feature/issue-5146-finish-reasons
Open

QuarkOS wants to merge 2 commits into
spring-projects:mainfrom
QuarkOS:feature/issue-5146-finish-reasons

Conversation

@QuarkOS

@QuarkOS QuarkOS commented Jan 5, 2026 •

Copy link
Copy Markdown

Fixes #5146

Problem

ChatGenerationMetadata.getFinishReason() returns raw provider-specific strings that vary across providers:

Provider Completed Truncated Tool Call Filtered
OpenAI STOP LENGTH TOOL_CALLS CONTENT_FILTER
Anthropic end_turn max_tokens tool_use -
Gemini STOP MAX_TOKENS - SAFETY
Azure OpenAI stop length tool_calls content_filter
Bedrock end_turn max_tokens tool_use -

This forces applications to implement custom mapping logic for consistent audits/metrics/alerts.

Solution

Introduce a normalized FinishReasonCategory enum with a static categorize() method:

public enum FinishReasonCategory {
    COMPLETED,   // Normal stop
    TRUNCATED,   // Length/token limits
    TOOL_CALL,   // Tool invocation
    FILTERED,    // Content filtering
    OTHER,       // Known but uncategorized
    UNKNOWN      // Null/empty/unrecognized
}

Add a default method to ChatGenerationMetadata:

default FinishReasonCategory getFinishReasonCategory() {
    return FinishReasonCategory.categorize(getFinishReason());
}

Usage

ChatResponse response = chatClient.call(prompt);
FinishReasonCategory category = response.getResult()
    .getMetadata()
    .getFinishReasonCategory();

// Provider-agnostic handling
if (category == FinishReasonCategory.TRUNCATED) {
    log.warn("Output was truncated");
}

Changes

  • New: FinishReasonCategory.java - Enum with categorize() method
  • Modified: ChatGenerationMetadata.java - Added getFinishReasonCategory() default method
  • New: FinishReasonCategoryTests.java - Parameterized tests for all providers
  • New: ChatGenerationMetadataTests.java - Integration tests

Checklist

  • Signed-off-by in commit (DCO)
  • Rebased on latest main
  • Unit tests added
  • All tests pass (./mvnw test -pl spring-ai-model)
  • Code formatted with spring-javaformat:apply

Copilot AI review requested due to automatic review settings January 5, 2026 22:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces provider-agnostic finish reason normalization to address the inconsistency in raw finish reason strings across different AI providers (OpenAI, Anthropic, Gemini, Azure OpenAI, Bedrock). It adds a new FinishReasonCategory enum with a static categorize() method that normalizes provider-specific strings into standard categories (COMPLETED, TRUNCATED, TOOL_CALL, FILTERED, OTHER, UNKNOWN).

  • Added FinishReasonCategory enum with case-insensitive categorization logic for all major AI providers
  • Extended ChatGenerationMetadata interface with a getFinishReasonCategory() default method
  • Included comprehensive parameterized tests covering all providers and edge cases

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
spring-ai-model/src/main/java/org/springframework/ai/chat/metadata/FinishReasonCategory.java New enum providing normalized finish reason categories with static categorize() method supporting case-insensitive mapping for OpenAI, Anthropic, Gemini, Azure OpenAI, and Bedrock
spring-ai-model/src/main/java/org/springframework/ai/chat/metadata/ChatGenerationMetadata.java Added default getFinishReasonCategory() method that delegates to FinishReasonCategory.categorize()
spring-ai-model/src/test/java/org/springframework/ai/chat/metadata/FinishReasonCategoryTests.java Comprehensive parameterized tests validating categorization for all provider finish reasons and edge cases (null, blank, unrecognized)
spring-ai-model/src/test/java/org/springframework/ai/chat/metadata/ChatGenerationMetadataTests.java Integration tests verifying the default method implementation and proper category resolution through the metadata interface

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@QuarkOS
QuarkOS force-pushed the feature/issue-5146-finish-reasons branch from 0bea17c to a060903 Compare January 5, 2026 23:04
@ilayaperumalg ilayaperumalg added model client enhancement New feature or request labels Jan 7, 2026
@QuarkOS

QuarkOS commented Sep 28, 2026

Copy link
Copy Markdown
Author

Closing this as part of a cleanup of my stale/slow-moving PRs. Thanks for your time!

Introduce FinishReasonCategory and a default
ChatGenerationMetadata accessor. getFinishReason() still
returns the raw provider string, including null.

Null and blank reasons are UNKNOWN. A non-blank value
that does not match a known alias is OTHER, so an
unrecognized reason stays distinct from an absent one.

Fixes spring-projects#5146

Signed-off-by: Quark <quackingquark@gmail.com>

Co-authored-by: Emilio Schwaiger <QuarkOS@users.noreply.github.com>
@cursor
cursor Bot force-pushed the feature/issue-5146-finish-reasons branch from a060903 to 9d1d947 Compare October 12, 2026 18:49

@CryoThrust CryoThrust left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rechecked 9d1d9475 after the update on #5146. The default accessor and the explicit raw-string-only contract address the compatibility concern I raised there. Keeping absent reasons separate from unrecognized values is also now documented and tested.

One concrete alias is missing from the truncation mapping: model_context_window_exceeded. The Anthropic SDK version used by this branch (2.64.0) already declares that stop reason (source). Both AnthropicChatModel.buildGenerations and its message_delta path pass the provider reason through to generation metadata.

I compiled the exact enum from this head and called it directly:

max_tokens                    -> TRUNCATED
context_window_exceeded       -> TRUNCATED
model_context_window_exceeded -> OTHER

The last value means the model hit its context window, so it should have the same TRUNCATED category. Could you add the actual provider alias to that switch arm and a regression case? A metadata assertion that the raw value remains model_context_window_exceeded would also protect the additive API contract.

This finding is from the mapper probe and the pinned SDK/model sources; I did not run the Spring AI module suite or a live Anthropic request.

Classify model_context_window_exceeded as TRUNCATED.
The raw finish reason string stays unchanged.

See spring-projects#5146

Signed-off-by: Quark <quackingquark@gmail.com>

Co-authored-by: Emilio Schwaiger <QuarkOS@users.noreply.github.com>

@CryoThrust CryoThrust left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rechecked 53fe935c. model_context_window_exceeded now maps to TRUNCATED; the exact-enum JDK17 probe passes six controls, including the uppercase/whitespace form and the absent/unknown distinctions. The added metadata test also explicitly checks that the raw Anthropic string is retained.

This resolves the alias finding from my earlier review. My validation is the mapper probe plus source inspection of the metadata regression, not a Spring AI module-suite run or live provider request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request model client

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Normalize / categorize LLM finish reasons across providers in ChatGenerationMetadata (keep raw reason)

4 participants