Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,11 @@ unreachable model within minutes, and the pipeline no longer imports the chat. N
- **The data-change event moved out of the chat.** `chat.suggest.SuggestionDataChanged` is now `event.DataChanged`;
`analytics`, `application` and `archive` import nothing of the chat, and `ChatIsolationTest` checks imports and every
`chat_*` table the migrations create.
- **Spring AI 2.0.2, with openai-java held at 4.65.0.** 2.0.2 brings openai-java 4.80.0, and from 4.66.0 on a stopped
turn or one past its deadline leaves the model call running (openai/openai-java#1027; 4.96.0 still does). The hold
moves from Spring AI to openai-java, so Spring AI's own fixes land: Anthropic streaming chunks carry the thinking
text, and the configured OpenAI timeout is honoured again. Renovate proposes openai-java in a pull request of its own.
- **Spring AI 2.0.2, with the openai-java it brings (4.80.0).** Its own fixes land: Anthropic streaming chunks carry
the thinking text, and the configured OpenAI timeout is honoured again. From openai-java 4.66.0 on
(openai/openai-java#1027) closing a stream from another thread no longer ends the HTTP call, so a stopped turn, a
deleted one or one past its deadline left the model generating; closing a model response now cancels its call first,
and no openai-java version is held back any more.
- **The frontend image builds on Node 26** (`node:26.13.0-alpine`, from 24.22.1). The Angular CLI accepts
`>=26.0.0`; the build output is the same static files, served by the same nginx stage. A root `.nvmrc` names
the major, and CI and the release gate now set up that Node with `actions/setup-node` instead of building on
Expand All @@ -44,8 +45,8 @@ unreachable model within minutes, and the pipeline no longer imports the chat. N
`''` for a `grid-template-columns` nothing computed, and the shortlist now reads both as "no grid", which a
browser can answer too. TypeScript stays on 6.0, which is what Angular 22 accepts.
- **Backend: jsoup 1.24.1 and Spotless 8.11.0.** palantir-java-format 2.103.0 reformats four files and stays at
2.98.0 for a commit of its own; Gradle 9.9.0 waits for its `gradle:9.9.0-jdk25` image. openai-java 4.101.0 still
fails the four chat-stop tests, so the hold at 4.65.0 stays.
2.98.0 for a commit of its own. Gradle 9.9.0; the backend image keeps building on `gradle:9.8.2-jdk25` until a
`gradle:9.9.0-jdk25` image is published.

### Fixed

Expand Down
2 changes: 2 additions & 0 deletions backend/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ takes `ChatBudget.take()` against `chat.max_calls_per_day` instead — reasoning
`docs/decisions/retrieval.md`.
- **An UPDATE cannot reference its own target table from a LATERAL item in its FROM clause.** Compute the pairs in a
`WITH` and update `FROM` that. Reasoning in `docs/decisions/pipeline-dedupe-filter.md`.
- **Every OpenAI-compatible chat client gets `CancelOnClose.CUSTOMIZER`, as `ChatModels` passes it, or a stopped
stream leaves the model generating.** — reasoning in `docs/decisions/chat.md` § Stop and regenerate.
- **Two constructors on a `@Component` are none, and the error names the wrong thing.** Spring picks neither and
reports `No default constructor found` — a message about a constructor that was never meant to exist, rather than
about the ambiguity. Every context in the suite fails at once: measured at 194 failures from one added convenience
Expand Down
10 changes: 0 additions & 10 deletions backend/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -123,16 +123,6 @@ dependencies {
// The BOM governs the two starters' versions and everything they drag in, so the
// Anthropic and OpenAI client SDKs are pinned by one line rather than by three.
implementation(platform(libs.spring.ai.bom))
// With one exception: openai-java is held below what the BOM brings, because from 4.66.0 on a
// stop no longer ends the model call. `strictly`, so a newer one dragged in by anything else
// fails the resolution instead of quietly winning. The reason and the way out sit next to
// `openaiJava` in libs.versions.toml.
constraints {
implementation(libs.openai.java.core) {
version { strictly(libs.versions.openaiJava.get()) }
because("openai-java 4.66.0+ leaves a cancelled stream's connection open (openai/openai-java#1027)")
}
}
// Naming two vendors here is deliberate and is the one place the "nothing is wired in"
// rule does not reach: a dependency is not a value. No host, key or model name appears
// in any committed file, and `base_url` still decides who answers.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
/*
* SPDX-License-Identifier: Apache-2.0
*
* Copyright 2026 Marcello Muscara (codeministry)
*
* Licensed under the Apache License, Version 2.0. You may obtain a copy of the
* License at http://www.apache.org/licenses/LICENSE-2.0
*/
package de.codeministry.leadgen.llm;

import java.io.IOException;
import okhttp3.Call;
import okhttp3.Interceptor;
import okhttp3.MediaType;
import okhttp3.Response;
import okhttp3.ResponseBody;
import okio.BufferedSource;
import org.springframework.ai.openai.http.okhttp.OpenAiHttpClientBuilderCustomizer;

/**
* Makes closing a model response cancel its HTTP call before the body is closed, so a stopped
* stream ends the request on the wire.
*
* <p><b>Why.</b> A chat turn is stopped, deleted or past its deadline by closing its stream from a
* thread other than the one blocked reading it. From openai-java 4.66.0 on (openai/openai-java#1027)
* that close shuts the OkHttp response before the reader, under the read still in flight: okio
* throws "Unbalanced enter/exit", the socket stays open, and the model keeps generating an answer
* nobody reads, paid for in full. {@link Call#cancel()} is OkHttp's own thread-safe way to end a
* call from outside: it closes the socket, the blocked read fails, and the closes after it find
* nothing left to race.
*
* <p>Cancelling a call whose body was read to its end changes nothing, so the calls that are not
* streamed, the judge's and the classifier's, close as before.
*/
final class CancelOnClose implements Interceptor {

/** Hands the interceptor to Spring AI's OkHttp client, for the sync and the async one alike. */
static final OpenAiHttpClientBuilderCustomizer CUSTOMIZER = builder -> builder.interceptor(new CancelOnClose());

@Override
public Response intercept(Chain chain) throws IOException {
Response response = chain.proceed(chain.request());
ResponseBody body = response.body();
if (body == null) {
return response;
}
return response.newBuilder().body(new Cancelling(body, chain.call())).build();
}

/** The response body as it was, except that closing it cancels the call first. */
private static final class Cancelling extends ResponseBody {

private final ResponseBody delegate;
private final Call call;

Cancelling(ResponseBody delegate, Call call) {
this.delegate = delegate;
this.call = call;
}

@Override
public MediaType contentType() {
return delegate.contentType();
}

@Override
public long contentLength() {
return delegate.contentLength();
}

@Override
public BufferedSource source() {
return delegate.source();
}

@Override
public void close() {
try {
call.cancel();
} finally {
delegate.close();
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ private ChatModel openAi(String baseUrl, String apiKey, String model, Duration t
null,
ObservationRegistry.NOOP,
null,
List.of()))
List.of(CancelOnClose.CUSTOMIZER)))
// Both clients, and the asynchronous one is not optional: left unset, the
// builder makes its own from its own empty fields and fails with "at least
// one credential source must be specified" — a credential error naming a key
Expand All @@ -174,7 +174,7 @@ private ChatModel openAi(String baseUrl, String apiKey, String model, Duration t
null,
ObservationRegistry.NOOP,
null,
List.of()))
List.of(CancelOnClose.CUSTOMIZER)))
// Same trap as in `EmbeddingModels`: without this, `AbstractOpenAiOptions`
// fills in its 60 s DEFAULT_TIMEOUT and the per-call value overrides the
// client's. The Anthropic options below carry no such default.
Expand Down
15 changes: 15 additions & 0 deletions docs/decisions/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -429,6 +429,21 @@ leaves nowhere to put them. The model is asked with tool execution left to the c
the turn reads between chunks and fires a signal that cancels the model request waiting for its
next one, so a local model stops generating rather than finishing into a closed pipe
(`ChatStreamTest`: the stub's connection closed within a second, its last chunk never written).

**Closing the stream cancels its HTTP call first (`llm/CancelOnClose`).** The stop, a delete and the
deadline all close the model's stream from a thread other than the one blocked reading it. openai-java
4.65.0 closed the reader before the response and got away with it; from 4.66.0 on
(openai/openai-java#1027) it closes the OkHttp response first, under the read still in flight, okio throws
"Unbalanced enter/exit", and the socket stays open while the model writes an answer nobody reads. Measured:
the four stop tests in `ChatStreamTest`, `ChatConversationTest` and `ChatTurnDeadlineTest` red on
4.66.1, 4.80.0, 4.92.0, 4.96.0 and 4.101.0. For a while the fix was holding openai-java at 4.65.0 below
what Spring AI 2.0.2 brings; that pinned a client Spring AI was never built against and needed a release
upstream to end. The interceptor ends it here instead: Spring AI's OkHttp client takes one through
`OpenAiSetup`'s customizer list, `ChatModels` hands it to the sync and the async client, and closing a
response body calls `Call.cancel()` before the close — OkHttp's own thread-safe way to end a call from
outside. A body read to its end has nothing left to cancel, so the judge and the classifier, which call
rather than stream, are untouched. Green on 4.80.0 and 4.101.0 alike.

The partial answer is kept, its citations listed, and the turn stored `STOPPED`; the composer takes
the next question.

Expand Down
12 changes: 0 additions & 12 deletions gradle/libs.versions.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,6 @@ flexmark = "0.64.8"
# third model module would buy nothing. Pinned rather than tracked for the same reason jsoup
# and freemarker are — a judge that suddenly answers differently should show up as a diff.
springAi = "2.0.2"
# openai-java, held below the version Spring AI brings (2.0.2 brings 4.80.0) and enforced as a
# `strictly` constraint in backend/build.gradle.kts. Since 4.66.0 (openai/openai-java#1027) closing a
# stream from another thread closes the OkHttp body under the in-flight read, which throws
# "Unbalanced enter/exit" and leaves the connection open, so stopping a chat turn or hitting its
# deadline no longer ends the model call. 4.65.0 is the last release before that; 4.66.1, 4.92.0, 4.96.0 and
# 4.101.0 fail the same four tests in ChatStreamTest, ChatConversationTest and ChatTurnDeadlineTest.
# Held here rather than holding Spring AI, so its other fixes land. Every openai-java method that
# spring-ai-openai 2.0.2 calls exists in 4.65.0 (checked against the bytecode, 499 references).
# Renovate proposes it in a pull request of its own; when that one turns green, the hold can go —
# drop the constraint and this entry together, and the next Spring AI picks its own version again.
openaiJava = "4.65.0"
# The GraalVM build tools. Applied for BOTH images and not only the native one: applying
# it is what makes the Boot plugin register `processAot`, and Spring AOT is half of the JVM
# image's startup win. It also carries the version of the reachability-metadata repository
Expand Down Expand Up @@ -61,7 +50,6 @@ spring-ai-chat = { module = "org.springframework.ai:spring-ai-client-chat" }
spring-ai-anthropic = { module = "org.springframework.ai:spring-ai-anthropic" }
spring-ai-openai = { module = "org.springframework.ai:spring-ai-openai" }
spring-ai-mcp-server = { module = "org.springframework.ai:spring-ai-starter-mcp-server-webmvc" }
openai-java-core = { module = "com.openai:openai-java-core", version.ref = "openaiJava" }
jsoup = { module = "org.jsoup:jsoup", version.ref = "jsoup" }
lombok = { module = "org.projectlombok:lombok", version.ref = "lombok" }
greenmail-junit5 = { module = "com.icegreen:greenmail-junit5", version.ref = "greenmail" }
Expand Down
8 changes: 1 addition & 7 deletions renovate.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,17 +36,11 @@
"groupName": "spring boot"
},
{
"description": "Spring AI gets its own pull request. Its BOM decides the model SDKs, and openai-java is held below the one it brings, so a Spring AI release is the update most likely to need that hold revisited; on its own, its CI result says so without turning the backend group red.",
"description": "Spring AI gets its own pull request. Its BOM decides the model SDKs, so a Spring AI release can change the client that streams the chat; the stop tests then say so on its own CI run without turning the backend group red.",
"matchPackageNames": ["/^org\\.springframework\\.ai/"],
"groupName": null,
"automerge": false
},
{
"description": "openai-java is held at the last release before the stream-cancel regression (reason next to openaiJava in libs.versions.toml). Its own pull request, so a red one stays out of the backend group and a green one says the hold can go.",
"matchPackageNames": ["com.openai:openai-java-core"],
"groupName": null,
"automerge": false
},
{
"description": "native-build-tools carries the GraalVM reachability-metadata repository version with it, so a bump changes which third-party metadata the native image is compiled against. Its own pull request, read before merging.",
"matchPackageNames": ["/^org\\.graalvm\\.buildtools/"],
Expand Down
Loading