ReadException "too many concurrent streams" - 2

[dpevunov-cp]

When using sttp's HttpClientFs2Backend (which uses JDK's java.net.http.HttpClient) with HTTP/2, high concurrent load causes immediate failures with java.io.IOException: too many concurrent streams instead of queuing requests

JDK HttpClient creates one HTTP/2 connection per host and fails immediately when concurrent requests exceed the server's SETTINGS_MAX_CONCURRENT_STREAMS limit (typically 100). This is a known unresolved JDK bug: https://bugs.openjdk.org/browse/JDK-8225647.

I would expect this setting to work like some backpressure mechanism but not just failing requests(at least not right away and waiting till timeout comes)

Is there any suggestion or workaround for this to be fixed on sttp lvl?

Here is a gist for reproducing the issue

[OlegYch]

fwiw armeria backend works fine

[adamw]

As I understand, what you'd like is to have an additional setting, maxConnectQueueLength, which would keep connection requests, until a slot is available?

(Next to a maxConnectionInProgress, which should be automatically filled-in basing on the server's advertised settings, however that is not possible due to the JVM bug you linked).

And only after there's more connections requests than slots in the queue, you would get an exception? Is that how Armeria works, btw.? Probably some time-out would be good to have as well.

However I think at some point of saturation, there's always going to be a threshold where you'll get immediate errors. With queueing, you might only push this higher to handle bursts of connections.

As a work-around, adding a backend wrapper which uses a semaphore to allow only 100 concurrent requests at any time could work maybe? (per host)