Shared synchronous Session may cause 300-second stalls on silently stale pooled connections #186

Summary

DashScope SDK 1.26.7 uses a process-wide shared synchronous requests.Session.

In a long-running Dify plugin process, embedding requests that use the SDK-managed shared Session intermittently take approximately 301 seconds, while normal requests complete in approximately 145–180 ms.

The affected request eventually succeeds because the caller retries the first transport failure. The initial exception is therefore not surfaced in the final Dify response.

Environment

• DashScope SDK: 1.26.7
• Python: 3.12
• requests: 2.34.2
• Integration: Dify Tongyi plugin 0.2.15
• API: TextEmbedding.call
• Model: text-embedding-v4
• Deployment: long-running Docker plugin processes

Observed behavior

Normal embedding invocation:

latency: approximately 145–180 ms
status: HTTP 200

Intermittently affected invocation:

latency: 301327 ms
status: HTTP 200 after caller-level retry

The corresponding knowledge-retrieval operation took 301.722 seconds.

The duration is consistent with:

1. the SDK's 300-second default read timeout;
2. the caller's 1-second retry backoff;
3. a fast successful retry.

The first exception is caught by the integration's retry helper, and the plugin runtime does not forward the Python warning to its container log. Therefore, we do not currently have the original ReadTimeout traceback and do not claim that the exception type has been directly captured.

Why this appears related to the shared Session

When no explicit Session is supplied, independent embedding invocations use the SDK-managed process-wide Session and connection pool.

The same deployment does not show this behavior on the Tongyi LLM path after that integration was changed to create and close a dedicated Session for every invocation.

We have implemented the same scoped mitigation for embedding calls:

• create one Session per embedding invocation;
• reuse it within that invocation's batches and retries;
• close it when the invocation completes;
• prevent pooled connections from crossing invocation boundaries.

Related Dify reports and changes:

• Embedding report: Tongyi text embedding: shared DashScope session can cause intermittent 300-second stalls langgenius/dify-official-plugins#3815
• Embedding mitigation PR: fix(tongyi): isolate DashScope sessions per embedding invocation langgenius/dify-official-plugins#3817
• Previous LLM Session isolation: fix(tongyi): isolate DashScope sessions per LLM invocation langgenius/dify-official-plugins#3659

DashScope PR #175 already addressed pooled connections that fail with a pre-response ConnectionError, but deliberately excludes ReadTimeout:

#175

Requested investigation

Please investigate whether the process-wide shared synchronous Session can reuse silently stale pooled connections and block until the default read timeout.

The SDK should ensure that stale pooled connections do not cause otherwise normal requests to stall for approximately 300 seconds. The specific connection-lifecycle strategy is left to the maintainers.

Expected behavior

A connection that has become silently stale should not block a normal embedding request until the full 300-second read timeout, and long-running applications should have a supported way to control the lifecycle of the SDK-managed shared connection pool.