feat(client): cap the requests in flight at 50 per client

CHANGELOG.md

@@ -38,6 +38,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `stocks.quotes()` now requests `stocks/quotes/?symbols=...` instead of the deprecated `stocks/bulkquotes/`; the method, its parameters and its output are unchanged (#74)
 - **BREAKING**: `UserRateLimits` speaks in API credits, as the product does (SDK requirements §8.1): `requests_limit` → `credit_limit`, `requests_remaining` → `credits_remaining`, `requests_consumed` → `credits_consumed`, `requests_reset` → `reset_time`. The old names are gone, with no aliases; its string form now reads `Credits used X/Y, remaining: Z, reset at: <ISO timestamp>` (#48)
 - The pre-flight credit check reads a private, thread-safe tracker fed by every response that carries credit headers, error answers included; out-of-order responses (an older reset window, a higher remaining in the same window) are ignored, and every response logs its credits at DEBUG. An envelope with a `credit_limit` of zero is not recorded: the API answers a missing or unknown token with demo data and a `0/0` envelope from another window, which would otherwise tell the check this account has no credits. `client.utilities.user()` is asked for precisely to learn the balance, so its answer replaces the state rather than being weighed against it (#49)
+- A client sends at most 50 requests at a time, the concurrent-request limit the API documents (it states that requests above it get a `429`, SDK requirements §12). Every request of the client takes one of 50 slots for the length of its HTTP exchange, whichever endpoint, fan-out or thread sends it, and a request that finds none waits for a slot instead of being sent. Before, `stocks.candles()` and `options.quotes()` each ran 50 workers of their own, so two calls at the same time put 100 requests on the wire. A retry waiting out its backoff holds no slot. The cap is fixed and has no constructor parameter, and it belongs to the client: two clients on the same account have 50 slots each, so share one client between threads rather than building one per thread (#65)
 
 ### Added
 

README.md

@@ -905,8 +905,10 @@ The SDK uses Pydantic for input validation:
 
 The SDK uses concurrent requests for efficient data fetching in specific scenarios:
 
-- **`stocks.candles()`**: For intraday resolutions (minutely/hourly), large date ranges are automatically split into year-long, non-overlapping chunks and fetched concurrently (up to 50 concurrent requests by default)
-- **`options.quotes()`**: Multiple option symbols are fetched concurrently (up to 50 concurrent requests by default)
+- **`stocks.candles()`**: For intraday resolutions (minutely/hourly), large date ranges are automatically split into year-long, non-overlapping chunks and fetched concurrently
+- **`options.quotes()`**: Multiple option symbols are fetched concurrently
+
+A client sends at most 50 requests at a time, the concurrent-request limit the API documents (it states that requests above it get a `429`). Every request takes one of the client's 50 slots for as long as its HTTP exchange lasts, whichever endpoint, fan-out or thread sends it, and a request that finds none waits for a slot to free instead of being sent. A retry waiting out its backoff holds no slot. The cap is fixed and has no constructor parameter. It belongs to the client: two clients on the same account have 50 slots each, so share one client between threads rather than building one per thread.
 
 When concurrent requests are used, responses are automatically merged into a single result. A chunk or symbol with no data is left out of the merge; if every part has no data the call returns an empty result, and if the API answered with nothing usable at all a `MarketdataHttpError` is raised.
 

docs/options.md

@@ -575,7 +575,7 @@ if csv_file:
 
 ### `quotes()`
 
-Fetches options quotes for one or more option symbols. This method includes API status checking, automatic retry logic, and supports concurrent requests for multiple symbols (up to 50 concurrent requests by default). Each symbol's request retries on its own, so a failed symbol never re-sends the healthy ones. A symbol whose response body cannot be decoded raises `ParseError`, as in every other method, on every output format; in CSV output the merged file keeps the header the API sent (the requested `columns`, the human-readable names) and every row of every symbol. On the other formats the symbols are merged on the model's columns among those the API sent, in the order it sent them (the request order under `columns=`), and a symbol whose answer lacks one of them, or carries one that is not a list as long as its others, raises `ParseError` as well, since merging it would put the next symbol's values on its row.
+Fetches options quotes for one or more option symbols. This method includes API status checking, automatic retry logic, and supports concurrent requests for multiple symbols (within the client's 50 request slots). Each symbol's request retries on its own, so a failed symbol never re-sends the healthy ones. A symbol whose response body cannot be decoded raises `ParseError`, as in every other method, on every output format; in CSV output the merged file keeps the header the API sent (the requested `columns`, the human-readable names) and every row of every symbol. On the other formats the symbols are merged on the model's columns among those the API sent, in the order it sent them (the request order under `columns=`), and a symbol whose answer lacks one of them, or carries one that is not a list as long as its others, raises `ParseError` as well, since merging it would put the next symbol's values on its row.
 
 > **Note:** The `symbols` parameter can be passed as the first positional argument or as a keyword argument. All other parameters must be keyword-only.
 
@@ -612,7 +612,7 @@ Fetches options quotes for one or more option symbols. This method includes API
 - If `output_format=OutputFormat.CSV`: A string containing the filename where CSV data was written (merged from all requested symbols)
 - Raises a `BaseMarketdataException` subclass if an error occurs (rate limits, validation errors, request failures, no valid responses received, etc.); see the [README](../README.md#error-handling)
 
-> **Note:** The method returns the requested output, an empty result when the API has no data for every symbol (`404` with `s: "no_data"`, see the [README](../README.md#no-data-is-not-an-error)), or raises. The method uses concurrent requests (up to 50 concurrent requests by default) to fetch quotes for multiple symbols efficiently.
+> **Note:** The method returns the requested output, an empty result when the API has no data for every symbol (`404` with `s: "no_data"`, see the [README](../README.md#no-data-is-not-an-error)), or raises. The method uses concurrent requests, within the client's 50 request slots (see the [README](../README.md#concurrent-requests)), to fetch quotes for multiple symbols efficiently.
 
 #### Examples
 

docs/stocks.md

@@ -320,7 +320,7 @@ print(df)
 
 ### `candles()`
 
-Fetches stock candles (OHLCV data) for a symbol with support for various timeframes and date ranges. This method includes API status checking and automatic retry logic. For intraday resolutions (minutely/hourly), large date ranges are automatically split into year-long chunks and fetched concurrently using up to 50 concurrent requests. Each chunk's request retries on its own, so a failed chunk never re-sends the others.
+Fetches stock candles (OHLCV data) for a symbol with support for various timeframes and date ranges. This method includes API status checking and automatic retry logic. For intraday resolutions (minutely/hourly), large date ranges are automatically split into year-long chunks and fetched concurrently, within the client's 50 request slots (see the [README](../README.md#concurrent-requests)). Each chunk's request retries on its own, so a failed chunk never re-sends the others.
 
 > **Note:** The `symbol` parameter can be passed as the first positional argument or as a keyword argument. All other parameters must be keyword-only.
 
@@ -331,8 +331,8 @@ Fetches stock candles (OHLCV data) for a symbol with support for various timefra
   - Numeric with unit: `"1"`, `"15M"`, `"1H"`, `"1D"`, `"1W"`, `"1M"`, `"1Y"`
   - Unit only: `"M"`, `"H"`, `"D"`, `"W"`, `"M"`, `"Y"`
   - Descriptive: `"minutely"`, `"hourly"`, `"daily"`, `"weekly"`, `"monthly"`, `"yearly"`
-- `from_date` (datetime.date, optional): The start date to fetch candles for. When both `from_date` and `to_date` are provided, the date range is automatically split into year-long chunks and fetched concurrently for intraday resolutions (minutely/hourly) using up to 50 concurrent requests. For non-intraday resolutions (daily/weekly/monthly/yearly), the entire date range is fetched in a single request without splitting.
-- `to_date` (datetime.date, optional): The end date to fetch candles for. When both `from_date` and `to_date` are provided, the date range is automatically split into year-long chunks and fetched concurrently for intraday resolutions (minutely/hourly) using up to 50 concurrent requests. For non-intraday resolutions (daily/weekly/monthly/yearly), the entire date range is fetched in a single request without splitting.
+- `from_date` (datetime.date, optional): The start date to fetch candles for. When both `from_date` and `to_date` are provided, the date range is automatically split into year-long chunks and fetched concurrently for intraday resolutions (minutely/hourly), within the client's 50 request slots. For non-intraday resolutions (daily/weekly/monthly/yearly), the entire date range is fetched in a single request without splitting.
+- `to_date` (datetime.date, optional): The end date to fetch candles for. When both `from_date` and `to_date` are provided, the date range is automatically split into year-long chunks and fetched concurrently for intraday resolutions (minutely/hourly), within the client's 50 request slots. For non-intraday resolutions (daily/weekly/monthly/yearly), the entire date range is fetched in a single request without splitting.
 - `countback` (int, optional): The number of candles to fetch (alternative to date range)
 - `extended` (bool, optional): Whether to fetch extended candles (pre-market and after-hours data)
 - `adjust_splits` (bool, optional): Whether to adjust for stock splits. Uses API alias `adjustsplits`.
@@ -368,7 +368,7 @@ Fetches stock candles (OHLCV data) for a symbol with support for various timefra
 
 When both `from_date` and `to_date` are provided, the method behavior depends on the resolution:
 
-- **For intraday resolutions (minutely/hourly)**: The date range is automatically split into year-long chunks, fetched concurrently using a thread pool executor (up to 50 concurrent requests), and merged into a single response. Chunks never share a calendar day, so no candle is requested twice. This allows efficient fetching of large historical date ranges without manual pagination.
+- **For intraday resolutions (minutely/hourly)**: The date range is automatically split into year-long chunks, fetched concurrently using a thread pool executor within the client's 50 request slots, and merged into a single response. Chunks never share a calendar day, so no candle is requested twice. This allows efficient fetching of large historical date ranges without manual pagination.
 
 - **For non-intraday resolutions (daily/weekly/monthly/yearly)**: The entire date range is fetched in a single request without splitting.
 

src/marketdata/client.py

@@ -1,3 +1,4 @@
+import threading
 import time
 from importlib.metadata import version
 from logging import DEBUG, INFO, Logger
@@ -21,6 +22,7 @@
 from marketdata.input_types.base import UserUniversalAPIParams
 from marketdata.internal_settings import (
     HEADER_REQUEST_ID,
+    MAX_CONCURRENT_REQUESTS,
     MAX_CREDIT_WINDOW_SECONDS,
     MAX_RETRY_ATTEMPTS,
     NO_TOKEN_VALUE,
@@ -53,6 +55,18 @@ def __init__(
         logger: Logger = None,
         max_retries: int = MAX_RETRY_ATTEMPTS,
     ):
+        """Build the client and, when it has a token, seed the credit tracker.
+
+        Args:
+            token: The API token. Falls back to the ``MARKETDATA_TOKEN``
+                setting; with neither, the client starts in demo mode.
+            logger: The logger to write to. The SDK's own when omitted.
+            max_retries: Retries after the first attempt of each request.
+
+        Raises:
+            ValueError: ``max_retries`` is negative.
+            BaseMarketdataException: The start-up ``/user/`` request failed.
+        """
         if max_retries < 0:
             raise ValueError("max_retries must be >= 0")
         self.token = token or settings.marketdata_token
@@ -75,7 +89,12 @@ def __init__(
         self.client = self._get_client()
         self.default_params = UserUniversalAPIParams()
 
-        # The private credit tracker behind the pre-flight check (#49); the
+        # One budget for every request of this client, whichever endpoint or
+        # fan-out sends it. It exists before `_setup_rate_limits`, whose
+        # `/user/` request already takes a slot.
+        self._request_slots = threading.BoundedSemaphore(MAX_CONCURRENT_REQUESTS)
+
+        # The private credit tracker behind the pre-flight check; the
         # request-scoped numbers travel with each result (marketdata.get_meta).
         self._rate_limits = RateLimitTracker()
         self._setup_rate_limits()
@@ -298,6 +317,38 @@ def _make_request(
         response_log_level: int = INFO,
         **kwargs,
     ) -> Response:
+        """Send one HTTP request and map its outcome to the SDK's exceptions.
+
+        The request holds one of the client's request slots only for the HTTP
+        exchange itself: it waits for a free slot after the pre-flight check,
+        and a retry sleeping through its backoff holds none.
+
+        Args:
+            method: The HTTP method.
+            url: The path, relative to the base URL.
+            check_rate_limits: Refuse the request when the account is known to
+                have no credits left.
+            part_of_result: Record the response's metadata in the caller's
+                scope.
+            include_api_version: Prefix the path with the API version.
+            authoritative_credits: Let this response replace the tracked credit
+                state instead of being weighed against it.
+            response_log_level: The level of the line logged for the response.
+            **kwargs: Passed to ``httpx.Client.request``.
+
+        Returns:
+            The response, including a 404 without ``errmsg``, which is the
+            API's "no data" answer.
+
+        Raises:
+            RateLimitError: The pre-flight check refused the request, or the
+                API answered 429.
+            NetworkError: There was no usable answer (a connection failure, a
+                timeout, a protocol or proxy error).
+            ParseError: The body does not match its ``Content-Encoding``.
+            MarketdataHttpError: The API answered with an error status; the
+                subclass names it.
+        """
         if self.token is NO_TOKEN_VALUE:
             check_rate_limits = False
 
@@ -311,7 +362,8 @@ def _make_request(
             # No `timeout=` here: the client carries the fixed one (§10),
             # so there is one place to read it and no argument a resource
             # could use to give itself a longer bound.
-            response = self.client.request(method, url, **kwargs)
+            with self._request_slots:
+                response = self.client.request(method, url, **kwargs)
         except DecodingError as exc:
             # The API answered but the body does not match its Content-Encoding
             # (an intercepting proxy): the answer is unusable, not missing.