[MarketDataDev02]

Closes #111

Summary

The API adds columns without an SDK release. Each one made every OutputFormat.INTERNAL call of its resource raise (ParseError since #50, a bare TypeError in 1.x), because a model's constructor refuses a keyword it does not declare. The models are now built from the columns they declare, and the rest of the answer is kept on the model for marketdata.get_extra(model), direction 2 of the issue: on every resource that builds through columns, the fan-outs' merged answer included, nothing the API sent is lost, and the column is named once per answer at DEBUG. Only utilities.user, which reads its own keys, keeps nothing. A column the model needs and the body lacks still raises ParseError, and so does an answer whose columns, declared or not, are lists of different lengths, on every format that decodes it, the rule the fan-outs' merge already applied.

The same failure hit the CSV output of the two fan-outs, stocks.candles and options.quotes: their merge refused a header with a column the model does not know, which is how it tells an HTML error page from a CSV. That output now keeps the column, as the CSV of every single-request resource does, and refuses only a header with none of the resource's columns. The CSV of options.chain, options.expirations, options.lookup and stocks.earnings applies the same rule, so a body that is not an answer of these resources raises ParseError on every format. Without a header row, under add_headers=False, the CSV of those four resources and of the fan-outs refuses an answer whose Content-Type names JSON or HTML, or whose body reads as either.

sdk-go (json.Unmarshal into structs, no DisallowUnknownFields) and sdk-csharp (reads the keys it looks for) ignore an unknown key on their typed output, so neither fails on an added column, and neither keeps it for the caller the way get_extra does.

Version impact

Major (1.3.0 -> 2.0.0): this ships in 2.0.0 with the rest of [Unreleased], which is not released yet. The CHANGELOG entry is under Changed (BREAKING) with its migration, get_extra is under Added, and the breaking changes are on the feat(models)!: commit d483055 and the feat(resources)!: commits 548338c, 7759248, 4fb87ed and a6cd05e, whose BREAKING CHANGE: footers name every breaking behavior of this PR. What a caller sees change, and what to do:

• INTERNAL of every resource that builds through columns (funds.candles, markets.status, options.chain, options.expirations, options.lookup, options.quotes, stocks.candles, stocks.earnings, stocks.news, stocks.prices, stocks.quotes and client.utilities.status, both name styles): an answer with a column the model does not declare returns the model, where it raised ParseError. get_extra(model) returns the column as the API sent it: the whole column on a single-object model, each row's own value on a list result. The column is named once per answer at DEBUG on marketdata.logger:
  The API sent the columns ['brandNew'], which StockNews does not declare; get_extra() holds them

  The line goes to marketdata.logger by name, as the response-metadata line in meta.py does, so a client built with its own logger= does not get it in that logger. Migration: code that caught that ParseError gets the result; read the column with get_extra() until the model declares it.

• StockEarnings.from_dict, StockPrice.from_dict, StockQuote.from_dict and their human-readable twins: a key that names no field is kept for get_extra(), where it raised TypeError. Migration: an except TypeError meant for an added key is no longer reached.

• options.chain, options.expirations, options.lookup and stocks.earnings, on every format: a body that is not a JSON object raises ParseError. INTERNAL raised a bare AttributeError on the human-readable models, JSON returned the body, DataFrame built a frame from it (for null an empty frame, which reads like an answer with no data) and CSV wrote it to the file. An object with none of the resource's columns, like a proxy's JSON error page, raises ParseError on JSON and DataFrame too, where they returned it, as it did on INTERNAL. On CSV, a header row with none of the resource's columns raises ParseError, and under add_headers=False, with no header to check, so does an answer whose Content-Type names JSON or HTML, a lone string, number or boolean included, or whose body reads as either; any other headerless body is written as it came. Migration: catch ParseError (or BaseMarketdataException) where the value used to come back.

• CSV of stocks.candles and options.quotes, and utils.merge_csv_responses: a column the API adds is kept, where it raised ParseError: ... (unknown columns [...]); only a header with none of the resource's columns is refused. Migration: none; the file has one more column.

• stocks.candles and options.quotes, and utils.json_answer_columns: a column the model does not declare is carried through the merge on every format, INTERNAL through get_extra(), where INTERNAL, JSON and DataFrame left it out. Like a declared column, it must come in every chunk or symbol as a list as long as its rows: an answer without it fails the call with ParseError on every format, as the merged CSV already did for a header that differs. Under add_headers=False, their CSV refuses an answer labeled or shaped as JSON or HTML, as the four single-object resources do. Migration: none for an API that sends the column in every answer; a call whose answers disagree on a column raises.

• resources.utilities.base.render: the INTERNAL rows of client.utilities.status keep an added column for get_extra(), where the call raised ParseError. Migration: as for INTERNAL above.

• Every other resource that answers in columns (funds.candles, markets.status, options.chain, options.expirations, options.lookup, stocks.earnings, stocks.news, stocks.prices, stocks.quotes and client.utilities.status, both name styles): an answer whose lists, declared or not, differ in length raises ParseError (columns of different lengths {...}, the fan-outs' message) on INTERNAL, JSON and DataFrame, and on the CSV of client.utilities.status, which is written from the decoded answer. On main, INTERNAL built the rows up to the shortest declared column (and refused an added one, as it refused any added column), DataFrame padded the short columns with nulls into extra rows, and JSON returned the body. A single value instead of a list is kept as sent, on every format. Migration: catch ParseError (or BaseMarketdataException); no answer of the live API is uneven (see Review round 8).

Unchanged:

• A column the model needs and the body lacks: ParseError.
• An object with none of the model's columns (a proxy's JSON error page) on the list resources: ParseError on INTERNAL, with the same message, while JSON, DataFrame and CSV return it, as on main; that stays with Reading a body can raise a non-SDK exception: RecursionError on nested JSON, TypeError/UnicodeError/AssertionError on an exotic charset #110. On the four single-object resources and the fan-outs every format refuses it (see Version impact).
• DataFrame, JSON and CSV of the single-request resources carry the column through when it holds one value per row, as before.
• utilities.headers keeps every key as a header; utilities.user reads only its own keys.
• A model's fields, == and repr: get_extra reads the columns from the model's __dict__, where the response metadata already lives, not from a field.

The change

• output_types/columns.py:
  • _split_fields is the one place that keys an answer by field. It returns the keys that name a column or a field of the model, renamed to the field, and every other key but the status flag s as the API sent it, logging those once at DEBUG. An answer that is not an object, or that carries none of the model's columns, comes back as it is with nothing else, so the model refuses it as before. _to_fields is gone: every caller goes through _split_fields.
  • get_extra, public and re-exported from marketdata (in __all__), reads the columns kept on a model. _with_extra keeps them on a model, and _with_row_extra on the rows of a list result: a value per row from a list as long as the rows, which an added list is whenever the answer carries declared lists, since the length check below holds it to their length, and a single value as sent on every row.
• Every INTERNAL path that builds through columns goes through _split_fields once per answer, before a column-oriented answer is split into rows:
  • stocks.candles and options.quotes, on their merged answer; the model each builds from one answer alone, to name the answer a refused value came from, ignores the rest;
  • options.chain, options.expirations and options.lookup, both models;
  • funds.candles, markets.status, stocks.news, stocks.prices, stocks.quotes and client.utilities.status. stocks.prices and stocks.quotes build each row with the constructor, as the other list resources do, so the log is written once per answer.
• StockEarnings.from_dict, StockPrice.from_dict, StockQuote.from_dict and their human-readable twins go through _split_fields and keep the rest for get_extra, with Args/Returns/Raises docstrings. stocks.earnings builds through from_dict.
• utils.merge_csv_responses refuses only a header with none of the resource's columns; its docstring says a column that matches none of them is kept. Without a header row, it refuses an answer whose Content-Type names JSON or HTML or, under any other label or none, whose body starts with [, { or <, or is null (_reads_as_json_or_html).
• utils.json_answer_columns returns, besides the model's columns, every other key but s that holds a list, and holds it to the rule a declared column follows. Its length check is utils._check_column_lengths, which the single-request resources share: by default it compares every key but s that holds a list.
• resources/base.py: _parse_json_object decodes a body that must be a JSON object carrying at least one of the model's columns, with every list one length, as parse_json does, and refuses anything else with ParseError. _parse_json_columns decodes and checks the lengths for the INTERNAL, JSON and DataFrame branches of funds.candles, markets.status, stocks.news, stocks.prices and stocks.quotes, and resources/utilities/base.render checks a column-oriented answer before any format, its CSV included. _check_csv_header refuses a CSV header row with none of the resource's columns, read as the fan-out merge reads it, and, without a header row, an answer labeled or shaped as JSON or HTML, through the same _reads_as_json_or_html. The INTERNAL, JSON and DataFrame branches of options.chain, options.expirations, options.lookup and stocks.earnings decode through the first, and their CSV branch checks through the second.
• CHANGELOG.md: the INTERNAL raises a bare TypeError the day the API adds a column #111 entry under Changed (BREAKING), with the length rule and its migration, get_extra under Added, and the Use Decimal for monetary fields in internal models (drop float for money) #50 line above.
• README.md: a "Columns a model does not declare" section under Output Formats, next to the one on money: which results carry get_extra, whole or per row, how the values are decoded, how long they last (the object's __dict__, through copy, deepcopy and pickle, outside ==, the repr and fields()), what the other formats do, and that a list must hold one value per row.
• docs/funds.md, markets.md, options.md, stocks.md and utilities.md: a "Columns a model does not declare" section before the methods, saying what get_extra holds for that resource's results (whole or per row), where a fan-out requires the column in every answer, and which utilities keep nothing, with a link to the README section.

Overlap with main

#157 (#102), #154, #158, #160, #161 and #163 landed on main after this branch was cut, #157 on 2026-10-02 at 14:45 UTC. b442e64 merges main into this branch, with the two conflicts this section announced:

• resources/base.py, the imports: this PR adds import csv, fix(stocks): build INTERNAL quotes that carry the 52-week range #157 added Collection to the collections.abc import. Both stay.
• StockQuote.from_dict: its docstring and its last line. fix(stocks): build INTERNAL quotes that carry the 52-week range #157 built through _to_fields, this PR through _split_fields and _with_extra. This PR's side stays.

Everything else merged on its own. With #157 in, the 52-week range is a field of StockQuote, so get_extra no longer holds it: live, at 15:52 UTC, stocks.quotes("AAPL", use_52_week=True, output_format=OutputFormat.INTERNAL) returns fiftyTwoWeekHigh=Decimal('345.34') and fiftyTwoWeekLow=Decimal('242.7592'), get_extra returns {}, and no undeclared column is logged. The live run below, on d483055, predates the merge. No other pull request is open.

Proof

What main does. A scratch matrix built every INTERNAL resource, plain and human, from nine bodies: the fixture plus a column (extra), the fixture minus one (missing), [1], "x", 5, null, {}, {"error": "x"} and {"s": "ok", "error": "x"}. That is 216 cells, run on main bc0619c and on this branch and compared by exception class and the first 70 characters of the message. What changed:

• extra, on the 18 single-request cases: ParseError became a result, which since round 3 holds the column in get_extra. The fan-outs returned one already, without the column; since round 4 they hold it in get_extra too. The two utilities that do not build through columns returned one already.
• [1], "x", 5 and null on human options.chain, options.expirations and stocks.earnings: a bare AttributeError became ParseError (12 cells). Since rounds 2 and 3 the non-objects on these resources and options.lookup, with either names, raise ParseError on all four formats; see Review round 3.

Every other cell is unchanged, including {} and the non-objects on the list resources, which stay with #110.

The fan-outs, on main. With a column added to the answer of stocks.candles and options.quotes, INTERNAL, JSON and DataFrame return without it, and CSV raises ParseError: ... (unknown columns ['brandNew']). On this branch every format has it.

Suite. main c4ee8e1: 1979 passed. This branch, on a6cd05e (main merged in): 2164 passed, coverage 100% (3297 statements, 0 missed). test_a_body_whose_keys_the_model_does_not_take_is_a_parse_error is gone: its two extra cases pinned the old answer. Its missing case stays as test_a_body_missing_a_column_the_model_needs_is_a_parse_error, now on options.expirations, markets.status and utilities.user. 188 tests are new, those 3 included.

The new tests fail on main. With main's src/marketdata (c4ee8e1) checked out on this branch, 147 of the 188 fail (measured on a6cd05e):

• the 7 cases of test_a_foreign_object_is_a_parse_error_on_every_format;
• the 22 cases of test_a_column_no_model_declares_is_kept_for_get_extra and the 22 of test_a_column_no_model_declares_reaches_every_format;
• the 35 cases of test_a_body_that_is_not_an_object_is_a_parse_error_on_every_format;
• the 4 cases of test_a_headerless_csv_that_reads_as_json_or_html_is_a_parse_error and the 8 of test_a_fan_out_headerless_csv_that_reads_as_json_is_a_parse_error;
• test_a_csv_header_the_csv_module_cannot_read_is_a_parse_error, test_an_undeclared_value_that_is_not_a_list_goes_to_every_row, test_get_extra_is_empty_for_an_object_with_no_dict and test_get_extra_survives_copy_and_pickle;
• the 21 cases of test_an_undeclared_list_of_another_length_is_a_parse_error_on_every_format and 15 of the 19 of test_a_declared_column_of_another_length_is_a_parse_error_on_every_format;
• test_options_quotes_an_added_column_one_symbol_lacks_is_a_parse_error and the new case of test_json_answer_columns_name_the_answer_that_breaks_the_merge;
• the 3 CSV tests of the fan-outs and the merge;
• the 2 from_dict tests of test_stocks_prices.py and the 2 of test_stocks_quotes.py.

The 41 that pass pin what does not change: the missing column (3), the object with none of the model's columns on INTERNAL (22), the CSV without a header row that the API labels CSV (12), and a declared column of another length on the fan-outs (4), which their merge already refused.

Mutations, each caught (on 6a6b3db, before _to_fields became _split_fields):

• _to_fields also builds from an object with none of the model's columns: 11 fail, all on the list resources.
• _to_fields also logs the status flag: 5 fail, on the check that the fixtures log nothing.
• stocks.news renames each row instead of the answer: 2 fail, one log per row.

Live. src/tests/integration: 16 passed and 2 failed. Both failures are the API's answers today, and the earnings one fails on main too:

• test_funds_live.py::test_candles_return_recent_fund_prices: funds candles answers "This API endpoint is currently offline".
• test_stocks_live.py::test_earnings_return_expected_shape: earnings("AAPL", countback=4) answers one upcoming report with fiscalYear: null.

On 2026-09-28 both pass again on this branch. On 2026-10-02, on b442e64, the 20 live tests pass.

Run at DEBUG, no answer in the live suite had a column its model does not declare. A column the API sends today does: stocks.quotes("AAPL", use_52_week=True, output_format=OutputFormat.INTERNAL), which raises on main (#102). On d483055, at 2026-09-28 16:25 UTC, it returns the quote, logs the two columns and keeps them. The same run asked the live API for the CSV of the four single-object resources, in both name styles and under columns=, and every file passed the header check (headers cut at 100 characters):

options.chain CSV, API names: written, 5 lines, header 'optionSymbol,underlying,expiration,side,strike,firstTraded,dte,updated,bid,bidSize,mid,ask,askSize,l'
options.expirations CSV, API names: written, 26 lines, header 'expirations'
options.lookup CSV, API names: written, 2 lines, header 'optionSymbol'
stocks.earnings CSV, API names: written, 2 lines, header 'symbol,fiscalYear,fiscalQuarter,date,reportDate,reportTime,currency,reportedEPS,estimatedEPS,surpris'
options.chain CSV, human: written, 5 lines, header 'Symbol,Underlying,Expiration Date,Option Side,Strike,First Traded,Days To Expiration,Date,Bid,Bid Si'
options.expirations CSV, human: written, 26 lines, header 'Expirations'
options.lookup CSV, human: written, 2 lines, header 'Symbol'
stocks.earnings CSV, human: written, 2 lines, header 'Symbol,Fiscal Year,Fiscal Quarter,Date,Report Date,Report Time,Currency,Reported EPS,Estimated EPS,S'
options.chain CSV, columns=['strike', 'bid']: written, 5 lines, header 'strike,bid'
stocks.quotes('AAPL', use_52_week=True, output_format=INTERNAL):
  DEBUG: The API sent the columns ['52weekHigh', '52weekLow'], which StockQuote does not declare; get_extra() holds them
  get_extra: {'52weekHigh': Decimal('345.34'), '52weekLow': Decimal('242.7592')}

On 7f929dc, at 2026-09-28 17:35 UTC, both fan-outs answer on every format with the real API: the hourly candles over two yearly chunks, the option quotes over two symbols. No answer carried a column its model does not declare, so get_extra holds nothing and every chunk and symbol passed the merge's rule:

stocks.candles AAPL hourly 2023-06-01..2024-06-28 INTERNAL: 1891 rows, get_extra keys [] (responses: 2)
stocks.candles AAPL hourly 2023-06-01..2024-06-28 JSON: JSON, 6 keys (responses: 2)
stocks.candles AAPL hourly 2023-06-01..2024-06-28 DATAFRAME: DataFrame (1891, 5) (responses: 2)
stocks.candles AAPL hourly 2023-06-01..2024-06-28 CSV: CSV, 1892 lines, header 't,o,h,l,c,v' (responses: 2)
stocks.candles AAPL hourly, human INTERNAL: 1891 rows, get_extra keys [] (responses: 2)
stocks.candles AAPL hourly, human JSON: JSON, 6 keys (responses: 2)
stocks.candles AAPL hourly, human DATAFRAME: DataFrame (1891, 5) (responses: 2)
stocks.candles AAPL hourly, human CSV: CSV, 1892 lines, header 'Date,Open,High,Low,Close,Volume' (responses: 2)
options.quotes, two symbols INTERNAL: OptionsQuotes, get_extra keys [] (responses: 2)
options.quotes, two symbols JSON: JSON, 26 keys (responses: 2)
options.quotes, two symbols DATAFRAME: DataFrame (2, 24) (responses: 2)
options.quotes, two symbols CSV: CSV, 3 lines, header 'optionSymbol,underlying,expiration,side,strike,firstTraded,dte,updated' (responses: 2)
options.quotes, two symbols, human INTERNAL: OptionsQuotesHumanReadable, get_extra keys [] (responses: 2)
options.quotes, two symbols, human JSON: JSON, 25 keys (responses: 2)
options.quotes, two symbols, human DATAFRAME: DataFrame (2, 24) (responses: 2)
options.quotes, two symbols, human CSV: CSV, 3 lines, header 'Symbol,Underlying,Expiration Date,Option Side,Strike,First Traded,Days' (responses: 2)

Tests

• src/tests/test_money_decimal.py:
  • test_a_column_no_model_declares_is_kept_for_get_extra, on every INTERNAL resource that builds through columns, plain and human, the fan-outs included (22). The fixture logs no undeclared column. The fixture plus one column builds the same fields, get_extra holds the column (a row its own value) and it is logged once.
  • test_a_column_no_model_declares_reaches_every_format, on the same 22: the same answer carries the column on INTERNAL through get_extra, on JSON and DataFrame as a column, and in the CSV file.
  • test_get_extra_survives_copy_and_pickle: a copy, a deep copy and a pickled row keep what get_extra returns.
  • test_an_undeclared_value_that_is_not_a_list_goes_to_every_row: a single value goes to every row on INTERNAL, JSON and DataFrame.
  • test_an_undeclared_list_of_another_length_is_a_parse_error_on_every_format, on every case but options.lookup, which sends no list (21): a column the model does not declare with one value more than the rows raises one identical ParseError (columns of different lengths {...}) on INTERNAL, JSON and DataFrame, and on the CSV of utilities.status. test_a_declared_column_of_another_length_is_a_parse_error_on_every_format (19, without the two options.expirations cases, which carry one list): the same for a declared column one value short.
  • test_get_extra_is_empty_for_an_object_with_no_dict.
  • test_an_object_with_none_of_the_model_columns_is_a_parse_error, on the same 22: {"error": "upstream timeout"} raises ParseError.
  • test_a_foreign_object_is_a_parse_error_on_every_format, on the single-object models, plain and human (7): the same object raises one identical ParseError on INTERNAL, JSON and DataFrame, the unknown-columns one on CSV and the JSON-or-HTML one on the headerless CSV.
  • test_a_body_that_is_not_an_object_is_a_parse_error_on_every_format: [1], null, "x", 5 and true, answered as JSON, on the single-object models, plain and human (35). INTERNAL, JSON and DataFrame raise one identical ParseError, CSV the one for a header with none of the resource's columns, the headerless CSV the JSON-or-HTML one, and no file is written.
  • test_a_csv_without_a_header_row_is_written_as_it_came, on the four single-object resources and the two fan-outs, a row and a lone number, both labeled text/csv as the API labels CSV (12), and test_a_csv_header_the_csv_module_cannot_read_is_a_parse_error.
  • test_a_headerless_csv_that_reads_as_json_or_html_is_a_parse_error (4: a JSON object with no label, an HTML page labeled text/plain, text labeled text/html, a string labeled application/problem+json) and test_a_fan_out_headerless_csv_that_reads_as_json_is_a_parse_error (8: [1] with no label, and "x", 5 and true labeled application/json, on each fan-out). The helper that mocks the answer takes its Content-Type, application/json unless a test says otherwise.
  • test_a_body_missing_a_column_the_model_needs_is_a_parse_error (3), above.
• src/tests/test_stocks_candles.py: test_stocks_candles_csv_keeps_a_column_the_api_adds, over two chunks.
• src/tests/test_options_quotes.py: test_options_quotes_csv_keeps_a_column_the_api_adds, over two symbols, and test_options_quotes_an_added_column_one_symbol_lacks_is_a_parse_error: a symbol whose answer lacks the column another one sends fails the call on all four formats, naming that symbol.
• src/tests/test_utils.py: test_merge_csv_responses_keeps_a_column_the_resource_does_not_have, and a case of test_json_answer_columns_name_the_answer_that_breaks_the_merge for an undeclared column one answer lacks. test_json_answer_columns_follow_the_order_the_answers_send now expects an undeclared list to be a column, and a key with one value not to be.
• src/tests/test_stocks_prices.py: test_stock_price_from_dict_keeps_an_undeclared_column_for_get_extra and test_stock_prices_human_readable_from_dict_keeps_an_undeclared_column. stocks.prices no longer builds its rows through either from_dict, so both are pinned directly.
• src/tests/test_stocks_quotes.py: test_stock_quote_from_dict_keeps_an_undeclared_column_for_get_extra and test_stock_quotes_human_readable_from_dict_keeps_an_undeclared_column, for the same reason: stocks.quotes builds its rows with the constructor. Each builds the quote from a row plus brandNew, checks it equals the quote built without it, and that get_extra holds brandNew.
• The CSV test of options.chain, options.expirations, options.lookup and stocks.earnings answered the placeholder AS RECEIVED FROM API, which names none of their columns; it now answers a CSV with a header and still checks the file as it came.

Review round 1

Measured on 2026-09-28, on b8035b2.

2243a397: the logger is private. columns.py names it _logger, so this PR adds no public name and the version impact is a patch. It is the same marketdata.logger logger, and the DEBUG line does not change. The top-level names that each module this PR changes defines without a leading underscore, read with ast on main, on the reviewed head and on this one (imports left out, as the review rules leave them out of the surface):

marketdata/output_types/columns.py
  bc0619c: []
  6a6b3db: added ['logger'], removed []
  b8035b2: same as bc0619c
13 other modules: the same public names at every revision

8c8d6586: no sibling SDK refuses a column when it merges a CSV fan-out. None of them reads a chunk's column names: each keeps the first chunk's header and drops the repeated one from the later chunks. Read on each default branch on 2026-09-28:

• Go: mergeTextChunks (internal/response/formatted.go) drops a later chunk's first line when it equals the first chunk's, and keeps it as data otherwise.
• C#: StocksApi.MergeCandleCsv drops the first line of every chunk after the first.
• Java: StocksCsvResource.mergeCsvBodies does the same.
• PHP: the candles merge (src/Endpoints/Stocks.php) and the option quotes merge (src/Endpoints/Options.php) keep the first header and strip it from the later chunks.
• JS: stocks/candles.ts merges the chunks' JSON and has no CSV merge, per the check of 2026-09-23 for perf(csv): merge the fan-out bodies a row at a time #148; no commit has landed on sdk-js since.

Go, C# and Java do not merge option quotes in CSV: they answer a map per symbol, per the same check, and no commit since touches it. So the CSV merge needs no sibling issue.

Checks on b8035b2. uv run pytest -n 4 --cov=marketdata: 2017 passed, coverage 100% (3207 statements, 0 missed); no test was added or changed, so the numbers above stand. ./lint.sh clean. git merge-tree is clean against main, #154 and #158; against #157 it conflicts only in the five docstring lines of StockQuote.from_dict described above.

Review round 2

Measured on 2026-09-28, on 9f65a39.

16a66cb0: a body that is not a JSON object is one ParseError on every format that decodes it. options.chain, options.expirations, options.lookup and stocks.earnings decode through _parse_json_object on their INTERNAL, JSON and DataFrame branches. It decodes as parse_json does, with exact=True where INTERNAL had it, and raises ParseError for anything but an object. The check sits in each branch rather than ahead of the dispatch because a CSV request asks the API for format=csv, and that body is not JSON. The CSV file stays the API's text as it came, as on every resource: that format decodes nothing, the same reason test_a_csv_file_is_the_api_text_whatever_numbers_it_carries pins for a number the other formats refuse.

The seven cases (options.chain, options.expirations and stocks.earnings with both names, and options.lookup) through the public client, answered with each body on each format. all 7 means the seven agree once the model's name is set aside as <Model>.

On the reviewed head, b8035b2:

[1]   INTERNAL  all 7: ParseError: Response body is not a valid answer of this resource (<Model>() argument after ** must be a mapping, not list): '[1]'
[1]   JSON      all 7: returns ResultList '[1]'
[1]   DATAFRAME all 7: returns DataFrame '   0\n0  1'
[1]   CSV       all 7: writes '[1]'
null  INTERNAL  all 7: ParseError: Response body is not a valid answer of this resource (<Model>() argument after ** must be a mapping, not NoneType): 'null'
null  JSON      all 7: returns NoneType 'None'
null  DATAFRAME all 7: returns DataFrame 'Empty DataFrame\nColumns: []\nIndex: []'
null  CSV       all 7: writes 'null'

On 9f65a39:

[1]   INTERNAL  all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): '[1]'
[1]   JSON      all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): '[1]'
[1]   DATAFRAME all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): '[1]'
[1]   CSV       all 7: writes '[1]'
null  INTERNAL  all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): 'null'
null  JSON      all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): 'null'
null  DATAFRAME all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): 'null'
null  CSV       all 7: writes 'null'

On main bc0619c, JSON, DataFrame and CSV answer as on b8035b2, and INTERNAL raises a bare AttributeError ('list' object has no attribute 'items') on the three human-readable models. The DataFrame of null was the worst of them: an empty frame, which reads like an answer with no data.

test_a_body_that_is_not_an_object_is_a_parse_error_on_single_object_models became test_a_body_that_is_not_an_object_is_one_parse_error_on_every_decoded_format. Each of its 14 cases calls the four formats, asserts one identical message on INTERNAL, JSON and DataFrame, and checks that the CSV file holds the body's bytes. With main's src/marketdata all 14 fail, where 6 did. The test count does not change.

fe3053d2: INTERNAL keeps leaving out a column its model does not declare, by design. It is direction 1 of #111, which the issue weighed against keeping the column and against raising:

• INTERNAL returns the model, and a column the model has no field for has nowhere to go. The format already answers with the model's columns rather than the body's: columns= is ignored on INTERNAL and every field comes back, as the CHANGELOG says under the options.strikes removal.
• On main the same body made INTERNAL raise ParseError while JSON, DataFrame and CSV returned it: the output format decided whether the call raised, which the house rules make a blocking finding. This PR removes that, and the DEBUG line names the column until the model declares it.
• Keeping the column on the object (an extra mapping, direction 2) adds public surface to the 14 models and their human-readable twins, and can be an issue of its own. Leaving it out of the other formats takes a column away from their callers, and the CSV would have to be parsed where it is written as it came.
• sdk-go and sdk-csharp leave an unknown key out of their typed output the same way (see Summary).
• The Unchanged list above stands: on the single-request resources JSON, DataFrame and CSV carry the column; on stocks.candles and options.quotes, JSON and DataFrame leave it out when they merge, as on main, and only the CSV keeps it.

Checks on 9f65a39. uv run pytest -n 4 --cov=marketdata: 2017 passed, coverage 100% (3212 statements, 0 missed). The public names of round 1, over the 16 modules this PR now changes, are the same on bc0619c, b8035b2 and 9f65a39; _parse_json_object is private. ./lint.sh clean. git merge-tree is clean against main, #154 and #158; against #157 it conflicts only in the five docstring lines of StockQuote.from_dict, and with #157 merged the suite passes, 2033 tests.

Review round 3

Measured on 2026-09-28, on d483055. This round replaces what rounds 1 and 2 say about the version impact (patch), the public surface, the argument for fe3053d2 and the CSV that wrote a body that is not an object: the column is now kept, get_extra is a new public name, CSV refuses that body, and the impact is major.

1643b4a2: INTERNAL keeps the undeclared column, for get_extra. _split_fields hands every model built from an answer the columns it does not declare, and get_extra(model) returns them as the API sent them: the whole column on a single-object model, each row's own value on a list result, on every single-request resource and on the six from_dict. They live in the model's __dict__, where the response metadata already does, so a model's fields, == and repr do not change. The fan-outs are left as they are, by decision: their merge keeps only declared columns on INTERNAL, JSON and DataFrame, as on main, while their CSV keeps the column, as it has since this PR's first commit. Carrying the column through their merge on the other three formats is a change of its own. test_a_column_no_model_declares_reaches_every_format checks one answer on all four formats for the 18 single-request cases; its CSV is the API's text with the same added column, or on client.utilities.status the file written from the same decoded answer.

16a66cb0: CSV refuses the same body. The CSV branch of the four resources checks the answer's header row with _check_csv_header before writing the file: a header with none of the resource's columns raises ParseError, with the rule and the message the fan-out merge already uses, read as it reads it (a BOM off, blank lines skipped, a row the csv module cannot read refused). A header that passes leaves the file as the API's text, byte for byte. The seven cases, through the public client, on each format:

[1]   INTERNAL  all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): '[1]'
[1]   JSON      all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): '[1]'
[1]   DATAFRAME all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): '[1]'
[1]   CSV       all 7: ParseError: Response body is not a valid answer of this resource (unknown columns ['[1]']): '[1]'
null  INTERNAL  all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): 'null'
null  JSON      all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): 'null'
null  DATAFRAME all 7: ParseError: Response body is not a valid answer of this resource (not a JSON object): 'null'
null  CSV       all 7: ParseError: Response body is not a valid answer of this resource (unknown columns ['null']): 'null'

The three formats that decode the body give one message. CSV gives the one a header with none of the resource's columns gets, since what is wrong with that text, read as CSV, is its header. Under add_headers=False there is no header row to check, and the file is written as it came. Live, the CSV of the four resources passes the check in both name styles and under columns= (see Live above).

c7feefea: major. Version impact says major (1.3.0 -> 2.0.0), the release this ships in, with the migration for each affected call. The #111 entry moved from Fixed to Changed (BREAKING), with its migration, and get_extra is under Added. The breaking change is on the feat(models)!: commit d483055, whose BREAKING CHANGE: footer names every breaking behavior of this PR, the earlier commits' included. As in #146 (a1f4764), the history is not rewritten.

Surface. The public names of round 1, over the 17 modules this PR now changes. The only new one is get_extra, which marketdata/__init__.py re-exports and lists in __all__:

marketdata/output_types/columns.py
  bc0619c: []
  9f65a39: same as bc0619c
  d483055: added ['get_extra'], removed []
16 other modules: the same public names at every revision

Checks on d483055. uv run pytest -n 4 --cov=marketdata: 2043 passed, coverage 100% (3259 statements, 0 missed); 63 of the 92 new tests fail with main's src/marketdata. ./lint.sh clean. git merge-tree is clean against main, #154 and #158; against #157 it conflicts as Overlap describes, and with #157 merged and resolved that way the suite passes, 2059 tests. The head, bbeea3c, only adds docstrings (render and the four CSV tests), and the suite (2043 passed, coverage 100%, 3259 statements, 0 missed), git merge-tree and ./lint.sh ran on it again with the same results.

Live, at 2026-09-28 16:35 UTC, columns= with the API's aliases does not trip the CSV header check: the API answers the canonical name in the header (price comes back as mid, date as updated on options.chain), and on options.expirations with date and stocks.earnings with price it answers an empty body, which is written as it came.

Review round 4

Measured on 2026-09-28, on 7f929dc. This round replaces what round 3 says about the fan-outs and about the CSV without a header row.

b4bc19ed: the fan-outs carry the column through their merge. json_answer_columns now returns every key but the status flag that holds a list, the model's or not, and holds each to the rule a declared column follows: every chunk or symbol must send it as a list as long as its rows. So the merged answer of stocks.candles and options.quotes carries an added column on JSON and DataFrame by the API's name, and INTERNAL keeps it for get_extra, each row's value on candles and the whole column on option quotes, as the merged CSV already kept it. An answer that lacks it, or sends another length, fails the call with ParseError naming that answer on every format, as the CSV merge already did for a header that differs from the first answer's. test_a_column_no_model_declares_reaches_every_format now runs on the fan-outs too (22 cases), and test_options_quotes_an_added_column_one_symbol_lacks_is_a_parse_error checks the refusal on the four formats. Live, both fan-outs answer on every format with the real API over two symbols and two yearly chunks (see Live).

248e8985: a headerless CSV that reads as JSON or HTML is refused. Without a header row there is nothing to match against the resource's columns, and comparing the width of the rows would refuse an answer the moment the API adds a column. What a CSV row of these resources never does is start with [, { or <, or read null: its first value is a symbol, a date, a number or a quoted text, under columns= too. So _reads_as_json_or_html refuses such a body, with ParseError: Response body is not a valid answer of this resource (JSON or HTML, not CSV): '[1]', in the CSV branch of the four single-object resources and in the fan-out merge, whose headerless path had the same gap. The single-object cases of test_a_body_that_is_not_an_object_is_a_parse_error_on_every_format ask for the headerless CSV as well, and any other headerless body is still written as it came.

d848f9bb: get_extra is documented. README.md has a "Columns a model does not declare" section under Output Formats, next to the one on money: which results carry it, whole or per row, how its values are decoded, how long they last (on the object itself, through copy, deepcopy and pickle, outside ==, the repr and dataclasses.fields(), which test_get_extra_survives_copy_and_pickle pins), and what the other formats do. The review rules leave *.md out of the diff, so this is where the section is announced.

Checks on 7f929dc. uv run pytest -n 4 --cov=marketdata: 2054 passed, coverage 100% (3266 statements, 0 missed); 74 of the 103 new tests fail with main's src/marketdata. get_extra is still the only new public name, over the 19 modules this PR now changes. ./lint.sh clean. git merge-tree is clean against main, #154 and #158; against #157 it conflicts in the same two places Overlap describes, and with #157 merged and resolved that way the suite passes, 2070 tests. The breaking changes of this round are on the feat(resources)!: commit 548338c, with a BREAKING CHANGE: footer; 7f929dc adds the README section.

Review round 5

Measured on 2026-09-28, on 7759248.

7ba21780: a foreign object is refused on every format of the single-object resources. _parse_json_object, which the INTERNAL, JSON and DataFrame branches of options.chain, options.expirations, options.lookup and stocks.earnings decode through, now also requires at least one of the model's columns, the rule the fan-out merge applies, with its message: ParseError: Response body is not a valid answer of this resource (none of this resource's fields): '{"error": "upstream timeout"}'. The CSV already refused that body by its header. {"error": "upstream timeout"} through the public client, on each format, on 7f929dc:

stocks.earnings              INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError
stocks.earnings human        INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError
options.chain                INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError
options.chain human          INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError
options.expirations          INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError
options.lookup               INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError
options.expirations human    INTERNAL=raises ParseError  JSON=returns ResultDict  DATAFRAME=returns DataFrame  CSV=raises ParseError

and on 7759248:

stocks.earnings              INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError
stocks.earnings human        INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError
options.chain                INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError
options.chain human          INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError
options.expirations          INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError
options.lookup               INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError
options.expirations human    INTERNAL=raises ParseError  JSON=raises ParseError  DATAFRAME=raises ParseError  CSV=raises ParseError

test_a_foreign_object_is_a_parse_error_on_every_format checks the four formats for the seven cases. The options.lookup case of test_a_body_missing_a_column_the_model_needs_is_a_parse_error moved to options.expirations: the lookup model has one column, so removing it leaves an object with none of the model's columns, which is now refused for that reason instead.

The list resources (funds.candles, markets.status, stocks.news, stocks.prices, stocks.quotes, client.utilities.status) are left as they are: INTERNAL refuses the object, and JSON, DataFrame and CSV return it, as on main. That is the malformed-body work of #110, as this PR said from the start.

Live, at 17:59 UTC, the answers of the four resources pass the new rule on the three formats that decode them, in both name styles and under columns=, an alias included:

options.chain, API names: INTERNAL=OptionsChain, JSON=ResultDict, DATAFRAME=DataFrame
options.expirations, API names: INTERNAL=OptionsExpirations, JSON=ResultDict, DATAFRAME=DataFrame
options.lookup, API names: INTERNAL=OptionsLookup, JSON=ResultDict, DATAFRAME=DataFrame
stocks.earnings, API names: INTERNAL=StockEarnings, JSON=ResultDict, DATAFRAME=DataFrame
options.chain, human: INTERNAL=OptionsChainHumanReadable, JSON=ResultDict, DATAFRAME=DataFrame
options.expirations, human: INTERNAL=OptionsExpirationsHumanReadable, JSON=ResultDict, DATAFRAME=DataFrame
options.lookup, human: INTERNAL=OptionsLookupHumanReadable, JSON=ResultDict, DATAFRAME=DataFrame
stocks.earnings, human: INTERNAL=StockEarningsHumanReadable, JSON=ResultDict, DATAFRAME=DataFrame
options.chain, columns=['strike', 'bid']: INTERNAL=OptionsChain, JSON=ResultDict, DATAFRAME=DataFrame
options.chain, columns=['price']: INTERNAL=OptionsChain, JSON=ResultDict, DATAFRAME=DataFrame

Checks on 7759248. uv run pytest -n 4 --cov=marketdata: 2061 passed, coverage 100% (3268 statements, 0 missed); 81 of the 110 new tests fail with main's src/marketdata. ./lint.sh clean. git merge-tree is clean against main, #154 and #158; against #157 it conflicts in the same two places Overlap describes, and with #157 merged and resolved that way the suite passes, 2077 tests. The breaking change of this round is on the feat(resources)!: commit 7759248, with a BREAKING CHANGE: footer.

Review round 6

Measured on 2026-09-28, on 21efa02.

Proof and 4f4bb881: both quote from_dict, pinned directly. stocks.quotes builds its rows with the constructor, so no endpoint test reaches StockQuote.from_dict or StockQuotesHumanReadable.from_dict. 1519681 adds a direct test for each, next to the existing from_dict tests and away from the tests #157 appends to that file, as test_stocks_prices.py does for the price models. The same row plus brandNew, through both methods, with main's src/marketdata (bc0619c):

StockQuote.from_dict: TypeError: StockQuote.__init__() got an unexpected keyword argument 'brandNew'
StockQuotesHumanReadable.from_dict: TypeError: StockQuotesHumanReadable.__init__() got an unexpected keyword argument 'brandNew'

and on 21efa02:

StockQuote.from_dict: StockQuote, get_extra -> {'brandNew': 1}
StockQuotesHumanReadable.from_dict: StockQuotesHumanReadable, get_extra -> {'brandNew': 1}

Both tests fail with main's src/marketdata and pass on this branch; the suite with them is the coverage run below.

eb622b26: get_extra on each resource page. 21efa02 adds a "Columns a model does not declare" section to docs/funds.md, markets.md, options.md, stocks.md and utilities.md, before their methods: what get_extra holds for that resource's results, whole on a single-object result and per row on a list, that candles() and quotes() need the column in every chunk or symbol, that utilities.user() and utilities.headers() keep nothing, and a link to the README section for how the values are decoded and how long they last. The review rules leave docs/** out of the diff, so this is where it is announced.

Checks on 21efa02. uv run pytest -n 4 --cov=marketdata: 2063 passed, coverage 100% (3268 statements, 0 missed); 83 of the 112 new tests fail with main's src/marketdata. ./lint.sh clean. git merge-tree is clean against main, #154 and #158; against #157 it conflicts in the same two places Overlap describes (docs/stocks.md and test_stocks_quotes.py merge on their own), and with #157 merged and resolved that way the suite passes, 2079 tests.

Review round 7

Measured on 2026-10-02, on b442e64.

8f792ec9: a headerless CSV answer labeled JSON or HTML is refused. The body alone cannot tell a JSON scalar from a CSV answer, because several answers the API sends under add_headers=False are valid JSON; the Content-Type can. What the API sent, live on 2026-10-02:

stocks/quotes/AAPL/ columns=last, CSV without header          200 text/csv; charset=utf-8  '333.435\r\n'
stocks/candles/D/AAPL/ columns=t countback=1, same            200 text/csv; charset=utf-8  '1790740800\r\n'
options/quotes/AAPL271217C00250000/ columns=inTheMoney, same  200 text/csv; charset=utf-8  'True\r\n'
stocks/candles/D/AAPL/ on 2026-01-01, same                    200 text/csv; charset=utf-8  '""\r\n'
stocks/quotes/NOTASYMBOLXQ/, same                             400 text/csv; charset=utf-8  'error,"Bad parameters, please check API documentation."\r\n'
stocks/quotes/AAPL/ columns=last, JSON                        203 application/json         '{"last":[333.34]}'
stocks/nosuchendpoint/AAPL/, CSV without header               404 application/json         '{"s": "no_data"}'

333.435, 1790740800 and the empty-answer placeholder "" parse as JSON, so a check that decodes the body would refuse the API's own answers; a boolean comes as True, which does not. Every CSV answer, the error and the empty one included, is labeled text/csv, and every JSON one application/json, down to a route the API does not have, which answers a CSV request with JSON. So _reads_as_json_or_html reads the label first: an answer labeled */json, *+json, text/html or application/xhtml+xml is refused with the same ParseError (JSON or HTML, not CSV), whatever its body, and under any other label or none the check on the body's shape stays. It runs where it ran: the CSV branch of options.chain, options.expirations, options.lookup and stocks.earnings, and the fan-out merge of stocks.candles and options.quotes. With a header row nothing changes: "x", 5 and true already fail the header check (unknown columns ['x']).

Through the public client, each answer mocked with its label, under add_headers=False, on 21efa02:

"x"         application/json  4 single-object: writes '"x"'   2 fan-outs: writes 'x\r\n'
5           application/json  4 single-object: writes '5'     2 fan-outs: writes '5\r\n'
true        application/json  4 single-object: writes 'true'  2 fan-outs: writes 'true\r\n'
1790740800  text/csv          all 6: writes '1790740800\r\n'

and on b442e64:

"x"         application/json  all 6: ParseError: Response body is not a valid answer of this resource (JSON or HTML, not CSV): '"x"'
5           application/json  all 6: ParseError: Response body is not a valid answer of this resource (JSON or HTML, not CSV): '5'
true        application/json  all 6: ParseError: Response body is not a valid answer of this resource (JSON or HTML, not CSV): 'true'
1790740800  text/csv          all 6: writes '1790740800\r\n'

The fan-outs did not even write the body: their merge read it as a CSV cell, so "x" became x. The single-object resources raise ParseError for the same three answers on INTERNAL, JSON, DataFrame and the CSV with a header, and the fan-outs on JSON, so the headerless CSV now agrees with the other formats.

Live, at 15:51 UTC on b442e64, the headerless CSV of the six resources through the SDK: every answer is labeled text/csv; charset=utf-8 and every file is written, a lone number and a lone True included (first 40 bytes; stocks.candles with columns=["t"] and countback=1, options.quotes with columns=["inTheMoney"]):

options.expirations: wrote b'1790913600\r\n1791172800\r\n1791345600\r\n1791'
options.lookup: wrote b'AAPL271217C00250000\r\n'
stocks.earnings: wrote b'AAPL,2026,4,1790740800,1793246400,after '
options.chain: wrote b'AAPL271217C00250000,AAPL,1829077200,call'
stocks.candles: wrote b'1790740800\r\n'
options.quotes: wrote b'True\r\n'

What stays: an answer with no label, or labeled text/plain or text/csv, whose body is a lone string, number or boolean is still written. Nothing on the wire tells it from a one-cell CSV, and text/csv is how the API labels its own. The placeholder "" is read as the empty answer before any check, on every format, as before.

Siblings: none of them checks the body of a CSV answer (see round 1), so each writes a JSON scalar, or a JSON object, as CSV. sdk-go already reads the Content-Type, to choose the parser of an error envelope (parseErrorBody in internal/http/client.go), and sdk-php sends an Accept per format; sdk-js and sdk-java do not mention text/csv. No sibling issue is proposed.

Tests: test_a_body_that_is_not_an_object_is_a_parse_error_on_every_format adds "x", 5 and true (21 cases), test_a_fan_out_headerless_csv_that_reads_as_json_is_a_parse_error adds them labeled JSON (6), test_a_headerless_csv_that_reads_as_json_or_html_is_a_parse_error adds text labeled HTML and a string labeled application/problem+json (2), and test_a_csv_without_a_header_row_is_written_as_it_came adds the fan-outs and a lone number labeled text/csv (8). With 21efa02's src/marketdata, the 29 refusals fail and the 8 written cases pass.

Checks on b442e64. uv run pytest -n 4 --cov=marketdata: 2125 passed, coverage 100% (3284 statements, 0 missed); 112 of the 149 new tests fail with main's src/marketdata. src/tests/integration -m integration: 20 passed. This round adds no public name: _reads_as_json_or_html is private. ./lint.sh clean. git merge-tree is clean against main, and no other pull request is open. The breaking change of this round is on the feat(resources)!: commit 4fb87ed, with a BREAKING CHANGE: footer.

Review round 8

Measured on 2026-10-02, on a6cd05e.

fbc495c7: an answer whose columns differ in length is one ParseError on every decoded format. On b442e64 no format raised for it: neither pandas nor polars fails to build the frame, as the finding expected; both build one row more than INTERNAL, with nulls in the declared columns. The formats disagreed on the rows instead. funds.candles (7 rows) through the public client, with an added brandNew of 8 values, and with t one value short; DataFrame is the same on pandas and polars:

added brandNew of 8
  main c4ee8e1  INTERNAL  ParseError: ... (FundsCandle.__init__() got an unexpected keyword argument 'brandNew')
                JSON      returns the body, brandNew [0, ..., 7]
                DATAFRAME 8 rows; the 8th has brandNew 7 and nulls in t, o, h, l, c
  b442e64       INTERNAL  7 rows, each with get_extra {'brandNew': [0, 1, 2, 3, 4, 5, 6, 7]}
                JSON      returns the body
                DATAFRAME 8 rows, as on main
  a6cd05e       INTERNAL, JSON, DATAFRAME: ParseError: Response body is not a valid answer of this resource (columns of different lengths {'t': 7, 'o': 7, 'h': 7, 'l': 7, 'c': 7, 'brandNew': 8})
t one value short
  main, b442e64 INTERNAL  6 rows, the 7th dropped without a word
                JSON      returns the body
                DATAFRAME 7 rows, a null in t
  a6cd05e       INTERNAL, JSON, DATAFRAME: ParseError: Response body is not a valid answer of this resource (columns of different lengths {'t': 6, 'o': 7, 'h': 7, 'l': 7, 'c': 7})

The rule is the one the fan-outs' merge applied, with its message: every key but the status flag s that holds a list, declared or not, holds one as long as the others. json_answer_columns and the single-request resources share it (_check_column_lengths), through _parse_json_object on options.chain, options.expirations, options.lookup and stocks.earnings, _parse_json_columns on funds.candles, markets.status, stocks.news, stocks.prices and stocks.quotes, and render on client.utilities.status, before any branch. options.chain had the same extra DataFrame row; options.quotes, whose answer has the same shape, already raised. The lines the finding quotes, _with_row_extra in columns.py, are unchanged: the check runs on the decoded answer before any row is built, so a list of another length than the declared lists no longer reaches them. test_an_undeclared_list_of_another_length_is_a_parse_error_on_every_format (21) and test_a_declared_column_of_another_length_is_a_parse_error_on_every_format (19) check one identical message on the three formats for every case, in both name styles, the fan-outs included.

The fourth format, CSV:

• client.utilities.status writes its CSV from the decoded answer, so its CSV raises the same ParseError, and both tests check it.
• Every other resource writes the API's text, which holds rows, not columns of their own length: the counterpart of a longer column is a CSV whose last row is short, t,brandNew\r\nx,1\r\ny,2\r\n3\r\n. On main and on this branch, funds.candles, markets.status, stocks.news, stocks.prices, stocks.quotes, options.chain, options.expirations and stocks.earnings write it as it came, and the fan-outs refuse it by its width, row ['3'] does not have 2 values (on main by its header, unknown columns ['brandNew']). A width check on the CSV of the single-request resources is malformed-body work, which stays with Reading a body can raise a non-SDK exception: RecursionError on nested JSON, TypeError/UnicodeError/AssertionError on an exotic charset #110, as the Unchanged list says for the list resources.

A single value instead of a list, "brandNew": "new", gives every row that value on INTERNAL, JSON and DataFrame, on both libraries, before and after this round. test_an_undeclared_value_that_is_not_a_list_goes_to_every_row now pins it on the three formats; the "a list of another length" case of the test it replaces pinned the INTERNAL reading the finding names.

Live, the rule breaks no answer the API sends. At 18:11 UTC, the JSON of 13 endpoints, in both name styles, had every list one length:

stocks.quotes AAPL,MSFT,TSLA,SPY,F,SIRI 52week  13 lists of 6     stocks.candles D AAPL countback=30   6 lists of 30
stocks.quotes AAPL                              11 lists of 1     markets.status September             2 lists of 31
stocks.prices AAPL,MSFT,SPY,GME                  5 lists of 4     utilities.status                     6 lists of 12
stocks.news AAPL                                 5 lists of 10    funds.candles D VFINX countback=10   5 lists of 10
stocks.earnings AAPL countback=8                12 lists of 1     options.chain AAPL strikeLimit=40   25 lists of 80
options.expirations AAPL                         1 list of 25     options.lookup                       no list
options.quotes AAPL271217C00250000              25 lists of 1

At 18:25 UTC, on a6cd05e, through the SDK, INTERNAL, JSON and DataFrame of the ten resources, in both name styles (utilities.status has one), returned their result: stocks.quotes 6 rows with the 52-week range, stocks.prices 4, stocks.news 10, stocks.earnings 1, markets.status 31, funds.candles 10, options.chain 20 contracts, options.expirations 25, options.lookup 1 and utilities.status 12, none refused. src/tests/integration -m integration: 20 passed.

No sibling SDK keeps a column its model does not declare, and none builds a DataFrame, so none has this case, as the review says.

Checks on a6cd05e. uv run pytest -n 4 --cov=marketdata: 2164 passed, coverage 100% (3297 statements, 0 missed). 147 of the 188 new tests fail with main's src/marketdata. Of this round's 41, 32 fail with b442e64's: the 9 that pass are the eight fan-out cases, which the merge already refused, and the single value. This round adds no public name: _check_column_lengths and _parse_json_columns are private. ./lint.sh clean. git merge-tree is clean against main, and no other pull request is open. The breaking change of this round is on the feat(resources)!: commit a6cd05e, with a BREAKING CHANGE: footer.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (c4ee8e1) to head (a6cd05e).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #159   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           62        62           
  Lines         3208      3297   +89     
=========================================
+ Hits          3208      3297   +89     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[marketdata-code-review]

✅ Ready — Code review

This adds marketdata.get_extra, preserves undeclared API columns across output formats, and consistently rejects malformed or uneven column payloads. Version impact: major (1.3.0 -> 2.0.0). The described sibling review found no equivalent typed-output defect in sdk-go or sdk-csharp and no equivalent strict CSV merge in sdk-go, sdk-csharp, sdk-java, sdk-php, or sdk-js.

Rating patch 6/6 (exemplary) · proof 6/6 (comprehensive) · overall 6/6

Since last review

• Resolved: fbc495c7
• 0 still open, 0 new

---

Reviewed a6cd05e · revision 9 · codex (gpt-5.6-sol) · claude unavailable · rules: default, sdk

[MarketDataDev02]

@marketdata-code-review re-review