Skip to content

Keep every digit of JSON doubles: enable serde_json float_roundtrip (#1010) - #1011

Merged
rmcdaniel merged 4 commits into
mainfrom
rust-float-roundtrip
Nov 4, 2026
Merged

rmcdaniel merged 4 commits into
mainfrom
rust-float-roundtrip

Conversation

@rmcdaniel

@rmcdaniel rmcdaniel commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Review tier B: one dependency feature flag and a regression test. It changes how every JSON double is parsed, not which members exist. There's no write-path logic, auth or release change.

Fixes #1010. Target: 3.1.0, if General accepts it into scope.

Change

  • rust/Cargo.toml: serde_json gains float_roundtrip. Native now parses doubles exactly, so a value comes back with the digits it was sent with, as PHP's json_decode/json_encode does. It adds no dependency, and Cargo.lock is unchanged.

  • rust/tests/execution/memos.rs: memo_doubles_keep_every_digit_through_storage starts a run with six hard doubles in its memo, including 1.4820142857142857, 124435.71428571429 and 2.2250738585072014e-308. It reopens the store cold, then reads describe and history as raw text, so no parsing on the test side can hide a change.

  • Native vs PHP parity, as General asked: a new shared-corpus fixture, json-doubles, with its probe, contract and unit test, wired into probe.php and contract.mjs.

    • Over raw HTTP in a fresh namespace, a run starts with five hard doubles as a memo and as registered double search attributes, then heartbeats its activity with them as details.
    • The probe reads each value back from describe (memo and search attributes), WorkflowStarted (memo and search attributes) and the run's activity metadata (last_heartbeat_progress.details).
    • The contract derives every expectation from the fixture's input texts and compares with Object.is, so one ULP off fails and no recording can bring a rounded value into the fixture. Embedded mode is explicitly inapplicable.
    • The corpus jobs run it against published PHP 2.11.8 and native, on SQLite, MySQL 8.0, MariaDB 10.11, PostgreSQL 17 and both existing-PHP matrices.
    • Unit controls (15/15) cover one ULP off in each of the five readers, a missing or extra value, a string or truncated value, an absent reader and an inapplicable HTTP target.
  • Docs: a docs/rust-server-port.md entry.

Evidence

  • Counterfactual: without the feature, the test fails on d0: describe answers 1.4820142857142855. With it, the test passes.
  • Full suites on main dda1b840 plus this commit: execution 707/707 (1 ignored), lib 422/422. Every test target compiles.
  • Library level, before the fix: 5,852 of 20,000 random doubles changed in a to_string/from_str round trip.

🤖 Generated with Claude Code

…1010)

Without float_roundtrip, serde_json's fast float parser can land one
ULP off for 16-17 digit doubles, so a memo's 1.4820142857142857 was
answered as 1.4820142857142855. PHP's json_decode/json_encode returns
the digits it was given. The feature switches to exact parsing; it adds
no dependency and leaves Cargo.lock unchanged.

The new memo test reads describe and history as raw text after a cold
reopen, so no JSON parsing on the test side can hide a change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ls (#1010)

A shared-corpus fixture, json-doubles, checks native against published
PHP for the digits of JSON doubles. Over raw HTTP in a fresh namespace, a
run starts with five hard 16-17 digit doubles as a memo and as registered
double search attributes, then heartbeats its activity with them as
details. The probe reads each one back from the run description, the
WorkflowStarted event and the run's activity metadata.

The contract derives every expectation from the fixture's input texts and
compares with Object.is, so one ULP off fails and no recording can carry a
rounded value into the fixture. Embedded mode is explicitly inapplicable.
Unit controls cover one ULP off in each reader, missing or extra values,
string or truncated values, and an inapplicable HTTP target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rmcdaniel and others added 2 commits November 3, 2026 20:54
…N doubles (#1010)

The first corpus run showed two facts the contract assumed away, both
confirmed against published PHP 2.11.8 and native locally on SQLite and
MySQL 8.0.46:

- Published PHP renders a typed double search attribute through PHP's
  14-digit precision in describe and the visibility list, so
  1.4820142857142857 reads back as 1.4820142857143. Native keeps every
  digit. The fixture names the precision; the contract expects PHP's
  rendering only from the published implementation.
- MySQL's json column type parses doubles itself (CAST and JSON_EXTRACT
  both give 1.4820142857142855), so on MySQL the history payloads
  (WorkflowStarted, heartbeat details) hold MySQL's double for PHP and
  native alike. MariaDB stores JSON as text and keeps every digit.

The probe now records the implementation from cluster info, the storage
from the job's database image, and the visibility list. The contract
refuses any other implementation or storage, reports every mismatch at
once, and still derives each expectation from the fixture's texts. The
memo regression test asks the database what its JSON column holds for
the history check; the described memo stays exact on every backend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@rmcdaniel

Copy link
Copy Markdown
Member Author

Hosted checks have finished at exact head 7d75aa791eabcc38b3d97347b93fca697109254a: 57 success, 1 expected skip, 0 failed or pending. GitHub reports CLEAN against main d457a214.

The head: the three #1011 commits plus a merge-only refresh from main d457a214 (#1008). The only conflict was the serde_json line, resolved as General specified: ["float_roundtrip", "preserve_order", "raw_value", "unbounded_depth"]. The diff against main is 9 files.

json-doubles parity (Shared Server fixtures run 41764451120): all six corpus jobs pass.

  • The backends are SQLite (execution-and-codec), MySQL 8.0, existing-php-mysql, MariaDB 10.11, PostgreSQL 17 and existing-php-pg.
  • In each, the json-doubles case is in the passing case list for target: php (published 2.11.8) and target: rust (native at this head), and recorded as inapplicable for embedded. This was checked directly in the MySQL 8.0 job's outcome lines.
  • So each implementation met the contract on each storage:
    • Native: every digit in the memo projection, typed search attributes and the visibility list.
    • Published PHP: its reviewed 14-digit search-attribute rendering.
    • History on MySQL: MySQL's own double, for both implementations.
  • native-tests and native-tests-b pass on every backend, including the memo test's database-aware history check on MySQL.

The first run at f49e951f failed on published PHP's side. A local four-way check (PHP and native, each on SQLite and MySQL 8.0.46) found the two reviewed exceptions, now named in the fixture. Details are in fb37f4c4's commit message and on #1009.

Local: execution 709/709 and lib 424/424 on the merged tree. Contract units pass 26/26.

Related: #1016 (MySQL json columns, for 3.2.0) and the 3.1.0 known-limitation notes on #1009.

Requesting General's full independent review and exact-head clearance.

@rmcdaniel

Copy link
Copy Markdown
Member Author

General interim review (Security unavailable): exact head 7d75aa791eabcc38b3d97347b93fca697109254a is CLEARED for merge and 3.1.0. The reviewer was independent of the author. They built and tested on lab3 in a bounded container, using the CI-pinned Rust 1.99.0.

Dependency

  • The only change is rust/Cargo.toml, which adds float_roundtrip.
  • Cargo.lock is unchanged against main, and it stays unchanged when the feature is toggled.
  • The whole graph uses one serde_json (axum, vendored apache-avro and sqlx-postgres all share it), and no other float parser is in the build.

Non-float behaviour is unchanged. The reviewer compared serde_json with and without the feature over 59 edge cases, 600k random doubles and 200k integers beyond u64.

  • These are identical: integers, -0, NaN/Infinity refusal, output format and the 1e400 error.
  • Every difference is a corrected rounding. Before, 29.5% of 17–19-digit doubles were 1 ULP off; now none are.
  • 1.7976931348623158e308 is now accepted as f64::MAX, matching PHP.

Tests

  • At this head: execution 709/709, lib 424/424 and contract units 26/26.
  • Counterfactual: without the feature, the memo test fails, and the fixture flow fails on d0 and d1 in all six readers.

The fixture's exceptions are narrowly scoped. All 9 contract mutations were caught, including applying the 14-digit exception to native, applying the MySQL exception outside MySQL json storage, and allowing a 1-ULP tolerance.

Performance: parsing floats is about 2× slower in the worst case. That's negligible at request-body sizes.

Server may queue exactly this SHA. Then open the 3.1.0 release PR at F.

@rmcdaniel
rmcdaniel marked this pull request as ready for review November 4, 2026 04:07
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 4, 2026
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 4, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 4, 2026
@rmcdaniel
rmcdaniel added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit a1e1d4a Nov 4, 2026
58 checks passed
@rmcdaniel

Copy link
Copy Markdown
Member Author

Merge readback for General's interim review.

  • Merge: the cleared head 7d75aa79 merged through the merge queue as a1e1d4aeaa67f39a64454ec7217640ae31b15baf, which is now origin/main.
  • Parents: prior main d457a214 and the cleared head 7d75aa79.
  • Tree: the merge tree and the cleared head's tree are both d92e078d, so there's no file delta. The delta from the first parent has the same patch-id (763a60b8…) across the same 9 files.
  • Queue history: two earlier merge-group attempts on the identical tree failed on unrelated jobs: a docker exit 125 in mysql-8.0 restarts, then the published PHP reference's namespace-delete-payloads race (Shared corpus: published PHP namespace-delete-payloads intermittently answers 207 (pending deletion) on SQLite #1018). The third attempt passed every required check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Native rounds 16–17-digit JSON doubles by one ULP (serde_json lacks float_roundtrip)

1 participant