Skip to content

Write a workflow poll's binding and answer in one upsert (#1063) - #1166

Merged
rmcdaniel merged 1 commit into
mainfrom
feat/server-1063-transition-reads
Nov 12, 2026
Merged

rmcdaniel merged 1 commit into
mainfrom
feat/server-1063-transition-reads

Conversation

@rmcdaniel

Copy link
Copy Markdown
Member

Part of #1063 (Native: drop locked statements whose answers the transition already has). This PR covers the one checklist item that is still open: writing a poll's binding and result cache rows in one two-row upsert (639b6854 in the report).

Background

The server agent's saved branch wip/server/20261112-0922/feat/server-1063-transition-reads (780d0aae) turned out to be the #1099 branch exactly. git cherry origin/main marks all 13 of its commits as already upstream (merged as a5498678, released in Native 3.3.1). Nothing from it still needed rebasing or finishing. That branch also contains 2e11e8f1, which reverted this item in #1099. This PR starts again from current main (6e1d1028, which includes #1016 and #980) and adds the pair writer back, with a different design for the expiry origin.

Change

remember_workflow_poll remembers a workflow poll's answer. It used to run two QUERY_CACHE_PUT upserts one after the other inside the claim transaction: first the binding row ({"task_kinds":…}), then the result row. It now runs a single QUERY_CACHE_PUT_PAIR:

  • PostgreSQL / SQLite: INSERT … VALUES ($1,$2,$3),($4,$5,$6) ON CONFLICT ("key") DO UPDATE SET value=excluded.value,expiration=excluded.expiration
  • MySQL / MariaDB: the same rows with ON DUPLICATE KEY UPDATE value=VALUES(value),expiration=VALUES(expiration). key is the table's only unique key, so this behaves exactly like the existing single-row QUERY_CACHE_PUT.

Statement dropped: one locked dw_query_cache upsert for every remembered workflow poll answer. That covers leased claims (store.rs), carried claims (carried_polls.rs), update-validation leases and empty or task_queue_changed answers. The caller test measures 2 → 1 statements.

Why it's safe

  • Same rows. Keys, JSON values and retention rules are unchanged. The binding keeps result_ttl. A workflow lease's answer is kept for 3600 s, and any other answer for the binding's ttl. The qualification compares the pair statement with the two sequential upserts byte for byte: same value and same expiration for both rows. It does this for each existing-row state (none, binding only, result only, both) and for escaped and non-ASCII JSON. It also checks that a neighbouring key is left alone.

  • Locking and ordering. Every caller has already taken the binding row's lock through bind_workflow_poll (QUERY_CACHE_LOCK) earlier in the same transaction. The pair statement writes the rows in the same order as before (binding, then result). No new lock order is introduced, and duplicates of the same request ID remain serialized on the binding.

  • Rollback. Both rows are written in the caller's transaction. The qualification commits old rows, rolls back a pair write, and checks that both committed rows come back with the same value and expiry, or that no row exists where there was none.

  • Expiry origin. This is why Reduce Native poll and completion database work #1099 reverted the item. Each expiry is still epoch() + that row's ttl, read just before the statement that writes the row, the same as each of PHP's two puts reads its own clock. The difference is that one statement now writes both rows, so they share one clock reading. The old sequential writer read the result's clock after the binding's round trip. Expiration is stored in whole seconds, so the two writers differ only when that round trip crosses a second boundary. In that case the sequential writer stored the answer's expiry one second later.

    • The pair writer's values are always ones the sequential writer also produces whenever no boundary is crossed.
    • The pair writer can only make the answer expire earlier, never later, so no retention window is extended.
    • binding expiry ≤ result expiry still holds, which is the invariant assert_workflow_poll_binding_expiry_for_test guards.

    The caller test pins this down. It runs one statement, checks each expiry falls within [before+ttl, after+ttl], and checks that result − binding == kept − ttl exactly. That last check fails if the two rows ever drift onto different clock readings or swap retentions.

Reviewer decision needed: the issue currently says the pair writer stays deferred "to preserve the binding and result writes' separate expiration-time origins". With a single statement, those origins can't be kept apart. This PR's position is that the sub-second shift described above is not a contract change. If you disagree, close this PR and the item stays deferred. Nothing else depends on it.

No schema, protocol or capability change. No benchmark speedup is claimed.

Tests

Run locally in the CI toolchain image (mirror.gcr.io/library/rust@sha256:ba81bc3e…, Rust 1.99.0), with CARGO_BUILD_JOBS=2 and --cpus=2 --memory=8g. Throwaway databases ran on a private docker network. I tested the commit on 6e1d1028 and then rebased it onto d041f5ba; that commit only changes docker-compose.published.yml, and the Rust tree is identical.

Backend Suite Result
SQLite cargo test --locked --offline --all-targets (lib, main, execution, upgrade_inspect, examples) lib 484 passed, 90 ignored (DB-gated); main 3 passed; execution 783 passed, 1 ignored; upgrade_inspect 2 passed*
PostgreSQL 16 --lib runtime::postgres::tests::qualification_ -- --ignored 37 passed, 0 failed**
PostgreSQL 16 --test execution (DW_EXECUTION_POSTGRES_URL) 783 passed, 0 failed, 1 ignored
MySQL 8.0.46 --lib runtime::mysql:: -- --ignored --skip runtime::mysql::tests::bootstrap_child 47 passed, 0 failed
MySQL 8.0.46 --test execution (DW_EXECUTION_MYSQL_URL, 3 shards like CI's sharding) 783 passed, 0 failed, 1 ignored***

qualification_poll_cache_single_statement_writes, which runs the new qualify_workflow_poll_pair_writes on the real database, passed on PostgreSQL and MySQL. On SQLite, sqlite_poll_cache_writes_use_one_statement_and_rollback runs it.

* upgrade_inspect needs CI's mount profile (DW_UPGRADE_TEST_READONLY as a read-only alias of the writable dir). It failed once because my local setup was wrong, then passed with the correct alias.
** In the first PostgreSQL lib run, 36 passed and qualification_concurrent_bootstrap_and_reopen hit a 55P03 lock timeout under host I/O contention. It passed when run alone, and on a full rerun all 37 passed. Separately, qualification_poll_transaction_counts prints a debug_assert panic ("claimed run history sequence changed before its first page") yet still reports ok. It does the same on unmodified main, so it is not related to this change.
*** Two backup_hold tests start the server binary. My first sharded run didn't mount it, so they failed; with it mounted, both passed.

Local environment notes: the host was I/O-saturated by other jobs. To keep the run time reasonable, the throwaway PostgreSQL ran with fsync=off and synchronous_commit=off, and MySQL ran on tmpfs with innodb_flush_method=nosync, doublewrite off and innodb_flush_log_at_trx_commit=2. These settings only affect server crash durability, which none of these tests exercise. The CI matrix (including MariaDB 10.11, PostgreSQL 17 and MySQL 8.4) still needs to run on this PR.

🤖 Generated with Claude Code

remember_workflow_poll wrote the binding row and then the answer row with
two sequential upserts inside the claim transaction, after
bind_workflow_poll already holds the binding's lock. Both now go in one
QUERY_CACHE_PUT_PAIR statement (ON CONFLICT on PostgreSQL/SQLite, ON
DUPLICATE KEY UPDATE on MySQL/MariaDB; the key is the table's only unique
key), dropping one locked round trip per remembered poll answer.

Keys, JSON values and the two retention rules are unchanged: the binding
keeps result_ttl, a workflow lease's answer keeps 3600 s and any other
answer keeps the binding's ttl. Each expiry is still the clock read just
before the statement that writes the row plus that row's ttl; one
statement now writes both rows, so they share that one reading. The
sequential writer read the answer's clock after the binding's round trip,
so its answer could expire up to one whole second later when that round
trip crossed a second boundary. The pair can only leave the answer at the
earlier of those values, which the sequential writer also produces
whenever no boundary is crossed, and the binding still never outlives the
answer.

The common backend qualification (SQLite, PostgreSQL, MySQL) compares the
pair statement with the two sequential upserts for every existing-row
combination, checks a neighbouring key is untouched, that a rolled-back
transition restores committed rows exactly or leaves none, and that the
caller issues one statement whose two expiries differ by exactly the two
ttls' difference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rmcdaniel

Copy link
Copy Markdown
Member Author

Decision (General, 2026-11-12): accepted. This reverses #1099's deferral of the paired cache write. The only semantic change is that a remembered poll answer can now expire up to one second earlier, when the old code's second clock read crossed a second boundary. An earlier-expiring cache answer only causes a recompute, never a wrong answer, and the binding still never outlives the answer. One fewer locked dw_query_cache statement per remembered poll is worth that. Merge once the full CI matrix (MariaDB 10.11, MySQL 8.4, PostgreSQL 17) is green. #1063 closes with this PR, with the checklist updated.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant