diff --git a/docker-compose.published.yml b/docker-compose.published.yml index 985a66fc9..738229ec0 100644 --- a/docker-compose.published.yml +++ b/docker-compose.published.yml @@ -16,7 +16,7 @@ name: durable-workflow-server # release whose default names another image). Set DW_SERVER_IMAGE to run # any other native image, by tag or digest; pin a digest for production # change control. -x-server-image: &server-image ${DW_SERVER_IMAGE:-durableworkflow/server:3.7.0} +x-server-image: &server-image ${DW_SERVER_IMAGE:-durableworkflow/server:3.8.0} services: server: diff --git a/docs/releases/3.8.0.md b/docs/releases/3.8.0.md new file mode 100644 index 000000000..48217a71f --- /dev/null +++ b/docs/releases/3.8.0.md @@ -0,0 +1,167 @@ +# Durable Workflow Server 3.8.0 (native) + +Server 3.8.0 lets an operator pause activity dispatch on one task queue, lets +redrive reuse side effects and version markers, and removes database round +trips from every workflow task. Its exact images are +`durableworkflow/server:3.8.0` and `ghcr.io/durable-workflow/server:3.8.0`; +the published Compose default selects 3.8.0. Rolling aliases move separately +after published-image qualification. + +## Upgrade + +**Schema upgrade required on every backend.** This release adds one +native-only table, `dw_task_queue_pauses`, which is created empty: + +| Backend | Schema | +|------------|---------| +| SQLite | 11 → 12 | +| PostgreSQL | 12 → 13 | +| MariaDB | 12 → 13 | +| MySQL | 13 → 14 | + +The step only creates the table; it rewrites no rows and takes seconds. A +server never upgrades its schema at startup. Started on the earlier schema, +3.8.0 refuses with `native_schema_upgrade_required`. + +For the single-node Compose deployment: + +1. Stop the Server and leave MySQL running: + + ```bash + docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml stop server + ``` +2. Back up MySQL and `server_storage`, and record the current image + reference. This backup is the only way back to 3.7.x. +3. Change `DW_SERVER_IMAGE` to the 3.8.0 image, pull it and run + `schema-upgrade` once: + + ```bash + docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml pull server + docker compose --env-file durable-workflow.prod.env \ + -f docker-compose.published.yml run --rm --no-deps server schema-upgrade + ``` + + From 3.7.x on MySQL it reports + `"from_schema_version":13,"schema_version":14,"status":"upgraded"`. Run + again, it reports `"status":"current"`. +4. Start the Server with + `docker compose --env-file durable-workflow.prod.env -f docker-compose.published.yml up -d --wait`, + then confirm `/api/ready`, `/api/cluster/info` (`version` is `3.8.0`) and + worker registration. + +With several Servers on one database, stop every one of them before step 3 +and start them all on 3.8.0 afterwards. On MySQL and MariaDB, an interrupted +upgrade resumes when you run `schema-upgrade` again; SQLite and PostgreSQL +upgrade in one transaction. + +**Rolling back to 3.7.x requires restoring the backup.** 3.7.0 refuses the +upgraded database with `native_schema_newer_than_binary`. + +**From earlier releases,** the 3.8.0 image's `schema-upgrade` applies every +earlier step in the same run. On MySQL from 3.5.x or 3.6.x (schema 12), that +includes the [3.7.0 JSON rebuild](3.7.0.md#upgrade), with its downtime of +about 2 minutes per million history rows. From 3.4.x or earlier, it includes +the [3.5.0 step](3.5.0.md#upgrade). + +On Kubernetes, scale the Server to zero, back up, run `helm upgrade` with +`native.schemaUpgrade.enabled=true` so the 3.8.0 image runs `schema-upgrade` +once, then scale back up. Chart 1.6.0, which defaults to Server 3.8.0, adds +its notes to the chart's +[upgrade notes](../../k8s/helm/durable-workflow/docs/UPGRADING.md). + +The worker protocol stays at `1.20` and the control plane at version `2`. + +## Pause activity dispatch on a task queue + +An operator can hold new activity dispatch on one task queue of one namespace +and later resume it +([#577](https://github.com/durable-workflow/server/issues/577)): + +- `POST /api/task-queues/{task_queue}/activity-dispatch/pause`, with an + optional `reason` of up to 1000 characters, answers `outcome: paused` or + `already_paused`. +- `POST /api/task-queues/{task_queue}/activity-dispatch/resume` answers + `outcome: resumed` or `not_paused`. +- `GET /api/task-queues/{task_queue}/activity-dispatch` reads the state. + Task queue describe and list carry the same `activity_dispatch` object, and + a paused queue stays listed. + +`activity_dispatch` reports `state`, `paused`, `reason`, `paused_at`, +`paused_by`, `resumed_at` and `resumed_by`. Both principals are the +authenticated caller, never one named in the request. The routes need the +operator or admin role, like build-id drain, so a worker token gets 403. They +work in `default` and in any named namespace, and a credential bound to one +namespace is refused for the others. Queue names compare exactly, so pausing +`Orders` does not pause `orders`. + +While a queue is paused, no activity task on it is leased. Ready tasks stay +ready with no attempt used, and after the resume the same task is leased as +attempt 1. Activity deadlines keep running, so a schedule-to-start or +schedule-to-close timeout still fires. Activities already running finish +normally, and a retry they schedule waits for the resume. Workflow, signal, +query and update delivery is unchanged, and the worker protocol too: a held +activity poll times out empty. The pause survives restarts and applies to +every node. A concurrency cap of `0` is not a pause. + +Cluster info advertises `capabilities.task_queue_activity_dispatch_pause: +true`. The control-plane OpenAPI document moves from revision 14 to 15 for +the two new paths. See +[Task queue activity dispatch pause](../contracts/task-queue-activity-dispatch-pause.md). + +## Redrive reuses side effects and version markers + +Redrive no longer refuses a failed run because its history records side +effects or version markers before the failed step +([#980](https://github.com/durable-workflow/server/issues/980)). As Workflow +does, the new run records each of them with the original payload, +`reused_from_run_id` and `reused_recorded_at`, the time the step first ran. +Workflow time after those steps matches the failed run, including across a +chain of redrives. + +A run whose workflow code upserted a memo or search attribute now refuses +with `service_redrive_history_not_qualified` (previously `unsupported_history`) +until the SDKs can replay reused upserts. A side effect whose result is stored +externally refuses with `external_side_effect_result_not_supported`. Describe's +`can_redrive` and `redrive_blocked_reason` report the same decision. Runs with +fired timers still refuse with `unsupported_history`; later slices of #980 +cover them. + +## Fewer database round trips per workflow task + +- Each workflow task runs three fewer statements: the sticky-execution record + no longer reads the run status and the worker's process metrics again, and + the replay clock counts redrives in the same read as the opening events + ([#1171](https://github.com/durable-workflow/server/issues/1171)). +- A workflow poll's binding and answer are written to `dw_query_cache` in one + statement instead of two + ([#1063](https://github.com/durable-workflow/server/issues/1063)). +- On MySQL, MariaDB and PostgreSQL, each pool connection now caches up to 200 + prepared statements instead of SQLx's 100, which holds the hot path of about + 160 statements. On MySQL, prepares and closes per workflow drop from about + 25 to 0.2 ([#1174](https://github.com/durable-workflow/server/issues/1174)). + +**Check MySQL's prepared-statement budget.** Every client's prepared +statements count toward the server-wide `max_prepared_stmt_count` (default +16382). A native node has 4 pool connections, so at the new default it holds +up to 800, and the cluster needs + +```text +nodes × 4 × capacity + other clients' statements ≤ max_prepared_stmt_count +``` + +That is up to 20 native nodes on a default MySQL server. For more, raise +`max_prepared_stmt_count` or lower the capacity with +`DW_NATIVE_DB_STATEMENT_CACHE_CAPACITY` (or `statement-cache-capacity` in +`DB_URL`), a whole number from 0 to 4095. A Server given an invalid value, or +two different values, refuses to start. `Prepared_stmt_count` in +`SHOW GLOBAL STATUS` reports current use. SQLite is unchanged. See +[Prepared-statement cache](../../rust/README.md#prepared-statement-cache). + +## Other changes + +- Tests now cover registering a namespace's first search attribute while + workflows upsert other keys, and deleting or re-registering one while a + workflow uses it. Behavior is unchanged from 3.5.0 + ([#571](https://github.com/durable-workflow/server/issues/571)).