Prepare native 3.8.0 release notes and Compose default

docker-compose.published.yml

@@ -16,7 +16,7 @@ name: durable-workflow-server
 # release whose default names another image). Set DW_SERVER_IMAGE to run
 # any other native image, by tag or digest; pin a digest for production
 # change control.
-x-server-image: &server-image ${DW_SERVER_IMAGE:-durableworkflow/server:3.7.0}
+x-server-image: &server-image ${DW_SERVER_IMAGE:-durableworkflow/server:3.8.0}
 
 services:
   server:

docs/releases/3.8.0.md

@@ -0,0 +1,167 @@
+# Durable Workflow Server 3.8.0 (native)
+
+Server 3.8.0 lets an operator pause activity dispatch on one task queue, lets
+redrive reuse side effects and version markers, and removes database round
+trips from every workflow task. Its exact images are
+`durableworkflow/server:3.8.0` and `ghcr.io/durable-workflow/server:3.8.0`;
+the published Compose default selects 3.8.0. Rolling aliases move separately
+after published-image qualification.
+
+## Upgrade
+
+**Schema upgrade required on every backend.** This release adds one
+native-only table, `dw_task_queue_pauses`, which is created empty:
+
+| Backend    | Schema  |
+|------------|---------|
+| SQLite     | 11 → 12 |
+| PostgreSQL | 12 → 13 |
+| MariaDB    | 12 → 13 |
+| MySQL      | 13 → 14 |
+
+The step only creates the table; it rewrites no rows and takes seconds. A
+server never upgrades its schema at startup. Started on the earlier schema,
+3.8.0 refuses with `native_schema_upgrade_required`.
+
+For the single-node Compose deployment:
+
+1. Stop the Server and leave MySQL running:
+
+   ```bash
+   docker compose --env-file durable-workflow.prod.env \
+     -f docker-compose.published.yml stop server
+   ```
+2. Back up MySQL and `server_storage`, and record the current image
+   reference. This backup is the only way back to 3.7.x.
+3. Change `DW_SERVER_IMAGE` to the 3.8.0 image, pull it and run
+   `schema-upgrade` once:
+
+   ```bash
+   docker compose --env-file durable-workflow.prod.env \
+     -f docker-compose.published.yml pull server
+   docker compose --env-file durable-workflow.prod.env \
+     -f docker-compose.published.yml run --rm --no-deps server schema-upgrade
+   ```
+
+   From 3.7.x on MySQL it reports
+   `"from_schema_version":13,"schema_version":14,"status":"upgraded"`. Run
+   again, it reports `"status":"current"`.
+4. Start the Server with
+   `docker compose --env-file durable-workflow.prod.env -f docker-compose.published.yml up -d --wait`,
+   then confirm `/api/ready`, `/api/cluster/info` (`version` is `3.8.0`) and
+   worker registration.
+
+With several Servers on one database, stop every one of them before step 3
+and start them all on 3.8.0 afterwards. On MySQL and MariaDB, an interrupted
+upgrade resumes when you run `schema-upgrade` again; SQLite and PostgreSQL
+upgrade in one transaction.
+
+**Rolling back to 3.7.x requires restoring the backup.** 3.7.0 refuses the
+upgraded database with `native_schema_newer_than_binary`.
+
+**From earlier releases,** the 3.8.0 image's `schema-upgrade` applies every
+earlier step in the same run. On MySQL from 3.5.x or 3.6.x (schema 12), that
+includes the [3.7.0 JSON rebuild](3.7.0.md#upgrade), with its downtime of
+about 2 minutes per million history rows. From 3.4.x or earlier, it includes
+the [3.5.0 step](3.5.0.md#upgrade).
+
+On Kubernetes, scale the Server to zero, back up, run `helm upgrade` with
+`native.schemaUpgrade.enabled=true` so the 3.8.0 image runs `schema-upgrade`
+once, then scale back up. Chart 1.6.0, which defaults to Server 3.8.0, adds
+its notes to the chart's
+[upgrade notes](../../k8s/helm/durable-workflow/docs/UPGRADING.md).
+
+The worker protocol stays at `1.20` and the control plane at version `2`.
+
+## Pause activity dispatch on a task queue
+
+An operator can hold new activity dispatch on one task queue of one namespace
+and later resume it
+([#577](https://github.com/durable-workflow/server/issues/577)):
+
+- `POST /api/task-queues/{task_queue}/activity-dispatch/pause`, with an
+  optional `reason` of up to 1000 characters, answers `outcome: paused` or
+  `already_paused`.
+- `POST /api/task-queues/{task_queue}/activity-dispatch/resume` answers
+  `outcome: resumed` or `not_paused`.
+- `GET /api/task-queues/{task_queue}/activity-dispatch` reads the state.
+  Task queue describe and list carry the same `activity_dispatch` object, and
+  a paused queue stays listed.
+
+`activity_dispatch` reports `state`, `paused`, `reason`, `paused_at`,
+`paused_by`, `resumed_at` and `resumed_by`. Both principals are the
+authenticated caller, never one named in the request. The routes need the
+operator or admin role, like build-id drain, so a worker token gets 403. They
+work in `default` and in any named namespace, and a credential bound to one
+namespace is refused for the others. Queue names compare exactly, so pausing
+`Orders` does not pause `orders`.
+
+While a queue is paused, no activity task on it is leased. Ready tasks stay
+ready with no attempt used, and after the resume the same task is leased as
+attempt 1. Activity deadlines keep running, so a schedule-to-start or
+schedule-to-close timeout still fires. Activities already running finish
+normally, and a retry they schedule waits for the resume. Workflow, signal,
+query and update delivery is unchanged, and the worker protocol too: a held
+activity poll times out empty. The pause survives restarts and applies to
+every node. A concurrency cap of `0` is not a pause.
+
+Cluster info advertises `capabilities.task_queue_activity_dispatch_pause:
+true`. The control-plane OpenAPI document moves from revision 14 to 15 for
+the two new paths. See
+[Task queue activity dispatch pause](../contracts/task-queue-activity-dispatch-pause.md).
+
+## Redrive reuses side effects and version markers
+
+Redrive no longer refuses a failed run because its history records side
+effects or version markers before the failed step
+([#980](https://github.com/durable-workflow/server/issues/980)). As Workflow
+does, the new run records each of them with the original payload,
+`reused_from_run_id` and `reused_recorded_at`, the time the step first ran.
+Workflow time after those steps matches the failed run, including across a
+chain of redrives.
+
+A run whose workflow code upserted a memo or search attribute now refuses
+with `service_redrive_history_not_qualified` (previously `unsupported_history`)
+until the SDKs can replay reused upserts. A side effect whose result is stored
+externally refuses with `external_side_effect_result_not_supported`. Describe's
+`can_redrive` and `redrive_blocked_reason` report the same decision. Runs with
+fired timers still refuse with `unsupported_history`; later slices of #980
+cover them.
+
+## Fewer database round trips per workflow task
+
+- Each workflow task runs three fewer statements: the sticky-execution record
+  no longer reads the run status and the worker's process metrics again, and
+  the replay clock counts redrives in the same read as the opening events
+  ([#1171](https://github.com/durable-workflow/server/issues/1171)).
+- A workflow poll's binding and answer are written to `dw_query_cache` in one
+  statement instead of two
+  ([#1063](https://github.com/durable-workflow/server/issues/1063)).
+- On MySQL, MariaDB and PostgreSQL, each pool connection now caches up to 200
+  prepared statements instead of SQLx's 100, which holds the hot path of about
+  160 statements. On MySQL, prepares and closes per workflow drop from about
+  25 to 0.2 ([#1174](https://github.com/durable-workflow/server/issues/1174)).
+
+**Check MySQL's prepared-statement budget.** Every client's prepared
+statements count toward the server-wide `max_prepared_stmt_count` (default
+16382). A native node has 4 pool connections, so at the new default it holds
+up to 800, and the cluster needs
+
+```text
+nodes × 4 × capacity + other clients' statements ≤ max_prepared_stmt_count
+```
+
+That is up to 20 native nodes on a default MySQL server. For more, raise
+`max_prepared_stmt_count` or lower the capacity with
+`DW_NATIVE_DB_STATEMENT_CACHE_CAPACITY` (or `statement-cache-capacity` in
+`DB_URL`), a whole number from 0 to 4095. A Server given an invalid value, or
+two different values, refuses to start. `Prepared_stmt_count` in
+`SHOW GLOBAL STATUS` reports current use. SQLite is unchanged. See
+[Prepared-statement cache](../../rust/README.md#prepared-statement-cache).
+
+## Other changes
+
+- Tests now cover registering a namespace's first search attribute while
+  workflows upsert other keys, and deleting or re-registering one while a
+  workflow uses it. Behavior is unchanged from 3.5.0
+  ([#571](https://github.com/durable-workflow/server/issues/571)).