diff --git a/docs/workflow-version-usage.md b/docs/workflow-version-usage.md new file mode 100644 index 000000000..ff4fa00c2 --- /dev/null +++ b/docs/workflow-version-usage.md @@ -0,0 +1,48 @@ +# Open-run version usage + +`GET /api/workflow-version-usage` reports which open runs retain an older +version-marker decision for one workflow type and change point. Discover support +by requiring `capabilities.workflow_version_usage` to be exactly `true` on +`GET /api/cluster/info`. A missing capability or unsuccessful request is not an +empty cohort. Published Server versions without this capability do not serve +this contract. + +Use an operator or admin token, the existing control-plane version header, and +the selected namespace. Required query parameters are exact `workflow_type` +(1–255 UTF-8 bytes), exact `change_id` (1–1024 bytes), and signed 32-bit +`min_version`, the proposed minimum supported version. Optional `state` is +`legacy`, `current`, or `unknown`; `limit` is 1–200 (default 50). Continue with +only the returned opaque `cursor` and the same identity/classification filters. +Invalid or mismatched cursors return HTTP 422, rather than restarting a page. +Native namespace echo is always the selected string, including `default`. + +A valid retained marker below `min_version` is `legacy`; a value at or above it +is `current`. Missing markers remain `unknown/marker_absent`, even for runs with +a current code fingerprint. Pruned, incomplete or unsupported history remains +`unknown/history_unavailable`. Malformed ranges, integer fields or conflicting +decisions remain `unknown/history_invalid`. More than 1,000 inspected marker +events in one run also makes its history unavailable. The query reads no +application arguments, results, memo or external payload objects. + +Reused markers must carry both `reused_from_run_id` and `reused_recorded_at`, +or neither. The source ID must be nonempty, exact and at most 255 UTF-8 bytes, +without surrounding whitespace or control characters. The clock must be a valid +explicit-offset RFC3339 timestamp (`Z` or `+/-HH:MM`), with optional fractional +seconds of at most nine digits. Malformed pairs remain `history_invalid`, even +on a different change ID. Reading these fields never rewrites retained strings +or establishes SDK replay-clock qualification. + +`counts` includes the entire bounded open-run cohort before the optional state +filter. `runs` is ordered by ascending run ID. The first page fixes the greatest +participating ID as `through_run_id`; continuations retain that upper bound and +advance strictly after the last returned ID. Each response's counts and records +share one read-only database snapshot. Between pages, closures and marker +recording can change counts and membership. A newly matching state behind a +cursor requires a fresh traversal. This live view is not a deployment fence and +never authorizes branch removal; unknowns must stay visible while admissions +are drained or fenced through the operator's rollout process. + +The normative fields are in the [shared v1 contract](https://github.com/durable-workflow/workflow/blob/30341b139323c4bf3fa26a34c2c562f88c075bbe/docs/version-usage.md) +and the [control-plane OpenAPI](../resources/platform-protocol-specs/control-plane-api.openapi.yaml). +The endpoint changes no marker producer, history, workflow definition, schema +or replay behavior. diff --git a/resources/platform-protocol-specs/control-plane-api.openapi.yaml b/resources/platform-protocol-specs/control-plane-api.openapi.yaml index e6bf02c02..36cbc9752 100644 --- a/resources/platform-protocol-specs/control-plane-api.openapi.yaml +++ b/resources/platform-protocol-specs/control-plane-api.openapi.yaml @@ -1,7 +1,7 @@ openapi: 3.1.0 info: title: durable-workflow.v2.control-plane-api - version: "15" + version: "16" summary: Durable Workflow control-plane HTTP+JSON API description: > Normative OpenAPI specification for the standalone Durable Workflow @@ -201,6 +201,98 @@ paths: "201": { $ref: "#/components/responses/ControlPlaneEnvelope" } "409": { $ref: "#/components/responses/ControlPlaneError" } "422": { $ref: "#/components/responses/ControlPlaneError" } + /workflow-version-usage: + get: + tags: [workflows] + operationId: getWorkflowVersionUsage + summary: Count and page open runs by retained version-marker decision + description: > + Requires capabilities.workflow_version_usage=true and the authenticated + namespace context. Reads open pending/running/waiting runs of the exact + workflow type and change ID. No marker, unavailable history and invalid + decisions remain unknown; unknown never means zero legacy usage or safe + removal. Counts include all states before the optional state filter. + Each response uses one read-only storage snapshot. Pagination is a live + ascending run-ID keyset, fixed to the first page's through_run_id, not a + durable fleet snapshot or deployment fence. A marker inspection budget + of 1000 per run yields history_unavailable when exceeded. + Reused marker provenance must be a complete source-ID and explicit-offset + RFC3339 clock pair, otherwise history_invalid. Native clocks retain up to + nine fractional digits and retained strings are never rewritten. Arguments, + results and external payloads are not read. The shared v1 contract is + https://github.com/durable-workflow/workflow/blob/30341b139323c4bf3fa26a34c2c562f88c075bbe/docs/version-usage.md. + parameters: + - $ref: "#/components/parameters/ControlPlaneVersionHeader" + - in: query + name: workflow_type + required: true + description: Exact nonempty workflow type, at most 255 UTF-8 bytes. + schema: { type: string, minLength: 1 } + - in: query + name: change_id + required: true + description: Exact nonempty change-point ID, at most 1024 UTF-8 bytes. + schema: { type: string, minLength: 1 } + - in: query + name: min_version + required: true + schema: { type: integer, minimum: -2147483648, maximum: 2147483647 } + - in: query + name: state + schema: { type: string, enum: [legacy, current, unknown] } + - in: query + name: limit + schema: { type: integer, minimum: 1, maximum: 200, default: 50 } + - in: query + name: cursor + description: Opaque namespace/type/change/minimum/state-bound continuation. Invalid or mismatched cursors are errors. + schema: { type: string } + responses: + "200": + description: Open-run version usage v1; no safe-to-remove flag. + content: + application/json: + schema: + type: object + additionalProperties: false + required: [schema_version, namespace, workflow_type, change_id, min_version, state, consistency, through_run_id, counts, runs, limit, returned_count, has_more, next_cursor] + properties: + schema_version: { const: 1 } + namespace: { type: string } + workflow_type: { type: string } + change_id: { type: string } + min_version: { type: integer, minimum: -2147483648, maximum: 2147483647 } + state: { enum: [null, legacy, current, unknown] } + consistency: { const: live } + through_run_id: { type: [string, "null"] } + counts: + type: object + additionalProperties: false + required: [total, legacy, current, unknown] + properties: + total: { type: integer, minimum: 0 } + legacy: { type: integer, minimum: 0 } + current: { type: integer, minimum: 0 } + unknown: { type: integer, minimum: 0 } + runs: + type: array + maxItems: 200 + items: + type: object + additionalProperties: false + required: [instance_id, run_id, status, state, recorded_version, unknown_reason] + properties: + instance_id: { type: string } + run_id: { type: string } + status: { enum: [pending, running, waiting] } + state: { enum: [legacy, current, unknown] } + recorded_version: { type: [integer, "null"], minimum: -2147483648, maximum: 2147483647 } + unknown_reason: { enum: [null, marker_absent, history_unavailable, history_invalid] } + limit: { type: integer, minimum: 1, maximum: 200 } + returned_count: { type: integer, minimum: 0, maximum: 200 } + has_more: { type: boolean } + next_cursor: { type: [string, "null"] } + "422": { $ref: "#/components/responses/ControlPlaneError" } /workflows/import/embedded-v2: post: tags: [workflows] diff --git a/rust/src/runtime/cluster_info.rs b/rust/src/runtime/cluster_info.rs index c4c778001..db8282071 100644 --- a/rust/src/runtime/cluster_info.rs +++ b/rust/src/runtime/cluster_info.rs @@ -54,6 +54,7 @@ pub(super) fn capabilities() -> Value { "schedule_jitter", "schedule_max_runs", "search_attributes", + super::version_usage::CAPABILITY, "history_export", "continue_as_new", "child_workflows", @@ -217,8 +218,9 @@ mod tests { } // PHP's names plus native's child turn, occurrence-id reopen, // occurrence deadline, update outcome reads, task queue pauses and - // command request keys. - assert_eq!(caps.as_object().unwrap().len(), 62); + // command request keys and open-run version usage. + assert_eq!(caps.as_object().unwrap().len(), 63); + assert_eq!(caps[super::super::version_usage::CAPABILITY], json!(true)); assert_eq!(caps["task_queue_activity_dispatch_pause"], json!(true)); assert_eq!(caps["command_request_keys"], json!(true)); assert_eq!(caps["update_outcome_reads"], json!(true)); diff --git a/rust/src/runtime/control_plane.rs b/rust/src/runtime/control_plane.rs index e0b512860..bd120be0d 100644 --- a/rust/src/runtime/control_plane.rs +++ b/rust/src/runtime/control_plane.rs @@ -6,6 +6,7 @@ use std::collections::HashMap; pub(super) enum Operation<'a> { Start, List, + VersionUsage, ListRuns, Describe, DescribeRun, @@ -99,6 +100,7 @@ impl Operation<'_> { let (operation, required, success): (&str, &[&str], &[&str]) = match self { Self::Start => ("start", &[], &["workflow_id", "outcome"]), Self::List => ("list", &[], &[]), + Self::VersionUsage => ("version_usage", &[], &[]), Self::ListRuns => ("list_runs", &["workflow_id"], &[]), Self::Describe => ("describe", &["workflow_id"], &[]), Self::DescribeRun => ("describe_run", &["workflow_id"], &["run_id"]), @@ -321,7 +323,7 @@ impl Operation<'_> { selected_run_not_current configured_workflow_type_invalid v1_projection_read_only \ backend_lock_pressure control_plane_internal_error", ), - Self::List | Self::ListRuns | Self::Describe | Self::Observe | Self::Debug | Self::ImportEmbeddedV2 => return None, + Self::List | Self::VersionUsage | Self::ListRuns | Self::Describe | Self::Observe | Self::Debug | Self::ImportEmbeddedV2 => return None, }) } } @@ -345,6 +347,7 @@ pub(super) async fn admission_response(request: Request, mut body: Value) -> Val ("POST", ["api", "workflows", _, "query", name]) | ("POST", ["api", "workflows", _, "runs", _, "query", name]) => Operation::Query(name), ("GET", ["api", "workflows"]) => Operation::List, + ("GET", ["api", "workflow-version-usage"]) => Operation::VersionUsage, ("GET", ["api", "workflows", _]) => Operation::Describe, ("GET", ["api", "workflows", _, "runs"]) => Operation::ListRuns, ("GET", ["api", "workflows", _, "runs", _]) => Operation::DescribeRun, diff --git a/rust/src/runtime/execution.rs b/rust/src/runtime/execution.rs index 2b4c2e646..2aa4c8b88 100644 --- a/rust/src/runtime/execution.rs +++ b/rust/src/runtime/execution.rs @@ -1122,6 +1122,13 @@ impl Runtime { Storage::MySql(store) => store.list_workflows(&filter).await, } } + pub(super) async fn version_usage(&self, filter: super::version_usage::Filter) -> Result { + match &*self.storage { + Storage::Sqlite(store) => store.version_usage(&filter, &self.token).await, + Storage::Postgres(store) => store.version_usage(&filter, &self.token).await, + Storage::MySql(store) => store.version_usage(&filter, &self.token).await, + } + } pub(crate) async fn update_workflow( &self, workflow_id: &str, diff --git a/rust/src/runtime/http.rs b/rust/src/runtime/http.rs index 8dd4d9c59..e12e23147 100644 --- a/rust/src/runtime/http.rs +++ b/rust/src/runtime/http.rs @@ -69,6 +69,7 @@ pub fn router(runtime: Runtime) -> Router { delete(delete_search_attribute), ) .route("/api/workflows", get(list_workflows).post(start)) + .route("/api/workflow-version-usage", get(version_usage)) .route("/api/bridge-adapters/webhook/{adapter}", post(bridge_webhook)) .route("/api/workflows/import/embedded-v2", post(import_embedded_v2)) .route("/api/workflows/import/waterline-v1", post(import_waterline_v1)) @@ -932,6 +933,7 @@ fn namespace_route(path: &str) -> Option { fn exact_namespace_route(path: &str) -> Option { Some(match path { "/api/runtime-credentials" | "/api/schedules" => Allowed::Any, + "/api/workflow-version-usage" => READ, "/api/workflows" | "/api/activities" | "/api/search-attributes" => READ_WRITE, // The metrics describe the namespace's own state, read by GET or POST. "/api/system/metrics" => READ_WRITE, @@ -1467,6 +1469,14 @@ async fn list_workflows( Ok(Json(runtime.list_workflows(query.validate()?).await?)) } +async fn version_usage( + Extension(runtime): Extension, + query: std::result::Result, axum::extract::rejection::QueryRejection>, +) -> Result> { + let Query(query) = query.map_err(|_| super::version_usage::invalid("query", "The version-usage query parameters are invalid."))?; + Ok(Json(runtime.version_usage(query.validate(&runtime.token)?).await?)) +} + async fn describe_current( Extension(runtime): Extension, Path(workflow_id): Path, diff --git a/rust/src/runtime/mod.rs b/rust/src/runtime/mod.rs index f016ce89a..0261a533a 100644 --- a/rust/src/runtime/mod.rs +++ b/rust/src/runtime/mod.rs @@ -207,6 +207,7 @@ mod update_validation; mod updates; pub mod upgrade; mod visibility; +mod version_usage; mod visibility_query; mod wait_history; mod wait_projection; diff --git a/rust/src/runtime/version_usage.rs b/rust/src/runtime/version_usage.rs new file mode 100644 index 000000000..9945dde9e --- /dev/null +++ b/rust/src/runtime/version_usage.rs @@ -0,0 +1,671 @@ +//! Read-only version-marker usage. Missing evidence cannot establish that a +//! legacy branch is unused. +use super::{Result, RuntimeError}; +use axum::http::StatusCode; +use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD}; +use futures_util::TryStreamExt; +use hmac::{Hmac, KeyInit, Mac}; +use serde::{Deserialize, Serialize}; +use serde_json::{Value, json}; +use sha2::Sha256; +use std::collections::BTreeSet; + +const MARKER_BUDGET: usize = 1000; +pub(super) const CAPABILITY: &str = "workflow_version_usage"; + +fn valid_reuse_clock(value: &str) -> bool { + if !value.is_ascii() || !(20..=35).contains(&value.len()) || value.as_bytes()[10] != b'T' { + return false; + } + let zone = if value.ends_with('Z') { value.len()-1 } else { + let at = value.len()-6; + if !matches!(value.as_bytes()[at], b'+' | b'-') || value.as_bytes()[at+3] != b':' { return false; } + at + }; + if zone < 19 { return false; } + let fraction = &value[19..zone]; + let precision = fraction.is_empty() || fraction.strip_prefix('.').is_some_and(|digits| + (1..=9).contains(&digits.len()) && digits.bytes().all(|byte| byte.is_ascii_digit())); + precision && chrono::DateTime::parse_from_rfc3339(value).is_ok() +} + +fn valid_reuse_provenance(payload: &Value) -> bool { + match (payload.get("reused_from_run_id"), payload.get("reused_recorded_at")) { + (None, None) => true, + (Some(Value::String(source)), Some(Value::String(clock))) => + !source.is_empty() && source.len() <= 255 && source.trim() == source.as_str() + && !source.chars().any(char::is_control) && valid_reuse_clock(clock), + _ => false, + } +} + +#[derive(Default, Deserialize)] +pub(super) struct Query { + workflow_type: Option, + change_id: Option, + min_version: Option, + state: Option, + limit: Option, + cursor: Option, +} + +pub(super) struct Filter { + workflow_type: String, + change_id: String, + min_version: i32, + state: Option, + limit: usize, + cursor: Option, +} + +#[derive(Deserialize, Serialize)] +#[serde(deny_unknown_fields)] +struct Cursor { + version: u8, + namespace: String, + workflow_type: String, + change_id: String, + min_version: i32, + state: Option, + after: String, + through: String, +} + +impl Cursor { + fn mac(key: &str, body: &[u8]) -> Hmac { + let mut mac = Hmac::::new_from_slice(key.as_bytes()).expect("HMAC accepts any key length"); + mac.update(b"durable-workflow/version-usage/v1\0"); + mac.update(body); + mac + } + + fn encode(&self, key: &str) -> Result { + let body = serde_json::to_vec(self)?; + Ok(format!("{}.{}", URL_SAFE_NO_PAD.encode(&body), + URL_SAFE_NO_PAD.encode(Self::mac(key, &body).finalize().into_bytes()))) + } + + fn decode(token: &str, key: &str) -> Result { + let bad = || invalid("cursor", "The version-usage cursor is invalid."); + // Legal UTF-8 query values can expand sixfold in JSON escaping. + if token.len() > 16384 { return Err(bad()); } + let (body, tag) = token.split_once('.').ok_or_else(bad)?; + let body = URL_SAFE_NO_PAD.decode(body).map_err(|_| bad())?; + let tag = URL_SAFE_NO_PAD.decode(tag).map_err(|_| bad())?; + Self::mac(key, &body).verify_slice(&tag).map_err(|_| bad())?; + serde_json::from_slice(&body).map_err(|_| bad()) + } +} + +pub(super) fn invalid(field: &str, message: &str) -> RuntimeError { + RuntimeError::Protocol { + status: StatusCode::UNPROCESSABLE_ENTITY, + response: super::control_plane::Operation::VersionUsage.response(json!({"reason":"validation_failed", "message":message, + "errors":{(field):[message]}, "validation_errors":{(field):[message]}})), + } +} + +impl Query { + pub(super) fn validate(self, key: &str) -> Result { + let required = |value: Option, key, maximum| { + value.filter(|value| !value.trim().is_empty() && value.len() <= maximum) + .ok_or_else(|| invalid(key, "A nonempty string within the supported byte limit is required.")) + }; + let workflow_type = required(self.workflow_type, "workflow_type", 255)?; + let change_id = required(self.change_id, "change_id", 1024)?; + let min_version = self.min_version.as_deref().and_then(|value| value.parse::().ok()) + .ok_or_else(|| invalid("min_version", "A signed 32-bit integer is required."))?; + if self.state.as_deref().is_some_and(|state| !matches!(state, "legacy" | "current" | "unknown")) { + return Err(invalid("state", "State must be legacy, current, or unknown.")); + } + let limit = match self.limit { + Some(value) => value.parse::().ok().filter(|value| (1..=200).contains(value)) + .ok_or_else(|| invalid("limit", "Limit must be an integer between 1 and 200."))?, + None => 50, + }; + let cursor = self.cursor.map(|token| { + let cursor = Cursor::decode(&token, key)?; + if cursor.version != 1 || cursor.workflow_type != workflow_type || cursor.change_id != change_id + || cursor.min_version != min_version || cursor.state != self.state + || cursor.after.is_empty() || cursor.through.is_empty() || cursor.after > cursor.through + || cursor.after.len() > 255 || cursor.through.len() > 255 + { + return Err(invalid("cursor", "The cursor does not match this version-usage query.")); + } + Ok(cursor) + }).transpose()?; + Ok(Filter { workflow_type, change_id, min_version, state: self.state, limit, cursor }) + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(super) enum Usage { + Legacy(i32), + Current(i32), + Unknown(Unknown), +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(super) enum Unknown { + HistoryUnavailable, + MarkerMissing, + MarkerInvalid, + ConflictingMarkers, +} + +/// Accumulate only the requested change point, without retaining run history. +/// A second marker cannot silently replace evidence for the same change ID. +pub(super) struct Evidence<'a> { + change_id: &'a str, + history_available: bool, + started: bool, + version: Option, + logical_sequences: BTreeSet, + invalid: bool, + conflicting: bool, + markers: usize, + sequence: i64, +} + +impl<'a> Evidence<'a> { + pub(super) fn new(change_id: &'a str, history_available: bool) -> Self { + Self { + change_id, + history_available, + started: false, + version: None, + logical_sequences: BTreeSet::new(), + invalid: false, + conflicting: false, + markers: 0, + sequence: 0, + } + } + + pub(super) fn record(&mut self, event_type: &str, payload: &Value) { + match event_type { + "WorkflowStarted" => self.started = true, + "VersionMarkerRecorded" => { + self.markers = self.markers.saturating_add(1); + if self.markers > MARKER_BUDGET { + self.history_available = false; + return; + } + let Some(change_id) = payload.get("change_id").and_then(Value::as_str).filter(|value| !value.is_empty()) else { + // Its identity is unavailable; it might be the requested marker. + self.invalid = true; + return; + }; + let integer = |key| payload.get(key).and_then(Value::as_i64).and_then(|value| i32::try_from(value).ok()); + let (Some(version), Some(minimum), Some(maximum)) = (integer("version"), integer("min_supported"), integer("max_supported")) else { + self.invalid = true; + return; + }; + let logical = payload.get("sequence").and_then(Value::as_i64); + if minimum > version || version > maximum + || !logical.is_some_and(|sequence| sequence > 0 && !self.logical_sequences.contains(&sequence)) + || !valid_reuse_provenance(payload) + { + self.invalid = true; + return; + } + // Native retains a task snapshot in addition to the portable + // decision fields. Unknown extensions need a reader upgrade. + if payload.as_object().expect("a decision object").keys().any(|key| !matches!(key.as_str(), + "change_id" | "version" | "min_supported" | "max_supported" | "sequence" + | "task" | "reused_from_run_id" | "reused_recorded_at")) + || payload.get("task").is_some_and(|task| !task.is_object()) + { + self.history_available = false; + return; + } + self.logical_sequences.insert(logical.expect("validated logical sequence")); + if change_id != self.change_id { + return; + } + if self.version.is_some_and(|previous| previous != version) { + self.conflicting = true; + } + self.version = Some(version); + } + _ => {} + } + } + + fn event(&mut self, sequence: i64, event_type: &str, payload: Option<&str>) { + // Ordinary native starts/redrives retain StartAccepted before + // WorkflowStarted; child/continued runs start with WorkflowStarted. + let prefix_valid = match (self.sequence, self.started, event_type) { + (0, false, "StartAccepted" | "WorkflowStarted") => true, + (1, false, "WorkflowStarted") => true, + (_, true, "StartAccepted" | "WorkflowStarted") => false, + (_, true, _) => true, + _ => false, + }; + if self.sequence.checked_add(1) != Some(sequence) || !prefix_valid + || !super::workflow_import::HISTORY_EVENT_TYPES.contains(&event_type) + { + self.history_available = false; + } + self.sequence = sequence; + if event_type == "VersionMarkerRecorded" { + if self.markers >= MARKER_BUDGET { + self.history_available = false; + return; + } + match payload.and_then(|payload| serde_json::from_str(payload).ok()) { + Some(payload) => self.record(event_type, &payload), + None => { self.markers += 1; self.invalid = true; }, + } + } else if event_type == "WorkflowStarted" { + self.started = true; + } + } + + pub(super) fn classify(&self, minimum_version: i32) -> Usage { + if !self.history_available || !self.started { + Usage::Unknown(Unknown::HistoryUnavailable) + } else if self.invalid { + Usage::Unknown(Unknown::MarkerInvalid) + } else if self.conflicting { + Usage::Unknown(Unknown::ConflictingMarkers) + } else { + match self.version { + Some(version) if version < minimum_version => Usage::Legacy(version), + Some(version) => Usage::Current(version), + None => Usage::Unknown(Unknown::MarkerMissing), + } + } + } +} + +impl Usage { + fn fields(self) -> (&'static str, Option, Option<&'static str>) { + match self { + Self::Legacy(version) => ("legacy", Some(version), None), + Self::Current(version) => ("current", Some(version), None), + Self::Unknown(reason) => ("unknown", None, Some(match reason { + Unknown::HistoryUnavailable => "history_unavailable", + Unknown::MarkerMissing => "marker_absent", + Unknown::MarkerInvalid | Unknown::ConflictingMarkers => "history_invalid", + })), + } + } +} + +struct Run<'a> { + id: String, + instance: String, + status: String, + last_sequence: i64, + evidence: Evidence<'a>, +} + +#[derive(Default, Serialize)] +struct Counts { total: u64, legacy: u64, current: u64, unknown: u64 } + +fn finish(run: Run<'_>, filter: &Filter, counts: &mut Counts, page: &mut Vec) { + let usage = if run.evidence.sequence != run.last_sequence { + Usage::Unknown(Unknown::HistoryUnavailable) + } else { + run.evidence.classify(filter.min_version) + }; + let (state, version, reason) = usage.fields(); + counts.total += 1; + match state { + "legacy" => counts.legacy += 1, + "current" => counts.current += 1, + _ => counts.unknown += 1, + } + if filter.cursor.as_ref().is_some_and(|cursor| run.id <= cursor.after) + || filter.state.as_deref().is_some_and(|selected| selected != state) + || page.len() > filter.limit + { + return; + } + page.push(json!({"instance_id":run.instance,"run_id":run.id,"status":run.status, + "state":state,"recorded_version":version,"unknown_reason":reason})); +} + +// Fixed SQL templates keep all caller values in binds. Explicit binary/C +// comparisons make namespace, type, and keyset identity independent of collation. +const SQLITE_ROWS: &str = "SELECT r.id,r.workflow_instance_id,r.status,r.details_pruned_at,r.import_source,r.last_history_sequence,i.id AS owner_id,h.sequence,h.event_type,CASE WHEN h.event_type='VersionMarkerRecorded' THEN h.payload ELSE NULL END AS marker,CASE WHEN h.event_type='WorkflowStarted' THEN CASE WHEN json_valid(h.payload) THEN CASE WHEN json_type(h.payload)='object' THEN 1 ELSE 0 END ELSE 0 END ELSE 1 END AS start_format_valid FROM workflow_runs r LEFT JOIN workflow_instances i ON i.id COLLATE BINARY=r.workflow_instance_id COLLATE BINARY AND i.namespace COLLATE BINARY=$1 LEFT JOIN workflow_history_events h ON h.workflow_run_id COLLATE BINARY=r.id COLLATE BINARY WHERE r.namespace COLLATE BINARY=$2 AND r.workflow_type COLLATE BINARY=$3 AND r.status COLLATE BINARY IN ('pending','running','waiting') AND ($4 IS NULL OR r.id COLLATE BINARY<=$5) ORDER BY r.id COLLATE BINARY,h.sequence"; +const POSTGRES_ROWS: &str = "SELECT r.id,r.workflow_instance_id,r.status,r.details_pruned_at,r.import_source,r.last_history_sequence,i.id AS owner_id,h.sequence,h.event_type,CASE WHEN h.event_type='VersionMarkerRecorded' THEN h.payload::text ELSE NULL END AS marker,CASE WHEN h.event_type='WorkflowStarted' THEN CASE WHEN jsonb_typeof(h.payload::jsonb)='object' THEN 1 ELSE 0 END ELSE 1 END AS start_format_valid FROM workflow_runs r LEFT JOIN workflow_instances i ON i.id COLLATE \"C\"=r.workflow_instance_id COLLATE \"C\" AND i.namespace COLLATE \"C\"=$1 LEFT JOIN workflow_history_events h ON h.workflow_run_id COLLATE \"C\"=r.id COLLATE \"C\" WHERE r.namespace COLLATE \"C\"=$2 AND r.workflow_type COLLATE \"C\"=$3 AND r.status COLLATE \"C\" IN ('pending','running','waiting') AND ($4 IS NULL OR r.id COLLATE \"C\"<=$5 COLLATE \"C\") ORDER BY r.id COLLATE \"C\",h.sequence"; +const MYSQL_ROWS: &str = "SELECT r.id,r.workflow_instance_id,r.status,r.details_pruned_at,r.import_source,r.last_history_sequence,i.id AS owner_id,h.sequence,h.event_type,CASE WHEN h.event_type='VersionMarkerRecorded' THEN CAST(h.payload AS CHAR) ELSE NULL END AS marker,CASE WHEN h.event_type='WorkflowStarted' THEN CASE WHEN JSON_TYPE(h.payload)='OBJECT' THEN 1 ELSE 0 END ELSE 1 END AS start_format_valid FROM workflow_runs r LEFT JOIN workflow_instances i ON BINARY i.id=BINARY r.workflow_instance_id AND BINARY i.namespace=BINARY $1 LEFT JOIN workflow_history_events h ON BINARY h.workflow_run_id=BINARY r.id WHERE BINARY r.namespace=BINARY $2 AND BINARY r.workflow_type=BINARY $3 AND BINARY r.status IN ('pending','running','waiting') AND ($4 IS NULL OR BINARY r.id<=BINARY $5) ORDER BY BINARY r.id,h.sequence"; +const SQLITE_UPPER: &str = "SELECT r.id FROM workflow_runs r LEFT JOIN workflow_instances i ON i.id COLLATE BINARY=r.workflow_instance_id COLLATE BINARY AND i.namespace COLLATE BINARY=$1 WHERE r.namespace COLLATE BINARY=$2 AND r.workflow_type COLLATE BINARY=$3 AND r.status COLLATE BINARY IN ('pending','running','waiting') ORDER BY r.id COLLATE BINARY DESC LIMIT 1"; +const POSTGRES_UPPER: &str = "SELECT r.id FROM workflow_runs r LEFT JOIN workflow_instances i ON i.id COLLATE \"C\"=r.workflow_instance_id COLLATE \"C\" AND i.namespace COLLATE \"C\"=$1 WHERE r.namespace COLLATE \"C\"=$2 AND r.workflow_type COLLATE \"C\"=$3 AND r.status COLLATE \"C\" IN ('pending','running','waiting') ORDER BY r.id COLLATE \"C\" DESC LIMIT 1"; +const MYSQL_UPPER: &str = "SELECT r.id FROM workflow_runs r LEFT JOIN workflow_instances i ON BINARY i.id=BINARY r.workflow_instance_id AND BINARY i.namespace=BINARY $1 WHERE BINARY r.namespace=BINARY $2 AND BINARY r.workflow_type=BINARY $3 AND BINARY r.status IN ('pending','running','waiting') ORDER BY BINARY r.id DESC LIMIT 1"; + +store_methods! { + pub(super) async fn version_usage(&self, filter: &Filter, key: &str) -> Result { + if filter.cursor.as_ref().is_some_and(|cursor| cursor.namespace != self.namespace.as_ref()) { + return Err(invalid("cursor", "The cursor does not match the authenticated namespace.")); + } + let (upper_sql, rows_sql) = match DB::PHP_CONNECTION { + "mysql" => (MYSQL_UPPER, MYSQL_ROWS), + "pgsql" => (POSTGRES_UPPER, POSTGRES_ROWS), + _ => (SQLITE_UPPER, SQLITE_ROWS), + }; + let mut tx = self.begin_repeatable_read().await?; + let through = match &filter.cursor { + Some(cursor) => Some(cursor.through.clone()), + None => Self::query(upper_sql).bind(self.namespace.as_ref()).bind(self.namespace.as_ref()) + .bind(filter.workflow_type.as_str()).fetch_optional(&mut *tx).await? + .map(|row| DB::string(&row, "id")).transpose()?, + }; + let mut counts = Counts::default(); + let mut page = Vec::with_capacity(filter.limit + 1); + if through.is_some() { + let mut rows = Self::query(rows_sql).bind(self.namespace.as_ref()).bind(self.namespace.as_ref()) + .bind(filter.workflow_type.as_str()).bind(through.as_deref()).bind(through.as_deref()) + .fetch(&mut *tx); + let mut current: Option> = None; + while let Some(row) = rows.try_next().await? { + let id = DB::string(&row, "id")?; + if current.as_ref().is_some_and(|run| run.id != id) { + finish(current.take().expect("previous run"), filter, &mut counts, &mut page); + } + if current.is_none() { + let source = DB::optional_string(&row, "import_source")?; + let retained = DB::optional_instant(&row, "details_pruned_at")?.is_none() + && DB::optional_string(&row, "owner_id")?.is_some() + && matches!(source.as_deref(), None | Some("embedded_v2")); + current = Some(Run { id, instance: DB::string(&row,"workflow_instance_id")?, + status: DB::string(&row,"status")?, last_sequence: DB::number(&row,"last_history_sequence")?, + evidence: Evidence::new(&filter.change_id, retained) }); + } + if let Some(sequence) = DB::optional_number(&row, "sequence")? { + let evidence = &mut current.as_mut().expect("current run").evidence; + if DB::number(&row, "start_format_valid")? != 1 { + evidence.history_available = false; + } + evidence.event(sequence, &DB::string(&row,"event_type")?, + DB::optional_string(&row,"marker")?.as_deref()); + } + } + if let Some(run) = current { + finish(run, filter, &mut counts, &mut page); + } + } + tx.commit().await?; + let has_more = page.len() > filter.limit; + page.truncate(filter.limit); + let next_cursor = if has_more { + Some(Cursor { + version: 1, namespace: self.namespace.to_string(), workflow_type: filter.workflow_type.clone(), + change_id: filter.change_id.clone(), min_version: filter.min_version, state: filter.state.clone(), + after: page.last().expect("nonempty page")["run_id"].as_str().expect("run ID").to_owned(), + through: through.clone().expect("bounded cohort"), + }.encode(key)?) + } else { None }; + Ok(json!({"schema_version":1,"namespace":self.namespace.as_ref(),"workflow_type":filter.workflow_type, + "change_id":filter.change_id,"min_version":filter.min_version,"state":filter.state, + "consistency":"live","through_run_id":through,"counts":counts,"returned_count":page.len(), + "runs":page,"limit":filter.limit,"has_more":has_more,"next_cursor":next_cursor})) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + fn cursor_key() -> String { + let mut bytes = [0_u8; 32]; + getrandom::fill(&mut bytes).unwrap(); + URL_SAFE_NO_PAD.encode(bytes) + } + + fn marker(change_id: &str, version: i32) -> Value { + json!({"change_id":change_id,"version":version,"min_supported":i32::MIN,"max_supported":i32::MAX,"sequence":1}) + } + + fn evidence<'a>(change_id: &'a str) -> Evidence<'a> { + let mut evidence = Evidence::new(change_id, true); + evidence.record("WorkflowStarted", &json!({})); + evidence + } + + #[test] + fn absent_marker_never_proves_current_usage() { + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded", &marker("unrelated", 99)); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::MarkerMissing)); + } + + #[test] + fn versions_are_compared_as_signed_integers_at_the_boundary() { + for (version, expected) in [(-1, Usage::Legacy(-1)), (0, Usage::Legacy(0)), (1, Usage::Current(1)), (2, Usage::Current(2)), (i32::MAX, Usage::Current(i32::MAX))] { + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded", &marker("pricing-v2", version)); + assert_eq!(e.classify(1), expected); + } + } + + #[test] + fn unavailable_or_incomplete_history_overrides_a_retained_current_marker() { + for available in [false, true] { + let mut e = Evidence::new("pricing-v2", available); + if !available { + e.record("WorkflowStarted", &json!({})); + } + e.record("VersionMarkerRecorded", &marker("pricing-v2", 2)); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::HistoryUnavailable)); + } + } + + #[test] + fn malformed_and_conflicting_markers_remain_unknown() { + for payload in [json!({}), json!({"change_id":4,"version":2}), json!({"change_id":"pricing-v2","version":"2"}), json!({"change_id":"pricing-v2","version":2.0}), json!({"change_id":"pricing-v2","version":u64::MAX})] { + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded", &payload); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::MarkerInvalid)); + } + let mut e = evidence("pricing-v2"); + for (index, version) in [0, 2, 2].into_iter().enumerate() { + let mut payload = marker("pricing-v2", version); + payload["sequence"] = json!(index + 1); + e.record("VersionMarkerRecorded", &payload); + } + assert_eq!(e.classify(1), Usage::Unknown(Unknown::ConflictingMarkers)); + } + + #[test] + fn marker_identity_is_exact_and_duplicate_identical_evidence_is_allowed() { + let mut e = evidence("λ-Pricing"); + for (index, change_id) in ["Λ-Pricing", "λ-pricing", "λ-Pricing "].into_iter().enumerate() { + let mut payload = marker(change_id, 0); + payload["sequence"] = json!(index + 1); + e.record("VersionMarkerRecorded", &payload); + } + for sequence in 4..=5 { + let mut payload = marker("λ-Pricing", 2); + payload["sequence"] = json!(sequence); + e.record("VersionMarkerRecorded", &payload); + } + assert_eq!(e.classify(1), Usage::Current(2)); + } + + #[test] + fn marker_budget_and_history_gaps_do_not_report_a_partial_current_decision() { + let mut e = Evidence::new("pricing-v2", true); + e.event(1, "WorkflowStarted", None); + for sequence in 2..=1001 { + let mut payload = marker("pricing-v2", 2); + payload["sequence"] = json!(sequence - 1); + e.event(sequence, "VersionMarkerRecorded", Some(&payload.to_string())); + } + assert_eq!(e.classify(1), Usage::Current(2)); + let payload = marker("pricing-v2", 2).to_string(); + e.event(1002, "VersionMarkerRecorded", Some(&payload)); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::HistoryUnavailable)); + let mut e = Evidence::new("pricing-v2", true); + e.event(1, "WorkflowStarted", None); + e.event(3, "VersionMarkerRecorded", Some(&payload)); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::HistoryUnavailable)); + } + + #[test] + fn recorded_range_and_positive_logical_sequence_are_required() { + for replacement in [json!({"version":2147483648_i64}),json!({"min_supported":3}),json!({"max_supported":0}),json!({"sequence":0}),json!({"sequence":"1"})] { + let mut payload = marker("pricing-v2", 2); + for (key, value) in replacement.as_object().unwrap() { payload[key] = value.clone(); } + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded", &payload); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::MarkerInvalid)); + } + } + + #[test] + fn repeated_versions_can_expand_the_range_but_cannot_reuse_a_logical_sequence() { + let mut e = evidence("pricing-v2"); + for (sequence, minimum, maximum) in [(1,0,1), (2,-1,2)] { + e.record("VersionMarkerRecorded", &json!({"change_id":"pricing-v2","version":1, + "min_supported":minimum,"max_supported":maximum,"sequence":sequence})); + } + assert_eq!(e.classify(1), Usage::Current(1)); + e.record("VersionMarkerRecorded", &json!({"change_id":"pricing-v2","version":1, + "min_supported":-1,"max_supported":2,"sequence":2})); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::MarkerInvalid)); + } + + #[test] + fn cursor_authentication_rejects_unsigned_or_modified_bounds() { + let key = cursor_key(); + let other_key = cursor_key(); + assert_ne!(key, other_key); + let cursor = Cursor { version:1, namespace:"default".into(), workflow_type:"echo".into(), + change_id:"pricing-v2".into(), min_version:1, state:None, after:"A".into(), through:"Z".into() }; + let token = cursor.encode(&key).unwrap(); + assert!(Cursor::decode(&token, &key).is_ok()); + assert!(Cursor::decode(&token, &other_key).is_err()); + let body = serde_json::to_vec(&cursor).unwrap(); + assert!(Cursor::decode(&URL_SAFE_NO_PAD.encode(&body), &key).is_err()); + let (_, tag) = token.split_once('.').unwrap(); + for (field, value) in [("after","B"),("through","ZZ")] { + let mut modified = serde_json::to_value(&cursor).unwrap(); + modified[field] = json!(value); + let modified = format!("{}.{tag}",URL_SAFE_NO_PAD.encode(serde_json::to_vec(&modified).unwrap())); + assert!(Cursor::decode(&modified,&key).is_err()); + } + } + + #[test] + fn native_start_admission_prefix_and_child_start_prefix_are_supported() { + let payload = marker("pricing-v2", 1).to_string(); + for prefix in [vec!["StartAccepted","WorkflowStarted"], vec!["WorkflowStarted"]] { + let mut e = Evidence::new("pricing-v2", true); + for (index, event_type) in prefix.iter().enumerate() { + e.event(index as i64 + 1, event_type, None); + } + e.event(prefix.len() as i64 + 1, "VersionMarkerRecorded", Some(&payload)); + assert_eq!(e.classify(1), Usage::Current(1)); + } + let mut e = Evidence::new("pricing-v2", true); + e.event(1,"StartAccepted",None); + e.event(2,"VersionMarkerRecorded",Some(&payload)); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::HistoryUnavailable)); + } + + #[test] + fn cursor_round_trips_maximum_query_values_even_when_json_escapes_them() { + let key = cursor_key(); + let cursor = Cursor { version:1, namespace:"default".into(), workflow_type:"\0".repeat(255), + change_id:"\0".repeat(1024), min_version:1, state:None, after:"A".into(), through:"Z".into() }; + let token = cursor.encode(&key).unwrap(); + assert!(token.len() > 8192); + assert!(Cursor::decode(&token, &key).is_ok()); + } + + #[test] + fn malformed_start_prefixes_never_establish_a_current_decision() { + for prefix in [ + vec!["StartAccepted", "StartAccepted", "WorkflowStarted"], + vec!["StartAccepted", "OtherEvent", "WorkflowStarted"], + vec!["WorkflowStarted", "StartAccepted"], + vec!["WorkflowStarted", "WorkflowStarted"], + vec!["OtherEvent", "WorkflowStarted"], + ] { + let mut e = Evidence::new("pricing-v2", true); + for (index, event_type) in prefix.iter().enumerate() { + e.event(index as i64 + 1, event_type, None); + } + e.event(prefix.len() as i64 + 1, "VersionMarkerRecorded", Some(&marker("pricing-v2",1).to_string())); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::HistoryUnavailable), "{prefix:?}"); + } + } + + #[test] + fn parity_unsupported_event_after_a_current_marker_remains_unavailable() { + let mut e = Evidence::new("pricing-v2", true); + e.event(1,"WorkflowStarted",None); + e.event(2,"VersionMarkerRecorded",Some(&marker("pricing-v2",1).to_string())); + e.event(3,"FutureHistoryEvent",None); + assert_eq!(e.classify(1), Usage::Unknown(Unknown::HistoryUnavailable)); + } + + #[test] + fn parity_unknown_marker_fields_require_a_reader_upgrade() { + for change_id in ["pricing-v2","other-change"] { + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded",&marker("pricing-v2",1)); + let mut future = marker(change_id,1); + future["sequence"] = json!(2); + future["future_format"] = json!(2); + e.record("VersionMarkerRecorded",&future); + assert_eq!(e.classify(1),Usage::Unknown(Unknown::HistoryUnavailable)); + } + } + + #[test] + fn parity_malformed_other_change_decisions_cannot_be_ignored() { + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded",&marker("pricing-v2",1)); + let mut invalid = marker("other-change",1); + invalid["sequence"] = json!(2); + invalid["min_supported"] = json!(2); + e.record("VersionMarkerRecorded",&invalid); + assert_eq!(e.classify(1),Usage::Unknown(Unknown::MarkerInvalid)); + } + + #[test] + fn reuse_metadata_must_be_a_complete_valid_pair_for_every_change() { + for change_id in ["pricing-v2", "other-change"] { + for fields in [ + json!({"reused_from_run_id":"source"}), + json!({"reused_recorded_at":"2026-01-02T03:04:05Z"}), + json!({"reused_from_run_id":[],"reused_recorded_at":"2026-01-02T03:04:05Z"}), + json!({"reused_from_run_id":"","reused_recorded_at":"2026-01-02T03:04:05Z"}), + json!({"reused_from_run_id":" source ","reused_recorded_at":"2026-01-02T03:04:05Z"}), + json!({"reused_from_run_id":"source","reused_recorded_at":null}), + json!({"reused_from_run_id":"source","reused_recorded_at":""}), + json!({"reused_from_run_id":"source","reused_recorded_at":"not-a-clock"}), + json!({"reused_from_run_id":"source","reused_recorded_at":"2026-02-30T03:04:05Z"}), + json!({"reused_from_run_id":"source","reused_recorded_at":"2026-01-02T03:04:05"}), + json!({"reused_from_run_id":"source","reused_recorded_at":"2026-01-02T03:04:05.1234567890Z"}), + ] { + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded",&marker("pricing-v2",1)); + let mut payload = marker(change_id,1); + payload["sequence"] = json!(2); + payload.as_object_mut().unwrap().extend(fields.as_object().unwrap().clone()); + e.record("VersionMarkerRecorded",&payload); + assert_eq!(e.classify(1),Usage::Unknown(Unknown::MarkerInvalid),"{payload}"); + } + } + } + + #[test] + fn reuse_metadata_accepts_native_precision_and_explicit_offsets() { + for clock in ["2026-01-02T03:04:05Z","2026-01-02T03:04:05+00:00", + "2026-01-02T03:04:05.123Z","2026-01-02T03:04:05.123456+00:00", + "2026-01-02T03:04:05.123456789+02:30"] { + let mut payload = marker("pricing-v2",1); + payload["reused_from_run_id"] = json!("source-run"); + payload["reused_recorded_at"] = json!(clock); + let mut e = evidence("pricing-v2"); + e.record("VersionMarkerRecorded",&payload); + assert_eq!(e.classify(1),Usage::Current(1),"{clock}"); + } + } +} diff --git a/rust/src/runtime/workflow_import.rs b/rust/src/runtime/workflow_import.rs index 754a5ad04..bdce7195d 100644 --- a/rust/src/runtime/workflow_import.rs +++ b/rust/src/runtime/workflow_import.rs @@ -42,7 +42,7 @@ const SOURCE_RUNTIME: &str = "embedded"; const RUN_STATUSES: &[&str] = &["pending", "running", "waiting", "cancelled", "terminated", "completed", "failed"]; const TERMINAL_STATUSES: &[&str] = &["cancelled", "terminated", "completed", "failed"]; -const HISTORY_EVENT_TYPES: &[&str] = &[ +pub(super) const HISTORY_EVENT_TYPES: &[&str] = &[ "StartAccepted", "StartRejected", "WorkflowStarted", "WorkflowContinuedAsNew", "ChildWorkflowScheduled", "ChildCancellationRequested", "ChildCancellationResolved", "ChildRunStarted", "ChildRunCompleted", "ChildRunFailed", "ChildRunCancelled", "ChildRunTerminated", "ServiceCallStarted", "ServiceCallCompleted", "ServiceCallFailed", diff --git a/rust/tests/execution.rs b/rust/tests/execution.rs index 5fdac43b9..40f3d5439 100644 --- a/rust/tests/execution.rs +++ b/rust/tests/execution.rs @@ -24,6 +24,9 @@ mod visibility; #[path = "execution/visibility_collation.rs"] mod visibility_collation; +#[path = "execution/version_usage.rs"] +mod version_usage; + #[path = "execution/memos.rs"] mod memos; diff --git a/rust/tests/execution/redrive.rs b/rust/tests/execution/redrive.rs index 497300070..14efcb6f2 100644 --- a/rust/tests/execution/redrive.rs +++ b/rust/tests/execution/redrive.rs @@ -258,7 +258,7 @@ async fn a_redriven_run_cancels_and_terminates_like_php() { /// A run like `failed_at_second_activity`, started with `start` fields and /// with `extra` commands, one step each, in its first workflow task. -async fn failed_with(app: &Router, workflow: &str, start: Value, extra: Vec) -> Value { +pub(super) async fn failed_with(app: &Router, workflow: &str, start: Value, extra: Vec) -> Value { let failed_step = 2 + extra.len(); let mut body = json!({"workflow_id":workflow,"workflow_type":"echo","task_queue":"test","input":envelope(Payload::Array(vec![]))}); for (key, value) in start.as_object().unwrap() { diff --git a/rust/tests/execution/version_usage.rs b/rust/tests/execution/version_usage.rs new file mode 100644 index 000000000..8af6deb9e --- /dev/null +++ b/rust/tests/execution/version_usage.rs @@ -0,0 +1,360 @@ +use super::*; +use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD}; + + +// Fixture statements are compiled templates. All run identities remain bound, +// including after MySQL's placeholder-only dialect conversion. +async fn fixture(database: &TestDatabase, run: &Value, statement: &'static str) { + let run_id = run["run_id"].as_str().unwrap(); + match database { + TestDatabase::Sqlite(directory) => { + let pool = sqlx::SqlitePool::connect(&format!("sqlite://{}", directory.path().join("runtime.sqlite").display())).await.unwrap(); + sqlx::query(statement).bind(run_id).execute(&pool).await.unwrap(); + pool.close().await; + } + TestDatabase::Postgres { options, .. } => { + let pool = sqlx::postgres::PgPoolOptions::new().connect_with(options.as_ref().clone()).await.unwrap(); + sqlx::query(statement).bind(run_id).execute(&pool).await.unwrap(); + pool.close().await; + } + TestDatabase::MySql { options, .. } => { + let pool = sqlx::mysql::MySqlPoolOptions::new().connect_with(options.as_ref().clone()).await.unwrap(); + sqlx::query(sqlx::AssertSqlSafe(statement.replace("$1", "?"))).bind(run_id).execute(&pool).await.unwrap(); + pool.close().await; + } + } +} + +const QUERY: &str = "workflow_type=echo&change_id=pricing-v2&min_version=1"; + +#[tokio::test] +async fn version_usage_does_not_reserve_an_existing_workflow_id() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + let started = start(&app, "version-usage").await; + let (status, described) = request(&app, "GET", "/api/workflows/version-usage", Value::Null).await; + assert_eq!(status, StatusCode::OK, "{described}"); + assert_eq!(described["workflow_id"], "version-usage"); + assert_eq!(described["run_id"], started["run_id"]); + assert_eq!(usage(&app, QUERY).await["counts"]["total"], 1); + runtime.close().await; + database.remove().await; +} + +async fn usage(app: &Router, query: &str) -> Value { + let (status, body) = request(app, "GET", &format!("/api/workflow-version-usage?{query}"), Value::Null).await; + assert_eq!(status, StatusCode::OK, "{body}"); + body +} + +async fn recorded(app: &Router, workflow: &str, version: i32) -> Value { + let run = start(app, workflow).await; + let task = poll(app, "version-reader-worker", "workflow").await; + assert_eq!(task["run_id"], run["run_id"]); + let (status, body) = finish_task(app, &task, + json!([{"type":"record_version_marker", "change_id":"pricing-v2", + "version":version,"min_supported":-1,"max_supported":2}])).await; + assert_eq!(status, StatusCode::OK, "{body}"); + run +} + +#[tokio::test] +async fn repeated_decisions_with_a_wider_valid_range_keep_the_recorded_version() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + register(&app, "version-reader-worker", json!(["echo"]), json!([])).await; + let run = start(&app, "version-expanding-range").await; + let task = poll(&app, "version-reader-worker", "workflow").await; + assert_eq!(task["run_id"], run["run_id"]); + let (status, body) = finish_task(&app, &task, json!([ + {"type":"record_version_marker","change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":1}, + {"type":"record_version_marker","change_id":"pricing-v2","version":1,"min_supported":-1,"max_supported":2} + ])).await; + assert_eq!(status, StatusCode::OK, "{body}"); + let history = run_history(&app, &run["workflow_id"], &run["run_id"]).await; + let answer = usage(&app, QUERY).await; + let metadata: Vec<_> = history.as_array().unwrap().iter().map(|event| + json!({"sequence":event["sequence"],"event_type":event["event_type"],"type":event["type"]})).collect(); + assert_eq!(answer["counts"], json!({"total":1,"legacy":0,"current":1,"unknown":0}), + "answer={answer}; history={metadata:?}"); + assert_eq!(answer["runs"][0]["recorded_version"], 1); + assert_eq!(answer["runs"][0]["unknown_reason"], Value::Null); + assert_eq!(run_history(&app, &run["workflow_id"], &run["run_id"]).await, history); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn open_runs_report_all_counts_before_filter_and_never_upgrade_missing_markers() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + register(&app, "version-reader-worker", json!(["echo"]), json!([])).await; + let legacy = recorded(&app, "version-usage-old", 0).await; + let current = recorded(&app, "version-usage-new", 1).await; + let closed = recorded(&app, "version-usage-closed", -1).await; + fixture(&database, &closed, "UPDATE workflow_runs SET status='completed' WHERE id=$1").await; + let absent = start(&app, "version-usage-unreached").await; + let page = usage(&app, QUERY).await; + assert_eq!(page["schema_version"], 1); + assert_eq!(page["namespace"], "default"); + assert_eq!(page["consistency"], "live"); + assert_eq!(page["counts"], json!({"total":3,"legacy":1,"current":1,"unknown":1})); + assert_eq!(page["returned_count"], 3); + assert!(page.get("safe_to_remove").is_none()); + let runs = page["runs"].as_array().unwrap(); + for (run, state, version, reason) in [(&legacy,"legacy",json!(0),Value::Null),(¤t,"current",json!(1),Value::Null),(&absent,"unknown",Value::Null,json!("marker_absent"))] { + let actual = runs.iter().find(|value| value["run_id"] == run["run_id"]).unwrap(); + assert_eq!(actual["state"], state); + assert_eq!(actual["recorded_version"], version); + assert_eq!(actual["unknown_reason"], reason); + } + let filtered = usage(&app, &format!("{QUERY}&state=unknown&limit=1")).await; + assert_eq!(filtered["counts"], page["counts"]); + assert_eq!(filtered["runs"][0]["run_id"], absent["run_id"]); + assert_eq!(filtered["has_more"], false); + assert_eq!(filtered["next_cursor"], Value::Null); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn pruned_missing_prefix_and_conflicting_history_remain_unknown() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + register(&app, "version-reader-worker", json!(["echo"]), json!([])).await; + let pruned = recorded(&app, "version-usage-pruned", 2).await; + let prefix = recorded(&app, "version-usage-prefix", 2).await; + let conflict = recorded(&app, "version-usage-conflict", 2).await; + let unsupported = recorded(&app, "version-usage-format", 2).await; + let late_start = recorded(&app, "version-usage-late-start", 2).await; + fixture(&database, &pruned, "UPDATE workflow_runs SET details_pruned_at='2026-01-01 00:00:00' WHERE id=$1").await; + fixture(&database, &prefix, "DELETE FROM workflow_history_events WHERE workflow_run_id=$1 AND sequence=1").await; + // A retained malformed decision must not become current merely because its + // version is newer. This is private storage-corruption regression data. + fixture(&database, &conflict, r#"UPDATE workflow_history_events SET payload='{"change_id":"pricing-v2","version":2,"min_supported":3,"max_supported":4,"sequence":1}' WHERE workflow_run_id=$1 AND event_type='VersionMarkerRecorded'"#).await; + fixture(&database, &unsupported, "UPDATE workflow_history_events SET payload='[]' WHERE workflow_run_id=$1 AND event_type='WorkflowStarted'").await; + fixture(&database, &late_start, "INSERT INTO workflow_history_events(id,workflow_run_id,sequence,event_type,payload,recorded_at) SELECT 'version-late-admission',workflow_run_id,MAX(sequence)+1,'StartAccepted','{}','2026-01-01 00:00:00' FROM workflow_history_events WHERE workflow_run_id=$1 GROUP BY workflow_run_id").await; + fixture(&database, &late_start, "UPDATE workflow_runs SET last_history_sequence=last_history_sequence+1 WHERE id=$1").await; + let page = usage(&app, QUERY).await; + assert_eq!(page["counts"], json!({"total":5,"legacy":0,"current":0,"unknown":5})); + for run in [&pruned, &prefix, &unsupported, &late_start] { + let record = page["runs"].as_array().unwrap().iter().find(|value| value["run_id"] == run["run_id"]).unwrap(); + assert_eq!(record["unknown_reason"], "history_unavailable"); + } + let record = page["runs"].as_array().unwrap().iter().find(|value| value["run_id"] == conflict["run_id"]).unwrap(); + assert_eq!(record["unknown_reason"], "history_invalid"); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn keyset_cursors_bind_query_identity_and_keep_the_initial_upper_bound() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + for workflow in ["version-page-a", "version-page-b", "version-page-c"] { + start(&app, workflow).await; + } + let first = usage(&app, &format!("{QUERY}&limit=1")).await; + assert_eq!(first["has_more"], true); + let cursor = first["next_cursor"].as_str().unwrap(); + let (body, tag) = cursor.split_once('.').unwrap(); + let cursor_body: Value = serde_json::from_slice(&URL_SAFE_NO_PAD.decode(body).unwrap()).unwrap(); + for (field, value) in [("after", "0"), ("through", "ZZZZ")] { + let mut altered = cursor_body.clone(); + altered[field] = json!(value); + let altered = format!("{}.{tag}", URL_SAFE_NO_PAD.encode(serde_json::to_vec(&altered).unwrap())); + let (status, body) = request(&app, "GET", &format!("/api/workflow-version-usage?{QUERY}&cursor={altered}"), Value::Null).await; + assert_eq!(status, StatusCode::UNPROCESSABLE_ENTITY, "{body}"); + assert_eq!(body["reason"], "validation_failed"); + } + for changed in ["workflow_type=other&change_id=pricing-v2&min_version=1", "workflow_type=echo&change_id=other&min_version=1", "workflow_type=echo&change_id=pricing-v2&min_version=2"] { + let (status, body) = request(&app, "GET", &format!("/api/workflow-version-usage?{changed}&cursor={cursor}"), Value::Null).await; + assert_eq!(status, StatusCode::UNPROCESSABLE_ENTITY, "{body}"); + assert_eq!(body["reason"], "validation_failed"); + } + // Cross the ULID millisecond boundary so this admission is above the + // captured upper ID rather than relying on same-millisecond entropy order. + tokio::time::sleep(std::time::Duration::from_millis(2)).await; + let later = start(&app, "version-page-later").await; + assert!(later["run_id"].as_str().unwrap() > first["through_run_id"].as_str().unwrap()); + fixture(&database, &first["runs"][0], "UPDATE workflow_runs SET status='completed' WHERE id=$1").await; + let second = usage(&app, &format!("{QUERY}&limit=200&cursor={cursor}")).await; + assert_eq!(second["through_run_id"], first["through_run_id"]); + assert_eq!(second["counts"]["total"], 2); + assert_eq!(second["returned_count"], 2); + assert_eq!(second["has_more"], false); + let first_id = first["runs"][0]["run_id"].as_str().unwrap(); + assert!(second["runs"].as_array().unwrap().iter().all(|run| run["run_id"].as_str().unwrap() > first_id)); + assert_eq!(usage(&app, QUERY).await["counts"]["total"], 3); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn named_namespace_and_exact_workflow_type_do_not_leak_other_runs() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + for namespace in ["version-tenant-a", "version-tenant-b"] { + namespaces::create(&app, namespace).await; + namespaces::start_scoped(&app, namespace, &format!("{namespace}-run")).await; + } + start(&app, "version-default").await; + start(&app, "version-default-second").await; + let default_page = usage(&app, &format!("{QUERY}&limit=1")).await; + let foreign_cursor = default_page["next_cursor"].as_str().unwrap(); + let (status, body) = namespaces::scoped(&app, "version-tenant-a", "GET", + &format!("/api/workflow-version-usage?{QUERY}&cursor={foreign_cursor}"), Value::Null).await; + assert_eq!(status, StatusCode::UNPROCESSABLE_ENTITY, "{body}"); + assert_eq!(body["reason"], "validation_failed"); + let path = format!("/api/workflow-version-usage?{QUERY}&limit=1"); + for namespace in ["version-tenant-a", "version-tenant-b"] { + let (status, body) = namespaces::scoped(&app, namespace, "GET", &path, Value::Null).await; + assert_eq!(status, StatusCode::OK, "{body}"); + assert_eq!(body["namespace"], namespace); + assert_eq!(body["counts"]["total"], 1); + assert_eq!(body["runs"][0]["instance_id"], format!("{namespace}-run")); + } + let wrong_case = usage(&app, "workflow_type=ECHO&change_id=pricing-v2&min_version=1").await; + assert_eq!(wrong_case["counts"]["total"], 0); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn all_canonical_open_statuses_participate_including_an_archived_open_run() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + for (name, statement) in [ + ("pending", "UPDATE workflow_runs SET status='pending' WHERE id=$1"), + ("running", "UPDATE workflow_runs SET status='running' WHERE id=$1"), + ("waiting", "UPDATE workflow_runs SET status='waiting',archived_at='2026-01-01 00:00:00' WHERE id=$1"), + ("completed", "UPDATE workflow_runs SET status='completed' WHERE id=$1"), + ("failed", "UPDATE workflow_runs SET status='failed' WHERE id=$1"), + ("cancelled", "UPDATE workflow_runs SET status='cancelled' WHERE id=$1"), + ("terminated", "UPDATE workflow_runs SET status='terminated' WHERE id=$1"), + ] { + let run = start(&app, &format!("version-status-{name}")).await; + fixture(&database, &run, statement).await; + } + let answer = usage(&app, QUERY).await; + assert_eq!(answer["counts"], json!({"total":3,"legacy":0,"current":0,"unknown":3})); + let mut statuses: Vec<_> = answer["runs"].as_array().unwrap().iter().map(|run| run["status"].as_str().unwrap()).collect(); + statuses.sort_unstable(); + assert_eq!(statuses, ["pending", "running", "waiting"]); + assert_eq!(request(&app,"GET","/api/cluster/info",Value::Null).await.1["capabilities"]["workflow_version_usage"], true); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn invalid_requests_refuse_instead_of_returning_an_empty_success() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + for query in ["", "workflow_type=echo&change_id=pricing-v2", "workflow_type=echo&change_id=pricing-v2&min_version=2147483648", "workflow_type=echo&change_id=pricing-v2&min_version=1&limit=0", "workflow_type=echo&change_id=pricing-v2&min_version=1&limit=201", "workflow_type=echo&change_id=pricing-v2&min_version=1&state=closed", "workflow_type=echo&change_id=pricing-v2&min_version=1&cursor=invalid"] { + let (status, body) = request(&app, "GET", &format!("/api/workflow-version-usage?{query}"), Value::Null).await; + assert_eq!(status, StatusCode::UNPROCESSABLE_ENTITY, "{query}: {body}"); + assert_eq!(body["reason"], "validation_failed"); + } + let empty = usage(&app, QUERY).await; + assert_eq!(empty["counts"], json!({"total":0,"legacy":0,"current":0,"unknown":0})); + assert_eq!(empty["through_run_id"], Value::Null); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn parity_missing_or_foreign_instance_metadata_stays_in_unknown_counts() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + register(&app,"version-reader-worker",json!(["echo"]),json!([])).await; + let missing = recorded(&app,"version-missing-owner",1).await; + let foreign = recorded(&app,"version-foreign-owner",1).await; + fixture(&database,&json!({"run_id":missing["workflow_id"]}),"DELETE FROM workflow_instances WHERE id=$1").await; + fixture(&database,&json!({"run_id":foreign["workflow_id"]}),"UPDATE workflow_instances SET namespace='unrelated-owner' WHERE id=$1").await; + let answer = usage(&app, &format!("{QUERY}&limit=1")).await; + assert_eq!(answer["counts"],json!({"total":2,"legacy":0,"current":0,"unknown":2})); + assert_eq!(answer["runs"][0]["unknown_reason"],"history_unavailable"); + assert_eq!(answer["has_more"],true); + let next = usage(&app,&format!("{QUERY}&limit=1&cursor={}",answer["next_cursor"].as_str().unwrap())).await; + assert_eq!(next["counts"],answer["counts"]); + assert_eq!(next["runs"][0]["unknown_reason"],"history_unavailable"); + assert_ne!(next["runs"][0]["run_id"],answer["runs"][0]["run_id"]); + assert_eq!(next["has_more"],false); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn parity_future_history_formats_do_not_report_current_usage() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + register(&app,"version-reader-worker",json!(["echo"]),json!([])).await; + let future = recorded(&app,"version-future-event",1).await; + let marker = recorded(&app,"version-future-marker",1).await; + fixture(&database,&future,"INSERT INTO workflow_history_events(id,workflow_run_id,sequence,event_type,payload,recorded_at) SELECT 'version-usage-future-event',workflow_run_id,MAX(sequence)+1,'FutureHistoryEvent','{}','2026-01-01 00:00:00' FROM workflow_history_events WHERE workflow_run_id=$1 GROUP BY workflow_run_id").await; + fixture(&database,&future,"UPDATE workflow_runs SET last_history_sequence=last_history_sequence+1 WHERE id=$1").await; + fixture(&database,&marker,r#"UPDATE workflow_history_events SET payload='{"change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":2,"sequence":1,"future_format":2}' WHERE workflow_run_id=$1 AND event_type='VersionMarkerRecorded'"#).await; + let answer = usage(&app,QUERY).await; + assert_eq!(answer["counts"],json!({"total":2,"legacy":0,"current":0,"unknown":2})); + assert!(answer["runs"].as_array().unwrap().iter().all(|run|run["unknown_reason"]=="history_unavailable")); + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn malformed_reuse_provenance_is_unknown_in_stored_history() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + register(&app,"version-reader-worker",json!(["echo"]),json!([])).await; + let run = recorded(&app,"version-provenance",1).await; + for statement in [ + r#"UPDATE workflow_history_events SET payload='{"change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":1,"sequence":1,"reused_from_run_id":"source"}' WHERE workflow_run_id=$1 AND event_type='VersionMarkerRecorded'"#, + r#"UPDATE workflow_history_events SET payload='{"change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":1,"sequence":1,"reused_from_run_id":[],"reused_recorded_at":"2026-01-02T03:04:05Z"}' WHERE workflow_run_id=$1 AND event_type='VersionMarkerRecorded'"#, + r#"UPDATE workflow_history_events SET payload='{"change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":1,"sequence":1,"reused_from_run_id":"source","reused_recorded_at":""}' WHERE workflow_run_id=$1 AND event_type='VersionMarkerRecorded'"#, + r#"UPDATE workflow_history_events SET payload='{"change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":1,"sequence":1,"reused_from_run_id":"source","reused_recorded_at":"not-a-clock"}' WHERE workflow_run_id=$1 AND event_type='VersionMarkerRecorded'"#, + ] { + fixture(&database,&run,statement).await; + let answer = usage(&app,QUERY).await; + assert_eq!(answer["counts"],json!({"total":1,"legacy":0,"current":0,"unknown":1})); + assert_eq!(answer["runs"][0]["unknown_reason"],"history_invalid"); + } + runtime.close().await; + database.remove().await; +} + +#[tokio::test] +async fn an_actual_redriven_marker_retains_a_queryable_recorded_decision() { + let database = TestDatabase::new().await; + let runtime = database.open().await.unwrap(); + let app = router(runtime.clone()); + let registered = request(&app,"POST","/api/worker/register",json!({"worker_id":"worker","task_queue":"test", + "runtime":"php","supported_workflow_types":["echo"],"supported_activity_types":["a","b"], + "workflow_definition_fingerprints":{"echo":redrive::FINGERPRINT}})).await; + assert_eq!(registered.0,StatusCode::CREATED,"{}",registered.1); + let source = redrive::failed_with(&app,"version-reused-marker",json!({}),vec![ + json!({"type":"record_version_marker","change_id":"pricing-v2","version":1,"min_supported":0,"max_supported":1}) + ]).await; + let (status, successor) = request(&app,"POST",&format!("/api/workflows/version-reused-marker/runs/{}/redrive",source["run_id"].as_str().unwrap()),json!({})).await; + assert_eq!(status,StatusCode::ACCEPTED,"{successor}"); + let before = run_history(&app,&source["workflow_id"],&successor["run_id"]).await; + let marker = before.as_array().unwrap().iter().find(|event|event["event_type"]=="VersionMarkerRecorded").unwrap(); + assert_eq!(marker["payload"]["reused_from_run_id"],source["run_id"]); + assert!(chrono::DateTime::parse_from_rfc3339(marker["payload"]["reused_recorded_at"].as_str().unwrap()).is_ok()); + let answer = usage(&app,QUERY).await; + assert_eq!(answer["counts"],json!({"total":1,"legacy":0,"current":1,"unknown":0}),"{answer}"); + assert_eq!(answer["runs"][0]["run_id"],successor["run_id"]); + assert_eq!(run_history(&app,&source["workflow_id"],&successor["run_id"]).await,before); + runtime.close().await; + database.remove().await; +} diff --git a/tests/Fixtures/ServerParity/cluster-capabilities.json b/tests/Fixtures/ServerParity/cluster-capabilities.json index ca6f776ed..ec47b0b48 100644 --- a/tests/Fixtures/ServerParity/cluster-capabilities.json +++ b/tests/Fixtures/ServerParity/cluster-capabilities.json @@ -456,6 +456,10 @@ } }, "reason": "Native's client_compatibility (server#1005) cites only protocols it serves beside the block: the auth composition contract, the control plane with its request contract, and worker protocol 1.20. Published PHP's block also cites the skew refusal matrix and five worker_protocol external execution contracts, which native does not publish; the CLI checks every protocol the block requires, so native must not cite them." + }, + "capabilities.workflow_version_usage": { + "value": true, + "reason": "Native serves GET /api/workflow-version-usage for namespace-scoped open-run legacy/current/unknown counts and live keyset pages. Missing markers and unavailable provenance remain unknown, with no safe-to-remove flag (Server #1220). Published PHP has no such endpoint." } } } diff --git a/tests/Unit/ServerParityClusterCapabilitiesContractTest.mjs b/tests/Unit/ServerParityClusterCapabilitiesContractTest.mjs index 97d6e89fd..6b4a88399 100644 --- a/tests/Unit/ServerParityClusterCapabilitiesContractTest.mjs +++ b/tests/Unit/ServerParityClusterCapabilitiesContractTest.mjs @@ -49,6 +49,8 @@ test('every reviewed native addition is a member published PHP lacks and gives a for (const [name, mutate] of [ ['native without its child start turn', state => { delete state.capabilities.child_workflow_start_turn; }], + ['native without version usage', state => { delete state.capabilities.workflow_version_usage; }], + ['native declaring version usage unavailable', state => { state.capabilities.workflow_version_usage = false; }], ['native copying a manifest flag it does not publish', state => { state.capabilities.timer_runtime_contract = true; }], ['native advertising response compression', state => { state.capabilities.response_compression = ['gzip']; }], ['native without its unqualified list', state => { state.native_unqualified = []; }],