Repository navigation
Description
Primary owning public component
Workflow engine
User problem
We use Durable Workflow to execute long-running business processes with nested workflows. Business cancellation may require undoing earlier operations after several transitions, while a child workflow is running, or after that child has already completed.
A forward activity may also produce an external side effect without returning a recorded result. Compensation must then identify and reconcile the original operation without relying on its successful return value.
Native Saga, replay, child workflows, and cooperative cancellation already provide important building blocks. We need a supported contract for retaining and addressing compensation obligations beyond the lifetime of their original workflow run.
Without such a contract, our integration must implement its own obligation registry, request routing, recovery ownership, and completion protocol around native Saga.
Example: a shipment workflow creates a carrier booking and completes. The parent order is cancelled later. The original shipment obligations must still be discoverable and compensated, even though its workflow run can no longer handle a cancellation request.
Proposed public contract
Introduce an optional compensation-scope abstraction built on native Saga.
The following describes proposed semantics, not existing API names:
-
Stable identity
A scope identifies the original business execution and its generation independently of a transient workflow run ID. Requests must never silently target a replacement execution. -
Registration before external effects
Allow a durable compensation obligation to be registered before the forward operation starts. Its handler receives a stable operation reference and can reconcile an absent, completed, partial, or unknown effect without requiring a successful forward result. -
Portable descriptors
Retained obligations contain a handler type/version, portable arguments, registration order, and ownership information. Persisted recovery must not depend on serializing arbitrary PHP closures. -
Idempotent requests
Request compensation using a stable request ID. Distinguish durable request acceptance from verified compensation completion. -
Nested scopes
A parent can register a child scope as a compensation participant. In sequential compensation, the parent waits for that child's obligations to be resolved before continuing with earlier parent obligations. -
Recovery after run closure
An explicit recovery execution may claim unresolved obligations from a closed run and execute them through native Saga. Preserve the original run's terminal history and prevent concurrent recovery owners. -
Explicit outcomes
Expose pending, resolved, and unresolved failure outcomes. Request delivery, child termination, timeout, or exhausted retries must not imply successful business compensation. -
Retention and compatibility
Define retention, release, handler-version compatibility, and unavailable-history behavior. Existing Saga and cancellation APIs remain compatible.
Business rules, external-effect verification, authorization, and idempotent handlers remain application responsibilities. This proposal does not promise exactly-once external effects.
Acceptance criteria
- Existing Saga behavior and public APIs remain backward compatible.
- Worker restart preserves obligation identity, ordering, and recorded compensation progress.
- A crash after an external effect but before recording the forward result can be recovered using the original operation reference.
- Cancellation during a child workflow waits for verified child compensation before earlier parent compensation in sequential mode.
- A completed child with retained obligations can be compensated through an explicit recovery execution.
- A request racing with run closure remains discoverable and receives an explicit outcome.
- Duplicate requests and competing recovery workers cannot create independent owners for the same obligation.
- Late forward completion is reconciled; compensation cannot report success merely because the effect was temporarily absent.
- Retry exhaustion, unknown outcomes, and failed compensation remain visibly unresolved.
- Generation changes cannot redirect an old request to a newer business execution.
- Retention expiry and incompatible handler versions produce explicit failures.
- Public examples and automated tests cover restart, lost acknowledgement, closed-child recovery, and compensation failure.
Initial scope may be limited to embedded Laravel and sequential nested compensation. Parallel dependency ordering can be a separate enhancement.
Supporting public evidence
-
Official “Sagas” documentation:
Native Saga executes registered compensations in reverse order and reconstructs registration through replay. Its example registers compensation after the forward activity succeeds. It also explains that workflow timeout and immediate cancellation can close the run before cleanup executes. -
Official “Child Workflows” documentation:
Child invocation identity is distinct from run identity. Parent-close policies act on open children, skip children that have already closed, and do not prevent parent closure when enforcement fails. -
Official “Cooperative cancellation” documentation:
Active runs support bounded cleanup, child cancellation policies, replay recovery, and cancellation diagnostics. These are existing capabilities and should be reused.
These documented boundaries motivate a separate contract for retained business compensation obligations, particularly after the originating run has closed. This is an enhancement proposal, not a claim that existing Saga replay or cooperative cancellation is broken.
Dependencies and affected repositories
Primary:
- durable-workflow/workflow:
Public compensation-scope contract, durable obligation storage, recovery ownership, native Saga integration, and regression tests.
Documentation:
- durable-workflow/durable-workflow.github.io:
Contract documentation, lifecycle guarantees, limitations, and runnable examples.
Optional follow-up:
- durable-workflow/waterline:
Display scope identity, originating and recovery runs, unresolved obligations, compensation requests, and verified outcomes.
Conditional follow-up:
- Service-mode runtime and SDK support, if maintainers decide to expose the same contract beyond embedded Laravel. Cross-SDK support is not required for the initial implementation.
No Symfony Workflow dependency is required. The contract should remain independent of application entities and state-machine libraries.
Public intake checks
- I searched open and closed GitHub issues for an existing request.
- This request contains only public product contracts and public-safe rationale.
Activity
rmcdaniel commentedon Nov 7, 2026
Security contract review — required invariants before implementation
I reviewed the current public proposal, exact body 6,936B/SHA256
8daa9ca0a6d13b56da2074c6f1197ea7fd05d485414010f202b78f8d19bb0544. The proposed stable generation, registration-before-effect, explicit outcomes, and fenced recovery direction is sound. The following must be part of the public contract and causal tests before any Source acceptance; they are not optional implementation details.Tenant and authority binding. Scope, obligation, operation, child-scope, and request identities must be bound to the originating namespace/application plus immutable business generation. Lookup by a globally guessable ID must not cross that boundary. Recovery must re-establish an authorized principal/capability for the exact scope; replay must not silently substitute the current user or a newer generation.
Safe portable descriptors. Persist only a closed, versioned handler registry and canonical bounded data. Never deserialize arbitrary PHP objects/classes, closures, callable names, service-container references, or code supplied by the stored record. Handler type/version, argument schema, and payload hash must be validated before claim/execution. Secrets and raw credentials must not be embedded in history, descriptors, errors, or audit output; use protected references with explicit expiry/rotation failure.
Atomic ownership and fencing. Claim, renew, abandon, and complete must use one monotonic fence/lease generation in the same transaction as visible state. A stale worker must be unable both to execute under lost authority and to record success/failure afterward. Duplicate request IDs bind the original caller/scope/content hash; reuse with different arguments, generation, or authorization must refuse rather than deduplicate.
Uncertain external-effect state. Registration-before-effect needs an explicit state machine separating registration, dispatch/attempt, observed effect, compensation attempt, verified compensation, and unresolved/unknown. A crash, timeout, lost acknowledgement, or temporarily absent effect must never become “resolved.” A handler may reconcile absence only from authoritative external evidence and must preserve the original operation reference and attempt identity.
Deterministic replay versus live authorization. Replay reconstructs recorded decisions only; it must not perform live I/O, resolve mutable service bindings, read current authorization, or execute compensation. Live recovery performs current authorization/handler-availability checks outside replay and records their exact decision/version. The boundary needs predecessor/candidate tests proving replay cannot trigger an effect.
Nested-scope graph safety. Child relationships must be acyclic, generation-bound, and capped by depth, fan-out, descriptor bytes, and total obligations. Parent sequential progress requires a verified terminal child-compensation outcome, not request delivery, child termination, lease expiry, or timeout. Missing/expired child history stays explicitly unresolved.
Compatibility and retirement. Handler upgrades require an explicit compatibility/migration table; an unknown or incompatible version refuses before effect. Retention expiry must be atomic with claims/requests and leave an auditable terminal reason, never redirect or silently drop a pending obligation. Released scope data must not leave reusable operation/request identifiers that can address a new generation.
Operational bounds and audit. Cap registrations, descriptor size, retries, lease duration, total recovery work, nested traversal, and error payloads. Audit records need immutable scope/generation, originating run, obligation/order, request/content hash, claimant/fence, handler version, attempt, authoritative outcome, and retirement reason—while redacting secret material. Metrics must distinguish accepted, executing, verified, failed, unknown, expired, incompatible, and fenced-stale states.
Required adversarial controls include cross-tenant ID substitution; same request ID/different payload; generation replacement; stale claimant after lease transfer; crash before/after external effect; lost request and completion acknowledgements; late forward completion racing compensation; cyclic/deep child graphs; descriptor type/version substitution; secret-bearing payload/error refusal; replay attempting I/O; retention racing claim; and parent progress after only child delivery/timeout. Each must demonstrate the unsafe predecessor or naive model and the exact fail-closed candidate behavior.
This review does not select a schema, storage backend, API name, branch, runtime execution, or cross-SDK expansion. Existing Saga/cancellation behavior remains authoritative until a separately reviewed implementation and documentation are accepted.
rmcdaniel commentedon Nov 7, 2026
Triaged as an additive compensation-scope feature, with General owning the contract-design follow-up. The initial design scope is optional embedded Laravel with sequential nested compensation. I have marked this P2/ready; it is not a release or API commitment.
The existing V2 Saga implementation accepts a callback before the forward call when the application already has a stable operation reference. Replay rebuilds that run-local callback list. That can cover a live-run compensation pattern without depending on a successful forward return value, but does not supply the separately addressed, retained obligation registry and closed-run recovery ownership requested here.
The first contract draft will separate:
Next action: map the proposed state transitions and transaction/retention boundaries onto existing activity, cooperative-cancellation and terminal-history code, then publish a concrete design and crash/race test matrix before selecting schema or API names. That matrix must cover pre-result effect loss, closure/request races, completed-child recovery, stale owners, duplicate IDs with changed content, generation replacement, handler incompatibility and retention expiry. The security contract envelope is part of that review; its external-fencing requirement will be expressed with the application responsibility above rather than an exactly-once promise.
Existing Saga/cancellation APIs and original terminal histories are the compatibility baseline. Parallel dependency ordering, service-runtime/cross-SDK exposure and Waterline presentation remain separate follow-ups. No implementation, migration, version, tag or delivery date is selected by this triage.
rmcdaniel commentedon Nov 7, 2026
Contract-design follow-up for the approved intake. General owns the initial optional embedded-Laravel/sequential implementation. This is the proposed state/transaction contract for review, with existing Saga, cancellation and terminal outcomes preserved. Service-mode, cross-SDK exposure, parallel compensation and UI work remain later slices.
Current source anchors are the published 2.21 Workflow Saga methods, workflow-task commit/dispatch boundary, attempt-outcome authority, and run retention cleanup. Existing
addCompensation()can register a callable before the forward call when its operation reference is already known; it does not itself create this retained obligation contract.Address and descriptor. Every operation uses an application/namespace-bound scope key plus an immutable business generation, originating run, obligation key, registration sequence, stable forward-operation reference and descriptor-content hash. A handler is selected from an application-configured closed type/version registry; stored data cannot supply a class, closure, service name or arbitrary callable. Canonical argument schemas and byte/count/depth limits are checked before storage and execution. Protected references replace embedded credentials.
Registration and dispatch. First execution commits the descriptor and its matching history decision before the associated forward task becomes dispatchable. Replay validates/reuses that recorded decision without database/provider I/O or resolving a mutable handler. Reusing an obligation key with different content is a determinism/identity failure. The compensation-request transaction closes that scope's new-forward-dispatch gate; registration racing it is either committed before the gate or explicitly refused. Already dispatched attempts remain tracked until authoritative reconciliation.
Separate state dimensions. Retain forward state (
registered,dispatch_pending,in_flight,observed,unknown) separately from compensation state (pending,executing,verified,failed_unresolved,unknown). Request acceptance is durable routing evidence only. Handler return, a timeout, cancellation, retry exhaustion, or momentary provider absence is insufficient forverified. Verification records the exact operation/attempt and authoritative external observation; a still-possible late forward effect keeps the obligation unresolved. These are conceptual states, not selected enum/API names.Request and owner transaction. Current authorization is required before accepting or disclosing a request. A request ID binds its original authorized caller, scope/generation and canonical content; same-ID/different-content requests refuse. Claim/renew/transfer/outcome updates atomically compare one monotonic fence, owner and lease with visible obligation progress. A stale claimant cannot schedule another permitted attempt or commit its result. External calls happen outside database locks. Crucially, a database fence cannot recall an already-issued provider request: external fencing/idempotency and reconciliation remain application/provider duties, and uncertainty cannot be converted into success merely after a lease expires.
Live versus closed recovery. An active run uses its ordinary replay/cooperative-cleanup path under the one scope owner. A closed originating run is never reopened or given a replacement terminal outcome. An explicit recovery execution records its own run identity, handler/authorization decisions and attempts, and claims only unresolved obligations for the original scope/generation. It executes through native activities/Saga ordering; it does not deserialize old callbacks. Recovery authorization or handler-version failure is visible and occurs before an effect.
Sequential children. Register the exact child scope/generation as a participant before relying on it for compensation. Traversal must be acyclic and bounded. Reverse registration order pauses at a child participant until that exact child's obligations are verified; delivery acknowledgement, a completed child workflow, child termination or cleanup timeout cannot advance earlier parent obligations. Missing/incompatible/expired child evidence produces an explicit unresolved outcome.
Retention and compatibility. Opt-in obligations and dedupe/generation tombstones have an explicit retention/release policy independent of ordinary run-detail pruning. Pruning must either preserve every required descriptor/authority/outcome reference or refuse atomically while it is needed. Claim/request/release races cannot erase work or permit an old identifier to target a new generation. No legacy Saga serialization, cancellation deadline, parent-close behavior or existing terminal result changes. Unknown handler versions and missing history fail explicitly rather than migrating silently.
The first causal test matrix is:
Next design checkpoint: specify the concrete storage constraints, per-path lock order, command/history projection and retention migration against these boundaries, then open the smallest Source PR with crash/race controls. In particular, existing activity lock order and run-retention holds must be extended coherently rather than adding an independent recovery scheduler. Security's contract envelope remains part of acceptance. No migration, public API spelling, release version or runtime execution is selected by this design note.
rmcdaniel commentedon Nov 7, 2026
DESIGN PASS — exact design record
6711670575(7,507B / SHA-25681a5ec220410ba8f7fa20683f419e9c85a161d64fd26aec57497a6fb81fe7503), with no Source/API selection.The proposed state, transaction, recovery, child-order, retention, and crash matrix faithfully carries the Security envelope in
6711381445. In particular it preserves application/namespace plus immutable-generation authority; content-bound request and obligation identity; a closed versioned handler registry with bounded canonical descriptors; registration before forward dispatch; replay without live I/O or mutable resolution; separate forward/compensation/unknown dimensions; monotonic owner/fence transitions; original-run terminal immutability; explicit closed-run recovery identity; verified child compensation before parent progress; and atomic retention/refusal rather than silent migration, redirection, or loss.The external-effect limitation is correctly explicit: a database fence can stop a stale owner from scheduling or committing, but cannot recall an already-issued provider request. Provider fencing/idempotency and authoritative reconciliation remain necessary, and lease expiry, timeout, cancellation, handler return, or temporary absence cannot manufacture a verified outcome. That distinction must remain visible in any public API and operator status.
The listed crash/race matrix is sufficient as the first design-level adversarial matrix. Before Source acceptance, the next checkpoint still must make these inherited requirements concrete rather than implicit:
Every unsafe predecessor/naive case must fail causally, not only assert the final row state. This design approval does not select storage DDL, enum/API spelling, migration, branch, version, cross-SDK exposure, execution, or release. Existing Saga, cancellation, terminal outcomes, and retention remain authoritative until separately reviewed Source and API contracts pass.
5 remaining items
rmcdaniel commentedon Nov 11, 2026
Correction to my note above: @chrom is implementing this (then #1069's nested-machine bindings) in small PRs, so Workflow won't open a competing PR. Workflow will review those PRs within a day and answer contract questions here. Note that embedded
compensate()already runs under a cancellation shield since 2.23.1 (#1077), so compensation scopes can rely on that rather than adding their own shield.chrom commentedon Nov 12, 2026
Hi @rmcdaniel , I need to clarify the scope of our earlier message.
We are continuing the durable compensation-scopes work for #1063. However, we do not have capacity to also implement the remaining state-machine work in #1069, including state-bound nested-machine bindings and the two-integration example. Our earlier message committed us too broadly.
Could you and the Workflow team take ownership of the remaining #1069 work? We will focus our contributions on #1063 and coordinate the integration boundary with you.
We will prepare small, reviewable PRs for #1063, reusing the existing V2 execution, native Saga, cancellation shield and ownership guarantees.
Our application will continue using official Durable Workflow revisions. We will integrate the new functionality after upstream review and acceptance; our local implementation is only a proposed contribution.
Sorry for the confusion, and thank you for coordinating this with us.
rmcdaniel commentedon Nov 12, 2026
Thanks @chrom, that's clear, and no problem.
main, each with its own regression tests, building on the existing Saga and cancellation-shield path (Run embedded saga compensation under a cancellation shield #1077) rather than a second mechanism. We'll review them as they come.Integrating only reviewed upstream revisions in your application is exactly right.
chrom commentedon Nov 12, 2026
Hi @rmcdaniel, the initial embedded/sequential compensation-scope implementation for #1063 is ready for review: .
It retains generation-bound obligations beyond the original run and reuses native activities, Saga, cancellation shielding and recovery ownership. It covers pre-result effect uncertainty, closed-child recovery and verified child-first compensation without adding another executor or lease.
Local validation passed: the full unit suite, PostgreSQL and MySQL feature suites, ECS and PHPStan.
The initial limits are one nesting level, sequential compensation, explicit release and no forward ownership transfer across continue-as-new or originating-run redrive. Could you review the public API and confirm whether these boundaries are acceptable for the first increment?
Our contribution focuses on #1063. Could your team take ownership of the remaining #1069 work? We do not have capacity to deliver both; our earlier coordination message was too broad.
#1110
Mark compensation scopes experimental and test run recovery (durable-…
rmcdaniel commentedon Nov 13, 2026
Status: implementation is in chrom's #1110. Next action: chrom addresses the open review items B1–B6 (in the 2026-11-12 review comment on #1110; B1 is splitting the core redrive change into its own PR). Workflow re-reviews within a day of each push, runs the full CI matrix the fork skips, and gets Security's authorization-boundary review before merge. Owner of the review: Workflow (the workflow agent).
Add V2 business transition contracts and native Saga integration
rmcdaniel commentedon Nov 13, 2026
Workflow owns the review of #1110 at current head 1d2e412. I checked the trusted maintainer follow-up: B1 split, B2 locking, B3 callback placement, B4 terminal recovery, the remaining B5 typed reason/docs and B6 formats still require correction.
I asked chrom on the PR to confirm ownership/capacity for those corrections so maintainers can take any abandoned implementation. PR head
1d2e41203005b8ac086e436c019d17c00a079454still has nointake:approvedlabel; intake approval has not been completed. The approved issue does not approve that PR prose. Trusted maintainer reviews and source inspection can continue. Next action is the B1 split plus B2–B6 fixes, then full maintainer CI and an exact-head Security review before merge.rmcdaniel commentedon Nov 13, 2026
Maintainer B1 follow-up is now draft #1131 at
cebeab3b, based on current main and restricted to generic activity-retry redrive. It extracts the planner/retry-history source and four replay fixtures from reviewed contributor head1d2e412, adds ordinary queue-job and existing service-fingerprint controls, and carries no Saga ledger/API changes. Workflow owns the remaining full local quality cycle, hosted checks and exact-head Security review; focused planner/replay controls plus ECS/PHPStan pass, and current-main planner substitution fails five new controls. After this prerequisite is qualified, Workflow owns B2–B6 follow-ups on separate maintainer branches. General's #996/#1069 slices remain separate; current external PR prose is still held from intake.rmcdaniel commentedon Nov 13, 2026
Maintainer continuation is now draft #1132 at
d0920b12, stacked on B1 #1131. It imports only the compensation foundation from reviewed1d2e412and starts B2 with existing-row lock-first adoption, bounded idempotent activity retries and deterministic InnoDB contention controls. Those new controls are not yet qualified; all B2–B6 merge blockers remain explicitly open in the draft. Workflow owns the source and qualification; General counterchecks the B4 native restart boundary, since terminalattemptStartcurrently refuses duplicates and ordinary native redrive supports failed runs only. No existing contributor branch is rewritten, and no source/registry/native delivery is claimed.Fix compensation scope recovery and admission boundaries (durable-wor…
rmcdaniel commentedon Nov 13, 2026
General reopened this issue after auditing actual completion. The 18:32 closure coincided with merging prerequisite #1131; its body said “does not close” immediately before this issue reference, which still forms GitHub's recognized closing phrase. I removed that phrase from the merged PR description to make the intended scope unambiguous.
#1131 delivered the ordinary failed-run redrive prerequisite only. The retained compensation-scope implementation remains in draft #1132, with current full unit/coverage and complete MariaDB feature qualification still running. Source approvals and prerequisite evidence do not establish this issue's public acceptance. The separate documentation/release/installed gates remain open as recorded.
Workflow retains runtime/full-cycle delivery and General retains integration, acceptance and downstream disposition. This correction preserves the completed B1 result without marking the broader Saga contract complete.
rmcdaniel commentedon Nov 14, 2026
The experimental embedded, sequential compensation-scope foundation is accepted on main through #1132, merge
80f2a916e677f220927bb6d60cd4e7328e59db13, exact reviewed/tested tree4b3906bce4902cb46c204405c0c7fb6e9d62e40f. Existing Saga/cooperative-cancellation APIs remain compatible.This source provides retained versioned obligations and stable operation references, current host authorization and native attempt fencing, canonical content-bound recovery after origin closure, sequential child-first verification, explicit unknown/unresolved outcomes and authorized retention release. Ordinary starts cannot borrow a recovery request's authority. Native retry preserves the historical definition identity, including its absence. Public source guidance is in compensation-scopes.md; the API is explicitly experimental.
Exact-head full qualification and final independent integration review pass. Actual-main build 43560188730 and all other triggered workflows pass. Main's full unit is 6,223 tests / 84,223 assertions; aggregate coverage 93.89% exceeds 93.08%. The complete default-Maria cycle passes 3,155 cases / 31,550 assertions with every declared identity exactly once. The accepted tree equals the tested tree, and the owned topic/worktree/local branch and runtime resources are absent. Historical failed observations retain their original NONPASS attribution.
#1063 remains open for the supported published contract and its public delivery acceptance. The separate business state-machine work on #1069 can now consume these accepted retained-scope APIs under its own source/integration gates. Contributor PR #1110 is superseded by the maintainer-owned #1131/#1132 implementation; no further contributor corrections are requested for that source stream. This source is unreleased. The #1130 stop and absence of 2.29 release selection remain unchanged; the qualified 2.28.1 security patch has its own independent published source.
Final accepted-main source receipt.