Repository navigation
Extend Durable Workflow with composable business state machines, pluggable transition policies, and nested execution #1069
Description
Primary owning public component
Workflow engine
User problem
We need a supported business-state-machine layer built on Durable Workflow's existing execution capabilities.
Our application processes orders across multiple sales-channel and carrier integrations. The meaning of the business states is shared, but each integration can require different allowed transitions, conditions, actions, confirmation evidence, and compensation behavior.
For example, two orders can use the same Order state machine while selecting different integration-specific implementations of a transition. One integration may require an external operation before the transition becomes possible; another may already provide the required evidence and need no action.
A state can also host a nested machine. An Order may remain in DeliveryInProgress while a Shipment machine performs its own transitions. The parent needs the child's verified business outcome and must coordinate cancellation and compensation without losing the child's identity after retries, state changes, or worker restarts.
DW already provides the underlying primitives: workflow execution, activities, durable waits, child workflows, replay, retries, versioning, and Saga compensation. We currently have to assemble their relationship with business states and transition policies ourselves.
This request is to extend DW with a reusable public contract and execution layer for these scenarios, preserving the existing runtime and APIs.
Proposed public contract
Add an optional business-state-machine capability integrated with Workflow V2. Extend existing state-machine support and reuse DW execution, history, waits, retries, and native Saga.
The names below illustrate the requested contract; they are not claims about existing APIs.
- Shared machine definition and replaceable transition policies
A machine definition declares:
- Stable machine identity and definition version.
- State identifiers and their shared meaning.
- Transition identifiers and machine-wide invariants.
- Named slots for state-bound nested machines.
A versioned policy profile supplies:
- Allowed edges within that machine's state catalogue.
- Sets of target-state conditions.
- Conditions permitting execution to start.
- One or more actions, including nested-machine operations.
- Required result evidence and compensation behavior.
- Waiting, failure, and retry-policy references.
Select the profile from trusted application context, such as the instance's integration or configuration. Different instances of the same machine can select different profiles without duplicating the runner or redefining state meanings.
Profiles cannot bypass machine-wide invariants. Configuration references registered rule/action types with validated parameters, rather than arbitrary executable code.
Pin the selected definition, profile, rules, and compensation versions. Deployment or configuration changes must not silently replace them for an active execution. Any supported migration must be explicit and preserve existing compensation obligations.
- One transition-step contract for all machine types
Preserve four business operations:
canTransition(TransitionContext $context): bool
canExecute(TransitionContext $context): bool
execute(TransitionContext $context): ExecutionReceipt
compensate(
TransitionContext $context,
ExecutionReference $originalExecution
): void
The contract must not depend on Order, Shipment, a particular integration, or an ORM model.
execute() may perform ordinary actions, start/adopt a nested machine, or durably await its result. Suspension and recovery belong to DW.
- Defined execution sequence
Event
-> resolve the instance's pinned policy
-> select a structurally applicable transition
-> [1] evaluate target-state conditions
satisfied:
skip unnecessary actions and proceed to [4]
unsatisfied:
-> evaluate execution-start permission
-> if permitted, establish execution identity and compensation coverage
-> [2] execute prescribed work
-> obtain a fresh recorded fact snapshot
-> [3] evaluate target-state conditions again
-> [4] perform a protected, idempotent business-state commit
Unavailable transitions, prohibited work, missing evidence, and unknown external outcomes must remain distinguishable. A false check must not automatically authorize an action, retry, or compensation.
When execution is unnecessary, preserve existing obligations or explicitly establish adoption responsibility for existing effects.
- Pure, explainable rule evaluation
Conditions operate on immutable, recorded facts. Evaluation exposes:
- Satisfied, unsatisfied, or unknown.
- Stable reason codes.
- Evidence references and missing facts.
- Rule/profile version and fact-snapshot identity.
- The evaluated condition structure.
The boolean methods bridge this typed evaluation; a mutable shared "last result" cache is not an acceptable result channel.
Fresh database/provider reads occur through recorded execution boundaries. Replay reuses recorded inputs. Guards do not perform I/O, dispatch actions, or wait.
If several transitions match an event, selection must follow a declared deterministic policy or report ambiguity.
- State-bound nested machines
Allow any declared parent state to host a nested machine using the same machine and transition contracts.
A durable binding records:
- Authorized application/client scope.
- Parent business instance and original owning execution.
- Binding slot and immutable generation.
- Hosting parent state.
- Child machine and business-execution identity.
- Pinned parent/child policies and required outcome.
Separate starting/adopting the child from awaiting its outcome.
For example, a transition into DeliveryInProgress can confirm the Shipment start/adoption milestone before committing the parent state. A later transition can await the same child's delivery result.
A queued request alone is not confirmed start. Retries reuse the original binding. Observation does not create another child or acquire a second compensation obligation.
Leaving the hosting state does not implicitly terminate the child or erase responsibility. The parent evaluates the child's concrete business outcome; a technically completed child run is not sufficient evidence.
- Compensation through native Saga
Compensation is a separate branch, invoked by an explicit business decision.
Identify the original execution and establish compensation coverage before external effects. Compensation must work even when execute() has not returned a receipt.
The original owning step resolves its effects; observer steps do not duplicate its undo. One step may own multiple effects, with explicit operation identity and ordering.
In sequential nested compensation:
- Request compensation of the exact bound child.
- Wait for verified resolution of its required obligations.
- Then continue earlier parent compensations.
compensate(): void succeeds only after required effects are resolved or their absence is authoritatively established. Pending, unknown, refused, and technical-failure outcomes require distinct handling.
Integrate with #1063 for retained compensation scopes and recovery after originating runs close. Preserve terminal history and original business identity.
- Cancellation, retries, and external uncertainty
Accepting business cancellation prevents new ordinary forward work and competing completion commits. Already dispatched operations remain tracked and reconciled.
Use existing DW activities, timers, retry facilities, and Saga ordering. Retry the specific safe failed operation rather than restarting an entire transition or compensation sequence.
Provide integration points for operation budgets, provider backoff, and circuit-breaker decisions. Waiting for permission is not successful execution.
Technical cancellation or timeout must not imply business cancellation success. Multi-day business waits require explicit policies independent of bounded technical cleanup.
- Persistence and inspection boundaries
Provide a persistence integration contract for:
- Expected source state and revision.
- Transition/decision identity.
- Authority and cancellation checks.
- Atomic business state, decision result, and publication intent where they share a store.
- Idempotent recovery after lost commit acknowledgement.
Cross-store execution requires an explicit durable handoff; no distributed atomicity is assumed.
Expose structured inspection data for the current business state, selected profile, blocked transition, reasons, pending work, child binding, and compensation outcome.
Business state remains distinct from workflow-run status. Application-specific states, authorization decisions, provider idempotency, and evidence validation remain application responsibilities.
Initial scope: embedded Laravel, single-state machines, and sequential nested compensation. Preserve existing workflows and avoid a separate scheduler, retry engine, or compensation executor.
Acceptance criteria
- Two integrations use the same machine and state meanings with different conditions, action sequences, allowed edges, and compensation handlers.
- Profile changes affect only explicitly selected new executions or an authorized migration; replay and outstanding compensation retain their pinned versions.
- The same four-method transition contract supports parent and child machines without entity-specific dependencies.
- Already-satisfied target conditions skip unnecessary actions.
- Execution-start refusal prevents effects; post-action checks use a fresh recorded snapshot.
- Undefined/source-ineligible transitions, business prohibition, missing evidence, and unknown outcomes produce distinguishable decisions.
- Multiple matching transitions follow deterministic selection or return an explicit ambiguity.
- A transition can execute several actions and compensate partial work when no final receipt exists.
- Nested machines can be bound to different parent states through configuration without special-casing Order or Shipment.
- Starting/adopting and awaiting a child are separate operations using the same immutable binding generation.
- Worker restart, duplicate events, and lost start acknowledgements do not create duplicate logical children or compensation owners.
- Cancellation while awaiting a child resolves required child compensation before earlier parent compensation.
- Closed-child and closed-parent recovery use the retained-scope guarantees of Support durable Saga compensation scopes across nested workflow lifecycles #1063; technical termination alone never satisfies business completion.
- Lost, delayed, or uncertain external results are reconciled before unsafe retries or successful compensation acknowledgement.
- Failed compensation preserves completed evidence and unresolved obligations; exhausted retries cannot produce a false business Cancelled state.
- Cancellation-versus-completion races and lost commit acknowledgements preserve one authoritative business decision.
- Replay does not perform unrecorded provider calls, mutable configuration reads, or live guard I/O.
- Cross-client binding substitution and stale-generation requests are rejected before protected access or effects.
- Invalid definitions, incompatible handler versions, cyclic child bindings, and configured resource limits produce explicit outcomes.
- Diagnostics explain blocked transitions and correlate business execution, child work, and compensation.
- Automated crash/race tests and a runnable two-integration example demonstrate the contract using existing DW execution facilities.
- Existing workflow, Saga, and cancellation APIs remain backward compatible.
Supporting public evidence
The requested capability builds on existing DW primitives:
-
Child Workflows documents durable parent/child execution, stable invocation identity, child handles, and outcome propagation.
-
Condition Waits documents replay-safe waiting on recorded workflow state and requires waits to execute in workflow context.
-
Versioning documents compatibility mechanisms for long-running executions. The proposed machine/profile version contract applies these guarantees to configurable business behavior.
-
Sagas documents native compensation registration and ordering. This feature should compose business transitions with that existing mechanism.
-
Cooperative cancellation documents bounded cleanup and operation policies. Business-state cancellation must preserve the distinction between technical run control and verified business resolution.
-
Issue Support durable Saga compensation scopes across nested workflow lifecycles #1063 covers retained compensation obligations and closed-run recovery. This feature should consume that contract rather than introduce a competing compensation subsystem.
These sources establish the existing execution foundation. The requested addition is a supported, composable business-state-machine API connecting those capabilities.
Dependencies and affected repositories
Primary repository:
- durable-workflow/workflow
Extend existing state-machine support with the public machine/profile/transition contracts, V2 execution integration, nested bindings, persistence boundaries, inspection data, and regression tests.
Related dependency:
- Support durable Saga compensation scopes across nested workflow lifecycles #1063 — retained Saga compensation scopes across nested workflow lifecycles.
Reuse its identity, ownership, verification, retention, and closed-run recovery guarantees. Scenarios depending on those capabilities must not be marked complete before that integration is verified.
Documentation:
- durable-workflow/durable-workflow.github.io
Document authoring contracts, profile selection/versioning, nested-machine behavior, compensation, persistence guarantees, and runnable examples.
Optional presentation follow-up:
- durable-workflow/waterline
Display business states, blocked-transition reasons, selected profiles, nested bindings, and compensation progress using the core inspection contract.
Initial compatibility boundary:
- Embedded Laravel and Workflow V2.
- Existing workflows remain usable without adopting the new state-machine layer.
- No mandatory Symfony Workflow dependency. Existing graph libraries may integrate through a defined boundary.
- Service-mode and cross-SDK support may follow separately.
- No application-specific Order/Shipment models, integration names, or business status catalogue belong in DW core.
Public intake checks
- I searched open and closed GitHub issues for an existing request.
- This request contains only public product contracts and public-safe rationale.
Activity
rmcdaniel commentedon Nov 8, 2026
Intake approved and reviewed. Maintainer triage: P2 feature, ready for contract design; owner General. This is separate from the current release work. The proposed API names remain proposals until the public contract is reviewed.
The published 2.22.0 source provides graph-edge checking in StateMachine and model-state configuration in StateConfig. The existing state-machine feature test exercises WorkflowStub. Those surfaces do not establish the additional pinned-policy, business-evidence, nested-binding and protected-commit guarantees requested here. The closed state-machine work (#333/#334) therefore does not resolve this broader request.
The first deliverable is a reviewable contract and execution protocol for embedded Laravel/V2, single-state machines and sequential nested compensation, with a two-integration worked example and an acceptance-to-test map. It will settle these boundaries before implementation:
Dependency: #1063 remains open. Retained obligations, original ownership and closed-child/closed-parent recovery must consume that contract and cannot be accepted here before their integration is verified. Definition/rule/adapter design can proceed without claiming those recovery guarantees already exist.
The implementation plan will preserve existing state-machine, workflow, Saga and cancellation APIs, avoid application-specific entities and a second scheduler/retry/compensation engine, and keep service-mode/cross-SDK and optional Waterline presentation outside the initial acceptance. Public documentation follows the reviewed core contract. No version, delivery date or new runtime behavior is committed by intake approval.
Next action — General: produce that contract/sequence and crash-race test plan after the active release qualification work, then open a draft implementation PR for the agreed first slice. This issue remains the acceptance record; its checkboxes stay unchecked until corresponding evidence exists.
chrom commentedon Nov 9, 2026
@rmcdaniel pls check MR #1070
rmcdaniel commentedon Nov 9, 2026
Intake approved by the operator for the first slice, PR #1070. General reviewed it: the design is sound, and I pushed boundary/shape-validation/experimental-marker fixes so the full matrix can run. This slice doesn't complete the proposal: state-bound child bindings, durable compensation scopes (#1063) and the two-integration example remain, as the PR's design notes say.
rmcdaniel commentedon Nov 9, 2026
Main after merge
0673e4dc: all 18 test jobs passed in push run 42670327804 (MySQL ×4, PostgreSQL ×4, MariaDB, coverage, coverage-unit, both Laravel embedded-upgrade cells, quality, regression corpus, tracing). The final aggregatebuildand Sonar jobs were never created, because GitHub has an ongoing major Actions/Checks incident (since 18:24 UTC). A fresh main run will be dispatched once Actions recovers.rmcdaniel commentedon Nov 9, 2026
Main
0673e4dcis green: run 42676776260 attempt 2 succeeded, including the aggregatebuildand the Sonar quality gate. Attempt 1 ran right after GitHub's Actions incident and had two time-limit stops (MySQL shard 0, PostgreSQL shard 2 at 96%) and one timing-sensitive watchdog test; all of them pass on rerun, and none touch the StateMachine code.8 remaining items
chrom commentedon Nov 12, 2026
Hi @rmcdaniel, the follow-up is in
1d2e412.Changes addressing your review conditions:
Workflow\V2\Compensationas@experimentaland documented the stability exception indocs/api-stability.md. Internal types remain internal.46fd88b1, including Hand open children to the continued run at continue-as-new (#868) #1104, Retry a handed-over child's failure under its recorded retry policy #1107 and Redrive failed runs whose completed steps include a child workflow #1109.ScopeRecovery, child-first compensation and preservation of the original terminal history. Forward scope ownership stays with the original run.The core redrive change is still included in this PR. Its isolated patch was also tested against main without the Compensation namespace: 20 tests / 282 assertions passed, providing evidence for a possible split.
Completed local validation:
All final runs finished with zero failures/errors. Unit coverage is 79.91%, above the 60% requirement. ECS, PHPStan, all 70 corpus-policy tests and the enforced corpus validator passed.
Validation used Laravel 13.40 and PHP 8.4. The final MySQL run used a clean isolated tmpfs database with the repository's CI settings. The interrupted preliminary MySQL run is not counted above. The unit suite reported 2 PHPUnit deprecations and 15 notices.
The agreed first-increment limits, native Saga and cancellation shielding are unchanged. These are local results; upstream CI and security review remain separate.
rmcdaniel commentedon Nov 12, 2026
Thanks @chrom. I've replied on #1110 with where each review item stands at
1d2e412. Let's keep the compensation-scope discussion on the PR.rmcdaniel commentedon Nov 13, 2026
Status: blocked on #1110 (compensation scopes, #1063). Clearing action: #1110 merges after its remaining review items (B1–B6, listed in the 2026-11-12 review comment on #1110). After that, the Workflow team builds the rest of this issue on top of the scope foundation: the state-bound nested-machine bindings and the two-integration example. Owner: Workflow (the workflow agent). I'm following the #1110 review thread and will re-review within a day of each push.
Add V2 business transition contracts and native Saga integration
rmcdaniel commentedon Nov 13, 2026
Maintainers are driving the remaining implementation. Contributor availability is not a prerequisite for continuing this issue.
The transition foundation already landed in #1071 and is present on current main: pinned
PolicyProfile, the four-methodTransitionStep, recorded snapshots/admission/commit throughActivityTransitionRuntime, and native Saga registration before work. The existing tests cover two profiles, skipped work, partial-effect compensation, decision refusal, stale evidence and cold replay. Those controlled activity responses do not establish the full application or nested-machine contract.The remaining work stays in this issue:
General has finished the focused #996 authority countercheck and the B4 native recovery correction; Workflow owns their remaining full quality and landing gates. The native start milestone is now concrete in draft #1135; General retains exact authority/history review, the full quality cycle and the subsequent state-bound binding contract with persisted worker controls. A start/adopt milestone must confirm the exact durable child identity before the parent commits its hosting state. Later observation/await must reuse that binding and add no second compensation owner. The existing
child()terminal wait does not establish that separate milestone. This issue remains open until the original acceptance criteria are met; the transition foundation does not complete it.rmcdaniel commentedon Nov 14, 2026
The confirmed native child start/separate-await increment is now source-delivered in #1135, merge63991eca with exact reviewed tree and all four actual-main workflows passing. The normal main build passed the complete unit suite and both full database matrices; combined unit/MySQL coverage is93.86% against93.08%.
General remains owner of the next immutable business-binding increment: authorized application scope, original parent/business execution, named slot and immutable generation, hosting state, exact child receipt, pinned policies and required business outcome. Start/adopt must be distinct from observe/await, and duplicate/lost-acknowledgement recovery must reuse the original generation. Leaving/re-entering a state must not redirect old requests or add another compensation owner. I have re-read the existing #1071 transition foundation and am building on it, preserving its four methods.
The accepted compensation API from #1063/#1132 is still required for retained ownership and verified child-first recovery integration. Its dependency does not stop independent binding-contract preparation. The runnable two-integration persistence/crash/race example remains required afterward. This issue stays open and in progress; the new primitive is experimental on main, not a stable package or full business-layer delivery.
rmcdaniel commentedon Nov 14, 2026
General's immutable binding value checkpoint is now concrete at
e352683805ab92a685da9751570b5b992ed6e8f9ongeneral/1069-state-bound-binding-20261114, based on accepted63991eca. This is source preparation, not a complete binding protocol or merge-ready increment.BindingIdentityexplicitly versions the original authorized application/business execution, native namespace/parent instance/run, named slot, immutable generation, hosting state, child type, parent/child policy fingerprints, input fingerprint, required business outcome and native child options. The original-generation lookup key is separate from the complete immutable descriptor fingerprint, so an existing key cannot silently adopt changed policy/input/outcome/state.ChildBindingpairs it with the exact confirmed native admission and rejects another namespace, parent owner or workflow type. Decoding rejects unknown versions/fields and wrong types; fixed version1 lookup/descriptor vectors guard persisted format stability.The three exact source/test files passed ECS and30 focused pure-value tests/45assertions/zero warnings on PHP8.4.27/PHPUnit13.4.6. Reflection bound the two classes to the mounted source. Formatter changes were copied back only to these new files. Both captured QA IDs and exact label sets are independently absent. Full quality, corpus, native/persistence/crash/race acceptance and source counter-review have not run for this checkpoint.
Next, General is connecting the values to recorded application activities and actual persistence, with exact-generation reservation/confirmation/loading and outcome assessment. Ownership must be registered through the accepted retained-scope API before native child dispatch; observation adds no compensation owner. The current scope foundation remains in1132, so retained ownership/recovery integration follows its qualified API. Native admission replay must reuse a lost acknowledgement, and stale generation/state re-entry must not redirect an old request. Cross-parent-run use of a StartedChild receipt remains refused. The two-integration application proof and all original1069 requirements remain pending; this issue stays open.
rmcdaniel commentedon Nov 14, 2026
General selects an additive native admission/closure contract for this integration, following the accepted-source readback of the pre-admission compensation window.
At accepted Saga source80f2, participating in a future child scope before dispatch is conservative: a child scope that never opened remains unresolved, and earlier parent compensation cannot advance. The existing missing-child control correctly protects that boundary. Lookup absence, technical cancellation or an invented empty ledger cannot prove permanent non-admission.
For #1069, the engine needs an authoritative fence/receipt for the exact reserved native child identity, original parent owner and immutable binding generation. Closing a never-admitted reservation must serialize with every admission route for that reserved child. It may return a durable never-admitted receipt only when admission is permanently closed and no child was admitted. If admission won, return the exact original admission receipt; a lost reply must recover it without creating another logical child. Late or retried work cannot cross a closed fence. Authority, namespace/parent ownership, source identity and generation must be checked before protected access or mutation.
Ownership: Workflow owns the additive native engine primitive and its crash/concurrency controls under the existing #1063/#1069 source handoff. General retains the StateMachine binding persistence, policy/authorization adapter, business-outcome and compensation integration. The primitive must supply evidence the sole original compensation owner can consume; it must not silently mark a missing child scope verified or clear an existing
dw.child-scopeparticipant. Existing child APIs preserve their current behavior, and the new binding path must use the existing native admission/locking machinery.The concrete source deliverable is one focused draft with the proposed API/receipt schema and causal controls for close-before-admit, admit-before-close, lost acknowledgement, late/duplicate workers and cross-owner/stale-generation refusal. An exact source review precedes adoption into General's binding branch. Normal source QA is in scope; no release, #1130 qualifier, 2.29 selection or held SDK/runtime/collector path is selected. The full #1069 contract remains open through integrated real persistence/recovery/race tests and public delivery.
rmcdaniel commentedon Nov 14, 2026
General ownership correction after the explicit delivery check: Workflow has not accepted or started the additional native reservation/admission-close implementation proposed in 6872718293. Its accepted #1132 delivery and missing-child semantic readback are complete; they do not establish ownership or progress on this new primitive. No draft, source, QA or runtime exists for it yet.
General retains the additive native primitive together with StateMachine binding persistence, authorization/policy, business outcomes and Saga integration. The contract and causal acceptance criteria in the earlier decision remain proposed work, not implemented behavior. General will prepare the concrete source/API change and review it under the existing repository gates. The accepted missing-child conservative behavior remains unchanged; lookup absence or an invented empty scope still cannot settle compensation. No release, #1130, 2.29, held SDK/runtime/collector or operator boundary changes.