You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Close activity attempts without locking the attempt after task, execution and run #1000
Lease renewal (DefaultActivityTaskBridge::observeAttempt) locks rows in the order attempt → execution → run → task.
ActivityOutcomeRecorder::record() takes the opposite order on its ordinary path: task, then execution, then run. On the stale, cancelled and terminated branches it then calls closeAttempt(), which runs ActivityAttempt::query()->lockForUpdate()->find(...). If a renewal of the same attempt is in flight, the renewal holds the attempt and waits for the execution, while the outcome holds the execution and waits for the attempt. PostgreSQL and MySQL resolve that by aborting one transaction with a deadlock error.
How reachable it is: since dec2aff, refusedExpiredAttempt() refuses most expired or settled callbacks under the run lock before taking any task or execution lock. So the ordinary stale branch is mostly reached in races where the attempt is still Running with an unexpired lease when the fence reads it. The code path is still live, and the cancelled/terminated branch closes attempts the same way.
Found during Security's review of #901, which removed the same reversal from the new audit append (ActivityOutcomeRejection::record() now reads the attempt without a lock). This issue covers the pre-existing closeAttempt() lock, which #901 does not change.
Acceptance criteria
No outcome path requests the attempt row lock after holding the task, execution or run lock. Either lock the attempt first, in renewal order, or close it with a guarded conditional update that needs no prior row lock.
A deterministic PostgreSQL and MySQL interleaving (same process harness as ActivityOutcomeRetentionTest::testLeaseRenewalAndLateOutcomeAuditCannotDeadlock in Record bounded diagnostic history for rejected late activity outcomes #901) covers renewal against the stale and cancelled outcome branches. It fails with a deadlock on the current code.
Owner: Workflow, with Security reviewing the existing claim/outcome fencing boundary. Current ActivityOutcomeRecorder::closeAttempt still takes lockForUpdate after the task/execution/run locks, so the issue's reverse-order site remains live.
Next source action is the deterministic renewal-versus-stale/cancelled outcome interleaving on PostgreSQL and MySQL, preserving the current expired-attempt fence. Prefer a guarded conditional terminal update if it can prove the exact attempt/current authority and unchanged stale-write behavior; otherwise adopt one canonical lock order across both paths. I have separated this from #1030's pre-effect profile reads, which do not solve the deadlock. No lease extension or new capability is inferred.
#1030 was closed as superseded by #1076, so this issue no longer waits on it. The next source action recorded above is unchanged. Owner: Workflow. It's queued behind the open defect PRs #1077–#1083.
Fixed by #1102 (General, 2026-11-12). closeAttempt() is now a guarded conditional update (WHERE status = 'running', closed_at = COALESCE(closed_at, now)) with no attempt row lock, so an outcome never requests the attempt lock after holding the task, execution and run. That removes the cycle with lease renewal. The new test asserts the lock order on the PR's PostgreSQL feature suite, and all six shards passed. It ships in the next Workflow release.
Activity
rmcdaniel commentedon Nov 4, 2026
Owner: Workflow, with Security reviewing the existing claim/outcome fencing boundary. Current ActivityOutcomeRecorder::closeAttempt still takes lockForUpdate after the task/execution/run locks, so the issue's reverse-order site remains live.
Next source action is the deterministic renewal-versus-stale/cancelled outcome interleaving on PostgreSQL and MySQL, preserving the current expired-attempt fence. Prefer a guarded conditional terminal update if it can prove the exact attempt/current authority and unchanged stale-write behavior; otherwise adopt one canonical lock order across both paths. I have separated this from #1030's pre-effect profile reads, which do not solve the deadlock. No lease extension or new capability is inferred.
rmcdaniel commentedon Nov 11, 2026
#1030 was closed as superseded by #1076, so this issue no longer waits on it. The next source action recorded above is unchanged. Owner: Workflow. It's queued behind the open defect PRs #1077–#1083.
Close activity attempts without locking them after the task, executio…
rmcdaniel commentedon Nov 12, 2026
Fixed by #1102 (General, 2026-11-12).
closeAttempt()is now a guarded conditional update (WHERE status = 'running',closed_at = COALESCE(closed_at, now)) with no attempt row lock, so an outcome never requests the attempt lock after holding the task, execution and run. That removes the cycle with lease renewal. The new test asserts the lock order on the PR's PostgreSQL feature suite, and all six shards passed. It ships in the next Workflow release.