Skip to content

Bound Agent Workflow tracking so that local state is automatically pruned - #1935

Draft
ericclemmons wants to merge 7 commits into
mainfrom
codex/workflow-tracking-retention
Draft

Bound Agent Workflow tracking so that local state is automatically pruned#1935
ericclemmons wants to merge 7 commits into
mainfrom
codex/workflow-tracking-retention

Conversation

@ericclemmons

@ericclemmons ericclemmons commented Jul 13, 2026

Copy link
Copy Markdown
Member

tl;dr – Automatically prune terminal Agent Workflow tracking rows from local SQLite.

Agent Workflow tracking currently grows without bound even after the Workflow platform expires instance state. This persists normalized success/error retention, derives an indexed expires_at after the terminal outcome is known, and maintains one earliest-expiry cleanup alarm per Agent or facet owner.

Cleanup is armed when runWorkflow() first writes tracking state, reconciles missed terminal callbacks after eviction or crashes, deletes expired local rows, and stops when no cleanup remains. Omitted retention uses a bounded 30-day local fallback because the SDK cannot detect Workers Free versus Paid; this improves the existing unbounded behavior.

This deletes only Agent-local SQLite tracking. It never deletes Workflow platform instances, state, or logs.


References: retention passthrough PR, Agents cleanup strategy

@changeset-bot

changeset-bot Bot commented Jul 13, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: ebc399c

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
agents Patch
@cloudflare/agent-think Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new

pkg-pr-new Bot commented Jul 13, 2026

Copy link
Copy Markdown

Open in StackBlitz

agents

npm i https://pkg.pr.new/agents@1935

@cloudflare/ai-chat

npm i https://pkg.pr.new/@cloudflare/ai-chat@1935

@cloudflare/codemode

npm i https://pkg.pr.new/@cloudflare/codemode@1935

hono-agents

npm i https://pkg.pr.new/hono-agents@1935

@cloudflare/shell

npm i https://pkg.pr.new/@cloudflare/shell@1935

@cloudflare/think

npm i https://pkg.pr.new/@cloudflare/think@1935

@cloudflare/voice

npm i https://pkg.pr.new/@cloudflare/voice@1935

@cloudflare/worker-bundler

npm i https://pkg.pr.new/@cloudflare/worker-bundler@1935

commit: ebc399c

@ericclemmons ericclemmons self-assigned this Jul 13, 2026
@ericclemmons

Copy link
Copy Markdown
Member Author

/bonk review

@ask-bonk

ask-bonk Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

UnknownError: ProviderInitError

github run

@ask-bonk

ask-bonk Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

@ericclemmons Bonk workflow failed. Check the logs for details.

View workflow run · To retry, trigger Bonk again.

@ericclemmons
ericclemmons force-pushed the codex/workflow-tracking-retention branch from 430722e to cd9cf26 Compare July 27, 2026 22:46
@ericclemmons
ericclemmons changed the base branch from main to codex/workflow-retention-tracking July 27, 2026 22:46
@ericclemmons
ericclemmons force-pushed the codex/workflow-retention-tracking branch from cedeedc to 05181d6 Compare August 14, 2026 14:46
@ericclemmons
ericclemmons force-pushed the codex/workflow-tracking-retention branch from cd9cf26 to 7a6d226 Compare August 14, 2026 14:46
@ericclemmons
ericclemmons force-pushed the codex/workflow-retention-tracking branch from 05181d6 to 4884a81 Compare August 14, 2026 17:56
@ericclemmons
ericclemmons force-pushed the codex/workflow-tracking-retention branch from 7a6d226 to b48a179 Compare August 14, 2026 17:56
@ericclemmons
ericclemmons force-pushed the codex/workflow-retention-tracking branch from 45eeb85 to 6ea30dd Compare August 14, 2026 19:28
@ericclemmons
ericclemmons force-pushed the codex/workflow-tracking-retention branch from b48a179 to ba7dcb6 Compare August 14, 2026 19:28
Base automatically changed from codex/workflow-retention-tracking to main August 14, 2026 20:22
@ericclemmons
ericclemmons force-pushed the codex/workflow-tracking-retention branch from ba7dcb6 to 0eb5018 Compare August 14, 2026 20:27
@ericclemmons

Copy link
Copy Markdown
Member Author

/bonk review this plz

completed_at = ${completedAt}
completed_at = ${completedAt},
expires_at = ${expiresAt}
WHERE workflow_id = ${workflowId}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_updateWorkflowTracking unconditionally overwrites completed_at and expires_at for every status:

const completedAt = completedStatuses.includes(statusName) ? now : null;
const expiresAt = completedAt ? ... : null;
// ...
SET completed_at = ${completedAt}, expires_at = ${expiresAt}

If reconciliation polls a still-running instance (status: "running"), the UPDATE blankets completed_at = NULL, expires_at = NULL over whatever was there. This is fine for genuinely non-terminal rows, but if a race delivers a terminal callback between the workflow.get() call in _cleanupWorkflowTracking and this UPDATE, the already-persisted expiry is silently erased and the row reverts to unbounded — the exact state the PR aims to eliminate.

Consider a WHERE guard like the one onWorkflowCallback already uses (AND status NOT IN ('terminated', 'paused')), or at minimum skip writing completed_at/expires_at when the new status is non-terminal and the existing row is already terminal.

this.sql`
DELETE FROM cf_agents_workflows
WHERE expires_at IS NOT NULL AND expires_at <= ${now}
`;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The expired-row deletion at L6056-6059 already runs before reconciliation. This second identical DELETE at the bottom of the method is redundant — reconciliation calls _updateWorkflowTracking which sets expires_at but never sets it into the past, so no new expired rows can appear between the two DELETE passes in the same invocation.

@ask-bonk

ask-bonk Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

The review was submitted with 2 inline comments. Here is the summary:


Submitted 2 inline findings on the PR. Summary of additional observations that didn't warrant inline comments:

Overall assessment: The design is sound — deriving expires_at at terminal-outcome time, indexing it, and using the existing alarm infrastructure for cleanup is the right shape. The reconciliation loop for missed callbacks is a solid recovery path. The facet integration correctly delegates alarm ownership to the root Agent. Tests cover the key scenarios well.

Minor notes not posted inline (non-blocking):

  • The _nextWorkflowCleanupAt() query (packages/agents/src/index.ts:6775) is duplicated verbatim inside _scheduleNextAlarm() (packages/agents/src/index.ts:6720). Consider extracting it so the MIN logic lives in one place.

  • normalizeWorkflowRetention in workflow-retention.ts accepts a WorkflowRetentionDuration which is string | number. When a number is passed, itty-time's ms() treats it as milliseconds and returns it unchanged, so Math.ceil(durationMs / 1000) converts ms→seconds. The test "rounds numeric retention up from milliseconds" confirms this, but the Workflows API docs describe the numeric form as milliseconds too, so there is no semantic mismatch — just worth noting that local and platform retention share the same input units by construction.

  • The new itty-time dependency is pinned to 2.0.1 (not a range). This is fine for reproducibility.

github run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant