feat(dynaflow): DynamoDB checkpointer for run/thread state (#280)

[shivamm-gupta]

Description

Closes #280

Implements persistent run and thread state checkpointing for agent orchestration (dynaflow pillar), enabling long-running and human-in-the-loop workflows to survive restarts and support deterministic time-travel resumption.

Changes

1. Core Checkpointing Abstractions:

   • Introduced Checkpoint data model with thread_id, checkpoint_id, parent_checkpoint_id lineage tracking, execution step numbers, node identifiers, state dictionaries, metadata, and ISO timestamps.
   • Defined BaseCheckpointer abstract contract providing save(), get(), list(), and convenient put() operations.

2. In-Memory Checkpointer:

   • Implemented MemoryCheckpointer with thread-safe lock management for local development, rapid unit testing, and ephemeral agent executions.

3. DynamoDB-Backed Checkpointer:

   • Implemented DynamoDBCheckpointer aligned with dynavec's single-table schema:
     • Checkpoint records: pk = "{namespace}#checkpoint#{thread_id}#{checkpoint_id}"
     • Thread manifest records: pk = "{namespace}#thread#{thread_id}" maintaining pointers to latest checkpoint and chronological history.
   • Handles lossless conversion of complex Python objects, dictionaries, lists, and numbers using _to_dynamo and _from_dynamo.

4. Engine and Agent Integration:

   • Wired BaseCheckpointer into ReActAgent (__init__, run, arun).
   • Automatically writes state checkpoints after each reasoning step and tool invocation.
   • Added thread resumption (resume_from or existing thread_id) restoring messages, prior steps, and tool execution counts.

Verification

• Added 6 comprehensive tests in tests/test_checkpointer.py covering:
  • Checkpoint serialization round-trips
  • MemoryCheckpointer CRUD and time-travel lookups
  • Moto-based AWS DynamoDB table tests for save/get/list and thread isolation
  • ReActAgent step-by-step checkpointing and interrupted thread resumption
• Verified all existing agent tests pass: pytest tests/test_agents.py tests/test_checkpointer.py (14 passed).
• Verified clean formatting and linting via ruff check and ruff format.

[codeforstartups]

Thanks @shivamm-gupta — reviewed, and the checkpointer design (closes #280) looks great. It needs a quick rebase onto latest development before I can merge, because two PRs landed under it today:

• feat(agents): add tool registry and built-in connectors #328 (tool registry) refactored agents/react.py: the monolithic run/arun were split into helpers — _init_messages, _dispatch_tool_call / _adispatch_tool_call, _process_tool_calls / _aprocess_tool_calls, and _finalize_result. Your thread_id/resume_from checkpoint logic was written against the old single-function loop, so the 4 conflicts are all in react.py where your hooks meet the new helpers.
• feat(dynalogs): structured logs per-run / per-node with secret redaction (#300) #333 (structured logging) added an export block to src/dynavec/__init__.py, which collides with your checkpoint exports (1 trivial conflict — keep both blocks).

To rebase:

git fetch upstream && git rebase upstream/development
# resolve: re-apply the checkpoint save/load around the new helper calls in run()/arun()
uv run --no-sync mypy && uv run --no-sync pytest -q
git push --force-with-lease

The react.py resolution is the real one: keep your thread_id/resume_from params and checkpoint-at-each-step behavior, but call the extracted helpers (_process_tool_calls, _finalize_result) instead of the old inline code. I didn't want to rewrite your resume semantics against a refactor you hadn't seen, hence the ask. Ping me once pushed and I'll re-review + merge promptly. 🙏

[shivamm-gupta]

Rebased onto latest \upstream/development! Integrated the checkpoint save/load logic around the new _process_tool_calls\ and _finalize_result\ helpers in both
un()\ and \�run(). All unit tests and ruff lint checks passing cleanly.