feat(dynaflow): conversation / short-term memory store on DynamoDB (#290)

[shivamm-gupta]

Description

Closes #290

Implements short-term conversation memory and scratchpad persistence on Amazon DynamoDB under the dynaflow pillar, complementing dynavec's native long-term semantic vector memory.

Changes

1. Memory Interface and Policies:

   • Defined BaseMemory interface supporting append(), get_messages(), trim(), clear(), get_scratchpad(), and set_scratchpad().
   • Implemented three configurable memory retention and compaction policies:
     • WindowMemoryPolicy: retains the last $k$ messages while optionally preserving initial system instructions.
     • TokenBudgetMemoryPolicy: calculates approximate token costs and trims oldest history to fit a maximum token budget.
     • SummaryMemoryPolicy: compacts overflowing conversation turns into a structured summary message block.

2. In-Memory Store:

   • Implemented InMemoryMemoryStore with thread-safe locking for local agent loops, debugging, and fast unit tests.

3. DynamoDB-Backed Store:

   • Implemented DynamoDBMemoryStore conforming to dynavec's single-table schema (pk = "{namespace}#memory#{thread_id}").
   • Converts rich message structures and scratchpad state seamlessly to DynamoDB types via _to_dynamo and _from_dynamo.

4. Agent Integration:

   • Wired memory directly into ReActAgent (run and arun).
   • Enables multi-turn conversation memory across calls on the same thread_id without manually threading past message arrays.

Verification

• Added 7 unit tests in tests/test_memory.py covering:
  • Message dictionary serialization round-trips
  • Sliding window, token budget, and summary policies
  • InMemoryMemoryStore and DynamoDBMemoryStore CRUD with moto
  • Multi-thread isolation and scratchpad persistence
  • ReActAgent multi-turn conversation recall across turns
• Full test suite passed offline: pytest tests/test_memory.py tests/test_agents.py (15 passed).
• Formatted and verified clean via ruff check and ruff format.

[codeforstartups]

Thanks @shivamm-gupta — reviewed, and the conversation-memory store (closes #290) looks great. Same situation as #334: it needs a quick rebase onto latest development, because two PRs landed under it today:

• feat(agents): add tool registry and built-in connectors #328 (tool registry) refactored agents/react.py into helpers (_init_messages, _dispatch_tool_call/_adispatch_tool_call, _process_tool_calls/_aprocess_tool_calls, _finalize_result). Your memory read/write hooks were written against the old monolithic run/arun, so the 4 react.py conflicts are where your hooks meet the new helpers.
• feat(dynalogs): structured logs per-run / per-node with secret redaction (#300) #333 (structured logging) added an export block to src/dynavec/__init__.py that collides with your memory exports (2 trivial conflicts — keep both blocks).

git fetch upstream && git rebase upstream/development
# resolve: re-apply the memory load-before / persist-after around the new helper calls in run()/arun()
uv run --no-sync mypy && uv run --no-sync pytest -q
git push --force-with-lease

Keep your memory load (before the loop) and persist (after each turn) behavior, but wrap the extracted helpers rather than the old inline loop. Ping me when pushed and I'll re-review + merge. 🙏

(If you rebase #334 first, rebase this one on top of that result so the react.py helper structure matches.)

[shivamm-gupta]

Rebased onto latest \upstream/development! Integrated conversation memory load/persist behavior around the new extracted
eact.py\ helpers (_init_messages, _process_tool_calls/_aprocess_tool_calls, and _finalize_result). All unit tests and ruff lint checks passing cleanly.

[shivamm-gupta]

Rebased onto latest \upstream/development\ (on top of merged #334)! Unifying conversation memory with the checkpointer and new
eact.py\ helpers. All unit tests (memory, checkpointer, agents) and ruff lint checks passing cleanly.