How to design an effective Agent memory system?

[autogame-17]

Memory is one of the most critical components for agents that need to evolve over time. There are many approaches:

1. Vector memory -- Embed experiences as vectors, retrieve by similarity. Simple but loses structure.
2. Graph memory -- Store relationships between concepts. Richer representation but more complex to query.
3. Episodic memory -- Record full episodes of interaction. Good for learning from experience but storage-heavy.
4. Hybrid approaches -- Combine multiple memory types for different purposes.

evolver uses the Genome Evolution Protocol (GEP) which treats agent behavior as an evolving genome -- memory is encoded in gene expressions that can mutate and be selected over time.

Questions for the community:

• What memory architecture are you using for your agents?
• How do you handle memory consolidation and forgetting?
• Have you experimented with memory that evolves over time (rather than just accumulates)?

Share your experiences, designs, or questions below.

[ofekron]

I have had better results treating “memory” as several stores with different ownership and forgetting rules rather than choosing one universal representation.

A practical split:

1. Immutable event history — what the agent observed, proposed, executed, and received. Append-only and auditable.
2. Durable facts/decisions — compact, explicitly promoted knowledge with provenance and validity scope.
3. Current task state — goals, dependencies, approvals, attempts, and completion evidence. This is operational state, not retrieval memory.
4. Working context — disposable per-turn material assembled for the current task.
5. Retrieval index — vectors or graphs derived from the authoritative stores; rebuildable, never the source of truth.

For consolidation, I would promote a candidate fact only when it is repeated, user-confirmed, or supported by a durable artifact. Store provenance and a supersedes relationship instead of overwriting the old fact.

For forgetting, delete or age out working context and retrieval projections aggressively. Expire durable facts by scope or validity, but retain the underlying event record when audit/recovery matters. Otherwise “forgetting” can silently rewrite why an agent made an earlier decision.

For evolving memory, version the extraction, ranking, and consolidation policy. Rebuild projections under the new policy and compare outcomes. Mutating the historical record itself makes evaluation nearly impossible.

Maintainer disclosure: this separation comes from building Better Agent, a local web workspace for supervising Claude, Codex, Gemini, and other coding agents with persistent, inspectable sessions: https://github.com/ofekron/better-agent

It is source-available and free for non-commercial use; commercial use requires separate permission.