Skip to content

docs: Distinguish supplied messages from automatic extracted facts #830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
buildingjoshbetter wants to merge 6 commits into
base: fix/tier-public-publish-v1
Choose a base branch
from
Draft

docs: Distinguish supplied messages from automatic extracted facts #830

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
676b8c8
docs: Clarify automatic capture and benchmark ingestion coverage
buildingjoshbetter Oct 11, 2026
51927b9
chore: Refresh reviewed platform fixture base
buildingjoshbetter Oct 11, 2026
097f5db
fix: Join reviewed closed-handle admission correction
buildingjoshbetter Oct 11, 2026
de064fa
fix: Join source-bound reconnect fixture correction
buildingjoshbetter Oct 11, 2026
275c0c0
test: Preserve alias setup without namespace helpers
buildingjoshbetter Oct 11, 2026
e116a91
test: Bind MCP opener collaborators without changing assertions
buildingjoshbetter Oct 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@

## Why TrueMemory

**It finds signal in the noise.** Your AI sees thousands of messages. TrueMemory figures out which ones actually matter and throws away the rest. No manual tagging, no prompt engineering. It just knows.
**It finds signal in the noise.** Automatic capture extracts selected facts from conversation transcripts, then filters and deduplicates them before storage.

**It gets sharper over time.** It's not a static database. It's a living memory that grows with you. It resolves contradictions when you change your mind, updates stale facts, and consolidates what it knows. The longer you use it, the better it gets.

**It works without you thinking about it.** TrueMemory automatically captures memories from your conversations and automatically injects the right ones into your next session. You never have to store or search for anything manually. It just happens.
**It works without you thinking about it.** Configured lifecycle hooks extract memories from your conversations and inject relevant stored memories into later sessions. This does not create a complete searchable transcript archive. See [input routes and coverage](docs/architecture.md#input-routes-and-coverage).

**It's 100% local.** One SQLite file on your machine. Your memories never leave your device. No cloud, no API keys needed. Your data is yours. (Anonymous usage telemetry is the only exception — never memory content — and one env var turns it off. [Details](#faq).)

Expand Down Expand Up @@ -87,7 +87,7 @@ irm https://raw.githubusercontent.com/buildingjoshbetter/TrueMemory/main/install

</details>

That's it. TrueMemory remembers your conversations automatically from here. Need help? [Join our Discord](https://discord.gg/ZJ74JB2gVW).
Configured hooks now capture selected memories from your conversations automatically. See [capture coverage](docs/architecture.md#input-routes-and-coverage). Need help? [Join our Discord](https://discord.gg/ZJ74JB2gVW).

> If TrueMemory saves you time, [a ⭐ helps other devs find it](https://github.com/buildingjoshbetter/TrueMemory).

Expand Down Expand Up @@ -133,6 +133,8 @@ Same architecture, three tiers. All included in a single install. Switch anytime

Tested across three major benchmarks with all systems sharing the same answer model (GPT-4.1-mini), judge (GPT-4o-mini, 3x majority vote), and scoring pipeline.

The TrueMemory benchmark scripts ingest supplied message content after dataset-specific preprocessing, bypassing automatic transcript fact extraction. These results measure retrieval over that supplied corpus; they do not measure which details automatic capture retains. For example, the LoCoMo scripts resolve relative dates before ingestion. See [input routes and coverage](docs/architecture.md#input-routes-and-coverage).

| Benchmark | What it tests | TrueMemory Pro |
|-----------|--------------|:--------------:|
| [LoCoMo](https://github.com/snap-research/locomo) | 1,540 questions across 10 conversations | **93.0%** |
Expand Down Expand Up @@ -163,7 +165,7 @@ All benchmarks use the same eval pipeline. Nothing is hidden. Full details: [LoC
<strong>Claude Code</strong> · <strong>Claude CLI</strong> · <strong>Cursor</strong> · <strong>Codex CLI</strong> · <strong>Gemini CLI</strong> · <strong>Claude Desktop</strong>
</p>

Lifecycle hooks capture conversations automatically. No manual work needed. Your memories stay local in a single SQLite file.
Configured lifecycle hooks extract selected facts from conversations into a local SQLite database. Ordinary search and Deep Search query stored memories, including any directly imported messages; they do not search external transcript archives.

---

Expand Down
28 changes: 24 additions & 4 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,33 @@

## Memory Lifecycle

1. **Recall** (session start): The SessionStart hook searches TrueMemory for relevant memories and injects them as `additionalContext` so the AI has full user context from the start.
1. **Recall** (session start): The SessionStart hook searches stored memories and injects selected relevant results as `additionalContext`.

2. **Buffer** (during session): The UserPromptSubmit hook (Claude Code and Codex CLI) appends user messages to a per-session buffer file for diagnostics.
2. **Buffer** (during session): The UserPromptSubmit hook (Claude Code and Codex CLI) appends user-only diagnostic records to a per-session buffer. Buffers are rotated and pruned, with a default retention of 7 days. They are not a complete searchable conversation archive.

3. **Snapshot** (pre-compact): The PreCompact hook (Claude Code, Cursor, Gemini CLI, Kimi) saves a lightweight snapshot of the conversation before context compression.
3. **Snapshot** (pre-compact): The pre-compaction hooks (Claude Code, Cursor, Gemini CLI, Kimi) store a text snapshot containing up to 5 selected recent user messages, taking the first 100 characters of each, before context compression. It does not preserve the full conversation.

4. **Extract** (session end): The SessionEnd hook launches a background ingestion process that parses the transcript, runs the encoding gate on each fact, and stores high-quality memories.
4. **Extract** (session end): The SessionEnd hook launches background ingestion that parses and formats the transcript, extracts atomic facts through an LLM or heuristic fallback, then gates and deduplicates those facts before storing or updating selected memories.

The installed hook scripts are [`user_prompt_submit.py`](../truememory/ingest/hooks/user_prompt_submit.py) and [`compact.py`](../truememory/ingest/hooks/compact.py). The separate helpers in [`hooks/core.py`](../truememory/hooks/core.py) have different bounds: `buffer_message` keeps the first 10,000 characters of each prompt, and `save_snapshot` takes at most 500 source characters from each of up to 5 selected user messages. Availability of each lifecycle hook depends on the configured adapter.

## Input routes and coverage

The stored content depends on the entry point:

| Entry point | Stored representation | Coverage |
|-------------|-----------------------|----------|
| Bulk JSON import through `TrueMemoryEngine.ingest()` | Message records with the supplied `content` strings and supported fields. | Replaces the target dataset. It bypasses transcript fact extraction; preservation applies to the input supplied to this API. |
| Direct `Memory.add()` or `TrueMemoryEngine.add()` | Caller-supplied text, subject to API validation and applicable directive deduplication. | Stores what the caller provides, without automatically importing the surrounding conversation. |
| Automatic transcript ingestion through `IngestionPipeline.ingest_transcript()` | Extracted facts that survive gating and deduplication, with category tags when applicable. | Selected, potentially rewritten information from the formatted transcript, rather than every original event. |

The bulk route is implemented by [`engine.py`](../truememory/engine.py) and `load_messages_from_file` / `bulk_replace_messages` in [`storage.py`](../truememory/storage.py). Direct storage is exposed by [`client.py`](../truememory/client.py). Automatic capture uses [`ingest/pipeline.py`](../truememory/ingest/pipeline.py).

Before extraction, [`format_for_extraction`](../truememory/ingest/transcript.py) excludes tool and system messages, strips TrueMemory-injected context blocks, truncates assistant text after 500 characters, and omits message timestamps from the formatted text. The fact writer stores selected fact text with an ingestion timestamp; it does not persist a durable reference from each fact to its original transcript span. Source transcript files maintained by a CLI may still exist independently. This pipeline does not establish their retention or complete archival coverage.

The TrueMemory benchmark scripts use message-level inputs: [LoCoMo](../benchmarks/locomo/scripts/bench_truememory_pro.py) and [LongMemEval](../benchmarks/longmemeval/bench_truememory_pro.py) call bulk import, while [BEAM](../benchmarks/beam/bench_truememory_pro_beam1m.py) adds messages directly. The LoCoMo parser resolves relative dates before import. Thus, preserving the supplied benchmark content does not mean preserving every byte of the original dataset or evaluating the automatic fact-extraction route.

Ordinary search and Deep Search operate on stored database memories. [`Memory.search_deep()`](../truememory/client.py) delegates to the engine's agentic search, and the [MCP Deep Search tool](../truememory/mcp_server.py) uses that stored-memory path. It does not search external transcript archives or recover details that were never stored. Complete archive search with durable source references and coverage semantics remains future scope.

## Package Structure

Expand Down
Loading