A technical deep dive into how Cortex works, why each design decision was made, and how to port the system to any LLM tool.
CAPTURE-STORE-SEARCH-INJECT LOOP
+-----------+ +-----------+ +-----------+ +-----------+
| CAPTURE | | STORE | | SEARCH | | INJECT |
+-----------+ +-----------+ +-----------+ +-----------+
| | | | | | | |
| Hooks: | | Dedup: | | Scoring: | | Tiered: |
| - PostTool| --> | - Hash | ... | - IDF | --> | - HOT |
| - Error | | - Jaccard | | - Stems | | - WARM |
| - Correct | | - 3-way | | - Tags | | - COLD |
| - Stop | | decide | | - Decay | | |
| | | | | - Boost | | Cluster |
| Classify: | | Append OR | | | | merge HOT |
| - Pattern | | Supersede | | Cache: | | items |
| - LLM | | OR Skip | | - SQLite | | |
| | | | | - 12-56x | | Budget: |
+-----------+ +-----------+ | speedup | | 4000 char |
| | +-----------+ | hard cap |
| | | +-----------+
| | | |
v v v v
+---------------------------------------------------------------+
| memory_log.jsonl |
| (append-only JSONL -- source of truth for all entries) |
+---------------------------------------------------------------+
| memory_cache.db |
| (SQLite acceleration layer -- pre-computed tokens/stems) |
+---------------------------------------------------------------+
| core_tagged.jsonl |
| (distilled knowledge -- no recency decay applied) |
+---------------------------------------------------------------+
Read path: User prompt --> extract_intent --> parallel search -->
score --> tier --> cluster-merge --> inject context
Write path: Hook fires --> classify --> dedup check --> store -->
invalidate cache
The read path fires on every UserPromptSubmit hook. The write path fires on PostToolUse (errors, test results), UserPromptSubmit (corrections, preferences), and Stop (session-end extraction). The loop closes because better memories produce better Claude responses, which produce better memories.
The scoring pipeline is the heart of Cortex. Every search result passes through this function:
score = match_score * tag_boost * type_boost * priority_boost * decay * coverage_factor
Token matching happens in a strict priority cascade. If a higher tier matches, lower tiers are only used for the remaining unmatched tokens.
| Tier | Method | Weight | Description |
|---|---|---|---|
| 1 | Exact match | 1.0x (IDF-weighted) | Query token appears verbatim in entry |
| 2 | Stem match | 0.7x (IDF-weighted) | _stem(query_token) == _stem(entry_token) |
| 3 | Substring match | 0.5x (IDF-weighted) | One token is a substring of the other (min 50% length ratio) |
The stemmer is a lightweight suffix-stripping function (no NLTK dependency). It handles common English suffixes: -tion, -ing, -ed, -ly, -er, -ment, -ness, -able, -ful, -ous, -ive, plurals, and about 30 others. This is intentionally simple -- it catches morphological variants ("deploying" -> "deploy") without the weight of a full NLP pipeline.
idf(token) = log(1 + N / df)Where N is the total number of entries and df is the count of entries containing that token. Smoothed IDF (log(1 + ...)) ensures tokens appearing in every entry still have positive weight (~0.69) rather than zero.
The match score uses IDF as both numerator and denominator weights:
# Numerator: sum of IDF weights for matched tokens (at their tier weight)
exact_w = sum(idf[t] for t in exact_matches) # 1.0x
stem_w = sum(idf[t] for t in stem_only_matches) * 0.7 # 0.7x
sub_w = substring_score * 0.5 # 0.5x
# Denominator: total IDF weight of all query tokens
total_w = sum(idf[t] for t in query_tokens)
match_score = (exact_w + stem_w + sub_w) / total_wThe effect: rare, domain-specific tokens (like "kubernetes" or "dockerfile") contribute far more than common tokens (like "config" or "error"). If a 4,400-entry corpus has "kubernetes" in 12 entries and "error" in 800, a match on "kubernetes" is worth ~40x more than a match on "error".
IDF weights are cached in SQLite and refreshed when >50 new entries are added or >24 hours have elapsed. The change from a single supersede is <0.01% across thousands of entries -- negligible.
Aliases are a static JSON map (e.g., {"myapp": ["my-app", "my_app"]}) that allows named-entity matching without polluting the scoring denominator. When a query token has aliases, those aliases are checked against entry tokens, but the denominator stays as the count of original query tokens. This means aliases improve recall without diluting precision.
| Boost | Multiplier | Condition |
|---|---|---|
| Tag boost | 2.0x | Query tokens overlap with entry's tags field |
| Type boost | 1.5x | Entry type is user_correction, repeated_correction, user_preference, frustrated_correction, or universal_directive |
| Priority boost | 1.0 + (priority * 0.1) | For distilled entries with explicit priority field |
All boosts are gated by multiplier eligibility:
multiplier_eligible = (
match_score >= 0.3 # strong match
or (match_score >= 0.15 and len(exact_matches) > 0) # decent match + real overlap
) and len(query_tokens) >= 3This prevents weak matches (stem-only or substring-only with low base scores) from being promoted to HOT tier by a 2x tag boost. A match_score of 0.12 with a 2x tag boost would be 0.24 -- just under HOT threshold. Without the gate, this would inject full content for a marginal match.
decay = e^(-0.03 * age_in_days)| Age | Retention |
|---|---|
| 1 day | 97% |
| 7 days | 81% |
| 30 days | 41% |
| 60 days | 17% |
| 90 days | 7% |
The rate of 0.03 was tuned empirically. Fast enough that old entries naturally fade. Slow enough that a correction from last week still matters. Core tagged entries (distilled knowledge) are exempt from decay -- they are timeless.
A bidirectional relevance penalty for superficial matches in large entries:
if exact_matches and len(entry_tokens) > 3:
entry_coverage = len(exact_matches & entry_tokens) / sqrt(len(entry_tokens))
coverage_factor = max(0.15, min(1.0, entry_coverage * 3))
else:
coverage_factor = 1.0An entry with 200 tokens that matches 2 query tokens is probably a cross-topic coincidence. The coverage factor penalizes this. The square root in the denominator provides diminishing penalty -- an entry twice as long is not penalized twice as harshly.
SQLite FTS5 was A/B tested as both a primary search replacement and a pre-filter. The test used 10 representative queries against a 4,400+ entry corpus.
FTS5 misses:
- Alias expansion: "myapp" doesn't FTS5-match "my-app"
- Stem matching: FTS5's built-in stemmer is English Porter, but the scoring pipeline uses custom suffix stripping tuned for technical terms
- Tag boosting: FTS5 ranks by BM25 only -- no way to boost entries whose tags match the query
- Recency decay: FTS5 has no temporal signal
- Coverage factor: FTS5 has no bidirectional relevance check
The idea was to use FTS5 to narrow candidates, then score them with _score_entry(). But FTS5 filters out entries that only match via stems, aliases, or substrings -- the exact entries the custom pipeline exists to find. Miss rates ranged from 19% (simple queries) to 96% (queries relying on alias/stem matching).
Custom scoring beats general-purpose full-text search for this use case. FTS5 is designed for document retrieval where the query vocabulary closely matches the corpus vocabulary. Memory entries are written by hooks in a different session context than the user's current query, so vocabulary mismatch is the norm, not the exception.
Soundex (American phonetic encoding) was originally used for fuzzy matching. It maps words to 4-character codes based on pronunciation: "kubernetes" and "kubernates" both encode to K165.
Below ~2,000 entries, Soundex was useful. It caught typos and phonetic variants that exact matching would miss.
The Soundex alphabet produces only ~7,000 unique codes for the English language. With 4,400+ entries across diverse domains (Rust/iced GUI, memory system, Docker deployment, DNS configuration), collisions became rampant:
- Rust/iced GUI entries matching memory system queries
- Docker entries matching database queries
- Completely unrelated entries sharing Soundex codes
The false positive rate made search results unreliable.
Three existing features already covered every legitimate Soundex use case:
| Use Case | Before (Soundex) | After |
|---|---|---|
| Typos ("kubernates") | Phonetic match | Substring match (50% length ratio gate) |
| Named entities ("myapp" for "my-app") | Phonetic match | Alias expansion (aliases.json) |
| Morphological variants ("deploying"/"deploy") | Phonetic match (incidental) | Stem matching |
Soundex was removed from both the scoring function and the cache pre-filter. The _soundex() function remains in the codebase as dead code (SQLite schema still references the column) but is never used in scoring decisions.
Every new entry goes through a three-way decision before being stored. This is inspired by Mem0's ADD/UPDATE/DELETE/NOOP pattern, but uses Jaccard similarity instead of an LLM call.
new entry arrives
|
[O(1) hash check]
/ \
match no match
| |
SKIP [Jaccard scan of last 50 entries]
(exact dupe) / | \
J > 0.5 0.3-0.5 J < 0.3
| AND new |
SKIP is longer APPEND
(near-dupe) | (new topic)
SUPERSEDE
(update in-place)
def jaccard(a, b):
tokens_a = set(tokenize(a))
tokens_b = set(tokenize(b))
return len(tokens_a & tokens_b) / len(tokens_a | tokens_b)| Outcome | Condition | Action |
|---|---|---|
| SKIP | Jaccard > 0.5 OR content hash match | Do nothing. Near-duplicate. |
| SUPERSEDE | Jaccard 0.3-0.5 AND new entry is longer | Replace old entry in-place. Mark cache dirty. |
| APPEND | Jaccard < 0.3 | Append as new entry. Append to cache inline. |
Before the Jaccard scan, a content hash check runs against the SQLite cache:
content_hash = md5(content.lower())[:16]
SELECT 1 FROM entry_cache WHERE content_hash = ? LIMIT 1This catches exact duplicates in constant time. The Jaccard scan only runs for entries that pass the hash check.
When an entry is superseded (replaced in-place), the line-based search cache is invalidated via mark_dirty(). This creates a .cache-dirty marker file that triggers a full cache rebuild on the next search. Append-only operations update the cache inline without a rebuild.
Search results are divided into three tiers based on their final score:
| Tier | Score Range | Injection Style | Context Cost |
|---|---|---|---|
| HOT | >= 0.3 | Full content | 100-500 chars per entry |
| WARM | 0.15 - 0.3 | Preview (first 2 lines, 150 chars) | ~150 chars per entry |
| COLD | < 0.15 | Not injected | 0 |
Even if an entry scores >= 0.3, it must have at least 2 exact token matches against the original query to be injected as HOT:
if r.get('exact_matches', 0) < 2:
continue # Demoted to nothing, not even WARMThis is defense in depth. An entry that scores high purely via stem or substring matching without real token overlap is often a cross-domain false positive. The gate prevents these from wasting context budget.
## Past session learnings
Relevant context from previous sessions:
### [USER_CORRECTION]
Never run docker prune -a without explicit user confirmation.
### [WORKING_SOLUTION]
The kubernetes networking fix requires updating CoreDNS configmap. (+2 related)
## POSSIBLE MATCHES (may be relevant)
- [CONFIG_INSIGHT] DNS propagation takes up to 48h for TTL changes...When multiple HOT items overlap significantly, they are clustered to reduce context consumption.
- Separate corrections from mergeable entries. Corrections are never merged -- every word in "never run docker prune -a" matters.
- For each mergeable entry, compute its token set.
- Greedy clustering: for each unassigned entry, find all other entries with > 40% Jaccard overlap against the growing cluster token set.
- The first entry in each cluster is the representative. Others are summarized as
(+N related).
Lower thresholds (20-30%) catch unrelated entries that happen to share common technical vocabulary. Higher thresholds (50-60%) miss genuine duplicates that use different phrasing. 40% was validated against real search results: it correctly groups entries about the same topic while keeping distinct topics separate.
As a cluster grows, its union token set grows faster than the intersection with new candidates. This means Jaccard naturally decreases for each additional candidate, creating a self-limiting cluster size. A hard cap of 5 items per cluster provides a safety net.
Cluster-merge reduces context injection by ~40-50% in multi-result scenarios with zero information loss. The representative entry's full content is preserved; only the redundant entries are summarized.
Query expansion adds semantic recall that keyword matching alone cannot provide. "Delete the deployment" should find entries about "remove", "erase", "teardown", and "destroy".
Pass 1: Expand query with LLM-generated synonyms
Pass 2: Score results against the ORIGINAL query (not expanded)
This is the critical design decision. Expansion improves recall -- it finds entries that use different vocabulary for the same concept. But scoring against the expanded query would hurt precision -- an entry matching "erase" but not "delete" would score as well as one matching "delete" directly.
The two-pass pattern gives you both: expanded vocabulary for candidate discovery, anchored scoring for relevance ranking.
| Priority | Backend | Latency | Cost | Notes |
|---|---|---|---|---|
| 1 | Gemini Flash Lite | ~1s | Free tier | Cheapest, fastest |
| 2 | Gemini Flash | ~1s | Free tier | Smarter, still fast |
| 3 | Anthropic Haiku API | ~2s | Per-token | If ANTHROPIC_API_KEY set |
| 4 | claude -p subprocess |
~12s | Included | Always available, Node.js cold start overhead |
Query: "delete the deployment"
Generate 3-5 direct synonyms or alternate spellings for the key concepts.
Focus on what the user literally means, not tangentially related topics.
Return ONLY comma-separated keywords, nothing else.
Expansions are cached on disk (JSON file, max 100 entries, content hash keys). Identical queries across sessions skip the LLM call entirely. No TTL -- expansion synonyms do not change.
If no LLM backend is available (no API keys, no network), expansion is skipped silently. The system falls back to keyword + stem + substring matching, which handles most queries. Expansion is an enhancement, not a requirement.
The SQLite cache provides 12-56x speedup over full JSONL scanning with identical results. Verified across 10 test queries.
For each entry in memory_log.jsonl, the cache stores:
| Column | Content |
|---|---|
tokens |
Space-separated set of 3+ char tokens from content + tags + type + section |
stems |
Space-separated set of stemmed tokens |
soundex_codes |
Space-separated Soundex codes (legacy, not used in scoring) |
tag_tokens |
Space-separated tokens from tags only |
content_hash |
MD5 prefix for O(1) dedup check |
raw_json |
Full JSON line for deserialization |
1. Load cache into memory (one-time per process)
2. For each cached entry:
a. Set intersection: query_tokens & entry_tokens
b. Set intersection: query_stems & entry_stems
c. If no overlap: skip (costs ~0.001ms)
d. If overlap: call _score_entry() (costs ~0.1ms)
3. Sort by score, return top N
The set intersection pre-filter skips ~90% of entries before the expensive _score_entry() call. This is why the speedup is 12-56x rather than 2-3x.
The cache is an acceleration layer. If the cache DB is missing, corrupt, or stale, search_memory_log() falls back to scanning the raw JSONL file. The results are identical -- the cache merely pre-computes what the JSONL scan computes on the fly.
This is a deliberate architectural choice. The JSONL file is the source of truth. The cache can be deleted and regenerated at any time with no data loss.
| Event | Invalidation Strategy |
|---|---|
| New entry appended | append_entry() -- adds to cache inline, no rebuild |
| Entry superseded | mark_dirty() -- creates .cache-dirty marker, triggers full rebuild on next search |
| Cache DB missing | Falls back to JSONL scan, triggers async rebuild |
| Stale detection | Checks (mtime, fsize, .cache-dirty) on load |
| Concurrent builds | BEGIN EXCLUSIVE in SQLite prevents double-rebuild |
IDF weights are stored alongside the cache in the same SQLite database:
CREATE TABLE idf_weights (token TEXT PRIMARY KEY, weight REAL);
CREATE TABLE idf_meta (key TEXT PRIMARY KEY, value TEXT);Refreshed when >50 new entries or >24 hours since last refresh. The 24-hour/50-entry window catches IDF drift from normal usage without recomputing after every append.
Sessions in Claude Code are ephemeral. When a user starts a new session, Claude has no context about what happened in the previous one. Session bridging provides that context in ~250 characters.
At session end, the stop_learning_extractor.py hook writes a SESSION_CHECKPOINT entry:
{
"type": "SESSION_CHECKPOINT",
"content": "Session abc12345: Fixed DNS propagation for cloudflare | Updated docker-compose.yml ports | Resolved CORS issue in API gateway",
"session_id": "abc12345-...",
"timestamp": "2026-03-29T15:30:00Z"
}On next session start, get_recent_session_summary() finds the most recent checkpoint from a different session and injects it:
## Session Context
Session abc12345: Fixed DNS propagation for cloudflare | Updated docker-compose.yml ports | Resolved CORS issue in API gateway
For entries written before checkpointing was added, or if checkpoint writing failed, the fallback reconstructs topics from raw entry tags:
- Read last 4KB of
memory_log.jsonl - Find entries from the most recent different session (compare
session_id[:8]) - Extract unique tags, skip generic ones (
stop_hook,llm_extracted,auto_detected, etc.) - Format:
"Last session topics: kubernetes, dns, cloudflare, docker"
The fallback must stay permanently. Old sessions will never have checkpoints, and checkpoint writing is wrapped in try/except (it must never block session end).
Complete pipeline from user message to context injection:
USER TYPES: "How do I fix the kubernetes networking issue?"
|
v
[UserPromptSubmit hook fires]
|
v
[extract_intent()]
| Strip stop words: "fix", "kubernetes", "networking", "issue"
| IDF selection: if >10 tokens, keep top 10 by rarity
|
v
[Parallel search via ThreadPoolExecutor]
|
+--> [Keyword search: search_memory_log(intent)]
| |
| +--> [Cache path: set intersection pre-filter]
| | skip ~90% of entries
| | _score_entry() on remaining ~10%
| |
| +--> [JSONL fallback if no cache]
| scan all entries, _score_entry() each
|
+--> [Semantic search: expand_query(intent)]
|
+--> [LLM: "kubernetes, k8s, networking, CNI, pod network"]
+--> [search_memory_log(expanded)]
+--> [rescore_results(raw, original_intent)]
^-- scoring anchored to original query
|
v
[Merge + dedup keyword and semantic results]
| keep highest score per content[:80] key
| cross-source dedup vs core_tagged results
|
v
[Compute exact_matches per result]
|
v
[format_tiered_context()]
|
+--> HOT (>= 0.3 AND >= 2 exact matches): full content
+--> WARM (0.15-0.3 AND >= 2 exact matches): preview
+--> COLD (< 0.15 or < 2 exact matches): dropped
|
v
[cluster_merge() on HOT items]
| group by Jaccard > 0.4 overlap
| corrections exempt from merging
|
v
[Budget check: total < 4000 chars]
| truncate if over budget
|
v
[Inject as additionalContext in hook output]
|
v
CLAUDE SEES:
## Session Context
Last session topics: kubernetes, dns, docker
## Relevant Knowledge (from past sessions, advisory only)
**[Networking]** CoreDNS configmap must be updated after...
## Past session learnings
### [WORKING_SOLUTION]
The kubernetes networking fix requires updating CoreDNS configmap...
### [USER_CORRECTION]
Never modify kube-system namespace resources without backup...
(Memory is auto-captured in background -- no manual action needed.)
To implement Cortex for a different LLM tool (Cursor, Copilot, Windsurf, or any tool with a hook/plugin system):
-
Hook into the prompt lifecycle. You need a way to inject context before the LLM sees the user's message, and a way to observe tool outputs after execution. In Claude Code, these are
UserPromptSubmitandPostToolUsehooks. -
Implement
_score_entry(). This is the core algorithm. Copy the multi-signal scoring pipeline. The function is ~120 lines of Python with zero dependencies. -
Use JSONL for storage. Append-only, one JSON object per line. No schema migrations, no database setup, no versioning. The JSONL file is human-readable and trivially parseable.
-
Add the three-way dedup. Without it, the memory log grows with duplicates and the signal-to-noise ratio degrades. Jaccard similarity at 0.5/0.3 thresholds is the minimum viable dedup.
-
Add the SQLite cache only if search latency matters. At <1,000 entries, JSONL scanning is fast enough. Above that, the cache provides 12-56x speedup. The cache is optional -- it is an acceleration layer, not a feature.
-
Add query expansion only if you have an LLM API available. The system works well without it. Expansion helps most when the memory corpus uses different vocabulary than the user's current query.
-
Tune thresholds for your corpus size. The defaults (HOT=0.3, WARM=0.15, decay=0.03) were tuned for a 4,400+ entry corpus. Smaller corpora may benefit from lower thresholds; larger corpora may need higher ones.