v0.2.0: bound search queries, record the embedding window, refresh unchanged stats, lock the inbox

CHANGELOG.md

@@ -9,6 +9,10 @@ A write path, through an inbox you approve.
 - `puremind serve --read-only`, or `[inbox] enabled = false` in `config.toml`, leaves out `puremind_propose`, so the server has only its three read-only tools. `[inbox] folder` sets where approved notes go. `puremind init --force` keeps both settings.
 - A Homebrew formula in `packaging/homebrew/puremind.rb`, for use from a tap of your own (no public tap yet).
 - README: results by question type for the benchmark (Recall@5 and judged QA for each of the six LongMemEval types).
+- `puremind_search` refuses a query over 4,096 characters, in the MCP schema and in the handler, instead of tokenizing it and echoing it back.
+- The index records its embedding window (`embed_max_seq`). Changing the window is refused like a model change, until `puremind index --rebuild`, so long notes no longer keep vectors cut at the old window. The error names the recorded and the configured window, and a model change made at the same time is reported as a model change. An index that never recorded the window adopts the current one without a rebuild. The README and the `config.toml` template say that changing `embed_max_seq` takes a rebuild.
+- An unchanged note whose mtime or size moved (an identical save, a sync tool) now has them updated without being embedded again, so `puremind_recent` orders it correctly.
+- `puremind_propose` holds a lock across the duplicate check, the 100-proposal cap and the file creation, so two agents proposing at once can no longer pass the cap or save the same proposal twice.
 - The MCP server's instructions and `puremind connect` now say that the server cannot write to the vault but accepts proposals. The test that listed exactly three tools now lists four.
 
 ## 0.1.6 (2026-10-22)

README.md

@@ -58,7 +58,7 @@ The MCP server offers four tools. Three only read your vault and are marked read
 
 | Tool | Purpose |
 |---|---|
-| `puremind_search(query, limit=10)` | Hybrid search. Returns up to 50 matching passages, best first, each with its note path, heading path and score. |
+| `puremind_search(query, limit=10)` | Hybrid search. Returns up to 50 matching passages, best first, each with its note path, heading path and score. A query is at most 4,096 characters. |
 | `puremind_get(path)` | Reads one note by the path search returned. It serves only the notes the indexer reads (see Vault below) and cuts text beyond 200,000 characters, saying so. |
 | `puremind_recent(days=7, limit=20)` | Lists the notes modified in the last `days` days, newest first, as of the last index run. |
 | `puremind_propose(title, content)` | Proposes a memory. It is saved in an inbox outside the vault and does nothing else until you approve it. |
@@ -92,7 +92,7 @@ Limits: a title of at most 200 characters, content of at most 20,000, and 100 pr
 - **Chunker.** Heading-aware: splits on `#` to `####` and keeps a breadcrumb such as `Project > Setup`. Headings inside fenced code blocks and YAML frontmatter do not count. A heading with no text of its own joins the subsection under it, and most sections under 100 characters merge with the one before. A section longer than 2,048 characters is cut into near-even windows with about 20% overlap, preferably at blank lines. A code block or frontmatter that fits in a window is usually kept whole, but one that is longer, or that starts late in a long section, can be cut; every part of a cut block keeps its own opening and closing fence lines. Frontmatter of 100 characters or more is a chunk of its own.
 - **Store.** One SQLite file, `~/.puremind/index.sqlite`, readable only by you: files and hashes, chunks, an FTS5 keyword index (used by `puremind search --lexical-only`), and float16 vectors. Vectors are loaded into a numpy matrix for dense search.
 - **Retrieval.** BM25 over English-stemmed words (top 50) and Qwen3-Embedding-0.6B cosine search (top 50, 1024 dimensions, with the query instruction prefix) are fused with reciprocal rank fusion (k=60). The bge-reranker-v2-m3 cross-encoder then re-scores the 50 fused candidates, reading the first 800 characters of each. `--no-rerank` skips that last step.
-- **Languages.** The embedder is multilingual, so search by meaning works in most languages. Keyword matching uses an English stemmer. In hybrid search, Chinese, Japanese and Korean text is matched by single characters and overlapping pairs of characters; `search --lexical-only` does not do this, and treats an unspaced run of those characters as one word. Thai, Lao, Khmer and Burmese are not split into words for keyword matching. In scripts that need more than one token per character, such as Telugu, Burmese, Khmer and Ethiopic, a full-length chunk is longer than the 2,048 tokens the embedder reads, so search by meaning misses its end (keyword search does not); raise `embed_max_seq` to about 3,200 if most of your notes are in such a script.
+- **Languages.** The embedder is multilingual, so search by meaning works in most languages. Keyword matching uses an English stemmer. In hybrid search, Chinese, Japanese and Korean text is matched by single characters and overlapping pairs of characters; `search --lexical-only` does not do this, and treats an unspaced run of those characters as one word. Thai, Lao, Khmer and Burmese are not split into words for keyword matching. In scripts that need more than one token per character, such as Telugu, Burmese, Khmer and Ethiopic, a full-length chunk is longer than the 2,048 tokens the embedder reads, so search by meaning misses its end (keyword search does not); raise `embed_max_seq` to about 3,200 if most of your notes are in such a script, then run `puremind index --rebuild`.
 - **Watcher.** `puremind watch` indexes once, then again each time the vault has been quiet for `--debounce` seconds (default 2) after a change to a visible note or folder. Reads, hidden folders and other file types never trigger a run, and runs never overlap. It follows a vault directory that is replaced or made again (for example by a remount or a sync tool) or that cannot be read for a while, and if nothing has happened for 10 minutes it scans anyway, in case a change was missed. A run that fails is reported on one line and watching goes on, except when no later run could succeed (a vault symlink that now names another directory than the index's, or a system limit such as the inotify watch limit): then `puremind watch` stops with an error.
 
 ## Memory and speed
@@ -130,7 +130,7 @@ The reranker costs about 0.2 seconds per candidate on a CPU, so with the default
 
 - **Model revisions.** The default models are pinned to `Qwen/Qwen3-Embedding-0.6B` at `97b0c614be4d77ee51c0cef4e5f07c00f9eb65b3` and `BAAI/bge-reranker-v2-m3` at `953dc6f6f85a1b2dbfca4c34a2796e7dde08d41e`. Another model uses its repository's default branch as it was at the first download, and is not refreshed afterwards unless you set a revision or clear the cache.
 - **Model names.** A name is a Hugging Face model id unless it starts with `/`, `./`, `../` or `~`, which mark a local directory. A directory in your working directory is never picked up by accident. Models load with `trust_remote_code` off.
-- **Changing the embedding model**, or `embedder_revision`, takes `puremind index --rebuild`, since vectors from different models are not comparable. Until then `puremind index` and `puremind watch` stop with an error and change nothing, and searches are refused. A `puremind watch` keeps the model it started with, so restart it after the change; one left running stops instead of wiping the rebuilt index. With `--rebuild`, the new model must first load and embed a test passage, so a mistyped name leaves the existing index untouched. The index is then emptied, and searches find nothing until the run finishes. `puremind index --rebuild` with the same model, by contrast, replaces one note at a time, so searches keep working.
+- **Changing the embedding model**, `embedder_revision` or `embed_max_seq` takes `puremind index --rebuild`, since vectors from different models are not comparable, and a new window changes the vectors of long chunks. Until then `puremind index` and `puremind watch` stop with an error and change nothing, and searches are refused. A `puremind watch` keeps the model and window it started with, so restart it after the change; one left running stops instead of wiping the rebuilt index. A running `puremind serve` keeps them too, and refuses searches until it is restarted. With `--rebuild`, the new model must first load and embed a test passage, so a mistyped name leaves the existing index untouched. The index is then emptied, and searches find nothing until the run finishes. `puremind index --rebuild` with the same model, by contrast, replaces one note at a time, so searches keep working.
 - **The reranker window.** 800 characters is the setting the benchmark below was measured with. Raising `rerank_chars` lets the reranker see more of each passage, at the cost of slower searches.
 - **Indexes from 0.1.1 or earlier** are brought up to date by the next `puremind index` or `puremind watch` run, which cuts and embeds every note again. It takes about as long as the first index did, and searches keep working meanwhile.
 
@@ -147,7 +147,7 @@ The index is a second copy of your notes' text. It is created readable only by y
 | Command | What it does |
 |---|---|
 | `puremind init <vault_dir> [--force]` | Writes `config.toml`. |
-| `puremind index [vault_dir] [--rebuild]` | Indexes the vault incrementally; `--rebuild` embeds every note again, and is needed after changing the embedding model. |
+| `puremind index [vault_dir] [--rebuild]` | Indexes the vault incrementally; `--rebuild` embeds every note again, and is needed after changing the embedding model or `embed_max_seq`. |
 | `puremind watch [vault_dir] [--debounce SECONDS]` | Indexes, then re-indexes when notes change. |
 | `puremind search <query> [--limit N] [--no-rerank] [--lexical-only]` | Searches from the terminal; `--limit` is 1 to 50. |
 | `puremind serve [--no-rerank] [--read-only]` | Runs the MCP server on stdio. Your agent starts this for you. `--read-only` leaves out `puremind_propose`. |

src/puremind_mac/config.py

@@ -277,6 +277,7 @@ def render_config(vault: Path, cfg: Config) -> str:
         '# "auto" picks mps on Apple Silicon, cuda if present, else cpu.',
         f"device = {toml_string(cfg.device)}",
         "# Tokens per chunk the embedder reads; a chunk is at most 2,048 characters.",
+        "# Changing it takes `puremind index --rebuild`.",
         f"embed_max_seq = {cfg.embed_max_seq}",
         "",
         "[retrieval]",

src/puremind_mac/inbox.py

@@ -9,6 +9,7 @@
 from __future__ import annotations
 
 import errno
+import fcntl
 import json
 import os
 import re
@@ -128,35 +129,38 @@ def propose(home: Path, title: str, content: str) -> tuple[Proposal, bool]:
     root = inbox_dir(home)
     _private_dir(home, create=True)
     _private_dir(root, create=True)
-    pending, unreadable = list_proposals(home)
-    for p in pending:
-        if p.title == title and p.content == content:
-            return p, False
-    if len(pending) + len(unreadable) >= MAX_PENDING:
-        raise InboxError(
-            f"the inbox already holds {MAX_PENDING} proposals; ask the user to review them with `puremind inbox`"
-        )
-    now = datetime.now(UTC)
-    for _ in range(8):
-        pid = f"{now:%Y%m%d}-{secrets.token_hex(3)}"
-        proposal = Proposal(pid, now.strftime("%Y-%m-%dT%H:%M:%SZ"), title, content)
-        try:
-            fd = os.open(root / f"{pid}.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0), 0o600)
-        except FileExistsError:
-            continue
-        # Written in place: a reader that meets the half-written file counts it as unreadable, and
-        # the writer is a short-lived server process that holds the whole text already.
-        try:
-            with os.fdopen(fd, "w", encoding="utf-8") as fh:
-                json.dump(proposal.__dict__, fh)
-        except BaseException:
+    flags = os.O_RDWR | os.O_CREAT | getattr(os, "O_NOFOLLOW", 0)
+    with os.fdopen(os.open(root / ".propose.lock", flags, 0o600), "a") as lock:
+        fcntl.flock(lock.fileno(), fcntl.LOCK_EX)
+        pending, unreadable = list_proposals(home)
+        for p in pending:
+            if p.title == title and p.content == content:
+                return p, False
+        if len(pending) + len(unreadable) >= MAX_PENDING:
+            raise InboxError(
+                f"the inbox already holds {MAX_PENDING} proposals; ask the user to review them with `puremind inbox`"
+            )
+        now = datetime.now(UTC)
+        for _ in range(8):
+            pid = f"{now:%Y%m%d}-{secrets.token_hex(3)}"
+            proposal = Proposal(pid, now.strftime("%Y-%m-%dT%H:%M:%SZ"), title, content)
             try:
-                os.unlink(root / f"{pid}.json")
-            except OSError:
-                pass
-            raise
-        return proposal, True
-    raise InboxError("could not save the proposal; try again")
+                fd = os.open(root / f"{pid}.json", os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0), 0o600)
+            except FileExistsError:
+                continue
+            # Written in place: a reader that meets the half-written file counts it as unreadable, and
+            # the writer is a short-lived server process that holds the whole text already.
+            try:
+                with os.fdopen(fd, "w", encoding="utf-8") as fh:
+                    json.dump(proposal.__dict__, fh)
+            except BaseException:
+                try:
+                    os.unlink(root / f"{pid}.json")
+                except OSError:
+                    pass
+                raise
+            return proposal, True
+        raise InboxError("could not save the proposal; try again")
 
 
 def find(home: Path, ref: str) -> Proposal:

src/puremind_mac/indexer.py

@@ -22,7 +22,15 @@
     frontmatter_end,
 )
 from .chunker import VERSION as CHUNKER_VERSION
-from .models import Embedder, ModelLoadError, embedder_revision, model_key, model_label
+from .models import (
+    Embedder,
+    ModelLoadError,
+    embedder_revision,
+    embedding_config,
+    embedding_config_label,
+    model_key,
+    model_label,
+)
 from .store import Store, StoreError
 
 log = logging.getLogger("puremind.index")
@@ -46,7 +54,7 @@ class VaultMismatch(ValueError):
 
 
 class ModelMismatch(ValueError):
-    """The index was built with another embedding model or revision, and no rebuild was asked for.
+    """The index was built with another embedding model, revision or configuration, and no rebuild was asked for.
     Nothing was changed."""
 
 
@@ -285,10 +293,16 @@ def __init__(self, vault: Path, store: Store, embedder: Embedder, stats: IndexSt
         # embedded again, and none cut by an earlier chunker.
         current = set() if every else store.chunked_with(CHUNKER_VERSION)
         self.keep = {rel: digest for rel, digest in self.old.items() if rel in current}
+        self.stat = {row.path: (row.mtime, row.size) for row in store.list_files()}
         self.seen: set[str] = set()
         self.queue: list[_Job] = []
         self.queued = 0
 
+    def refresh_stat(self, rel: str, digest: str, st: os.stat_result) -> None:
+        """Keep an unchanged note's mtime and size current for ``puremind_recent``, writing only when they moved."""
+        if self.stat.get(rel) != (st.st_mtime, st.st_size):
+            self.store.update_file_metadata(rel, digest, st.st_mtime, st.st_size)
+
     def fail(self, rel: str, why: str) -> None:
         self.stats.skipped.append(f"{rel} ({why})")
 
@@ -332,6 +346,7 @@ def visit(self, path: Path) -> None:
         self.seen.add(rel)
         digest = hashlib.sha256(raw).hexdigest()
         if self.keep.get(rel) == digest:
+            self.refresh_stat(rel, digest, st)
             self.stats.unchanged += 1
             return
         try:
@@ -398,6 +413,7 @@ def commit(self, job: _Job, vectors) -> None:
                 if digest == job.digest:
                     break
                 if self.keep.get(job.rel) == digest:
+                    self.refresh_stat(job.rel, digest, st)
                     self.stats.unchanged += 1  # edited back to the indexed text
                     return
                 if job.attempt >= RECHECK_ATTEMPTS:
@@ -450,7 +466,7 @@ def index_vault(
     """Bring the index in line with the vault.
 
     ``rebuild`` embeds every note again, replacing each one's rows in turn, so searches keep
-    finding the old version meanwhile. A new embedding model, or a new revision of it, is refused
+    finding the old version meanwhile. A new embedding model, revision or configuration is refused
     with ``ModelMismatch`` unless ``rebuild`` is set, and then discards the index first, since
     vectors from two models cannot be searched together. Notes cut by an earlier chunker are cut
     and embedded again. ``should_stop`` is polled between notes; once it returns True the run
@@ -476,23 +492,34 @@ def index_vault(
     known_model = store.get_meta("embedder")
     known_rev = store.get_meta("embedder_revision")  # None in an index from 0.1.1, which pinned none
     revision = embedder_revision(embedder)
-    changed = known_model is not None and (
+    config = embedding_config(embedder)
+    known_config = store.get_meta("embedding_config")  # None in an index built before the window was recorded
+    model_changed = known_model is not None and (
         model_key(known_model) != model_key(embedder.name) or (known_rev is not None and known_rev != revision)
     )
+    config_changed = known_model is not None and known_config is not None and known_config != config
+    changed = model_changed or config_changed
     if changed and store.stats()["files"]:
         if not rebuild:  # else a `watch` left on the old config and a fresh `index` wipe each other in turn
-            built = model_label(known_model) + (f" at revision {known_rev}" if known_rev else "")
-            wanted = model_label(embedder.name) + (f" at revision {revision}" if revision else "")
-            raise ModelMismatch(f"the index was built with {built} but {wanted} is configured; nothing was changed")
+            if model_changed:  # the larger change; a new window alongside it needs the same rebuild
+                built = model_label(known_model) + (f" at revision {known_rev}" if known_rev else "")
+                wanted = model_label(embedder.name) + (f" at revision {revision}" if revision else "")
+                raise ModelMismatch(f"the index was built with {built} but {wanted} is configured; nothing was changed")
+            raise ModelMismatch(
+                f"the index was built with another embedding configuration ({embedding_config_label(known_config)}, "
+                f"but {embedding_config_label(config)} is configured); run `puremind index --rebuild`; "
+                "nothing was changed"
+            )
         _embed(embedder, [PROBE_TEXT])  # a model that cannot run must not cost the user an index
     if changed:
         log.warning(
-            "embedder changed (%s%s -> %s%s); rebuilding the index",
+            "embedding model or configuration changed (%s%s -> %s%s); rebuilding the index",
             known_model, f" at {known_rev}" if known_rev else "", embedder.name, f" at {revision}" if revision else "",
         )
         store.clear()
     store.set_meta("embedder", model_key(embedder.name))
     store.set_meta("embedder_revision", revision)
+    store.set_meta("embedding_config", config)
     store.set_meta("vault_root", str(vault))
 
     scan = _scan(vault)