Add CPU Rust BM25 search task

.env.example

@@ -41,7 +41,7 @@ AGENT_CODEX_CONFIG=
 # ---------------------------------------------------------------------------
 # 2. RewardKit 0.2.0 trajectory judge
 # ---------------------------------------------------------------------------
-# Required by every current task except agentic-search, regardless of which coding
+# Required by every current task except agentic-search and rust-bm25-search, regardless of which coding
 # agent is selected. The judge is fixed to deepseek/deepseek-v4.1-flash on
 # OpenRouter. Use an OpenRouter key for VERIFIER_OPENAI_API_KEY.
 VERIFIER_OPENAI_BASE_URL=https://openrouter.ai/api/v1

README.md

@@ -90,7 +90,7 @@ VERIFIER_OPENAI_API_KEY=YOUR_OPENROUTER_KEY
 `DEEPSEEK_API_KEY` authenticates the Pi agent for `deepseek/deepseek-flash`.
 The `VERIFIER_*` pair runs the independent `deepseek/deepseek-v4.1-flash` trajectory judge
 through OpenRouter and RewardKit 0.2.0 and is required by every task except
-`agentic-search`. Use an OpenRouter key for `VERIFIER_OPENAI_API_KEY`; the Pi agent
+`agentic-search` and `rust-bm25-search`. Use an OpenRouter key for `VERIFIER_OPENAI_API_KEY`; the Pi agent
 uses its separate DeepSeek key. The launcher passes verifier credentials only
 to the verifier container.
 

README_zh.md

@@ -81,7 +81,7 @@ VERIFIER_OPENAI_API_KEY=YOUR_OPENROUTER_KEY
 ```
 
 `DEEPSEEK_API_KEY` 用于 `deepseek/deepseek-flash` 的 Pi 智能体认证；`VERIFIER_*`
-通过 RewardKit 0.2.0 和 OpenRouter 运行固定为 `deepseek/deepseek-v4.1-flash` 的独立轨迹评审，除 `agentic-search`
+通过 RewardKit 0.2.0 和 OpenRouter 运行固定为 `deepseek/deepseek-v4.1-flash` 的独立轨迹评审，除 `agentic-search` 和 `rust-bm25-search`
 外的所有任务都需要。`VERIFIER_OPENAI_API_KEY` 使用 OpenRouter 密钥，Pi 智能体
 使用独立的 DeepSeek 密钥；启动器只把 verifier 凭据传进 verifier 容器。
 

docs/evaluation.md

@@ -69,6 +69,7 @@ the shared images locally.
 | `query-encoder-alignment` | 1 NVIDIA GPU | Yes | — | — |
 | `agentic-search` | CPU | — | — | `OPENROUTER_API_KEY`, `JINA_API_KEY` |
 | `sparse-retrieval-pruning` | CPU | Yes | — | — |
+| `rust-bm25-search` | CPU | — | — | — |
 
 The table identifies credential groups, not the complete resource budget. Read
 `tasks/TASK_NAME/README.md` and `task.toml` before launching. In particular,
@@ -213,7 +214,7 @@ steps and observations. Failed runs are exported before the error is reported.
 
 ## Configure judges
 
-All current tasks except `agentic-search` use the trajectory judge. Add:
+All current tasks except `agentic-search` and `rust-bm25-search` use the trajectory judge. Add:
 
 ```dotenv
 VERIFIER_OPENAI_BASE_URL=https://openrouter.ai/api/v1

docs/network-policy.md

@@ -130,6 +130,7 @@ access.
 | `query-encoder-alignment` |  OpenRouter/Jina resources only | allowlist: `openrouter.ai`, `api.jina.ai` | task hosts + model host | task hosts (OpenRouter also serves the trajectory judge) |
 | `agentic-search` | OpenRouter/Jina resources only | allowlist: `openrouter.ai`, `api.jina.ai` | task hosts + model host | task hosts only; no model judge |
 | `sparse-retrieval-pruning` | Offline index building and search | no network | model host only | `openrouter.ai` only |
+| `rust-bm25-search` | Offline Rust BM25 indexing and search | no network | model host only | no network; deterministic verifier |
 
 Allowing the judge endpoint does not expose verifier credentials to submitted
 commands. `scientific-paper-qa` similarly removes both judge credential groups before

scripts/tests/test_network_policies.py

@@ -40,6 +40,7 @@
     "query-encoder-alignment": ("allowlist", TASK_HOSTS, "allowlist", TASK_HOSTS, "allowlist", TASK_HOSTS),
     "agentic-search": ("allowlist", TASK_HOSTS, "allowlist", TASK_HOSTS, "allowlist", TASK_HOSTS),
     "sparse-retrieval-pruning": ("no-network", [], "no-network", [], "allowlist", ["openrouter.ai"]),
+    "rust-bm25-search": ("no-network", [], "no-network", [], "no-network", []),
 }
 
 
@@ -103,7 +104,7 @@ def test_task_resource_docs_match_the_network_profiles(self):
             )
             self.assertTrue(resource_doc.is_file(), task_name)
 
-        for task_name in ("memory-constrained-dense-retrieval", "long-document-reranking", "code-embedding-finetuning", "sparse-retrieval-pruning"):
+        for task_name in ("memory-constrained-dense-retrieval", "long-document-reranking", "code-embedding-finetuning", "sparse-retrieval-pruning", "rust-bm25-search"):
             text = (
                 REPO / "tasks" / task_name / "environment/docs/available_resources.md"
             ).read_text().lower()

tasks/rust-bm25-search/.gitignore

@@ -0,0 +1,8 @@
+/data/
+/models/
+__pycache__/
+*.py[cod]
+/target/
+/authoring/output/
+/authoring/downloads/
+/:memory:.ses

tasks/rust-bm25-search/README.md

@@ -0,0 +1,134 @@
+# Rust BM25 search
+
+This CPU task asks an agent to implement a complete Rust inverted index and
+BM25 searcher from the 8,841,823 raw MS MARCO Passage v1 passages. Generic crates
+are allowed; existing search engines and prebuilt indexes are not. A separate
+verifier recompiles the source, rebuilds an empty index, and evaluates all
+6,980 standard dev queries.
+
+**Contribution status: submitted for review; official asset pins pending.** The exact Pyserini
+quality floor has been measured, and an independently built author Rust
+implementation passed it on all dev queries (MRR@10 0.1872948219). A Harbor
+Oracle trial also passed the actual source-artifact transfer and independent
+verifier flow. An independent GPT-5.6 Sol / max trial through Codex collaboration
+also completed the full verifier: MRR@10 0.1839805908 falls just below the exact
+floor, so it earns zero. Frozen model and author sources were revalidated after
+review fixes; full outputs match their previous runs. Current timings and
+regression evidence are in [review-fixes-observation.json](authoring/review-fixes-observation.json).
+The coding trial used a context-free hosted agent with an isolated task container; Harbor's Codex API
+route remains untested because the API preflight found exhausted credits.
+Official asset publication and terms review remain necessary before merge. See
+[validation.md](authoring/validation.md) for the evidence and limitations.
+Author: MXueguang (`x93ma@edu.uwaterloo.ca`).
+
+The reference is Pyserini 1.4.0, BM25 k1=0.9, b=0.4, default English analysis,
+text only. It omits stored raw text, positions and document vectors. Its
+MRR@10 is exactly **3236243/17589600 = 0.18398616227770956**. The candidate must
+meet this exact floor, rather than a rounded 0.18 or 0.1840.
+
+For valid candidates, quality, hot batch time and index bytes have equal
+geometric weights, normalized as `U/(1+U)` with reference-equivalent score 0.5.
+Reference timing and disk are remeasured inside each verifier run, rather than
+copied from one author's hardware. Five fresh-process startup measurements,
+hot p50/p95, sequential QPS, build time and sampled process RSS are also
+reported. Cold means a new process with no query warmup, **without evicting the
+OS file cache**. QPS is concurrency-one throughput, not server saturation.
+RSS is diagnostic process memory, while the container cap includes children
+and page cache. These limitations are explicit in both instructions.
+
+Both phases use the shared CPU image, Rust 1.85.1 and the same offline Cargo
+lock. Resource envelope: 4 CPUs, 16 GiB memory, 64 GiB workspace, no network or
+GPU. Agent budget is 3 hours; compilation 10 minutes, index build 2 hours,
+individual requests 120 seconds and each warmup/measured batch 1 hour. These
+are engineering budgets to validate with a real model, not universal claims
+about hardware performance. The host needs Linux Landlock ABI 1 or newer and
+seccomp filters; root-owned files additionally enforce read-only search on older ABIs.
+
+Only `/app/submission` transfers to the verifier. Candidate code runs as UID
+10001, separate from root-owned tests, qrels and reward files. Search is
+restricted to its immutable index, binary and system libraries. Source-only
+recompilation and these controls support the contract; they do not replace
+review that the implementation is actually self-built BM25 and contains no
+hard-coded answers. No API or model judge is part of deterministic grading.
+All candidate phases block persistent IPC, kernel keyrings and extended-attribute
+access, including through io_uring. Index files, directories and the root must
+have no extended attributes. These checks prevent IPC and xattr payloads from
+bypassing the measured index-file bytes.
+
+## Restore and check
+
+From the repository root, with the host dependencies documented by the repo:
+
+```bash
+python scripts/download_assets.py --task rust-bm25-search
+python scripts/download_assets.py --task rust-bm25-search --verify-only
+python -m unittest discover -s tasks/rust-bm25-search/authoring -p 'test_*.py'
+python scripts/check_tasks.py tasks/rust-bm25-search
+python scripts/check_release.py
+docker build -t search-swe-local:rust-bm25-agent tasks/rust-bm25-search/environment
+docker build -t search-swe-local:rust-bm25-verifier tasks/rust-bm25-search/tests
+docker run --rm --network none --cpus 4 --memory 16g --memory-swap 16g --pids-limit 256 \
+  --mount "type=bind,source=$PWD/tasks/rust-bm25-search/authoring/check_persistent_state.py,target=/check.py,readonly" \
+  search-swe-local:rust-bm25-verifier python /check.py
+```
+
+The data manifest currently pins existing Tevatron repositories. No upload
+has been made. Hidden queries/qrels are small files in `tests/data`, with
+immutable source URLs and fingerprints. Restore them if necessary using
+`python tasks/rust-bm25-search/authoring/prepare_queries.py`. Input terms and
+toolchain provenance are in [SOURCES.md](authoring/SOURCES.md). Official
+publication must preserve applicable MS MARCO/TREC terms and use the reviewed
+`search-swe/Search-SWE` task path before merge.
+
+## Reproduce the reference
+
+The author wrappers invoke the same trusted converter and reference code used
+by the verifier. Create fresh output paths; these commands refuse to overwrite
+existing indexes or measurements. The measured reference is recorded in
+[reference-observation.json](authoring/reference-observation.json); its timing
+is a local observation, not a portable threshold.
+
+```bash
+python tasks/rust-bm25-search/authoring/prepare_corpus.py \
+  --corpus tasks/rust-bm25-search/data/corpus.jsonl.gz \
+  --output tasks/rust-bm25-search/authoring/output/collection
+```
+
+Run these commands inside the pinned verifier image with the task mounted at
+`/task-package`, 4 CPUs, 16 GiB memory, no network, and a hostname resolvable
+through `/etc/hosts`. No API credential is needed:
+
+```bash
+python /task-package/authoring/reference.py build \
+  --collection /task-package/authoring/output/collection \
+  --index /task-package/authoring/output/reference-index --threads 4
+python /task-package/authoring/reference.py measure \
+  --index /task-package/authoring/output/reference-index \
+  --queries /task-package/tests/data/dev.queries.jsonl \
+  --qrels /task-package/tests/data/dev.qrels.tsv \
+  --warmup /task-package/data/warmup.jsonl.gz \
+  --output /task-package/authoring/output/reference-measurement
+```
+
+Pyserini 1.4.0 imports optional encoder modules alongside LuceneSearcher; one
+constructs an unused OpenAI client. The reference worker supplies a non-secret
+sentinel solely to satisfy that constructor. BM25 makes no API request and
+runs with networking disabled. Startup measurements include Python/JVM/module
+startup as well as index opening, as required by the process-cold definition.
+
+## Reviewed precedent and instruction audit
+
+The structural precedent is `tasks/sparse-retrieval-pruning`, reviewed on base
+commit `46197a43f8cbb364e72b6d63d5db873ae7d2cac8`. Reused patterns: CPU runtime,
+explicit source artifact, separate verifier, clean rebuild, unprivileged
+candidate and baseline-relative optimization behind a quality gate. Differences:
+raw text and complete Rust BM25 indexing, standard passage dev MRR, measured
+Pyserini reference, startup measurements and disk in the objective. None of
+that task's data IDs, thresholds, resource/API policies or author fields was
+copied. This draft deliberately needs no trajectory-judge API.
+
+The structured contract is [raw-instruction.md](raw-instruction.md). The
+agent-facing [instruction.md](instruction.md) is a prose rewrite; the mapping
+and review boundaries are in [instruction-equivalence.md](authoring/instruction-equivalence.md).
+Author candidates, full indexes and trial logs live in ignored local paths;
+they are not included in the agent image or a public solution directory.

tasks/rust-bm25-search/assets.json

@@ -0,0 +1,27 @@
+{
+  "schema_version": 1,
+  "files": [
+    {
+      "path": "data/corpus.jsonl.gz",
+      "size_bytes": 1118496727,
+      "sha256": "634543cee77a992613bbf32d0aa4863659dffd064c405f062454400aa80aea3f",
+      "source": {
+        "repo_id": "Tevatron/msmarco-passage-corpus",
+        "repo_type": "dataset",
+        "revision": "21a2e1a8c6e7b1747c92f82cd2259a97077dc48d",
+        "filename": "corpus.jsonl.gz"
+      }
+    },
+    {
+      "path": "data/warmup.jsonl.gz",
+      "size_bytes": 1492,
+      "sha256": "331e829922e2bd8ee1f01766fb7253c0d672ef79498011e91ad5e3e93f17a54a",
+      "source": {
+        "repo_id": "Tevatron/msmarco-passage",
+        "repo_type": "dataset",
+        "revision": "330a160b40a739d4c07d52c7ce562e2c549214c0",
+        "filename": "dl20.jsonl.gz"
+      }
+    }
+  ]
+}

tasks/rust-bm25-search/authoring/SOURCES.md

@@ -0,0 +1,45 @@
+# Input and tool provenance
+
+The fixed runtime inputs are listed with exact sizes, SHA-256 digests and
+immutable HF commits in `../assets.json`. The corpus is the existing
+`Tevatron/msmarco-passage-corpus` mirror, revision
+`21a2e1a8c6e7b1747c92f82cd2259a97077dc48d`. The 54 DL20 query texts come from
+`Tevatron/msmarco-passage`, revision
+`330a160b40a739d4c07d52c7ce562e2c549214c0`. No new public upload was made.
+
+The corpus gzip was hashed locally and all 8,841,823 unique canonical IDs were
+checked during conversion. Only the `text` field is indexed; the mirror's
+additional `title` field is deliberately excluded. The query split was checked
+for overlap by query ID and normalized text.
+
+Verifier-only dev topics and qrels originate in `castorini/anserini-tools`,
+commit `8f1bc80988c2483d9519bc2519862b54ab730986`. Exact source URLs, source
+hashes and the derived query-file hash are in `../tests/data/sources.json`.
+`prepare_queries.py` restores them without accessing a mutable branch. The
+small verifier inputs are included only in the verifier image; they are not
+mounted in the agent environment.
+
+MS MARCO is a Microsoft dataset. Its official
+[dataset page and terms](https://microsoft.github.io/msmarco/Datasets#terms-and-conditions),
+checked on 2026-10-11, restrict use to non-commercial research and say that
+availability does not extend a license or other intellectual-property rights.
+The page separately licenses repository documentation and code; those notices
+do not establish a redistribution grant for the underlying passages. Tevatron
+mirroring does not establish new redistribution rights either. Maintainers must
+resolve the applicable MS MARCO/TREC terms and the proposed official mirror
+before publication. No new dataset license or redistribution grant is asserted
+here. The dataset page requests citation of the
+[MS MARCO dataset paper](https://arxiv.org/abs/1611.09268).
+
+Both images use `docker.io/hanhainebula/search-swe-base:cpu-py3.12-1.0.0`.
+The locally inspected base digest is
+`sha256:1350a3043b3c49388b48dd5a0c2e3d91e9071f87359dbe63a2c26f055acb0cd2`.
+It supplies Pyserini 1.4.0 and Java 21. The Rust 1.85.1 distribution's source
+URL and verified checksum are in the two matching `install-rust.sh` files.
+Cargo.lock pins every dependency by version and registry checksum. The agent
+and verifier use the same toolchain/cache seed.
+
+`reference-observation.json` records the full local baseline run and its
+input hashes. `../tests/reference-quality.json` pins its exact quality result,
+not its host-specific timings. Each valid candidate verifier run rebuilds and
+measures the reference again on the same container resource profile.

tasks/rust-bm25-search/authoring/check_isolation.py

@@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+"""Run inside a disposable verifier image; never execute on the host."""
+
+import json
+import os
+import subprocess
+import sys
+from pathlib import Path
+
+PROBE = r'''
+use std::fs::{self,OpenOptions};
+fn main() {
+    assert!(fs::read("/tests/data/dev.qrels.tsv").is_err(), "hidden labels readable");
+    assert!(fs::write("/logs/verifier/reward.txt", b"1").is_err(), "reward writable");
+    if std::env::args().nth(1).unwrap()=="search" {
+        assert_eq!(fs::read("/work/index/probe").unwrap(), b"index");
+        assert!(fs::read("/task/data/corpus.jsonl.gz").is_err(), "raw corpus readable in search");
+        assert!(fs::read("/tmp/outside-index").is_err(), "outside cache readable");
+        assert!(OpenOptions::new().write(true).truncate(true).open("/work/index/probe").is_err(), "index truncatable");
+        assert!(fs::write("/tmp/new-cache", b"cached").is_err(), "persistent cache writable");
+    } else {
+        assert!(fs::read("/task/data/corpus.jsonl.gz").is_ok(), "corpus not readable when indexing");
+        fs::write("/work/index/probe",b"index").unwrap();
+    }
+    println!("isolation probe passed");
+}
+'''
+
+
+def main():
+    if os.geteuid() != 0 or not Path("/.dockerenv").exists():
+        raise RuntimeError("Use a fresh disposable verifier container")
+    sys.path.insert(0, "/tests")
+    from grader import ENV, kill_submission_processes
+    Path("/work").mkdir(mode=0o711)
+    for name in ["index", "tmp"]:
+        path = Path("/work") / name
+        path.mkdir()
+        os.chown(path, 10001, 10001)
+    Path("/logs/verifier").mkdir(parents=True, mode=0o700)
+    Path("/logs/verifier/reward.txt").write_text("0\n")
+    Path("/task/data").mkdir(parents=True, exist_ok=True)
+    Path("/task/data/corpus.jsonl.gz").write_bytes(b"fixture corpus")
+    Path("/tmp/outside-index").write_text("cached state")
+    Path("/tmp/probe.rs").write_text(PROBE)
+    subprocess.run(["/opt/rust/bin/rustc", "/tmp/probe.rs", "-o", "/work/bm25-search"], check=True)
+    os.chmod("/work/bm25-search", 0o555)
+    for mode in ["index", "search"]:
+        if mode == "search":
+            os.chown("/work/index", 0, 0)
+            os.chown("/work/index/probe", 0, 0)
+            os.chmod("/work/index", 0o555)
+            os.chmod("/work/index/probe", 0o444)
+        try:
+            subprocess.run([sys.executable, "/tests/isolate.py", mode, "/work/bm25-search", mode],
+                           check=True, timeout=20, env=ENV)
+        finally:
+            kill_submission_processes()
+    assert Path("/logs/verifier/reward.txt").read_text() == "0\n"
+    print(json.dumps({"index_and_search_isolation": "passed", "reward_unchanged": True}))
+
+
+if __name__ == "__main__":
+    main()