Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,12 +227,13 @@ The default is to try and saturate your cpus.

### Proxy Support

The model setup script honors standard proxy environment variables:
The model setup scripts, and uv when it installs their dependencies, honor
standard proxy environment variables:

```bash
export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=http://proxy.company.com:8080
python scripts/direct_download.py
scripts/direct_download.py
```

## MCP (Model Context Protocol) Server
Expand Down Expand Up @@ -304,14 +305,18 @@ You can test the MCP server outside of Claude:

Note: this is mostly untested, and not required for general usage. I've been
trying nomic v2 and running it through model2vec to make things faster without
a GPU. There are scripts:
a GPU. There are scripts, which need [uv](https://docs.astral.sh/uv/)
to install their dependencies:

```
pip3 install -r scripts/requirements.txt
scripts/direct_download.py
scripts/nomic2vec.py
```

Where `env -S` is unavailable (e.g. BusyBox), run them as
`uv run --script scripts/<name>.py`. Without uv, install the dependencies
with `pip3 install -r scripts/requirements.txt` and run them with `python3`.

Take the resulting model directory and move it to ~/.cache/semcode/models/model2vec.

Then do a index run:
Expand Down
8 changes: 7 additions & 1 deletion scripts/direct_download.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
#!/usr/bin/env python3
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "huggingface_hub",
# ]
# ///
from huggingface_hub import snapshot_download
path = snapshot_download("nomic-ai/nomic-embed-text-v2-moe", revision="main")
path = snapshot_download("nomic-ai/nomic-bert-2048", revision="main")
33 changes: 23 additions & 10 deletions scripts/nomic2vec.py
Original file line number Diff line number Diff line change
@@ -1,21 +1,35 @@
#!/usr/bin/env python3
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = [
# "torch>=2.0.0",
# "transformers>=4.30.0",
# "model2vec[distill]>=0.1.0",
# "sentence-transformers>=2.2.0",
# "sentencepiece>=0.1.97",
# "einops>=0.8.1",
# ]
# ///
"""
Distill a locally-downloaded Nomic Embed v2 model from Hugging Face into a fast Model2Vec static model (offline).
Artifacts must be downloaded first using direct_download.py, which populates:
~/.cache/huggingface/hub/models--nomic-ai--nomic-embed-text-v2-moe/snapshots/latest
~/.cache/huggingface/hub/models--nomic-ai--nomic-embed-text-v2-moe/snapshots/<commit>

Usage:
# Use default cache location (after running direct_download.py):
python nomic2vec.py
scripts/nomic2vec.py

# Or specify custom location:
python nomic2vec.py -m /path/to/model/dir
scripts/nomic2vec.py -m /path/to/model/dir

# Optional parameters:
python nomic2vec.py -o ./embed_v2_m2v -v ./custom_vocab.txt --pca-dims 256 --device cuda
scripts/nomic2vec.py -o ./embed_v2_m2v --pca-dims 256 --device cuda

Requires:
pip install -U model2vec transformers sentence-transformers
uv, which installs the dependencies declared inline above. Where
`env -S` is unavailable, run `uv run --script scripts/nomic2vec.py`.
Without uv: pip3 install -r scripts/requirements.txt, then
python3 scripts/nomic2vec.py
"""

import os
Expand Down Expand Up @@ -151,7 +165,7 @@ def main():
if not model_dir.exists():
print(f"✗ Model directory does not exist: {model_dir}")
print(" Make sure to run direct_download.py first to download the model files.")
print(" Example: python direct_download.py")
print(" Example: scripts/direct_download.py")
sys.exit(1)

if not (model_dir / "config.json").exists():
Expand Down Expand Up @@ -197,9 +211,8 @@ def main():
except Exception as e1:
print(f"Failed to load with trust_remote_code=True: {e1}")
print("This model requires custom code from nomic-ai/nomic-bert-2048.")
print("Make sure you have downloaded both model repositories:")
print("1. python scripts/direct_download.py --main-only")
print("2. python scripts/direct_download.py --impl-only")
print("Make sure you have downloaded both model repositories")
print("(scripts/direct_download.py fetches both).")
print("\nTrying alternative loading strategy...")

try:
Expand Down
Loading