Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,13 @@ interfaces; each entry says what moved. Milestones (M0, M1, …) are described i
- Model cards open with Hugging Face Hub metadata (licence, library, task, tags), so an export can be
uploaded as is. A new optional Project API hook, `model_card_info()`, gives the licence, subject tags
and data statement; `abc_music` and `shakespeare_char` state theirs (ADR 0010). Export code version 4.
- `slm site`: a static web site where the model runs in the visitor's browser, with no server.
`model.js` is the network written out in JavaScript, with a key/value cache, tested against PyTorch in
headless Chrome (logits within 1e-4, greedy output token for token). The playground page uses it when
`info` says `"engine": "browser"`; project viewers work unchanged (ADR 0011).
- `slm runs list --project NAME` lists one project's runs (#8, thanks @KeuqS).
- `abc-folk` on the Hugging Face Hub ([m87techlabs/abc-folk](https://huggingface.co/m87techlabs/abc-folk)),
and a runbook for publishing an export there (`docs/runbooks/hf-hub.md`).

## [0.1.0] - 2026-10-13

Expand Down
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ same pipeline:

**Read before architectural work:** `docs/DESIGN.md` (architecture, contracts, rules),
`docs/MODEL.md` (the model spec: architecture, presets, file formats; keep it in sync with
`src/slmkit/model/` and `presets/model/`) and `docs/ROADMAP.md` (current milestone and exit
`src/slmkit/model/`, `presets/model/` and the browser copy in `src/slmkit/serve/static/model.js`) and `docs/ROADMAP.md` (current milestone and exit
criteria). Work on the current milestone only.

## Vocabulary
Expand Down Expand Up @@ -142,6 +142,7 @@ uv run slm runs list --project abc_music # only runs for one project
uv run slm runs compare <run_a> <run_b>
uv run slm export <run_id> --name <model> --version <n>
uv run slm serve --model <model>:<n>
uv run slm site <model>:<n> # static site; the model runs in the browser
uv run slm studio start | stop | status | run # "See it in action": the local web studio
uv run slm lineage <artifact_id>
```
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@ uv run slm studio start # "See it in action": everything you built, in your br
| | |
|---|---|
| [`docs/concepts/two-models.md`](docs/concepts/two-models.md) | **No music background?** What the two models do, and how to read their output |
| [`docs/concepts/browser-inference.md`](docs/concepts/browser-inference.md) | The model in a browser: the same network in JavaScript, with no server (`slm site`) |
| `slm studio` | **See it in action.** A local web app over your runs, artifacts, models and machine ([runbook](docs/runbooks/studio.md)) |
| [`docs/LIFECYCLE.md`](docs/LIFECYCLE.md) | **Start here.** The model development lifecycle, with diagrams of the pipeline, architecture and roadmap |
| [`docs/GLOSSARY.md`](docs/GLOSSARY.md) | Every abbreviation and term, in plain language |
Expand Down
3 changes: 3 additions & 0 deletions docs/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -563,6 +563,9 @@ one allowed exception is `export --to-windows`.
first project that has one (chess, M3).
- `/` is a browser playground (one static page). A project may ship a viewer (`web_viewer()`,
ADR 0008) that the export carries as `ui/`: for ABC, sheet music and in-browser playback.
- `slm site abc-folk:3` builds a static site where the model runs in the browser: the same
playground page, `serve/static/model.js` (the network in JavaScript, parity-tested against PyTorch in
headless Chrome) and the model's files. No server at all; any static host serves it (ADR 0011).
- For anything worth keeping online, put an existing gateway in front rather than exposing
`slm serve` directly — it handles auth, rate limiting and TLS, which this app deliberately
does not.
Expand Down
12 changes: 10 additions & 2 deletions docs/GLOSSARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -438,8 +438,16 @@ hub. "HF format" is the de-facto standard model layout, which is why slmkit expo
repository. A model's page is its `README.md`, and a YAML block at the top gives its licence, task and
tags. The registry analogy: a container registry for model files.

**Space.** A demo app hosted on the Hub, built from its own git repository. A *Docker* Space builds the
repository's Dockerfile and sends visitors to one port; slmkit's runs `slm serve` (ADR 0010).
**Space.** A demo app hosted on the Hub, from its own git repository. A *static* Space serves the
repository's files as a web site, free; a *Docker* Space builds a Dockerfile and runs it as a server
(now a paid plan). slmkit's demo is a static Space built by `slm site`, with the model running in the
visitor's browser (ADR 0011).

**Web Worker.** A background thread for a web page. The page and the worker exchange messages, so long
work (here, generating text with `model.js`) doesn't freeze the page.

**Static site.** A web site that is only files (HTML, JavaScript, data), served as they are, with no
program running on the server. `slm site` builds one per model.

**safetensors.** A tensor file format that is safe to load from untrusted sources, unlike
Python's `pickle`.
Expand Down
4 changes: 4 additions & 0 deletions docs/MODEL.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ presets, and the files a trained model ends up as. It is the specification the c
`src/slmkit/model/` implements (M1 Phase B). For terms, see [`GLOSSARY.md`](GLOSSARY.md); for the
walk-through of the actual code, `concepts/the-model.md` (lands with Phase B).*

*A second implementation exists: `src/slmkit/serve/static/model.js` runs the same network in a browser
(`slm site`, ADR 0011). A change to the architecture here has to be made there too;
`tests/unit/test_site.py` fails if the two disagree.*

---

## At a glance
Expand Down
12 changes: 8 additions & 4 deletions docs/ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,13 +166,17 @@ Goal: a public repo a newcomer can evaluate in a minute and contribute to withou
## Hugging Face Hub: the folk-tune model and a live demo ◐

Goal: anyone can load `abc-folk` with `transformers`, or try it in a browser, without installing
anything. *No GPU.* Decisions: ADR 0010.
anything. *No GPU.* Decisions: ADR 0010. Runbook: `runbooks/hf-hub.md`.

- [x] Model cards carry Hub metadata, with the licence and data statement from the project
(`model_card_info()`); `abc-folk:3` exported with it (runbook M2 §E.7).
- [x] The demo recipe tested locally: CPU PyTorch, slmkit v0.1.0, `slm serve` (one jig in 0.36 s).
- [ ] `abc-folk:3` published on the Hub (tag `v3`), and loads there with `transformers`.
- [ ] The Space is live, and the README links to it.
- [x] The Docker recipe tested locally: CPU PyTorch, slmkit v0.1.0, `slm serve` (one jig in 0.36 s).
Not used: Docker Spaces need a paid plan.
- [x] `abc-folk:3` published on the Hub (tag `v3`), byte-identical to the export, and loads there
anonymously with `transformers` (runbook H.4).
- [x] `slm site`: the model runs in the browser from static files, parity-tested against PyTorch in
headless Chrome (ADR 0011). Docker Spaces turned out to need a paid plan; static Spaces are free.
- [ ] The static Space is live, and the README links to it.

---

Expand Down
15 changes: 11 additions & 4 deletions docs/STACK.md
Original file line number Diff line number Diff line change
Expand Up @@ -358,17 +358,18 @@ Triton Inference Server, which are built for fleets of GPU models, not one 3.5 M

### Hugging Face Hub, Spaces and the `hf` CLI
**◐** (after v0.1.0; not a dependency) · `huggingface_hub 2.2` via `uvx` ·
[Hub](https://huggingface.co/docs/hub) · [Docker Spaces](https://huggingface.co/docs/hub/spaces-sdks-docker) ·
[Hub](https://huggingface.co/docs/hub) · [static Spaces](https://huggingface.co/docs/hub/spaces-sdks-static) ·
[CLI](https://huggingface.co/docs/huggingface_hub/guides/cli)

**What it is.** Hugging Face's hosting for models and demo apps, each a git repository, and the
command-line client that uploads to it.

**Why.** Publishing an export where anyone can load it with `transformers` or try it in a browser.
Exports are already HF-format and their cards carry the Hub's metadata (ADR 0010), so publishing is
`hf upload`. The demo is a *Docker* Space running `slm serve` itself. Run through `uvx`, the CLI
installs nothing into slmkit. Rejected: a Gradio Space (a second UI, and a framework slmkit doesn't
use); making `huggingface_hub` a dependency for an occasional, manual step.
`hf upload`. The demo is a *static* Space holding what `slm site` builds, with the model running in
the visitor's browser (ADR 0011). Run through `uvx`, the CLI installs nothing into slmkit. Rejected: a
Docker Space running `slm serve` (it now needs a paid plan); a Gradio Space (a second UI, and a framework
slmkit doesn't use); making `huggingface_hub` a dependency for an occasional, manual step.

### HTTPX
**●** extra `dev` · `0.28` · [docs](https://www.python-httpx.org/)
Expand Down Expand Up @@ -400,6 +401,12 @@ zoom and axis ticks are exactly the parts worth not writing).
studio and playground page is the check that the JavaScript works. The runbooks' screenshots were made
this way (studio.md §S.6). Nothing is installed; both browsers ship with Windows.

`tests/unit/test_site.py` also uses it (`--dump-dom`) to run `model.js`, the in-browser copy of the
model, against PyTorch (ADR 0011). It finds `google-chrome` or `chromium` on Linux (CI's Ubuntu runners
have Chrome) or the Windows browsers from WSL, honours `SLM_CHROME`, and skips if there is none.
Headless Chrome's virtual time doesn't advance Web Workers, so screenshots of the browser engine
generating were taken over the DevTools protocol instead, with a throwaway script outside the repo.

### Docker and Kubernetes
**◐ M4** (optional) · [Docker](https://docs.docker.com/) · [Kubernetes](https://kubernetes.io/docs/) ·
[kind](https://kind.sigs.k8s.io/) · [k3s](https://docs.k3s.io/)
Expand Down
177 changes: 177 additions & 0 deletions docs/concepts/browser-inference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,177 @@
# The model in a browser: inference with no server

*After v0.1.0. Code: [`src/slmkit/serve/static/model.js`](../../src/slmkit/serve/static/model.js) (the
network), [`engine-worker.js`](../../src/slmkit/serve/static/engine-worker.js) (runs it off the page's
thread), [`src/slmkit/serve/site.py`](../../src/slmkit/serve/site.py) (`slm site`). Tests:
[`tests/unit/test_site.py`](../../tests/unit/test_site.py). Decision:
[ADR 0011](../decisions/0011-browser-inference.md). Hands-on:
[`../runbooks/hf-hub.md`](../runbooks/hf-hub.md) §H.5. Terms: [`../GLOSSARY.md`](../GLOSSARY.md).*

`slm serve` runs a model on a server: a browser sends a prompt, Python and PyTorch compute, the text
comes back. For the models slmkit trains, there is a simpler option. The **browser** can do the
computing itself, so the "server" is just a folder of static files. `slm site abc-folk:3` builds that
folder.

The infrastructure analogy: moving work from an API server to the client, the way a single-page app
moved rendering into the browser. It only works when the client can afford the work, and here it can:
abc-folk has 864,256 parameters and its weights are a 3.5 MB file, smaller than many web pages' images.

---

## 1. Why it's possible here, and not for big models

Generating one token means multiplying the current state through every weight matrix once: about **one
multiply-add per parameter**, plus attention over the tokens so far. For abc-folk that is under a million
multiply-adds per token, which plain JavaScript does in well under a millisecond.

| | abc-folk | a 7-billion-parameter chat model |
|---|---|---|
| weights to download | 3.5 MB | ~14 GB (16-bit) |
| work per token | ~0.9 M multiply-adds | ~7,000 M |
| in a browser, plain JavaScript | **~1,500 tokens/s, measured** | impractical |

Big models in browsers exist, but they need the GPU (WebGPU), weights squeezed to 4 bits, and a large
download. slmkit's models need none of that.

**Measured:** in Chrome on the reference machine, one jig (189 tokens) in 0.12 s, about **1,500
tokens/s**. `slm serve` on the CPU makes the same tune at about **450 tokens/s**. The browser is faster
because at this size PyTorch's per-operation overhead, not arithmetic, dominates (serving.md §6).

---

## 2. The same network, written a second time

`model.js` is `src/slmkit/model/llama.py` again, in plain JavaScript with no library. It reads an
export's own files: `config.json` for the sizes, `tokenizer.json` for the characters, and
`model.safetensors` for the weights. That last format is simple enough to read by hand: an 8-byte
length, a JSON header naming each tensor's shape and byte range, then raw float32 numbers
(MODEL.md §6).

Every piece maps to a class in `llama.py`, and each has to match exactly, or the logits differ:

| llama.py | model.js | the detail that has to match |
|---|---|---|
| `RMSNorm` | `rmsnorm()` | the epsilon from `config.json`, the learned gain, no mean subtraction |
| `RotaryEmbedding` | `rope()`, tables in the constructor | pairs `(i, i + Hd/2)` ("rotate_half"), and angles rounded to float32 as PyTorch computes them |
| `Attention` | the head loop in `step()` | query head `h` reads key/value head `h ÷ (H/K)` (grouped-query) and scales by `1/√Hd` |
| `MLP` | the SwiGLU lines in `step()` | `down(silu(gate(x)) × up(x))` |
| tied `lm_head` | `this.head = this.embed` | the output layer *is* the embedding matrix, stored once |
| `filter_logits` | `filterLogits()` | temperature, then top-k, then top-p, in that order |

A linear layer is the one operation everything else is built from. In PyTorch it is `x @ W.T`; here it
is a loop:

```js
function matvec(W, x, out, rows, cols) { // W is [rows, cols], row-major, as nn.Linear stores it
for (let r = 0; r < rows; r++) {
let sum = 0;
for (let c = 0; c < cols; c++) sum += W[r * cols + c] * x[c];
out[r] = sum;
}
}
```

The weights stay float32, as stored. The arithmetic runs in JavaScript's float64, which is at least as
precise as PyTorch's float32, so results agree to rounding.

---

## 3. The key/value cache: why each token costs one position

`llama.py`'s sampler re-runs the whole context for every new token. On a GPU that costs nothing worth
saving, and the loop stays easy to read. In JavaScript it would be wasteful. For a 300-token tune, the
network would process 1 + 2 + … + 300 ≈ **45,000 positions** to produce 300 tokens.

The fix is the standard one. In attention, a position's *key* and *value* vectors depend only on that
position and the ones before it. They never change once computed, so `model.js` stores them. Each new
token then runs the network for **one position**, and reads the stored keys and values of all earlier
ones. That is 300 positions instead of 45,000.

The cache is `layers × positions × kv heads × head size` numbers for keys, and again for values. For
abc-folk that is 4 × 512 × 4 × 32 × 2 × 8 bytes ≈ **4 MB**. It is kept in float64, like the rest of the
arithmetic, so it adds no rounding of its own.

**At the context limit.** abc-folk sees at most 512 tokens. When the text passes that, `generate.py`
keeps the last 512 tokens, and they take positions 0 to 511 again. Every cached key was rotated for
its old position, so the cache is no longer valid. `model.js` does the same as `generate.py`: it
re-reads the last 512 tokens from scratch. That costs 512 positions instead of 1 for each token past
the limit, but most tunes end well before it.

---

## 4. Proving it is the same model

A second implementation is only useful if it computes the same function. The test does what
`slm export` does with `transformers` (serving.md §4), in a real browser. `tests/unit/test_site.py`
serves a model to headless Chrome, runs `model.js`, and compares with PyTorch:

- **logits at every position of a full context, within 1e-4**, for plain multi-head and for
grouped-query attention;
- **greedy generation, token for token**, for 24 tokens on a model with an 8-token context, so the
re-reading in §3 is exercised.

On the real abc-folk:3, the largest logit difference is **2.4e-6**. Greedy output from the Hub's
`transformers`, from `slm serve` and from the browser is the same text.

**A test that can't fail proves nothing.** A freshly initialised model gives nearly uniform logits: every
token scores about the same, so even a wrong implementation would land within 1e-4. The test scales the
random weights so that the logits span about ±5, like a trained model's. It doesn't scale them much
further. At ×8, logits reach ±29, and PyTorch itself, run in float32 and in float64, disagrees by 3e-4,
more than the tolerance.

---

## 5. Seeds: the same seed, a different tune

With temperature 0 (greedy), the browser and the server write the same text. With sampling, they don't:
`seed 3` in the browser gives a different tune than `seed 3` from `slm serve`.

A seed selects a sequence of random numbers, and the sequence comes from a specific generator. PyTorch's
is a Mersenne Twister with its own way of turning bits into a sampled token (`torch.multinomial`).
`model.js` uses a small generator (mulberry32) and samples by walking the cumulative probabilities.
Reproducing PyTorch's generator bit for bit in JavaScript would be possible, but it would be a
reimplementation of PyTorch internals for no benefit to anyone. So a seed **reproduces a result within
the browser** (the playground's "Copy link" still works), and the distribution the tokens are drawn from
is the same.

---

## 6. The page doesn't change

The playground page (serving.md §8) asks for `info`, then posts to `generate`. A site built by
`slm site` has a static file named `info` with two extra fields: `"engine": "browser"` and
`"files": "model/"`. When the page sees them, it starts `engine-worker.js` (a **Web Worker**: a
background thread for a page, so the page stays responsive while it computes) and sends the request
there instead. The worker returns the same JSON that `POST /generate` would, field for field, so the
page, its statistics and the project's viewer (sheet music and playback for ABC) work unchanged.

```
$SLM_HOME/sites/site-5c053f5e701f/
├── index.html the playground page, as slm serve sends it
├── model.js the network (§2, §3)
├── engine-worker.js runs model.js off the page's thread
├── info what /info returns, plus "engine": "browser"
├── model/ config.json, generation_config.json, tokenizer.json, model.safetensors
├── ui/viewer.js the project's viewer, from the export
└── manifest.json which model, which code
```

A site is an artifact: its ID hashes the model and the code version, it is written once, and
`slm lineage site-5c053f5e701f` walks from it back to the raw data. Any static host serves it: a
Hugging Face static Space (§H.5 of the runbook), GitHub Pages, or `python3 -m http.server`.

---

## 7. What it doesn't do yet

- **BPE models.** `model.js` reads slmkit's character-level tokenizer only. A BPE tokenizer needs its
merges applied in JavaScript; `slm site` refuses such models with a clear message until then.
- **Constrained decoding.** As with `slm serve`, a project's `logits_processor` is Python and doesn't
run here (serving.md §10).
- **Models much larger than ~10M parameters.** Nothing stops them, but the download and the work per
token grow with the parameter count; beyond tens of millions, a server or WebGPU is the better tool.

**Rejected:** transformers.js with an ONNX export. It would add a second model format and a library
standing in for exactly the part slmkit exists to write out. **Rejected:** WebGPU. It's needless at this
size, and not every browser has it. **Rejected:** a Docker Space running `slm serve`, the first plan.
It works, but Hugging Face now charges for Docker Spaces, and a server is more than a 3.5 MB model needs.
Loading
Loading