Repository navigation
feat: slm site (the model runs in the browser) and the Hugging Face publishing runbook - #9
Merged
Merged
Conversation
…verified live Every milestone-sized piece gets a runbook, and publishing is the one outward-facing step, so it gets the same treatment: what goes where, the exact commands, and checks a stranger can run against the public API with real output (metadata, tag, SHA-256 of the weights against the export, an anonymous transformers load). It also records the pitfalls met on the way (an organisation's display name versus its username, a password manager filling the account form), because those cost real round trips. The Space section stays open until the Space is live.
Creating a Docker Space under the organisation returned 402 Payment Required: Hugging Face now charges organisations for Docker Spaces, while personal accounts can still use the free CPU tier. The Space moves to the personal namespace and still serves the organisation's model. Recorded in ADR 0010 with the alternative (a static Space with in-browser inference) and as a runbook pitfall.
Hugging Face now charges for Docker Spaces, organisations and personal accounts alike; static Spaces stay free. slmkit's models are small enough to run in a visitor's browser, so a static site can serve one with no server at all. model.js is the same Llama-style network as model/llama.py, written out in JavaScript and reading an export's own files (config, tokenizer, safetensors), with a key/value cache so each token costs one position. The playground page is unchanged except that, when its info file says "engine": "browser", it generates in a Web Worker and gets back the same JSON /generate returns, so project viewers work as they are. slm site writes the page, the engine, the info file and the model's files as an immutable artifact. Parity is tested the way export is: in headless Chrome, logits at every position within 1e-4 of PyTorch (multi-head and grouped-query), and greedy generation token for token past the context length. On abc-folk:3 the difference is 2.4e-6, and the browser generates ~1,500 tokens/s. Rejected: transformers.js with an ONNX export (a second model format and a library standing in for the part slmkit writes itself); WebGPU (needless at 0.86M parameters); an embedded V8 in the tests instead of Chrome (a new dependency, and it wouldn't test a browser).
…lishing runbook browser-inference.md explains why a 0.86M-parameter model can run in a browser and a 7B one can't, maps every piece of model.js to llama.py, works through the key/value cache (300 positions instead of ~45,000 for a 300-token tune), how parity is proved and why the test scales its random weights only to x3, and why a seed can't reproduce PyTorch's samples. ADR 0011 replaces ADR 0010's Docker Space after both organisation and personal accounts returned 402. The runbook's H.5 now builds and checks the static site, with real output.
sandeeps85
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
abc-folk:3is live at https://huggingface.co/m87techlabs/abc-folk. The demo was planned as a DockerSpace running
slm serve, but Hugging Face returned402 Payment Requiredfor both the organisation(Team plan) and the personal account (PRO). Static Spaces are free. slmkit's models are small enough
to run in a browser, so this PR adds that, plus the runbook for publishing.
slm site: the model runs in the browserserve/static/model.jsis the same network asmodel/llama.py, written out in plain JavaScript:RMSNorm, rotate-half RoPE with float32 angle tables, grouped-query attention, SwiGLU, a tied head.
It reads the export's own files and keeps a key/value cache. At the context limit it re-reads the
last block, the way
generate.pycrops.The playground page: one branch. When
infosays"engine": "browser", it generates in a WebWorker and gets the same JSON as
POST /generate, so project viewers work unchanged.slm site <name:version>writes an immutable, content-addressed site artifact (lineage back tothe raw data): the page, the engine, a static
info, the model's files and the viewer. Char-levelmodels only; BPE models are refused.
Parity in headless Chrome (
tests/unit/test_site.py):Skipped without Chrome; CI's Ubuntu runners have it.
Measured on abc-folk:3:
transformersandslm serve;slm serveon the CPU.Docs
docs/runbooks/hf-hub.mdcovers:anonymous
transformersload;docs/concepts/browser-inference.md: why it works at this size,model.jsmapped tollama.py, thekey/value cache worked through, parity, and why seeds differ across engines.
step), STACK, GLOSSARY (Web Worker, static site, Space), CONTRIBUTING, README, CHANGELOG and ROADMAP.
Rejected:
out);
Checklist (CONTRIBUTING.md)
make lintandmake test(246 passed, including the browser parity tests) pass locally