Assay the weights before you trust them.
assay is an offline-first, single-binary scanner for ML model artifacts
(safetensors, GGUF, PyTorch pickle). It answers two questions about a model
you just downloaded:
- Is this file what it claims to be? (provenance and integrity)
- Does loading it put my machine at risk? (format-level safety, and the code the repo would have your loader execute)
A downloaded model is a multi-gigabyte opaque blob that people execute with
total trust. We would never do that with a random .exe. assay applies the
same supply-chain hygiene to model weights.
The name comes from metallurgy: an assay tests the purity and composition of a metal. A model is literally weights, so
assaytests whether those weights are pure (no contaminant) and authentic (real provenance).
assay never loads, imports, or executes a model. It reads bytes, and nothing
else. Files are memory-mapped and streamed, so peak RAM stays well under model
size even on a 70B baseline.
# from source
git clone https://github.com/Sn0wAlice/assay && cd assay
cargo install --path .
# from crates.io
cargo install assayNo runtime dependencies, no Python, no network access during a scan.
Each tagged release also publishes prebuilt tarballs for macOS and Linux (x86_64 and arm64) plus a Homebrew formula, built and hashed by the release workflow. Grab one from the releases page if you would rather not build.
# scan a single file
assay scan model.safetensors
# scan a whole model directory (Hugging Face style repo)
assay scan ./Qwen2.5-0.5B-Instruct/
# verify a signature or provenance bundle alongside the weights
assay verify ./model/ --bundle model.sig --key signer.pub
# inspect the weights themselves, not just the container
assay scan ./model/ --deep --profile
# ask the sharpest question: how does this model differ from a known-good one?
assay compare ./model-suspect/ ./model-known-good/
# CI mode: machine-readable, non-zero exit on findings
assay scan ./model/ --json --fail-on highTEST.md walks you through all of this on real public models in
about two minutes. DUMMIE.md is the complete illustrated
walkthrough of every check, with ASCII diagrams and annotated output.
Every artifact is identified from its magic bytes, with the file extension used
only as a hint. assay refuses to guess when the bytes disagree with the name.
A repo that mixes safetensors and pickle is itself a signal, and it is
reported as one.
safetensors exists precisely because Python pickle (.bin, .pt, .ckpt)
can run arbitrary code the moment you load it. assay:
- flags every pickle artifact as untrusted by default
- runs an opcode-level scan for dangerous patterns (
GLOBAL,REDUCE,STACK_GLOBAL, imports ofos/subprocess/builtins, and friends) - looks inside torch zip containers rather than stopping at the archive
- understands the legacy container: a
torch.savefile written without zip serialization is not one pickle but five concatenated streams (magic, protocol version, sys_info, the module, the storage keys) followed by raw tensor bytes.assaysplits them, tells you which stream carried the payload, and states plainly how far the opcodes went and where the storage payload begins, instead of stopping at the first non-opcode byte and calling every ordinary checkpoint truncated - flags bytes stapled after the last stream, by signature when the payload announces itself
- tells you when a clean
safetensorsequivalent sits in the same repo
PICKLE_TRUNCATED now means what it says: a stream that stopped before its
STOP opcode, reported with the byte offset where analysis ended.
Weights cannot execute. The files next to them can, and that is now the most
direct execution path in the ecosystem. A repo can ship modeling_foo.py plus
an auto_map entry in config.json; from_pretrained(trust_remote_code=True)
imports that module, which runs everything at its top level before a single
tensor is read. A scanner that only opens tensor containers never sees it.
So assay reports the repo, not just the weights. Every .py alongside a model
becomes an artifact with its own verdict and its own file hash, and the configs
are read to find out which of them a loader would actually execute:
REMOTE_CODE_AUTO_MAP: this file is wired to a loader entry point, so importing the model runs itREMOTE_CODE_EXTERNAL: the mapping points at another repo (org/name--module.Class), so loading downloads and executes code this repo does not even containREMOTE_CODE_UNRESOLVED: the mapping names a module that is not hereREMOTE_CODE_PRESENT: code ships with the weights and nothing points at it, which is not an alarm but is worth knowingPY_DANGEROUS_CALL: shell execution, subprocess,eval/exec, dynamic imports, pickle and marshal loading, network calls, native memory. Scoredhighat module level, because that runs on import, andmediuminside a function body. Reported with the line number and the linePY_OBFUSCATION: large encoded blobs and long hex-escape runs, which is how a payload avoids being read
Matching is call-aware, so ordinary torch code (model.eval(),
torch.compile()) is not mistaken for eval( and compile(. Symlinks out of
the repo are not followed: what gets judged is the repo itself.
A chat template is not inert text. transformers renders it for every
conversation, so it runs before the model produces a single token. It renders
in a Jinja sandbox, which means a payload cannot simply call os.system: it has
to escape first, by pivoting through Python's object graph (__class__,
__mro__, __subclasses__, __globals__) or through one of the Jinja globals
that leak a reference to it (lipsum, cycler, joiner).
That is what makes the check worth having: formatting a conversation needs none
of those. assay extracts the {{ … }} and {% … %} regions, which are the
only places a template executes anything, and reports what it finds there:
TEMPLATE_SANDBOX_ESCAPE(high): an object-graph pivot, named with the line it appears onTEMPLATE_GADGET_GLOBAL(medium):lipsum,cyclerorjoiner, which generate filler text and are mainly known as escape pivotsTEMPLATE_DYNAMIC_ATTRIBUTE: an attribute looked up by computed name (|attr,getattr), which is how a payload names__class__without writing it. Medium on its own, high when the name is also assembled from pieces, becausegetattr(ns, '__in' ~ 'it__')is not something anyone types by accidentTEMPLATE_OBFUSCATION(medium): names built at render time fromchr(), hex escapes, concatenation or|join
Templates are found wherever they live: chat_template.jinja,
chat_template.json, the chat_template field of tokenizer_config.json
(including the named-list form), and the tokenizer.chat_template key inside
GGUF metadata. A template with findings becomes its own artifact, with its own
verdict and file hash. A template with nothing to report stays silent, because
every modern repo ships one and an artifact per repo saying "a template is
present" is noise nobody reads.
namespace is deliberately not treated as a gadget: real chat templates use it
for bookkeeping. The suite checks that against reproductions of the templates in
wide use, and any finding on those would be a bug.
The format is safe by design, but the container still has attack surface
(overlapping offsets, malformed specs, denial of service at load time). assay:
- parses the
u64length prefix and the JSON header - validates every
data_offsets [begin, end]: in bounds,begin <= end, non-overlapping, no range pointing outside the data segment - rejects dtype and shape declarations that disagree with the byte ranges
- accounts for every byte: any run of bytes inside the file that no tensor
claims is reported with its exact offsets and a preview, because real writers
pack tensors contiguously and unclaimed bytes are a storage channel the format
cannot explain (
ST_UNREFERENCED_BYTES)
That last check matters more than it sounds, and it applies to GGUF too. A file
whose offsets are all valid can still carry an arbitrary payload in the hole
between two tensors, or after the last one. No loader reads those bytes, they never appear in a tensor, and
the manifest hash does not cover them, so the file looks pristine everywhere
else. assay names the region, shows the first bytes, and recognizes the
signature when the payload announces itself (ELF, Mach-O, PE, ZIP, gzip, xz,
7-Zip, PDF, PNG, a pickle stream, a shebang).
GGUF carries no executable code, but its metadata can embed a Jinja2 chat
template, which is a code-ish injection surface. assay:
- validates magic, version, tensor count, and the KV metadata block
- checks that every tensor offset lands inside the file
- surfaces embedded chat templates for human review instead of silently trusting them
- accounts for every byte, as it does for
safetensors: each tensor's size is computed from its ggml type and shape, the alignment padding the format requires is recognized as such, and anything left over is reported with its offsets and a preview (GGUF_UNREFERENCED_BYTES)
That last check has a safety valve. Tensor sizes come from the ggml type table,
and assay refuses to trust it blindly: the layouts of Q8_1 and the IQ
family have moved between ggml versions, so they are marked unknown rather than
guessed, and if the computed sizes contradict the declared offsets anywhere in
the file, the report says the accounting is unreliable
(GGUF_ACCOUNTING_INCOMPLETE, info) instead of inventing findings. An
admission is worth more than a confident lie.
Two digests, because there are two different questions.
The manifest hash is the model's identity: a per-tensor digest folded into one hash that covers tensor names, dtypes, shapes and content, and nothing else. It survives re-containerization, so renaming the file or repacking the archive does not change it. That is what you pin when you want to know the model is the same.
The file hash is the digest of the artifact exactly as it sits on disk, every byte included. Two files can share a manifest hash and still differ, for instance when one has an archive stapled after the last tensor, so the file hash is what you pin when you want to know that nothing at all changed. It is computed for every artifact, including pickles, which have no tensors and therefore no manifest hash but are still worth pinning.
Detached ed25519 signatures and model-transparency manifests are verified
against the computed hashes, and sidecar files next to the artifact are
auto-detected. A full Sigstore or cosign chain (Fulcio roots, Rekor
transparency log) is not implemented, so such bundles are reported as
present but unverified rather than treated as trusted. signed is only ever
printed on a real cryptographic pass.
Everything above judges the container. --deep inspects the numbers inside it,
without ever loading the model:
- per-tensor statistics in a single streaming pass: NaN and Inf integrity, mean, std, L2 and RMS, excess kurtosis, sparsity, and 6 sigma outlier mass
- per-layer profile with robust median and MAD anomaly detection across
layers, a terminal sparkline (
--profile) and an honest 1D SVG chart (--svg) - secret and string scanning over metadata, GGUF KV blocks, and sibling
config, tokenizer, template and adapter files (including
chat_template.jinja, where templates live now, andadapter_config.json): Hugging Face user and organization tokens, AWS, GitHub, GitLab, Slack, Docker Hub, Anthropic and Google keys, Google service account credentials, PEM private key blocks, and suspicious URLs. Evidence is always redacted - architectural fingerprint derived from naming scheme, layer count, hidden dimension, head count, and vocabulary size, which catches a model that claims to be X but is structurally Y
Weight analysis is most honest as a diff against a known-good reference, not as
a judgment of a model in isolation. A normally trained transformer is naturally
non-uniform across layers, so a standalone profile will flag legitimate peaks.
compare SUBJECT BASELINE makes the baseline the zero line:
- identical models are silent (
IDENTICAL) - a uniform fine-tune shows broad, even drift and stays quiet
- a localized tamper shows a single concentrated spike and is flagged
It streams matched tensor pairs in lockstep (both files mapped, never both
models fully in RAM), canonicalizes naming differences such as a
transformer. wrapper prefix, recognizes weight tying instead of calling it a
divergence, guards against cross-architecture comparison (ARCH_MISMATCH,
override with --force), and reports added, removed, or reshaped tensors as
STRUCTURAL_DIVERGENCE.
assay is explicit about how much confidence a result carries, and the two
kinds of result are never mixed.
| Verdicts | Signals | |
|---|---|---|
| Produced by | container and provenance checks (always on) | --deep and compare |
| Based on | external ground truth ("pickle can execute code") | statistics, no ground truth |
| Output | clean / untrusted / malformed |
scored findings with a severity |
| Means | this is what the file is | this is worth a human look |
A high score never means "malicious". It means anomalous. assay will always
tell you the confidence of a finding, and it never claims to catch what it
cannot.
Scan a file or a model directory.
| Flag | Meaning |
|---|---|
--json |
machine-readable report on stdout |
--fail-on SEVERITY |
exit non-zero at or above this severity (default high) |
--color auto|always|never |
colorize output (default auto) |
--no-progress |
disable the real-time progress display on stderr |
--allow-empty |
exit 0 instead of 4 when the path holds no scannable artifact |
--deep (alias --stats) |
inspect the weights, not just the container |
--profile |
print the per-layer sparkline (implies --deep) |
--svg FILE |
write the layer-profile chart as SVG (implies --deep) |
--mad-k K |
anomaly threshold in MADs from the cross-layer median (default 5.0) |
--scan-tensor-entropy |
experimental: flag near-maximal-entropy integer tensor regions |
Progress prints to stderr and is disabled automatically off a TTY.
How the tree is walked. Symlinked files are followed, because that is
exactly how a Hugging Face cache stores a repo: every file in a snapshot is a
link into blobs/, and refusing to follow them would mean scanning nothing.
Directories are walked through a set of canonical paths already visited, so a
link cycle (sub/back -> ..) terminates instead of rescanning the same weights
once per level. Files are deduplicated the same way: two names for one inode are
read once, reported once, and the other paths are listed as
ALIASED_ARTIFACT.
Same checks as scan, focused on the provenance answer.
| Flag | Meaning |
|---|---|
--bundle FILE |
explicit signature or provenance bundle |
--key FILE |
ed25519 public key (raw or hex) for detached signatures |
Plus every flag from scan.
Differential weight analysis against a known-good reference of the same architecture.
| Flag | Meaning |
|---|---|
--json |
machine-readable drift report |
--svg FILE |
write the drift-per-layer chart as SVG |
--mad-k K |
concentration threshold for layer drift, in MADs (default 5.0) |
--epsilon EPS |
elements differing by more than this count toward changed_frac (default 1e-6) |
--force |
compare across mismatched architectures (output is unreliable) |
--fail-on SEVERITY |
exit non-zero at or above this severity (default high) |
--color auto|always|never |
colorize output (default auto) |
| Code | Meaning |
|---|---|
0 |
clean, nothing at or above the threshold |
1 |
findings at or above --fail-on severity |
2 |
unreadable or malformed artifact (parse failure) |
3 |
internal error, or the given path does not exist |
4 |
the path exists but holds nothing scannable |
Worst outcome wins: a malformed artifact anywhere in a directory scan outranks a high-severity finding elsewhere in it.
Why 4 and not 0. A scan that checked nothing is not a pass. A mistyped
path, a glob that matched no file, a build step that moved the weights: any of
these would otherwise turn a supply-chain gate green without a single byte
having been verified, which is the one failure mode a gate must never have. It
is not an error either, since nothing actually failed, so it gets its own code
and its own line. Pipelines that scan a genuinely optional directory can pass
--allow-empty to turn it back into 0, and the finding is still reported.
$ assay scan ./models/gpt2 --deep --profile
[1/2] ./models/gpt2/model.safetensors CLEAN 3 finding(s) (22.90s)
[2/2] ./models/gpt2/pytorch_model.bin UNTRUSTED 3 finding(s) (1ms)
✓ scanned 2 artifact(s): 1 clean, 1 untrusted, 1.0 GiB in 22.91s
./models/gpt2/model.safetensors [safetensors] -> CLEAN
manifest: blake3:d4ceed607f7040ba84b91eadef010d98079f9d9d85ffd6faf13d77ce958eccdf
file: blake3:3bca1b7f6c327daecafc16e52d1319375299354e35413fb4e18d24e59b77ce06
signature: unsigned
[low] WEIGHT_OUTLIER_LAYER: layer 3 is anomalous on mean_kurtosis (6.0 MADs from the cross-layer median); worth a human look, not a verdict
- metric=mean_kurtosis, value=119.9821, mads=6.00
[low] WEIGHT_OUTLIER_LAYER: layer 11 is anomalous on l2 (7.3 MADs from the cross-layer median); worth a human look, not a verdict
- metric=l2, value=840.6817, mads=7.27
[info] ARCH_DETECTED: structural fingerprint: gpt2 (gpt2)
- layers=Some(12), hidden=Some(768), heads=Some(12), vocab=Some(50257)
./models/gpt2/pytorch_model.bin [pickle] -> UNTRUSTED
file: blake3:4216aebca2f4e9256cdb1e9955c1fd43c1fa179bd9e4c3c4a10bbce0a5acb213
signature: unsigned
[high] PICKLE_RCE_RISK: pickle artifact can execute code at load time
- stream 4/5: execution opcodes: REDUCE, BUILD
[info] PICKLE_TORCH_LEGACY: legacy torch container: 5 pickle stream(s) analyzed in full (bytes 0..25917); the remaining 548092160 byte(s) are the raw tensor storage payload, which carries no opcodes
- storage payload at file offsets 25917..548118077
[info] SAFE_ALTERNATIVE_AVAILABLE: a safetensors artifact is present in the same repo; prefer it
scanned 2 artifact(s); worst finding: high
./models/gpt2/model.safetensors
layer profile ▅▁▂▁▂▁▂▂▃▄▆█ (12 layers, metric=l2)
min=787.0994 max=840.6817
anomalous layers: 3, 11
The pickle line is the whole pitch: you were one
torch.loadaway from running someone else's code, with a cleansafetensorsfile sitting right next to it.Note layers 3 and 11 flagged on the clean file. On a model in isolation you cannot tell a legitimate peak from an injected one, because a well-trained transformer is naturally non-uniform. That is exactly why
compareexists.
$ assay compare ./models/gpt2 ./models/dialogpt
compare ./models/gpt2/model.safetensors vs ./models/dialogpt/model.safetensors
arch: gpt2 vs gpt2 (match)
normalized: stripped wrapper prefix from 160 baseline tensor name(s)
160 matched, 0 structural divergence(s), worst rel_l2: 1.4601
drift profile ▂▁▃▂▂▄▅▆█▇▇█ (12 layers, metric=rel_l2)
min=0.1385 max=0.2105
no anomalous layers
[info] TIED_WEIGHT: 'lm_head.weight' is tied to 'transformer.wte.weight'; weight tying is a serialization convention, not a divergence
- counterpart present on same side with equal values
DialoGPT is a full fine-tune of gpt2: every layer moved a little, drift is broad and homogeneous, so nothing is flagged. The
transformer.naming prefix is canonicalized away (160 matched, 0 structural divergences), and the tiedlm_head/wtepair is reported as info.
$ python make_tampered_gpt2.py ./models/gpt2/model.safetensors \
./models/gpt2-tampered/model.safetensors --layer 5 --scale 4.0
$ assay compare ./models/gpt2 ./models/gpt2-tampered
compare ./models/gpt2/model.safetensors vs ./models/gpt2-tampered/model.safetensors
arch: gpt2 vs gpt2 (match)
160 matched, 0 structural divergence(s), worst rel_l2: 0.7500
drift profile ▁▁▁▁▁█▁▁▁▁▁▁ (12 layers, metric=rel_l2)
min=0.0000 max=0.5359
anomalous layers: 5
[medium] LAYER_DRIFT_OUTLIER: layer 5 drift is a concentrated outlier (rel_l2=0.536, 12.0 MADs above the cross-layer drift level); worth a human look, not a verdict
- dominant tensor: h.5.mlp.c_fc.weight
[medium] TENSOR_DRIFT: tensor 'h.5.mlp.c_fc.weight' dominates the drift of layer 5
Only the tampered layer 5 spikes. Layers 3 and 11, the ones the standalone profile flagged above, stay silent here, because they do not move versus the baseline. That is the payoff of differential analysis.
See DUMMIE.md for a line-by-line explanation of every field in
this output.
--json prints one report per artifact. A single-file scan prints the report
object directly; a directory scan wraps them in { "artifacts": [ ... ] }:
{
"artifact": "./models/gpt2/pytorch_model.bin",
"format": "pickle",
"verdict": "untrusted",
"findings": [
{
"id": "PICKLE_RCE_RISK",
"severity": "high",
"detail": "pickle artifact can execute code at load time",
"evidence": ["execution opcodes: REDUCE, BUILD"]
},
{
"id": "PICKLE_TRUNCATED",
"severity": "medium",
"detail": "pickle opcode stream ended unexpectedly or hit an unknown opcode; analysis may be incomplete"
}
],
"hashes": {
"file": "blake3:4216aebca2f4e9256cdb1e9955c1fd43c1fa179bd9e4c3c4a10bbce0a5acb213"
},
"signature": "unsigned"
}hashes.file is present for every artifact. For safetensors and GGUF the
block also carries manifest and per_tensor, and with --deep the report
gains stats, layer_profile, and fingerprint. A compare --json report carries subject, baseline, arch,
structural_divergences, tensor_drift, layer_drift, findings, and
summary.
# fail the pipeline if any artifact is untrusted, and keep the report
assay scan ./models/ --json --fail-on high | tee assay-report.json- name: Assay model artifacts
run: |
cargo install assay
assay scan ./models/ --json --fail-on high | tee assay-report.jsonPin both digests from a known-good run and compare them on every build. A
changed manifest means the weights changed, whatever the filename says. A
changed file with an unchanged manifest means the model is the same but the
bytes around it are not, which is exactly what stapling a payload onto a valid
artifact looks like.
Container and provenance (verdict grade)
| ID | Severity |
|---|---|
PICKLE_RCE_RISK |
high |
PICKLE_TRAILING_DATA |
high with a known file signature, otherwise medium |
PICKLE_GLOBAL_REF, PICKLE_UNTRUSTED, PICKLE_TRUNCATED |
medium |
PICKLE_CONTAINER_NO_PICKLE, PICKLE_CONTAINER_UNREADABLE |
medium |
PICKLE_TORCH_LEGACY |
info |
ST_OFFSET_OOB, ST_OFFSET_OVERLAP |
high |
ST_UNREFERENCED_BYTES |
high with a known file signature, medium for other non-zero bytes, low for zero padding |
ST_HEADER_MALFORMED, ST_DTYPE_SHAPE_MISMATCH |
medium |
ST_DTYPE_UNKNOWN |
low |
GGUF_PARSE_ERROR, GGUF_BAD_VERSION, GGUF_OFFSET_OOB |
high |
GGUF_UNREFERENCED_BYTES |
high with a known file signature, medium for other non-padding bytes, low for excess zero padding |
GGUF_CHAT_TEMPLATE |
low |
GGUF_ACCOUNTING_INCOMPLETE |
info |
SIG_MISMATCH |
high |
SIG_NO_MANIFEST, SIG_KEY_UNREADABLE, SIG_BUNDLE_UNREADABLE, SIG_UNRECOGNIZED, SIG_ERROR |
low |
SIG_VERIFIED, SIG_MANIFEST_MATCH, SIG_NO_KEY, SIG_SIGSTORE_UNVERIFIED |
info |
IO_ERROR, PATH_NOT_FOUND |
high |
UNKNOWN_FORMAT |
medium |
SAFE_ALTERNATIVE_AVAILABLE, NO_ARTIFACTS, ALIASED_ARTIFACT |
info |
Chat templates
| ID | Severity |
|---|---|
TEMPLATE_SANDBOX_ESCAPE |
high |
TEMPLATE_DYNAMIC_ATTRIBUTE |
high with an assembled name, otherwise medium |
TEMPLATE_GADGET_GLOBAL, TEMPLATE_OBFUSCATION |
medium |
Code shipped with the weights
| ID | Severity |
|---|---|
REMOTE_CODE_AUTO_MAP, REMOTE_CODE_EXTERNAL |
high |
PY_DANGEROUS_CALL |
high at module level, medium inside a function |
REMOTE_CODE_UNRESOLVED, PY_OBFUSCATION, PY_NOT_READ |
medium |
REMOTE_CODE_PRESENT |
low |
Weight signals (--deep)
| ID | Severity |
|---|---|
WEIGHT_NAN_INF |
high |
EMBEDDED_SECRET |
high, medium, or low by confidence |
ARCH_MISMATCH |
medium |
WEIGHT_OUTLIER_LAYER |
low or medium by deviation |
ARCH_DETECTED, SUSPICIOUS_URL, STATS_DEFERRED_QUANTIZED, TENSOR_ENTROPY_ANOMALY |
info |
Drift signals (compare)
| ID | Severity |
|---|---|
STRUCTURAL_DIVERGENCE, COMPARE_ERROR |
high |
ARCH_MISMATCH |
high, or medium with --force |
LAYER_DRIFT_OUTLIER, TENSOR_DRIFT |
low, medium, or high by drift magnitude |
IDENTICAL, TIED_WEIGHT, DRIFT_DEFERRED_QUANTIZED |
info |
- Offline first. No network calls during a scan. Signature roots are bundled or supplied explicitly.
- Never load the model. No framework, no import, no forward pass. Bytes only.
- Single binary, no runtime deps. Drop it into a CI image or an air-gapped box and run it.
- Honest confidence. Every finding carries a severity, and a statistical signal is never dressed up as a verdict.
- Deterministic. The same bytes always produce the same digests and the same verdicts.
- Dogfoodable. Built to be run on real artifacts pulled off public hubs.
Worth knowing before you rely on it:
- Quantized GGUF is partially covered. Legacy quants (Q4_0, Q4_1, Q5_0,
Q5_1, Q8_0) and F32, F16, BF16 are dequantized for real statistics. K-quant
and IQ tensors are reported as
STATS_DEFERRED_QUANTIZED(structural information only) rather than computing garbage on raw block bytes. Full k-quant decoding is the next milestone. - Quantization-triggered payloads are not detected yet. A payload that only activates after GGUF quantization needs a full-precision reference and a per-weight quantization-error diff. The dequantization, streaming, and lockstep drift machinery is in place; the check itself is not shipped and is deliberately not promised.
- Sigstore and cosign chains are not verified. They are reported as unverified, never as trusted.
- Behavioral fingerprinting is permanently out of scope. Gradient or forward-pass identification would require executing the model, which breaks the invariant this tool is built on.
- Weight analysis finds anomalies, not intent. A signal is a reason to look, never a proof of malice.
cargo test --all # 72 unit tests + 24 end-to-end tests
cargo fmt --all --check # formatting gate
cargo clippy --all-targets -- -D warningsThe end-to-end tests drive the compiled binary against fixtures they build themselves (a forged pickle, hand-assembled safetensors and GGUF files, a tampered layer, a signed manifest), so the suite is hermetic: no network, no downloaded weights, under a second to run. CI runs all three gates on Linux and macOS for every push and pull request, then re-runs the pickle self-test against the release binary.
If you touch a user-visible message, remember the README and
DUMMIE.md quote real output; regenerate the examples rather
than editing them by hand.
DUMMIE.md: the complete illustrated walkthrough, with ASCII diagrams of every check and annotated real output.TEST.md: download real public models and tryassayin two minutes, including a harmless self-test that proves the pickle scanner fires.CHANGELOG.md: what changed, release by release.
Found a model assay should have flagged and did not? That is the best possible
bug report. Open an issue with the repo id.
