[im-rk]

What

The first release of the benchmark:

• 60 invented scripts: data/dev.jsonl (30) and data/test.jsonl (30), each 15 scam + 15 benign across 10 scam and 7 benign categories, with deliberately hard cases: genuine bank alerts that use scary words, innocent "gift card" and "wire the money" family talk, and polite scams with no scam vocabulary (landlord, boss, "I'm travelling").
• schema/script.schema.json (JSON Schema 2020-12, strict) and scripts/validate.mjs: schema, id/split/prefix rules, label-vs-category consistency, 15/15 quotas, >= 40% tricky benign, >= 6 scam categories, different author groups for dev and test, no URLs or phone numbers, near-duplicate detection (word 3-gram overlap).
• scripts/score.mjs: precision, recall, F1, false-positive rate at two operating points (hold and flag), per-category recall, misclassified list; divide-by-zero is null, never 0; missing predictions count as allow and are reported.
• CI runs the validator and the tests.
• Docs say plainly that this is a development yardstick, not an independent benchmark.

Test plan

• npm run validate: OK (dev: 30, test: 30)
• 10 tests: the shipped data passes; 12 kinds of data mistakes are each caught; hand-checked scoring arithmetic; perfect/empty/all-benign edge cases; bad predictions reported