Skip to content

Upgrade text model training and evaluation - #11

Merged
iPrakharV merged 2 commits into
mainfrom
ml-credibility-v2
May 11, 2026
Merged

iPrakharV merged 2 commits into
mainfrom
ml-credibility-v2

Conversation

@iPrakharV

Copy link
Copy Markdown
Owner

What changed

  • replaced the checked-in text artifact with tiny-text-mps-v2, a compact MLP over word and character n-gram features
  • kept worker inference dependency-free by exporting JSON weights and running the MLP in pure Python
  • preserved compatibility with the old linear artifact format
  • refreshed MPS training output, evaluation output, model card, report, and README numbers
  • added tests for the v2 artifact schema and loader

Training and evaluation

  • device: mps
  • train time: 0.3991 s
  • parameters: 2,273
  • challenge evaluation: 16/16 correct

Validation

  • python training/train_text_model.py
  • python training/evaluate_text_model.py
  • python -m ruff check .
  • python -m compileall -q gateway worker training bench scripts tests
  • python -m pytest -q
  • python scripts/smoke_check.py against locally started gateway and worker
  • python bench/run_scenarios.py --output-json /tmp/modelmesh-bench.json --output-md /tmp/modelmesh-bench.md

@iPrakharV
iPrakharV merged commit c10f346 into main May 11, 2026
2 checks passed
@iPrakharV
iPrakharV deleted the ml-credibility-v2 branch May 11, 2026 02:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant