diff --git a/README.md b/README.md index 59d7ed3..f00ba22 100644 --- a/README.md +++ b/README.md @@ -93,7 +93,7 @@ The full project report is in `docs/report.md`. ## Training -The worker can load a small trained text model artifact from `worker/app/artifacts/tiny_text_model.json`. The current model trains on generated service-status examples, so the point is the local training and serving path, not benchmark-grade NLP accuracy. +The worker can load a small trained text model artifact from `worker/app/artifacts/tiny_text_model.json`. The current model is a compact MLP over word and character n-gram features. It trains on generated service-status examples, so the point is the local training, export, evaluation, and serving path, not benchmark-grade NLP accuracy. On Apple Silicon, the training script uses PyTorch MPS. It refuses to run on CPU by default so a slow accidental CPU run fails fast. @@ -108,12 +108,16 @@ Latest training run on an M3 Pro: | Metric | Value | | --- | ---: | +| Model type | mlp_text_classifier | | Device | mps | -| Epochs | 320 | +| Epochs | 260 | | Train examples | 3,276 | | Test examples | 820 | -| Train time | 0.4351 s | -| Examples/sec | 2,409,512.37 | +| Features | 140 | +| Hidden units | 16 | +| Parameters | 2,273 | +| Train time | 0.3991 s | +| Examples/sec | 2,134,408.08 | | Train accuracy | 1.0 | | Test accuracy | 1.0 | @@ -124,7 +128,7 @@ Model evaluation and limitations are documented in `docs/model_card.md`. ## What is in here now - FastAPI gateway with `/predict`, `/health`, `/workers`, and `/metrics` -- Worker service with a tiny text classifier and optional trained artifact +- Worker service with a compact text classifier and optional trained artifact - Redis-backed cache when Redis is available, with an in-memory fallback - Retry on failed worker calls and basic load-aware routing - Scenario benchmark runner with saved results diff --git a/docs/architecture.md b/docs/architecture.md index e2b1ea2..a949114 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -32,7 +32,7 @@ The benchmark runner uses both modes against the same slow-worker scenario so th ## Training -`training/train_text_model.py` trains the tiny text model with PyTorch. On Apple Silicon it uses `mps`, and it exits instead of silently falling back to CPU unless `--allow-cpu` is passed. +`training/train_text_model.py` trains the compact text model with PyTorch. On Apple Silicon it uses `mps`, and it exits instead of silently falling back to CPU unless `--allow-cpu` is passed. The trained artifact is plain JSON so the worker can load it without depending on PyTorch at inference time. diff --git a/docs/model_card.md b/docs/model_card.md index c79fe37..60769a5 100644 --- a/docs/model_card.md +++ b/docs/model_card.md @@ -2,9 +2,11 @@ ## Model -`tiny-text-mps-v1` is a small linear text classifier used by ModelMesh workers. It classifies short service-status text as either `healthy` or `risky`. +`tiny-text-mps-v2` is a small MLP text classifier used by ModelMesh workers. It classifies short service-status text as either `healthy` or `risky`. -The model is intentionally small. Its purpose is to make the training and serving path testable inside an inference mesh, not to compete with production NLP models. +The model uses word-count features and character n-gram features. It trains with PyTorch, then exports to JSON so workers can run inference without PyTorch. + +The model is intentionally small. Its purpose is to make the training, export, evaluation, and serving path testable inside an inference mesh, not to compete with production NLP models. ## Training @@ -18,14 +20,16 @@ Latest checked-in run: | Metric | Value | | --- | ---: | +| Model type | mlp_text_classifier | | Device | mps | -| Epochs | 320 | +| Epochs | 260 | | Train examples | 3,276 | | Test examples | 820 | -| Vocab size | 37 | -| Parameters | 38 | -| Train time | 0.4351 s | -| Examples/sec | 2,409,512.37 | +| Features | 140 | +| Hidden units | 16 | +| Parameters | 2,273 | +| Train time | 0.3991 s | +| Examples/sec | 2,134,408.08 | | Train accuracy | 1.0 | | Test accuracy | 1.0 | @@ -44,23 +48,17 @@ The challenge set is a small handwritten set of service-status examples with mix | Metric | Value | | --- | ---: | | Examples | 16 | -| Correct | 15 | -| Accuracy | 0.9375 | +| Correct | 16 | +| Accuracy | 1.0 | Confusion matrix: | Expected | Predicted healthy | Predicted risky | | --- | ---: | ---: | -| healthy | 6 | 1 | +| healthy | 7 | 0 | | risky | 0 | 9 | -The missed example was: - -```text -worker recovered after timeout and is stable -``` - -The model predicted `risky`, mostly because `timeout` has a strong negative learned weight and the bag-of-words model does not understand recovery order. +There were no misses on this challenge set. That is useful as a regression check, but it is still a small handwritten set and should not be treated as broad NLP evidence. Full evaluation output: `training/evaluation/latest.md`. @@ -78,10 +76,10 @@ Use this model to exercise ModelMesh serving behavior: ## Limitations - Training data is generated service-status text, not a real production incident dataset. -- The model is bag-of-words and does not understand word order. +- The model uses word and character n-gram features, not a transformer or sequence model. - Accuracy numbers are useful for regression checks, not for claiming broad NLP performance. - The model is deliberately small so the project can focus on ML systems behavior. ## Next model improvement -The next useful upgrade is a compact sequence model or character n-gram model trained on a larger labeled service-log dataset, still with MPS timing and artifact export. +The next useful upgrade is a larger labeled service-log dataset or a compact sequence model, still with MPS timing, evaluation output, and JSON artifact export. diff --git a/docs/report.md b/docs/report.md index 85d5129..6bc02ac 100644 --- a/docs/report.md +++ b/docs/report.md @@ -33,8 +33,8 @@ flowchart LR gateway --> cache["Redis or memory cache"] gateway --> worker_a["Worker A"] gateway --> worker_b["Worker B"] - worker_a --> model_a["tiny-text-mps-v1"] - worker_b --> model_b["tiny-text-mps-v1"] + worker_a --> model_a["tiny-text-mps-v2"] + worker_b --> model_b["tiny-text-mps-v2"] gateway --> dashboard["Dashboard and metrics"] ``` @@ -49,13 +49,16 @@ Latest MPS training run on an M3 Pro: | Metric | Value | | --- | ---: | +| Model type | mlp_text_classifier | | Device | mps | -| Epochs | 320 | +| Epochs | 260 | | Train examples | 3,276 | | Test examples | 820 | -| Parameters | 38 | -| Train time | 0.4351 s | -| Examples/sec | 2,409,512.37 | +| Features | 140 | +| Hidden units | 16 | +| Parameters | 2,273 | +| Train time | 0.3991 s | +| Examples/sec | 2,134,408.08 | | Train accuracy | 1.0 | | Test accuracy | 1.0 | @@ -68,17 +71,17 @@ The model was also evaluated on a handwritten challenge set with mixed healthy a | Metric | Value | | --- | ---: | | Examples | 16 | -| Correct | 15 | -| Accuracy | 0.9375 | +| Correct | 16 | +| Accuracy | 1.0 | Confusion matrix: | Expected | Predicted healthy | Predicted risky | | --- | ---: | ---: | -| healthy | 6 | 1 | +| healthy | 7 | 0 | | risky | 0 | 9 | -The one miss was a recovery sentence containing `timeout`. That is a reasonable failure for a bag-of-words model and is listed in `docs/model_card.md`. +There were no misses on this challenge set. The set is small and handwritten, so the result is treated as a regression check rather than a broad NLP claim. ## Benchmark results diff --git a/requirements-train.txt b/requirements-train.txt index 7e9a987..ce95c64 100644 --- a/requirements-train.txt +++ b/requirements-train.txt @@ -1 +1,2 @@ torch>=2.5 +numpy>=2.0 diff --git a/tests/test_model.py b/tests/test_model.py index 42209b9..3bacfc5 100644 --- a/tests/test_model.py +++ b/tests/test_model.py @@ -1,6 +1,6 @@ import json -from worker.app.model import TinyTextModel, load_model +from worker.app.model import DEFAULT_ARTIFACT, MLPTextModel, TinyTextModel, load_model def test_tiny_model_scores_healthy_text_higher() -> None: @@ -28,3 +28,41 @@ def test_model_can_load_training_artifact(tmp_path) -> None: assert model.predict("stable")["label"] == "healthy" assert model.predict("broken")["label"] == "risky" assert model.version == "test-model" + + +def test_model_can_load_v2_mlp_artifact(tmp_path) -> None: + artifact = tmp_path / "model.json" + artifact.write_text(json.dumps({ + "model_version": "test-mlp", + "model_type": "mlp_text_classifier", + "feature_spec": { + "word_vocab": ["stable", "broken"], + "char_ngrams": [], + "char_ngram_range": [3, 4], + }, + "weights": { + "hidden": [[1.0, 0.0], [0.0, 1.0]], + "output": [2.0, -2.0], + }, + "biases": { + "hidden": [0.0, 0.0], + "output": 0.0, + }, + })) + + model = load_model(str(artifact)) + + assert isinstance(model, MLPTextModel) + assert model.predict("stable")["label"] == "healthy" + assert model.predict("broken")["label"] == "risky" + assert model.version == "test-mlp" + + +def test_checked_in_artifact_uses_v2_schema() -> None: + artifact = json.loads(DEFAULT_ARTIFACT.read_text()) + + assert artifact["model_version"] == "tiny-text-mps-v2" + assert artifact["model_type"] == "mlp_text_classifier" + assert set(artifact) >= {"feature_spec", "weights", "biases", "training"} + assert set(artifact["weights"]) == {"hidden", "output"} + assert set(artifact["biases"]) == {"hidden", "output"} diff --git a/tests/test_text_model_evaluation.py b/tests/test_text_model_evaluation.py index 404f697..048f692 100644 --- a/tests/test_text_model_evaluation.py +++ b/tests/test_text_model_evaluation.py @@ -5,6 +5,7 @@ def test_challenge_evaluation_has_expected_shape() -> None: result = evaluate() assert result["examples"] == 16 - assert result["correct"] >= 12 + assert result["model_type"] == "mlp_text_classifier" + assert result["correct"] >= 14 assert set(result["confusion_matrix"]) == {"healthy", "risky"} assert set(result["per_label"]) == {"healthy", "risky"} diff --git a/training/evaluate_text_model.py b/training/evaluate_text_model.py index 565232d..2a5d754 100644 --- a/training/evaluate_text_model.py +++ b/training/evaluate_text_model.py @@ -107,6 +107,7 @@ def evaluate(model_artifact: Path | None = None) -> dict[str, Any]: return { "model_version": model.version, + "model_type": getattr(model, "model_type", "unknown"), "dataset": "handwritten service-status challenge set", "examples": total, "accuracy": round(correct / total, 4), @@ -122,6 +123,7 @@ def render_markdown(result: dict[str, Any]) -> str: "# Model evaluation", "", f"Model version: `{result['model_version']}`", + f"Model type: `{result['model_type']}`", "", "| Metric | Value |", "| --- | ---: |", diff --git a/training/evaluation/latest.json b/training/evaluation/latest.json index b22997c..0f7704a 100644 --- a/training/evaluation/latest.json +++ b/training/evaluation/latest.json @@ -1,13 +1,14 @@ { - "model_version": "tiny-text-mps-v1", + "model_version": "tiny-text-mps-v2", + "model_type": "mlp_text_classifier", "dataset": "handwritten service-status challenge set", "examples": 16, - "accuracy": 0.9375, - "correct": 15, + "accuracy": 1.0, + "correct": 16, "confusion_matrix": { "healthy": { - "healthy": 6, - "risky": 1 + "healthy": 7, + "risky": 0 }, "risky": { "healthy": 0, @@ -17,10 +18,10 @@ "per_label": { "healthy": { "precision": 1.0, - "recall": 0.8571 + "recall": 1.0 }, "risky": { - "precision": 0.9, + "precision": 1.0, "recall": 1.0 } }, @@ -29,7 +30,7 @@ "text": "fast stable reliable service", "expected": "healthy", "predicted": "healthy", - "score": 0.9993, + "score": 1.0, "correct": true }, { @@ -42,99 +43,99 @@ { "text": "worker recovered after timeout and is stable", "expected": "healthy", - "predicted": "risky", - "score": 0.6689, - "correct": false + "predicted": "healthy", + "score": 1.0, + "correct": true }, { "text": "cache hit but worker failed with timeout", "expected": "risky", "predicted": "risky", - "score": 0.9887, + "score": 1.0, "correct": true }, { "text": "great latency but retry caused timeout", "expected": "risky", "predicted": "risky", - "score": 0.9572, + "score": 1.0, "correct": true }, { "text": "responsive route completed request", "expected": "healthy", "predicted": "healthy", - "score": 0.9998, + "score": 1.0, "correct": true }, { "text": "unhealthy worker returned fast response", "expected": "risky", "predicted": "risky", - "score": 0.9949, + "score": 1.0, "correct": true }, { "text": "stable gateway handled burst traffic", "expected": "healthy", "predicted": "healthy", - "score": 0.6106, + "score": 1.0, "correct": true }, { "text": "prediction failed after worker crash", "expected": "risky", "predicted": "risky", - "score": 0.994, + "score": 1.0, "correct": true }, { "text": "service is healthy but slow", "expected": "risky", "predicted": "risky", - "score": 0.6146, + "score": 1.0, "correct": true }, { "text": "reliable cache and responsive worker", "expected": "healthy", "predicted": "healthy", - "score": 0.9994, + "score": 1.0, "correct": true }, { "text": "broken route recovered with retry", "expected": "risky", "predicted": "risky", - "score": 0.9964, + "score": 1.0, "correct": true }, { "text": "cache recovered and request completed", "expected": "healthy", "predicted": "healthy", - "score": 0.993, + "score": 1.0, "correct": true }, { "text": "gateway retry loop after timeout", "expected": "risky", "predicted": "risky", - "score": 0.9937, + "score": 1.0, "correct": true }, { "text": "responsive worker handled traffic burst", "expected": "healthy", "predicted": "healthy", - "score": 0.9185, + "score": 1.0, "correct": true }, { "text": "mesh returned broken prediction error", "expected": "risky", "predicted": "risky", - "score": 0.9992, + "score": 1.0, "correct": true } ] diff --git a/training/evaluation/latest.md b/training/evaluation/latest.md index ffc26e3..2cfc7f3 100644 --- a/training/evaluation/latest.md +++ b/training/evaluation/latest.md @@ -1,29 +1,28 @@ # Model evaluation -Model version: `tiny-text-mps-v1` +Model version: `tiny-text-mps-v2` +Model type: `mlp_text_classifier` | Metric | Value | | --- | ---: | | Examples | 16 | -| Correct | 15 | -| Accuracy | 0.9375 | +| Correct | 16 | +| Accuracy | 1.0 | ## Confusion matrix | Expected | Predicted healthy | Predicted risky | | --- | ---: | ---: | -| healthy | 6 | 1 | +| healthy | 7 | 0 | | risky | 0 | 9 | ## Label metrics | Label | Precision | Recall | | --- | ---: | ---: | -| healthy | 1.0 | 0.8571 | -| risky | 0.9 | 1.0 | +| healthy | 1.0 | 1.0 | +| risky | 1.0 | 1.0 | ## Misses -| Text | Expected | Predicted | Score | -| --- | --- | --- | ---: | -| worker recovered after timeout and is stable | healthy | risky | 0.6689 | +No misses on this challenge set. diff --git a/training/runs/latest.json b/training/runs/latest.json index a70acc2..fc260cc 100644 --- a/training/runs/latest.json +++ b/training/runs/latest.json @@ -1,16 +1,20 @@ { "model_path": "worker/app/artifacts/tiny_text_model.json", + "model_type": "mlp_text_classifier", "device": "mps", - "epochs": 320, - "learning_rate": 0.05, + "epochs": 260, + "learning_rate": 0.03, "samples_per_class": 2048, "train_examples": 3276, "test_examples": 820, - "vocab_size": 37, - "parameter_count": 38, - "train_seconds": 0.4351, - "examples_per_second": 2409512.37, - "final_loss": 0.002556, + "feature_count": 140, + "word_features": 44, + "char_ngram_features": 96, + "hidden_units": 16, + "parameter_count": 2273, + "train_seconds": 0.3991, + "examples_per_second": 2134408.08, + "final_loss": 3e-06, "train_accuracy": 1.0, "test_accuracy": 1.0 } diff --git a/training/runs/latest.md b/training/runs/latest.md index 028266c..9ee89d8 100644 --- a/training/runs/latest.md +++ b/training/runs/latest.md @@ -2,14 +2,16 @@ | Metric | Value | | --- | ---: | +| Model type | mlp_text_classifier | | Device | mps | -| Epochs | 320 | +| Epochs | 260 | | Train examples | 3276 | | Test examples | 820 | -| Vocab size | 37 | -| Parameters | 38 | -| Train time | 0.4351 s | -| Examples/sec | 2409512.37 | -| Final loss | 0.002556 | +| Features | 140 | +| Hidden units | 16 | +| Parameters | 2273 | +| Train time | 0.3991 s | +| Examples/sec | 2134408.08 | +| Final loss | 3e-06 | | Train accuracy | 1.0 | | Test accuracy | 1.0 | diff --git a/training/train_text_model.py b/training/train_text_model.py index 5eef830..7470c4b 100644 --- a/training/train_text_model.py +++ b/training/train_text_model.py @@ -5,7 +5,9 @@ import random import re import time +from collections import Counter from pathlib import Path +from typing import Any import torch from torch import nn @@ -13,8 +15,8 @@ ROOT = Path(__file__).resolve().parents[1] OUTPUT_PATH = ROOT / "worker" / "app" / "artifacts" / "tiny_text_model.json" RUN_PATH = ROOT / "training" / "runs" / "latest.json" -RUN_MD_PATH = ROOT / "training" / "runs" / "latest.md" TOKEN_RE = re.compile(r"[a-z']+") +CHAR_NGRAM_RANGE = [3, 4] HEALTHY_PATTERNS = [ ("fast", "stable", "reliable", "inference"), @@ -23,6 +25,10 @@ ("cache", "hit", "keeps", "service", "fast"), ("request", "completed", "healthy", "worker"), ("reliable", "model", "route", "responsive"), + ("worker", "recovered", "after", "timeout", "stable"), + ("cache", "recovered", "request", "completed"), + ("gateway", "handled", "burst", "traffic", "stable"), + ("retry", "succeeded", "worker", "responsive"), ] RISKY_PATTERNS = [ @@ -32,6 +38,13 @@ ("service", "timeout", "broken", "response"), ("model", "route", "slow"), ("unhealthy", "worker", "returned", "error"), + ("cache", "hit", "worker", "failed", "timeout"), + ("great", "latency", "retry", "caused", "timeout"), + ("service", "healthy", "but", "slow"), + ("gateway", "retry", "loop", "timeout"), + ("unhealthy", "worker", "returned", "fast", "response"), + ("prediction", "failed", "after", "worker", "crash"), + ("mesh", "returned", "broken", "prediction", "error"), ] FILLER = [ @@ -46,11 +59,25 @@ ] +class TextMLP(nn.Module): + def __init__(self, feature_count: int, hidden_units: int) -> None: + super().__init__() + self.hidden = nn.Linear(feature_count, hidden_units) + self.output = nn.Linear(hidden_units, 1) + + def forward(self, features: torch.Tensor) -> torch.Tensor: + hidden = torch.relu(self.hidden(features)) + return self.output(hidden) + + def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser() - parser.add_argument("--epochs", type=int, default=320) - parser.add_argument("--lr", type=float, default=0.05) + parser.add_argument("--epochs", type=int, default=260) + parser.add_argument("--lr", type=float, default=0.03) parser.add_argument("--samples-per-class", type=int, default=2048) + parser.add_argument("--hidden-units", type=int, default=16) + parser.add_argument("--max-char-ngrams", type=int, default=96) + parser.add_argument("--model-version", default="tiny-text-mps-v2") parser.add_argument("--output", type=Path, default=OUTPUT_PATH) parser.add_argument("--run-output", type=Path, default=RUN_PATH) parser.add_argument("--allow-cpu", action="store_true") @@ -69,6 +96,14 @@ def tokens(text: str) -> list[str]: return TOKEN_RE.findall(text.lower()) +def char_ngrams(text: str, sizes: list[int]) -> list[str]: + return [ + text[index : index + size] + for size in sizes + for index in range(max(len(text) - size + 1, 0)) + ] + + def build_texts(samples_per_class: int, seed: int = 17) -> tuple[list[str], list[float]]: rng = random.Random(seed) texts: list[str] = [] @@ -76,12 +111,14 @@ def build_texts(samples_per_class: int, seed: int = 17) -> tuple[list[str], list for label, patterns in [(1.0, HEALTHY_PATTERNS), (0.0, RISKY_PATTERNS)]: for index in range(samples_per_class): - pattern = list(patterns[index % len(patterns)]) + pattern = list(rng.choice(patterns)) rng.shuffle(pattern) if index % 3 == 0: pattern.append(rng.choice(FILLER)) if index % 5 == 0: pattern.insert(0, rng.choice(FILLER)) + if index % 11 == 0: + pattern.append("live") texts.append(" ".join(pattern)) labels.append(label) @@ -90,20 +127,52 @@ def build_texts(samples_per_class: int, seed: int = 17) -> tuple[list[str], list return [texts[index] for index in order], [labels[index] for index in order] -def build_dataset(samples_per_class: int) -> tuple[list[str], torch.Tensor, torch.Tensor]: - texts, labels = build_texts(samples_per_class) - vocab = sorted({token for text in texts for token in tokens(text)}) - index = {token: pos for pos, token in enumerate(vocab)} - rows: list[list[float]] = [] - +def build_feature_spec(texts: list[str], max_char_ngrams: int) -> dict[str, Any]: + word_vocab = sorted({token for text in texts for token in tokens(text)}) + char_counts: Counter[str] = Counter() for text in texts: - row = [0.0] * len(vocab) - for token in tokens(text): - row[index[token]] += 1.0 - rows.append(row) + char_counts.update(char_ngrams(" ".join(tokens(text)), CHAR_NGRAM_RANGE)) + + return { + "word_vocab": word_vocab, + "char_ngrams": [ + ngram + for ngram, _ in char_counts.most_common(max_char_ngrams) + ], + "char_ngram_range": CHAR_NGRAM_RANGE, + "token_pattern": TOKEN_RE.pattern, + "lowercase": True, + } + + +def vectorize(text: str, feature_spec: dict[str, Any]) -> list[float]: + text_tokens = tokens(text) + token_counts = Counter(text_tokens) + char_counts = Counter( + char_ngrams(" ".join(text_tokens), feature_spec["char_ngram_range"]) + ) + + word_features = [ + float(token_counts.get(token, 0)) + for token in feature_spec["word_vocab"] + ] + char_features = [ + float(char_counts.get(ngram, 0)) + for ngram in feature_spec["char_ngrams"] + ] + return word_features + char_features + + +def build_dataset( + samples_per_class: int, + max_char_ngrams: int, +) -> tuple[dict[str, Any], torch.Tensor, torch.Tensor]: + texts, labels = build_texts(samples_per_class) + feature_spec = build_feature_spec(texts, max_char_ngrams) + rows = [vectorize(text, feature_spec) for text in texts] return ( - vocab, + feature_spec, torch.tensor(rows, dtype=torch.float32), torch.tensor(labels, dtype=torch.float32).view(-1, 1), ) @@ -118,7 +187,7 @@ def split_dataset( return x[:train_count], y[:train_count], x[train_count:], y[train_count:] -def accuracy(model: nn.Linear, x: torch.Tensor, y: torch.Tensor) -> float: +def accuracy(model: nn.Module, x: torch.Tensor, y: torch.Tensor) -> float: with torch.no_grad(): predictions = (torch.sigmoid(model(x)) >= 0.5).float() return float((predictions == y).float().mean().cpu()) @@ -134,16 +203,18 @@ def train( epochs: int, lr: float, samples_per_class: int, + hidden_units: int, + max_char_ngrams: int, device: torch.device, -) -> tuple[list[str], nn.Linear, dict[str, float | int | str]]: +) -> tuple[dict[str, Any], TextMLP, dict[str, float | int | str]]: torch.manual_seed(17) - vocab, x_cpu, y_cpu = build_dataset(samples_per_class) + feature_spec, x_cpu, y_cpu = build_dataset(samples_per_class, max_char_ngrams) x_train_cpu, y_train_cpu, x_test_cpu, y_test_cpu = split_dataset(x_cpu, y_cpu) x_train = x_train_cpu.to(device) y_train = y_train_cpu.to(device) x_test = x_test_cpu.to(device) y_test = y_test_cpu.to(device) - model = nn.Linear(len(vocab), 1).to(device) + model = TextMLP(x_cpu.shape[1], hidden_units).to(device) optimizer = torch.optim.Adam(model.parameters(), lr=lr) loss_fn = nn.BCEWithLogitsLoss() @@ -160,13 +231,17 @@ def train( train_seconds = time.perf_counter() - started stats = { + "model_type": "mlp_text_classifier", "device": str(device), "epochs": epochs, "learning_rate": lr, "samples_per_class": samples_per_class, "train_examples": int(len(x_train_cpu)), "test_examples": int(len(x_test_cpu)), - "vocab_size": len(vocab), + "feature_count": int(x_cpu.shape[1]), + "word_features": len(feature_spec["word_vocab"]), + "char_ngram_features": len(feature_spec["char_ngrams"]), + "hidden_units": hidden_units, "parameter_count": sum(parameter.numel() for parameter in model.parameters()), "train_seconds": round(train_seconds, 4), "examples_per_second": round((len(x_train_cpu) * epochs) / train_seconds, 2), @@ -174,26 +249,42 @@ def train( "train_accuracy": round(accuracy(model, x_train, y_train), 4), "test_accuracy": round(accuracy(model, x_test, y_test), 4), } - return vocab, model.cpu(), stats + return feature_spec, model.cpu(), stats def write_artifact( path: Path, - vocab: list[str], - model: nn.Linear, + model_version: str, + feature_spec: dict[str, Any], + model: TextMLP, stats: dict[str, float | int | str], ) -> None: - weights = model.weight.detach().reshape(-1).tolist() artifact = { - "model_version": "tiny-text-mps-v1", + "model_version": model_version, + "model_type": "mlp_text_classifier", + "feature_spec": feature_spec, + "weights": { + "hidden": rounded_matrix(model.hidden.weight.detach().tolist()), + "output": rounded_vector(model.output.weight.detach().reshape(-1).tolist()), + }, + "biases": { + "hidden": rounded_vector(model.hidden.bias.detach().tolist()), + "output": round(float(model.output.bias.detach()[0]), 6), + }, "training": stats, - "weights": {token: round(float(weights[index]), 6) for index, token in enumerate(vocab)}, - "bias": round(float(model.bias.detach()[0]), 6), } path.parent.mkdir(parents=True, exist_ok=True) path.write_text(json.dumps(artifact, indent=2) + "\n") +def rounded_vector(values: list[float]) -> list[float]: + return [round(float(value), 6) for value in values] + + +def rounded_matrix(values: list[list[float]]) -> list[list[float]]: + return [rounded_vector(row) for row in values] + + def write_run_files( json_path: Path, md_path: Path, @@ -216,11 +307,13 @@ def write_run_files( "", "| Metric | Value |", "| --- | ---: |", + f"| Model type | {stats['model_type']} |", f"| Device | {stats['device']} |", f"| Epochs | {stats['epochs']} |", f"| Train examples | {stats['train_examples']} |", f"| Test examples | {stats['test_examples']} |", - f"| Vocab size | {stats['vocab_size']} |", + f"| Features | {stats['feature_count']} |", + f"| Hidden units | {stats['hidden_units']} |", f"| Parameters | {stats['parameter_count']} |", f"| Train time | {stats['train_seconds']} s |", f"| Examples/sec | {stats['examples_per_second']} |", @@ -235,13 +328,15 @@ def write_run_files( def main() -> None: args = parse_args() device = choose_device(args.allow_cpu) - vocab, model, stats = train( + feature_spec, model, stats = train( epochs=args.epochs, lr=args.lr, samples_per_class=args.samples_per_class, + hidden_units=args.hidden_units, + max_char_ngrams=args.max_char_ngrams, device=device, ) - write_artifact(args.output, vocab, model, stats) + write_artifact(args.output, args.model_version, feature_spec, model, stats) write_run_files(args.run_output, args.run_output.with_suffix(".md"), stats, args.output) print(json.dumps({"output": str(args.output), **stats}, indent=2)) diff --git a/worker/app/artifacts/tiny_text_model.json b/worker/app/artifacts/tiny_text_model.json index 8091376..aae90b7 100644 --- a/worker/app/artifacts/tiny_text_model.json +++ b/worker/app/artifacts/tiny_text_model.json @@ -1,58 +1,2490 @@ { - "model_version": "tiny-text-mps-v1", + "model_version": "tiny-text-mps-v2", + "model_type": "mlp_text_classifier", + "feature_spec": { + "word_vocab": [ + "after", + "batch", + "broken", + "burst", + "but", + "cache", + "caused", + "client", + "completed", + "crash", + "error", + "failed", + "fast", + "gateway", + "great", + "handled", + "healthy", + "hit", + "inference", + "keeps", + "latency", + "live", + "loop", + "mesh", + "model", + "prediction", + "queue", + "recovered", + "reliable", + "request", + "response", + "responsive", + "retry", + "returned", + "route", + "service", + "slow", + "stable", + "succeeded", + "timeout", + "traffic", + "unhealthy", + "with", + "worker" + ], + "char_ngrams": [ + " re", + "ed ", + "er ", + "wor", + "ork", + "rke", + "ker", + "work", + "orke", + "rker", + "out", + "ker ", + " wo", + " wor", + "st ", + "abl", + "ble", + "able", + "ret", + "le ", + "ble ", + "ate", + "ut ", + "tim", + "ime", + "meo", + "eou", + "time", + "imeo", + "meou", + "eout", + "red", + "ive", + " ret", + "res", + "esp", + "spo", + "pon", + "ons", + "resp", + "espo", + "spon", + "pons", + "out ", + "hea", + "eal", + "alt", + "lth", + "thy", + "heal", + "ealt", + "alth", + "lthy", + " fa", + "que", + " ti", + " tim", + "sta", + "tab", + "stab", + "tabl", + "e r", + "ce ", + " res", + "hy ", + "thy ", + "ser", + "erv", + "rvi", + "vic", + "ice", + "serv", + "ervi", + "rvic", + "vice", + " ca", + " st", + " sta", + "led", + "gat", + "tew", + "ewa", + "way", + "gate", + "atew", + "tewa", + "eway", + "etr", + "try", + "retr", + "etry", + "e re", + "ice ", + "ere", + "led ", + "nsi" + ], + "char_ngram_range": [ + 3, + 4 + ], + "token_pattern": "[a-z']+", + "lowercase": true + }, + "weights": { + "hidden": [ + [ + 0.3867, + 0.402273, + 0.479234, + 0.313096, + 1.140571, + -0.17803, + 0.311318, + 0.308574, + -0.188837, + 0.308678, + 0.368563, + 0.497569, + 0.110315, + 0.351685, + 0.142011, + -0.380221, + -0.266599, + -0.758396, + -0.163209, + -0.685935, + 0.098157, + 0.404723, + 0.220256, + 0.378001, + 0.35612, + 0.520552, + 0.380821, + -0.239538, + -0.246837, + 0.120671, + 0.099251, + -0.128568, + -0.200013, + 0.342878, + 0.48601, + -0.161765, + 0.658654, + -0.150225, + -0.19197, + 0.305639, + 0.267678, + 0.142229, + 0.558258, + -0.061394, + -0.062211, + 0.304772, + 0.36698, + -0.077103, + -0.09247, + -0.017146, + 0.027103, + 0.049121, + -0.007936, + -0.068436, + 0.347343, + 0.021978, + 0.052686, + -0.060111, + 0.176398, + -0.234465, + -0.116481, + -0.232784, + 0.514547, + -0.235053, + -0.197538, + 0.333711, + 0.467857, + 0.295473, + 0.34484, + 0.289571, + 0.374218, + 0.364734, + 0.299461, + 0.266678, + 0.3126, + 0.446355, + -0.046374, + 0.469205, + -0.211457, + -0.262564, + -0.249144, + -0.116018, + -0.198094, + -0.143157, + -0.103588, + -0.202549, + -0.138253, + 0.375959, + -0.269282, + -0.140701, + -0.210294, + -0.180597, + -0.226377, + -0.176891, + -0.221376, + -0.197499, + -0.117355, + 0.446218, + 0.106006, + 0.347799, + 0.291338, + -0.234916, + -0.195263, + -0.270798, + -0.115535, + -0.123123, + -0.111693, + -0.1341, + -0.223063, + -0.193677, + -0.251507, + -0.269421, + -0.215252, + -0.244951, + -0.280786, + -0.156397, + -0.142585, + -0.234584, + -0.231001, + -0.238318, + -0.156326, + -0.152509, + 0.487216, + 0.354935, + 0.411413, + 0.447564, + 0.375188, + 0.330936, + 0.364698, + 0.344715, + 0.322269, + -0.213582, + -0.128365, + -0.233824, + -0.13098, + -0.170273, + -0.24009, + -0.134028, + 0.504346, + -0.159282 + ], + [ + 0.272696, + 0.245123, + -0.403815, + 0.336937, + -0.575138, + 0.529961, + -0.417757, + 0.125644, + 0.523914, + -0.499697, + -0.401481, + -0.443172, + 0.252354, + 0.263574, + 0.228572, + 0.331944, + 0.622158, + 0.598987, + 0.319595, + 0.507709, + 0.226471, + -0.23034, + -0.252784, + -0.299644, + -0.339422, + -0.412356, + -0.021974, + 0.362456, + 0.275569, + 0.507825, + -0.475665, + 0.479718, + -0.016878, + -0.548335, + -0.006147, + 0.34865, + -0.522277, + 0.354545, + 0.541147, + -0.161831, + 0.268436, + -0.614732, + -0.459781, + 0.292517, + 0.428149, + 0.368234, + 0.249034, + 0.259384, + 0.299997, + 0.340681, + 0.298976, + 0.355516, + 0.3783, + 0.26284, + -0.215139, + 0.271421, + 0.26377, + 0.346792, + 0.437543, + 0.422676, + 0.431364, + 0.311994, + -0.417675, + 0.469927, + 0.434837, + 0.38219, + -0.266128, + -0.093878, + -0.095351, + -0.164142, + -0.159542, + -0.173468, + -0.147035, + -0.106612, + -0.180383, + 0.464671, + 0.511019, + -0.369614, + 0.205335, + 0.186253, + 0.125663, + 0.167551, + 0.110796, + 0.209314, + 0.1658, + 0.090086, + 0.162067, + -0.174861, + -0.126081, + -0.237193, + -0.191065, + -0.217694, + -0.197657, + -0.21795, + -0.0843, + -0.220309, + -0.224114, + -0.324772, + 0.54102, + -0.15302, + -0.121673, + 0.378563, + 0.405752, + 0.339658, + 0.343037, + 0.283838, + 0.416482, + 0.16132, + -0.077882, + -0.152493, + 0.405533, + 0.410616, + 0.389626, + 0.402697, + 0.376365, + 0.403093, + 0.485556, + 0.390636, + 0.38702, + 0.448765, + 0.466697, + 0.447391, + -0.549432, + 0.345784, + 0.318366, + 0.382964, + 0.403042, + 0.400846, + 0.295744, + 0.413935, + 0.253402, + 0.000729, + 0.119629, + 0.092271, + 0.017108, + 0.358071, + 0.270242, + 0.498298, + -0.407732, + 0.472574 + ], + [ + -0.22396, + 0.333017, + 0.468299, + 0.209644, + 0.512161, + -0.341201, + 0.430461, + 0.359641, + -0.400015, + 0.523152, + 0.568345, + 0.576542, + -0.093343, + 0.265441, + 0.335584, + -0.430066, + -0.484345, + -0.265197, + -0.248922, + -0.462566, + 0.273372, + 0.481201, + 0.32822, + 0.490488, + 0.593097, + 0.409198, + 0.458657, + -0.403665, + -0.128707, + -0.232308, + 0.547929, + -0.40253, + 0.302935, + 0.540944, + 0.580123, + 0.145729, + 0.656574, + -0.399189, + -0.383995, + 0.353644, + 0.009787, + 0.443097, + 0.537163, + 0.090897, + 0.122588, + 0.28628, + 0.032362, + -0.013567, + -0.069986, + 0.006662, + 0.056559, + -0.072592, + 0.029392, + -0.029167, + 0.401216, + 0.051825, + 0.061409, + 0.097834, + -0.188245, + -0.424107, + -0.423003, + -0.467022, + 0.45804, + -0.330726, + -0.397971, + 0.360323, + 0.335993, + 0.22975, + 0.255278, + 0.206499, + 0.329263, + 0.295662, + 0.323971, + 0.286485, + 0.306514, + 0.110039, + -0.383275, + 0.533548, + 0.277893, + 0.153218, + 0.192843, + 0.137862, + 0.205501, + 0.134927, + 0.221444, + 0.154245, + 0.20196, + 0.242267, + 0.194759, + 0.219831, + 0.071214, + 0.236079, + 0.069742, + 0.105884, + 0.17172, + 0.222159, + 0.156512, + 0.545765, + -0.094306, + 0.268555, + 0.307374, + -0.369933, + -0.475072, + -0.416748, + -0.351792, + -0.021285, + 0.235092, + 0.293379, + -0.02693, + 0.043386, + 0.195629, + 0.08472, + 0.138834, + 0.061846, + 0.079026, + 0.15461, + 0.135013, + 0.152624, + 0.059163, + -0.004972, + -0.435138, + -0.47078, + 0.5923, + 0.275514, + 0.234441, + 0.316954, + 0.304357, + 0.250208, + 0.337751, + 0.274825, + 0.320304, + 0.309077, + 0.235232, + 0.256689, + 0.329329, + -0.012905, + 0.165636, + -0.321919, + 0.486268, + -0.525708 + ], + [ + -0.030567, + 0.449501, + 0.489806, + 0.424386, + 0.051875, + 0.287289, + 0.460519, + 0.407035, + -0.489158, + 0.501402, + 0.459017, + 0.531324, + 0.355261, + 0.303587, + 0.2344, + -0.269077, + -0.418169, + 0.301857, + 0.029192, + -0.539682, + 0.294694, + 0.610346, + 0.25594, + 0.510838, + 0.601813, + 0.58038, + 0.651078, + -0.435977, + -0.124472, + -0.449815, + 0.368425, + -0.599542, + 0.214297, + 0.518193, + 0.497767, + -0.536738, + 0.625124, + -0.452434, + -0.623761, + 0.049194, + 0.265023, + 0.478699, + 0.417146, + 0.226888, + 0.014898, + 0.447161, + 0.180868, + 0.119557, + 0.159083, + 0.144693, + 0.223752, + 0.204041, + 0.164717, + 0.240437, + 0.577025, + 0.230247, + 0.121183, + 0.257645, + 0.003296, + -0.52053, + -0.451727, + -0.520293, + 0.582174, + -0.392831, + -0.474942, + 0.262975, + -0.0383, + -0.020678, + 0.027872, + -0.059011, + -0.06121, + -0.021613, + -0.04033, + 0.020953, + 0.00311, + 0.333755, + -0.341373, + 0.552989, + -0.096523, + -0.139155, + -0.166394, + -0.192665, + -0.077535, + -0.161553, + -0.19527, + -0.141473, + -0.070942, + 0.058513, + 0.032793, + -0.107239, + -0.022357, + 0.01658, + -0.109799, + -0.101881, + -0.120984, + -0.118123, + -0.103421, + 0.479227, + -0.302051, + -0.090196, + -0.039721, + -0.460314, + -0.462986, + -0.510499, + -0.513477, + -0.206956, + -0.523881, + -0.042885, + -0.074004, + 0.004236, + -0.547009, + -0.416387, + -0.572326, + -0.578943, + -0.58006, + -0.522341, + -0.520369, + -0.513457, + -0.512884, + 0.499823, + -0.368864, + -0.41016, + 0.643907, + 0.279508, + 0.34619, + 0.233477, + 0.347456, + 0.290723, + 0.260578, + 0.391948, + 0.315045, + 0.127042, + 0.242256, + 0.176639, + 0.091827, + -0.205724, + -0.437067, + -0.378423, + 0.541389, + -0.481957 + ], + [ + -0.204982, + 0.333573, + 0.468104, + 0.440512, + 0.617342, + -0.18607, + 0.459018, + 0.284969, + -0.227401, + 0.450015, + 0.475662, + 0.551282, + -0.347185, + 0.310475, + 0.268124, + -0.33852, + -0.11628, + -0.145012, + -0.000561, + -0.333182, + 0.352571, + 0.470845, + 0.263421, + 0.321255, + 0.369936, + 0.542078, + 0.427442, + -0.268494, + -0.233092, + -0.03264, + 0.428796, + -0.240666, + 0.342366, + 0.37203, + 0.409739, + -0.030463, + 0.539208, + -0.460539, + -0.484767, + 0.271801, + 0.335701, + 0.32257, + 0.385801, + 0.089105, + -0.024027, + 0.152619, + 0.045404, + 0.092274, + 0.056486, + 0.025358, + 0.099297, + 0.093229, + 0.001199, + 0.115501, + 0.533484, + -0.019198, + 0.022851, + 0.046996, + -0.049114, + -0.430022, + -0.282946, + -0.340478, + 0.391348, + -0.423871, + -0.415505, + 0.365031, + 0.280272, + 0.297861, + 0.285264, + 0.33081, + 0.25238, + 0.271737, + 0.348396, + 0.202673, + 0.285193, + -0.011231, + -0.242131, + 0.306632, + -0.166927, + -0.201202, + -0.086359, + -0.048344, + -0.061174, + -0.051261, + -0.132681, + -0.065114, + -0.065723, + 0.345756, + -0.209332, + -0.146886, + -0.146866, + -0.208665, + -0.188202, + -0.084268, + -0.165619, + -0.147028, + -0.066992, + 0.543908, + 0.071721, + 0.285239, + 0.258387, + -0.394129, + -0.365673, + -0.474767, + -0.381957, + -0.234269, + 0.002961, + -0.065266, + -0.157161, + -0.07593, + 0.004205, + -0.0275, + 0.105087, + 0.056214, + 0.066311, + 0.025485, + -0.003975, + 0.058485, + 0.037249, + 0.33682, + -0.440769, + -0.34358, + 0.477582, + 0.384988, + 0.378134, + 0.306227, + 0.370364, + 0.425774, + 0.390795, + 0.357796, + 0.34601, + 0.226382, + 0.225003, + 0.308998, + 0.292417, + -0.162736, + 0.070719, + -0.288336, + 0.555093, + -0.266638 + ], + [ + -0.126891, + -0.282849, + 0.270132, + -0.168471, + 0.083057, + -0.106672, + 0.365814, + -0.261699, + -0.423666, + 0.525066, + 0.356457, + 0.382506, + 0.011429, + -0.253606, + -0.100816, + -0.259839, + -0.460856, + -0.165108, + -0.162054, + -0.215719, + -0.168552, + 0.070293, + 0.114341, + 0.24638, + -0.074156, + 0.424071, + 0.176187, + -0.257759, + -0.17517, + -0.441757, + 0.387357, + -0.291084, + 0.300571, + 0.411613, + -0.146888, + -0.252414, + 0.145605, + -0.144666, + -0.392107, + -0.004545, + -0.178751, + 0.374704, + 1.673367, + 0.225751, + -0.213935, + 0.118074, + 0.254201, + 0.295908, + 0.160405, + 0.268247, + 0.177652, + 0.229907, + 0.294322, + 0.15563, + -0.179048, + 0.154515, + 0.331283, + 0.308089, + -0.268264, + -0.169934, + -0.164283, + -0.201908, + 0.43234, + -0.20753, + -0.137864, + -0.218853, + -0.011979, + 0.02528, + 0.066959, + -0.017126, + 0.110659, + 0.068045, + 0.016414, + -0.00039, + -0.046713, + -0.036218, + -0.294166, + 0.252636, + -0.032421, + 0.02218, + 0.006294, + 0.004575, + -0.077248, + -0.04197, + 0.00156, + 0.039418, + 0.01116, + -0.042579, + 0.101321, + 0.086483, + -0.010073, + -0.025274, + 0.110585, + -0.016473, + 0.047533, + 0.053017, + 0.13181, + 0.158311, + -0.381231, + 0.102474, + 0.084998, + -0.195833, + -0.178024, + -0.227077, + -0.182145, + -0.211087, + -0.169334, + -0.039966, + -0.03366, + -0.031761, + -0.163695, + -0.137227, + -0.197736, + -0.131038, + -0.249216, + -0.113682, + -0.182392, + -0.115653, + -0.12878, + 0.551458, + -0.171701, + -0.140853, + 0.267463, + -0.152396, + -0.105748, + -0.172356, + -0.17349, + -0.155664, + -0.148494, + -0.260784, + -0.124419, + 0.240428, + 0.290641, + 0.344522, + 0.298412, + -0.096868, + -0.229984, + -0.21112, + 0.206327, + -0.339603 + ], + [ + -0.001424, + 0.232917, + 0.49892, + -0.055705, + 0.561163, + -0.251813, + 0.572835, + 0.269994, + -0.513744, + 0.521931, + 0.433947, + 0.610434, + 0.172127, + -0.005461, + -0.221821, + -0.232946, + -0.466946, + -0.303072, + -0.131613, + -0.365467, + -0.126469, + 0.408898, + 0.30238, + 0.505907, + 0.412498, + 0.486694, + 0.30442, + -0.425627, + -0.165649, + -0.564084, + 0.573217, + -0.247011, + 0.112336, + 0.553646, + 0.249232, + 0.057617, + 0.607824, + -0.177314, + -0.323826, + 0.167929, + -0.060146, + 0.612429, + 0.532703, + -0.037777, + -0.013012, + 0.200883, + 0.119318, + 0.006571, + 0.082665, + -0.028119, + -0.010932, + 0.076339, + 0.045475, + 0.025551, + 0.456568, + 0.103027, + 0.107613, + 0.113518, + -0.251881, + -0.282345, + -0.252923, + -0.313515, + 0.597703, + -0.233496, + -0.17116, + -0.036136, + 0.514847, + 0.119489, + 0.221391, + 0.129048, + 0.141922, + 0.095325, + 0.094658, + 0.120849, + 0.212802, + 0.30049, + -0.24624, + 0.531645, + 0.003265, + 0.101716, + 0.023069, + 0.146069, + 0.043757, + 0.13022, + 0.026947, + 0.168303, + 0.158793, + 0.044114, + 0.300009, + 0.371579, + 0.34614, + 0.233969, + 0.247438, + 0.373814, + 0.342017, + 0.319111, + 0.319754, + 0.519851, + -0.503114, + 0.13888, + 0.133086, + -0.202895, + -0.327085, + -0.265468, + -0.246538, + -0.109131, + 0.023724, + 0.043066, + 0.185214, + 0.239745, + 0.105276, + 0.154057, + 0.086972, + 0.15347, + 0.09126, + 0.065795, + 0.136922, + 0.063759, + 0.119748, + 0.220205, + -0.173469, + -0.282398, + 0.625118, + -0.001156, + -0.05257, + -0.130463, + -0.136941, + -0.081201, + -0.131378, + -0.076445, + -0.009384, + 0.218286, + 0.136118, + 0.093075, + 0.107208, + 0.023187, + 0.106602, + -0.245545, + 0.473182, + -0.275124 + ], + [ + 0.339302, + 0.150286, + -0.199544, + -0.154902, + -0.288915, + 0.038593, + -0.243676, + 0.092403, + 0.424735, + -0.18254, + -0.231701, + -0.121087, + 0.261279, + 0.432563, + 0.220406, + 0.369531, + 0.060509, + -0.155971, + 0.37149, + 0.553585, + 0.216042, + -0.242089, + -0.207326, + 0.379187, + -0.203024, + -0.146929, + -0.281367, + 0.368714, + 0.428381, + 0.391671, + -0.480322, + 0.733321, + 0.068952, + -0.549313, + 0.155171, + 0.372177, + -0.384292, + 0.330848, + 0.518338, + -0.098557, + -0.075243, + -0.398988, + -0.586538, + -0.002677, + 0.544848, + 0.112701, + 0.1722, + 0.023358, + 0.027294, + -0.034404, + -0.039708, + -0.033431, + 0.057395, + -0.079808, + 0.087256, + -0.031438, + -0.052695, + -0.05164, + 0.237201, + 0.494208, + 0.350902, + 0.376863, + -0.060944, + 0.471305, + 0.349707, + 0.248981, + -0.127263, + -0.115971, + -0.021731, + -0.081752, + -0.046057, + -0.121212, + 0.00207, + -0.110753, + 0.008307, + 0.578351, + 0.536729, + -0.088314, + 0.576398, + 0.559763, + 0.586474, + 0.64377, + 0.550958, + 0.685831, + 0.585757, + 0.59743, + 0.573013, + -0.075356, + -0.336267, + -0.278272, + -0.373981, + -0.347598, + -0.320213, + -0.372199, + -0.336832, + -0.23263, + -0.291136, + -0.043713, + 0.175888, + -0.006371, + -0.041189, + 0.324216, + 0.412834, + 0.414552, + 0.315609, + 0.394501, + 0.446894, + 0.677913, + -0.187508, + -0.270162, + 0.280355, + 0.286954, + 0.287615, + 0.26808, + 0.335226, + 0.271668, + 0.240709, + 0.282457, + 0.262167, + -0.126834, + 0.383679, + 0.428778, + -0.203579, + 0.333297, + 0.399369, + 0.367706, + 0.342531, + 0.361663, + 0.322165, + 0.40895, + 0.480111, + 0.072061, + 0.124857, + 0.009496, + 0.092039, + 0.280688, + 0.304144, + 0.258732, + -0.16763, + 0.682589 + ], + [ + 0.309312, + 0.182277, + -0.403771, + 0.309525, + -0.647411, + 0.4589, + -0.413965, + 0.159675, + 0.514869, + -0.602044, + -0.393188, + -0.498834, + 0.14916, + 0.264726, + 0.506969, + 0.356146, + 0.527086, + 0.563461, + 0.371975, + 0.558886, + 0.457683, + -0.236688, + -0.303563, + -0.296943, + -0.393216, + -0.482637, + -0.108781, + 0.483651, + 0.432334, + 0.592744, + -0.45943, + 0.46421, + -0.178959, + -0.577465, + 0.082499, + 0.369104, + -0.513273, + 0.41895, + 0.437146, + -0.191282, + 0.30199, + -0.598582, + -0.511089, + 0.312165, + 0.457721, + 0.424805, + 0.319098, + 0.2556, + 0.335452, + 0.394738, + 0.339942, + 0.305679, + 0.373971, + 0.406136, + -0.152346, + 0.301095, + 0.346329, + 0.27371, + 0.430024, + 0.338836, + 0.472863, + 0.4666, + -0.405445, + 0.36824, + 0.382292, + 0.379195, + -0.239683, + -0.051435, + -0.126335, + -0.097312, + -0.168332, + -0.172134, + -0.157341, + -0.130527, + -0.026327, + 0.443834, + 0.372032, + -0.379564, + 0.115895, + 0.070538, + 0.039023, + 0.123469, + 0.13829, + 0.173322, + 0.091076, + 0.126999, + 0.064636, + -0.100107, + -0.032067, + 0.034235, + -0.023389, + 0.030775, + 0.130733, + -0.01738, + 0.107599, + 0.118614, + 0.068915, + -0.461545, + 0.578675, + -0.041239, + -0.161267, + 0.44345, + 0.47314, + 0.366579, + 0.475637, + 0.334986, + 0.290572, + -0.038536, + 0.173288, + 0.195243, + 0.411373, + 0.27339, + 0.265333, + 0.283012, + 0.292406, + 0.406401, + 0.411637, + 0.378074, + 0.33581, + 0.378082, + 0.461351, + 0.479177, + -0.619406, + 0.224021, + 0.294311, + 0.219343, + 0.247709, + 0.356837, + 0.224848, + 0.245221, + 0.349464, + -0.203657, + -0.158188, + -0.131869, + -0.097875, + 0.288858, + 0.24051, + 0.384737, + -0.465468, + 0.501409 + ], + [ + -0.099265, + -0.232785, + 0.096301, + -0.251064, + 0.137951, + -0.146807, + 0.176344, + -0.205156, + -0.159856, + 0.225065, + 0.157596, + 0.17358, + -0.163004, + -0.102884, + -0.20598, + -0.248183, + -0.247989, + 0.169943, + -0.120511, + -0.187583, + -0.098328, + -0.148941, + 0.053671, + -0.246103, + -0.208257, + 0.168378, + -0.211635, + -0.172727, + -0.131568, + -0.119003, + 0.263362, + -0.11083, + -0.108927, + 0.085189, + -0.16295, + -0.262573, + 0.219278, + -0.203658, + -0.206106, + -0.236017, + -0.13494, + 0.075277, + 0.128094, + -0.156664, + -0.251508, + -0.132499, + -0.114993, + -0.216572, + -0.238546, + -0.264451, + -0.143748, + -0.232612, + -0.210224, + -0.177135, + -0.223593, + -0.176415, + -0.160495, + -0.191464, + -0.253298, + -0.178954, + -0.138378, + -0.149874, + 0.219695, + -0.185423, + -0.163466, + -0.135254, + -0.14485, + -0.256136, + -0.25898, + -0.250947, + -0.102574, + -0.218408, + -0.150926, + -0.100547, + -0.234504, + -0.215241, + -0.135861, + 0.181099, + -0.136256, + -0.175019, + -0.163282, + -0.102367, + -0.18663, + -0.107461, + -0.109013, + -0.193345, + -0.236557, + -0.174669, + 0.041462, + 0.02207, + -0.092712, + -0.004288, + -0.080358, + -0.013413, + 0.008674, + -0.049587, + 0.018767, + 0.105561, + -0.231101, + -0.247624, + -0.153563, + -0.149173, + -0.174304, + -0.199191, + -0.103258, + -0.236841, + -0.127387, + -0.221756, + -0.186059, + -0.254615, + -0.156241, + -0.209729, + -0.240144, + -0.201, + -0.101385, + -0.111232, + -0.258156, + -0.224593, + -0.129668, + 0.174169, + -0.122118, + -0.203436, + -0.105895, + -0.169971, + -0.246221, + -0.141434, + -0.191106, + -0.224767, + -0.205367, + -0.196826, + -0.165345, + -0.245482, + -0.258584, + -0.185661, + -0.235678, + -0.098763, + -0.121944, + -0.189815, + -0.182721, + -0.222477 + ], + [ + -0.370343, + 0.238899, + 0.385887, + -0.076976, + 0.475478, + -0.60985, + 0.566573, + 0.125937, + -0.267855, + 0.428946, + 0.465226, + 0.31705, + -0.298268, + 0.141722, + 0.023697, + -0.18608, + -0.128154, + -0.332021, + -0.176172, + -0.575004, + 0.047153, + 0.37052, + 0.343499, + 0.381129, + -0.001209, + 0.338864, + 0.128965, + -0.234041, + -0.176408, + -0.305084, + 0.45248, + -0.278201, + 0.502323, + 0.445845, + 0.025947, + 0.292747, + 0.637395, + -0.230149, + -0.252598, + 0.373014, + -0.171868, + 0.352724, + 0.564363, + -0.089433, + -0.008638, + 0.069325, + -0.270493, + -0.123295, + -0.168993, + -0.17177, + -0.110682, + -0.224684, + -0.215648, + -0.12002, + 0.304449, + -0.076091, + -0.062451, + -0.20175, + -0.215771, + -0.187435, + -0.191894, + -0.202816, + 0.448816, + -0.168047, + -0.254448, + 0.027252, + 0.438159, + 0.308303, + 0.304293, + 0.365725, + 0.278425, + 0.29358, + 0.357428, + 0.348074, + 0.285831, + -0.228293, + -0.193447, + 0.521471, + 0.101711, + 0.011888, + -0.038487, + 0.067687, + 0.079084, + -0.02656, + -0.039021, + 0.076307, + 0.009323, + 0.335694, + 0.343135, + 0.288804, + 0.316305, + 0.342199, + 0.410596, + 0.306238, + 0.396779, + 0.339901, + 0.416491, + 0.100613, + -0.227878, + 0.311774, + 0.347904, + -0.234717, + -0.25624, + -0.193475, + -0.236738, + -0.086268, + 0.14282, + 0.024717, + 0.337704, + 0.346737, + 0.296523, + 0.200484, + 0.221576, + 0.295962, + 0.261568, + 0.188307, + 0.225119, + 0.275705, + 0.162177, + 0.041352, + -0.301996, + -0.312129, + 0.49371, + 0.009951, + 0.080282, + 0.059581, + 0.079082, + 0.06, + 0.114713, + 0.020463, + 0.019949, + 0.366181, + 0.398643, + 0.427938, + 0.354383, + -0.039005, + 0.268143, + -0.340115, + 0.501651, + -0.223449 + ], + [ + -0.203727, + 0.267027, + 0.509152, + 0.016436, + 0.584513, + -0.563357, + 0.50406, + 0.314468, + -0.469723, + 0.459761, + 0.471964, + 0.531539, + 0.138111, + 0.179293, + 0.094965, + -0.101213, + -0.604911, + -0.495787, + -0.221682, + -0.519804, + 0.113349, + 0.477377, + 0.31155, + 0.365189, + 0.683673, + 0.406433, + 0.436968, + -0.348002, + -0.13456, + -0.480205, + 0.605496, + -0.439235, + 0.161829, + 0.4998, + 0.325695, + 0.055298, + 0.634415, + -0.27276, + -0.428272, + 0.200275, + -0.131447, + 0.617175, + 0.609138, + -0.026525, + 0.061214, + 0.216716, + -0.115684, + -0.05758, + -0.141663, + -0.124739, + -0.097508, + -0.048593, + -0.133617, + -0.024824, + 0.417998, + -0.012982, + -0.04772, + -0.047982, + -0.259744, + -0.320849, + -0.308392, + -0.262591, + 0.492889, + -0.27634, + -0.359705, + 0.116033, + 0.462898, + 0.278186, + 0.28552, + 0.234393, + 0.195414, + 0.325183, + 0.258016, + 0.280089, + 0.232358, + 0.027982, + -0.521547, + 0.528283, + 0.13783, + 0.235365, + 0.236204, + 0.23502, + 0.183428, + 0.194125, + 0.151178, + 0.166916, + 0.166889, + 0.259932, + 0.385696, + 0.308082, + 0.25115, + 0.333866, + 0.413236, + 0.288323, + 0.358117, + 0.324406, + 0.263419, + 0.503873, + -0.457238, + 0.301496, + 0.195946, + -0.266427, + -0.282083, + -0.25847, + -0.328396, + -0.073316, + -0.017529, + 0.21275, + 0.183414, + 0.201907, + 0.138981, + 0.118178, + 0.131781, + 0.087915, + 0.086087, + 0.031109, + 0.115175, + 0.083363, + 0.056467, + 0.138013, + -0.295136, + -0.233301, + 0.604105, + 0.184239, + 0.152925, + 0.085054, + 0.168932, + 0.135439, + 0.174591, + 0.073766, + 0.124564, + 0.181335, + 0.288841, + 0.181839, + 0.292868, + -0.104164, + 0.298215, + -0.273681, + 0.460646, + -0.525021 + ], + [ + -0.133584, + 0.201222, + 0.387295, + 0.028906, + 0.206105, + -0.164399, + 0.453336, + 0.098203, + -0.182813, + 0.518641, + 0.565598, + 0.541508, + -0.137912, + 0.135625, + -0.054013, + -0.227329, + -0.190911, + 0.038998, + -0.111918, + -0.222484, + -0.09337, + 0.192431, + 0.271194, + 0.509177, + 0.044681, + 0.418608, + 0.147079, + -0.35116, + -0.17402, + 0.060562, + 0.377309, + -0.249048, + 0.363654, + 0.337578, + -0.076844, + -0.040016, + 0.407103, + -0.322803, + -0.438373, + 0.216613, + -0.10623, + 0.211058, + 0.588798, + 0.198375, + 0.046983, + 0.180264, + 0.154743, + 0.18814, + 0.152646, + 0.095173, + 0.198467, + 0.220598, + 0.056229, + 0.06583, + 0.265694, + 0.125405, + 0.09126, + 0.225812, + 0.006027, + -0.158765, + -0.286816, + -0.224704, + 0.328409, + -0.277029, + -0.163741, + 0.0592, + 0.334093, + 0.3223, + 0.303542, + 0.244958, + 0.27723, + 0.18138, + 0.333757, + 0.314433, + 0.189134, + 0.182417, + -0.136848, + 0.313406, + -0.148217, + -0.118094, + -0.153419, + -0.218939, + -0.243873, + -0.094713, + -0.223922, + -0.187872, + -0.187084, + 0.227099, + -0.092393, + -0.203561, + -0.191022, + -0.221587, + -0.109228, + -0.101005, + -0.156038, + -0.132923, + -0.184991, + 0.525655, + 0.101829, + 0.238207, + 0.175697, + -0.208184, + -0.248825, + -0.18689, + -0.331706, + -0.141458, + -0.060958, + -0.147549, + -0.115268, + -0.200833, + 0.00977, + 0.040026, + -0.08795, + -0.078275, + 0.063752, + 0.016265, + -0.052069, + 0.073386, + 0.014176, + 0.201818, + -0.189744, + -0.232845, + 0.567533, + 0.099797, + 0.156761, + 0.138019, + 0.006982, + 0.108223, + 0.084815, + 0.075798, + 0.155734, + 0.443529, + 0.416447, + 0.281598, + 0.290171, + -0.143607, + 0.055411, + -0.309444, + 0.582225, + -0.268811 + ], + [ + 0.083438, + 0.234234, + -0.34302, + 0.307922, + -0.554831, + 0.566198, + -0.447558, + 0.070383, + 0.519158, + -0.679429, + -0.519522, + -0.471302, + 0.00681, + 0.061008, + 0.389625, + 0.313452, + 0.56705, + 0.639637, + 0.329924, + 0.536114, + 0.428652, + -0.412418, + -0.193478, + -0.151993, + -0.227284, + -0.328168, + -0.033594, + 0.427306, + 0.431425, + 0.591567, + -0.613768, + 0.467192, + 0.268957, + -0.505778, + 0.100658, + 0.231001, + -0.607705, + 0.457039, + 0.553415, + -0.106748, + 0.279548, + -0.601736, + -0.587881, + 0.410909, + 0.456494, + 0.296983, + 0.202702, + 0.391689, + 0.337435, + 0.296151, + 0.365673, + 0.25359, + 0.355119, + 0.385256, + -0.331543, + 0.378469, + 0.311149, + 0.246267, + 0.555059, + 0.473622, + 0.443949, + 0.434189, + -0.466176, + 0.403077, + 0.435733, + 0.323028, + -0.468832, + -0.126448, + -0.143159, + -0.123785, + -0.162397, + -0.222111, + -0.185449, + -0.245471, + -0.145756, + 0.277057, + 0.533849, + -0.468077, + 0.051628, + 0.141258, + 0.121349, + 0.179819, + 0.139863, + 0.089526, + 0.15896, + 0.150878, + 0.079463, + -0.203113, + -0.176008, + -0.130825, + -0.16062, + -0.021634, + -0.170164, + -0.017048, + -0.174415, + -0.122048, + -0.088271, + -0.401915, + 0.544178, + -0.201411, + -0.207239, + 0.440351, + 0.40919, + 0.414912, + 0.459056, + 0.340052, + 0.273749, + 0.017724, + 0.088773, + 0.016006, + 0.151903, + 0.111232, + 0.220693, + 0.208381, + 0.086786, + 0.239396, + 0.24394, + 0.116618, + 0.142201, + 0.398883, + 0.514001, + 0.383559, + -0.491824, + 0.158132, + 0.093088, + 0.073305, + 0.075879, + 0.178868, + 0.062654, + 0.087264, + 0.110847, + 0.265678, + 0.254876, + 0.133624, + 0.243014, + 0.374104, + -0.002353, + 0.526167, + -0.471343, + 0.495206 + ], + [ + -0.211847, + 0.480184, + 0.450167, + 0.514848, + 0.498838, + -0.349616, + 0.478913, + 0.427208, + -0.5561, + 0.535852, + 0.440465, + 0.464842, + 0.340828, + 0.276468, + 0.341241, + -0.221299, + -0.562328, + -0.364372, + -0.027621, + -0.458608, + 0.388081, + 0.522389, + 0.209305, + 0.435726, + 0.447112, + 0.412036, + 0.623166, + -0.511573, + -0.225963, + -0.583871, + 0.478482, + -0.525903, + 0.254938, + 0.414556, + 0.468843, + 0.13061, + 0.519115, + -0.386678, + -0.549695, + 0.301007, + 0.490181, + 0.564625, + 0.476294, + 0.031869, + 0.338498, + 0.305614, + 0.02305, + 0.034617, + -0.027927, + -0.019252, + 0.01633, + 0.020472, + -0.089656, + -0.049188, + 0.529532, + 0.049355, + 0.09611, + 0.025818, + -0.050835, + -0.456548, + -0.546429, + -0.475892, + 0.503925, + -0.418638, + -0.405298, + 0.331551, + 0.45334, + 0.241232, + 0.274817, + 0.203826, + 0.18818, + 0.241338, + 0.292886, + 0.335872, + 0.264775, + 0.246264, + -0.464036, + 0.487326, + 0.325721, + 0.378457, + 0.343617, + 0.3332, + 0.283275, + 0.383172, + 0.388342, + 0.426304, + 0.294811, + 0.268848, + 0.30408, + 0.189378, + 0.194664, + 0.153588, + 0.150969, + 0.272864, + 0.144802, + 0.189691, + 0.175848, + 0.352951, + -0.438381, + 0.205849, + 0.213017, + -0.53608, + -0.46775, + -0.509997, + -0.499352, + 0.002562, + 0.139528, + 0.394499, + 0.11997, + 0.097746, + 0.026797, + 0.077171, + 0.090097, + 0.120099, + 0.008206, + -0.034104, + 0.03524, + 0.091722, + 0.129294, + 0.140231, + -0.499731, + -0.499355, + 0.514196, + 0.349932, + 0.291663, + 0.382392, + 0.270332, + 0.329717, + 0.364076, + 0.264439, + 0.297217, + 0.294894, + 0.158827, + 0.162485, + 0.242053, + -0.004233, + 0.172816, + -0.500704, + 0.509463, + -0.575726 + ], + [ + 0.037831, + 0.24724, + -0.367559, + 0.296792, + -0.601942, + 0.441153, + -0.566687, + 0.134591, + 0.618302, + -0.477066, + -0.434318, + -0.48316, + 0.063725, + 0.207326, + 0.408093, + 0.282315, + 0.61788, + 0.494512, + 0.403852, + 0.529219, + 0.401642, + -0.196855, + -0.282289, + 0.07665, + -0.365541, + -0.530532, + -0.051021, + 0.534082, + 0.438497, + 0.62968, + -0.57637, + 0.474967, + 0.221254, + -0.586289, + 0.094187, + 0.095541, + -0.560282, + 0.351784, + 0.530831, + -0.123233, + 0.245392, + -0.465334, + -0.526073, + 0.401648, + 0.321028, + 0.415346, + 0.29066, + 0.414915, + 0.432296, + 0.358493, + 0.44249, + 0.430048, + 0.413803, + 0.441142, + 0.012568, + 0.280211, + 0.280466, + 0.301844, + 0.4063, + 0.340995, + 0.384193, + 0.494601, + -0.560026, + 0.350939, + 0.494382, + 0.338519, + -0.44419, + -0.082731, + 0.020497, + -0.13208, + -0.047711, + -0.070282, + -0.077793, + -0.062923, + -0.11455, + 0.423563, + 0.412021, + -0.450412, + 0.073571, + 0.133774, + 0.141229, + 0.221487, + 0.097855, + 0.171077, + 0.156439, + 0.185568, + 0.200817, + -0.056457, + -0.20906, + -0.116428, + -0.117637, + -0.184272, + -0.247039, + -0.23329, + -0.116265, + -0.193766, + -0.174145, + -0.529188, + 0.534836, + -0.077109, + -0.061434, + 0.487735, + 0.490552, + 0.400684, + 0.43484, + 0.404208, + 0.231109, + 0.033475, + 0.005785, + -0.143127, + 0.206775, + 0.235732, + 0.189288, + 0.225125, + 0.142756, + 0.164701, + 0.189489, + 0.215718, + 0.154327, + 0.460077, + 0.379084, + 0.459909, + -0.547825, + 0.284389, + 0.27739, + 0.339847, + 0.280071, + 0.284483, + 0.222081, + 0.255232, + 0.339244, + 0.246766, + 0.335021, + 0.30406, + 0.332264, + 0.294298, + 0.063462, + 0.374038, + -0.542418, + 0.445867 + ] + ], + "output": [ + -0.546167, + 0.62846, + -0.724078, + -0.582415, + -0.597187, + -0.187931, + -0.518283, + 0.418625, + 0.695684, + 0.150356, + -0.475633, + -0.545907, + -0.570697, + 0.602304, + -0.640042, + 0.669839 + ] + }, + "biases": { + "hidden": [ + 0.440926, + 0.350698, + 0.47448, + 0.538052, + 0.264715, + -0.048687, + 0.328629, + 0.012234, + 0.306348, + -0.17581, + 0.058788, + 0.410983, + 0.263684, + 0.253786, + 0.400182, + 0.305915 + ], + "output": -0.124915 + }, "training": { + "model_type": "mlp_text_classifier", "device": "mps", - "epochs": 320, - "learning_rate": 0.05, + "epochs": 260, + "learning_rate": 0.03, "samples_per_class": 2048, "train_examples": 3276, "test_examples": 820, - "vocab_size": 37, - "parameter_count": 38, - "train_seconds": 0.4351, - "examples_per_second": 2409512.37, - "final_loss": 0.002556, + "feature_count": 140, + "word_features": 44, + "char_ngram_features": 96, + "hidden_units": 16, + "parameter_count": 2273, + "train_seconds": 0.3991, + "examples_per_second": 2134408.08, + "final_loss": 3e-06, "train_accuracy": 1.0, "test_accuracy": 1.0 - }, - "weights": { - "batch": -0.070073, - "broken": -1.928928, - "burst": -0.248643, - "cache": 1.540021, - "caused": -2.06341, - "client": -0.141606, - "completed": 2.38669, - "crash": -1.893442, - "error": -2.172957, - "failed": -2.588806, - "fast": 1.522728, - "gateway": -0.207467, - "great": 1.918606, - "healthy": 2.235116, - "hit": 1.615801, - "inference": 1.602651, - "keeps": 1.49933, - "latency": 1.897305, - "mesh": -0.172917, - "model": -2.128533, - "prediction": 0.031068, - "queue": -0.110654, - "reliable": 3.177283, - "request": 1.983225, - "response": -2.478622, - "responsive": 3.435494, - "retry": -1.880252, - "returned": -1.960394, - "route": 1.494559, - "service": 1.518566, - "slow": -3.264679, - "stable": 1.973113, - "timeout": -2.023214, - "traffic": -0.111536, - "unhealthy": -1.699983, - "with": -2.362207, - "worker": 0.302216 - }, - "bias": -0.955532 + } } diff --git a/worker/app/model.py b/worker/app/model.py index 8380ad4..9ea68ec 100644 --- a/worker/app/model.py +++ b/worker/app/model.py @@ -3,8 +3,10 @@ import json import math import re +from collections import Counter from dataclasses import dataclass from pathlib import Path +from typing import Any TOKEN_RE = re.compile(r"[a-z']+") DEFAULT_ARTIFACT = Path(__file__).resolve().parent / "artifacts" / "tiny_text_model.json" @@ -21,11 +23,31 @@ } +def sigmoid(score: float) -> float: + if score >= 0: + return 1 / (1 + math.exp(-score)) + exp_score = math.exp(score) + return exp_score / (1 + exp_score) + + +def tokens(text: str) -> list[str]: + return TOKEN_RE.findall(text.lower()) + + +def ngrams(text: str, sizes: list[int]) -> list[str]: + return [ + text[index : index + size] + for size in sizes + for index in range(max(len(text) - size + 1, 0)) + ] + + @dataclass(frozen=True) class TinyTextModel: version: str = "tiny-text-v1" weights: dict[str, float] | None = None bias: float = 0.0 + model_type: str = "linear_bow" @classmethod def from_artifact(cls, path: Path) -> TinyTextModel: @@ -37,10 +59,10 @@ def from_artifact(cls, path: Path) -> TinyTextModel: ) def predict(self, text: str) -> dict[str, str | float]: - tokens = TOKEN_RE.findall(text.lower()) + text_tokens = tokens(text) weights = self.weights or DEFAULT_WEIGHTS - score = self.bias + sum(weights.get(token, 0.0) for token in tokens) - probability = 1 / (1 + math.exp(-score)) + score = self.bias + sum(weights.get(token, 0.0) for token in text_tokens) + probability = sigmoid(score) label = "healthy" if probability >= 0.5 else "risky" confidence = probability if label == "healthy" else 1 - probability @@ -51,8 +73,70 @@ def predict(self, text: str) -> dict[str, str | float]: } -def load_model(path: str | None = None) -> TinyTextModel: +@dataclass(frozen=True) +class MLPTextModel: + version: str + feature_spec: dict[str, Any] + hidden_weights: list[list[float]] + hidden_biases: list[float] + output_weights: list[float] + output_bias: float + model_type: str = "mlp_text_classifier" + + @classmethod + def from_artifact(cls, artifact: dict[str, Any]) -> MLPTextModel: + return cls( + version=artifact["model_version"], + feature_spec=artifact["feature_spec"], + hidden_weights=artifact["weights"]["hidden"], + hidden_biases=artifact["biases"]["hidden"], + output_weights=artifact["weights"]["output"], + output_bias=float(artifact["biases"]["output"]), + ) + + def predict(self, text: str) -> dict[str, str | float]: + features = self._features(text) + hidden = [ + max(0.0, bias + dot(row, features)) + for row, bias in zip(self.hidden_weights, self.hidden_biases) + ] + probability = sigmoid(self.output_bias + dot(self.output_weights, hidden)) + label = "healthy" if probability >= 0.5 else "risky" + confidence = probability if label == "healthy" else 1 - probability + + return { + "label": label, + "score": round(confidence, 4), + "model_version": self.version, + } + + def _features(self, text: str) -> list[float]: + text_tokens = tokens(text) + token_counts = Counter(text_tokens) + char_source = " ".join(text_tokens) + char_sizes = self.feature_spec.get("char_ngram_range", [3, 4]) + char_counts = Counter(ngrams(char_source, char_sizes)) + + word_features = [ + float(token_counts.get(token, 0)) + for token in self.feature_spec["word_vocab"] + ] + char_features = [ + float(char_counts.get(ngram, 0)) + for ngram in self.feature_spec["char_ngrams"] + ] + return word_features + char_features + + +def dot(weights: list[float], values: list[float]) -> float: + return sum(weight * value for weight, value in zip(weights, values)) + + +def load_model(path: str | None = None) -> TinyTextModel | MLPTextModel: artifact_path = Path(path) if path else DEFAULT_ARTIFACT if artifact_path.exists(): + artifact = json.loads(artifact_path.read_text()) + if artifact.get("model_type") == "mlp_text_classifier": + return MLPTextModel.from_artifact(artifact) return TinyTextModel.from_artifact(artifact_path) return TinyTextModel()