model-auditing

AI Red Team Operations Console

A structured audit framework for ML pipelines. 14 stages, 100+ checks, and every leakage pattern that silent model failures are made of.

Steering vectors with receipts: make one, catch one, deploy a calibrated one. pip install hidden-directions

Autonomous Update Review Architecture for Federated Learning. A full-stack platform for inspecting, scoring, and auditing model updates using SHAP explainability, anomaly detection, and ledger-based tracking. Built with FastAPI, React 19, and Flower.

Third-party-verifiable model-authenticity audit for the javis.bot Claude/GPT relay — open-source probe, runs on GitHub Actions, code/runs/results all public.

Shift type, not magnitude, determines ML failure modes under deployment shift — a cross-domain audit protocol and benchmark

Reproducible pipeline for silent-failure auditing in ECG accept-sets (MIT-BIH) with Newton–Puiseux onset scoring

A production-grade LLM Evaluation & Benchmarking Framework for systematic model auditing. Features parallel benchmarking, fairness/bias detection, MMLU integration, and a real-time analytics dashboard powered by React and FastAPI.

A method-neutral protocol for recording and verifying evidence that latent model states causally influenced downstream decisions, outputs, or actions, without requiring storage of raw internal activations.

Open-source toolkit for Fourier-based audits of trained quantum neural networks.

Production-grade platform for auditing AI/ML models for fairness, explainability, and robustness. JWT auth, SHA-256 model verification, paginated REST API, dark web dashboard. 30/30 tests passing.

🔬 1- A Human-Centered AI & Data Science hub for rigorous Machine Learning model evaluation, comparison, auditing, and selection, combining performance analysis, validation, hyperparameter optimization, calibration, reproducibility, and responsible real-world applications.

Audit a public AI text classifier for bias and explain its decisions. Counterfactual fairness probing (Equity Evaluation Corpus style) + occlusion-based token attribution, on a black-box model using only inputs and outputs. Python · Transformers · PyTorch.

studying model misalignment via spectral analysis

Workspace-lens audit cards and training-workflow scaffolds for open-weight language models.

Calibration-backed auditing of secret-loyalty model organisms — Apart Research Hackathon 2026

Research-grade, reproducible clinical-LLM auditing and mitigation workbench. Research use only.

SHAP-based bias detection and model interpretability auditing framework. Feature importance analysis and ML audit automation.

Subgroup-stratified, calibration-aware fairness auditing for ML models: DeLong AUC confidence intervals, per-subgroup calibration error, multiple-comparison-corrected significance, and a novel five-axis cross-platform protocol (CPFE). Grounded in peer-reviewed methods.