Skip to content

Repository files navigation

BioDevOps reproducibility repository

This portable snapshot supports developmental assurance claims. It is not a clinical validation package.

Layers

formal_models contains bounded models and controls; agent_specification contains frozen prompts, advisory schemas and the bounded orchestrator; executable_governance contains Rego, SHACL/OWL, mappings and conformance tests; evaluation_protocol contains rubrics and labels; data_availability contains complete synthetic data and MAUDE identifiers/digests only; results contains stored auditable summaries, including the ten-model RQ2/RQ4 matched campaign (raw per-arm traces and paired_results.json), the two-series scaling sweep and retained single-series qwen2.5 sweep, the field-provenance/redundancy audit, the qwen2.5 natural-output raw inputs in results/qwen25_natural_output_raw/, and the SHACL-informed in-loop revision campaigns (qwen2.5:3b frozen baseline plus the qwen3.5:9b/gemma3:12b cross-series extension). Mock fallback runs are not empirical evidence.

Reproduction

From biodevops_rag, use Python 3.10+, install requirements.txt, build the vector index with scripts/ingest_corpus.py, then run python -m unittest tests/test_bounded_agent.py tests/test_transition_matrix.py tests/test_planner_contract.py tests/test_inloop_revision.py tests/test_governance_conformance.py. Use local Ollama for real inference and record model digest, seed, and source in every trace. The matched development and held-out bounded-agent campaigns (agent_specification/matched_agentic_campaign/), the RQ2 ten-model campaign, and the in-loop revision campaigns under results/ are frozen, hash-authorized, one-shot runs; do not rerun them to "reproduce" a table --- rerun the table-regeneration scripts against the stored raw outputs instead (rq2_multimodel_cluster_ci.py, build_scaling_summary_table.py, redundancy_audit.py, inloop_revision_cross_series_20260806/aggregate_inloop_cross_series.py, inloop_revision_cross_series_20260806/verify_rq2_table.py). scripts/run_bounded_arm_inloop_revision.py is the runner that produced the in-loop revision campaigns and can be rerun on new scenarios or models under the same frozen protocol.

Data availability and reuse

Synthetic scenarios are included and marked synthetic. FDA MAUDE narratives are not redistributed: maude_record_index.json provides FDA IDs, URLs and SHA-256 provenance. No license is asserted for third-party standards, ontologies, model weights, or FDA source text.

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages