Repository navigation

Add llm evaluation against human accessibility audits - #5

Open

reiganarmstrong wants to merge 152 commits into

mainfrom

feat/model-evaluation

Open

Add llm evaluation against human accessibility audits#5
reiganarmstrong wants to merge 152 commits into
mainfrom
feat/model-evaluation

Commits on Sep 19, 2026
- chore(git): ignore generated web artifacts
  reiganarmstrong
  committed
  aecb061
- chore(agents): configure repository workflows
  reiganarmstrong
  committed
  74e9b0d
- docs(evaluation): record domain and architecture decisions
  reiganarmstrong
  committed
  21d5734
- docs(evaluation): add model evaluation specification
  reiganarmstrong
  committed
  fe861a6
- docs(evaluation): add implementation ticket graph
  reiganarmstrong
  committed
  9cc5b95
- feat(audit): add explicit request configuration
  reiganarmstrong
  committed
  c371979
- style(audit): address review findings
  reiganarmstrong
  committed
  c395efa
- feat(evaluation): bootstrap private CLI workspace
  reiganarmstrong
  committed
  aa77ee9
- fix(evaluation): enforce private workspace safety
  reiganarmstrong
  committed
  b99017f
- fix(evaluation): anchor private paths to worktree
  reiganarmstrong
  committed
  e9485bf
- docs(agents): define ticket resolution convention
  reiganarmstrong
  committed
  e3de4fa
- feat(evaluation): freeze immutable benchmark snapshot
  reiganarmstrong
  committed
  0bd719c
- fix(evaluation): harden snapshot publication
  reiganarmstrong
  committed
  fc630cb
- docs(evaluation): resolve benchmark snapshot ticket
  reiganarmstrong
  committed
  e4f9a55
- feat(evaluation): complete guarded model evaluation workflow
  reiganarmstrong
  committed
  472500c
- fix(evaluation): enforce execution and review invariants
  reiganarmstrong
  committed
  8bfe6af
- style(evaluation): wrap CLI fixture metadata
  reiganarmstrong
  committed
  1cb479a
- fix(evaluation): bind reports to verified run inputs
  reiganarmstrong
  committed
  2129616
- docs(evaluation): resolve framework implementation tickets
  reiganarmstrong
  committed
  68d34d9
- feat(evaluation): streamline run-oriented CLI workflow
  reiganarmstrong
  committed
  2448042
- fix(evaluation): bind execution to planned run destination
  reiganarmstrong
  committed
  6a1e6c0
- docs(evaluation): resolve run-oriented CLI workflow
  reiganarmstrong
  committed
  442c4cb
Commits on Sep 20, 2026
Commits on Sep 22, 2026
Commits on Sep 23, 2026
Commits on Sep 24, 2026
Commits on Sep 27, 2026
- feat(skills): add link to matt pocock skills for claude
  reiganarmstrong
  committed
  28c8a86
- docs(evaluation): explain homepage run JSON files
  reiganarmstrong
  committed
  9c014de
- refactor(evaluation)!: name raw and normalized audit files explicitly
  
  reiganarmstrong
  and
  claude
  committed
  d4a87cf
- docs(scratch): resolve rename-audit-artifacts spec
  
  reiganarmstrong
  and
  claude
  committed
  14c07a7
- refactor(evaluation): rename pricing.v2.json to pricing.json
  
  reiganarmstrong
  and
  claude
  committed
  25cd709
- fix(evaluation): render original defect evidence verbatim in report.md
  
  reiganarmstrong
  and
  claude
  committed
  30ca95f
Commits on Sep 29, 2026
Commits on Oct 1, 2026
- fix(evaluation): clarify report checks and format explanation bullets
  reiganarmstrong
  committed
  b8e3a36
- feat(evaluation)!: organize supporting run artifacts
  reiganarmstrong
  committed
  1bc12d9
- docs(tracker): resolve run directory layout spec
  reiganarmstrong
  committed
  49b50c5
- feat(evaluation): select Haiku, Opus, or Sonnet presets
  reiganarmstrong
  committed
  bd1d9f6
- fix(evaluation): explain presets and cost limit in CLI help
  reiganarmstrong
  committed
  896cceb
- docs(evaluation): resolve model selection spec after review
  reiganarmstrong
  committed
  5994cda
- fix(evaluation): use medium effort for Opus and Sonnet
  reiganarmstrong
  committed
  f9e2186
- docs(evaluation): record medium effort verification
  reiganarmstrong
  committed
  d61ed55
- tests: consolidate evaluator suite and make scratch tracker local
  
  reiganarmstrong
  and
  claude
  committed
  605a694
- tests: cover declined approval and excluded-row claims after review
  
  reiganarmstrong
  and
  claude
  committed
  ce345d1
- fix(evaluation): parse LLM responses exactly as the API does
  
  reiganarmstrong
  and
  claude
  committed
  b93586d
- fix(evaluation): catch exactly the parse errors the API skips
  
  reiganarmstrong
  and
  claude
  committed
  ab42455
- fix(evaluation): count LLM findings with the tool's per-check rules
  
  reiganarmstrong
  and
  claude
  committed
  77c791b
- fix(evaluation): keep table evidence intact and reject text issue lists
  
  reiganarmstrong
  and
  claude
  committed
  8a59f67
- docs(readme): add review prompt and move evaluator section to end
  reiganarmstrong
  committed
  eabee40
Commits on Oct 3, 2026
- docs(readme): clarify evaluator response handling
  reiganarmstrong
  committed
  b219345
- chore(readme): wording and formatting
  reiganarmstrong
  committed
  b83a09b
Commits on Oct 9, 2026
Commits on Oct 10, 2026
Commits on Oct 11, 2026
Commits on Oct 15, 2026
Commits on Oct 16, 2026
Commits on Oct 22, 2026
Commits on Oct 24, 2026
Commits on Oct 25, 2026

Commits

Commits on Sep 19, 2026

Commits on Sep 20, 2026

Commits on Sep 22, 2026

Commits on Sep 23, 2026

Commits on Sep 24, 2026

Commits on Sep 27, 2026

Commits on Sep 29, 2026

Commits on Oct 1, 2026

Commits on Oct 3, 2026

Commits on Oct 9, 2026

Commits on Oct 10, 2026

Commits on Oct 11, 2026

Commits on Oct 15, 2026

Commits on Oct 16, 2026

Commits on Oct 22, 2026