You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Foundation: fixture specification and validator #65
One fixture format across sixty recipes, checked by a tool, so fixtures stay readable and honest.
Scope
A written specification for fixtures/: inputs, gold labels, and model responses, in JSON or JSON Lines, with stable identifiers linking the three.
Split support (validation, test, and others as needed) recorded per example.
Required provenance on every response: synthetic or recorded, plus model string and date when recorded.
A JSON Schema and a validator: python -m jev_cookbook.fixtures validate recipes/NN-slug.
Checks beyond schema: every evaluated input has a label, identifiers are unique, no response without a matching input, and a scan for strings that look like keys or tokens.
Acceptance criteria
Validator passes on a good example and fails with a specific message on each kind of bad one, with tests for both
The specification is one page in docs/ and is linked from CONTRIBUTING.md
The validator exits non-zero on failure so CI can use it
Lint and tests pass in CI, or locally with the output pasted in the pull request if CI does not exist yet
Activity
Foundation: repository hygiene for secrets, dependencies, and noteboo…
Foundation: reconcile recipe catalog integration and orchestration ru…
Foundation: decision backends with typed answers, replay, and scripte…
Foundation: fixture specification and validator (#94)
Hygiene: never echo or miss a secret used as a dict key; integrate fi…
Foundation: repair audited hygiene coverage and align Markdown pre-co…
Fix round 4 (M1, M2/M3): uid-boundary sandbox for notebook execution,…
Notebook CI: execute every notebook offline, validate fixtures, check…