Repository navigation
Recipe 45: Memory reconciliation #45
Copy link
Copy link
Closed
Labels
cat: memory & contextCategory: Memory & contextCategory: Memory & contextdecision: choiceUses a Choice questionUses a Choice questionlevel: 4 advancedStateful workflows, provenance, adversarial cases, calibrationStateful workflows, provenance, adversarial cases, calibrationneeds: simulatorDepends on the simulation and state primitivesDepends on the simulation and state primitivespriority: p2Scheduled after p1 workScheduled after p1 worksize: LTwo to three days; consider splitting the workTwo to three days; consider splitting the workstatus: readyNo open blockers; can be picked upNo open blockers; can be picked uptype: recipeA cookbook notebook recipeA cookbook notebook recipe
Milestone
Description
Recipe 45 of 60 · Level 4 (Advanced) · Memory & context ·
Choice
Use case
Classify a new memory statement as corroborating, superseding, contradicting, or unrelated to existing evidence so a versioned store can preserve provenance and unresolved conflicts.
What to build
The folder recipes/45-memory-reconciliation/ with notebook.ipynb, README.md, and fixtures/, meeting the recipe contract. Start from python tools/new_recipe.py 45 and the template in recipes/_template/.
Build notes
- Outcomes:
corroborating,superseding,contradicting,unrelated. - A versioned store in Python keeps provenance and unresolved conflicts. It never deletes.
- Tests cover the store's invariants.
Decision design
- Choice. Options are a fixed set supplied by Python, including the fallback outcome this use case names. Show the
choice, the per-optionprobabilities, and theconfidencefor individual examples before aggregating.
Level 4 scope
Stateful workflows, provenance, adversarial cases, model calibration, or several integrated decision stages.
Acceptance criteria
-
recipes/45-memory-reconciliation/notebook.ipynbruns top to bottom offline, with no key and no network, and its outputs are committed - Live mode is opt-in (
JEV_COOKBOOK_LIVE=1withTYPESAFE_API_KEY) and uses the same question definitions as the offline run - Fixtures are synthetic, include the hard cases named in the build notes, carry gold labels, and pass the fixture validator
- Every response fixture records its provenance, and the notebook states which mode it ran in
- Evaluation: Label accuracy, and the store invariant tests.
- State, provenance, or policy rules are enforced in
helpers.pyand covered by tests that hold for any model answer - Fixtures include adversarial, conflicting, or stateful cases, and the notebook walks through at least one
- No statement about Jev's quality, latency, or cost appears without recorded live inference; in a synthetic run, metrics are labelled as pipeline checks
-
recipes/45-memory-reconciliation/README.mdsays what the recipe teaches, how to run it, and what was and was not measured live - The pull request changes only
recipes/45-memory-reconciliation/and CI is green
Dependencies
- Blocked by Foundation: CI executes every notebook offline and validates fixtures #69 (CI executes every notebook offline and validates fixtures)
- Blocked by Foundation: simulation and state primitives #70 (simulation and state primitives)
- Tracked in Epic: Level 4 · Advanced recipes #78
Sources
Read these before writing the questions.
- S01 · TypeSafe AI: Introduction: Typed questions over supplied state and atomic judgments composed in code.
- S02 · TypeSafe AI: Primitives (Questions): Choice, Noul, Score, independent questions, and dependent follow-up calls.
- S03 · TypeSafe AI: Confidence: Distribution-derived confidence and task-specific thresholds informed by observed outcomes.
- S04 · TypeSafe AI: How to build with TypeSafe: Narrow semantic judgments integrated with workflows, deterministic logic, and side effects controlled by code.
Out of scope
- Changes to shared code, the catalog, or the root README. If the shared package is missing something, open a foundation issue and link it here.
- Real side effects of any kind. Actions are simulated.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
cat: memory & contextCategory: Memory & contextCategory: Memory & contextdecision: choiceUses a Choice questionUses a Choice questionlevel: 4 advancedStateful workflows, provenance, adversarial cases, calibrationStateful workflows, provenance, adversarial cases, calibrationneeds: simulatorDepends on the simulation and state primitivesDepends on the simulation and state primitivespriority: p2Scheduled after p1 workScheduled after p1 worksize: LTwo to three days; consider splitting the workTwo to three days; consider splitting the workstatus: readyNo open blockers; can be picked upNo open blockers; can be picked uptype: recipeA cookbook notebook recipeA cookbook notebook recipe
Activity