Repository navigation
memory/graph slice 4: ingest.t27 -- the extraction state machine and the LLM effect slot #8425
Description
Part of #7828, slice 4. Slices 1 (#7829, PR #7830), 2 (#7881) and 3 (#7882, PR #7885) have
landed: specs/memory/graph/ already holds types.t27, temporal.t27, scope.t27 and
crypt.t27. ingest.t27 does not exist yet.
Slice boundary. The ## User Scenarios / ## Requirements / ## Success Criteria sections
the Queen wrote on the epic describe slice 1 (records and time fields). They are not the
criteria for this issue. This issue is slice 4 only: the extraction state machine, and the LLM
call as an explicit effect slot. Slice 5 (dedup.t27, index.t27, search.t27) and slice 6
(attest.t27) stay out.
What slice 4 states
specs/memory/graph/ingest.t27 -- how an episode becomes entities and facts, and every way that
can refuse. Graphiti does this in Python with an LLM call in the middle of the write path and no
place where the privacy cost of that call is written down. Here the call is an effect slot:
the state machine never performs it, it only decides whether a request may be handed out, and a
request that may be handed out carries requires_plaintext and trust_domain, as the epic asks.
The scope and temporal policy is cited, not restated: ingest.t27 calls
scope::may_write, scope::raw_expired, temporal::known_as_of, types::fact_endpoints_ok
and types::fact_well_formed. There is no second copy of the owner/user/thread rule.
The state machine
States: RECEIVED -> ADMITTED -> EXTRACTING -> COMMITTED, with REJECTED as the other
terminal. Events: ADMIT, EMIT, RESULT, FAIL, ABORT.
A step answers four things at once -- next state, whether it emitted the effect, whether it
committed, and why it refused -- so one function decides all four and they cannot drift apart.
Decided before implementing
- Retry.
FAILinEXTRACTINGwith attempts remaining returns toRECEIVED, not to
ADMITTED. A retry therefore re-runs the whole admission gate before a second disclosure of
plaintext, and consent withdrawn between attempts stops the retry. Attempts are capped.
Exhausted attempts settle asREJECTEDwith reasonEFFECT. - Idempotency / repeated completion.
COMMITTEDandREJECTEDare absorbing. A repeated
RESULTon either leaves the state unchanged, commits nothing, and reportsSETTLED-- so an
at-least-once effect runtime cannot write the same episode twice. Any other event on a settled
episode leaves the state unchanged and reportsSTATE. - Fail closed. An unknown state, or an unknown event in a live state, settles the episode as
REJECTEDwith reasonSTATE. An unknown reason code reads as a failure. - The gate is read at the emission point.
EMITre-checks the admission verdict it is given.
ReachingADMITTEDis not sufficient: a bad verdict atEMITrejects and emits nothing.
Acceptance criteria
specs/memory/graph/ingest.t27compiles with thet27cbuilt from the tree of the PR, and
generates for the sealed backend set (gen=c,rust,verilog,zig;tests=zig).- Every pair of (state, event) over a range that includes codes which are neither a state nor
an event has a defined outcome. Proven by an exhaustive check, not by spot cases: exactly one
pair emits the extraction effect, exactly one pair commits, and nothing else does. - No extraction effect is emitted for a scope the principal may not write, for a trust domain
that is unknown, or where the owner's consent does not cover that domain. Proven by an
exhaustive check over every (trust domain, consent) pair in a range including invalid codes,
asserting emission iff consent covers the domain. - An emitted effect carries both
requires_plaintextand a knowntrust_domain; a step that
emits nothing carries neither. - Owner/user/thread and temporal validity are enforced by calling slices 1-3, and the gate
reports which check refused first, so the reasons are distinguishable. - Executable tests cover: the valid path commits; a foreign scope; missing and denied plaintext
consent; an invalid trust domain; a malformed extraction result; an effect failure that
retries and then settles; a repeated completion that writes once. End-to-end through the
machine, asserting observable outcomes (state, emitted, committed, reason) -- not constants. - Negative controls run and recorded: a deliberately wrong expected outcome and a deliberately
widened transition each make the suite fail. .trinity/seals/graph_MemoryGraphIngest.jsonwritten by that same compiler, and one row in
docs/reports/t27b_expectations.json.
Explicitly not in this slice
No real LLM is called, no service is deployed, no real conversation is read, no production KMS is
configured, and nothing here claims privacy against the operator -- holding an unwrapped data key
still means reading plaintext (crypt.t27 says so; slice 6 attest.t27 is where that changes).
Fixtures are synthetic. The executable store that obeys these rules lives in
dmitrii-f-t27/trinity-memory (#136, #137).
Related: gHashTag/999-multibots-telegraf#3876, #3877.
Requirements
FR-001: specs/memory/graph/ingest.t27 MUST compile with the t27c compiler from the PR tree and MUST generate artifacts for the sealed backend set gen=c,rust,verilog,zig;tests=zig.
FR-002: The specification MUST define a state machine with states RECEIVED, ADMITTED, EXTRACTING, COMMITTED, REJECTED and events ADMIT, EMIT, RESULT, FAIL, ABORT such that every pair of (state, event) over a range including codes that are neither a state nor an event has a defined outcome.
FR-003: The state machine transition function MUST return four values atomically: next state, whether the extraction effect was emitted, whether the episode was committed, and the refusal reason (if any), so these four decisions cannot drift apart.
FR-004: Exactly one (state, event) pair MUST emit the extraction effect, exactly one pair MUST commit, and no other pair MUST do either. This MUST be proven by an exhaustive check, not spot cases.
FR-005: On FAIL in EXTRACTING with attempts remaining, the machine MUST return to RECEIVED (not ADMITTED), so a retry re-runs the whole admission gate before a second disclosure of plaintext. Exhausted attempts MUST settle as REJECTED with reason EFFECT.
FR-006: COMMITTED and REJECTED MUST be absorbing states. A repeated RESULT on either MUST leave the state unchanged, commit nothing, and report SETTLED. Any other event on a settled episode MUST leave the state unchanged and report STATE.
FR-007: An unknown state or an unknown event in a live state MUST settle the episode as REJECTED with reason STATE. An unknown reason code MUST be read as a failure.
FR-008: The EMIT transition MUST re-check the admission verdict it is given. Reaching ADMITTED MUST NOT be sufficient; a bad verdict at EMIT MUST reject and emit nothing.
FR-009: The admission gate MUST enforce scope policy by calling scope::may_write, temporal policy by calling temporal::known_as_of and scope::raw_expired, and fact well-formedness by calling types::fact_endpoints_ok and types::fact_well_formed. The gate MUST report which check refused first so reasons are distinguishable.
FR-010: An emitted extraction effect MUST carry both requires_plaintext and a known trust_domain. A step that emits nothing MUST carry neither.
FR-011: For every (trust domain, consent) pair in a range including invalid codes, the machine MUST emit the effect if and only if consent covers the domain. No effect MUST be emitted for a scope the principal may not write, for an unknown trust domain, or where the owner's consent does not cover that domain. This MUST be proven by an exhaustive check.
FR-012: Executable tests MUST cover end-to-end through the machine asserting observable outcomes (state, emitted, committed, reason) for: the valid path commits; a foreign scope; missing plaintext consent; denied plaintext consent; an invalid trust domain; a malformed extraction result; an effect failure that retries and then settles; a repeated completion that writes once.
FR-013: Negative control tests MUST run and be recorded: a deliberately wrong expected outcome and a deliberately widened transition MUST each make the test suite fail.
FR-014: The compiler MUST write .trinity/seals/graph_MemoryGraphIngest.json and add one row to docs/reports/t27b_expectations.json.
Boundary
- specs/memory/graph/ingest.t27
- .trinity/seals/graph_MemoryGraphIngest.json
- docs/reports/t27b_expectations.json
Activity
docs: the coordination entry this branch needs to land
memory/graph slice 4: ingest.t27 -- the extraction state machine and …