diff --git a/CURRENT_RESULTS.md b/CURRENT_RESULTS.md index f5627a7..973af11 100644 --- a/CURRENT_RESULTS.md +++ b/CURRENT_RESULTS.md @@ -129,3 +129,7 @@ The archived root README and original manuscript are retained as part of the imm [Executed report](graph_synthesis/post_certificate/RESULTS.md), [frozen protocol](graph_synthesis/post_certificate/PROTOCOL.md), [corrections](graph_synthesis/post_certificate/CORRECTIONS.md), and [five figures](graph_synthesis/post_certificate/figures/). Recorded target outcomes: **H1: met; H2: not met; H3: met; H4: met; H5: met.** H4 covers only endpoint scenarios; its broader continuous-interval minimax claim is unsupported, with an interior counterexample retained. Counted H1 reduction: 99.26%; local H2 reduction: 90.11%, connected-control saving: 0.00%. No fresh Jev calls, independent semantic-accuracy claim, external-system superiority or production-policy change. + +## Transactional fault-injection follow-up (issue #50) + +[The separately frozen protocol](graph_synthesis/post_certificate_h2_followup/PROTOCOL.md), [executed report](graph_synthesis/post_certificate_h2_followup/RESULTS.md), and [all control records](graph_synthesis/post_certificate_h2_followup/results.json) distinguish affected-node fault injection from an unaffected primitive update. The new controlled conjunction passes: 320/320 accepted transactions, 64/64 affected injections, 64/64 unaffected updates, 64/64 stale and malformed rejections, and 8/8 cycle rejections. **The original PR #33 H2 result remains failed at 63/64**; its fixture and denominator were not changed. These are in-memory graph tests, not semantic accuracy, concurrency, crash durability or elapsed-time evidence. diff --git a/graph_synthesis/post_certificate_h2_followup/PROTOCOL.md b/graph_synthesis/post_certificate_h2_followup/PROTOCOL.md new file mode 100644 index 0000000..a834d5c --- /dev/null +++ b/graph_synthesis/post_certificate_h2_followup/PROTOCOL.md @@ -0,0 +1,65 @@ +# Frozen follow-up protocol: transactional fault injection + +Date: 2026-09-28. Baseline: merged PR #33 at `7a38d854424a1add6912668fc7def1e3bbaac292`. +This protocol must be committed before the new runner, tests, or outcomes. It is a +new study, not a revision of PR #33's frozen H2 fixture population or its failed +63/64 result. No provider or paid model calls are involved. + +## Question and causal condition + +PR #33 selected one primitive with no affected derived node, so its intended +evaluation-failure control did not inject a failure. An evaluation fault can be +tested only at a derived node in the reverse-dependency closure of a primitive +whose Boolean value actually changes. A primitive with an empty closure is a +valid no-op for derived evaluation and should commit successfully. These two +cases are distinct controls. + +## Frozen population and implementation + +- Use Python 3.12 and an independent `random.Random(20260929)` stream. +- Construct 64 modular DAGs by calling the already merged + `graph_synthesis.post_certificate.run.make_modular_dag` once per graph in + index order. This gives 64 primitives and 256 derived nodes per graph. +- Initialize every primitive from `rng.getrandbits(1)` in numeric order. +- For each graph, execute five accepted transactions. Each draws a module with + `rng.randrange(8)`, a change count with `rng.randint(1,4)`, then primitives + via `rng.sample(range(module*8,(module+1)*8), count)`. Toggle each selected + primitive. Compare the returned values and revision to independent full + recomputation and the expected revision after every transaction. +- After the five accepted transactions, select the lowest numbered primitive + with a nonempty reverse-dependency closure. Toggle it and inject failure at + the lowest numbered derived node in that closure. Require `RuntimeError`, + unchanged caller-owned values and revision, and no returned partial overlay. +- On each graph, also reject one stale revision (`current-1`) and one malformed + update targeting derived node 64. Require their specific exceptions and + unchanged caller-owned state/revision. +- For an unaffected primitive control, use a separate DAG with two primitives + and one derived node `2 = xor(0)`. Toggle primitive 1 at revision 0; require + a successful revision-1 transaction, zero derived evaluations, and agreement + with independent full recomputation. Repeat for 64 initial primitive pairs + from the same RNG stream; this tests the no-op causal boundary without + manufacturing a failure. +- Construct eight invalid DAGs with a two-node derived cycle. Require graph + construction to reject each with `ValueError` before any transaction. +- Reuse the original 16 connected controls from `make_connected_dag`, with + primitive initialization and transaction specified in PR #33's H2 runner. + Report their mean saving separately from local modular-DAG savings. + +## Predeclared endpoints + +The new conjunction passes only if all 320 accepted transactions match full +recomputation and revision, all 64 affected injection controls raise and leave +caller-owned state unchanged, all 64 stale and malformed controls reject with +the expected exception and unchanged state, all 64 unaffected controls commit +with zero derived evaluations and correct values, all eight cycles reject, and +local modular-DAG derived evaluations are at least 80% below 320 full +recomputations of 256 derived nodes each. The connected-control result is a +reported boundary, not part of the 80% denominator. + +Retain each control's graph index, primitive, selected fail node, affected +closure size, actual exception class, and pass/fail status. Retain failures in +the denominator. A mutation check must demonstrate that suppressing an expected +injection exception makes the follow-up fail. Publish exact counts, usage-free +work counters, an immutable input/source hash record, and a report that repeats +the original PR #33 H2 failure separately. No semantic accuracy, crash +durability, concurrency isolation, or general performance claim follows. diff --git a/graph_synthesis/post_certificate_h2_followup/RESULTS.md b/graph_synthesis/post_certificate_h2_followup/RESULTS.md new file mode 100644 index 0000000..0edf39f --- /dev/null +++ b/graph_synthesis/post_certificate_h2_followup/RESULTS.md @@ -0,0 +1,22 @@ +# Transactional fault-injection follow-up + +This is a separate precommitted study. The original PR #33 H2 result remains **failed (63/64)**. + +New conjunction: **met**. + +| Endpoint | Passed / total | +|---|---:| +| accepted | 320/320 | +| affected injection | 64/64 | +| stale revision | 64/64 | +| malformed update | 64/64 | +| unaffected noop | 64/64 | +| cycle rejection | 8/8 | +| connected boundary | 16/16 | + +Local derived evaluations: 8,153 / 81,920 full recomputation evaluations (90.05% reduction). +Connected control mean saving: 0.00%; reported separately. + +Affected injections require an actual derived dependency and an exception. Unaffected primitive updates correctly commit without derived evaluation. This measures in-memory behavior only; it does not establish crash durability, concurrency isolation, semantic accuracy, or wall-clock performance. + +See [the frozen protocol](PROTOCOL.md) and [all control records](results.json). diff --git a/graph_synthesis/post_certificate_h2_followup/__init__.py b/graph_synthesis/post_certificate_h2_followup/__init__.py new file mode 100644 index 0000000..c129c53 --- /dev/null +++ b/graph_synthesis/post_certificate_h2_followup/__init__.py @@ -0,0 +1 @@ +"""Precommitted follow-up to the post-certificate H2 control.""" diff --git a/graph_synthesis/post_certificate_h2_followup/results.json b/graph_synthesis/post_certificate_h2_followup/results.json new file mode 100644 index 0000000..aa42a0b --- /dev/null +++ b/graph_synthesis/post_certificate_h2_followup/results.json @@ -0,0 +1,5142 @@ +{ + "connected_mean_reduction": 0.0, + "controls": { + "accepted": [ + { + "evaluations": 27, + "graph_index": 0, + "passed": true, + "primitive_ids": [ + 15, + 13, + 11 + ], + "transaction_index": 0 + }, + { + "evaluations": 25, + "graph_index": 0, + "passed": true, + "primitive_ids": [ + 50 + ], + "transaction_index": 1 + }, + { + "evaluations": 29, + "graph_index": 0, + "passed": true, + "primitive_ids": [ + 62, + 59, + 63 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 0, + "passed": true, + "primitive_ids": [ + 42, + 44, + 41 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 0, + "passed": true, + "primitive_ids": [ + 8, + 10 + ], + "transaction_index": 4 + }, + { + "evaluations": 23, + "graph_index": 1, + "passed": true, + "primitive_ids": [ + 46 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 1, + "passed": true, + "primitive_ids": [ + 35, + 37, + 36, + 39 + ], + "transaction_index": 1 + }, + { + "evaluations": 32, + "graph_index": 1, + "passed": true, + "primitive_ids": [ + 37, + 35 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 1, + "passed": true, + "primitive_ids": [ + 54, + 53, + 52 + ], + "transaction_index": 3 + }, + { + "evaluations": 25, + "graph_index": 1, + "passed": true, + "primitive_ids": [ + 59, + 63 + ], + "transaction_index": 4 + }, + { + "evaluations": 29, + "graph_index": 2, + "passed": true, + "primitive_ids": [ + 38, + 33, + 32, + 37 + ], + "transaction_index": 0 + }, + { + "evaluations": 29, + "graph_index": 2, + "passed": true, + "primitive_ids": [ + 58, + 61, + 60 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 2, + "passed": true, + "primitive_ids": [ + 17, + 16 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 2, + "passed": true, + "primitive_ids": [ + 6, + 1, + 0, + 4 + ], + "transaction_index": 3 + }, + { + "evaluations": 24, + "graph_index": 2, + "passed": true, + "primitive_ids": [ + 4 + ], + "transaction_index": 4 + }, + { + "evaluations": 25, + "graph_index": 3, + "passed": true, + "primitive_ids": [ + 63, + 58, + 57 + ], + "transaction_index": 0 + }, + { + "evaluations": 11, + "graph_index": 3, + "passed": true, + "primitive_ids": [ + 24 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 3, + "passed": true, + "primitive_ids": [ + 55, + 50, + 48, + 52 + ], + "transaction_index": 2 + }, + { + "evaluations": 9, + "graph_index": 3, + "passed": true, + "primitive_ids": [ + 35 + ], + "transaction_index": 3 + }, + { + "evaluations": 20, + "graph_index": 3, + "passed": true, + "primitive_ids": [ + 51, + 48, + 53 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 4, + "passed": true, + "primitive_ids": [ + 52 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 4, + "passed": true, + "primitive_ids": [ + 35, + 34, + 33, + 36 + ], + "transaction_index": 1 + }, + { + "evaluations": 19, + "graph_index": 4, + "passed": true, + "primitive_ids": [ + 28, + 31 + ], + "transaction_index": 2 + }, + { + "evaluations": 15, + "graph_index": 4, + "passed": true, + "primitive_ids": [ + 22, + 21 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 4, + "passed": true, + "primitive_ids": [ + 12, + 13 + ], + "transaction_index": 4 + }, + { + "evaluations": 27, + "graph_index": 5, + "passed": true, + "primitive_ids": [ + 1, + 6 + ], + "transaction_index": 0 + }, + { + "evaluations": 30, + "graph_index": 5, + "passed": true, + "primitive_ids": [ + 37, + 32, + 34 + ], + "transaction_index": 1 + }, + { + "evaluations": 26, + "graph_index": 5, + "passed": true, + "primitive_ids": [ + 28, + 31 + ], + "transaction_index": 2 + }, + { + "evaluations": 29, + "graph_index": 5, + "passed": true, + "primitive_ids": [ + 21, + 16 + ], + "transaction_index": 3 + }, + { + "evaluations": 20, + "graph_index": 5, + "passed": true, + "primitive_ids": [ + 2 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 6, + "passed": true, + "primitive_ids": [ + 34, + 33, + 37, + 35 + ], + "transaction_index": 0 + }, + { + "evaluations": 27, + "graph_index": 6, + "passed": true, + "primitive_ids": [ + 12, + 11 + ], + "transaction_index": 1 + }, + { + "evaluations": 32, + "graph_index": 6, + "passed": true, + "primitive_ids": [ + 40, + 46, + 42 + ], + "transaction_index": 2 + }, + { + "evaluations": 27, + "graph_index": 6, + "passed": true, + "primitive_ids": [ + 34, + 37 + ], + "transaction_index": 3 + }, + { + "evaluations": 21, + "graph_index": 6, + "passed": true, + "primitive_ids": [ + 55, + 49 + ], + "transaction_index": 4 + }, + { + "evaluations": 27, + "graph_index": 7, + "passed": true, + "primitive_ids": [ + 50, + 54 + ], + "transaction_index": 0 + }, + { + "evaluations": 30, + "graph_index": 7, + "passed": true, + "primitive_ids": [ + 47, + 45 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 7, + "passed": true, + "primitive_ids": [ + 36, + 39, + 37 + ], + "transaction_index": 2 + }, + { + "evaluations": 29, + "graph_index": 7, + "passed": true, + "primitive_ids": [ + 37, + 39 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 7, + "passed": true, + "primitive_ids": [ + 13, + 11 + ], + "transaction_index": 4 + }, + { + "evaluations": 32, + "graph_index": 8, + "passed": true, + "primitive_ids": [ + 56, + 63, + 61 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 8, + "passed": true, + "primitive_ids": [ + 25, + 26, + 29, + 28 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 8, + "passed": true, + "primitive_ids": [ + 47, + 41, + 42, + 46 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 8, + "passed": true, + "primitive_ids": [ + 43 + ], + "transaction_index": 3 + }, + { + "evaluations": 9, + "graph_index": 8, + "passed": true, + "primitive_ids": [ + 6 + ], + "transaction_index": 4 + }, + { + "evaluations": 12, + "graph_index": 9, + "passed": true, + "primitive_ids": [ + 11 + ], + "transaction_index": 0 + }, + { + "evaluations": 25, + "graph_index": 9, + "passed": true, + "primitive_ids": [ + 41, + 42, + 47, + 44 + ], + "transaction_index": 1 + }, + { + "evaluations": 16, + "graph_index": 9, + "passed": true, + "primitive_ids": [ + 34 + ], + "transaction_index": 2 + }, + { + "evaluations": 30, + "graph_index": 9, + "passed": true, + "primitive_ids": [ + 22, + 20 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 9, + "passed": true, + "primitive_ids": [ + 34, + 38, + 39, + 32 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 10, + "passed": true, + "primitive_ids": [ + 26, + 30, + 24 + ], + "transaction_index": 0 + }, + { + "evaluations": 24, + "graph_index": 10, + "passed": true, + "primitive_ids": [ + 30, + 29 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 10, + "passed": true, + "primitive_ids": [ + 53, + 49 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 10, + "passed": true, + "primitive_ids": [ + 57, + 59, + 58, + 60 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 10, + "passed": true, + "primitive_ids": [ + 11, + 9, + 8, + 13 + ], + "transaction_index": 4 + }, + { + "evaluations": 4, + "graph_index": 11, + "passed": true, + "primitive_ids": [ + 47 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 11, + "passed": true, + "primitive_ids": [ + 28, + 30, + 27 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 11, + "passed": true, + "primitive_ids": [ + 2, + 4, + 0 + ], + "transaction_index": 2 + }, + { + "evaluations": 26, + "graph_index": 11, + "passed": true, + "primitive_ids": [ + 58, + 62 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 11, + "passed": true, + "primitive_ids": [ + 48, + 53, + 52, + 49 + ], + "transaction_index": 4 + }, + { + "evaluations": 23, + "graph_index": 12, + "passed": true, + "primitive_ids": [ + 60 + ], + "transaction_index": 0 + }, + { + "evaluations": 24, + "graph_index": 12, + "passed": true, + "primitive_ids": [ + 32, + 37, + 33 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 12, + "passed": true, + "primitive_ids": [ + 48, + 54, + 52, + 51 + ], + "transaction_index": 2 + }, + { + "evaluations": 28, + "graph_index": 12, + "passed": true, + "primitive_ids": [ + 45, + 44 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 12, + "passed": true, + "primitive_ids": [ + 2, + 6, + 0 + ], + "transaction_index": 4 + }, + { + "evaluations": 26, + "graph_index": 13, + "passed": true, + "primitive_ids": [ + 47 + ], + "transaction_index": 0 + }, + { + "evaluations": 11, + "graph_index": 13, + "passed": true, + "primitive_ids": [ + 6, + 0, + 4 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 13, + "passed": true, + "primitive_ids": [ + 39, + 33 + ], + "transaction_index": 2 + }, + { + "evaluations": 30, + "graph_index": 13, + "passed": true, + "primitive_ids": [ + 10, + 9, + 13, + 14 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 13, + "passed": true, + "primitive_ids": [ + 62, + 57 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 14, + "passed": true, + "primitive_ids": [ + 34, + 35, + 36, + 32 + ], + "transaction_index": 0 + }, + { + "evaluations": 29, + "graph_index": 14, + "passed": true, + "primitive_ids": [ + 61, + 58 + ], + "transaction_index": 1 + }, + { + "evaluations": 23, + "graph_index": 14, + "passed": true, + "primitive_ids": [ + 51 + ], + "transaction_index": 2 + }, + { + "evaluations": 17, + "graph_index": 14, + "passed": true, + "primitive_ids": [ + 22 + ], + "transaction_index": 3 + }, + { + "evaluations": 2, + "graph_index": 14, + "passed": true, + "primitive_ids": [ + 60 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 15, + "passed": true, + "primitive_ids": [ + 43, + 41, + 47 + ], + "transaction_index": 0 + }, + { + "evaluations": 24, + "graph_index": 15, + "passed": true, + "primitive_ids": [ + 61, + 56, + 59 + ], + "transaction_index": 1 + }, + { + "evaluations": 13, + "graph_index": 15, + "passed": true, + "primitive_ids": [ + 43, + 44 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 15, + "passed": true, + "primitive_ids": [ + 6, + 0, + 3 + ], + "transaction_index": 3 + }, + { + "evaluations": 2, + "graph_index": 15, + "passed": true, + "primitive_ids": [ + 8 + ], + "transaction_index": 4 + }, + { + "evaluations": 32, + "graph_index": 16, + "passed": true, + "primitive_ids": [ + 53, + 55, + 48, + 49 + ], + "transaction_index": 0 + }, + { + "evaluations": 27, + "graph_index": 16, + "passed": true, + "primitive_ids": [ + 11 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 16, + "passed": true, + "primitive_ids": [ + 20, + 22 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 16, + "passed": true, + "primitive_ids": [ + 52, + 54, + 55, + 50 + ], + "transaction_index": 3 + }, + { + "evaluations": 27, + "graph_index": 16, + "passed": true, + "primitive_ids": [ + 35, + 34, + 38 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 17, + "passed": true, + "primitive_ids": [ + 19, + 21, + 22, + 16 + ], + "transaction_index": 0 + }, + { + "evaluations": 11, + "graph_index": 17, + "passed": true, + "primitive_ids": [ + 2 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 17, + "passed": true, + "primitive_ids": [ + 53, + 52, + 55, + 51 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 17, + "passed": true, + "primitive_ids": [ + 47, + 41, + 40, + 44 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 17, + "passed": true, + "primitive_ids": [ + 12, + 8, + 14 + ], + "transaction_index": 4 + }, + { + "evaluations": 32, + "graph_index": 18, + "passed": true, + "primitive_ids": [ + 37, + 34, + 38, + 36 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 18, + "passed": true, + "primitive_ids": [ + 8, + 13 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 18, + "passed": true, + "primitive_ids": [ + 41, + 45, + 43, + 42 + ], + "transaction_index": 2 + }, + { + "evaluations": 26, + "graph_index": 18, + "passed": true, + "primitive_ids": [ + 28, + 26 + ], + "transaction_index": 3 + }, + { + "evaluations": 19, + "graph_index": 18, + "passed": true, + "primitive_ids": [ + 14 + ], + "transaction_index": 4 + }, + { + "evaluations": 29, + "graph_index": 19, + "passed": true, + "primitive_ids": [ + 1, + 4, + 0, + 3 + ], + "transaction_index": 0 + }, + { + "evaluations": 26, + "graph_index": 19, + "passed": true, + "primitive_ids": [ + 24 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 19, + "passed": true, + "primitive_ids": [ + 52, + 55, + 51, + 50 + ], + "transaction_index": 2 + }, + { + "evaluations": 16, + "graph_index": 19, + "passed": true, + "primitive_ids": [ + 19, + 23 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 19, + "passed": true, + "primitive_ids": [ + 28, + 29, + 27, + 26 + ], + "transaction_index": 4 + }, + { + "evaluations": 16, + "graph_index": 20, + "passed": true, + "primitive_ids": [ + 22 + ], + "transaction_index": 0 + }, + { + "evaluations": 5, + "graph_index": 20, + "passed": true, + "primitive_ids": [ + 38, + 34 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 20, + "passed": true, + "primitive_ids": [ + 57, + 59 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 20, + "passed": true, + "primitive_ids": [ + 0, + 6, + 4, + 3 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 20, + "passed": true, + "primitive_ids": [ + 47, + 46, + 42 + ], + "transaction_index": 4 + }, + { + "evaluations": 16, + "graph_index": 21, + "passed": true, + "primitive_ids": [ + 14 + ], + "transaction_index": 0 + }, + { + "evaluations": 28, + "graph_index": 21, + "passed": true, + "primitive_ids": [ + 10, + 14, + 13 + ], + "transaction_index": 1 + }, + { + "evaluations": 16, + "graph_index": 21, + "passed": true, + "primitive_ids": [ + 48 + ], + "transaction_index": 2 + }, + { + "evaluations": 26, + "graph_index": 21, + "passed": true, + "primitive_ids": [ + 61, + 56 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 21, + "passed": true, + "primitive_ids": [ + 29, + 30, + 24, + 28 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 22, + "passed": true, + "primitive_ids": [ + 33, + 37, + 34 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 22, + "passed": true, + "primitive_ids": [ + 4, + 5, + 2 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 22, + "passed": true, + "primitive_ids": [ + 2, + 1, + 0, + 6 + ], + "transaction_index": 2 + }, + { + "evaluations": 25, + "graph_index": 22, + "passed": true, + "primitive_ids": [ + 55, + 52 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 22, + "passed": true, + "primitive_ids": [ + 29, + 26 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 23, + "passed": true, + "primitive_ids": [ + 22, + 16, + 23, + 21 + ], + "transaction_index": 0 + }, + { + "evaluations": 28, + "graph_index": 23, + "passed": true, + "primitive_ids": [ + 35, + 38, + 34 + ], + "transaction_index": 1 + }, + { + "evaluations": 23, + "graph_index": 23, + "passed": true, + "primitive_ids": [ + 2, + 0 + ], + "transaction_index": 2 + }, + { + "evaluations": 23, + "graph_index": 23, + "passed": true, + "primitive_ids": [ + 57, + 62, + 61, + 56 + ], + "transaction_index": 3 + }, + { + "evaluations": 16, + "graph_index": 23, + "passed": true, + "primitive_ids": [ + 57 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 24, + "passed": true, + "primitive_ids": [ + 18, + 16, + 21 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 24, + "passed": true, + "primitive_ids": [ + 47, + 42, + 43 + ], + "transaction_index": 1 + }, + { + "evaluations": 22, + "graph_index": 24, + "passed": true, + "primitive_ids": [ + 0, + 7 + ], + "transaction_index": 2 + }, + { + "evaluations": 14, + "graph_index": 24, + "passed": true, + "primitive_ids": [ + 32 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 24, + "passed": true, + "primitive_ids": [ + 59, + 57, + 56 + ], + "transaction_index": 4 + }, + { + "evaluations": 28, + "graph_index": 25, + "passed": true, + "primitive_ids": [ + 3, + 4 + ], + "transaction_index": 0 + }, + { + "evaluations": 30, + "graph_index": 25, + "passed": true, + "primitive_ids": [ + 1, + 3, + 4 + ], + "transaction_index": 1 + }, + { + "evaluations": 24, + "graph_index": 25, + "passed": true, + "primitive_ids": [ + 28, + 26 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 25, + "passed": true, + "primitive_ids": [ + 0, + 6, + 1 + ], + "transaction_index": 3 + }, + { + "evaluations": 26, + "graph_index": 25, + "passed": true, + "primitive_ids": [ + 56 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 26, + "passed": true, + "primitive_ids": [ + 19, + 22, + 18 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 26, + "passed": true, + "primitive_ids": [ + 36, + 35, + 37 + ], + "transaction_index": 1 + }, + { + "evaluations": 29, + "graph_index": 26, + "passed": true, + "primitive_ids": [ + 41 + ], + "transaction_index": 2 + }, + { + "evaluations": 29, + "graph_index": 26, + "passed": true, + "primitive_ids": [ + 59, + 61 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 26, + "passed": true, + "primitive_ids": [ + 11, + 10, + 8, + 12 + ], + "transaction_index": 4 + }, + { + "evaluations": 18, + "graph_index": 27, + "passed": true, + "primitive_ids": [ + 14 + ], + "transaction_index": 0 + }, + { + "evaluations": 12, + "graph_index": 27, + "passed": true, + "primitive_ids": [ + 37 + ], + "transaction_index": 1 + }, + { + "evaluations": 27, + "graph_index": 27, + "passed": true, + "primitive_ids": [ + 49, + 51, + 55 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 27, + "passed": true, + "primitive_ids": [ + 61, + 60 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 27, + "passed": true, + "primitive_ids": [ + 19, + 21, + 23, + 17 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 28, + "passed": true, + "primitive_ids": [ + 57, + 56, + 60 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 28, + "passed": true, + "primitive_ids": [ + 1, + 3, + 0, + 5 + ], + "transaction_index": 1 + }, + { + "evaluations": 16, + "graph_index": 28, + "passed": true, + "primitive_ids": [ + 59 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 28, + "passed": true, + "primitive_ids": [ + 5, + 0, + 1 + ], + "transaction_index": 3 + }, + { + "evaluations": 24, + "graph_index": 28, + "passed": true, + "primitive_ids": [ + 32, + 33 + ], + "transaction_index": 4 + }, + { + "evaluations": 28, + "graph_index": 29, + "passed": true, + "primitive_ids": [ + 5 + ], + "transaction_index": 0 + }, + { + "evaluations": 2, + "graph_index": 29, + "passed": true, + "primitive_ids": [ + 3, + 4, + 2 + ], + "transaction_index": 1 + }, + { + "evaluations": 27, + "graph_index": 29, + "passed": true, + "primitive_ids": [ + 52, + 49, + 54, + 48 + ], + "transaction_index": 2 + }, + { + "evaluations": 27, + "graph_index": 29, + "passed": true, + "primitive_ids": [ + 25, + 29 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 29, + "passed": true, + "primitive_ids": [ + 24, + 31, + 29, + 25 + ], + "transaction_index": 4 + }, + { + "evaluations": 13, + "graph_index": 30, + "passed": true, + "primitive_ids": [ + 7 + ], + "transaction_index": 0 + }, + { + "evaluations": 26, + "graph_index": 30, + "passed": true, + "primitive_ids": [ + 3, + 1 + ], + "transaction_index": 1 + }, + { + "evaluations": 32, + "graph_index": 30, + "passed": true, + "primitive_ids": [ + 40, + 47, + 46, + 41 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 30, + "passed": true, + "primitive_ids": [ + 40, + 47, + 41, + 45 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 30, + "passed": true, + "primitive_ids": [ + 63, + 60 + ], + "transaction_index": 4 + }, + { + "evaluations": 3, + "graph_index": 31, + "passed": true, + "primitive_ids": [ + 62 + ], + "transaction_index": 0 + }, + { + "evaluations": 14, + "graph_index": 31, + "passed": true, + "primitive_ids": [ + 50 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 31, + "passed": true, + "primitive_ids": [ + 39, + 34 + ], + "transaction_index": 2 + }, + { + "evaluations": 28, + "graph_index": 31, + "passed": true, + "primitive_ids": [ + 37, + 38 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 31, + "passed": true, + "primitive_ids": [ + 46, + 40, + 42 + ], + "transaction_index": 4 + }, + { + "evaluations": 28, + "graph_index": 32, + "passed": true, + "primitive_ids": [ + 1 + ], + "transaction_index": 0 + }, + { + "evaluations": 29, + "graph_index": 32, + "passed": true, + "primitive_ids": [ + 55, + 50, + 51 + ], + "transaction_index": 1 + }, + { + "evaluations": 22, + "graph_index": 32, + "passed": true, + "primitive_ids": [ + 21, + 22 + ], + "transaction_index": 2 + }, + { + "evaluations": 29, + "graph_index": 32, + "passed": true, + "primitive_ids": [ + 47, + 40, + 43, + 44 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 32, + "passed": true, + "primitive_ids": [ + 22, + 23 + ], + "transaction_index": 4 + }, + { + "evaluations": 26, + "graph_index": 33, + "passed": true, + "primitive_ids": [ + 2, + 7 + ], + "transaction_index": 0 + }, + { + "evaluations": 15, + "graph_index": 33, + "passed": true, + "primitive_ids": [ + 58 + ], + "transaction_index": 1 + }, + { + "evaluations": 32, + "graph_index": 33, + "passed": true, + "primitive_ids": [ + 45, + 47, + 41, + 44 + ], + "transaction_index": 2 + }, + { + "evaluations": 12, + "graph_index": 33, + "passed": true, + "primitive_ids": [ + 16 + ], + "transaction_index": 3 + }, + { + "evaluations": 25, + "graph_index": 33, + "passed": true, + "primitive_ids": [ + 14, + 11 + ], + "transaction_index": 4 + }, + { + "evaluations": 26, + "graph_index": 34, + "passed": true, + "primitive_ids": [ + 36, + 35 + ], + "transaction_index": 0 + }, + { + "evaluations": 15, + "graph_index": 34, + "passed": true, + "primitive_ids": [ + 53 + ], + "transaction_index": 1 + }, + { + "evaluations": 15, + "graph_index": 34, + "passed": true, + "primitive_ids": [ + 4 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 34, + "passed": true, + "primitive_ids": [ + 22, + 23, + 19 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 34, + "passed": true, + "primitive_ids": [ + 37, + 36 + ], + "transaction_index": 4 + }, + { + "evaluations": 25, + "graph_index": 35, + "passed": true, + "primitive_ids": [ + 2, + 0, + 1 + ], + "transaction_index": 0 + }, + { + "evaluations": 28, + "graph_index": 35, + "passed": true, + "primitive_ids": [ + 17, + 19, + 18 + ], + "transaction_index": 1 + }, + { + "evaluations": 16, + "graph_index": 35, + "passed": true, + "primitive_ids": [ + 19, + 16, + 18 + ], + "transaction_index": 2 + }, + { + "evaluations": 21, + "graph_index": 35, + "passed": true, + "primitive_ids": [ + 5 + ], + "transaction_index": 3 + }, + { + "evaluations": 28, + "graph_index": 35, + "passed": true, + "primitive_ids": [ + 56 + ], + "transaction_index": 4 + }, + { + "evaluations": 29, + "graph_index": 36, + "passed": true, + "primitive_ids": [ + 17, + 23, + 18 + ], + "transaction_index": 0 + }, + { + "evaluations": 15, + "graph_index": 36, + "passed": true, + "primitive_ids": [ + 27 + ], + "transaction_index": 1 + }, + { + "evaluations": 24, + "graph_index": 36, + "passed": true, + "primitive_ids": [ + 12 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 36, + "passed": true, + "primitive_ids": [ + 41, + 42 + ], + "transaction_index": 3 + }, + { + "evaluations": 26, + "graph_index": 36, + "passed": true, + "primitive_ids": [ + 9, + 8 + ], + "transaction_index": 4 + }, + { + "evaluations": 27, + "graph_index": 37, + "passed": true, + "primitive_ids": [ + 12 + ], + "transaction_index": 0 + }, + { + "evaluations": 27, + "graph_index": 37, + "passed": true, + "primitive_ids": [ + 16, + 17 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 37, + "passed": true, + "primitive_ids": [ + 61, + 56, + 59, + 62 + ], + "transaction_index": 2 + }, + { + "evaluations": 22, + "graph_index": 37, + "passed": true, + "primitive_ids": [ + 60 + ], + "transaction_index": 3 + }, + { + "evaluations": 28, + "graph_index": 37, + "passed": true, + "primitive_ids": [ + 0, + 3 + ], + "transaction_index": 4 + }, + { + "evaluations": 23, + "graph_index": 38, + "passed": true, + "primitive_ids": [ + 36 + ], + "transaction_index": 0 + }, + { + "evaluations": 28, + "graph_index": 38, + "passed": true, + "primitive_ids": [ + 0, + 6, + 3 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 38, + "passed": true, + "primitive_ids": [ + 10, + 15 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 38, + "passed": true, + "primitive_ids": [ + 39, + 32, + 34 + ], + "transaction_index": 3 + }, + { + "evaluations": 23, + "graph_index": 38, + "passed": true, + "primitive_ids": [ + 8 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 39, + "passed": true, + "primitive_ids": [ + 62, + 60, + 63 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 39, + "passed": true, + "primitive_ids": [ + 63, + 60, + 59, + 57 + ], + "transaction_index": 1 + }, + { + "evaluations": 19, + "graph_index": 39, + "passed": true, + "primitive_ids": [ + 52 + ], + "transaction_index": 2 + }, + { + "evaluations": 26, + "graph_index": 39, + "passed": true, + "primitive_ids": [ + 29, + 27 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 39, + "passed": true, + "primitive_ids": [ + 47, + 40 + ], + "transaction_index": 4 + }, + { + "evaluations": 32, + "graph_index": 40, + "passed": true, + "primitive_ids": [ + 29, + 27, + 25, + 30 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 40, + "passed": true, + "primitive_ids": [ + 13, + 9, + 10, + 8 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 40, + "passed": true, + "primitive_ids": [ + 7, + 3 + ], + "transaction_index": 2 + }, + { + "evaluations": 21, + "graph_index": 40, + "passed": true, + "primitive_ids": [ + 9 + ], + "transaction_index": 3 + }, + { + "evaluations": 26, + "graph_index": 40, + "passed": true, + "primitive_ids": [ + 55, + 49, + 54, + 51 + ], + "transaction_index": 4 + }, + { + "evaluations": 29, + "graph_index": 41, + "passed": true, + "primitive_ids": [ + 57, + 62, + 58 + ], + "transaction_index": 0 + }, + { + "evaluations": 29, + "graph_index": 41, + "passed": true, + "primitive_ids": [ + 47, + 43, + 42, + 41 + ], + "transaction_index": 1 + }, + { + "evaluations": 25, + "graph_index": 41, + "passed": true, + "primitive_ids": [ + 47, + 40 + ], + "transaction_index": 2 + }, + { + "evaluations": 30, + "graph_index": 41, + "passed": true, + "primitive_ids": [ + 40, + 44, + 47 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 41, + "passed": true, + "primitive_ids": [ + 52 + ], + "transaction_index": 4 + }, + { + "evaluations": 29, + "graph_index": 42, + "passed": true, + "primitive_ids": [ + 30, + 26, + 28 + ], + "transaction_index": 0 + }, + { + "evaluations": 10, + "graph_index": 42, + "passed": true, + "primitive_ids": [ + 4 + ], + "transaction_index": 1 + }, + { + "evaluations": 32, + "graph_index": 42, + "passed": true, + "primitive_ids": [ + 9, + 13, + 11, + 8 + ], + "transaction_index": 2 + }, + { + "evaluations": 18, + "graph_index": 42, + "passed": true, + "primitive_ids": [ + 2, + 1 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 42, + "passed": true, + "primitive_ids": [ + 36, + 32, + 34, + 38 + ], + "transaction_index": 4 + }, + { + "evaluations": 32, + "graph_index": 43, + "passed": true, + "primitive_ids": [ + 33, + 35, + 37, + 39 + ], + "transaction_index": 0 + }, + { + "evaluations": 30, + "graph_index": 43, + "passed": true, + "primitive_ids": [ + 36, + 37, + 38, + 32 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 43, + "passed": true, + "primitive_ids": [ + 52, + 48, + 55 + ], + "transaction_index": 2 + }, + { + "evaluations": 27, + "graph_index": 43, + "passed": true, + "primitive_ids": [ + 19 + ], + "transaction_index": 3 + }, + { + "evaluations": 27, + "graph_index": 43, + "passed": true, + "primitive_ids": [ + 39, + 38, + 33 + ], + "transaction_index": 4 + }, + { + "evaluations": 28, + "graph_index": 44, + "passed": true, + "primitive_ids": [ + 47, + 44, + 42 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 44, + "passed": true, + "primitive_ids": [ + 13, + 9, + 8, + 11 + ], + "transaction_index": 1 + }, + { + "evaluations": 26, + "graph_index": 44, + "passed": true, + "primitive_ids": [ + 15, + 11 + ], + "transaction_index": 2 + }, + { + "evaluations": 28, + "graph_index": 44, + "passed": true, + "primitive_ids": [ + 42, + 46 + ], + "transaction_index": 3 + }, + { + "evaluations": 27, + "graph_index": 44, + "passed": true, + "primitive_ids": [ + 57, + 62, + 58 + ], + "transaction_index": 4 + }, + { + "evaluations": 11, + "graph_index": 45, + "passed": true, + "primitive_ids": [ + 61 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 45, + "passed": true, + "primitive_ids": [ + 40, + 45, + 46, + 41 + ], + "transaction_index": 1 + }, + { + "evaluations": 19, + "graph_index": 45, + "passed": true, + "primitive_ids": [ + 54 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 45, + "passed": true, + "primitive_ids": [ + 61, + 63, + 62, + 56 + ], + "transaction_index": 3 + }, + { + "evaluations": 25, + "graph_index": 45, + "passed": true, + "primitive_ids": [ + 18, + 22, + 21 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 46, + "passed": true, + "primitive_ids": [ + 59, + 60, + 57, + 56 + ], + "transaction_index": 0 + }, + { + "evaluations": 6, + "graph_index": 46, + "passed": true, + "primitive_ids": [ + 7 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 46, + "passed": true, + "primitive_ids": [ + 41, + 43, + 40, + 46 + ], + "transaction_index": 2 + }, + { + "evaluations": 20, + "graph_index": 46, + "passed": true, + "primitive_ids": [ + 43 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 46, + "passed": true, + "primitive_ids": [ + 19, + 17, + 21 + ], + "transaction_index": 4 + }, + { + "evaluations": 29, + "graph_index": 47, + "passed": true, + "primitive_ids": [ + 18, + 23, + 19 + ], + "transaction_index": 0 + }, + { + "evaluations": 21, + "graph_index": 47, + "passed": true, + "primitive_ids": [ + 30, + 25 + ], + "transaction_index": 1 + }, + { + "evaluations": 17, + "graph_index": 47, + "passed": true, + "primitive_ids": [ + 4, + 1 + ], + "transaction_index": 2 + }, + { + "evaluations": 32, + "graph_index": 47, + "passed": true, + "primitive_ids": [ + 55, + 48, + 49 + ], + "transaction_index": 3 + }, + { + "evaluations": 10, + "graph_index": 47, + "passed": true, + "primitive_ids": [ + 4 + ], + "transaction_index": 4 + }, + { + "evaluations": 16, + "graph_index": 48, + "passed": true, + "primitive_ids": [ + 21 + ], + "transaction_index": 0 + }, + { + "evaluations": 12, + "graph_index": 48, + "passed": true, + "primitive_ids": [ + 51, + 55, + 53 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 48, + "passed": true, + "primitive_ids": [ + 60 + ], + "transaction_index": 2 + }, + { + "evaluations": 10, + "graph_index": 48, + "passed": true, + "primitive_ids": [ + 55 + ], + "transaction_index": 3 + }, + { + "evaluations": 28, + "graph_index": 48, + "passed": true, + "primitive_ids": [ + 1, + 4 + ], + "transaction_index": 4 + }, + { + "evaluations": 10, + "graph_index": 49, + "passed": true, + "primitive_ids": [ + 41 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 49, + "passed": true, + "primitive_ids": [ + 2, + 5, + 4, + 0 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 49, + "passed": true, + "primitive_ids": [ + 49 + ], + "transaction_index": 2 + }, + { + "evaluations": 26, + "graph_index": 49, + "passed": true, + "primitive_ids": [ + 17, + 21 + ], + "transaction_index": 3 + }, + { + "evaluations": 23, + "graph_index": 49, + "passed": true, + "primitive_ids": [ + 44, + 43 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 50, + "passed": true, + "primitive_ids": [ + 38, + 34, + 33, + 39 + ], + "transaction_index": 0 + }, + { + "evaluations": 27, + "graph_index": 50, + "passed": true, + "primitive_ids": [ + 29, + 30 + ], + "transaction_index": 1 + }, + { + "evaluations": 11, + "graph_index": 50, + "passed": true, + "primitive_ids": [ + 3 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 50, + "passed": true, + "primitive_ids": [ + 56, + 63, + 62 + ], + "transaction_index": 3 + }, + { + "evaluations": 28, + "graph_index": 50, + "passed": true, + "primitive_ids": [ + 22, + 20, + 21 + ], + "transaction_index": 4 + }, + { + "evaluations": 15, + "graph_index": 51, + "passed": true, + "primitive_ids": [ + 28 + ], + "transaction_index": 0 + }, + { + "evaluations": 29, + "graph_index": 51, + "passed": true, + "primitive_ids": [ + 51, + 48, + 53 + ], + "transaction_index": 1 + }, + { + "evaluations": 23, + "graph_index": 51, + "passed": true, + "primitive_ids": [ + 9 + ], + "transaction_index": 2 + }, + { + "evaluations": 29, + "graph_index": 51, + "passed": true, + "primitive_ids": [ + 41, + 47, + 46 + ], + "transaction_index": 3 + }, + { + "evaluations": 29, + "graph_index": 51, + "passed": true, + "primitive_ids": [ + 24, + 26 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 52, + "passed": true, + "primitive_ids": [ + 0, + 5, + 7 + ], + "transaction_index": 0 + }, + { + "evaluations": 30, + "graph_index": 52, + "passed": true, + "primitive_ids": [ + 16, + 23, + 17, + 19 + ], + "transaction_index": 1 + }, + { + "evaluations": 26, + "graph_index": 52, + "passed": true, + "primitive_ids": [ + 38, + 33, + 35 + ], + "transaction_index": 2 + }, + { + "evaluations": 28, + "graph_index": 52, + "passed": true, + "primitive_ids": [ + 8 + ], + "transaction_index": 3 + }, + { + "evaluations": 26, + "graph_index": 52, + "passed": true, + "primitive_ids": [ + 31, + 27 + ], + "transaction_index": 4 + }, + { + "evaluations": 26, + "graph_index": 53, + "passed": true, + "primitive_ids": [ + 18, + 23, + 20, + 16 + ], + "transaction_index": 0 + }, + { + "evaluations": 27, + "graph_index": 53, + "passed": true, + "primitive_ids": [ + 14, + 13 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 53, + "passed": true, + "primitive_ids": [ + 28, + 26, + 30, + 27 + ], + "transaction_index": 2 + }, + { + "evaluations": 19, + "graph_index": 53, + "passed": true, + "primitive_ids": [ + 51, + 50 + ], + "transaction_index": 3 + }, + { + "evaluations": 26, + "graph_index": 53, + "passed": true, + "primitive_ids": [ + 22, + 20, + 23, + 21 + ], + "transaction_index": 4 + }, + { + "evaluations": 22, + "graph_index": 54, + "passed": true, + "primitive_ids": [ + 55, + 51 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 54, + "passed": true, + "primitive_ids": [ + 5, + 1, + 0, + 7 + ], + "transaction_index": 1 + }, + { + "evaluations": 1, + "graph_index": 54, + "passed": true, + "primitive_ids": [ + 39 + ], + "transaction_index": 2 + }, + { + "evaluations": 25, + "graph_index": 54, + "passed": true, + "primitive_ids": [ + 52 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 54, + "passed": true, + "primitive_ids": [ + 18, + 23, + 20 + ], + "transaction_index": 4 + }, + { + "evaluations": 17, + "graph_index": 55, + "passed": true, + "primitive_ids": [ + 13 + ], + "transaction_index": 0 + }, + { + "evaluations": 24, + "graph_index": 55, + "passed": true, + "primitive_ids": [ + 17, + 19 + ], + "transaction_index": 1 + }, + { + "evaluations": 17, + "graph_index": 55, + "passed": true, + "primitive_ids": [ + 46 + ], + "transaction_index": 2 + }, + { + "evaluations": 15, + "graph_index": 55, + "passed": true, + "primitive_ids": [ + 21 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 55, + "passed": true, + "primitive_ids": [ + 58, + 56, + 63 + ], + "transaction_index": 4 + }, + { + "evaluations": 25, + "graph_index": 56, + "passed": true, + "primitive_ids": [ + 6 + ], + "transaction_index": 0 + }, + { + "evaluations": 29, + "graph_index": 56, + "passed": true, + "primitive_ids": [ + 56, + 62, + 59 + ], + "transaction_index": 1 + }, + { + "evaluations": 29, + "graph_index": 56, + "passed": true, + "primitive_ids": [ + 29, + 26, + 28 + ], + "transaction_index": 2 + }, + { + "evaluations": 21, + "graph_index": 56, + "passed": true, + "primitive_ids": [ + 50, + 49 + ], + "transaction_index": 3 + }, + { + "evaluations": 27, + "graph_index": 56, + "passed": true, + "primitive_ids": [ + 62, + 61, + 56 + ], + "transaction_index": 4 + }, + { + "evaluations": 24, + "graph_index": 57, + "passed": true, + "primitive_ids": [ + 42, + 47 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 57, + "passed": true, + "primitive_ids": [ + 24 + ], + "transaction_index": 1 + }, + { + "evaluations": 30, + "graph_index": 57, + "passed": true, + "primitive_ids": [ + 62, + 59 + ], + "transaction_index": 2 + }, + { + "evaluations": 26, + "graph_index": 57, + "passed": true, + "primitive_ids": [ + 49, + 54, + 55 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 57, + "passed": true, + "primitive_ids": [ + 24, + 31, + 27 + ], + "transaction_index": 4 + }, + { + "evaluations": 32, + "graph_index": 58, + "passed": true, + "primitive_ids": [ + 16, + 22 + ], + "transaction_index": 0 + }, + { + "evaluations": 28, + "graph_index": 58, + "passed": true, + "primitive_ids": [ + 31, + 30, + 29, + 27 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 58, + "passed": true, + "primitive_ids": [ + 6, + 7 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 58, + "passed": true, + "primitive_ids": [ + 11, + 10, + 14, + 9 + ], + "transaction_index": 3 + }, + { + "evaluations": 32, + "graph_index": 58, + "passed": true, + "primitive_ids": [ + 2, + 1 + ], + "transaction_index": 4 + }, + { + "evaluations": 30, + "graph_index": 59, + "passed": true, + "primitive_ids": [ + 6, + 1, + 0 + ], + "transaction_index": 0 + }, + { + "evaluations": 7, + "graph_index": 59, + "passed": true, + "primitive_ids": [ + 53, + 55 + ], + "transaction_index": 1 + }, + { + "evaluations": 32, + "graph_index": 59, + "passed": true, + "primitive_ids": [ + 39, + 33, + 36, + 37 + ], + "transaction_index": 2 + }, + { + "evaluations": 11, + "graph_index": 59, + "passed": true, + "primitive_ids": [ + 47, + 44 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 59, + "passed": true, + "primitive_ids": [ + 57, + 59, + 58, + 63 + ], + "transaction_index": 4 + }, + { + "evaluations": 3, + "graph_index": 60, + "passed": true, + "primitive_ids": [ + 35 + ], + "transaction_index": 0 + }, + { + "evaluations": 31, + "graph_index": 60, + "passed": true, + "primitive_ids": [ + 32, + 37, + 39, + 33 + ], + "transaction_index": 1 + }, + { + "evaluations": 18, + "graph_index": 60, + "passed": true, + "primitive_ids": [ + 21 + ], + "transaction_index": 2 + }, + { + "evaluations": 28, + "graph_index": 60, + "passed": true, + "primitive_ids": [ + 58, + 63 + ], + "transaction_index": 3 + }, + { + "evaluations": 19, + "graph_index": 60, + "passed": true, + "primitive_ids": [ + 7 + ], + "transaction_index": 4 + }, + { + "evaluations": 22, + "graph_index": 61, + "passed": true, + "primitive_ids": [ + 51 + ], + "transaction_index": 0 + }, + { + "evaluations": 14, + "graph_index": 61, + "passed": true, + "primitive_ids": [ + 6, + 3 + ], + "transaction_index": 1 + }, + { + "evaluations": 31, + "graph_index": 61, + "passed": true, + "primitive_ids": [ + 26, + 31, + 27 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 61, + "passed": true, + "primitive_ids": [ + 46, + 47, + 41, + 43 + ], + "transaction_index": 3 + }, + { + "evaluations": 14, + "graph_index": 61, + "passed": true, + "primitive_ids": [ + 44 + ], + "transaction_index": 4 + }, + { + "evaluations": 27, + "graph_index": 62, + "passed": true, + "primitive_ids": [ + 14, + 9, + 13 + ], + "transaction_index": 0 + }, + { + "evaluations": 30, + "graph_index": 62, + "passed": true, + "primitive_ids": [ + 28, + 26, + 27 + ], + "transaction_index": 1 + }, + { + "evaluations": 23, + "graph_index": 62, + "passed": true, + "primitive_ids": [ + 23, + 18 + ], + "transaction_index": 2 + }, + { + "evaluations": 31, + "graph_index": 62, + "passed": true, + "primitive_ids": [ + 38, + 36, + 39 + ], + "transaction_index": 3 + }, + { + "evaluations": 31, + "graph_index": 62, + "passed": true, + "primitive_ids": [ + 29, + 24, + 30 + ], + "transaction_index": 4 + }, + { + "evaluations": 31, + "graph_index": 63, + "passed": true, + "primitive_ids": [ + 19, + 21, + 18 + ], + "transaction_index": 0 + }, + { + "evaluations": 32, + "graph_index": 63, + "passed": true, + "primitive_ids": [ + 44, + 40, + 47, + 43 + ], + "transaction_index": 1 + }, + { + "evaluations": 28, + "graph_index": 63, + "passed": true, + "primitive_ids": [ + 51, + 49, + 50 + ], + "transaction_index": 2 + }, + { + "evaluations": 30, + "graph_index": 63, + "passed": true, + "primitive_ids": [ + 34, + 39, + 37 + ], + "transaction_index": 3 + }, + { + "evaluations": 30, + "graph_index": 63, + "passed": true, + "primitive_ids": [ + 32, + 37, + 34 + ], + "transaction_index": 4 + } + ], + "affected_injection": [ + { + "affected_closure_size": 6, + "exception": "RuntimeError", + "fail_node": 76, + "graph_index": 0, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 28, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 1, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 21, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 2, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 28, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 3, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 20, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 4, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 25, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 5, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 17, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 6, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 23, + "exception": "RuntimeError", + "fail_node": 65, + "graph_index": 7, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 26, + "exception": "RuntimeError", + "fail_node": 66, + "graph_index": 8, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 20, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 9, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 19, + "exception": "RuntimeError", + "fail_node": 68, + "graph_index": 10, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 22, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 11, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 20, + "exception": "RuntimeError", + "fail_node": 68, + "graph_index": 12, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 2, + "exception": "RuntimeError", + "fail_node": 79, + "graph_index": 13, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 21, + "exception": "RuntimeError", + "fail_node": 66, + "graph_index": 14, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 20, + "exception": "RuntimeError", + "fail_node": 68, + "graph_index": 15, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 23, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 16, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 6, + "exception": "RuntimeError", + "fail_node": 72, + "graph_index": 17, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 14, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 18, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 19, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 19, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 17, + "exception": "RuntimeError", + "fail_node": 72, + "graph_index": 20, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 29, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 21, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 11, + "exception": "RuntimeError", + "fail_node": 77, + "graph_index": 22, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 15, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 23, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 10, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 24, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 5, + "exception": "RuntimeError", + "fail_node": 80, + "graph_index": 25, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 25, + "exception": "RuntimeError", + "fail_node": 69, + "graph_index": 26, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 18, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 27, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 9, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 28, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 31, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 29, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 24, + "exception": "RuntimeError", + "fail_node": 66, + "graph_index": 30, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 21, + "exception": "RuntimeError", + "fail_node": 68, + "graph_index": 31, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 30, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 32, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 24, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 33, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 22, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 34, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 13, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 35, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 25, + "exception": "RuntimeError", + "fail_node": 66, + "graph_index": 36, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 26, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 37, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 21, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 38, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 27, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 39, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 5, + "exception": "RuntimeError", + "fail_node": 74, + "graph_index": 40, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 22, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 41, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 12, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 42, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 30, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 43, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 13, + "exception": "RuntimeError", + "fail_node": 73, + "graph_index": 44, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 15, + "exception": "RuntimeError", + "fail_node": 66, + "graph_index": 45, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 14, + "exception": "RuntimeError", + "fail_node": 75, + "graph_index": 46, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 31, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 47, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 16, + "exception": "RuntimeError", + "fail_node": 65, + "graph_index": 48, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 24, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 49, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 10, + "exception": "RuntimeError", + "fail_node": 70, + "graph_index": 50, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 17, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 51, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 9, + "exception": "RuntimeError", + "fail_node": 74, + "graph_index": 52, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 8, + "exception": "RuntimeError", + "fail_node": 70, + "graph_index": 53, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 9, + "exception": "RuntimeError", + "fail_node": 74, + "graph_index": 54, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 18, + "exception": "RuntimeError", + "fail_node": 65, + "graph_index": 55, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 2, + "exception": "RuntimeError", + "fail_node": 92, + "graph_index": 56, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 17, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 57, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 1, + "exception": "RuntimeError", + "fail_node": 93, + "graph_index": 58, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 28, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 59, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 16, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 60, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 24, + "exception": "RuntimeError", + "fail_node": 67, + "graph_index": 61, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 15, + "exception": "RuntimeError", + "fail_node": 64, + "graph_index": 62, + "passed": true, + "primitive": 0 + }, + { + "affected_closure_size": 9, + "exception": "RuntimeError", + "fail_node": 72, + "graph_index": 63, + "passed": true, + "primitive": 0 + } + ], + "connected_boundary": [ + { + "control_index": 0, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 1, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 2, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 3, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 4, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 5, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 6, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 7, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 8, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 9, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 10, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 11, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 12, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 13, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 14, + "evaluations": 256, + "passed": true, + "saving": 0.0 + }, + { + "control_index": 15, + "evaluations": 256, + "passed": true, + "saving": 0.0 + } + ], + "cycle_rejection": [ + { + "control_index": 0, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 1, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 2, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 3, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 4, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 5, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 6, + "exception": "ValueError", + "passed": true + }, + { + "control_index": 7, + "exception": "ValueError", + "passed": true + } + ], + "malformed_update": [ + { + "exception": "ValueError", + "graph_index": 0, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 1, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 2, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 3, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 4, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 5, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 6, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 7, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 8, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 9, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 10, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 11, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 12, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 13, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 14, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 15, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 16, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 17, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 18, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 19, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 20, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 21, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 22, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 23, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 24, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 25, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 26, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 27, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 28, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 29, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 30, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 31, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 32, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 33, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 34, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 35, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 36, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 37, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 38, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 39, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 40, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 41, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 42, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 43, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 44, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 45, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 46, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 47, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 48, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 49, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 50, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 51, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 52, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 53, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 54, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 55, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 56, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 57, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 58, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 59, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 60, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 61, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 62, + "passed": true + }, + { + "exception": "ValueError", + "graph_index": 63, + "passed": true + } + ], + "stale_revision": [ + { + "exception": "RuntimeError", + "graph_index": 0, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 1, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 2, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 3, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 4, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 5, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 6, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 7, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 8, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 9, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 10, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 11, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 12, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 13, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 14, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 15, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 16, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 17, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 18, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 19, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 20, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 21, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 22, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 23, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 24, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 25, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 26, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 27, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 28, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 29, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 30, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 31, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 32, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 33, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 34, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 35, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 36, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 37, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 38, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 39, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 40, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 41, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 42, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 43, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 44, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 45, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 46, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 47, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 48, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 49, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 50, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 51, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 52, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 53, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 54, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 55, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 56, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 57, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 58, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 59, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 60, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 61, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 62, + "passed": true + }, + { + "exception": "RuntimeError", + "graph_index": 63, + "passed": true + } + ], + "unaffected_noop": [ + { + "affected_closure_size": 0, + "control_index": 0, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 1, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 2, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 3, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 4, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 5, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 6, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 7, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 8, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 9, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 10, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 11, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 12, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 13, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 14, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 15, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 16, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 17, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 18, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 19, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 20, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 21, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 22, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 23, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 24, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 25, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 26, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 27, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 28, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 29, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 30, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 31, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 32, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 33, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 34, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 35, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 36, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 37, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 38, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 39, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 40, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 41, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 42, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 43, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 44, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 45, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 46, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 47, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 48, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 49, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 50, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 51, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 52, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 53, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 54, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 55, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 56, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 57, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 58, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 59, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 60, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 61, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 62, + "evaluations": 0, + "passed": true, + "primitive": 1 + }, + { + "affected_closure_size": 0, + "control_index": 63, + "evaluations": 0, + "passed": true, + "primitive": 1 + } + ] + }, + "counts": { + "accepted": { + "passed": 320, + "total": 320 + }, + "affected_injection": { + "passed": 64, + "total": 64 + }, + "connected_boundary": { + "passed": 16, + "total": 16 + }, + "cycle_rejection": { + "passed": 8, + "total": 8 + }, + "malformed_update": { + "passed": 64, + "total": 64 + }, + "stale_revision": { + "passed": 64, + "total": 64 + }, + "unaffected_noop": { + "passed": 64, + "total": 64 + } + }, + "evaluation_reduction": 0.90047607421875, + "full_recompute_evaluations": 81920, + "local_evaluations": 8153, + "original_h2_result": { + "passed": 63, + "primary_target_met": false, + "total": 64 + }, + "primary_target_met": true, + "protocol_commit": "f0d39d568c66175ffba0f023e6eb36c5fbad39e0", + "schema_version": 1, + "seed": 20260929, + "source_sha256": { + "PROTOCOL.md": "c4878691c4efb7d8bf69f8ade4a50d0bb7b979fec89a81a911b7525b09baa810", + "run.py": "bc6a744bcb6eaa4bed15a40e8b77f0f89869bfb3e2624f7641e38cb5bac48a9d" + } +} diff --git a/graph_synthesis/post_certificate_h2_followup/run.py b/graph_synthesis/post_certificate_h2_followup/run.py new file mode 100644 index 0000000..2d45dc9 --- /dev/null +++ b/graph_synthesis/post_certificate_h2_followup/run.py @@ -0,0 +1,182 @@ +"""Execute the separate, frozen transactional fault-injection follow-up.""" +from __future__ import annotations + +import argparse +import hashlib +import json +import random +from pathlib import Path + +from graph_synthesis.post_certificate.methods import DependencyGraph +from graph_synthesis.post_certificate.run import make_connected_dag, make_modular_dag + +ROOT = Path(__file__).resolve().parent +SEED = 20260929 + + +def affected_nodes(graph: DependencyGraph, primitive: int) -> set[int]: + seen: set[int] = set() + frontier = [primitive] + while frontier: + for child in graph.reverse[frontier.pop()]: + if child not in seen: + seen.add(child) + frontier.append(child) + return seen + + +def exception_name(call, *, suppress_fault: bool = False) -> str | None: + try: + call() + except Exception as exc: + if suppress_fault and isinstance(exc, RuntimeError) and str(exc) == 'injected evaluation failure': + return None + return type(exc).__name__ + return None + + +def execute(*, suppress_fault: bool = False) -> dict: + rng = random.Random(SEED) + accepted = [] + injections = [] + stale = [] + malformed = [] + unaffected = [] + cycles = [] + connected = [] + local_evaluations = 0 + + for graph_index in range(64): + graph = make_modular_dag(rng) + primitives = {i: bool(rng.getrandbits(1)) for i in range(64)} + values = graph.full_recompute(primitives) + revision = 0 + for transaction_index in range(5): + module = rng.randrange(8) + count = rng.randint(1, 4) + selected = rng.sample(range(module * 8, (module + 1) * 8), count) + changes = {p: not values[p] for p in selected} + new_values, new_revision, evaluations = graph.transact( + values, revision, changes, expected_revision=revision + ) + recomputed = graph.full_recompute({i: new_values[i] for i in range(64)}) + passed = new_values == recomputed and new_revision == revision + 1 + accepted.append({'graph_index': graph_index, 'transaction_index': transaction_index, + 'primitive_ids': selected, 'evaluations': evaluations, 'passed': passed}) + local_evaluations += evaluations + values, revision = new_values, new_revision + + primitive = next(p for p in range(64) if affected_nodes(graph, p)) + closure = affected_nodes(graph, primitive) + fail_node = min(closure) + before = dict(values) + changes = {primitive: not values[primitive]} + caught = exception_name( + lambda: graph.transact(values, revision, changes, + expected_revision=revision, fail_node=fail_node), + suppress_fault=suppress_fault, + ) + injections.append({'graph_index': graph_index, 'primitive': primitive, + 'fail_node': fail_node, 'affected_closure_size': len(closure), + 'exception': caught, 'passed': caught == 'RuntimeError' + and values == before and revision == 5}) + caught = exception_name(lambda: graph.transact( + values, revision, changes, expected_revision=revision - 1)) + stale.append({'graph_index': graph_index, 'exception': caught, + 'passed': caught == 'RuntimeError' and values == before and revision == 5}) + caught = exception_name(lambda: graph.transact( + values, revision, {64: True}, expected_revision=revision)) + malformed.append({'graph_index': graph_index, 'exception': caught, + 'passed': caught == 'ValueError' and values == before and revision == 5}) + + for control_index in range(64): + graph = DependencyGraph(2, {2: ('xor', (0,))}) + primitives = {i: bool(rng.getrandbits(1)) for i in range(2)} + values = graph.full_recompute(primitives) + new_values, revision, evaluations = graph.transact( + values, 0, {1: not values[1]}, expected_revision=0) + recomputed = graph.full_recompute({i: new_values[i] for i in range(2)}) + unaffected.append({'control_index': control_index, 'primitive': 1, + 'affected_closure_size': len(affected_nodes(graph, 1)), + 'evaluations': evaluations, 'passed': revision == 1 + and evaluations == 0 and new_values == recomputed + and values == graph.full_recompute(primitives)}) + + for control_index in range(8): + first = 1 + 2 * control_index + second = first + 1 + caught = exception_name(lambda: DependencyGraph(1, { + first: ('xor', (second,)), second: ('xor', (first,))})) + cycles.append({'control_index': control_index, 'exception': caught, + 'passed': caught == 'ValueError'}) + + for control_index in range(16): + graph = make_connected_dag() + primitives = {j: bool((j + control_index) % 2) for j in range(64)} + values = graph.full_recompute(primitives) + new_values, revision, evaluations = graph.transact( + values, 0, {0: not values[0]}, expected_revision=0) + recomputed = graph.full_recompute({j: new_values[j] for j in range(64)}) + connected.append({'control_index': control_index, 'evaluations': evaluations, + 'saving': 1 - evaluations / 256, + 'passed': revision == 1 and new_values == recomputed}) + + reduction = 1 - local_evaluations / (320 * 256) + groups = {'accepted': accepted, 'affected_injection': injections, + 'stale_revision': stale, 'malformed_update': malformed, + 'unaffected_noop': unaffected, 'cycle_rejection': cycles, + 'connected_boundary': connected} + counts = {name: {'passed': sum(bool(row['passed']) for row in rows), 'total': len(rows)} + for name, rows in groups.items()} + target = all(counts[name]['passed'] == counts[name]['total'] + for name in groups if name != 'connected_boundary') + target = target and reduction >= .80 + hashes = {p.name: hashlib.sha256(p.read_bytes()).hexdigest() + for p in (ROOT / 'PROTOCOL.md', Path(__file__))} + return {'schema_version': 1, 'seed': SEED, 'protocol_commit': 'f0d39d568c66175ffba0f023e6eb36c5fbad39e0', + 'source_sha256': hashes, 'original_h2_result': {'passed': 63, 'total': 64, 'primary_target_met': False}, + 'counts': counts, 'local_evaluations': local_evaluations, + 'full_recompute_evaluations': 320 * 256, 'evaluation_reduction': reduction, + 'connected_mean_reduction': sum(row['saving'] for row in connected) / 16, + 'primary_target_met': target, 'controls': groups} + + +def render(result: dict) -> str: + lines = ['# Transactional fault-injection follow-up', '', + 'This is a separate precommitted study. The original PR #33 H2 result remains **failed (63/64)**.', '', + f"New conjunction: **{'met' if result['primary_target_met'] else 'not met'}**.", '', + '| Endpoint | Passed / total |', '|---|---:|'] + for name, count in result['counts'].items(): + lines.append(f"| {name.replace('_', ' ')} | {count['passed']}/{count['total']} |") + lines += ['', f"Local derived evaluations: {result['local_evaluations']:,} / " + f"{result['full_recompute_evaluations']:,} full recomputation evaluations " + f"({result['evaluation_reduction']:.2%} reduction).", + f"Connected control mean saving: {result['connected_mean_reduction']:.2%}; reported separately.", '', + 'Affected injections require an actual derived dependency and an exception. ' + 'Unaffected primitive updates correctly commit without derived evaluation. ' + 'This measures in-memory behavior only; it does not establish crash durability, ' + 'concurrency isolation, semantic accuracy, or wall-clock performance.', '', + 'See [the frozen protocol](PROTOCOL.md) and [all control records](results.json).', ''] + return '\n'.join(lines) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument('--check', action='store_true') + args = parser.parse_args() + result = execute() + output = json.dumps(result, sort_keys=True, indent=2) + '\n' + report = render(result) + if args.check: + if (ROOT / 'results.json').read_text(encoding='utf-8') != output: + raise SystemExit('Saved result differs from frozen replay') + if (ROOT / 'RESULTS.md').read_text(encoding='utf-8') != report: + raise SystemExit('Saved report differs from frozen replay') + else: + (ROOT / 'results.json').write_text(output, encoding='utf-8') + (ROOT / 'RESULTS.md').write_text(report, encoding='utf-8') + print(json.dumps({'target_met': result['primary_target_met'], 'counts': result['counts']})) + + +if __name__ == '__main__': + main() diff --git a/graph_synthesis/post_certificate_h2_followup/test_run.py b/graph_synthesis/post_certificate_h2_followup/test_run.py new file mode 100644 index 0000000..c24b99d --- /dev/null +++ b/graph_synthesis/post_certificate_h2_followup/test_run.py @@ -0,0 +1,22 @@ +import unittest + +from graph_synthesis.post_certificate_h2_followup.run import execute + + +class FrozenFollowupTests(unittest.TestCase): + def test_full_replay_meets_new_conjunction(self): + result = execute() + self.assertTrue(result['primary_target_met']) + self.assertEqual(result['original_h2_result'], + {'passed': 63, 'total': 64, 'primary_target_met': False}) + self.assertEqual(result['counts']['affected_injection'], {'passed': 64, 'total': 64}) + self.assertEqual(result['counts']['unaffected_noop'], {'passed': 64, 'total': 64}) + + def test_suppressed_injection_is_detected(self): + result = execute(suppress_fault=True) + self.assertFalse(result['primary_target_met']) + self.assertEqual(result['counts']['affected_injection'], {'passed': 0, 'total': 64}) + + +if __name__ == '__main__': + unittest.main()