Scientific implementation report — original-proposal track, with a frozen-program assessment on 256 fresh paired cases.
Experiments are stopped for an accountability review. The broader study is incomplete; no calibration or further search is running. The contribution record connects calls to executable changes and measured results. A separate supervision audit records usage omitted from experimental call totals. Paper synthesis follows completion of the study.
We implement the original Sakana ShinkaDreamer proposal: evolve an executable agent whose world-model updater integrates partial observations and whose planner uses that internal map to navigate a dynamic maze. Selection combines task reward and current-map reconstruction accuracy with the proposed weights of 0.6 and 0.4. The implementation preserves the 15×15 maze,5×5 observation window, keys, gating door, moving enemies and changing walls, with explicit repairs to the example code's coordinates, mechanics and evaluator interface. In an initial development characterization, the unevolved seed completed 32 valid episodes and escaped in none;20 ended in capture and 12 timed out. Mean map accuracy was 98.31%, with 32.90% mean final map coverage. These results characterize the starting agent and show that accurate remembered cells alone do not establish successful navigation. The first native ShinkaEvolve descendant escaped all five search cases. A subsequent paired development check of generation 2 found 32/32 escapes versus 0/32 for the seed, including all 27 cases outside the five-case search panel. Mean fitness rose from 0.4967 to 0.9353 and final map coverage from 32.90% to 83.64%; reported-cell accuracy decreased slightly to 98.04%. On a subsequent disjoint 64-case selection-validation panel, both descendants escaped 61/64 cases, versus 0/64 for the seed. The prespecified fitness rule selected generation 2, but its advantage over generation 1 was unresolved. Its fitness gain over the seed was 0.4171 (descriptive paired 95% interval 0.3817–0.4476). A final assessment of the frozen generation 2 on 256 fresh paired cases found 239/256 escapes (93.36%) versus 2/256 (0.78%) for the original seed. Mean fitness improved by 0.4026 (paired 95% interval 0.3835–0.4201), almost entirely through task reward; the change in reported-cell accuracy was unresolved. All 512 assessment episodes took 64.06 seconds with no experiment-model calls. This establishes improvement over the original weak seed on the declared task distribution. A subsequent 256-case control experiment found source-specific benefits from persistent spatial maps and the hardcoded future-risk heuristic; fitness superiority over a competent handcrafted pathfinder remains unresolved. Independent discovery reliability is still under study.
Partial observation makes memory and planning central to maze navigation. An agent must retain information about previously observed passages and objectives, reconcile that memory with a changing world, and choose actions using incomplete knowledge. The original Sakana proposal addresses this problem by jointly evolving two Python functions:
world_model_step(memory, local_obs, last_action) -> memory
planner(memory, local_obs) -> actionThe scientific question is whether program evolution improves task performance and internal-map quality through interpretable changes to these functions. Possible changes include pathfinding, uncertainty, remembered enemy sightings, exploration and replanning. Helpers and memory structures remain evolvable; this is not restricted to scalar tuning or a fixed algorithm catalog.
Here, world model means the agent's maintained representation of the current world. Future prediction and learning an unknown transition law are not required by this experiment. ShinkaDreamer is not an implementation of neural Dreamer. Earlier predictive-learning extensions are retained in a separate historical archive; none of their measurements or figures is presented as evidence for this objective.
The environment contains a 15×15 maze, two keys, a locked door gating the exit and three anonymous enemies. The agent receives a 5×5 local observation. It chooses one of nine attempted displacements, including waiting, and an interaction flag. Enemies also draw uniform attempted displacements; blocked moves remain stays. Walls change every 25 transitions and episodes end on capture, escape or the 200-transition horizon.
The implementation reuses the existing repaired maze. Border walls remain fixed, layouts have feasible objective routes, diagonal corner cutting is disallowed, and wall closures skip occupied cells. Interaction opens an adjacent cardinal door before movement when both keys are held and collects a key at the movement destination. Contact with an enemy before or after enemy movement is fatal. These are explicit interpretations and repairs of the proposal's prose and example, documented in implementation details.
Candidates receive local observations, inventory, door state and observable movement/interaction feedback. The hidden world, world seeds and scoring remain inside the evaluator. The candidate runs in the existing restricted Python process, with independent candidate randomness. No forecast interface, hidden movement-law condition or prediction-learning intervention is imposed.
The initial program closely follows the proposal: integrate observations into a persistent map, retain enemy sightings for five steps, move greedily toward the nearest remembered objective, and otherwise wander randomly. It has no engineered pathfinder or learned dynamics predictor. Repairs apply actual displacement before map integration, maintain observable inventory and door state, represent anonymous sightings, and permit interaction on a key already under the agent.
The native launcher uses upstream ShinkaEvolve revision
9912af1.
The proposal's nominal configuration has 100 total slots, four islands and five
development episodes per evaluated program. We implement it in separately reviewed
blocks. The first block has a six-slot ceiling, including the seed and failures,
30 actual provider calls across all roles, 1,200 summed provider seconds and a
40-minute invocation ceiling. It does not automatically continue toward 100 slots.
The bounded launcher retains native weighted parent selection, archive/top inspirations, migration, diff/full/crossover operators, reward-only model allocation, real local BGE embeddings, conditional novelty judging, summaries/recommendations and prompt coevolution. Mutation allocation uses Astra/high and Sol 6.1/high through the strict subscription route; all auxiliary roles use that same route, and embeddings run locally. Configured mechanisms and actual events are reported separately. See the frozen protocol, staged plan and source-level Shinka study.
For an episode with length
After each observation and planning call, but before applying its action, the
evaluator compares the reported believed_map with current terrain and enemy
occupancy. Relative map coordinates are translated privately using the initial
agent position. Let
An empty audit has
Before execution, we fixed 32 development cases, numbered 0–31, and evaluated only the initial program, with a maximum of 6,400 transitions. No fitting, candidate selection or mutation-model calls were performed. These are development cases, not a fresh assessment or evidence of generalization. The first case's replay was selected by index before observing its outcome.
This 32-case characterization is separate from the launcher's five-case selection default. Its score must not be imported as native search fitness; a search must evaluate its seed and descendants on the same configured panel.
The saved measurements, per-episode data and source manifest support all results and figures below. Every figure uses the shared Chromatic Fields style and only this original-proposal experiment's data.
| Measurement | Initial seed,32 episodes |
|---|---|
| Escape | 0/32 (0%) |
| Capture | 20/32 (62.5%) |
| Timeout | 12/32 (37.5%) |
| Invalid execution | 0/32 |
| Mean task score |
0.172325 |
| Mean map accuracy |
0.983149 |
| Mean combined fitness |
0.496655 |
| Mean final map coverage | 32.90% |
| Mean collected keys | 0.53125 |
| Opened door | 0/32 |
| Mean episode length | 137.06 transitions |
Figure 1. Initial-agent characterization. All 32 episodes are included. Outcome intervals are descriptive 95% Wilson intervals for the development panel; score and accuracy panels retain individual episodes. Coverage is reported independently of accuracy. These are results for one unevolved program, not an evolutionary comparison or a held-out success estimate.
Figure 2. Actual map memory in the first recorded episode. The first and last recorded pre-action states show evaluator truth alongside the agent's reported map, translated into common coordinates. Hidden truth is a retrospective visualization and was never supplied to the agent. Unknown map cells remain visually distinct. This case ended in capture; the last displayed state precedes the terminal action and is not a post-capture reconstruction.
Execution used 4,386 transitions. Summed measured episode wall time was 2.84 seconds on the current host, including candidate execution; this excludes implementation, tests and figure rendering. That initial characterization used zero experiment-model calls. The subsequent search has its own measurements below.
The search reused development cases 0–4 for every evaluated program. It reserved four of the six permitted slots: the seed, two valid descendants and one unfinished full-rewrite proposal. Both full-rewrite attempts timed out at the declared 240-second request boundary, triggering the prospective two-failure checkpoint. No valid source or episode result exists for that reserved slot. Three additional database rows are administrative island copies of the seed, not evaluated programs.
| Program | Episodes | Escape | Capture / timeout | Task |
Accuracy |
Fitness |
Coverage | Mean steps |
|---|---|---|---|---|---|---|---|---|
| Seed, slot 0 | 5 | 0/5 | 3 / 2 | 0.236000 | 0.986904 | 0.536362 | 34.31% | 150.0 |
| Generation 1 | 5 | 5/5 | 0 / 0 | 0.908240 | 0.982192 | 0.937821 | 88.89% | 70.6 |
| Generation 2 | 5 | 5/5 | 0 / 0 | 0.910880 | 0.982193 | 0.939405 | 90.58% | 77.2 |
| Slot 3, unfinished rewrite | 0 | — | — | — | — | — | — | — |
All 15 executed episodes were valid, with 1,489 transitions. Generation 2 is the search-fitness leader, not a held-out selected champion. Its paired mean fitness gain over the seed is 0.403044: the task term contributes +0.404928 and the accuracy term −0.001884. These gains are attributable to task performance in the objective, not improved reconstruction accuracy.
Generation 2 exceeds generation 1 by only 0.001584 fitness. Almost all of that difference comes from taking 6.6 more steps on average: the original objective rewards elapsed steps. Both escape all five cases, so this ranking does not establish superior navigation efficiency. We retain the proposed reward and disclose this limitation.
Figure 3. Search outcomes under the original objective. Episode points and program means use the same five development cases. The unfinished slot has no invented score. Accuracy and coverage remain separate; these are selection data.
Both successful programs are direct seed mutations without inspirations. They separate persistent terrain from transient enemy occupancy, add visit history and frontier exploration, and replace direct target movement with shortest-path planning through keys, door and exit. Their planners use accumulated map memory and fixed danger estimates based on the known uniform enemy movement rule. They also omit some stale dynamic cells from the scored map while retaining them internally for navigation. This selective export and its denominator matter when interpreting accuracy. No learned transition law or isolated causal effect of memory is established by this code inspection.
Figure 4. Actual native search activity. Both mutation models were selected. Real local embeddings and conditional novelty comparisons ran; one comparison exceeded the 0.95 threshold and invoked the subscription-backed judge, which accepted generation 1. Generation 2's similarity was below threshold and required no judge. Neither result is a scientific novelty claim.
The full configuration was enabled, but this short execution did not exercise every mechanism. Diff produced both valid descendants; full rewrite produced two timeouts. No inspiration or crossover was used, no migration occurred, and no summary, insight, recommendation or prompt-mutation provider call ran. The initial prompt archive received native descendant credit; it did not evolve a new prompt. Three programs remain pending in meta memory. At finalization the shared budget gate rejected further dispatch after the two transport failures. We therefore report a partially exercised full-native configuration, not a demonstration of benefits from the complete machinery.
Figure 5. Outcomes and computational cost. Outcome intervals are nominal 95% Wilson intervals on the development panel; 5/5 gives 56.6–100%. Reusing and selecting on these cases prevents interpreting that interval as held-out generalization evidence. Timed-out requests remain in resource totals; their unreported tokens are unknown.
Figure 6. Leader replay, development case 0. The case is chosen by index, not outcome. Evaluator truth and the reported map are shown retrospectively; hidden truth was never supplied to the agent. All recorded replay frames and nonwinning candidate sources are preserved in the compact results.
The controller ran for 846.11 seconds (14.10 minutes), excluding implementation and reporting. Five provider requests consumed 818.03 seconds: two successful mutations, two timed-out rewrite attempts and one successful novelty judgment. Reported usage from the three successful requests was 30,416 uncached input plus output tokens and 36,864 cached input tokens. Usage for the two timeouts is unknown; these are incomplete totals, not an estimate of subscription allowance.
The 15 episodes consumed 2.915 evaluator-plus-candidate CPU-seconds, of which 2.110 were candidate CPU-seconds. Separately sampled controller and embedding process CPU were 17.71 and 9.59 seconds; peak sampled combined resident memory was 1.09 GiB. The latter CPU samples are not a complete machine-wide resource audit. Supervising-assistant work is additional and is not converted into a weekly allowance percentage. Native API price estimates are not subscription charges.
The search is checkpointed with workers stopped. Its subsequent development comparison is reported below. The historical assessments remain paused.
Before execution, the Stage 3 protocol nominated generation 2 because it led both search fitness and task score. Its source, evaluator and limits were frozen. We evaluated it once on cases 0–31 and reused the seed's saved results after verifying all seven evaluator/dependency source hashes. No candidate was edited, no baseline was rerun and no model was called. The five overlapping search cases reproduced every recorded outcome and score exactly. Cases 5–31 are additional development cases, not a fresh or held-out assessment; this panel was already exposed in the seed characterization.
| Measurement | Saved seed | Generation 2 | Paired difference |
|---|---|---|---|
| Escape | 0/32 | 32/32 | +100 percentage points |
| Capture / timeout / invalid | 20 / 12 / 0 | 0 / 0 / 0 | — |
| Mean task |
0.172325 | 0.905300 | +0.732975 |
| Mean reconstruction |
0.983149 | 0.980418 | −0.002731 |
| Mean fitness |
0.496655 | 0.935347 | +0.438692 |
| Mean final map coverage | 32.90% | 83.64% | +50.74 percentage points |
| Mean keys / opened door | 0.53125 / 0 of 32 | 2 / 32 of 32 | — |
| Mean episode length | 137.06 | 63.25 | −73.81 transitions |
Generation 2 escaped 5/5 reused search cases and 27/27 additional development cases. Its successful episodes lasted 25–130 transitions. The aggregate length difference includes seed failures and is not an escape-speed comparison between successful policies. The fitness gain decomposes into +0.439785 from task reward and −0.001093 from reconstruction. Better task performance therefore persists beyond the five search cases without improved reported-cell accuracy.
Descriptive paired 95% intervals are [0.4137, 0.4623] for the fitness gain, [0.6914, 0.7728] for task score, [−0.0075, 0.0021] for reconstruction, and [43.93, 57.18] percentage points for coverage. These intervals quantify variation across the observed development cases; they do not correct selection bias or establish performance on unseen cases.
Figure 7. Development comparison, all 32 paired episodes. Intervals use 10,000 whole-episode paired bootstrap resamples, preserving the original mean of episode-level reconstruction ratios. The binary bootstrap degenerates when every observed pair has the same escape difference; it cannot establish a perfect population success rate. Separate nominal 95% Wilson intervals are 89.3–100% for generation 2 and 0–10.7% for the seed. These post hoc boundary diagnostics do not remove development exposure or candidate-selection bias.
Figure 8. Every development case. The five search cases and 27 additional development cases are identified explicitly. The comparison preserves all outcomes and uses the same environment case IDs. Reconstruction remains an on-policy current-map measurement under different actions, not a matched-experience prediction experiment.
The new evaluation executed 32 valid episodes and 2,024 transitions, using 5.55 seconds wall time and 5.45 total CPU-seconds, including 4.45 candidate CPU-seconds. No new proposals, mutation calls or auxiliary model calls ran. The seed's CPU usage was not measured in its initial characterization, so no CPU speedup is inferred. Analysis, rendering and supervising-assistant work are additional to these evaluation measurements. All episode data, a paired CSV, uncertainty estimates and rendering provenance are in the Stage 3 results.
Stage 3 is complete. The separately authorized selection-validation step follows.
The Stage 4 protocol fixed
the three sources and selection rule before generating 64 private cases. The
nomination rule retained generation 2 for best search
| Measurement | Original seed | Generation 1 | Generation 2, selected |
|---|---|---|---|
| Escape | 0/64 | 61/64 | 61/64 |
| Capture / timeout / invalid | 37 / 27 / 0 | 3 / 0 / 0 | 3 / 0 / 0 |
| Mean task |
0.173006 | 0.867581 | 0.868744 |
| Mean reconstruction |
0.982144 | 0.980801 | 0.981192 |
| Mean fitness |
0.496662 | 0.912869 | 0.913723 |
| Mean final map coverage | 35.67% | 85.15% | 83.77% |
| Mean keys | 0.46875 | 1.96875 | 1.953125 |
| Opened door | 0/64 | 61/64 | 61/64 |
| Mean episode length | 127.83 | 59.58 | 65.61 |
| Evaluator + candidate CPU-seconds | 5.146 | 10.632 | 11.535 |
The rule selects the descendant with the highest mean original
Generation 2's advantage over generation 1 is only
0.000854 [−0.036302, 0.037922] in
Figure 9. Selection-validation comparison. All 64 cases contribute to each arm. Shared bootstrap case indices preserve pairing across all programs and metrics; reconstruction retains its mean of within-episode ratios. The selected source is frozen, but this panel remains selection data, not final assessment.
Figure 10. Individual cases and failures. Generation 1 was captured on ordinals 11, 25 and 63; generation 2 on 7, 50 and 62. Neither failed on the other's three capture cases. All failures remain visible and in score denominators. The earlier 32/32 development result therefore does not imply universal escape.
Execution completed 192 valid episodes / 16,193 transitions in 28.18 seconds, using 27.62 total CPU-seconds, including 21.14 candidate CPU-seconds. The per-arm table excludes a small amount of controller/export CPU included in that total. There were zero experiment-model calls, new proposals or candidate edits; no additional Shinka mechanism was exercised in this evaluation stage. Each episode retained the 192 MiB/10 candidate CPU-second limits and had a declared 20-second wall guard. Analysis, rendering and supervising-assistant use are additional to evaluation costs. See the results and paired data.
The assessment freeze
bound generation 2, the original seed, evaluator, paired contrasts and 256-case
sample before Stage 5. The primary contrast is mean
The Stage 5 protocol bound the earlier scientific freeze to a 30-minute execution/reporting ceiling and 1,200 total evaluator/candidate CPU-seconds. The 256 private cases were drawn after the freeze, excluding all previous private pools, including Stage 4, and the public seed range below 1,000,000. Both unchanged programs ran on every case. There were no new candidates, model calls, sample-size extensions or selection decisions using these results. The assessment pool is now retired from future tuning and fresh confirmation.
| Measurement | Original seed | Frozen generation 2 | Paired difference, 95% interval |
|---|---|---|---|
| Escape | 2/256 (0.78%) | 239/256 (93.36%) | +92.58 [89.06, 95.70] percentage points |
| Capture / timeout / invalid | 154 / 100 / 0 | 16 / 1 / 0 | — |
| Mean fitness |
0.500541 | 0.903113 | +0.402573 [0.383537, 0.420061] |
| Mean task |
0.179711 | 0.850380 | +0.670669 [0.638845, 0.699953] |
| Mean reconstruction |
0.981785 | 0.982214 | +0.000429 [−0.001346, 0.002210] |
| Mean final map coverage | 35.72% | 79.78% | +44.05 [41.16, 46.79] percentage points |
| Mean keys | 0.53125 | 1.9375 | +1.40625 [1.3125, 1.4961] |
| Opened door | 2/256 | 239/256 | +92.58 [89.06, 95.70] percentage points |
| Mean episode length | 133.65 | 56.03 | −77.63 [−86.80, −68.31] transitions |
The primary fitness interval lies entirely above the prespecified meaningful difference of 0.05. Its gain decomposes into +0.402401 from task reward and +0.000172 from reconstruction. Reported-cell accuracy is similar, while the evolved program covers substantially more of the map. Accuracy is measured along each policy's own trajectory; this is not a matched-experience prediction test. Episode length includes captures and timeouts and is not a comparison of escape speed conditional on success.
Generation 2 escaped on both cases where the seed escaped, and on 237 additional cases. Separate-arm 95% Wilson escape intervals are 89.62–95.81% for generation 2 and 0.21–2.80% for the seed. All paired intervals use 10,000 whole-case bootstrap draws with the frozen RNG seed, sharing each draw across programs and metrics. Secondary intervals are unadjusted. The zero-width invalid-execution interval reflects no observed invalid runs, not a guarantee of population reliability.
Figure 11. Fresh assessment effects. All 256 pairs contribute. The orange
tick marks the prespecified meaningful difference for the sole primary endpoint,
Figure 12. Every assessment case. Generation 2 had 16 captures and one timeout; all remain in the analysis. Ordinal order has no temporal meaning, and private environment seeds are not exported. The original seed's two escapes are highlighted. The dashed line is the mean paired fitness gain.
The complete assessment executed 512 valid episodes / 48,558 transitions in 64.06 seconds wall time and 62.61 total CPU-seconds, including controller overhead. Summed episode evaluator/candidate CPU was 61.33 seconds, the quantity shown in Figure 12. Saved-data analysis, rendering and supervising-assistant usage are outside these measurements. Sources, all public episode records, paired CSV, uncertainty and figure provenance are in the Stage 5 results. Recovery verified these previously completed results without rerunning episodes.
The next experiment was frozen in commit 7bc6949 before drawing another
256 paired private cases. It compares unchanged generation 2 with the historical
handcrafted memory/pathfinding agent and two source-specific interventions.
The comparator retains fixed local enemy avoidance; its adapter adds the current-map
export and repairs key-underfoot interaction and diagonal closed-door entry.
The local-map intervention removes spatial maps outside the current 5×5 view,
retaining localization, inventory and visitation history. The no-risk intervention
removes the hardcoded next-enemy-move probability calculation while retaining
occupied-cell exclusion. Neither intervention disables all memory or isolates
learned dynamics. All three controls were checked on the existing 32 development
cases before freezing; there were no subsequent outcome-driven edits.
| Condition | Escapes | Capture / timeout / invalid | Mean |
Mean |
Final coverage |
|---|---|---|---|---|---|
| Generation 2 | 243/256 | 12 / 1 / 0 | 0.911277 | 0.981869 | 80.47% |
| Handcrafted memory pathfinder | 230/256 | 26 / 0 / 0 | 0.888316 | 0.980385 | 82.35% |
| Generation 2, current local map only | 134/256 | 18 / 104 / 0 | 0.800794 | 1.000000 | 9.64% |
| Generation 2, no next-step risk | 154/256 | 102 / 0 / 0 | 0.743233 | 0.985432 | 70.30% |
Three primary comparisons were specified: original
Access to remembered spatial maps increases escape by 42.58 [34.38, 50.78] percentage points relative to the local-map intervention; using next-step risk increases it by 34.77 [26.95, 42.58] points relative to no risk. Both family-adjusted intervals exclude zero, supported by exact paired binary tests. These are causal contrasts for the specified source changes on this task distribution. They do not show that Shinka reliably discovers these mechanisms or learns enemy dynamics. The local-only map achieves perfect reported-cell accuracy while escaping less often and covering far less terrain, illustrating the objective's denominator limitation.
Figure 13. Competent control and mechanism assessment. All 1,024 episodes are included. Thick paired intervals are marginal 95%; thin intervals adjust for the three primary endpoints. The orange mark is the 0.05 meaningful fitness gain for the pathfinder comparison. Arm-level escape intervals use Wilson intervals. See the protocol, all paired results and control provenance. Execution used 79,582 transitions, 160.95 measured elapsed seconds and 164.28 total CPU-seconds, with zero experiment-model calls. Timing follows the saved controller fields; analysis and supervision are additional.
A separate passive audit replayed the already exposed 79-frame generation-2 example, without new world transitions. Removing its stale-cell reporting withdrawal added 46 reported frame-cell claims, only four correct. Accuracy fell from 97.8163% to 97.4401%, while all 79 actions stayed identical. This is a reporting-denominator effect on one historical trajectory, not a population estimate or evidence of better control.
The frozen broader-study design compares three independent full-native searches with three original-seed rewrite searches. Each has a maximum of 24 provider calls across all roles, 5,400 provider seconds and two hours. Full-native search retains ancestry, islands, inspirations, two-model bandit selection, local embedding novelty and meta/prompt evolution. The rewrite arm draws independent full programs from the original seed with alternating models and no auxiliary calls. This compares whole search strategies; common ceilings do not imply equal realized computation. After all six searches, 64 fresh shared cases select frozen nominees and 512 further cases assess winners. Neither fresh pool has been drawn.
The first full-native repetition (full-r1) is complete. It evaluated the seed
and seven valid descendants on the same five public training cases: 40 episodes,
2,869 transitions, zero invalid executions. The highest training fitness belongs
to generation 5: F=0.943888, S=0.90648, A=1.000000, 5/5 escapes, with 17.07%
final reported-map coverage. The seed had F=0.536362 and 0/5 escapes. These are
training results under adaptive search; they do not establish generalization or
search-strategy superiority. Perfect reported-cell accuracy with sparse coverage
again illustrates the reconstruction objective's limitation.
Figure 14. First independent full-native search checkpoint. The incumbent curve uses the all-role call count when each proposal was accepted for evaluation. The right panel counts every dispatched provider request, including one timeout. All four islands were sampled, one migration occurred, four samples used inspirations and four used recommendations. Both mutation models were used; local embeddings, four novelty-judge calls and an evolved prompt were observed. These events establish mechanism execution, not their causal benefit.
The run stopped normally at 24 calls: nine mutation, four novelty, eight summary, one global-insight, one recommendation and one prompt-mutation request. One mutation request timed out; its usage is unknown and remains outside the reported token lower bound. All calls required subscription billing and passed the raw context audit; 23 returned reported subscription usage. Local CPU embeddings made nine calls separately. Execution consumed 44.93 measured wall-minutes, 43.38 provider-minutes and 1.10 GiB peak combined RSS. A final generated generation-8 proposal was held before its required novelty check because the call cap had been reached. It is preserved as unevaluated and cannot enter selection. Native lineage, sources, representative saved replays, resource and route audits are in the compact checkpoint. Native search guidance is exported as model-produced guidance, not validated scientific findings. A source inspection shows that sparse exported maps coexist with persistent internal terrain and enemy beliefs used by the planners. The risk propagation uses hardcoded transition assumptions, not learned dynamics. Generation 5 also includes conditional waiting near the exit. On these saved training cases its 0.0012 fitness advantage over generation 2 follows exactly from five additional mean steps, because the original reward assigns a positive step contribution. This arithmetic does not establish whether that waiting branch activated. One rewrite block was subsequently interrupted after four valid descendants; four planned searches and fresh selection/assessment remain unexecuted.
The first independent rewrite block was stopped during request6 after the user challenged allowance consumption and the clarity of scientific contributions. Four returned rewrites each escaped5/5training cases; generation4 has the best trainingF=0.943533. The seed and four descendants completed25episodes and1,849 transitions. Request3 hit the local480-second timeout; request6 was interrupted by the operator. This is an interrupted attempt, not a completed budget-matched comparison. Its measured runtime was30.20wall-minutes; summed request time was 29.46minutes. Programs, outcomes and audit and the operator checkpoint are preserved. No fresh selection/assessment or calibration followed.
The accountability review records the larger management problem: expensive model settings were not calibrated, diminishing returns were already evident, and headline experiment counts excluded supervising and delegated assistant work. Local metadata for 8October17:00 through9October13:00UTC shows two supervisor sessions and42subagent sessions with observed usage; both supervisors and16subagents used Ultra. Recorded uncached input plus output was3,342,956tokens for supervision and 7,903,680for subagents. These are partial local token records, not an authoritative weekly subscription meter. They cannot be converted to a weekly-limit percentage. No claim about productive versus unproductive tokens follows from the aggregate.
The seed accurately reconstructs the cells it reports, but rarely escapes. Its greedy planner can become blocked or pursue an unsuitable remembered target. The initial characterization establishes room for behavioral improvement. The subsequent search and 32-case paired check demonstrate improved task performance on development cases. The frozen-program assessment confirms a large gain over the original seed on fresh cases from the same distribution. Search, development, selection and assessment scores remain separate; only the five-case search panel supplies native search fitness.
The original accuracy term has specific limitations. An agent can obtain high accuracy by retaining easy or recently observed cells while covering little of the maze. Static terrain also contributes many targets. The 0.6/0.4 coefficients do not guarantee balanced selection pressure, and the task reward gives a positive contribution to elapsed steps. We disclose these properties while preserving the requested objective. High composite fitness must not be described as proof of understanding.
The two historical successful descendants are direct seed mutations with no inspirations. The new independent run uses accumulated ancestry and inspirations, but neither observation establishes their causal benefit. Persistent memory is used by their planners, but the contribution of mapping versus pathfinding versus risk heuristics is only partly isolated by the two specified interventions in section 4.6. Independent discovery reliability remains unmeasured. The known uniform enemy law is used as a heuristic; these results do not show learned transition dynamics.
The seed comparison is deliberately faithful to the original proposal, but its greedy/random planner is a weak control. The subsequent competent-control experiment does not establish a fitness advantage over the handcrafted pathfinder. The selection results also provide no clear basis for preferring one successful descendant's behavior over the other's, despite the required scalar ranking.
Use the existing environment created by bootstrap.sh. Run the focused correctness checks:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m unittest tests.test_proposalRe-evaluate the initial program into a new directory:
OPENBLAS_NUM_THREADS=1 .venv/bin/python proposal/evaluate.py \
--program_path proposal/initial.py \
--results_dir results/proposal-reproduction --episodes 32 --replay-firstRegenerate the README figures from the saved experiment, without executing agents:
OPENBLAS_NUM_THREADS=1 .venv/bin/python proposal/figures.pySee proposal/README.md for preparation and explicit native
launch commands. The launcher defaults to preparation; --run is required for
model calls. No historical campaign or assessment is resumed by these commands.
Rebuild the Stage 1 figures from the committed compact evidence, without the private runtime database or any new episodes/model calls:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.search_report --render-exportThe original bounded invocation was .venv/bin/python proposal/evolve_full.py --run.
Its saved failure gate and deadline prevent further model work; this is not an
instruction to restart it. A future evolutionary continuation requires a new
explicit resource grant and reconciliation of the held pre-source slot. The
run state records the exact boundary. The local native
dashboard can be reopened without model calls:
.venv/bin/python scripts/v3_webui.py \
--campaign results/proposal-full-native-01 --port 8001Rebuild the Stage 3 paired analysis and Chromatic Fields figures from saved data, without running episodes or models:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.development_reportReproduce the frozen generation-2 evaluation into a new directory:
OPENBLAS_NUM_THREADS=1 .venv/bin/python proposal/evaluate_full.py \
--campaign-manifest artifacts/proposal/stage3-development/protocol.json \
--program_path artifacts/proposal/full-native-01/programs/generation-002.py \
--results_dir results/proposal-stage3-reproduction \
--episodes 32 --seed-start 0 --record-first yesThese are reproduction commands, not automatic authorization for another stage.
Rebuild the Stage 4 analysis and figures from committed ordinal episode data:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.selection_reportThe exact Stage 4 execution command was:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.panel \
--protocol artifacts/proposal/stage4-selection/protocol.json \
--output results/proposal-stage4-selectionThat bounded invocation requires the locally preserved private pool and retains its original deadline; it is not a command to launch a fresh panel. The runner checkpoints each episode and refuses to silently retry an interrupted one. Public statistical reproduction needs neither the private pool nor new episodes.
Rebuild the Stage 5 assessment analysis and Chromatic Fields figures using only the committed public records:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.assessment_reportThe exact completed Stage 5 invocation was:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.assess \
--protocol artifacts/proposal/stage5-assessment/protocol.json \
--output results/proposal-stage5-assessmentIts deadline is closed. Preserve the private pool and checkpoints; do not redraw or rerun the assessment. The current checkpoint records recovery of the completed execution and the ongoing broader experimental study.
Rebuild the competent-control analysis and figure without new episodes or models:
OPENBLAS_NUM_THREADS=1 .venv/bin/python -m proposal.control_report- Sakana ShinkaDreamer proposal. Preserved original project description and example code.
- Lange, R.T., Imajuku, Y., and Cetin, E. ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution. Final ICLR 2026 paper; official implementation.