Repository navigation
feat(interp): a value shares its parts - #444
Conversation
The interpreter was a recursive tree walker: each call of the program used native stack (51 KB in a debug build, 9 KB in a release build), and the process aborted at a call depth of 161 (debug) or 907 (release) on a stack of 8 MB. Wasm reaches 16250 and gives a named trap. `eval` is now a loop with a stack of frames on the heap (owner decision 2026-11-10). An expression is lowered 1 time to code with shared parts (`Node`), so a frame can hold the part that runs next. A call uses no native stack: - 100000 calls deep give a value (0.14 s in a release build). - Calls with no end give the runtime error "call stack exhausted" at 1000000 frames, with exit code 1. The process does not abort. The behaviour of each form is the same, with the same order of evaluation and the same errors. An error leaves the environment as the recursive evaluator left it. A top-level function is lowered when its module is loaded, not copied at each call: `fib(27)` runs in 0.45 s, was 0.61 s. `Value::Closure` holds its body as `Code` (an opaque shared pointer), not as an `Expr`. No other crate reads that field. Not in this change: a `Value` still copies, prints and frees its parts with native recursion, so a program that builds a list of more than about 1000 elements can still abort. That is the next pull request. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u
A `Value` held its parts in boxes, and it copied, printed and freed them with native recursion. A list is a chain of 2 parts for each element, so a long list overflowed the native stack: the copy at 1000 (debug) to 6700 (release) elements, the print at 800 to 18000, and the free at 10000 to 64000. Each read of a variable copied the value, so a function on a list of n elements needed time and memory of the order n * n: a list of 2000 elements took 2.95 s and 3 GB. The parts of a value are now shared pointers (`Arc`): - A copy is 1 step, for each size of value. - `Drop for Value` frees the parts with a loop. - `Display` prints with a loop. `Debug` is the printed form. - `is_linear` uses a loop. `to_type` gives the type to 64 levels of parts, and the placeholder `()` below that. - A closure is `Value::Closure(Arc<Closure>)`. Measured in a release build: a list of 2000 elements takes 0.01 s and 12 MB; a list of 100000 elements takes 0.33 s and 179 MB; `fib(27)` takes 0.41 s (0.49 s before this change, 0.61 s with the recursive evaluator). With the evaluator of the pull request before this one, no step of the interpreter uses native recursion on the depth of the calls or on the size of a value. The recursion that remains is on the nesting of the source text (`lower`, the patterns of a `match`), as in the parser and in the type checker. No crate outside the interpreter builds or takes apart a value with parts. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 9 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
After an error, the environment stayed as it was at the error: the names of the call that failed stayed bound, and a region stayed active. The recursive evaluator did the same, but calls with no end are now an error that a caller can catch (the REPL), so the state is visible. An error now ends each frame, the top one first: the names, the environment and the regions are as they were before the evaluation. `Interpreter::call` also restores its environment after an error. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u
…ge-pass-ow72tj-values # Conflicts: # src/ephapax-interp/src/lib.rs
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u
…ge-pass-ow72tj-values
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u
A call in tail position found a frame on top that gives the environment back already. The call now adds no frame of its own, so a loop written as a tail call has no depth limit: 1200000 calls run in 15 MB. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u
…ge-pass-ow72tj-values # Conflicts: # src/ephapax-interp/src/lib.rs
…ass-ow72tj-values # Conflicts: # TEST-NEEDS.adoc # src/ephapax-interp/src/lib.rs
|
Codex Review Summary: ❌ FailedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
|
This security review couldn’t start because the paying account has exhausted its Codex allowance and has insufficient available credits. The payer can check their balance and add credits in the Codex usage dashboard, then try again. If you do not manage the paying account, contact this repository's admins. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fe317ca83
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Strings are linear resources | ||
| Value::String { .. } => return true, | ||
| // Pairs are linear if either component is linear | ||
| Value::Pair(l, r) => work.extend([&**l, &**r]), |
There was a problem hiding this comment.
When --unstable-core code repeatedly copies an unrestricted recursive value—for example, each iteration replacing a Tree with a node containing the two results of copy(tree)—the new Arc representation stores an O(n)-sized DAG, but this traversal pushes both aliases independently. Consequently, the linearity check before the next copy revisits the shared subtree along every logical path and grows exponentially (roughly 2^n), so a small valid program can stall even though this change makes the value itself cheap to copy. Track visited Arc allocations, or determine linearity from the declared type, so shared subgraphs are scanned once.
Useful? React with 👍 / 👎.
| Ok(Value::String { | ||
| data: d1 + &d2, | ||
| region: r1, | ||
| data: format!("{d1}{d2}"), |
There was a problem hiding this comment.
When a program builds a string through repeated appends, format!("{d1}{d2}") allocates a fresh buffer and recopies the entire accumulated left operand on every iteration. The previous d1 + &d2 consumed and extended the left String, allowing its spare capacity to be reused, so this change turns common incremental concatenation from amortized linear work into quadratic copying and increases peak memory. Extract the owned left string from left_val and append to it rather than formatting both borrowed operands into a new allocation.
Useful? React with 👍 / 👎.
Summary
Pull request 2 of 2 for the interpreter. Pull request 1, hyperpolymath/ephapax#443 (the evaluator with its own stack), is merged, and the base of this one is
main.A
Valueheld its parts in boxes, and it copied, printed and freed them with native recursion. A list is a chain of 2 parts for each element, so a long list overflowed the native stack:Each read of a variable copied the value. So a function on a list of n elements needed time and memory of the order n * n.
Changes
All in
src/ephapax-interp/src/lib.rs.Arc):Pair,Left,Right,Borrow,Tuple,List, andClosure(Arc<Closure>). A copy is 1 step.Drop for Valuefrees the parts with a loop.Displayprints with a loop.Debugis the printed form.is_linearuses a loop.to_typegives the type to 64 levels of parts, and the placeholder()below that. The REPL uses it to show a type.Value::pair,Value::left,Value::right,Value::borrow.No crate outside the interpreter builds or takes apart a value with parts.
Measurements
Release build:
sum(100000)fib(27)After this change, no step of the interpreter uses native recursion on the depth of the calls or on the size of a value. The recursion that remains is on the nesting of the source text (
lower, the patterns of amatch), as in the parser and in the type checker.Test plan
a_long_list_value_uses_no_native_stack: a list value of 1000000 elements is copied, printed, typed and freed on the 2 MB stack of a test thread.a_value_prints_each_of_its_parts: the printed form of each kind of value.the_interpreter_runs_a_long_list(recursive_types.rs): a program builds a list of 100000 elements and gets its length.cargo test --workspaceafter the merge ofmain: 650 pass, 0 fail.cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warningsare clean.run --interpon each of the 149.ephfiles of the repository gives the same output and exit code as before the two pull requests.🤖 Generated with Claude Code
https://claude.ai/code/session_0136P7ZmKrYLDC4WHi2yt11u