perf: GC scope contexts and captured-binding fixes (#10500, #10703, #10520)

changelog.d/11710-scope-context-objects.md

@@ -0,0 +1,17 @@
+Captured-and-mutated bindings now live in **scope context objects** (V8 `Context` / SpiderMonkey environment objects) instead of one GC cell per binding. The bindings of one statement list that the same closures capture share one movable `GC_TYPE_SCOPE` arena object with one NaN-boxed slot per binding. The defining frame keeps ONE GC root per object, and a capturing closure keeps ONE capture slot per object. Under RS4GC statepoints the relocation work is (#live GC pointers) × (#safepoints), so on a module factory with hundreds of boxed bindings this is the difference between hundreds of roots and a handful. The per-binding box registries, `CLOSURE_BOX_CELLS` and `BOX_CAPTURE_COUNTS` stay deleted (the scope-context change removes them).
+
+- **Where groups come from.** `perry_codegen::scope_env::group_scope_boxes` runs after the async/generator transforms. It groups each body's eligible bindings by (home statement list, TDZ seeding, set of capturing closures) and writes each group as one `PreallocateBoxes`/`PreallocateTdzBoxes` statement before the group's earliest home. Codegen allocates one scope object per such statement (`js_scope_alloc`) and validates dominance again (`ScopeMap::build`), so HIR that never went through the pass is still correct. A binding is eligible only when its first declaration dominates every reference and declaration of it, and it appears in at most one preallocation statement. Anything else keeps its own `GC_TYPE_BOX` cell.
+- **Retention.** Arrows, function expressions and callbacks are each their own capture class, so a surviving closure never keeps alive a binding that only a dead sibling captured. A body's hoisted function declarations count as one class, because they are created together at scope entry. A statement list that would still need more than 16 objects gets one object per TDZ kind: that is a bundler's module-scale wrapper, whose closures live as long as the module.
+- **Per-iteration bindings.** A loop body's group is allocated inside the body, so every iteration gets a fresh object. Closures from earlier iterations keep their own bindings. Async activations get one object per capture class, which `ReleaseBoxes` ends with a single release call. The generation-token semantics are unchanged.
+- **Access.** A slot read is a plain load: no call, and so no safepoint. Only a TDZ-seeded group adds a compare with a cold call that throws. A write is a store plus the ordinary write barrier, with the object as parent. Closure bodies cache one rooted base per captured object and derive slot addresses from it. `add` is now a transparent derivation for the root-reload pass. Async i32/i1 control words share the object with a non-pointer tag in the high half.
+- **Capture by value.** A top-level binding whose every write precedes every closure that captures it is no longer boxed: the closures snapshot its final value.
+- **Fixed on the way.** Array head write-backs for `push`/spread growth and numeric bulk-fill loops wrote the new head into a boxed binding's frame slot instead of its cell. The validated box reads hid this by answering `undefined`. The runtime-key `a[i] = v` path already routes through the cell on main.
+- **Runtime.** `GC_TYPE_SCOPE` (traced slot-by-slot, rewritten on evacuation, exact object starts recorded), `js_scope_alloc` / `js_scope_capture_base`. Thread transfer carries a captured scope object as `ScopeCapture`. Exact-arrow dispatch accepts a scope object where it accepted a box.
+
+Mapped sloppy-mode `arguments` parameters and captured-and-mutated parameters keep per-binding cells (parameters are not grouped). The #11506 mapping array (`ObjectMeta::arguments`) now holds each mapped parameter cell as a NaN-boxed pointer. The elements are stored through the array's element setter (layout note plus in-body barrier with the array as parent), so marking keeps the cell alive and a moving collection rewrites the element. `arguments[i] = v` re-derives the cell after the own-value write, which can allocate.
+
+Tests: six gap tests (`test_gap_gc_scope_{closures,loops,async,arguments,retention,splice}`), registered in the GC stress corpus. Each is byte-exact against Node 26.5.1 under default settings, under `PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1` with native and shadow roots, and under a rate-1 seeded schedule with from-space protection. Copying minors were asserted to have run. Codegen unit tests assert one root, one allocation and one capture slot per group, with a per-binding control arm. No version bump.
+
+This combined follow-up also fixes #10520's unnecessary cell for a never-reassigned closure-valued `const`: self-recursion analysis now requires the initializer itself to capture its binding. A codegen unit test checks both a captured `const` closure and an actual self-capturing initializer, and a gap fixture checks their behavior. Both `splice` lowering paths now use the scope-aware writeback helper so changing an array head cannot overwrite the scope object's root.
+
+For #10500, the current packed static-store path already accepts a pooled literal whose runtime-interned twin is a different string object. A regression test uses pooled `name`, `E`, `length` and `now` keys alongside control names and asserts that every overwrite primes the packed store cache. No new side table or version bump is included.

crates/perry-codegen/src/boxed_vars.rs

@@ -88,6 +88,11 @@ pub(crate) fn collect_boxed_param_ids(
     out
 }
 
+/// Ids named by this body's preallocation statements (closures excluded).
+pub(crate) fn collect_prealloc_box_ids_shallow(stmts: &[perry_hir::Stmt], out: &mut HashSet<u32>) {
+    collect_prealloc_box_ids_in_stmts(stmts, out);
+}
+
 fn collect_prealloc_box_ids_in_stmts(stmts: &[perry_hir::Stmt], out: &mut HashSet<u32>) {
     use perry_hir::Stmt;
     for s in stmts {
@@ -202,7 +207,7 @@ fn collect_boxed_vars_scope(stmts: &[perry_hir::Stmt]) -> HashSet<u32> {
     // the first read of fib from inside the body goes through
     // js_box_get which returns the real closure value.
     let mut self_recursive_ids: HashSet<u32> = HashSet::new();
-    collect_self_recursive_closure_ids(stmts, &closure_refs, &mut self_recursive_ids);
+    collect_self_recursive_closure_ids(stmts, &mut self_recursive_ids);
 
     // Box = (declared AND captured AND mutated) OR (self-recursive closure),
     // minus for-loop init vars.
@@ -216,12 +221,128 @@ fn collect_boxed_vars_scope(stmts: &[perry_hir::Stmt]) -> HashSet<u32> {
             continue;
         }
         if closure_refs.contains(id) && (closure_writes.contains(id) || outer_writes.contains(id)) {
+            // Capture by value when every write precedes every capturing
+            // closure: each closure then snapshots the binding's final value
+            // and no cell is needed (see `writes_all_precede_captures`).
+            if !closure_writes.contains(id) && writes_all_precede_captures(stmts, *id) {
+                continue;
+            }
             boxed.insert(*id);
         }
     }
     boxed
 }
 
+/// True when `id` is declared by exactly one `Stmt::Let` of these statements,
+/// directly in the body's top-level list, and every statement of that list
+/// that writes `id` (outside closures, including the declaration itself)
+/// comes strictly before every statement that creates a closure naming `id`,
+/// with no write or closure reference before the declaration.
+/// Then no write can happen after a capture — a loop around both is a loop
+/// around the declaration too, which makes a fresh binding per iteration — so
+/// snapshot capture is exact.
+fn writes_all_precede_captures(stmts: &[perry_hir::Stmt], id: u32) -> bool {
+    use perry_hir::Stmt;
+    fn find_home<'a>(stmts: &'a [Stmt], id: u32, out: &mut Vec<(&'a [Stmt], usize)>) {
+        for (i, s) in stmts.iter().enumerate() {
+            if matches!(s, Stmt::Let { id: d, .. } if *d == id) {
+                out.push((stmts, i));
+            }
+            match s {
+                Stmt::If {
+                    then_branch,
+                    else_branch,
+                    ..
+                } => {
+                    find_home(then_branch, id, out);
+                    if let Some(e) = else_branch {
+                        find_home(e, id, out);
+                    }
+                }
+                Stmt::While { body, .. } | Stmt::DoWhile { body, .. } => find_home(body, id, out),
+                Stmt::For { init, body, .. } => {
+                    if matches!(init.as_deref(), Some(Stmt::Let { id: d, .. }) if *d == id) {
+                        // A `for` head binding: never the snapshot case here.
+                        out.push((&[], 0));
+                        out.push((&[], 0));
+                    }
+                    find_home(body, id, out);
+                }
+                Stmt::Try {
+                    body,
+                    catch,
+                    finally,
+                } => {
+                    find_home(body, id, out);
+                    if let Some(c) = catch {
+                        find_home(&c.body, id, out);
+                    }
+                    if let Some(f) = finally {
+                        find_home(f, id, out);
+                    }
+                }
+                Stmt::Switch { cases, .. } => {
+                    for c in cases {
+                        find_home(&c.body, id, out);
+                    }
+                }
+                Stmt::Labeled { body, .. } => {
+                    find_home(std::slice::from_ref(body.as_ref()), id, out)
+                }
+                _ => {}
+            }
+        }
+    }
+    let mut homes = Vec::new();
+    find_home(stmts, id, &mut homes);
+    let [(list, home)] = homes.as_slice() else {
+        return false;
+    };
+    let (list, home) = (*list, *home);
+    // Every write and closure reference must sit in the home statement or a
+    // later sibling: compare the totals with what that range accounts for.
+    let count = |range: &[Stmt]| {
+        let mut writes = 0usize;
+        let mut captures = 0usize;
+        for s in range {
+            let mut w = HashSet::new();
+            collect_outer_writes_in_stmt(s, &mut w);
+            let mut r = HashSet::new();
+            let mut cw = HashSet::new();
+            collect_closure_refs_and_writes_in_stmt(s, &mut r, &mut cw);
+            writes += usize::from(w.contains(&id));
+            captures += usize::from(r.contains(&id));
+        }
+        (writes, captures)
+    };
+    let mut last_write = None;
+    let mut first_capture = None;
+    for (j, s) in list.iter().enumerate().skip(home) {
+        let (w, c) = count(std::slice::from_ref(s));
+        if w > 0 || (j == home && matches!(s, Stmt::Let { init: Some(_), .. })) {
+            last_write = Some(j);
+        }
+        if c > 0 && first_capture.is_none() {
+            first_capture = Some(j);
+        }
+    }
+    let (Some(last_write), Some(first_capture)) = (last_write, first_capture) else {
+        return false;
+    };
+    if last_write >= first_capture {
+        return false;
+    }
+    // Nothing before the home statement may write or capture the binding.
+    let (w_before, c_before) = count(&list[..home]);
+    if w_before > 0 || c_before > 0 {
+        return false;
+    }
+    // Only a declaration in the body's own top-level list qualifies: then
+    // there is no enclosing statement or sibling branch that could write or
+    // capture the binding outside the range checked above.
+    std::ptr::eq(list.as_ptr(), stmts.as_ptr())
+}
+
 /// Walk the given statements looking for `Expr::Closure` nodes, and
 /// for each one recursively run the boxing analysis on its body.
 /// Unions the resulting ids into `out`.
@@ -523,11 +644,7 @@ fn walk_for_self_capturing_closure(e: &perry_hir::Expr, let_id: u32, found: &mut
 /// When a Stmt::Let's Closure init captures the Let's own id, that id must
 /// be boxed so the closure body can read the live value instead of the
 /// stale 0.0 that was in the slot at capture time.
-fn collect_self_recursive_closure_ids(
-    stmts: &[perry_hir::Stmt],
-    closure_refs: &HashSet<u32>,
-    out: &mut HashSet<u32>,
-) {
+fn collect_self_recursive_closure_ids(stmts: &[perry_hir::Stmt], out: &mut HashSet<u32>) {
     use perry_hir::Stmt;
     for s in stmts {
         if let Stmt::Let {
@@ -553,14 +670,6 @@ fn collect_self_recursive_closure_ids(
             // closure reads.
             if init_expr_has_self_capturing_closure(init_expr, *id) {
                 out.insert(*id);
-            } else if matches!(init_expr, perry_hir::Expr::Closure { .. })
-                && closure_refs.contains(id)
-            {
-                // Pre-existing direct-closure-literal arm — kept as a
-                // belt-and-suspenders fallback in case the
-                // walk-the-init detection above misses an edge shape
-                // (e.g. a future HIR variant that holds a Closure).
-                out.insert(*id);
             }
         }
         // Recurse into nested blocks.
@@ -570,48 +679,43 @@ fn collect_self_recursive_closure_ids(
                 else_branch,
                 ..
             } => {
-                collect_self_recursive_closure_ids(then_branch, closure_refs, out);
+                collect_self_recursive_closure_ids(then_branch, out);
                 if let Some(eb) = else_branch {
-                    collect_self_recursive_closure_ids(eb, closure_refs, out);
+                    collect_self_recursive_closure_ids(eb, out);
                 }
             }
             Stmt::For { init, body, .. } => {
                 if let Some(init_stmt) = init {
                     collect_self_recursive_closure_ids(
                         std::slice::from_ref(init_stmt.as_ref()),
-                        closure_refs,
                         out,
                     );
                 }
-                collect_self_recursive_closure_ids(body, closure_refs, out);
+                collect_self_recursive_closure_ids(body, out);
             }
             Stmt::While { body, .. } | Stmt::DoWhile { body, .. } => {
-                collect_self_recursive_closure_ids(body, closure_refs, out);
+                collect_self_recursive_closure_ids(body, out);
             }
             Stmt::Try {
                 body,
                 catch,
                 finally,
             } => {
-                collect_self_recursive_closure_ids(body, closure_refs, out);
+                collect_self_recursive_closure_ids(body, out);
                 if let Some(c) = catch {
-                    collect_self_recursive_closure_ids(&c.body, closure_refs, out);
+                    collect_self_recursive_closure_ids(&c.body, out);
                 }
                 if let Some(f) = finally {
-                    collect_self_recursive_closure_ids(f, closure_refs, out);
+                    collect_self_recursive_closure_ids(f, out);
                 }
             }
             Stmt::Switch { cases, .. } => {
                 for case in cases {
-                    collect_self_recursive_closure_ids(&case.body, closure_refs, out);
+                    collect_self_recursive_closure_ids(&case.body, out);
                 }
             }
             Stmt::Labeled { body, .. } => {
-                collect_self_recursive_closure_ids(
-                    std::slice::from_ref(body.as_ref()),
-                    closure_refs,
-                    out,
-                );
+                collect_self_recursive_closure_ids(std::slice::from_ref(body.as_ref()), out);
             }
             _ => {}
         }
@@ -1666,7 +1770,54 @@ fn infer_refinable_type_without_context(init: &perry_hir::Expr) -> Option<perry_
 mod tests {
     use super::*;
     use perry_hir::types::Type;
-    use perry_hir::Expr;
+    use perry_hir::{Expr, Stmt};
+
+    fn closure(func_id: u32, captured_id: u32) -> Expr {
+        Expr::Closure {
+            func_id,
+            params: Vec::new(),
+            return_type: Type::Any,
+            body: vec![Stmt::Return(Some(Expr::LocalGet(captured_id)))],
+            captures: vec![captured_id],
+            mutable_captures: Vec::new(),
+            captures_this: false,
+            captures_new_target: false,
+            enclosing_class: None,
+            is_arrow: true,
+            is_async: false,
+            is_generator: false,
+            is_strict: false,
+        }
+    }
+
+    #[test]
+    fn captured_closure_value_is_boxed_only_for_real_self_capture() {
+        // #10520: the outer closure references `f`, but `f`'s initializer
+        // does not. Only `self_ref` captures itself before initialization.
+        let stmts = vec![
+            Stmt::Let {
+                id: 1,
+                name: "f".into(),
+                ty: Type::Any,
+                mutable: false,
+                init: Some(closure(10, 3)),
+            },
+            Stmt::Let {
+                id: 2,
+                name: "self_ref".into(),
+                ty: Type::Any,
+                mutable: false,
+                init: Some(closure(11, 2)),
+            },
+            Stmt::Expr(closure(12, 1)),
+        ];
+        let boxed = collect_boxed_vars(&stmts);
+        assert!(!boxed.contains(&1), "captured const closure needs no cell");
+        assert!(
+            boxed.contains(&2),
+            "a self-capturing initializer needs a cell"
+        );
+    }
 
     #[test]
     fn simple_refinement_uses_shared_hir_inference_for_constructed_values() {

crates/perry-codegen/src/codegen/arguments.rs

@@ -53,19 +53,17 @@ pub(crate) fn store_param_slot(
     let slot = blk.alloca(if boxed_param { I64 } else { DOUBLE });
     if boxed_param {
         let arg_bits = blk.bitcast_double_to_i64(arg_name);
-        let box_ptr = blk.call(I64, "js_box_alloc_bits", &[(I64, &arg_bits)]);
-        blk.store(I64, &box_ptr, &slot);
+        blk.store(I64, &arg_bits, &slot);
     } else {
         blk.store(DOUBLE, arg_name, &slot);
     }
     slot
 }
 
-/// #10464: a boxed parameter's cell is minted by this frame's entry block
-/// (`store_param_slot`), so the frame releases it before every `ret`.
-/// `materialize_arguments_object` withdraws a slot it maps into a sloppy-mode
-/// `arguments` object, which holds the raw cell without a counted edge.
-pub(crate) fn release_boxed_param_slots_at_exit(
+/// After every incoming parameter has a root slot, replace boxed parameters'
+/// value words with freshly allocated GC cells. An allocation can collect, so
+/// boxing during the initial spill loop would lose the later arguments.
+pub(crate) fn box_rooted_parameter_slots(
     lf: &mut crate::function::LlFunction,
     params: &[Param],
     boxed_vars: &HashSet<u32>,
@@ -76,19 +74,15 @@ pub(crate) fn release_boxed_param_slots_at_exit(
             continue;
         }
         if let Some(slot) = slots.get(&p.id) {
-            lf.add_pre_return_box_release(slot, "js_box_scope_release");
+            // All incoming arguments are rooted before the first allocation.
+            let blk = lf.block_mut(0).expect("parameter entry");
+            let bits = blk.load(I64, slot);
+            let cell = blk.call(I64, "js_box_alloc_bits", &[(I64, &bits)]);
+            blk.store(I64, &cell, slot);
         }
     }
 }
 
-/// The parameter ids a synthesized `arguments` object aliases.
-pub(crate) fn mapped_parameter_ids(params: &[Param]) -> HashSet<u32> {
-    mapped_arguments_params(params)
-        .into_iter()
-        .map(|(_, id)| id)
-        .collect()
-}
-
 pub(crate) fn materialize_arguments_object(
     ctx: &mut FnCtx<'_>,
     params: &[Param],
@@ -201,7 +195,6 @@ pub(crate) fn materialize_arguments_object(
     for (arg_index, param_id) in mapped {
         if let Some(param_slot) = ctx.locals.get(&param_id).cloned() {
             // #10464: the object aliases the cell for its own lifetime.
-            ctx.func.forget_pre_return_box_release(&param_slot);
             let box_ptr = ctx.block().load(I64, &param_slot);
             ctx.block().call_void(
                 "js_arguments_object_map_index",