perf: the read miss front takes the receiver as the fused test holds it (D4) - #11658
Merged
Merged
Conversation
added 15 commits
September 28, 2026 01:45
…a pointer compare A canonical key list stored whichever string its first grower passed, and a read site holds its module's pooled literal: two objects with the same bytes. Every key match against a shape's list therefore fell through to a byte compare, including the megamorphic read's confirm of its slot guess. Pool literals of at most 64 bytes are now minted as ATOMS at module init (js_string_pool_atom): the one string object for that text in the agent, shared by every module's pool. The intern cache's miss paths hand out the atom for its text, and canonical lists write the atom of every key they store (Appended::atomized on extend_slot's write paths and canonicalize's copy). The trie still validates edges by bytes, so which object a list holds never changes which node a probe reaches. The atom table is per agent, bounded by program text, strong (every atom is also a registered pool handle's value) and rewritten on move by the intern table root scanner. A pointer match proves equal text; a mismatch proves nothing (a list written before its atom existed), so every consumer keeps its byte fallback. The megamorphic shape answer now scans for identity before it compares any bytes.
…ceiver's key list first A site latched megamorphic sends every read that misses its compact word to js_object_get_field_ic_slow, which answered it from the receiver's shape only after decoding the word, classifying the receiver and scanning the key list. The site may hold one thing: a slot guess (the compact word's high half, the slot the receiver's shape answered last), which the receiver's own shape confirms or refutes. The slow entry now asks that first, and only at a latched site, so a site that can still be primed is primed as before: the receiver's ShapeId names its record; the record's POSITION BOUND says logical key position `guess` is inline slot `guess`; the key at that position must be this key (one pointer compare, S3b atoms); then the receiver's slot is the answer. Anything else continues down the unchanged path. Nothing is emitted at the site, so code size is unchanged. Whether a shape can answer by position is a FACT OF THE RECORD, stored in bit 15 of flags_and_kind (RECORD_POSITIONAL, in the pairwise-disjointness assert): an Ordinary, generation-0, hole-free shape with a keys array and no ACCESSOR key in its attribute summary. It is written by refresh_positional wherever an input can change (construction, with_summary, slab insert, the in-place stable-tombstone update), read with one load on the megamorphic path, and debug builds assert it against its definition on every read. The bound is then min(key count, live inline slots). A test walks every minted record of the agent and fails if the bit and its definition disagree (sabotage: dropping the slab-insert refresh fails it, 8 of 68 records). The in-place updaters only accept a private-epoch record (nonzero generation, never positional), so their refreshes cannot flip the bit today; a second test drives both updaters to zero holes and asserts that premise, so it is where those refreshes start to matter if it ever changes. Logical position i is read past the keys array's front offset (array_elements_ptr), so a shifted keys array is answered correctly. The confirm reads the record through a thread-local mirror of the ordinary page directory (pointer and length, republished whenever the slab's `pages` change, cleared before the slab is dropped): one thread-pointer-relative load and two directory loads, no runtime-state resolution. The step runs in the slow entry's frameless head; the rest of the entry moved out of line. The mirror has a per_thread verdict in thread_exit_address_globals.json.
Minting atoms through the intern cache flagged every pool literal GC_FLAG_INTERNED, which silently admitted literal keys to the interned-only own-property lanes (read lane, set fast paths, chain store, proxy put). On Zod the widened read lane misses for inherited keys: keys_find_slot_by_key_ptr 5014 -> 8022 calls, +0.3%. Atoms are now plain allocations, and the intern cache neither adopts nor hands them out.
… SSO unbox inventory atomized() replaced heap-string key slots with their atom and left every other slot alone. That was correct for short (SSO) strings, whose bits are their identity, but only implicitly, so the SSO unbox inventory (#11627) counted it as a new heap-only string reader. The SSO arm is now explicit.
Keep both sides of string/intern.rs: the atom table beside #11634's young-only intern log. The atom table holds strong heap pointers, so it gets its own young log (arm-before-publish in place(), cleared on rehash, debug-asserted) and is scanned in both minor and full passes.
The megamorphic read asks a receiver's shape record whether key position `guess` is inline slot `guess`. #11633 answered with bit 15 of flags_and_kind plus `min(logical_key_count, live_inline_slot_count)` on every ask. POSBOUND stores the answer: `position_bound: u32` at offset 40, 0 when the shape cannot answer by position, else the min. It replaces bit 15 (reserved again), is rewritten by `refresh_positional` wherever an input changes, and debug builds assert it against its definition on every read. The census test now compares the stored bound with the definition. The record grows 40 -> 48 bytes (4 bytes of tail padding). The slab's fast lookup takes the ordinary directory mirror's address (`ordinary_record_in`), so a caller that already holds it reads no thread-local.
A generic property read keeps only the ShapeId compare and the slot load inline. The compare's false edge makes one plain call to the GC-leaf js_object_get_field_ic_front(dir, handle, key_bits, cache_slot, packed), tests its answer against TAG_HOLE and, only on a decline, branches to the unchanged collecting js_object_get_field_ic_slow. Receiver-validation failures skip the front. --typed-feedback builds keep the old edge. The front (read_confirm.rs) answers from shape facts only, in order: - a polymorphic way (PIC_ID_TOKEN_BIT | ShapeId, slot); - a spill entry: the compact word holds the ShapeId flipped by PACKED_SPILL_FLIP, and the un-flipped id must be a real ShapeId; - a latched megamorphic site (D3): the slot guess in the compact word's high half, confirmed by the receiver's shape record (guess < POSBOUND and one key-atom word compare); a wrong guess gets one bounded scan of the first 32 positional keys, and the found position re-aims the site word unless it holds a stamp (D3b). It allocates, collects, locks, throws and calls nothing, so it is Leaf in the call-effects tables and nothing is spilled or relocated across it. The slow entry asks the inherited-read cache for a never-primed site, then runs the miss body. The directory operand is PERRY_AGENT_PTRS slot 0, which is never null (statically PERRY_EMPTY_SHAPE_DIR until the slab publishes its mirror): one initial-exec load on ELF executables, the TEB TLS array plus the runtime's PERRY_AGENT_PTRS_SECREL on Windows x86-64, the HotTls TSD read on Apple aarch64, and the perry_shape_dir_cell leaf accessor elsewhere (x86-64 Darwin, ELF dylib/staticlib outputs, wasm). A `length` site passes the empty directory. Absent directory pages and chunks are shared all-EMPTY statics, so the walk has no null tests. tsc: -0.40% instructions, .text -9.0% (127.28 -> 115.81 MB), RSS -1.2%; lead_mega1 213.1 -> 164.3 instr/iter, lead_poly4 at base.
…it (D4) First-read D4 (polymorphic ways). The ways stay site-owned and the GC-leaf miss front answers them; the inline site stays one ShapeId compare and one load. What a way hit paid beyond the front itself was the call edge, and the largest avoidable part of it was the receiver operand: the site passed the payload, which LLVM folds from `biased + floor` back into `bits - POINTER_TAG`, a 10-byte movabs, an add and a move. The front now takes `payload - RECEIVER_HANDLE_FLOOR`, exactly the fused receiver test's biased value (already in a register on the miss edge), and folds the floor into its own load displacements. A `length` site, which has no fused test, subtracts the floor itself. RECEIVER_HANDLE_FLOOR moves to perry-abi; codegen's HANDLE_FLOOR and the runtime's HANDLE_BAND_MAX are pinned to it. Q1: `pic_prime_get` debug-asserts that an overflow-encoded slot never cascades into a way, the fact that lets the front answer a way with a plain inline load and no spill re-test. lead_poly4 132.00 -> 129.74 instr/iter (-3 per way hit), lead_mega1 164.31 -> 161.49, lead_mega 189.06 -> 186.24, lead_lit 108.99 unchanged.
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: ⛔ Files ignored due to path filters (4)
📒 Files selected for processing (25)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
added 14 commits
September 29, 2026 07:14
main brings #11633 (squashed as 10ece99), step 5 P0/P1 (#11652, the record rep word) and step 4b (#11650). The shape record combines both growths: position_bound (POSBOUND) at offset 40, rep at 48, 56 bytes; the layout asserts are rebased to that. rep is not a POSBOUND input, and a rep-typed shape carries its own bound (tested).
main's stack guard (#10812) matches AgentPtrAccess, which this branch extended with WindowsTeb: the runtime publishes no stack limit on Windows, so no check is emitted there, as before. The census authority surfaces and the reallocating-chunk sabotage now name the Slot-based slab (ChunkCells, PageSlots, Page = Slot<PageSlots>) this branch introduced.
# Conflicts: # crates/perry-runtime/src/object/shapes_store.rs
… now-safe unsafe in the posbound test
# Conflicts: # crates/perry-abi/src/lib.rs # crates/perry-codegen/src/runtime_decls/objects.rs # crates/perry-runtime/src/agent_ptrs.rs
# Conflicts: # crates/perry-abi/src/lib.rs
… confirm and asserts POSBOUND
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #11657 (the D3 miss front). Retarget to main after #11657 merges.
What
The read miss front now takes the receiver the way the site's fused receiver test already holds it in a register (payload minus the handle floor). The site pays one register move instead of a 10-byte
movabs+add+mov, and the front folds the floor into its load offsets. That saves 3 instructions on every front call, way hits and latched megamorphic reads alike.RECEIVER_HANDLE_FLOORin perry-abi. Codegen's copy and the runtime'sHANDLE_BAND_MAXare pinned to it, and the pairing is checked at compile time.pic_prime_getchecks that an overflow-encoded slot never goes into a way. Ways never hold a spill entry, which is why the front has no spill re-test.Measured and not shipped:
Results (Linux x86_64, base = #11657 c519ea9, outputs identical to node)
.textVerification
codegen 2309/0; runtime 4738/0 (serial); gc_call_effects regen --check linux identical; wasm ABI, sso, file size and fmt OK; gc-root-dominance with stale 2 ≤ 2. Sabotage: letting an overflow slot cascade into a way turns 4 tests red; dropping the floor in the front turns 2 red.