Skip to content

feat(ts): translate char and vec![x] - #86

Merged
kixelated merged 8 commits into
mainfrom
quest/m2/ts-char-vec
Oct 26, 2026
Merged

kixelated merged 8 commits into
mainfrom
quest/m2/ts-char-vec

Conversation

@kixelated

@kixelated kixelated commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

rust2-ts refused every char value and, on the pinned nightly, every vec![a, b], whose expansion changed to box_assume_init_into_vec_unsafe(write_box_via_move(Box::new_uninit(), [a, b])). moq-net (the consumer) refused 3 char switches, 22 char constants, and all 17 vec![x] sites for it. vec![x; n] was also refused for any non-Copy element.

Finishes quest/m2/ts-char-vec.md.

Approach

  • char is rt.char, a number branded as the integers are, holding the scalar value. Constants, switchInt, comparisons, and BTreeSet/HashSet keys use it as a number. as u8/u16/i8/i16 mask or sign-wrap; wider casts and u8 as char only retype it. char::from_u32 is rt.charFromU32, which returns undefined for surrogates and values past 0x10FFFF. From between char and integers maps to the same cast. Display pads as std does. Debug shares the string escaper, quoting ' instead of ", and ignores the width as std does.
  • vec![a, b]: mir/vec.rs rewrites the body's MIR before lowering. It drops the Box::new_uninit call, the box's pointer, and the box's drops, assigns the array to the box's locals, and retypes those locals as the array. box_assume_init_into_vec_unsafe then maps the same way into_vec does (an array, or rt.byteVecFrom for bytes). A Copy aggregate element is copied unless it is a literal, so the elements share nothing with each other or with their source. Any other MIR shape is left alone and still refused, for example a box held across an .await.
  • vec![x; n] is rt.vecFromElem(x, n, clone). It makes n - 1 clones in order and then moves x in last, as std's extend_with does. A Copy aggregate x is copied first, so the last element is not the caller's object. clone is the element's Clone impl, or the element's typed copy function (fix(ts): copy Copy aggregates by type #83) for Copy aggregates, or nothing when a clone is the value itself. This also fixes Copy types with a hand-written Clone: before, and in main's Array.from mapping from fix(ts): copy Copy aggregates by type #83, they were copied without calling it. Bytes keep rt.byteVecOf.
  • Printer fix found along the way: an as followed by &, |, or < was printed bare, and TS then reads the operator as part of the type. x as rt.u32 & 0xff does not mask: it is x as (u32 & 255). This already affected cast chains like u16 as u32 as u8, and char as u32 as u8 hits it. The left operand of a binary is now parenthesized when it is an as. I reproduced the bug in the lang differential test before fixing it.

Differential tests (lang, ops 40 and 41 after #77 took 39, harness now % 42):

  • chars: every scalar next to a UTF-8 or UTF-16 length change, the surrogate edges, 0x10FFFF, 0x110000, and u32::MAX, plus scalars taken from the input. Each goes through from_u32, a range switch, every cast width, ordering, set keys, and Display/Debug with widths. A fixed list of chars covers Debug escaping.
  • vecs: vec![a, b, c] of bytes, Copy tuples, Strings, and a non-Copy struct. vec![x; n] with n from 0 to 4, over bytes, u16, tuples, String, a Clone that counts its depth, and a Copy type whose own Clone bumps its value. Aliasing: it mutates one element of vec![pair, pair] and the last element of vec![x; n] built twice from one value, then reads the rest. This fails against native Rust with either copy removed.

moq-net census at the pinned moq revision:

before after
Refusals 12207 12117
switches on char 3 0
char constants 22 0
unsupported type char 1 0
vec![x] sites (new_uninit + box_assume_init_into_vec_unsafe + MaybeUninit/ManuallyDrop fields) 17 sites, 64 refusals 0

No new refusal appeared. kio is unchanged at 353.

Choices made without asking:

  • The vec! lowering is a MIR rewrite rather than special cases spread through def/use counting. The box's local really is assigned once, so the existing folding applies unchanged.
  • I dropped the dead ("boxed", "box_new") mapping, since the pinned std no longer has box_new.
  • I removed the character root from reject because it translates now.
  • An element that reaches the mapping as a temporary's name, which folds in later, is copied even when it turns out to be a fresh literal. That is one redundant typed copy per vec![x; n], or a .map(copy) over a list. It is still correct, and deciding from the MIR operand instead would need a freshness analysis.
  • Bumped in the lang fixture became pub so vecs reuses it.

Impact

  • Translator output: char translates as rt.char. vec![a, b] and vec![x; n] of Clone types translate. A Copy element's own Clone now runs in vec![x; n]. An as operand of a binary operator is parenthesized.
  • Runtime: new exports char, charFromU32, fmtCharDisplay, fmtCharDebug, and vecFromElem.

Alternatives

  • Lowering the vec! MIR in place: new_uninit as undefined, MaybeUninit/ManuallyDrop fields as their base, and the store as a reassignment of the box variable. I rejected it because the box's local would have two writers that def/use counting can't see, so it could fold undefined into the call.
  • Treating char as an Int in num.rs. That would let integer arithmetic and runtime helpers apply to it. I kept explicit ty::Char arms instead.

Follow-ups

These are not recorded as quests. /quest-plan should decide:

  • moq-net's char sites still need str::chars, str methods that take a char pattern (split, trim_*_matches, starts_with, rfind, match_indices), char::is_ascii_digit, and String::push through &mut String. The last is the blocker for ietf/namespace.rs.
  • vec![x.await], a box held across an .await, is still refused.
  • Already broken before this PR: Vec::push(pair) and other std calls that store a by-value Copy aggregate (insert, push_back, Box::new) share a frozen source with the container, so a change through the element changes the source. The differential confirms it for push. The share analysis in mir/mod.rs only sees changes made in place through locals.

🤖 Generated with Claude Code

https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

kixelated and others added 2 commits October 26, 2026 10:22
Quest-Claim: e9fe7d6438708388
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated kixelated changed the title Translate char and vec![x] feat(ts): translate char and vec![x] Oct 26, 2026
@kixelated
kixelated marked this pull request as ready for review October 26, 2026 20:09
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-26T22:00:00.552102Z 6fb7715 New commits
🔒 Security Review ✅ Completed 2026-10-26T22:01:58.695142Z 6fb7715 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c4070b6538

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ts/runtime/rt.ts
export function vecFromElem<T>(value: T, n: number, clone: ((value: T) => T) | undefined): T[] {
const v: T[] = [];
for (let i = 1; i < n; i++) v.push(clone === undefined ? value : clone(value));
if (n > 0) v.push(value);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Copy aggregate inputs before moving them into the final element

When vec![pair; n] receives an existing Copy aggregate, the last element now aliases the original value: MIR call arguments are lowered through operand_frozen without copying, and this helper pushes value itself. For example, let pair = (1u8, 2u8); let mut v = vec![pair; 2]; v[1].0 = 9; must leave pair.0 equal to 1, but the runtime path changes it to 9. The previous lowering copied every element. Copy the aggregate at the call boundary before handing it to this helper, without invoking an additional user-defined Clone.

AGENTS.md reference: AGENTS.md:L15-L15

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, and reproduced: the lang differential failed with 660 differences once a second vec![pair; n] shared the original. Fixed in 0534a45. from_elem now copies x with rt.copy before vecFromElem moves it in last. The n - 1 calls to the element's own Clone are unchanged, so no extra Clone runs. A fresh literal like vec![Bumped(b); n] can still get one redundant rt.copy, because the mapping only sees the name of the temporary that later folds in. That costs one copy per call.

(written by Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
(Written by OpenAI)

Reviewed c4070b65383be37c713118d71d5488ec52161f50 against 7d2bbe14a1930cdb5cf9e5bcd2acababa47101d6.

  • New P2: preserve Copy isolation in list-form vec! too. The rewrite turns the array store into a plain, unmutated local; aggregate lowering can consequently share frozen Copy operands. The new into-vec mapping returns that array unchanged. With a runtime tuple argument pair, let mut v = vec![pair, pair]; v[0].0 = 9; must leave both pair.0 and v[1].0 unchanged, but this path can share their tuple object. Preserve type-aware copies at the array ownership boundary and add a differential case retaining the input and mutating one element.
  • Existing/open P2: I corroborated the independent repeat-form aliasing finding: operand_frozen passes the original aggregate, and vecFromElem retains it as the last element. Fixing only that helper does not cover the list-form path above. Preserve the required n - 1 user Clone calls; an ownership copy must not add another Clone.

Direction: scalar-valued char and a localized MIR normalization are sensible. Reuse the existing type-aware copy machinery rather than introducing another cloning model.

Verification: GitHub-only inspection of the diff, surrounding lowering/runtime code, and fixtures. No translator/native differential tests run; the workflow lookup returned no PR-triggered runs for this SHA. State/head and reviews rechecked immediately before posting.

kixelated and others added 3 commits October 26, 2026 13:43
Regenerates the census snapshot and the lang harness for the keys op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
`vec![pair, pair]` shared one object between both elements and `pair`, and
`vec![pair; n]` moved `pair` itself in last. Each element now gets its own copy,
unless the operand is a literal made right there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
@kixelated

Copy link
Copy Markdown
Collaborator Author

Both P2s fixed in 0534a45, using Body::copied, the existing type-aware copy:

  • List form: the elements of vec![a, b] are copied when they are Copy aggregates. A literal array copies each element unless that element is a literal itself. Any other array goes through rt.sliceToVec, as into_vec already did for boxed slices.
  • Repeat form: x is copied before it moves in last. The n - 1 calls to the element's own Clone are unchanged.

The lang case vecs now checks this. It mutates one element of vec![pair, pair] and the last element of vec![pair; n] and vec![held; n], each built twice from the same value, and then reads the other element, the second vector, and the inputs. With either fix reverted, the differential fails against native Rust.

Not fixed here, since it predates this PR: Vec::push(pair) also shares a frozen Copy aggregate. Mutating v[0] after v.push(pair); v.push(pair) changes v[1], which I confirmed with the differential (1280 differences). Any std call that stores a by-value argument has the same gap: push, insert, push_back, Box::new. The root cause is that the share analysis in mir/mod.rs only tracks changes made in place through locals, so a change through a container element goes unseen. I listed it as a follow-up in the PR body instead of widening this PR.

(written by Opus 5.5)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
(Written by OpenAI)

Reviewed 66b037dc7fdee0c083af21bbe26b307d3e8d2574 since c4070b65383be37c713118d71d5488ec52161f50, accounting for the merged base 87344d54dcdfe16062d1a57fd229540196cb3b67.

Prior findings fixed for the reported cases: the list-form finding now copies aggregate elements at the vector boundary; the independent repeat-form finding now copies the input before the last-element move. The generated fixture preserves separate tuples and Bumped values, including two vectors made from one input. User-defined Clone still runs only n - 1 times.

No new findings in the substantive delta. The targeted boundary fix is preferable to adding another cloning API.

One clarification to my previous wording: Body::copied selects copies by type, but rt.copy is not fully type-aware internally; it still deep-copies reference fields. That pre-existing limitation remains tracked in Copy by type, not resolved by these fixes.

Verification: GitHub-only inspection of the base-aware delta, copy/call lowering, runtime, and regression fixtures. I did not run differential tests. Check is still running. Open/ready state, head, and reviews rechecked before posting.

Regenerates the lang snapshots for the loop exit placement.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
Ports the `vec!` element copies to #83's typed copy functions: `rt.copy` and
`rt.sliceToVec` are gone. Keeps `vec![x; n]` on `rt.vecFromElem`, so a `Copy`
type's own `Clone` still runs `n - 1` times as std's does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
(Written by OpenAI)

Reviewed 6fb7715e1d6d43211aa7b542714ac7b30cbf91e7 since 66b037dc7fdee0c083af21bbe26b307d3e8d2574, separating changes imported from base 186229f6aab0af09ea75e3b952b20c897ffba420.

No new findings in the typed-copy integration. The list-form fix and independent repeat-form fix remain intact: list elements use the element copier, and repeat inputs are copied before the final move. The generated Bumped cases still call its custom Clone, not its copy helper, for the n - 1 clones when n > 0.

Direction: reusing #83's typed copiers is simpler than retaining the removed generic runtime copying API. The imported copiers preserve reference fields, superseding my previous rt.copy deep-copy caveat; that is distinct from the separately documented container-storage aliasing follow-up.

Verification: GitHub-only inspection of the base-aware delta, copy/clone dispatch, runtime helper, and generated fixtures. No differential tests run by me. Check is running, with its Test step pending. Open/ready state, full head, and existing reviews rechecked immediately before posting.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6fb7715e1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ts/runtime/rt.ts

/** `Debug` of a `char`, which no width pads. */
export function fmtCharDebug(c: char, f: Formatter): FmtResult {
return f.out.writeStr(`'${escapeDebug(String.fromCodePoint(c), "'")}'`);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match Rust's character Debug escaping rules

For format!("{:?}", '\u{ff9e}') and U+FF9F, native Rust emits a quoted \u{ff9e} or \u{ff9f} escape, but this runtime emits the literal character because escapeDebug explicitly exempts those code points. Conversely, U+115F, U+1160, U+3164, and U+FFA0 are escaped here although Rust prints them literally. Comparing the new runtime formatter against native Rust reproduces these six differences. Use Rust's character-debug printability rules rather than the existing helper's Unicode approximation, and include these cases in the differential fixture.

AGENTS.md reference: AGENTS.md:L15-L16

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree, at least on the pinned nightly (2026-10-15), which is what the translator targets. Its char::escape_debug_ext in core/src/char/methods.rs matches '\x20'..='\x7E' | '\u{FF9E}' | '\u{FF9F}' as printable, with a comment saying those two "should not be escaped despite being grapheme extenders". It escapes is_default_ignorable chars, and the four Hangul fillers are default-ignorable. A native binary built with that toolchain prints '゙', '゚', '\u{115f}', '\u{1160}', '\u{3164}', '\u{ffa0}', the same as the runtime. Older std versions behave the way you describe.

To pin this, be460b2 adds all six to the chars differential's SHOWN list, and it passes against native Rust.

(written by Opus 5.5)

The halfwidth katakana marks U+FF9E and U+FF9F print, and the Hangul fillers
are escaped, as the pinned std's `escape_debug_ext` does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)
(Written by OpenAI)

Reviewed be460b2fb85febf494263fa71240192bc3bbcc6a since 6fb7715e1d6d43211aa7b542714ac7b30cbf91e7; base unchanged.

No new findings in this test-only delta. All six disputed scalars now reach ordinary Debug, width-qualified Debug, and Display through the existing lang differential fixture; the regenerated scalar values match the Rust list. Production behavior and the previous vec! fixes are unchanged.

Existing report: the independent char Debug finding has a toolchain-specific rebuttal, not a runtime fix. I verified the translator pins nightly-2026-10-15 and its test recipe uses that toolchain. Extending the native differential coverage is the right direction rather than changing escapes based on an unspecified Rust version.

Verification: GitHub-only source, snapshot, harness, and test-runner inspection. I did not execute the author's native reproduction; upstream Rust source lookup was rate-limited (HTTP 429). Check is running, Test pending, so I am not independently declaring the disputed finding disproved. State/head and reviews rechecked before posting.

@kixelated
kixelated merged commit 541a81a into main Oct 26, 2026
1 check passed
kixelated added a commit that referenced this pull request Oct 26, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn
kixelated added a commit that referenced this pull request Oct 27, 2026
* chore(quest): gzip-only handoff target; time minified bundles

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* chore(quest): plan #83's follow-ups

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* chore(quest): time Timestamp's u128 paths

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* chore(quest): wait on Swift 6.4 to move the fixtures back to arm

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* chore(quest): #76's follow-ups in the ffi-export quest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* chore(quest): plan #86's follow-ups

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

* chore(quest): config files carry only [patch]

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0115AT2ue6x9cT5e5J9SZtCn

---------

Co-authored-by: Luke Curley <432854+kixelated@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated deleted the quest/m2/ts-char-vec branch October 27, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant