Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quest/m3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,6 @@ generic std gap is planned here even when moq never calls it.
- [An inclusive range folded through an adaptor](/quest/m3/ts-range-fold.md) - a `RangeInclusive` folded through `map`, `filter`, or `rev` ends in core's state, and `by_ref` of one type-checks
- [Binding gaps](/quest/m3/ts-bind-gaps.md) - bound types as keys, unit returns checked, renamed dependencies
- [Drop order in std's folds and iterators](/quest/m3/ts-drop-order.md) - the values that drop still refused in std's folds, zips, flattens, and collects drop as std's do
- [Ordering in std](/quest/m3/ts-ordering.md) - `cmp::max`/`min`, `is_sorted`, and `BinaryHeap`, and #173's cost on integers measured
- [Slice windows and chunks, Vec drain, and the order of byte arrays](/quest/m3/ts-slice-gaps.md) - `windows`, `chunks`, `Vec::drain`, and byte arrays' order translate
- [Return a std iterator's next from a generic function](/quest/m3/ts-next-return.md) - `it.next()` returned from a generic function is an `Option`, as `tsc` expects
- [Port the workflow guard to kixelated/quest](/quest/m3/quest-workflow-guard.md) - quest's environment guard parses YAML as this repository's does
22 changes: 0 additions & 22 deletions quest/m3/ts-ordering.md

This file was deleted.

17 changes: 17 additions & 0 deletions quest/m3/ts-slice-gaps.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# [S] Slice windows and chunks, Vec drain, and the order of byte arrays

## Goal

rust2-ts translates a slice's `windows` and `chunks`, `Vec::drain` of a range, and `<`,
`<=`, `>`, `>=`, and `cmp` of byte arrays such as `[u8; 2]`, matching std's iteration,
drop order of drained values, and comparison results. Proven by differential tests
against native Rust.

## Plan

Met by [#252](https://github.com/moq-dev/rust2/pull/252) on 2026-10-29: its ordering
fixtures and bench rewrote around each. rust2 aims to support all of std. Each refuses
with a span today; the byte-array comparison refuses because the mapping of a byte array
it is generic over would take the array, not a slice of it.

Public API: the translator's output.
6 changes: 6 additions & 0 deletions quest/m3/ts-unsafe.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,5 +45,11 @@ none of its own `unsafe`.
range, and each integer's `from_le_bytes` and `from_ne_bytes`, and `chunks_vectored`
needs `std::io::IoSlice`: 184 of moq-net's 196 refusals in `bytes`, the rest being 12
of [`f32`](/quest/m3/ts-f32.md).
- `BinaryHeap` and `PeekMut` translate from std's MIR, whose `Hole` moves elements
through raw pointers into the `Vec`'s storage, replacing `rt.BinaryHeap` and
`rt.PeekMut`, the hand-written port of std's sifts that
[#252](https://github.com/moq-dev/rust2/pull/252) mapped them to and that must track
std's by hand. The lang fixture's `orders` differential case proves the translation
still matches std's comparisons, pop order, and drops.

Public API: the translator's output.
2 changes: 1 addition & 1 deletion ts/MIGRATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
`true`.
- The runtime's `iterMax`, `iterMin`, `iterMaxByKey`, and `iterMinByKey` take a
`PartialOrd::lt` after the iterator and any key function, for items or keys of any type:
`rt.iterMax(it, rt.cmpLt(rt.cmp))`.
`rt.iterMax(it, rt.cmpLt(rt.cmpStr))`, or of integers, `rt.iterMax(it, rt.lt)`.
- `rt.FromUtf8Error` holds its `rt.Utf8Error` as std's does: `e.validUpTo` and
`e.errorLen` are `e.error.validUpTo` and `e.error.errorLen`.
- `ts.toml` no longer takes `ignore-macros`: `tracing`'s events are left out without it,
Expand Down
26 changes: 19 additions & 7 deletions ts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ from. `rt.ts` beside them is the
runtime: what std does that JS does not, with iterators in `rt/iter.ts`, `str`'s searches
in `rt/str.ts`, collections in `rt/collections.ts`, ownership, atomics, and async in
`rt/task.ts`, `f64`'s methods in `rt/float.ts`, and `Duration` in `rt/time.ts`. What translates from std's own MIR,
`Option`'s, `Result`'s, and `Poll`'s methods, goes in a module under its crate's name, as
`Option`'s, `Result`'s, and `Poll`'s methods and `cmp::max` and `min`, goes in a module under its crate's name, as
`core/option.ts` holds `map$Option_u64_u64_rescale_closure0`. A dependency's modules go
under its crate's name too, `kio.ts` and `kio/lock.ts`, with the instances of its generic
functions that the package's types make, so a module at the package's root named for a
Expand All @@ -95,7 +95,8 @@ crate translation reaches is refused, and so are two versions of one crate it re
| an iterator | `rt.Iter<Item>`, whose adaptors are runtime functions: `rt.iterMap(rt.iterSlice(v), f)` |
| `a..b`, `a..=b` | `rt.Range<T>`, `rt.RangeInclusive<T>`, which are iterators too |
| `a..`, `..b`, `..=b`, `Bound<T>` | `{ start }`, `{ end }`, `{ end, inclusive: true }`; `rt.RangeBound<T>`, a union tagged by variant |
| `HashMap`, `HashSet`, `BTreeMap`, `BTreeSet`, `VecDeque` | `rt.HashMap<K, V>` and so on, each a plain object whose functions are the runtime's: `rt.hashGet(map, key)` |
| `HashMap`, `HashSet`, `BTreeMap`, `BTreeSet`, `VecDeque`, `BinaryHeap` | `rt.HashMap<K, V>` and so on, each a plain object whose functions are the runtime's: `rt.hashGet(map, key)` |
| `PeekMut<T>` | `rt.PeekMut<T>`, the `rt.Place` of a heap's greatest element, which sifts it down as it drops once `DerefMut` lent it |
| `Ordering`, `DefaultHasher` | `rt.Ordering`, which is `-1`, `0`, or `1`; `rt.Hasher` |
| `Duration` | `rt.Duration`: `{ secs, nanos }`, std's whole seconds as an `rt.u64` and the nanoseconds past them, never changed once made |
| closure | an arrow function, with `[Symbol.dispose]` if it owns a value that drops |
Expand Down Expand Up @@ -254,7 +255,8 @@ impl, an `Arc`'s count, or a waker. Bodies lower from MIR, so each drop is where
elaboration put it. A panic drops nothing. A value without `[Symbol.dispose]` of its own, such as
an enum or a tuple, drops through glue lowered from rustc's `drop_in_place` shim, a
function of its type's own (`drop$Held`). `bool`'s, `Option`'s, `Result`'s, and `Poll`'s
methods lower from core's MIR, so they drop what core's do, where core's do. The runtime
methods and `cmp::max`, `min`, and their `_by` and `_by_key` forms lower from core's MIR,
so they drop what core's do, where core's do. The runtime
functions for std's collections, `Arc`, and the std functions that may discard an operand
(`OnceCell::get_or_init`, a map's `insert`) take what drops each value they may drop, and
drop it where std's do. So do iterators: one that owns its items, as a `Vec`'s `into_iter` or a
Expand Down Expand Up @@ -527,6 +529,14 @@ they need.
calling only the comparator, as std permits of an unstable sort: JS's `sort` moves
`undefined`, a `None`, to the end without comparing it, and each engine orders its
comparisons differently.
- A `BinaryHeap` is std's heap in an array, with std's sifts ported to the runtime: std's
move elements through raw pointers into its `Vec`, which no body of std's lowers
through, and the sifts decide which `PartialOrd` tests run and which of equal elements
leaves first. The sorts, `max` and `min`, the heap, and `is_sorted` take an integer's
tests as the runtime's `rt.lt`, `rt.le`, `rt.gt`, and `rt.ge` rather than through its
`partial_cmp`, `rt.cmpLt(rt.cmp)`: on `just ts bench`'s `order/` workloads that took
Firefox 0.8-0.86x the time on sorts and `max`/`min` and Chrome 0.66x on the heap, and
Chrome the same on the rest.
- Iterators are the runtime's lazy classes rather than generators or ES iterator
helpers: std's adaptors would lower from std's own bodies, whose slice iterators are
raw pointers, ES helpers lack most of them
Expand Down Expand Up @@ -581,7 +591,8 @@ what it captured, `mem::swap`, `replace`, or `take` of a reference's whole point
assigning an `Arc`, a waker, a `MutexGuard`, a pinned reference, or a future through a
reference, a switch on or a field of another crate's type without a mapping, and std
functions without a mapping in `src/lower/library.rs`, but for `bool`'s, `Option`'s,
`Result`'s, and `Poll`'s methods that take no `&mut`, which lower from core's MIR.
`Result`'s, and `Poll`'s methods that take no `&mut`, and `cmp`'s `max` and `min` forms,
which lower from core's MIR.

## Tests

Expand Down Expand Up @@ -676,11 +687,12 @@ frame headers decoded as they arrive a few bytes at a time, which prices a decod
comes up `Short` and retries, iterator chains over bytes, a `BTreeMap` of 16, 256, and
4096 keys, and timestamps converted between timescales as moq-net's `Timestamp::convert`
does, whose products pass 2^53, then added by its `checked_add` and ordered by its `Ord`,
whose cross products do too. Each runs four ways: natively, as wasm32 at moq-wasm's
whose cross products do too, and integers sorted, searched for their greatest and least,
and kept in a `BinaryHeap`. Each runs four ways: natively, as wasm32 at moq-wasm's
size-optimized profile, hand-written (moq's `js/net` in `bench/moq.ts` and its `Timestamp`
in `bench/timestamp.ts`, ES iterator helpers in `bench/iter.ts`, a sorted array in
`bench/sorted.ts`, and `bigint` in `bench/rescale.ts`, as moq's `js/hang` converts
timescales), and translated; every route's output must match
`bench/sorted.ts`, `bigint` in `bench/rescale.ts`, as moq's `js/hang` converts
timescales, and typed arrays and a heap of its own in `bench/order.ts`), and translated; every route's output must match
native Rust's first. The JS routes run minified, the same bundles whose size it reports,
in Node, Bun, Chrome, and Firefox, with the WASM route's copies across the boundary timed
apart from its compute, and an A/A copy of the
Expand Down
3 changes: 3 additions & 0 deletions ts/bench/moq.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import { Cursor, Reader, type StreamVersion, Writer } from "moq/js/net/src/strea
import * as Varint from "moq/js/net/src/varint.ts";

import { iterate } from "./iter.ts";
import { order } from "./order.ts";
import { rescale } from "./rescale.ts";
import { ordered } from "./sorted.ts";
import { interleave, stamp } from "./timestamp.ts";
Expand Down Expand Up @@ -59,6 +60,8 @@ export function run(input: Uint8Array): Uint8Array | Promise<Uint8Array> {
return stamp(data);
case 11:
return interleave(data);
case 12:
return order(selector, data);
default:
throw new Error(`unknown workload ${op}`);
}
Expand Down
61 changes: 61 additions & 0 deletions ts/bench/order.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
// The `order/` workloads' hand-written route: `bench.rs`'s op 12, as TypeScript orders
// numbers without a library: a typed array's `sort`, a stable `sort` by key, a loop for
// the greatest and least, and a binary heap of its own, as JS has none.

/** Run op 12, as `bench::run` does, for the selector and the count `data` starts with. */
export function order(selector: number, data: Uint8Array): Uint8Array {
const padded = new Uint8Array(8);
padded.set(data.subarray(0, 8));
const count = Math.min(Math.max(Number(new DataView(padded.buffer).getBigUint64(0)), 1), 1 << 16);
let v = Uint32Array.from({ length: count }, (_, i) => (i * 40503) % (1 << 16));
switch (selector) {
case 0:
v.sort();
break;
case 1:
v = Uint32Array.from(Array.from(v).sort((a, b) => (a % 1024) - (b % 1024)));
break;
case 2:
case 3: {
let hi = 0;
let lo = 0xffffffff;
for (const x of v) {
if (x > hi) hi = x;
if (x < lo) lo = x;
}
v = Uint32Array.of(hi, lo);
break;
}
default: {
const heap: number[] = [];
for (const x of v) {
let i = heap.push(x) - 1;
while (i > 0 && heap[(i - 1) >> 1] < x) {
heap[i] = heap[(i - 1) >> 1];
i = (i - 1) >> 1;
}
heap[i] = x;
}
for (let n = 0; n < count; n++) {
v[n] = heap[0];
const x = heap.pop() as number;
const len = heap.length;
if (len === 0) break;
let i = 0;
for (;;) {
let child = 2 * i + 1;
if (child >= len) break;
if (child + 1 < len && heap[child + 1] > heap[child]) child++;
if (heap[child] <= x) break;
heap[i] = heap[child];
i = child;
}
heap[i] = x;
}
}
}
const out = new Uint8Array(24);
const view = new DataView(out.buffer);
[v[0], v[v.length >> 1], v[v.length - 1]].forEach((x, i) => view.setBigUint64(8 * i, BigInt(x)));
return out;
}
10 changes: 8 additions & 2 deletions ts/bench/workloads.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@
export interface Workload {
/** What is coded, such as `varint/quic/2B` or `frames/video/lite-07`, iterated, or converted. */
name: string;
op: "encode" | "decode" | "iterate" | "map" | "convert" | "add" | "compare" | "handoff";
/** Varints, frame headers, messages, items, keys, timestamps, or handoffs per call, which each time is divided by. */
op: "encode" | "decode" | "iterate" | "map" | "convert" | "add" | "compare" | "order" | "handoff";
/** Varints, frame headers, messages, items, keys, timestamps, integers, or handoffs per call, which each time is divided by. */
items: number;
input: Uint8Array;
}
Expand Down Expand Up @@ -93,6 +93,12 @@ export function workloads(encode: (input: Uint8Array) => Uint8Array): Workload[]
const time = (op: number, params: bigint[]) => Uint8Array.of(op, 0, ...[BigInt(count), ...params].flatMap(u64));
out.push({ name: "time/checked-add/micros", op: "add", items: count, input: time(10, video) });
out.push({ name: "time/ord/micros-48k", op: "compare", items: count, input: time(11, [...video, ...audio]) });

// `bench.rs`'s op 12: integers ordered by std's own `Ord`, which the translation tests.
const orders = ["sort", "sort-by-key", "iter-max-min", "ord-max-min", "binary-heap"];
orders.forEach((name, selector) => {
out.push({ name: `order/${name}`, op: "order", items: count, input: Uint8Array.of(12, selector, ...u64(BigInt(count))) });
});
return out;
}

Expand Down
56 changes: 56 additions & 0 deletions ts/fixtures/codec/expected/bench.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ export function run(input: rt.ByteSlice): rt.ByteVec {
stamp(new rt.ByteSlice(input.data, input.start + 2, input.length - 2), out);
} else if (op === 11) {
interleave(new rt.ByteSlice(input.data, input.start + 2, input.length - 2), out);
} else if (op === 12) {
order(selector, new rt.ByteSlice(input.data, input.start + 2, input.length - 2), out);
} else {
decoded(decodeGroup(new rt.ByteSlice(input.data, input.start + 2, input.length - 2), liteVersion(selector), out), out);
}
Expand Down Expand Up @@ -302,6 +304,60 @@ export function ordered(data: rt.ByteSlice, out: rt.ByteVec): void {
putU64(out, sum);
}

/**
* `count` integers sorted, sorted by a key, searched for the greatest and least, or
* pushed onto a `BinaryHeap` and popped off, as `selector` picks, each by `u32`'s own
* `Ord`: the first, middle, and last of the order, or the greatest and least.
*/
export function order(selector: rt.u8, data: rt.ByteSlice, out: rt.ByteVec): void {
let v: rt.u32[] = rt.iterToArray(rt.iterMap(new rt.Range(0, rt.u64Low(rt.u64Clamp(takeU64(data)[0], 1, 65536))), (i: rt.u32): rt.u32 => {
return rt.u32Rem(rt.u32Mul(i, 40503), 1 << 16);
}));
if (selector === 0) {
rt.sliceSort(v, rt.lt);
} else if (selector === 1) {
rt.sliceSortByKey(v, (_210: rt.u32): rt.u32 => {
return rt.u32Rem(_210, 1024);
}, rt.lt);
} else if (selector === 2) {
v = [rt.optionUnwrap(rt.iterMax(rt.iterSlice(v), rt.lt)), rt.optionUnwrap(rt.iterMin(rt.iterSlice(v), rt.lt))];
} else if (selector === 3) {
const _35: [rt.u32, rt.u32] = rt.iterFold(rt.iterSlice(v), [0, 4294967295] as [rt.u32, rt.u32], (_2: [rt.u32, rt.u32], _33: rt.u32): [rt.u32, rt.u32] => {
const hi2: rt.u32 = _2[0];
const lo2: rt.u32 = _2[1];
const x4: rt.u32 = _33;
return [rt.max(hi2, x4), rt.min(lo2, x4)] as [rt.u32, rt.u32];
});
v = [_35[0], _35[1]];
} else {
const heap: rt.BinaryHeap<rt.u32> = rt.heapNew() as rt.BinaryHeap<rt.u32>;
for (const _47 of rt.replaceIn(v, [])) {
rt.heapPush(heap, _47, rt.le);
}
while (true) {
const _53: rt.u32 | undefined = rt.heapPop(heap, rt.le);
if (_53 !== undefined) {
v.push(_53);
} else {
break;
}
}
}
const _63: rt.u32[] = v;
const _61: rt.u32 = _63[rt.idx(_63, 0)];
const _66: rt.u32[] = v;
const _64: rt.u32 = _66[rt.idx(_66, rt.usizeDiv(v.length, 2))];
const _73: rt.u32[] = v;
const iter2: rt.Iter<rt.u32> = rt.iterSlice([_61, _64, _73[rt.idx(_73, rt.usizeSub(v.length, 1))]]);
while (true) {
const _79: rt.u32 | rt.Done = iter2.next();
if (_79 === rt.DONE) {
break;
}
putU64(out, _79 as rt.u64);
}
}

/**
* One iterator chain over every byte, picked by `chain`, with closures that cost little
* next to the iterator protocol they run through.
Expand Down
36 changes: 34 additions & 2 deletions ts/fixtures/codec/src/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@
//! decode's input. Op 6 runs iterator chains over its bytes instead, as moq-net's model
//! writes them, op 7 decodes frame headers as they arrive a few bytes at a time, op 8
//! keeps a `BTreeMap`, as moq-net's model keeps its state, op 9 converts timestamps
//! between timescales, and ops 10 and 11 add and order them as moq-net's `Timestamp` does.
//! between timescales, ops 10 and 11 add and order them as moq-net's `Timestamp` does, and
//! op 12 orders integers by std's own `Ord`.

use std::cmp::Ordering;
use std::collections::BTreeMap;
use std::collections::{BTreeMap, BinaryHeap};

use crate::coding::{Decode, DecodeError, Decoder, Encode, EncodeError, Encoder, Form, varint};
use crate::harness::{decode_error, encode_error, form, lite_version, put_u64, put_usize, take_u64};
Expand Down Expand Up @@ -35,6 +36,7 @@ pub fn run(input: &[u8]) -> Vec<u8> {
9 => rescale(selector, data, &mut out),
10 => stamp(data, &mut out),
11 => interleave(data, &mut out),
12 => order(selector, data, &mut out),
_ => decoded(decode_group(data, lite_version(selector), &mut out), &mut out),
}
out
Expand Down Expand Up @@ -207,6 +209,36 @@ fn ordered(data: &[u8], out: &mut Vec<u8>) {
put_u64(out, sum);
}

/// `count` integers sorted, sorted by a key, searched for the greatest and least, or
/// pushed onto a `BinaryHeap` and popped off, as `selector` picks, each by `u32`'s own
/// `Ord`: the first, middle, and last of the order, or the greatest and least.
fn order(selector: u8, data: &[u8], out: &mut Vec<u8>) {
let (count, _) = take_u64(data);
let count = count.clamp(1, 1 << 16) as u32;
let mut v: Vec<u32> = (0..count).map(|i| (i * 40503) % (1 << 16)).collect();
match selector {
0 => v.sort(),
1 => v.sort_by_key(|&x| x % 1024),
2 => v = vec![*v.iter().max().unwrap(), *v.iter().min().unwrap()],
3 => {
let (hi, lo) = v.iter().fold((0, u32::MAX), |(hi, lo), &x| (hi.max(x), lo.min(x)));
v = vec![hi, lo];
}
_ => {
let mut heap = BinaryHeap::new();
for x in std::mem::take(&mut v) {
heap.push(x);
}
while let Some(x) = heap.pop() {
v.push(x);
}
}
}
for x in [v[0], v[v.len() / 2], v[v.len() - 1]] {
put_u64(out, x as u64);
}
}

/// One iterator chain over every byte, picked by `chain`, with closures that cost little
/// next to the iterator protocol they run through.
fn iterate(chain: u8, data: &[u8], out: &mut Vec<u8>) {
Expand Down
Loading