Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -194,10 +194,7 @@ jobs:
- name: Build extension packages
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: >-
pnpm --filter "@codexo/exojs-particles" --filter "@codexo/exojs-tilemap" --filter "@codexo/exojs-tiled"
--filter "@codexo/exojs-physics" --filter "@codexo/exojs-tilemap-physics" --filter "@codexo/exojs-lighting" --filter "@codexo/exojs-pathfinding" --filter "@codexo/exojs-audio-fx"
--filter "@codexo/exojs-aseprite" --filter "@codexo/exojs-ldtk" --filter "@codexo/exojs-react" build
run: pnpm build:packages

- name: Verify production stripping against the built dist
env:
Expand Down
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,6 +173,12 @@ pnpm verify:lockstep pnpm typecheck:examples pnpm typecheck:guides
pnpm site:build pnpm test:examples:smoke
```

Scripts forward their arguments, so a variant is an argument rather than a
script of its own: `pnpm lint --fix`, `pnpm test --coverage`,
`pnpm test:browser:webgl --reporter=verbose`. The measurement harnesses run
through one runner, `pnpm perf <name>` (`pnpm perf` lists them); the
cross-library benchmarks through `pnpm bench` (see `packages/exojs-bench`).

## Shared configuration

`@codexo/exojs-config` (private, unpublished, never a runtime dependency) centralizes
Expand Down
132 changes: 51 additions & 81 deletions package.json

Large diffs are not rendered by default.

8 changes: 4 additions & 4 deletions packages/exojs-bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,8 @@ matrix cell; both share the harness's page, server and launch flags so their
numbers are produced under the same conditions.

```sh
pnpm perf:webgpu:stall # per-frame selection/capacity/upload/allocation counters
pnpm perf:webgpu:timer # per-frame timer methodology: raw vs attributed queue
pnpm perf webgpu:alloc # per-frame allocation, wall-clock or work-unit counters, one browser per cell
pnpm perf webgpu:timer # per-frame timer methodology: raw vs attributed queue
# latency vs hardware timestamps, plus serialized /
# canvas-clear / offscreen-clear control arms and the
# clock + scheduler controls behind the floor above
Expand Down Expand Up @@ -405,8 +405,8 @@ pnpm --filter @codexo/exojs-bench bench \
No `--` separator is needed with `pnpm --filter …`; pnpm forwards these straight
to the script. Running the same script from inside `packages/exojs-bench`
(`pnpm bench -- --out=…`) works too, and `--out` is then relative to the package
directory either way. The root `pnpm bench` is a different thing entirely — the
engine's own `vitest bench` micro-benchmarks — and there is no root
directory either way. The root `pnpm bench` forwards here as well; the engine's
own `vitest bench` micro-benchmarks are `pnpm bench:micro`, and there is no root
`bench:compare`.

The run writes `results.json`, `results.csv` and `results.md` into `--out`
Expand Down
8 changes: 4 additions & 4 deletions packages/exojs-bench/results/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,10 +142,10 @@ pnpm bench:compare -- \
--profile
```

The harness scripts live in `packages/exojs-bench` and are run from there: the
root `pnpm bench` is the engine's own micro-benchmark suite, not this harness.
The `--` separates the harness's flags from pnpm's own, and output directories
are relative to the package.
The harness scripts live in `packages/exojs-bench`; the root `pnpm bench` forwards
to the same harness, while `bench:compare` is run from the package. The `--`
separates the harness's flags from pnpm's own, and output directories are
relative to the package either way.

`--rendering` and `--physics` are repeatable, once per run, in run order. Repeat
the same three-run pattern for `--domain=physics` and pass both sets to one
Expand Down
6 changes: 3 additions & 3 deletions packages/exojs-bench/src/runTimerProbe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@
* D hardware sum of render-pass timestamp deltas (timestamp-query ground truth)
* ```
*
* pnpm perf:webgpu:timer -- --nodes 1000000 --config retained --frames 30 --warmup 40
* pnpm perf:webgpu:timer -- --mode serialized --frames 30
* pnpm perf:webgpu:timer -- --controls-only --repeats 12
* pnpm perf webgpu:timer --nodes 1000000 --config retained --frames 30 --warmup 40
* pnpm perf webgpu:timer --mode serialized --frames 30
* pnpm perf webgpu:timer --controls-only --repeats 12
*
* One browser process per cell: the timer question includes device bootstrap, so
* a cell that inherited another cell's warm device would not be the same
Expand Down
6 changes: 1 addition & 5 deletions scripts/ci/lanes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,11 +148,7 @@ export const LANES: readonly Lane[] = [
id: 'package',
stage: 'verify',
when: 'packageVerify',
run:
'pnpm size && pnpm size:summary && pnpm verify:exports && pnpm verify:declaration-imports && pnpm verify:lockstep && pnpm verify:release-matrix ' +
'&& pnpm pack --dry-run && pnpm --filter "@codexo/exojs-build" --filter "@codexo/exojs-particles" --filter "@codexo/exojs-tilemap" ' +
'--filter "@codexo/exojs-tiled" --filter "@codexo/exojs-physics" --filter "@codexo/exojs-tilemap-physics" --filter "@codexo/exojs-lighting" --filter "@codexo/exojs-pathfinding" --filter "@codexo/exojs-audio-fx" ' +
'--filter "@codexo/exojs-aseprite" --filter "@codexo/exojs-ldtk" --filter "@codexo/exojs-react" pack --dry-run && pnpm verify:publint',
run: 'pnpm size && pnpm size:summary && pnpm verify:exports && pnpm verify:declaration-imports && pnpm verify:lockstep && pnpm verify:release-matrix && pnpm verify:publish',
dist: true,
},
{
Expand Down
141 changes: 141 additions & 0 deletions scripts/perf.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
/**
* Runner for the measurement harnesses under `test/perf/` and the bench
* package's probes.
*
* pnpm perf # list the runs
* pnpm perf <name> [args...] # run one, forwarding the arguments
* pnpm perf smoke # what the `sync` gate group runs
*
* Every in-process harness needs the same Node invocation: the
* `@codexo/exojs-source` condition so package imports resolve to `src/`, the
* GLSL loader so shader imports resolve at all, and `tsx` for TypeScript.
* Spelling that prefix out once per harness in package.json is what let the
* scripts drift apart (some with `--expose-gc`, some without, one with a heap
* limit), so the prefix lives here and a run declares only what differs.
*/
import { spawnSync } from 'node:child_process';
import { pathToFileURL } from 'node:url';

interface PerfRun {
/** What the run measures, one line. */
readonly summary: string;
/** A `test/perf` module executed in-process against engine source. */
readonly module?: string;
/** Extra Node flags for `module` runs. */
readonly nodeFlags?: readonly string[];
/** A shell command instead of an in-process module. */
readonly command?: string;
/** Other runs to execute in order. */
readonly runs?: readonly string[];
}

const SOURCE_NODE_FLAGS = ['--conditions=@codexo/exojs-source', '--import', './scripts/glsl-register.ts', '--import', 'tsx/esm'] as const;
/** Cell probes take one large scene per process and read `gc()` for exact deltas. */
const CELL_NODE_FLAGS = ['--expose-gc', '--max-old-space-size=8192'] as const;

export const PERF_RUNS = {
rendering: { summary: 'renderer CPU submission benchmark', module: 'test/perf/rendering-benchmark.ts' },
audio: { summary: 'audio graph benchmark', module: 'test/perf/audio-benchmark.ts' },
collision: { summary: 'collision benchmark', module: 'test/perf/collision-benchmark.ts' },
'scene-graph': { summary: 'scene graph benchmark', module: 'test/perf/scene-graph-benchmark.ts' },
interaction: { summary: 'interaction benchmark', module: 'test/perf/interaction-benchmark.ts' },
'collect-phase': { summary: 'collect phase benchmark', module: 'test/perf/collect-phase-benchmark.ts' },
profile: { summary: 'profile benchmark', module: 'test/perf/profile-benchmark.ts' },
'profile:gc': { summary: 'profile benchmark with forced GC for exact memory deltas', module: 'test/perf/profile-benchmark.ts', nodeFlags: ['--expose-gc'] },
smoke: {
summary: 'every in-process benchmark once, for its exit code (the `sync` gate)',
runs: ['rendering', 'audio', 'collision', 'scene-graph', 'interaction', 'collect-phase', 'profile'],
},

renderers: { summary: 'structural renderer sweep, full matrix', command: 'tsx test/perf/rendering/run-sweep.ts full' },
'renderers:quick': { summary: 'structural renderer sweep, small matrix', command: 'tsx test/perf/rendering/run-sweep.ts quick' },
'renderers:browser': {
summary: 'structural renderer metrics against a real WebGL2 context',
command: 'vitest run --project=browser-webgl-chromium webgl2-renderer-perf',
},
'renderers:alloc': { summary: 'steady-state allocation per renderer scene', module: 'test/perf/rendering/run-allocation.ts' },
'renderers:alloc:cell': { summary: 'one allocation scene in a fresh process', module: 'test/perf/rendering/run-allocation-cell.ts' },
'renderers:cull-margin': { summary: 'culling margin sweep', module: 'test/perf/rendering/run-cull-margin.ts' },
'renderers:cull-margin:cell': {
summary: 'one culling margin cell in a fresh process',
module: 'test/perf/rendering/run-cull-margin-cell.ts',
nodeFlags: CELL_NODE_FLAGS,
},
'renderers:bootstrap': { summary: 'scene bootstrap allocation', module: 'test/perf/rendering/run-bootstrap-allocation.ts' },
'renderers:bootstrap:cell': {
summary: 'one bootstrap cell in a fresh process',
module: 'test/perf/rendering/run-bootstrap-cell.ts',
nodeFlags: CELL_NODE_FLAGS,
},
'renderers:instance-cost': {
summary: 'per-instance cost by node type',
module: 'test/perf/rendering/run-instance-cost.ts',
nodeFlags: CELL_NODE_FLAGS,
},

'webgpu:alloc': { summary: 'WebGPU allocation per scene, one browser per cell', command: 'tsx test/perf/webgpu/run-webgpu-allocation.ts' },
'webgpu:timer': { summary: 'WebGPU frame timer methodology probe', command: 'pnpm --filter @codexo/exojs-bench timer' },
} as const satisfies Record<string, PerfRun>;

export type PerfRunName = keyof typeof PERF_RUNS;

const runNames = Object.keys(PERF_RUNS) as PerfRunName[];

const printUsage = (): void => {
const width = Math.max(...runNames.map(name => name.length));
console.log('Usage: pnpm perf <name> [args...]\n');
for (const name of runNames) {
console.log(` ${name.padEnd(width)} ${PERF_RUNS[name].summary}`);
}
};

const execute = (name: PerfRunName, args: readonly string[]): number => {
const run: PerfRun = PERF_RUNS[name];

if (run.runs) {
for (const child of run.runs) {
console.log(`\n=== pnpm perf ${child} ===\n`);
const status = execute(child as PerfRunName, args);
if (status !== 0) return status;
}
return 0;
}

if (run.module) {
const result = spawnSync(process.execPath, [...(run.nodeFlags ?? []), ...SOURCE_NODE_FLAGS, run.module, ...args], { stdio: 'inherit' });
return result.status ?? 1;
}

// A shell so the pnpm/tsx/vitest shims resolve on Windows as well; arguments
// are re-quoted because the shell splits the joined line again.
const quoted = args.map(arg => (/\s/.test(arg) ? JSON.stringify(arg) : arg));
const result = spawnSync([run.command, ...quoted].join(' '), { stdio: 'inherit', shell: true });
return result.status ?? 1;
};

const main = (): void => {
const [requested, ...rest] = process.argv.slice(2);
// `pnpm perf <name> -- <args>` forwards the separator too; the harnesses do not expect it.
const args = rest[0] === '--' ? rest.slice(1) : rest;

if (!requested) {
printUsage();
process.exit(2);
}

if (!runNames.includes(requested as PerfRunName)) {
console.error(`Unknown perf run '${requested}'.\n`);
printUsage();
process.exit(2);
}

const status = execute(requested as PerfRunName, args);
if (status !== 0) {
console.error(`\nperf ${requested} failed (exit code ${status}).`);
}
process.exit(status);
};

if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
main();
}
Loading
Loading