diff --git a/agents/recording-analyst.md b/agents/recording-analyst.md new file mode 100644 index 0000000..3447a74 --- /dev/null +++ b/agents/recording-analyst.md @@ -0,0 +1,56 @@ +--- +name: recording-analyst +description: Answers quantitative questions about a PolySpatial recording (.qrec) strictly from the recorded per-frame scene state — how many times something rotated or changed color, how far it moved, when it appeared or disappeared, what changed around an annotation. Use it whenever a question about a recording or an annotation reference (`#`) can be settled by numbers; it never reads project source or scenes, so its answer is a measurement, not an interpretation. +tools: Bash +model: sonnet +--- + +You measure. You are handed a Unity project path, a recording (or an annotation reference that resolves +to one) and a question. You answer the question with numbers taken from the recording, with the frames +and seconds they came from, and nothing else. + +## Tools you use + +Only these, always with `--project-path `: + +- `unity command polyspatial_annotation_show --ref "" [--window N]` — resolve a reference: text, + frame, entity path, state at that frame, changes around it. +- `unity command polyspatial_annotation_list`, `polyspatial_recording_list`, `polyspatial_recording_metadata`. +- `unity command polyspatial_scene_state --recording R --entity "" --start-frame A --end-frame B [--properties p1,p2] [--summarize true]` +- `unity command polyspatial_scene_changes --recording R --start-frame A --end-frame B [--entity X] [--properties ...] [--group-depth 2]` +- `unity command polyspatial_entity_timeline --recording R --entity X --property position|worldPosition|rotation|worldRotation|scale --step 1` +- `unity command polyspatial_scene_export --out Temp/.ndjson ...` — same as scene_state, to a file. +- `python3` on files you exported or redirected into your scratch directory. + +Never read `Assets/`, `Packages/`, `ProjectSettings/`, `.unity`, `.prefab` or `.cs` files, never run +`grep` over the project, never `editor_play`, `simulate_*`, `capture_game_view` or anything that changes +the Editor. If the recording cannot answer the question, say exactly which data is missing. + +## How you work + +1. Resolve the reference first; note `entityPath`, `frame`, `frameEnd`, `recordingFrames`. A reply with `scene: true` is a note on the open scene, not on a recording: there is nothing to measure, say so and hand it back. +2. Decide which entity and property answer the question. A property that is constant on the annotated + entity usually lives on an ancestor: walk up the `entityPath` one level at a time. Names repeat; when + a command reports an ambiguous name, pass a path suffix such as `Parent/Child`. +3. Pull every frame you need (`--step 1`, or `--properties` to keep only the property you care about) + and redirect anything longer than a screen to a file. Compute on the file: + - rotation: for consecutive quaternions q0,q1 take delta = q1 * inverse(q0), convert to angle-axis, + accumulate signed angle about the dominant axis; net turns = sum/360, total turns = sum|angle|/360; + runs of |delta| > 0 are bursts. Report net and total separately. + - distance: sum |worldPosition(i) - worldPosition(i-1)|. + - counts: the length of a `[[frame,value],...]` keyframe array minus one. + - presence: `lifecycle` keyframes. +4. Convert frames to seconds with the `time` fields of `polyspatial_annotation_list` or by + `polyspatial_scene_changes --start-time/--end-time`. + +## What you return + +A short report, nothing else: + +- **Answer**: the number(s), with unit, and the frame range and seconds they cover. +- **Where it lives**: entity path and property that produced the answer (e.g. "rotation is on the + parent `MMCupcake`, the annotated `Cup` is rigid"). +- **Evidence**: 2–5 lines of the raw values or the bursts you found. +- **Commands**: the exact commands you ran, so the caller can reproduce. +- **Not measurable**: anything the question asked that the recording cannot show (pixels, script + variables, why something happened). diff --git a/skills/polyspatial-playtest/SKILL.md b/skills/polyspatial-playtest/SKILL.md new file mode 100644 index 0000000..412108a --- /dev/null +++ b/skills/polyspatial-playtest/SKILL.md @@ -0,0 +1,230 @@ +--- +name: polyspatial-playtest +description: Use when you must prove that gameplay, UI, or behavior in a Unity project actually works or is actually broken — "verify my change in Play mode", "does the button do X", "play-test this", "why does the cupcake stop spinning", "check the fix", or whenever someone hands you a PolySpatial annotation reference like `MyRecording-2026-9-11-101947#7d29bfdc…`. Records the Play session as a PolySpatial .qrec, drives the game with simulated input, then answers from the recorded per-frame scene state instead of eyeballing a screenshot. Requires a running Editor with com.unity.pipeline and the PolySpatial recording commands (`unity command --query polyspatial`). +allowed-tools: + - Bash + - Read +--- + +# PolySpatial play-test and debug + +Screenshots tell you what a frame looked like; a PolySpatial recording tells you what every +entity *was* on every frame — transforms, component properties, lifecycle, audio — and you can +query it after Play mode has ended. This skill is the loop: understand the request (often an +annotation someone left on a recording), change the code, then **play the game for real, record +it, and read the numbers back**. Use `capture_game_view` only to confirm what a frame looks like +once you already know from the data which frame matters. + +All commands below are `unity command [--flag value]`. Always pass `--project-path ` +when more than one Editor may be open. Read [references/commands.md](references/commands.md) for +every flag and the shape of each result. + +## 0. Preconditions (check once per session) + +```bash +unity status # an Editor for this project, state "ready" +unity command --query polyspatial --detail compact # the polyspatial_* commands must be listed +unity command set_autotick --enable true # an unfocused Editor barely advances frames otherwise +``` + +If `polyspatial_*` commands are missing, the project lacks the PolySpatial recording packages: say +so, and fall back to `editor_play` plus `capture_game_view`. Do not hand-edit `.unity`/`.prefab` +files while an Editor is reachable (see the `unity-cli` skill). + +The game must use the **Input System** for simulated input to reach it; legacy `Input.GetKey` code +cannot be driven. Check `ProjectSettings/ProjectSettings.asset` → `activeInputHandler` (1 or 2), or +just try `simulate_key` in Play mode and see whether the game reacts. + +## 1. Start from an annotation reference + +A reference looks like `#` (people copy it with the `Ref` button in +Window ▸ PolySpatial ▸ Annotations, and CoCreate's Playtest sends a whole set at once, see +[From a CoCreate Playtest](#from-a-cocreate-playtest)). It may also arrive as a bare id or as a +path to the annotation's `.json`. Resolve it first; never guess what it points at: + +```bash +unity command polyspatial_annotation_list # every annotation, one JSON line each +unity command polyspatial_annotation_show --ref "#" --window 30 +``` + +`polyspatial_annotation_show` returns: + +- `annotation.text` — what the person said, and `frame`/`time` — when. `kind` is `entity` when they + right-clicked an object (then `entityPath`, `worldPosition`, `worldBoundsSize`, `hitPoint` are set) + or `moment` when they annotated the whole frame. +- `state` — the entity's subtree at that frame: world transform, components and their properties. +- `changes` — every property of that subtree that varied within ±`window` frames, with `from`, `to`, + `firstChangeFrame`, `lastChangeFrame`. For a moment annotation you get `changedEntities` grouped by + the top two hierarchy levels instead; drill in with `polyspatial_scene_changes --entity`. + +Read text and data together. "The cupcake stopped spinning" at frame 400 plus `world.rotation` +changing through frame 430 means it did *not* stop — the person is describing the expected +behavior, or noticed something else. "The guy is looking back" as a moment annotation plus +`changedEntities` naming only `DudeContainer` tells you which object to inspect. + +### Delegate the measuring + +Raw recording data is large and the arithmetic is mechanical, so keep both out of the main +conversation. In Claude Code, hand every quantitative question to the `recording-analyst` subagent +that ships with this plugin (`Agent` with `subagent_type: recording-analyst`), passing the project +path, the reference or recording, and the question verbatim. It can only run `polyspatial_*` commands +and compute on their output, so what comes back is a measurement with frames and seconds, and you +decide afterwards whether the *why* needs the code. In Codex there is no plugin subagent: copy +[references/codex-recording-analyst.toml](references/codex-recording-analyst.toml) into the project's +`.codex/agents/` and delegate the same way, or follow the rules below yourself. + +### Answer from the recording first, explain from the code second + +When the question is about what happened ("how many times did it rotate", "did it ever leave the +platform", "how long was the button disabled"), the recording is the source of truth and the +answer is a number you compute from it. Do that **before** opening any scene or script, and report +it as soon as you have it. Only then, if the person asked *why*, read the code that drives the +entity — and say plainly which part of your answer is measured and which is inferred from code. +Do not spend the session grepping `.unity` files while the measured answer sits unreported. + +Worked example, "how many times did the cupcake rotate?". The recording gives you samples; +the arithmetic is yours — dump every frame to a file and compute in a short script, exactly as +you would with any dataset: + +```bash +unity command polyspatial_annotation_show --ref "#" # entityPath: …/Cupcake/Wiggle/MMCupcake/Cup +unity command polyspatial_entity_timeline --recording --entity "MMCupcake/Cup" --property rotation --step 1 --json > cup.json +unity command polyspatial_entity_timeline --recording --entity "Wiggle/MMCupcake" --property rotation --step 1 --json > mmcupcake.json +# python: for consecutive quaternions q0,q1 take delta = q1 * inverse(q0), convert to angle-axis, +# accumulate signed angle about the dominant axis; sum / 360 = turns; runs of |delta| > 0 = bursts. +``` + +Result on the sample recording: `Cup` never turns relative to its parent (one distinct rotation +in 779 frames); `MMCupcake` turns 13.9 times about x in two bursts, frames 336–380 (6.9) and +384–428 (6.9). Answer with those numbers and frames first: "the cupcake spun ~13.9 turns in two +~7-turn bursts at 7.4–8.4 s and 8.5–9.5 s; the rotation is on the parent MMCupcake, the annotated +Cup mesh is rigid." Then, only if asked why: the MMF_Rotation feedback on that object. + +The same pattern answers "how many times did the color change" (`polyspatial_scene_state +--entity X --start-frame 1 --end-frame N --properties Image.color` and count that keyframe array), +"how far did the player travel" (sum `worldPosition` deltas), "was it ever inactive" (the +`lifecycle` keyframes). There is no per-question command; there are samples and your computation. +When a slice is more than a screen of text, `polyspatial_scene_export --out Temp/x.ndjson ...` +writes it to a file: compute on the file, print only the result. + +Rules for measuring: + +- A property that is constant on the annotated entity usually lives on an ancestor: walk up the + `entityPath` one level at a time. `worldRotation`/`worldPosition` give the composed result. +- Net rotation and total rotation differ: a wiggle travels many degrees and nets zero. Report + the one the question asks for, and say which. +- `--step 1` gives every frame; the default samples ~200 points, which is fine for a curve but + not for counting. +- Convert frames to seconds with the `time` fields of `polyspatial_annotation_list` or `--start-time` + on `polyspatial_scene_changes`; quote both in the answer. + +To see the moment, replay and park on the frame, then capture: + +```bash +unity command polyspatial_playback --recording --frame 400 # rebuilds the recording in the open scene, parked on frame 400; no Play mode +unity command polyspatial_playback_seek --frame 430 # any frame, either direction +unity command capture_game_view --save_path Temp/annotation-400.png # camera capture; "screen" needs Play mode +R=UnityEditor.PolySpatial.Utilities.RecordingPlaybackScene +unity command eval --code "$R.StopPlayback(); return $R.IsPlayingBack;" # closes the replay and restores the scene's own objects +``` + +If the task is to change behavior, then go read the code that drives that entity (the hierarchy +path names the GameObjects), fix or implement, and prove it with section 2. + +### From a CoCreate Playtest + +CoCreate's Playtest hands over one message per triage: a header +`Playtest session — N notes on .qrec (mm:ss recorded)`, then one +`## Note k · ·