Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .agents/skills/uloop-raycast/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ unity_x = input_x
unity_y = gameViewHeight - input_y
```

- Device Simulator play view is supported. Prefer coordinates from `uloop screenshot --capture-mode rendering` (or its annotated `SimX`/`SimY`).

## Examples

```bash
Expand Down
10 changes: 8 additions & 2 deletions .agents/skills/uloop-screenshot/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ uloop screenshot [--window-name <name>] [--resolution-scale <scale>] [--match-mo

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `--window-name` | string | `Game` | Window name to capture. Ignored when `--capture-mode rendering`. |
| `--window-name` | string | `Game` | Window name to capture. Ignored when `--capture-mode rendering`. When the Game tab is Device Simulator and the title is `Simulator`, default `Game` falls back to `Simulator`. |
| `--resolution-scale` | number | `1.0` | Resolution scale (0.1 to 1.0) |
| `--match-mode` | enum | `exact` | Window name matching mode: `exact`, `prefix`, or `contains`. Ignored when `--capture-mode rendering`. |
| `--capture-mode` | enum | `window` | `window`=capture EditorWindow including toolbar, `rendering`=capture game rendering only (PlayMode required, coordinates match simulate-mouse) |
Expand All @@ -36,7 +36,13 @@ uloop screenshot [--window-name <name>] [--resolution-scale <scale>] [--match-mo

## Window Name

The window name is the text displayed in the window's title bar (tab). Common names: Game, Scene, Console, Inspector, Project, Hierarchy, Animation, Animator, Profiler. Custom EditorWindow titles are also supported.
The window name is the text displayed in the window's title bar (tab). Common names: Game, Simulator, Scene, Console, Inspector, Project, Hierarchy, Animation, Animator, Profiler. Custom EditorWindow titles are also supported.

## Device Simulator

- Prefer `--capture-mode rendering` for the annotate → click flow. It works with both the normal Game view and Device Simulator, and coordinates match `simulate-mouse-ui` / `simulate-mouse-input` / `raycast`.
- `--capture-mode window` captures Editor chrome (toolbar/borders). With Device Simulator as the play view, default `--window-name Game` falls back to `Simulator` when no Game tab exists; you can also pass `--window-name Simulator`.
- Simulator Fit to Screen / Scale / Safe Area overlays are chrome-only and do not change rendering-mode input coordinates. After device rotation, re-run annotate (or re-read `Screen` size) before clicking.

## Output

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Read this when using `uloop screenshot --capture-mode rendering --annotate-elements true` or `--annotate-raycast-grid true` output to find coordinates for `simulate-mouse-ui`, `simulate-mouse-input`, or `raycast`.

Device Simulator is supported for this flow: prefer `--capture-mode rendering` (not `window`) so coordinates match the simulated device resolution.

## AnnotatedElements Fields

`AnnotatedElements` is empty unless `--annotate-elements true` is used, or unless `--annotate-raycast-grid true` adds clustered 3D collider candidates (with or without `--raycast-layer-mask`). UI entries are sorted by z-order, frontmost first. Each item contains:
Expand Down
1 change: 1 addition & 0 deletions .agents/skills/uloop-simulate-mouse-input/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ unity_y = gameViewHeight - input_y
```

- `Mouse.current.position` uses bottom-left Unity coordinates, so the value read inside Unity may show the converted Y.
- Device Simulator play view is supported. Prefer rendering-mode screenshots for coordinates; they match the simulated device resolution, not Simulator chrome scale.

## Prerequisites

Expand Down
1 change: 1 addition & 0 deletions .agents/skills/uloop-simulate-mouse-ui/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ uloop simulate-mouse-ui --action <action> --x <x> --y <y> [options]
- Dragging on empty space (no draggable UI element) returns `Success = false`
- `--bypass-raycast` still uses coordinates for pointer event positions, but chooses the clicked, long-pressed, or dragged GameObject by `--target-path`
- If `--target-path` or `--drop-target-path` matches multiple active GameObjects, the command fails instead of choosing an arbitrary duplicate
- Device Simulator play view is supported. Prefer `uloop screenshot --capture-mode rendering --annotate-elements` for coordinates; they use the simulated device resolution (`Handles.GetMainGameViewSize()` / `Screen`), not the Simulator chrome scale.

## Pause Point Inspection (Standard for E2E)

Expand Down
2 changes: 2 additions & 0 deletions .claude/skills/uloop-raycast/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ unity_x = input_x
unity_y = gameViewHeight - input_y
```

- Device Simulator play view is supported. Prefer coordinates from `uloop screenshot --capture-mode rendering` (or its annotated `SimX`/`SimY`).

## Examples

```bash
Expand Down
10 changes: 8 additions & 2 deletions .claude/skills/uloop-screenshot/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ uloop screenshot [--window-name <name>] [--resolution-scale <scale>] [--match-mo

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `--window-name` | string | `Game` | Window name to capture. Ignored when `--capture-mode rendering`. |
| `--window-name` | string | `Game` | Window name to capture. Ignored when `--capture-mode rendering`. When the Game tab is Device Simulator and the title is `Simulator`, default `Game` falls back to `Simulator`. |
| `--resolution-scale` | number | `1.0` | Resolution scale (0.1 to 1.0) |
| `--match-mode` | enum | `exact` | Window name matching mode: `exact`, `prefix`, or `contains`. Ignored when `--capture-mode rendering`. |
| `--capture-mode` | enum | `window` | `window`=capture EditorWindow including toolbar, `rendering`=capture game rendering only (PlayMode required, coordinates match simulate-mouse) |
Expand All @@ -36,7 +36,13 @@ uloop screenshot [--window-name <name>] [--resolution-scale <scale>] [--match-mo

## Window Name

The window name is the text displayed in the window's title bar (tab). Common names: Game, Scene, Console, Inspector, Project, Hierarchy, Animation, Animator, Profiler. Custom EditorWindow titles are also supported.
The window name is the text displayed in the window's title bar (tab). Common names: Game, Simulator, Scene, Console, Inspector, Project, Hierarchy, Animation, Animator, Profiler. Custom EditorWindow titles are also supported.

## Device Simulator

- Prefer `--capture-mode rendering` for the annotate → click flow. It works with both the normal Game view and Device Simulator, and coordinates match `simulate-mouse-ui` / `simulate-mouse-input` / `raycast`.
- `--capture-mode window` captures Editor chrome (toolbar/borders). With Device Simulator as the play view, default `--window-name Game` falls back to `Simulator` when no Game tab exists; you can also pass `--window-name Simulator`.
- Simulator Fit to Screen / Scale / Safe Area overlays are chrome-only and do not change rendering-mode input coordinates. After device rotation, re-run annotate (or re-read `Screen` size) before clicking.

## Output

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Read this when using `uloop screenshot --capture-mode rendering --annotate-elements true` or `--annotate-raycast-grid true` output to find coordinates for `simulate-mouse-ui`, `simulate-mouse-input`, or `raycast`.

Device Simulator is supported for this flow: prefer `--capture-mode rendering` (not `window`) so coordinates match the simulated device resolution.

## AnnotatedElements Fields

`AnnotatedElements` is empty unless `--annotate-elements true` is used, or unless `--annotate-raycast-grid true` adds clustered 3D collider candidates (with or without `--raycast-layer-mask`). UI entries are sorted by z-order, frontmost first. Each item contains:
Expand Down
1 change: 1 addition & 0 deletions .claude/skills/uloop-simulate-mouse-input/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ unity_y = gameViewHeight - input_y
```

- `Mouse.current.position` uses bottom-left Unity coordinates, so the value read inside Unity may show the converted Y.
- Device Simulator play view is supported. Prefer rendering-mode screenshots for coordinates; they match the simulated device resolution, not Simulator chrome scale.

## Prerequisites

Expand Down
1 change: 1 addition & 0 deletions .claude/skills/uloop-simulate-mouse-ui/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ uloop simulate-mouse-ui --action <action> --x <x> --y <y> [options]
- Dragging on empty space (no draggable UI element) returns `Success = false`
- `--bypass-raycast` still uses coordinates for pointer event positions, but chooses the clicked, long-pressed, or dragged GameObject by `--target-path`
- If `--target-path` or `--drop-target-path` matches multiple active GameObjects, the command fails instead of choosing an arbitrary duplicate
- Device Simulator play view is supported. Prefer `uloop screenshot --capture-mode rendering --annotate-elements` for coordinates; they use the simulated device resolution (`Handles.GetMainGameViewSize()` / `Screen`), not the Simulator chrome scale.

## Pause Point Inspection (Standard for E2E)

Expand Down
60 changes: 60 additions & 0 deletions Assets/Tests/Editor/GameViewBridgeTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
using System.Reflection;
using NUnit.Framework;
using UnityEngine;

using io.github.hatayama.UnityCliLoop.InternalAPIBridge;

namespace io.github.hatayama.UnityCliLoop.Tests.Editor
{
/// <summary>
/// Verifies PlayModeView RenderTexture field resolution used by GameViewBridge.
/// </summary>
public class GameViewBridgeTests
{
private class FakePlayModeView
{
#pragma warning disable CS0414 // Field is assigned for reflection-based tests only.
private object m_TargetTexture = "texture";
#pragma warning restore CS0414
}

private class FakeSimulatorWindow : FakePlayModeView
{
}

[Test]
public void ResolveTargetTextureField_WhenCalledOnDeclaringType_FindsPrivateBaseField()
{
// Verifies GetField on the PlayModeView declaring type finds m_TargetTexture.
FieldInfo field = GameViewBridge.ResolveTargetTextureField(typeof(FakePlayModeView));

Assert.That(field, Is.Not.Null);
Assert.That(field.Name, Is.EqualTo("m_TargetTexture"));
Assert.That(field.DeclaringType, Is.EqualTo(typeof(FakePlayModeView)));
}

[Test]
public void ResolveTargetTextureField_WhenDerivedTypeGetFieldMissesPrivateBaseField_DeclaringTypeStillResolves()
{
// Verifies the reflection pitfall: derived-type GetField cannot see private base fields.
FieldInfo fromDerived = typeof(FakeSimulatorWindow).GetField(
"m_TargetTexture",
BindingFlags.Instance | BindingFlags.NonPublic);
FieldInfo fromDeclaring = GameViewBridge.ResolveTargetTextureField(typeof(FakePlayModeView));

Assert.That(fromDerived, Is.Null);
Assert.That(fromDeclaring, Is.Not.Null);

FakeSimulatorWindow instance = new();
object value = fromDeclaring.GetValue(instance);
Assert.That(value, Is.EqualTo("texture"));
}

[Test]
public void GetRenderTexture_WhenMembersResolved_DoesNotThrow()
{
// Verifies PlayModeView type/method/field resolution completes without throwing.
Assert.DoesNotThrow(() => GameViewBridge.GetRenderTexture());
}
}
}
11 changes: 11 additions & 0 deletions Assets/Tests/Editor/GameViewBridgeTests.cs.meta

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

85 changes: 85 additions & 0 deletions Assets/Tests/Editor/ScreenshotWindowNameResolverTests.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
using NUnit.Framework;

using io.github.hatayama.UnityCliLoop.FirstPartyTools;
using io.github.hatayama.UnityCliLoop.ToolContracts;

namespace io.github.hatayama.UnityCliLoop.Tests.Editor
{
/// <summary>
/// Verifies window-name fallback from Game to Device Simulator for screenshot capture.
/// </summary>
public class ScreenshotWindowNameResolverTests
{
[Test]
public void ShouldFallbackToSimulator_WhenDefaultGameExactMisses_ReturnsTrue()
{
// Verifies the default Game exact lookup falls back when no window matched.
bool shouldFallback = ScreenshotWindowNameResolver.ShouldFallbackToSimulator(
UnityCliLoopConstants.SCREENSHOT_DEFAULT_WINDOW_NAME,
WindowMatchMode.exact,
matchCount: 0);

Assert.That(shouldFallback, Is.True);
}

[Test]
public void ShouldFallbackToSimulator_WhenGameAlreadyMatched_ReturnsFalse()
{
// Verifies fallback is skipped when the Game window was found.
bool shouldFallback = ScreenshotWindowNameResolver.ShouldFallbackToSimulator(
UnityCliLoopConstants.SCREENSHOT_DEFAULT_WINDOW_NAME,
WindowMatchMode.exact,
matchCount: 1);

Assert.That(shouldFallback, Is.False);
}

[Test]
public void ShouldFallbackToSimulator_WhenRequestedNameIsNotGame_ReturnsFalse()
{
// Verifies explicit non-Game names do not silently switch to Simulator.
bool shouldFallback = ScreenshotWindowNameResolver.ShouldFallbackToSimulator(
"Inspector",
WindowMatchMode.exact,
matchCount: 0);

Assert.That(shouldFallback, Is.False);
}

[Test]
public void ShouldFallbackToSimulator_WhenMatchModeIsNotExact_ReturnsFalse()
{
// Verifies prefix/contains Game lookups do not trigger Simulator fallback.
bool shouldFallback = ScreenshotWindowNameResolver.ShouldFallbackToSimulator(
UnityCliLoopConstants.SCREENSHOT_DEFAULT_WINDOW_NAME,
WindowMatchMode.contains,
matchCount: 0);

Assert.That(shouldFallback, Is.False);
}

[Test]
public void ResolveCaptureWindowName_WhenFallbackApplies_ReturnsSimulator()
{
// Verifies the resolved capture title becomes Simulator after a Game miss.
string resolved = ScreenshotWindowNameResolver.ResolveCaptureWindowName(
UnityCliLoopConstants.SCREENSHOT_DEFAULT_WINDOW_NAME,
WindowMatchMode.exact,
primaryMatchCount: 0);

Assert.That(resolved, Is.EqualTo(UnityCliLoopConstants.SCREENSHOT_SIMULATOR_WINDOW_NAME));
}

[Test]
public void ResolveCaptureWindowName_WhenFallbackDoesNotApply_ReturnsRequestedName()
{
// Verifies Resolve keeps the requested title when Game already matched.
string resolved = ScreenshotWindowNameResolver.ResolveCaptureWindowName(
UnityCliLoopConstants.SCREENSHOT_DEFAULT_WINDOW_NAME,
WindowMatchMode.exact,
primaryMatchCount: 1);

Assert.That(resolved, Is.EqualTo(UnityCliLoopConstants.SCREENSHOT_DEFAULT_WINDOW_NAME));
}
}
}
11 changes: 11 additions & 0 deletions Assets/Tests/Editor/ScreenshotWindowNameResolverTests.cs.meta

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Assets/Tests/Editor/UnityCLILoop.Tests.Editor.asmdef
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
"rootNamespace": "io.github.hatayama.UnityCliLoop.Tests.Editor",
"references": [
"GUID:214998e563c124e8a88199b2dd1f522d",
"GUID:5079a8d3a72924a81aa1cbc25f65ed1b",
"GUID:5c4588558a3624eacbce0f50007cf1eb",
"GUID:18e6dd30a99d14f32a045f79a2956f46",
"GUID:2fca906c8834d45aea12a5cc143c032d",
Expand Down
2 changes: 2 additions & 0 deletions Packages/src/Editor/FirstPartyTools/Raycast/Skill/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ unity_x = input_x
unity_y = gameViewHeight - input_y
```

- Device Simulator play view is supported. Prefer coordinates from `uloop screenshot --capture-mode rendering` (or its annotated `SimX`/`SimY`).

## Examples

```bash
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,8 @@ public static string[] GetOpenWindowNames()
return names.ToArray();
}

// Captures game rendering by reading GameView's composited RenderTexture (PlayMode only).
// Captures game rendering by reading the Play Mode view RenderTexture (PlayMode only).
// Works for both GameView and Device Simulator via PlayModeView.m_TargetTexture.
// Contains all cameras + Screen Space Overlay Canvas, without tab bar or borders.
internal static async Task<(Texture2D? texture, GameRenderingImageInfo renderingImageInfo, bool timedOut)> CaptureGameRenderingAsync(
float resolutionScale,
Expand All @@ -189,13 +190,13 @@ public static string[] GetOpenWindowNames()
RenderTexture rt = GameViewBridge.GetRenderTexture();
if (rt == null)
{
Debug.LogWarning("[EditorWindowCaptureUtility] GameView RenderTexture is not available");
Debug.LogWarning("[EditorWindowCaptureUtility] Play Mode view RenderTexture is not available");
GameRenderingImageInfo unavailableInfo = renderingImageInfo ??
CreateUnavailableGameRenderingImageInfo(Handles.GetMainGameViewSize());
return (null, unavailableInfo, false);
}

// GameView RenderTexture can be shorter than the full input area, so raw image Y needs this offset.
// Play Mode view RenderTexture can be shorter than the full input area, so raw image Y needs this offset.
GameRenderingImageInfo captureInfo = renderingImageInfo ??
CreateGameRenderingImageInfo(Handles.GetMainGameViewSize(), rt.width, rt.height);

Expand Down
Loading
Loading