Skip to content

fix: Raycast grid annotation no longer misses rotated or split colliders - #1666

Merged
hatayama merged 4 commits into
v3-betafrom
unify-raycast-annotation-clustering
Jul 9, 2026
Merged

hatayama merged 4 commits into
v3-betafrom
unify-raycast-annotation-clustering

Conversation

@hatayama

@hatayama hatayama commented Jul 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • screenshot --annotate-raycast-grid (without --raycast-layer-mask) now always uses the same dense clustered raycast pass as the masked mode, instead of a coarse 5x5 preview that could miss rotated or disconnected collider shapes entirely.
  • The response now reports RaycastLayerNamesChecked, the physics layer names actually eligible to produce hits in this response, to help diagnose why an expected layer produced no results.

User Impact

  • Previously, calling --annotate-raycast-grid without a layer mask sampled only 25 points on a coarse grid. A rotated or split collider could fall between those sample points and appear completely undetected, even though the same object was hittable via raycast --x --y directly — misleadingly suggesting the collider's hit detection itself was broken.
  • Now every --annotate-raycast-grid call uses the same dense 40x40 sampling and clustering used by the masked mode, so collider shapes are outlined accurately regardless of orientation or whether the mask is specified.
  • RaycastLayerSummaries (which layers exist and how many hits they have) is now always populated, not only when no mask is given.
  • The new RaycastLayerNamesChecked field tells the caller exactly which layers were checked against the active camera, making it possible to tell "this layer produced no PhysicsCollider entries because the camera can't see it" from "because nothing is there."

Changes

  • Unified ScreenshotUseCase.CaptureRenderingAsync onto a single clustering code path; layer mask defaults to Physics.DefaultRaycastLayers when --raycast-layer-mask is not given.
  • Added RaycastLayerMaskResolver.CreateLayerNamesFromMask to convert a layer bitmask into layer names, reused for the new RaycastLayerNamesChecked field.
  • Removed the coarse 5x5 grid path (RaycastGridPoints field, RaycastGridPointInfo DTO, and related dead code in RaycastGridAnnotator), replaced with the internal RaycastLayerHitSample type used only for layer-summary aggregation.
  • Updated skill docs (annotated-elements.md, SimulateMouseInput/SKILL.md and their generated mirrors), README.md, and default-tools.json tool descriptions to match.

Verification

  • dist/darwin-arm64/uloop compile — 0 errors, 0 warnings
  • uloop run-tests for RaycastGridAnnotatorTests (32/32), ScreenshotUseCaseTests (4/4), DefaultToolsCatalogDriftTests (1/1) — all green
  • Manual smoke test in PlayMode on RaycastAnnotationPerspectiveDemoScene: confirmed the previously-missed rotated/split collider is now correctly outlined, RaycastLayerNamesChecked reflects the expected layer set with and without --raycast-layer-mask, and RaycastLayerSummaries remains populated in both cases

Review in cubic

hatayama added 2 commits July 10, 2026 00:21
--annotate-raycast-grid にはマスク無し時の5x5粗いグリッド予覧(RaycastGridPoints)と
マスク指定時の40x40密クラスタ化(輪郭付きPhysicsCollider)という2つの排他モードがあり、
5x5サンプリングが回転/分割コライダーを取りこぼして「当たり判定が壊れている」と誤解される
事態が発生した。実際には同じ40x40密データをRaycastLayerSummaries用に既に取得していたため、
粗い可視化を返す理由がなかった。

隣ペインのFable 5にセカンドオピニオンを求め、常時クラスタ化に統一する方針で合意。
マスク無し時はPhysics.DefaultRaycastLayers(Ignore Raycastのみ除外)でクラスタ化し、
RaycastLayerSummariesはマスクの有無を問わず常に返す。レスポンスJSONには実際にクラスタ化
対象になったレイヤー名一覧(RaycastLayerNamesChecked = 使用マスク & Camera.main.cullingMask)
を新規に追加した。

- RaycastGridPointInfo(5x5専用DTO)をToolContractsから削除し、RaycastGridAnnotator内部の
  RaycastLayerHitSample(Hit/HitLayerIndex/HitLayer/HitGameObjectPathの4フィールドのみ)に置換
- CollectRaycastGridPoints/CalculateGridInputPosition/CreateOverlayElementsの5x5専用デッドコードを削除
- RaycastLayerMaskResolver.CreateLayerNamesFromMaskを追加し、ビットマスクからレイヤー名一覧を生成
annotated-elements.mdからRaycastGridPointsセクションを削除し、
RaycastLayerSummariesが常時返る仕様とRaycastLayerNamesChecked
新フィールドの説明を追加。SimulateMouseInputのSKILL.mdと
default-tools.json、READMEからも5x5グリッド専用の言及を除去し、
AnnotatedElements[].SimX/SimYへの案内に統一した。

.claude/.agentsのミラーはuloop skills installで再生成し、
canonicalファイルとの差分が無いことを確認済み。
@coderabbitai

coderabbitai Bot commented Jul 9, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@hatayama, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 34 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: f771da97-7f81-4492-8b65-1ecb7a424f90

📥 Commits

Reviewing files that changed from the base of the PR and between d947a5e and 76a59db.

📒 Files selected for processing (1)
  • Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs
📝 Walkthrough

Walkthrough

Refactors raycast grid annotation from RaycastGridPointInfo-based point collection to dense RaycastLayerHitSample-based sampling. Removes RaycastGridPoints from screenshot output, adds RaycastLayerNamesChecked field, introduces CreateLayerNamesFromMask helper, and updates docs/schemas to reference AnnotatedElements[].SimX/SimY instead of grid point coordinates.

Changes

Raycast Layer Summary Refactor

Layer / File(s) Summary
Contract removal and new data container
Packages/src/Editor/ToolContracts/RaycastGridPointInfo.cs, Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs
Removes the RaycastGridPointInfo public contract class and introduces the internal RaycastLayerHitSample container (Hit, HitGameObjectPath, HitLayer, HitLayerIndex).
Aggregation and mask-to-name resolution
RaycastGridAnnotator.cs, RaycastLayerMaskResolver.cs
Updates CreatePointInfo/CreateLayerSummaries to build/aggregate RaycastLayerHitSample data and adds CreateLayerNamesFromMask to convert a layer bitmask into layer names.
ScreenshotUseCase output wiring
Packages/src/Editor/FirstPartyTools/Screenshot/ScreenshotUseCase.cs, Packages/src/Editor/ToolContracts/ScreenshotResponse.cs
Adds RaycastLayerNamesChecked field to ScreenshotInfo, reworks the AnnotateRaycastGrid flow to compute an effective clustering mask and checked layer names, removes grid overlay elements and RaycastGridPoints population.
Test suite updates
Assets/Tests/Editor/RaycastGridAnnotatorTests.cs
Renames grid-position tests to target CalculateGridInputPositionForGrid, removes an obsolete overlay test, adds three new CreateLayerNamesFromMask tests, and updates layer-summary tests to use RaycastLayerHitSample inputs.
Documentation and schema updates
.agents/skills/..., .claude/skills/..., Packages/src/Editor/FirstPartyTools/Screenshot/Skill/..., Packages/src/Editor/FirstPartyTools/SimulateMouseInput/Skill/SKILL.md, README.md, cli/common/tools/default-tools.json
Updates skill docs, README, and tool schema descriptions to reflect always-populated RaycastLayerSummaries, the new RaycastLayerNamesChecked field, and removal of RaycastGridPoints coordinate references in favor of AnnotatedElements[].SimX/SimY.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

  • hatayama/unity-cli-loop#1473: Both PRs modify RaycastGridAnnotator and its tests within the same raycast-grid/screenshot coordinate-verification pipeline.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main fix: raycast grid annotation no longer misses rotated or split colliders.
Description check ✅ Passed The description accurately matches the changeset, covering the dense raycast pass, new layer metadata, and removed coarse-grid path.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch unify-raycast-annotation-clustering

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs (2)

125-142: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider renaming CreatePointInfo to reflect the new RaycastLayerHitSample return type.

The method now creates RaycastLayerHitSample, not the removed RaycastGridPointInfo. The name CreatePointInfo is stale and could mislead future readers. Consider CreateLayerHitSample or CreateHitSample.

♻️ Proposed rename
-private static RaycastLayerHitSample CreatePointInfo(GameViewRaycastResult raycastResult)
+private static RaycastLayerHitSample CreateLayerHitSample(GameViewRaycastResult raycastResult)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs`
around lines 125 - 142, The method name CreatePointInfo is stale now that it
returns RaycastLayerHitSample, so rename it to match the actual purpose and
return type. Update the identifier used in RaycastGridAnnotator to something
like CreateLayerHitSample or CreateHitSample, and make sure any call sites or
references are updated to the new name for consistency.

31-63: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

CollectRaycastGridPointsForGrid duplicates the grid-iteration and raycast logic of CollectClusterSamples.

Both methods iterate the same 40×40 grid, call CalculateGridInputPositionForGrid, and invoke RaycastFromInputPosition. They differ only in mask (hardcoded vs parameter) and how results are stored (all samples vs hits-only). Extracting a shared dense-raycast helper that returns raw GameViewRaycastResult[] would eliminate the duplication and ensure both passes stay in sync if grid dimensions or raycast parameters change.

Additionally, the method name CollectRaycastGridPointsForGrid is redundant ("Grid…ForGrid") and references the removed RaycastGridPointInfo concept. Consider CollectDenseRaycastSamples or CollectLayerHitSamples.

♻️ Proposed refactor: extract shared grid-raycast helper
 private static List<RaycastLayerHitSample> CollectRaycastGridPointsForGrid(
     Vector2 renderingImageSize,
     int imageToInputOffsetY,
     int rowCount,
     int columnCount,
     int layerMask)
 {
-    List<RaycastLayerHitSample> samples = new List<RaycastLayerHitSample>();
-    Physics.SyncTransforms();
-
-    for (int row = 1; row <= rowCount; row++)
-    {
-        for (int column = 1; column <= columnCount; column++)
-        {
-            Vector2 inputPosition = CalculateGridInputPositionForGrid(
-                renderingImageSize,
-                imageToInputOffsetY,
-                rowCount,
-                columnCount,
-                row,
-                column);
-            GameViewRaycastResult raycastResult = GameViewRaycastUtility.RaycastFromInputPosition(
-                inputPosition,
-                UnityCliLoopConstants.RAYCAST_DEFAULT_MAX_DISTANCE,
-                layerMask,
-                false);
-
-            samples.Add(CreatePointInfo(raycastResult));
-        }
-    }
-
-    return samples;
+    List<GameViewRaycastResult> results = RaycastDenseGrid(
+        renderingImageSize, imageToInputOffsetY, rowCount, columnCount, layerMask);
+    return results.ConvertAll(r => CreatePointInfo(r));
 }

+private static List<GameViewRaycastResult> RaycastDenseGrid(
+    Vector2 renderingImageSize,
+    int imageToInputOffsetY,
+    int rowCount,
+    int columnCount,
+    int layerMask)
+{
+    List<GameViewRaycastResult> results = new List<GameViewRaycastResult>(rowCount * columnCount);
+    Physics.SyncTransforms();
+    for (int row = 1; row <= rowCount; row++)
+    {
+        for (int column = 1; column <= columnCount; column++)
+        {
+            Vector2 inputPosition = CalculateGridInputPositionForGrid(
+                renderingImageSize, imageToInputOffsetY, rowCount, columnCount, row, column);
+            results.Add(GameViewRaycastUtility.RaycastFromInputPosition(
+                inputPosition, UnityCliLoopConstants.RAYCAST_DEFAULT_MAX_DISTANCE, layerMask, false));
+        }
+    }
+    return results;
+}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs`
around lines 31 - 63, The grid raycast logic in CollectRaycastGridPointsForGrid
is duplicated from CollectClusterSamples, so extract a shared helper that
performs the grid iteration and RaycastFromInputPosition calls and returns the
raw GameViewRaycastResult values for both callers. Keep
CalculateGridInputPositionForGrid as the single source for grid coordinates, and
let each method handle its own filtering/storage (all samples vs hits-only)
after the shared raycast pass. Also rename CollectRaycastGridPointsForGrid to a
clearer, non-redundant name such as CollectDenseRaycastSamples or
CollectLayerHitSamples to match the updated behavior and avoid the old
RaycastGridPointInfo terminology.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@Packages/src/Editor/FirstPartyTools/Screenshot/ScreenshotUseCase.cs`:
- Around line 95-113: `ScreenshotUseCase` is building layer summaries without
respecting the resolved raycast mask, so the summary output can disagree with
`RaycastLayerNamesChecked`. Update the `CollectRaycastLayerSummaries` call in
`ScreenshotUseCase` to pass `effectiveLayerMask`, and thread that mask through
`RaycastGridAnnotator.CollectRaycastGridPointsForGrid` instead of hardcoding
`Physics.DefaultRaycastLayers`. Make the matching signature changes in
`RaycastGridAnnotator` and any helpers so all raycast-related results are
filtered by the same mask.

---

Nitpick comments:
In
`@Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs`:
- Around line 125-142: The method name CreatePointInfo is stale now that it
returns RaycastLayerHitSample, so rename it to match the actual purpose and
return type. Update the identifier used in RaycastGridAnnotator to something
like CreateLayerHitSample or CreateHitSample, and make sure any call sites or
references are updated to the new name for consistency.
- Around line 31-63: The grid raycast logic in CollectRaycastGridPointsForGrid
is duplicated from CollectClusterSamples, so extract a shared helper that
performs the grid iteration and RaycastFromInputPosition calls and returns the
raw GameViewRaycastResult values for both callers. Keep
CalculateGridInputPositionForGrid as the single source for grid coordinates, and
let each method handle its own filtering/storage (all samples vs hits-only)
after the shared raycast pass. Also rename CollectRaycastGridPointsForGrid to a
clearer, non-redundant name such as CollectDenseRaycastSamples or
CollectLayerHitSamples to match the updated behavior and avoid the old
RaycastGridPointInfo terminology.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: db0b8af5-6cdc-4a68-ba40-cab2cf7c97fe

📥 Commits

Reviewing files that changed from the base of the PR and between ae13a21 and d947a5e.

⛔ Files ignored due to path filters (1)
  • Packages/src/Editor/ToolContracts/RaycastGridPointInfo.cs.meta is excluded by none and included by none
📒 Files selected for processing (14)
  • .agents/skills/uloop-screenshot/references/annotated-elements.md
  • .agents/skills/uloop-simulate-mouse-input/SKILL.md
  • .claude/skills/uloop-screenshot/references/annotated-elements.md
  • .claude/skills/uloop-simulate-mouse-input/SKILL.md
  • Assets/Tests/Editor/RaycastGridAnnotatorTests.cs
  • Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastGridAnnotator.cs
  • Packages/src/Editor/FirstPartyTools/Screenshot/RaycastAnnotation/RaycastLayerMaskResolver.cs
  • Packages/src/Editor/FirstPartyTools/Screenshot/ScreenshotUseCase.cs
  • Packages/src/Editor/FirstPartyTools/Screenshot/Skill/references/annotated-elements.md
  • Packages/src/Editor/FirstPartyTools/SimulateMouseInput/Skill/SKILL.md
  • Packages/src/Editor/ToolContracts/RaycastGridPointInfo.cs
  • Packages/src/Editor/ToolContracts/ScreenshotResponse.cs
  • README.md
  • cli/common/tools/default-tools.json
💤 Files with no reviewable changes (1)
  • Packages/src/Editor/ToolContracts/RaycastGridPointInfo.cs

Comment thread Packages/src/Editor/FirstPartyTools/Screenshot/ScreenshotUseCase.cs
hatayama added 2 commits July 10, 2026 00:51
戻り値の型がRaycastGridPointInfoからRaycastLayerHitSampleに変わった際、
メソッド名が古い型名を指したまま残っていた。CodeRabbitのレビュー指摘を
受け、実際の戻り値の型に合わせて改名した。
粗い5x5モードが廃止されたことで"GridPointsForGrid"という名前は
過去DTO(RaycastGridPointInfo)への冗長な参照になっていた。
実際の役割は「密グリッド全点でレイヤーヒットを標本化する」ヘルパー
なのでCollectLayerHitSamplesにリネームし、レビュアーからの
可読性指摘に対応。呼び出し側はCollectRaycastLayerSummaries1箇所のみ。
@hatayama

hatayama commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

Disposition of CodeRabbit findings

Reviewed all 3 findings against current code, spec, and design intent. Two accepted (both applied), one rejected with rationale.

1. [Major] RaycastLayerSummaries ignores effectiveLayerMask — REJECTED. Replied inline. The two fields are intentionally different: RaycastLayerNamesChecked = what was actually clustered (mask ∩ cullingMask, diagnostic), RaycastLayerSummaries = always over Physics.DefaultRaycastLayers (constant discovery aid, "what else could I filter to next"). Making the summary respect the mask would defeat its purpose. Documented explicitly in annotated-elements.md.

2. [Nitpick] Rename CreatePointInfo → CreateLayerHitSample — ACCEPTED, applied in db9b4d9. Stale name after return-type change.

3. [Nitpick] Extract shared helper + rename CollectRaycastGridPointsForGrid — SPLIT:

  • Extract shared helper: REJECTED. The two callers differ meaningfully (fixed vs param mask, all-samples vs hits-only), and the codebase follows YAGNI — introducing a shared helper for hypothetical future consistency adds indirection now without a concrete need.
  • Rename: ACCEPTED, applied in 76a59db. CollectRaycastGridPointsForGrid → CollectLayerHitSamples. The "GridPointsForGrid" name referred to the now-removed RaycastGridPointInfo concept.

Verification after changes: uloop compile clean (0 errors, 0 warnings), RaycastGridAnnotatorTests 32/32 pass.

@hatayama

hatayama commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

Addressed CodeRabbit's review:

  • Actionable comment (RaycastLayerSummaries vs effectiveLayerMask): Not applied — replied inline explaining this is intentional. RaycastLayerSummaries is documented to always report against the fixed Physics.DefaultRaycastLayers set as a constant "what else could I filter to" discovery aid, independent of --raycast-layer-mask. RaycastLayerNamesChecked is the separate field that tracks the effective mask. Making them follow the same mask would defeat the discovery-aid purpose.
  • Nitpick: rename CreatePointInfo: Applied → CreateLayerHitSample (db9b4d9).
  • Nitpick: rename CollectRaycastGridPointsForGrid: Applied → CollectLayerHitSamples (76a59db).
  • Nitpick: extract shared dense-raycast helper to dedupe with CollectClusterSamples: Not applied. This duplication pre-dates this PR and isn't introduced by this diff; CodeRabbit itself flagged it as trivial/low-value. Out of scope for this fix, per project scope discipline.

uloop compile (0 errors/warnings) and RaycastGridAnnotatorTests (32/32) re-verified after the renames.

@hatayama
hatayama merged commit a92a8fa into v3-beta Jul 9, 2026
10 checks passed
@hatayama
hatayama deleted the unify-raycast-annotation-clustering branch July 9, 2026 16:20
@github-actions github-actions Bot mentioned this pull request Jul 11, 2026
RyanXie123 pushed a commit to RyanXie123/unity-cli-loop that referenced this pull request Sep 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant