Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .code-linter.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{

Check failure on line 1 in .code-linter.json

View workflow job for this annotation

GitHub Actions / code-linter / Code Linter

Policy Signature Required

Commit 37614f93 modified protected policy file '.code-linter.json' without a valid signature from a trusted code owner. Reason: Commit is unsigned (no cryptographic signature present).
"max_file_lines": 300,
"max_function_lines": 50,
"max_nesting_depth": 4,
Expand All @@ -13,6 +13,7 @@
".ci-gates",
".git",
"Tests~",
"Tests",
"tools~",
"Editor/tests",
"ThirdParty",
Expand Down
31 changes: 31 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -153,4 +153,35 @@ AUDIT_FINDINGS.md*
.mcp.json
.mcp.json.bak-*
CLAUDE.md
CLAUDE.md.meta
AGENTS.md.soma-backup-*

# Generated research / benchmark output (keep scripts and reports; ignore result dumps)
capture-validation-results.*
capture-validation-results.*.meta
capture_spikes_report.json
capture_spikes_report.json.meta
architecture-benchmark-results.*
architecture-benchmark-results.*.meta
agent-tasks-benchmark.json
agent-tasks-benchmark.json.meta
ambiguity-validation-results.json
ambiguity-validation-results.json.meta
nexus-mcp-tools.json
nexus-mcp-tools.json.meta
unity-cli-commands.json
unity-cli-commands.json.meta
unity-mcp-tools.json
unity-mcp-tools.json.meta
unity-pipeline-commands.json
unity-pipeline-commands.json.meta
captures/
captures.meta
stabilize-pipeline-capture-results.json
stabilize-pipeline-capture-results.json.meta
stabilize-domain-reload-results.json
stabilize-domain-reload-results.json.meta

# Local Assets-harness symlink so Unity Test Runner can load Tests~/Editor
EditModeTests/
EditModeTests.meta
1 change: 0 additions & 1 deletion .projectmem/.current_issue

This file was deleted.

12 changes: 10 additions & 2 deletions .projectmem/PROJECT_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Open source Unity Editor automation server for local AI tools and developer work
- Unity Package Manager package: `com.forkhorizon.nexus.unity` for Unity `6000.0+`.
- C# Editor implementation under `Editor/`, with runtime support under `Runtime/`.
- Python 3 MCP bridge under `Editor/nexus_unity_bridge.py` and `Editor/nexus_bridge/`.
- Tests: Unity EditMode tests under `Tests~/Editor/` and Python bridge tests under `Editor/tests/`.
- Tests: Unity EditMode tests under `Tests/Editor/` and Python bridge tests under `Editor/tests/`.
- Validation: `scripts/prepush-validate.sh`, GitHub Actions, and the .NET `tools~/NexusQualityGate` tool.

## Entry points
Expand All @@ -23,19 +23,27 @@ Open source Unity Editor automation server for local AI tools and developer work
- `Editor/` — Unity Editor server, raw tools, integration UI, and Python bridge.
- `MCPServer.cs` — server state/lifecycle; `MCPServer.Networking.cs` owns HTTP/WebSocket handling.
- `MCPServerMethods.cs` — tool dispatch; `MCPServerMethods.*.cs` group the raw tool implementations.
- `Capture/` — transport-independent Capture V2 (`ICaptureGateway`, DriverOwnedReadback, Game View source, JPEG/PNG encode).
- `Commands/` — canonical Nexus command descriptors and handlers (`nexus.project_map`, `nexus.group_compile_errors`, `nexus.capture_game_view`).
- `Pipeline/` — optional `UnityMCP.Editor.Pipeline` assembly with `[CliCommand]` wrappers; excluded when `com.unity.pipeline` is absent.
- `Runtime/` — transport adapters, Pipeline capability probe, and `legacy`/`pipeline`/`auto` runtime selection.
- `MCPServerMethods.HighValue.GameViewCapture.cs` — Legacy HTTP/MCP adapter that maps `capture_game_view_screenshot` onto `ICaptureGateway`.
- `NexusMcpConfigGenerator.cs` — generated MCP client configuration and bridge health checks.
- `nexus_unity_bridge.py` — MCP protocol entry point.
- `nexus_bridge/` — bridge transport, schemas, and manager routing.
- `tests/` — Python bridge unit tests.
- `Runtime/` — runtime assembly support used by Editor-facing features.
- `Tests~/Editor/` — Unity EditMode/API/security/integration configuration tests.
- `Tests/Editor/` — the single Unity EditMode/API/security/integration test suite.
- `scripts/` — package validation and optional agent-tooling smoke script.
- `tools~/NexusQualityGate/` — standalone .NET documentation and source-quality gate.
- `.github/workflows/validate.yml` — protected-branch CI validation and Unity package smoke.
- `README.md`, `DOCUMENTATION.MD`, `API_REFERENCE.MD`, `RELEASE.md`, `CHANGELOG.md` — public package and release documentation.

## Relationships
- `Editor/MCPServer.Networking.cs` validates a local request before `Editor/MCPServerMethods.cs` dispatches the raw JSON-RPC method on the Unity main thread.
- `Editor/MCPServerMethods.HighValue.GameViewCapture.cs` parses Legacy screenshot params, calls `Editor/Capture/CaptureGateway.cs`, then Base64-encodes `CaptureResult` into the existing JSON schema.
- `Editor/Commands/NexusLegacyCommandProjection.cs` registers canonical command handlers on the Legacy HTTP table; `Editor/Pipeline/NexusPipelineCommands.cs` calls the same handlers through `[CliCommand]` when Pipeline is present.
- `Editor/Runtime/NexusRuntimeHost.cs` selects Legacy vs Pipeline from `MCPSettings.RuntimeMode` and `Editor/Runtime/NexusRuntimeCapabilities.cs` without blocking Editor init.
- `Editor/nexus_unity_bridge.py` loads `Editor/nexus_bridge/routing.py`, which maps MCP manager tools to the raw Unity JSON-RPC API.
- `Editor/nexus_bridge/schemas.py` defines the public bridge contracts exercised by `Editor/tests/`.
- `Editor/NexusMcpConfigGenerator.cs` deploys the bridge and writes client configuration containing the current auth token.
Expand Down
26 changes: 23 additions & 3 deletions .projectmem/plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,33 @@
> never logged as events.

## Ideas
_Loose thoughts, not yet committed to._
- Dual backend now (Legacy HTTP/MCP + optional Unity Pipeline later); hybrid primary only after M4 gates.

## Active plans
_What we're working toward now. Use `- [ ]` / `- [x]` checklists._
- [x] M1 — Extract Capture V2 into transport-independent `Editor/Capture` behind `ICaptureGateway` and route `capture_game_view_screenshot` through it without changing the Legacy HTTP/MCP response schema.
- [x] M2 — Canonical commands + dual registration for `nexus.project_map`, `nexus.group_compile_errors`, `nexus.capture_game_view` only.
- [x] One command descriptor + one handler per POC command.
- [x] Legacy HTTP projection; Pipeline `[CliCommand]` wrappers optional.
- [x] No hard `com.unity.pipeline` dependency.
- [x] `capture_game_view_screenshot` HTTP schema unchanged.

- [x] M3 — Explicit Legacy and Unity Pipeline transport adapters; runtime setting stays `legacy` until M4.
- [x] `legacy` / `pipeline` / `auto` setting, default `legacy`; `auto` prefers Legacy.
- [x] Async Pipeline capability probe; never block init.
- [x] Explicit pipeline may skip 8081 when another project owns it.

- [x] M4 — `auto` prefers Pipeline on eligible installs; Legacy remains fallback; user can force Legacy.
- [x] Eligibility: Unity 6000, package, commands, session, healthy probe.
- [x] Default requested mode `auto`. HTTP is not removed.

- [x] M5 — Publish Legacy HTTP sunset timeline. Do not remove HTTP. Do not mark fully deprecated while CLI/Pipeline are pre-release.
- [x] Gate ledger in `get_server_status`.
- [x] Docs timeline; Legacy remains fully supported.

- [x] Stabilization / acceptance pass. M4 ACCEPTED (EditMode 152/152, two-editor A/B, 20 pending-GPU reloads, persistent MCP wait = 3 Editor ticks, overlay orientation). Legacy removal BLOCKED. Internal merge only, not RC. Do not start a new architecture.

## Next
_Queued, but not started._
- Legacy removal only after Unity CLI 1.0 stable, non-experimental Pipeline, and one Nexus stable release. Do not start.

## Someday / maybe

Expand Down
2 changes: 2 additions & 0 deletions .unity-quality-gate.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{

Check failure on line 1 in .unity-quality-gate.json

View workflow job for this annotation

GitHub Actions / code-linter / Code Linter

Policy Signature Required

Commit 37614f93 modified protected policy file '.unity-quality-gate.json' without a valid signature from a trusted code owner. Reason: Commit is unsigned (no cryptographic signature present).
"project": "",
"unity_path": "",
"include_paths": [
Expand All @@ -7,6 +7,8 @@
],
"exclude_paths": [
"Tests~/",
"Tests/",
"Research~/",
"tools~/",
"Plugins/"
],
Expand Down
21 changes: 18 additions & 3 deletions API_REFERENCE.MD
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Nexus Unity API Reference

Version: `1.5.0`
Version: `1.7.0` (development)

Nexus Unity exposes two supported public API surfaces:

- Raw HTTP JSON-RPC: 117 Unity Editor methods returned by `list_tools`.
- Raw HTTP JSON-RPC: 117 base Unity Editor methods plus 3 canonical Nexus commands, for 120 visible `list_tools` names in the current 1.7.0 development schema. The 3 `nexus_*` transport aliases remain dispatchable but hidden from the visible schema.
- MCP bridge: 14 consolidated `unity_` tools defined in `Editor/nexus_bridge/schemas.py`.

## Raw HTTP JSON-RPC
Expand Down Expand Up @@ -74,6 +74,9 @@ Schema compatibility notes:
- `list_player_prefs` / `unity_list_player_prefs`
- `capture_inspector_screenshot` / `unity_capture_inspector_screenshot`
- `capture_game_view_screenshot` / `unity_capture_game_view_screenshot`
- `nexus.project_map` / `unity_nexus.project_map`
- `nexus.group_compile_errors` / `unity_nexus.group_compile_errors`
- `nexus.capture_game_view` / `unity_nexus.capture_game_view`
- `generate_mermaid_diagram` / `unity_generate_mermaid_diagram`
- `semantic_find` / `unity_semantic_find`
- `enforce_forced_defaults` / `unity_enforce_forced_defaults`
Expand Down Expand Up @@ -157,7 +160,19 @@ Schema compatibility notes:
- `ui_click` / `unity_ui_click`
- `ui_input_text` / `unity_ui_input_text`

`batch_execute` accepts at most 50 requests and rejects nested `batch_execute` calls.
`batch_execute` accepts at most 50 requests and rejects nested `batch_execute` calls. Asynchronous methods (`capture_game_view_screenshot`, `nexus.capture_game_view`) cannot run inside a batch and return an error asking you to call them directly.

### Screenshot results

`capture_game_view_screenshot` and `capture_inspector_screenshot` return `success`, `message`, and `duration_ms`, plus a `data` object containing `width`, `height`, `format`, and `image_base64`. The default Game View format remains `png`. Optional Game View parameters are `format` (`png` or `jpg`/`jpeg`), `quality` / `jpeg_quality` (1–100, JPEG only), `width`, `height`, `max_long_edge`, and `include_telemetry`. Output size is capped at 8192 pixels on the longest edge. The legacy top-level `status`, `format`, and successful `image_base64` fields remain available for existing raw clients; Inspector responses also retain `ui_layout`.

Canonical high-level commands `nexus.project_map`, `nexus.group_compile_errors`, and `nexus.capture_game_view` (and their `nexus_*` aliases) share one handler and description per command. `nexus.capture_game_view` defaults to JPEG and returns `encoding`, `bytes`, `base64`, `source`, and stage timings. These three commands are also exposed as experimental Unity Pipeline `[CliCommand]`s when `com.unity.pipeline` is installed; that package is optional and is not a Nexus dependency.

`get_server_status` includes an additive `runtime` object: `requested` and `effective` are `legacy`, `pipeline`, or `auto`; `eligible` is true when Unity 6000, the Pipeline package, Nexus command registration, a session port file, and a healthy loopback probe all pass. The Project Settings **Runtime** mode defaults to `auto`, which prefers Pipeline on eligible installs and Legacy otherwise. Set the mode to `legacy` to force HTTP immediately. Explicit `pipeline` does not silently fall back to HTTP. HTTP remains supported and is not removed.

`runtime.legacy` publishes the sunset ledger: `deprecated` is false until maturity gates pass, `still_supported` is true, `sunset_status` is `announced`, and `removal` is `not_scheduled`. Removal requires Unity CLI 1.0 stable, non-experimental `com.unity.pipeline`, and one Nexus stable release after Pipeline-primary `auto`. `runtime.unity_cli` is informational (`detected`, `version`, `prerelease`, `stable_1_0_or_newer`); the Editor does not shell out to `unity --version`. `runtime.pipeline` includes `detected`, `version`, `experimental`, `supported`, plus health/session fields. `runtime.legacy_unavailable_reason` is `foreign_project` when another Unity project owns port 8081.

`list_tools` accepts optional `profile`: `core`, `visual`, `scene`, or `compat`. Canonical command ids are listed once; `nexus_*` aliases stay dispatchable without being advertised twice.

## MCP Bridge Tools

Expand Down
59 changes: 59 additions & 0 deletions BENCHMARKS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Nexus Unity Benchmarks

Current development line: `1.7.0`
Latest released version: `1.6.0`
Evidence date: 2026-09-22

This is the public benchmark ledger for Nexus Unity. It deliberately separates measured performance numbers from repeatable validation evidence. No latency, CPU, memory, or throughput improvement is claimed until a before/after measurement exists.

## Measurement rules

Every numeric benchmark should record:

- Unity version, package version, platform, branch, and commit;
- capture source, format, resolution, and warm/cold state;
- sample count, p50, p95, and outliers;
- CPU, memory, and allocation measurements when relevant;
- the baseline and comparison implementation.

## Current validation evidence

These are confirmed results, but they are not latency or resource benchmarks.

| Area | Configuration | Result |
|---|---|---|
| Unity EditMode suite | Parent interactive harness | 152 discovered, 152 passed, 0 failed, 0 skipped/inconclusive |
| Unity EditMode suite | Clean checkout batch mode | 152 discovered, 151 passed, 0 failed, 1 inconclusive because visible Game View is unavailable in batch mode |
| Python bridge | Static package validation | 43 tests passed |
| Capture blocking audit | Production Capture/Pipeline path | 0 `RequestIntoNativeArray`, `WaitForCompletion`, `Task.Result`, or `GetAwaiter().GetResult` patterns |
| Domain reload stability | Pending-GPU reload validation | 20/20 passed, 0 hangs |
| Multi-editor isolation | Two editors, Pipeline ports 7800 and 7801 | Passed; no Legacy HTTP 8081 bind fight and routing was correct |
| Clean compilation | Pipeline installed | Production and Pipeline assemblies compiled successfully |
| Clean compilation | Pipeline absent | Production assembly compiled with 0 C# errors; Pipeline assembly absent as expected |
| Capture smoke | Pipeline, PNG/JPEG, 1600×900 | Passed |
| API schema snapshot | Clean checkout | 120 visible tools, 3 hidden aliases, 29,160 UTF-8 bytes for the tools array |

## Numeric performance benchmarks

No numeric baseline has been recorded yet.

| Date | Scenario | Format | Resolution | Samples | p50 | p95 | CPU | RAM | Baseline |
|---|---|---|---:|---:|---:|---:|---:|---:|---|
| TBD | Game View capture | PNG | 1600×900 | TBD | TBD | TBD | TBD | TBD | TBD |
| TBD | Game View capture | JPEG | 1600×900 | TBD | TBD | TBD | TBD | TBD | TBD |
| TBD | Domain reload | Pending-GPU capture state | N/A | TBD | TBD | TBD | TBD | Previous capture implementation |

## What the current evidence supports

The current data supports these engineering claims:

- capture work no longer uses synchronous GPU waits on the Unity main thread;
- capture and domain reload paths are stable across the validated stress cases;
- two Unity editors can run concurrently without Legacy port contention;
- the package remains compilable with and without the optional Pipeline package.

The current data does **not** support claims of lower CPU use, lower memory use, higher capture throughput, or a percentage speed improvement.

## Future benchmark entry format

Add one row per scenario and keep the raw run output alongside the script or artifact that produced it. Recommended first numeric benchmark: 30 warm Game View captures for PNG and JPEG at 1600×900, reporting p50/p95 latency, peak RSS, and CPU time against the previous implementation.
7 changes: 7 additions & 0 deletions BENCHMARKS.md.meta

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 26 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,33 @@

All notable public changes to Nexus Unity are documented here.

## [Unreleased]
## [1.7.0] - Unreleased

### Added
- Canonical Nexus commands `nexus.project_map`, `nexus.group_compile_errors`, and `nexus.capture_game_view` (HTTP aliases `nexus_*`) share one handler and description each. When `com.unity.pipeline` is installed, optional `[CliCommand]` wrappers in `UnityMCP.Editor.Pipeline` project the same handlers; Pipeline is not a package dependency.
- Runtime setting `legacy` / `pipeline` / `auto` (default `auto`). `auto` prefers Unity Pipeline on eligible installs (Unity 6000, Pipeline package, registered Nexus commands, healthy session) and Legacy otherwise. Explicit `legacy` forces HTTP immediately. HTTP remains the fallback and is not removed.
- Published a Legacy HTTP sunset timeline: Legacy remains fully supported. Removal is not scheduled until Unity CLI 1.0 stable, non-experimental `com.unity.pipeline`, and one Nexus stable release after Pipeline-primary `auto`. `get_server_status.runtime.legacy` reports that ledger.

### Changed
- Package EditMode tests live under a single `Tests/Editor` suite (guarded by `UNITY_INCLUDE_TESTS`, so consumers only compile it when the test framework is installed and the package is listed in their `Packages/manifest.json` `testables`). Historical `Tests~/Editor` was merged into that suite and removed. The test asmdef uses Unity's `overrideReferences` + `Newtonsoft.Json.dll` / `nunit.framework.dll`.
- Runtime `get_server_status` now returns a main-thread-published snapshot. Requested mode is cached so the HTTP listener does not read EditorPrefs off-thread.
- Inspector/window screenshots use `EditorWindowPixelCapture` (ReadPixels). Game View remains Capture V2 DriverOwnedReadback only.
- Research reports and spike scripts live under `Research~/` and are not compiled.
- Extracted Game View capture into a transport-independent Capture V2 gateway (`ICaptureGateway`) using `AsyncGPUReadback.Request` plus `GetData<byte>()` (DriverOwnedReadback). Legacy `capture_game_view_screenshot` keeps its structured PNG JSON schema by default and now accepts optional `format`, `quality` / `jpeg_quality`, `width`, `height`, `max_long_edge`, and `include_telemetry` parameters. Canonical `nexus.capture_game_view` uses that same gateway with a JPEG default matching the hybrid proof of concept.
- Pipeline `[CliCommand]` Game View capture now returns `Task<NexusCaptureResult>` so Unity Pipeline can await GPU completion off the main thread. Production capture never calls `WaitForCompletion` or `.GetResult()` on the Unity main thread.
- `list_tools` can filter by profile (`core`, `visual`, `scene`, `compat`). Canonical ids are advertised once; transport aliases remain dispatchable.
- Pipeline health probing validates session PID liveness and Unity-like process identity, caches last-healthy state across domain reload, and reports `runtime.unity_cli` / `runtime.pipeline.detected|experimental|supported` without shelling out to the Unity CLI.

### Fixed
- Security: unauthenticated HTTP requests could bypass auth by containing `get_server_status` or `shutdown_server` anywhere in the body; only the actual JSON-RPC `method` is exempt now.
- Capture: readback timeouts no longer busy-loop or poll forever, encode failures no longer wedge the capture gate, the Game View wait resumes on the main thread, and capture size is capped at 8192 per edge.
- Runtime: Pipeline health probe retries and re-runs when the runtime mode changes; the Unity version gate accepts 6000 and newer.
- `batch_execute` reports async capture methods with a clear error; `find_objects` also matches the literal name; `list_tools` accepts array params.
- Reworked Game View, Inspector, and UI window screenshot capture to use Unity-native in-engine render textures and VisualElement captures with safe surface readback fallback instead of macOS `screencapture`.
- Added synchronous editor tracker rebuild for targeted Inspector captures (`instance_id`), resolved MSAA render textures, guarded against background desktop screen scraping via `isApplicationActive`, and provided structured PNG results while preserving legacy client compatibility.
- Added comprehensive screenshot stress-test tooling (`scripts/screenshot-stress-test.py`) and in-engine regression tests (`CaptureScreenshotStressTestRapidChurn`) covering rapid burst captures, multi-threaded swarms, selection churn invariants, and dynamic window geometry resizes.

## [1.6.0] - 2026-08-23
### Security
- Sanitize and redact sensitive exception messages in tool usage stats (`get_tool_usage_stats` and `RecordToolUsage`), replacing absolute filesystem paths, user directories, and multiline frames with generic placeholders and safe truncated summaries, and exposing `last_error_type` (#142).
- Require explicit confirmation (`confirm: true`) for `delete_asset`, enforce `AssetDatabase.MoveAssetToTrash` for OS trash recovery, and block creation, modification, moving, or deletion of `ProjectSettings/` and `Packages/` paths across all asset tools (#140).
Expand Down
Loading
Loading