Repository navigation
test(editor): establish repeatable large-workload performance acceptance #38
Description
Verified on branch / awaiting merge
Production implementation: dd5979c613b0fe8ebd9f7026f25001ceee9f8cee. Delivered head: dbae93e8f211d45a69cdffdb6afa3a33a0af0fa5; the subsequent diff contains documentation/acceptance evidence only. Exact-head CI: Engine CI and NuGet dependency graph passed; all six checks on this exact head are successful. All applicable acceptance conditions for this issue are verified; unavailable physical/manual device checks are explicitly scoped, not reported passed.
Owner: Controlled public-host static/input/command/product measurements. Implementation commits: 973a1aa7, bc88dc04, 1917b253.
Evidence: complete report, exact-head delivery attachment, portable test-case/raw-measurement/requirement indexes linked by the report. Specific proof: Frozen input/runtime hashes before/after; 150 five-pair original/current trial rows; Ten absolute complete-inspection processes; All regression flags require explicit review.
Commands: official tests/Invoke-EditorUiAcceptance.ps1 Contracts/Workflows/Visual/Lifecycle/Products with SDK-selected independent outputs; complete source/entry preparation; formal InnoEngine.Build scripts --ide for the three companions; tests/Invoke-WikiChecks.ps1; controlled public performance fixtures where applicable. The report records original command owners, TRX/binlogs, negative tests, failures, skips, input hashes and device limits.
Profiles overlap; do not sum their pass counts into a distinct-test claim. Windows-only sharing fixtures and Unix-only filesystem fixtures are scoped to the executing host. Real native contexts/simulated display metrics and hidden product feature journeys are not a claim of physical display, OS IME or manual Mac testing. No BGCS/vendor/package/generated-binding modification, parallel Core system, test hook, IVT or private reflection was used.
Keep this issue Open. PR #26 includes an individual Fixes #38 and will close it only when the user merges to master. No automatic merge or premature master-resolution claim.
Original defect, ownership and accepted work
The following baseline observations describe the issue as opened. The verification above supersedes intermediate pending checkpoints; original failed/negative logs remain preserved. Acceptance checkboxes are checked only after the exact delivery-head verification completes.
Parent and original requirement
Parent: #27. Work item E11 from the approved Editor UI closure plan and its original implementation/acceptance requirements.
Baseline facts and risks at issue creation
The existing UTF-8 microbenchmark proves an isolated buffer improvement, not original-plan UI CPU/allocation/GC/median/P95 requirements.
Owner and dependency boundary
Existing public Editor/native test Host, deterministic workload fixtures and fixed Windows measurement environment.
Dependencies: #29, #32, #33, #34.
Implementation
- Freeze current reviewed-source baseline before optimizations and reconstruct the original pre-refactor baseline with matching input identities.
- Measure FlappyBird normal layout, 10,000-item tree/assets and deterministic 2,000-node graphs.
- Cover idle, scroll, search, property drag, menu opening and cold versus warm startup.
- Use same machine/SDK/configuration/driver/window/DPI/Zoom and at least five paired alternating valid repeats.
- Keep all raw CPU/allocation/GC/frame median/P95 and work-count data.
Acceptance
- Functional results agree and hot-path allocation does not increase.
- Locate and fix reproducible CPU/P95 regressions exceeding 10 percent.
- Repeat noisy trials without selecting only the best sample.
- Do not reduce correctness, validation or diagnostic visibility to pass.
- Missing original baseline/input reconstruction is an explicit blocker, never replaced by microbenchmarks.
- Shared CI runners do not substitute for controlled machine performance.
Failure and lifecycle constraints
Preserve diagnostics, cancellation, atomic rollback/publication, Missing recovery and generation retirement. Reuse Core Events, Identity, IO, Serialization and History. Do not modify BGCS/vendor/generated code or add global/implicit imports, compatibility paths, IVT or private reflection.
Evidence and closure
Commits, exact commands, logs/TRX/data and remaining limits will be added during implementation. Until all acceptance passes this issue remains open. After verification mark Verified on branch / awaiting merge; add Fixes #38 with this issue's actual number to PR #26 only when complete. Close on merge to master, not on branch implementation.
Activity
FLwolfy commentedon Oct 27, 2026
Original-baseline input blocker is now reproduced through the official workflow. Unchanged b303d9f production source and its Build entry compile successfully; the formal baseline shader-tool build also succeeds (2,942 original vendor warnings, zero errors). Running that original Build entry's scripts/IDE workflow against a frozen copy of the exact current FlappyBird Assets/Plugins fails because current companion editor scripts require EditorUiSession, which the original API did not expose. Evidence: e11/original-flappy-projection.log; no compatibility shim or script rewrite was applied. Therefore identical-input original-vs-current FlappyBird product performance cannot currently be established, as distinct from a failed baseline engine build. This part remains Blocked under the approved E11 rule; deterministic shared-input native tree/list/graph comparisons continue. The PR cannot claim complete E11 acceptance from those narrower results.
FLwolfy commentedon Oct 28, 2026
The final controlled frame matrix is complete: 30 successful alternating original/current processes (five pairs each for hierarchy/assets/shader), plus five menu/statistics process pairs. All 12,778 frozen input/dependency files (583,137,380 bytes) match before/after. Median/P95: hierarchy 5117.21/5594.09 -> 345.14/402.69 ms; assets 4104.39/4320.28 -> 18.73/27.58 ms; shader 29.01/33.17 -> 28.78/33.39 ms. Shader P95 is +0.65%, CPU unchanged; all sampled allocations decrease. The previous relative graph regression is resolved. This does not close E11: full same-input original/current FlappyBird remains blocked, the complete interaction/cold-start matrix remains unfinished, and #46 tracks the poor absolute hierarchy cost. Code commit f7dce89 has passed managed Windows/macOS and dependency CI; full native CI is still running. Raw data and identity receipts are being added under records/architecture/acceptance/editor-ui-performance-2026-10-28. No GPU or whole-product performance claim.
FLwolfy commentedon Oct 28, 2026
Continuation plan after recovering the companion history: the prior report treated the current Rendering2D editor implementation as an immutable external input to the old engine. The actual UI change includes Rendering2D commit 067d4eb (editor sources/README only) and Samples commit 3e331fd (only the installed Rendering2D package). I will recover the consistent pre-change producer set (engine b303d9f, Rendering2D 3e8fafe, Samples 6eb52f3, unchanged Canvas eed5fd9) and verify byte-identical gameplay/assets/settings inputs and unchanged plugin runtime content before comparing the complete product. The companion editor-code/package identity differences will be explicit measured implementation changes, not hidden input substitutions. No compatibility shim or baseline production edit will be added. A standalone test-owned composition harness may set the existing public window visibility option to false so both historical and current Hosts run without desktop interaction. The previous failed same-current-package replay remains recorded; recovered multi-repository before/after results will be reported separately and only accepted if their input/implementation classification is demonstrated.
FLwolfy commentedon Oct 29, 2026
Continuation review found that the existing paired harness measured mostly static views. This does not complete the approved scrolling/search/continuous-edit workload matrix. The branch now adds actual native wheel/search input and an owned continuous Scene edit gesture followed by real panel drawing; no production testing API or alternate History path is introduced. Three initial cases passed. The search preflight exposed incomplete native key/character event settling in the fixture; its failed TRX is retained, and the corrected public-input sequence is being verified before controlled measurements. The historical producer uses its own public transaction API in the test-owned benchmark, rather than adding a compatibility layer to production. This issue remains Open and PR #26 remains Draft until the complete same-input, five-pair matrix and final gates pass.
FLwolfy commentedon Oct 30, 2026
Final-candidate quantitative review — unresolved graph tail
At
dd5979c613b0fe8ebd9f7026f25001ceee9f8cee, the complete controlled matrix produced 150 trial rows and 68 comparisons. All 16,406 frozen input/runtime files (3,207,829,122 bytes) match before/after. The two review flags are the same 2,000-node graph tail: median of per-process P95/max 29.9459 ms original versus 35.9288 ms current (+19.98%, three of five pairs above +10%). Median frame time is +3.33%, measured CPU median unchanged and allocation -24.91%; those improvements do not waive the tail flag. Preserve all raw records undercandidate-dd5979c6-quantitative.Each initial process measured ten frames, making nearest-rank P95 exactly its maximum. Before making a production change or accepting a noise explanation, run five additional alternating original/current process pairs through the same existing public fixture, using its normal five repetitions (50 measured frames per process, same five warm-up frames). Preserve every frame, CPU/allocation/GC/work counts, unchanged inputs and all outliers. Predeclare pooled per-process nearest-rank P95 and the same +10% repeatability rule; do not choose the best repetition or replace the initial data. If the repeat establishes a persistent regression, profile and correct its actual owner without reducing validation, changing APIs for timing or weakening architecture. Until this review resolves, this issue remains Open and PR #26 remains Draft.