Repository navigation
perf(serialization): reuse explicitly declared neutral capture keys #53
Description
Verified on branch / awaiting merge
Production implementation: dd5979c613b0fe8ebd9f7026f25001ceee9f8cee. Delivered head: dbae93e8f211d45a69cdffdb6afa3a33a0af0fa5; the subsequent diff contains documentation/acceptance evidence only. Exact-head CI: Engine CI and NuGet dependency graph passed; all six checks on this exact head are successful. All applicable acceptance conditions for this issue are verified; unavailable physical/manual device checks are explicitly scoped, not reported passed.
Owner: Core immutable snapshot/Writer proof, Scene complete neutral row inputs. Implementation commits: be44532f, 2434cd96, 1917b253.
Evidence: complete report, exact-head delivery attachment, portable test-case/raw-measurement/requirement indexes linked by the report. Specific proof: Core97: callback retirement/thread/metadata/identity/converter/effect/mutation/GC guards; Scene67: complete writes/restore/references/Missing/Prefab and removed-System GC; Ten complete-inspection processes, architecture/Wiki/products/CI.
Commands: official tests/Invoke-EditorUiAcceptance.ps1 Contracts/Workflows/Visual/Lifecycle/Products with SDK-selected independent outputs; complete source/entry preparation; formal InnoEngine.Build scripts --ide for the three companions; tests/Invoke-WikiChecks.ps1; controlled public performance fixtures where applicable. The report records original command owners, TRX/binlogs, negative tests, failures, skips, input hashes and device limits.
Profiles overlap; do not sum their pass counts into a distinct-test claim. Windows-only sharing fixtures and Unix-only filesystem fixtures are scoped to the executing host. Real native contexts/simulated display metrics and hidden product feature journeys are not a claim of physical display, OS IME or manual Mac testing. No BGCS/vendor/package/generated-binding modification, parallel Core system, test hook, IVT or private reflection was used.
Keep this issue Open. PR #26 includes an individual Fixes #53 and will close it only when the user merges to master. No automatic merge or premature master-resolution claim.
Original defect, ownership and accepted work
The following baseline observations describe the issue as opened. The verification above supersedes intermediate pending checkpoints; original failed/negative logs remain preserved. Acceptance checkboxes are checked only after the exact delivery-head verification completes.
Approved-plan mapping
Parent #27; required by #46, E07 #34 and E11 #38. This issue owns the minimal Core serialization boundary needed to eliminate redundant complete large-scene materialization without weakening authoritative state detection.
Baseline confirmed facts and risks
Complete due inspection still takes roughly 35–43 ms before the independent experiment. A historical whole-root prototype demonstrated lower idle cost but still incurred roughly 70 ms and 16.8 MB after an edit. The current branch instead certifies complete array-element inputs through the existing Writer and memoizes only bounded immutable canonical blocks. Preliminary ordinary-producer edit medians are 12.8–14.7 ms and about 2.51 MB; these do not replace final quiet process pairs. Earlier interfaces, primitive slots, context projections, property-only shortcuts and cached graph-ID maps are not retained.
Owners and dependency direction
- Core Serialization owns the borrowed builder, exact neutral key comparison, metadata/Converter guards, immutable snapshot hints and publication validation.
- Core Identity remains the only object authority; keys retain domain-qualified runtime values without resolving or retaining live objects.
- The metadata provider certifies actual pure primitive fields and explicit complete source revision access.
- Scene owns its existing topology/hierarchy, GameObject metadata/activation writes and Transform local/world/reset/restore writes.
- Editor Workspace reads current source catalog status and continues authoritative dirty/saved-baseline handling.
No UI/backend/platform branch, event system, identity registry, serializer, persistence format or global pool is introduced. BGCS/vendor/packages/generated bindings are unchanged.
Implementation plan and files
- Add an optional protected Converter capture-key callback, declining by default. Regular Serialize never uses it; arbitrary converters keep normal capture.
- Add a synchronous borrowed Core builder for capacity, owner revisions, proven property revisions and exact neutral text/numeric inputs. Failed TryAdd invalidates the entire key. Retire operation/context/metadata references on every outcome and reject later/cross-thread use.
- Reuse the current immutable snapshot/node pipeline. Guard the key by current Core runtime identity, immutable metadata and pinned Converter generation. Recheck a changed key after the normal write; changed inputs reject publication. Never mutate a prior node/key.
- Add explicit metadata capability methods and a revision-member declaration attribute. Sealed identity sources use private nonserialized nullable counters; exhausted/unavailable counters decline reuse. No public Transform interface/property is added.
- Extend the existing WriteObjectArray operation with an optional borrowed capture-key callback. Scene declares complete per-object rows: owner revision, source GUID, included parent source GUID, sibling position and every eligible component revision/source identity. The root, ordering and Systems retain ordinary writing. Missing/Prefab/untracked/contextual rows decline only their own reuse. Exact GUID values are compared without lossy hashes.
- Prevent enclosing-key publication whenever nested converters execute, including independently captured property payloads. Use only a transient neutral effect marker; arbitrary converters retain normal evaluation.
- Let the same canonical encoder memoize keyed immutable object blocks up to 64 KiB. Larger blocks continue streaming; prior snapshots remain immutable and public output arrays are independent.
- Cover every public and internal metadata/activation/local/world/reset/restore mutation. Current catalog queries do not import source data during label/dirty queries; source reconciliation remains the Asset owner.
- Update actual consumers, scripting exports, project references, XML, Wiki, architecture checks and the original-plan ledger together.
src/foundation/core/Inno.Core.Serialization/
Capture/SerializationCaptureKeyBuilder.cs
Metadata/SerializationRevisionAttribute.cs
Metadata/SerializationTypeMetadata.cs
Converters/SerializationConverter.cs
Internal/{ConverterRegistry,SerializationOperation,SerializationNode,ValuePipeline}.cs
Properties/ScriptingApi.cs
backends/DotNet/runtime/Inno.Adapter.Serialization.DotNet/
ReflectionSerializationMetadataSource.cs
Discovery/ReflectionStructMetadata.cs
src/content/scene/Inno.Scene/
Core/GameObject.cs
Concrete/Transform.cs
Serialization/{Converters/GameSceneConverter,Graph/SceneGraphSerialization}.cs
src/composition/editor/features/Inno.Editor.Scene/Documents/
EditorSceneWorkspace.SourceChanges.cs
tests/core/Inno.Core.Serialization.Tests/
tests/scene/Inno.Scene.Tests/
tests/editor/Inno.Editor.PlayMode.Tests/
docs/core/Inno.Core.Serialization.md
records/architecture/acceptance/
Acceptance
- Canonical bytes and dirty/clean results agree for all local/world/reset/restore writes, names/tags/layers/active state, hierarchy/topology, references, Missing, prefab, source reconciliation, History, Play and reload.
- Untracked fields/properties and stateful/contextual/ambiguous converters keep authoritative capture; opt-in is explicit and complete.
- Metadata/Converter/identity/domain changes, key decline, partial errors and mutation during capture never reuse or publish a false clean result.
- Retained snapshots/builders do not retain extension objects, metadata, delegates, Assembly or ALC; strict GC/finalizer/GC monitors remain enforced.
- Borrowed builder retirement, thread misuse, failures and nullable/exhausted revision semantics pass through public converter/provider boundaries without IVT/private reflection/testing APIs.
- Five alternating independent process pairs on fixed input pass median <16 ms, P95 <25 ms and <8 MiB per complete due inspection with the unchanged 100 ms policy. Raw data, CPU, GC, allocations and failures are retained.
- Full relevant Core/Scene/Editor, SDK/script exports, architecture, Wiki and final CI pass on the same candidate.
Failure, lifecycle and closure
Keys are transient neutral optimization facts, never encoded project data or a schema version. Unknown capability declines reuse; exceptions retain the prior snapshot/last-good owner and diagnostics. Source mutation during a declared capture rejects that capture. Default and unsupported cases use the same authoritative writer, not a parallel implementation or legacy path.
Keep Open until full acceptance is verified on branch, then add an individual Fixes reference to PR #26. Close only when that PR merges to master. No automatic merge is authorized.
Activity
FLwolfy commentedon Oct 29, 2026
Production correctness currently passes Core 90 / Scene 56, and the first complete twelve-suite SDK build completed with zero warnings/errors. Quiet due-inspection medians are 6.7–7.1 ms, but the first ten-sample P95 is 39.6 ms; this is not latency acceptance. The measured identity-check scratch allocates about 2.16 MB per inspection. A further Core-owned refinement borrows a cleared BCL buffer only during the synchronous callback, validates all identities before unchanged-node reuse, and copies checks only when an actual write needs post-write validation. No thresholds, timers, samples or ordinary writer semantics are relaxed. The next build was explicitly canceled for this refinement; its log remains canceled, not passed. Full final-source tests, controlled groups and product/CI reruns remain open.
FLwolfy commentedon Oct 29, 2026
The editing-path check prevents closure: changing one Transform per complete 10,000-object inspection still costs about 69–71 ms / 16.8 MB, despite the idle improvement. A property-only proof experiment passes Core contracts but reaches only 57–60 ms and initially violates unchanged-snapshot reuse; it will not be adopted as the solution. The next design applies the same neutral capture proof at existing structured-array element boundaries. Core Writer owns optional row-key capture/verification; Scene contributes complete object/component state, exact source/parent identities and sibling placement. Ordinary serialization and uncertified/Missing/Prefab/reference rows still execute the same writer. Removing the all-or-nothing Scene root hint avoids rescanning its complete key twice on edits; independently unchanged rows can reuse their existing immutable nodes. This requires one optional existing Writer callback parameter and exact Guid key input, public-boundary non-Scene row tests, canonical/retirement/failure tests and fresh idle/edit measurements. No new serializer, identity index, mutation service or platform/backend dependency is introduced. Existing failed experiments remain evidence, not production fallback paths.
FLwolfy commentedon Oct 29, 2026
Commit 2434cd9 replaces the all-or-nothing Scene root hint with optional keyed elements in the existing Writer array protocol, and adds exact Guid key values. Scene owns complete object/component, source/parent ID and sibling inputs. Core rejects enclosing reuse when nested converters execute (including neutral effects propagated from independent property payloads), preserving stateful conversion. The rejected property-only experiment is removed. The same canonical encoder memoizes immutable keyed blocks up to 64 KiB; larger nodes remain streamed. Core 97 passes, including independent non-Scene element reuse, exact IDs, failures, converter effects, concurrency, failed streams and large-node bounded allocation. Scene 56 and ten focused native cases pass at their recorded stages. Preliminary edit medians are 12.8–14.7 ms / ~2.51 MB, idle about 9.5 ms / ~2.36 MB; raw spikes remain retained. Complete frozen SDK graph, quiet five-process idle/edit pairs, profiles/products and exact-head CI are now required before closure. No BGCS, vendor, generated binding, platform policy or parallel foundational service was changed.