Skip to content

Measure model loading memory and prepare GPU baseline counters - #1429

Merged
Qiong Wu (qiowu) (DingmaomaoBJTU) merged 3 commits into
microsoft:mainfrom
DingmaomaoBJTU:codex/port-perf-memory-lifecycle
Sep 18, 2026
Merged

Qiong Wu (qiowu) (DingmaomaoBJTU) merged 3 commits into
microsoft:mainfrom
DingmaomaoBJTU:codex/port-perf-memory-lifecycle

Conversation

@DingmaomaoBJTU

@DingmaomaoBJTU Qiong Wu (qiowu) (DingmaomaoBJTU) commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Model construction can allocate memory before the existing perf baseline, while missing GPU process counters leave the pre-model delta unavailable. This ports the loading-memory and GPU-baseline changes from gim-home/winml-cli-mirror PRs #10 and #11 onto current main.

  • Add versioned load-only memory measurements from runtime/device readiness through model compilation, excluding benchmark inputs and inference. Keep sampled process peaks and OS lifetime high-water diagnostics separate. Preserve null counters and signed deltas.
  • Prepare absent counters with a model-free D3D12 device on the explicitly selected GPU, retain it through final sampling, and release it on success or failure. Connect preparation to the new process tracker used by the combined port.
  • Keep automatic Runtime outputs for older projections and explicit output requests for newer ones. Include the initializer-shape regression test; its implementation fix is already upstream.

Compatibility: the outer report remains schema version 2 and load_memory.version is 1. Classic memory_measurement.schema_version is 3 because its structure and legacy-named baseline/delta boundaries change; documentation describes migration and limitations. Existing upstream CGIR changes are retained.

Validation on Windows / Python 3.11:

  • Affected perf CLI/GenAI/lifecycle, all session unit tests, and initializer-shape regression: 1,449 passed, 6 existing skips.
  • Final focused memory/GPU integration rerun: 34 passed; Runtime compatibility suite: 40 passed.
  • Ruff across src/ and tests/, changed-file formatting, license headers, and git diff --check passed.
  • Mypy: no issues in 464 source files.

Review follow-up: distinguish configured polling delay from observed mean/max sampling intervals, including GPU-query overhead. Add native factory/adapter/device failure-injection tests and idempotent COM cleanup checks. The updated memory/GPU suite passes all 40 tests.

A fresh-process native smoke test on Qualcomm Adreno X1-85 successfully created the baseline device on the selected LUID, discovered valid PDH counters, recorded final observations, and released the retained device. This validates baseline preparation/cleanup; full model hardware benchmarks from the source PRs were not rerun.

Comment thread src/winml/modelkit/session/monitor/memory_tracker.py Fixed
Comment thread src/winml/modelkit/session/monitor/memory_tracker.py Fixed
@DingmaomaoBJTU
Qiong Wu (qiowu) (DingmaomaoBJTU) merged commit bd7fb42 into microsoft:main Sep 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants