From 4b3b93b61a100b23518f6d4b855d03b75e662fc6 Mon Sep 17 00:00:00 2001 From: JRPAN <25518778+JRPan@users.noreply.github.com> Date: Tue, 20 Oct 2026 21:57:26 -0400 Subject: [PATCH 1/2] Finalize the v2.0.1 release notes: date, multi-GPU tracing (#570), GPGPU-Sim #148 Co-Authored-By: Claude Opus 5.5 --- release.notes.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/release.notes.md b/release.notes.md index b649ab47b6..97bfa55fe3 100644 --- a/release.notes.md +++ b/release.notes.md @@ -1,10 +1,11 @@ # Release v2.0.1 -_October 2026._ +_Oct 20, 2026._ A maintenance release on top of 2.0.0: pre-collected traces move from the FTP server to -GitHub release assets, trace post-processing and parsing fixes, and a reworked CI. +GitHub release assets, multi-GPU tracing, trace post-processing and parsing fixes, and a +reworked CI. > **Requires the matching GPGPU-Sim:** `gpgpu-sim_distribution` tag `v2.0.1`. @@ -29,6 +30,9 @@ GitHub release assets, trace post-processing and parsing fixes, and a reworked C `TRYWAIT`s (#564). It used to read only the first lane of `base_stride` / `base_delta` lines, so a `TRYWAIT` on an `mbarrier` reached only through another lane could be dropped and the kernel could deadlock (#561). +- `run_hw_trace.py` traces on several GPUs at once: `-D 0,1,2,3` (or `all`) runs one tracing + worker per GPU, and `--cpu_workers` post-processing workers take traced apps from a queue + (`--queue_size`), so no GPU sits idle during post-processing. `-D 0` behaves as before (#570). ## Trace parsing - Opening a `.tracez` file checks every `fread`, so a truncated file fails with an error @@ -51,6 +55,8 @@ GitHub release assets, trace post-processing and parsing fixes, and a reworked C ## GPGPU-Sim v2.0.1 - Fixes 47 build warnings and two statistics that read uninitialized memory (#147). +- PTX mode honours `-gpgpu_ptx_force_max_capability` when loading cuobjdump PTX, so fat + binaries with `sm_90a` PTX (e.g. the `GPU_Microbenchmark` apps) load on older configs (#148). - CI picks its runner pool by repository visibility (#144); README badge cleanup (#146). # Release v2.0.0 — Accel-Sim 2.0 (full Hopper support) From 3164bcc56576eb47afd1e64d36bfc5c32a1f76f2 Mon Sep 17 00:00:00 2001 From: JRPAN <25518778+JRPan@users.noreply.github.com> Date: Tue, 20 Oct 2026 22:03:07 -0400 Subject: [PATCH 2/2] README: a Pre-collected Traces section with available traces, download and simulation Moves trace download out of the Tracer section into its own top-level section, listed in the contents, with the available H200 traces, the download commands, the extracted layout, and how to simulate and correlate them. Co-Authored-By: Claude Opus 5.5 --- README.md | 62 ++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 52 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index dc1a23d797..8cc98539c1 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ Each panel shows the Pearson correlation and mean absolute percentage error (MAP - [How to Cite](#how-to-cite) - [Dependencies](#dependencies) - [Quick Start](#quick-start) +- [**Pre-collected Traces**](#pre-collected-traces) — no GPU needed - [Full Hopper Support](#full-hopper-support) - [**Tracing with vLLM**](#tracing-llms-with-vllm) - [Core Components](#core-components) @@ -231,6 +232,56 @@ For **PTX execution-driven** mode (no traces required), drop `-T` and use a `*-P --- +## Pre-collected Traces + +No GPU needed: download SASS traces collected on real hardware and simulate them directly. +Traces ship as assets of the Accel-Sim release they were validated with, together with the Nsight Compute stats measured on the same GPU, so simulation results can be correlated against hardware right away. + +### Available traces + +| GPU | Release | Workloads | Configurations | Kernels | Download size | +|---|---|---|---|---|---| +| H200 | [v2.0.1](https://github.com/accel-sim/accel-sim-framework/releases/tag/v2.0.1) | BERT, ResNet-50, DLRMv2, 3D U-Net, RetinaNet, Whisper, SDXL, Llama 3.1 8B (inference and training), Qwen 2.5 7B, DeepSeek R1, Mixtral 8x7B | 20 | 8942 | 208 GB | + +Each configuration (workload + arguments) downloads on its own, from under 1 GB to about 50 GB. +`./get-accel-sim-traces.py -l` lists every configuration with its size, and the release's `README.md` has the same table. +A few kernels that do not complete in simulation are left out of both the traces and the Nsight Compute stats. + +### Download + +```bash +./get-accel-sim-traces.py -l # list what is available +./get-accel-sim-traces.py -a H200/inference_bert.sh/32_256 # one configuration +./get-accel-sim-traces.py -a H200/inference_sdxl.sh # every configuration of a workload +./get-accel-sim-traces.py -a H200/all # everything for one GPU +``` + +The script checks every download against the SHA-256 checksums in the release's `manifest.json`, resumes interrupted downloads, and extracts under `./hw_run/` in the standard Accel-Sim layout: + +``` +hw_run/H200/traces/device-0/12.8///traces/ kernelslist.g + kernel-*.tracez +hw_run/H200/device-0/12.8/// Nsight Compute stats (ncu.csv.*) +``` + +Options: `-d ` extracts somewhere else, `--keep_parts` keeps the downloaded tar parts, and `--repo` / `--tag` read another release (a private repository needs `GITHUB_TOKEN` or a `gh auth login` session). +Each configuration is one tar split into parts under 2 GB; the release's `README.md` also shows how to download and check them by hand. + +### Simulate and correlate + +The workloads are defined in [`define-ml-workloads.yml`](./util/job_launching/apps/define-ml-workloads.yml) (`bert_inference`, `resnet_inference`, `sdxl_inference`, `llama_inference`, …): + +```bash +./util/job_launching/run_simulations.py -B bert_inference -C H200-SASS \ + -T ./hw_run/H200/traces/device-0/12.8 -N bert-h200 +./util/job_launching/monitor_func_test.py -v -N bert-h200 +./util/job_launching/get_stats.py -R -k -K -N bert-h200 | tee bert-h200.csv +./util/plotting/plot-correlation.py -c bert-h200.csv -H ./hw_run/H200 +``` + +For large workloads, `run_simulations.py --per-kernel` runs each kernel as its own job (collect with `get_stats.py --per-kernel-dirs`). + +--- + ## Full Hopper Support Accel-Sim 2.0 models the asynchronous, warp-specialized, persistent execution style of modern Hopper kernels (FlashAttention-3, cuBLAS/CUTLASS GEMMs, NCCL). @@ -396,16 +447,7 @@ To disable it entirely, pass `--spinlock_handling none`: The detection tool lives in [`./util/tracer_nvbit/others/spinlock_tool/`](./util/tracer_nvbit/others/spinlock_tool/). **Pre-traced applications.** -Pre-collected traces are published as GitHub release assets, one tar per GPU / workload / argument set, split into parts under 2 GB with SHA-256 checksums in the release's `manifest.json`. -To list, download, verify and extract them under `./hw_run//` in the standard layout (`traces/device-0////traces/` plus the Nsight Compute stats in `device-0////`): -```bash -./get-accel-sim-traces.py -l # list what is available -./get-accel-sim-traces.py -a H200/inference_bert.sh/32_256 # one argument set -./get-accel-sim-traces.py -a H200/inference_sdxl.sh,H200/all # GPU[/workload[/args]], 'all' at any level -``` -Pass `-T ./hw_run//traces/device-0/` to `run_simulations.py` and `-H ./hw_run/` to `plot-correlation.py`. -The release's `README.md` lists every set and how to download one by hand. -Downloading from a private repository needs `GITHUB_TOKEN` or a `gh auth login` session. +To simulate without a GPU, download traces collected on real hardware — see [Pre-collected Traces](#pre-collected-traces). ### SASS Frontend and Simulation Engine