Repository navigation
Conversation
b300-build was cut from release, which split from dev in April 2021, so the work could not be merged directly. This replays it onto dev: 439 files that dev had not touched transfer unchanged, 25 are new, and the rest were reconciled by hand against what dev had done in the meantime. Where dev had solved the same problem differently its version was kept: andersen.cu keeps dev's fully-qualified thrust:: names (only the unary_function base, removed in CUDA 13, is dropped), dwt2d keeps dev's NUL off-by-one fixes, kmeans keeps dev's removal of the hardcoded cudaSetDevice(1), and the cutlass and heterosync recipes stay as dev has them. The parboil driver is taken from this work because dev's python 3 port left ppkg.py and sanity_check.py unparseable and three python2 shebangs in place. setup_environment keeps the architecture autodetection rather than dev's per-version table: the table is why dev stops at compute_100 and emits no native code for a GB300. dev's GPU_Microbenchmark restructure already feeds CUDA_CPPFLAGS into every ubench, so it inherits sm_103 from that change alone, and the 19 per-benchmark makefiles dev deleted needed no porting. cutlass-bench is a submodule in dev, so its clockRate fix belongs upstream. rodinia-3.1 kmeans is re-enabled in the build: dev had commented it out over the legacy texture API, which is what this work replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both arch tables predate Volta, so on a GB300 (SM 10.3) they fell off the end: lonestargpu printed "MapSMtoCores for SM 10.3 is undefined. Default to use 64 Cores/SM" and reported 152*64 instead of 152*128, and the sdk-compat copy -- which stopped at SM 3.0 -- returned -1 outright. Fill both in from SM 3.2 up to SM 12.1 so core counts are right on current hardware and the tables do not need touching again for a while. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test-12-8 job broke in three places, all from this branch assuming the toolkit is CUDA 13. setup_environment widened the gencode wish-list down to compute_10. On CUDA 13 that changes nothing -- sm_75 is the floor -- but on CUDA 12.8 it starts emitting compute_50/60/62, and <mma.h> only declares nvcuda::wmma for __CUDA_ARCH__ >= 700, so config_tensor, tensor_bw_half and tensor_lat_half failed with "wmma is not a class or namespace name". The suite has effectively required Volta since the CUDA 11 era, so the floor is now sm_70. Those three ubenches also declare MIN_ARCH = 70, which is what makes an explicit GPUAPPS_ARCHS override safe -- their GENCODE_SMxx assignments predate CUDA_CPPFLAGS and never applied to it. CUTLASS_ARCHS was hardcoded to 100a;103a, which no CUDA 12 nvcc accepts, and cutlass_mini had been switched onto it even though it builds the Hopper examples by name and needs 90a. Both lists are now filtered against nvcc --list-gpu-arch, and cutlass_mini keeps the 80;90;90a that ci has always used. cuda-samples is pinned at v13.2, whose CMakeLists hardcodes set(CMAKE_CUDA_ARCHITECTURES ... 110 ...) -- it shadows any -D we pass, and no CUDA 12 nvcc knows sm_110. Every cuda-samples tag targets exactly one toolkit, so the target skips with a message below CUDA 13 instead of failing the whole make. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Nothing in CI exercised the toolkit this branch is actually about, so the three CUDA 13 regressions fixed in the previous commit only surfaced in reverse -- as 12.8 breakage -- and a genuine CUDA 13 break would not have surfaced at all. The accel-sim org publishes no CUDA 13 container (the newest is Ubuntu-24.04-cuda-12.9), so this starts from the stock NVIDIA devel image and apt-installs what the minimal image would have provided. git goes in before the checkout so actions/checkout clones with git rather than falling back to the REST tarball, which would leave the submodules unavailable. 13.2 is the toolkit the Blackwell port was developed and verified against. Every target in `ci` builds on it locally: rodinia_2.0-ft, rodinia-3.1, GPU_Microbenchmark, cutlass_mini and cuda_samples. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CUTLASS's CMakeLists does find_package(Python3 ... REQUIRED), and the stock nvidia/cuda devel image ships no interpreter -- the accel-sim minimal image did, which is why this only shows up on the new job. rodinia_2.0-ft, rodinia-3.1 and GPU_Microbenchmark all built on 13.2 before cutlass_mini hit this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tgrogers
force-pushed
the
b300-build-on-dev
branch
from
October 26, 2026 12:05
6ccf1fa to
edac8b0
Compare
tgrogers
pushed a commit
to accel-sim/accel-sim-framework
that referenced
this pull request
Oct 26, 2026
Re-enabling it here was premature: it depends on the CUDA 13 texture-object port in accel-sim/gpu-app-collection#91, which has not merged. CI clones gpu-app-collection's default branch, so the app is not built there -- run_hw_trace.py correctly reported "SKIP - no binary for kmeans-rodinia-2.0-ft" and then run_simulations.py failed looking for traces that were never generated. SASS-Simulation has a second dependency besides #91: it replays a shared pre-collected trace corpus that contains no kmeans traces, so that corpus needs regenerating before this app can be turned back on. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds and runs the whole suite on a GB300 (Blackwell,
sm_103, aarch64) underCUDA 13.2.
makegoes from failing immediately ondevto completing with472 binaries and zero errors.
Why this is a re-port
This work was originally done on a branch cut from
release. That made a directmerge into
devproduce 36 conflicts, so the diff was re-applied ontodevinstead. Everything below is the
devversion of that work, plus the extraissues that only
dev's own code exposed.Architecture selection
setup_environmentpreviously carried a hand-maintained per-CUDA-version archtable that stopped at
compute_100. It now intersects a desired-arch wish-listwith whatever
nvcc --list-gpu-archreports, with a static fallback forpre-CUDA-11 toolkits. On CUDA 13.2 this resolves to 75/80/86/90/100/103.
Gencode uses the bracket form
code=[sm_X,compute_X]rather thancode="sm_X,compute_X": identical SASS and PTX, but it contains no spaces, sopassing it through
MAKE_ARGScannot split into a stray make goal. PTX staysembedded so GPGPU-Sim can still run these binaries.
Multi-arch is deliberate. The suite is built once and simulated as many
different machines, so each target wants the PTX a real build for that arch
would emit. Narrowing to the local GPU is available via
GPUAPPS_ARCHS="103"to cut build time.
CUDA 13 API removals
texture<>,cudaBindTexture,cudaBindTextureToArray,cudaUnbindTexture) is gone. Migrated ~68 files tocudaTextureObject_t, declaring the handles__constant__under theiroriginal names so every fetch site and dispatch macro is unchanged; the host
fills them with
cudaMemcpyToSymbol. Only the explicit template arguments(
tex1Dfetch<float>,tex2D<uint4>) differ.cudaDeviceProp::clockRate/::memoryClockRate/::computeModeare gone;replaced with
cudaDeviceGetAttribute. Five sites.cudaThreadSynchronize()is gone (lonestargpupta).thrust::unary_functionis gone, andusing namespace thrustnow collideswith
::cuda/::cubin nvcc-generated stubs.Blackwell / newer-toolkit build breaks
arch_filter.mk(new) intersects any hardcodedARCHlist withnvcc --list-gpu-arch, so the five ubenches pinningsm_101and friends stopbeing fatal. A
MIN_ARCHmechanism incommon.mkdrops gencodes below abenchmark's floor (cp.async needs 80).
cutlass-benchv4.1.0 to v4.8.1 (4.1.0 warnsunsupported or deprecated compute capabilities 103a),cuda-samplesv12.8to v13.2 (12.8 pins
compute_50, fatal under CUDA 13).heterosynchardcodedcompute_60; cleared soCUDA_CPPFLAGSdecides.ispass-2009/WPlinked-lGLalthough no source includes a GL header.MapSMtoCorestables stopped at SM 7.0 and SM 3.0; a GB300 fell off the endand reported 152x64 cores, or -1. Both now cover SM 3.2 to 12.1.
aarch64
-m64is x86-only; made conditional inpathfinderandpathfinder_k1,matching the pattern already used in
huffmanandmummergpu.charis unsigned by default, sogetopt_longreturning -1 compared as 255.Fixed in
dwt2dalong with a missing{0,0,0,0}longopts sentinel.Correctness bugs found by running, not building
These are pre-existing and would silently corrupt results:
rodinia-3.1kmeans used the pre-CUDA-5cudaMemcpyToSymbol("c_clusters",...)string-literal form, which passes the address of the string. Unquoted.
pathfinderwas missing two__syncthreads(), a real data race.sgemmdeclaredreadColMajorMatrixFileasbooland fell off theend, giving SIGTRAP under a modern compiler.
mummergpuhas aif (this)null check inside a member function, which isUB that modern GCC deletes; built with
-fno-delete-null-pointer-checks.CUTLASS build
Examples 10 and 11 link
cutlass_library, which drags in all 56,823 generatedkernels spanning sm_50 to sm_103; a serial sub-make then spent over six minutes
in a single
ptxason an FP4 kernel for an arch with no FP4 hardware. Filteringto
CUTLASS_LIBRARY_KERNELS=*planar_complex*keeps all 97 examples buildingwhile cutting generated sources to 1,363. The profiler is no longer built, since
the install step only ever copied from
build/examples. Arches are aCUTLASS_ARCHSvariable instead of a hardcoded80;90;90a.Makefile reconciliation
devhad drifted from the sub-Makefiles: 24 stalemvlines in therodinia-3.1target relocated binaries that now link straight intoBINDIR.Removed.
Re-enabled
devdisabled several apps thatreleasebuilds. They build and run cleanunder CUDA 13, so they are back:
ispass-2009AES and MUM,rodinia-2.0-ftkmeans, lonestar
bfs-wlcandsssp-wlc(which needed uncommenting in twoplaces -- the
VARIANTSlist and the installmv), andparboilinall:.Verification
Full
makefrom clean: exit 0, zero errors, 472 binaries.All nine suites run under the accel-sim
run_hw.pyharness at exit 0:rodinia_2.0-ft, rodinia-3.1, ispass-2009, parboil, polybench, pannotia,
proxy-apps-doe, lonestargpu-2.0, GPU_Microbenchmark -- 94 apps, 130 run
configurations, with no app missing a binary.
Known pre-existing failures, unchanged by this PR: lonestar
bhdeadlocks andptagoes out of bounds under Volta+ independent thread scheduling (both arepre-Volta warp-synchronous code),
mnistCUDNNneeds APIs removed in cuDNN 9,and
shoc-Stabilitytries a 256GB allocation.Hardware counter collection could not be exercised here: this machine has
RmProfilingAdminOnly: 1, so everyncuinvocation failsERR_NVGPUCTRPERM.The applications themselves still ran to completion under the harness.
Deliberately out of scope
src/cuda/gpuwattch-ubenchstill callscudaThreadSynchronize(), which CUDA 13removed, so it will not compile on a CUDA 13 toolkit. This PR does not fix it.
An earlier revision did -- a mechanical
cudaThreadSynchronize()->cudaDeviceSynchronize()rename across 181 files -- but no Makefile targetreferences that directory (the only wattch target points at
accelwattch-ubench), so nothing compiled the result and the change could notbe verified. It was 40% of this PR's file count for zero tested benefit, so it
has been dropped. The rename is almost certainly still correct and belongs in
its own commit, alongside a target that actually builds the suite.
Also reverted to
dev's content:src/cuda/shoc-master/build-aux/config.guessand
config.sub. Those were regenerated as a byproduct of running autotoolsduring a shoc build attempt, not a deliberate change, and shoc is not re-enabled
here. They were ~16% of the line count.
Together these took the diff from 482 files / +15,277 -3,825 to 298 files /
+12,471 -1,457.
Companion changes not in this PR
aborts a whole suite, the three runner scripts are deduplicated, NVBit is
fetched for the host architecture, and the app list is corrected.
to build
accel-sim.outhere: x86-only-msse2/-mfpmath=sse, an x86-onlyasm("int $03"), andcudaDeviceProp::clockRate.Ubuntu-24.04-cuda-13.2-minimalimage recipe, sothe CUDA 13 CI job here can eventually drop its apt step and start from a
published tag.
get_data.shis missinglonestargpu-2.0/lonestar-bfs-wla/data -> ../inputs, so that one app cannotfind its graph. Fixing it needs access to the published archive.
🤖 Generated with Claude Code