Repository navigation
HSA decode: define the scratchpad as a variable instead of baking it in - #178
Merged
erwei-xilinx merged 1 commit intoOct 27, 2026
Merged
Conversation
ROCR's AIE loader resolves a full ELF's control scratchpad against a variable defined with hsa_executable_agent_global_variable_define, and flushes it on every dispatch (ROCm/rocm-systems#14004). So the decode allocates the scratchpad, loads the cached hsaco with it defined, and writes L with plain stores. This drops the ELF relocation rewriting in driver.py, the per-decoder repack and prepare_bytes, the host-side flush, and the pointer_info probe it needed. Pin rocm-sdk-core 10.3.0a20261027, the first nightly with both ROCm/rocm-systems#14004 and #13908. On an older ROCR, repeated full-ELF dispatches give wrong results and the decode fails to load. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #158; based on its branch, so the diff is only this change. Retarget to
mainonce #158 merges.ROCR now resolves a full ELF's control scratchpad against a variable the application defines before loading (ROCm/rocm-systems#14004), and keys full-ELF PDI placements by section (ROCm/rocm-systems#13908). With both:
scratch-pad-ctrldefined, and writes L with plain stores.driver.py, the per-decoder repack andtriton_npu_hsa_prepare_bytes,triton_npu_hsa_scratchpad_flush, and thepointer_infoprobe.triton_npu_hsa_prepare_with_variables;triton_npu_hsa_prepareis unchanged.rocm-sdk-core==10.3.0a20261027, the first nightly with both fixes. Without #13908, Switch HSA/ROCR dispatch to hsaco/kernel-object loading #158 gives wrong HSA results on repeated full-ELF dispatches (Llama decode diverges at token 2,matmul_bf16_m64_n64_k6475% wrong).Test plan
On Krackan, with the pinned wheel:
AMD_TRITON_NPU_RUNTIME=hsa:axpy,matmul_bf16_m64_n64_k64,test_softmaxpass.examples/llm_q4nx/test_decode_artifact.pypasses.scripts/test_hsa_hsaco_pack.pyfails in aircc on Switch HSA/ROCR dispatch to hsaco/kernel-object loading #158 as well, unchanged by this PR.🤖 Generated with Claude Code