Skip to content

Split pointwise ops over concat inputs to enable fused_concat fusion - #5182

Open
pfultz2 wants to merge 8 commits into
developfrom
fuse-pointwise-concat-split
Open

Split pointwise ops over concat inputs to enable fused_concat fusion#5182
pfultz2 wants to merge 8 commits into
developfrom
fuse-pointwise-concat-split

Conversation

@pfultz2

@pfultz2 pfultz2 commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Patterns like rotate-half (used in RoPE) produce a concat of slices that feeds a pointwise op. Since the concat's inputs are not pointwise, fuse_concat cannot fuse this into a fused_concat kernel, so the concat and the pointwise op each launch a separate kernel and materialize an extra intermediate buffer. By distributing the pointwise op over the concat segments, the whole pattern becomes a concat of pointwise ops, which the existing find_concat_pointwise matcher can fuse into a single kernel.

Technical Details

Two new matchers are added to the fuse_concat pass, run before the existing fusions:

  • find_pointwise_concat_split: matches a single-use pointwise op with a single-use concat input whose inputs are all non-pointwise. For each concat segment, it clones the pointwise submodule and re-applies it to the segment, slicing the remaining pointwise inputs along the concat axis to the segment's range. The original pointwise op is then replaced by a concat of these per-segment pointwise ops. It bails out when the concat has fewer than two inputs or is used more than once by the pointwise op.
  • find_nested_concat: flattens a single-use concat that feeds another concat on the same axis into its parent, so the concat produced by the split (and any outer concat around it) collapses into one concat before the pointwise-concat fusions run.

Each matcher is followed by dead_code_elimination, and the existing find_pointwise_concat_pointwise/find_concat_pointwise matchers then fuse the resulting concat-of-pointwise into a fused_concat.

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

    • Added: New functionality.
    • Changed: Changes to existing functionality.
    • Removed: Functionality or support that has been removed. (Compared to a previous release)
    • Optimized: Component performance that has been optimized or improved.
    • Resolved Issues: Known issues from a previous version that have been resolved.
    • Not Applicable: This PR is not to be included in the changelog.

Follow the LLVM AI Tool Use Policy for contributions using AI.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Distributes pointwise operations across concat segments to enable GPU fused_concat fusion and avoid intermediate buffers.

Changes:

  • Adds pointwise-concat splitting and nested-concat flattening.
  • Adds structural fusion and bailout tests.
  • Review used a single pass without agent fan-out.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
src/fuse_concat.cpp Implements the new fusion rewrites.
test/fuse_concat.cpp Tests splitting, outer concat flattening, and multi-use bailout.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/fuse_concat.cpp
Comment on lines +271 to +274
if(concat_ins->inputs().size() < 2)
return;
if(std::count(ins->inputs().begin(), ins->inputs().end(), concat_ins) != 1)
return;
Comment thread src/fuse_concat.cpp
Comment on lines +309 to +310
// Merge a same-axis concat that feeds directly into another concat
struct find_nested_concat
Comment thread src/fuse_concat.cpp

void fuse_concat::apply(module_pass_manager& mpm) const
{
match::find_matches(mpm, find_pointwise_concat_split{});
@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Batch New Rate (b3e94e) Old Rate (3a503c)* Diff Status
torchvision-resnet50 64 3,298.94 3,264.92 1.04%
torchvision-resnet50_fp16 64 7,887.67 7,548.67 4.49%
torchvision-densenet121 32 2,542.97 2,483.99 2.37%
torchvision-densenet121_fp16 32 5,038.52 5,004.24 0.69%
torchvision-inceptionv3 32 2,060.01 2,058.51 0.07%
torchvision-inceptionv3_fp16 32 4,437.82 4,416.99 0.47%
cadene-inceptionv4 16 818.04 820.61 -0.31%
cadene-resnext64x4 16 784.89 782.78 0.27%
slim-mobilenet 64 8,371.94 8,386.36 -0.17%
slim-nasnetalarge 64 228.86 228.86 0.00%
slim-resnet50v2 64 3,228.93 3,180.91 1.51%
bert-mrpc-onnx 8 1,168.90 1,168.84 0.00%
bert-mrpc-tf 1 502.21 498.63 0.72%
pytorch-examples-wlang-gru 1 376.63 473.35 -20.43% 🔴
pytorch-examples-wlang-lstm 1 555.14 384.83 44.26% 🔆
torchvision-resnet50_1 1 1,055.21 1,046.63 0.82%
cadene-dpn92_1 1 443.36 437.32 1.38%
cadene-resnext101_1 1 364.59 365.89 -0.36%
onnx-taau-downsample 1 846.88 844.09 0.33%
dlrm-criteoterabyte 1 32.32 32.42 -0.31%
dlrm-criteoterabyte_fp16 1 51.69 51.80 -0.21%
agentmodel 1 11,766.92 9,209.12 27.77% 🔆
unet_fp16 2 58.36 58.80 -0.75%
resnet50v1_fp16 1 1,428.36 1,366.11 4.56%
resnet50v1_int8 1 1,782.36 1,883.96 -5.39% 🔴
bert_base_cased_fp16 64 1,099.73 1,098.16 0.14%
bert_large_uncased_fp16 32 347.17 345.59 0.46%
bert_large_fp16 1 207.37 206.59 0.38%
distilgpt2_fp16 16 2,099.48 2,092.89 0.31%
yolov5s 1 560.70 558.33 0.42%
tinyllama 1 45.81 45.83 -0.05%
vicuna-fastchat 1 44.32 44.20 0.26%
whisper-tiny-encoder 1 412.77 411.87 0.22%
whisper-tiny-decoder 1 408.60 408.48 0.03%
llama2_7b 1 20.87 20.84 0.15%
qwen1.5-7b 1 23.67 23.58 0.39%
phi3-3.8b 1 28.37 26.72 6.20% 🔆
llama3-8b 1 22.75 21.80 4.36%
whisper-large-encoder 1 10.17 10.18 -0.08%
whisper-large-decoder 1 106.17 105.30 0.82%
mistral-7b 1 23.78 23.78 0.03%
FLUX.1-schnell 1 784.43 755.22 3.87%

Regressions detected 🔴

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

@gh-app-migraphx-bot-pr-write

Copy link
Copy Markdown
Test Status Result
bert-mrpc-onnx PASSED: MIGraphX meets tolerance
bert-mrpc-tf ERROR - check error output
traceback
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 313, in main
import tensorflow as tf
File "/usr/local/lib/python3.12/dist-packages/tensorflow/init.py", line 40, in
from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow # pylint: disable=unused-import
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 37, in
self_check.preload_check()
File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/platform/self_check.py", line 63, in preload_check
from tensorflow.python.platform import _pywrap_cpu_feature_guard
ImportError: libnuma.so.1: cannot open shared object file: No such file or directory
pytorch-examples-wlang-gru 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
pytorch-examples-wlang-lstm 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
dlrm-criteoterabyte PASSED: MIGraphX meets tolerance
agentmodel PASSED: MIGraphX meets tolerance
unet PASSED: MIGraphX meets tolerance
resnet50v1 PASSED: MIGraphX meets tolerance
bert_base_cased_fp16 PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_large PASSED: MIGraphX meets tolerance
yolov5s PASSED: MIGraphX meets tolerance
tinyllama PASSED: MIGraphX meets tolerance
vicuna-fastchat PASSED: MIGraphX meets tolerance
whisper-tiny-encoder PASSED: MIGraphX meets tolerance
whisper-tiny-decoder PASSED: MIGraphX meets tolerance
distilgpt2_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
llama2_7b PASSED: MIGraphX meets tolerance
qwen1.5-7b PASSED: MIGraphX meets tolerance
phi3-3.8b PASSED: MIGraphX meets tolerance
llama3-8b PASSED: MIGraphX meets tolerance
whisper-large-encoder PASSED: MIGraphX meets tolerance
whisper-large-decoder PASSED: MIGraphX meets tolerance
mistral-7b PASSED: MIGraphX meets tolerance
FLUX.1-schnell PASSED: MIGraphX meets tolerance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants