Clarify attention KV-index and look-back semantics

[erizheng-amd]

Motivation

This PR ports over the change on PR #403 from the rocmlirTriton side. This is a follow-up to ROCm/rocmlirTriton#356, which introduced sliding-window attention.

The old names obscure the actual contract: currentSeqLen is a zero-based, inclusive last-valid K/V index, not a sequence length. Likewise, a sliding-window value is a look-back distance. This follows ROCm/rocmlirTriton#356 (comment).

Technical Details

• Rename the runtime KV-cache operand to lastValidKVIndex.
• Rename the window attribute and tooling option to slidingWindowLookBack.
• Make the valid bounds explicit:
  • 0 <= P < seq_len_k
  • 1 <= L < seq_len_k
  • L requires P.
• Update TOSA-to-Rock matching, Rock lowering, rocmlir-gen masking, performance tooling, and regression coverage to use the inclusive-index contract.
• Reject malformed per-group KV-index lists before rocmlir-gen indexes them.
• Leave out-of-range sliding-window masks explicit during TOSA-to-Rock matching instead of creating Rock IR that fails verification.

Test Plan

-- ninja check-rocmlir
-- PR CI
-- PR nightly CI
-- attentionSweep

Test Result

-- ninja check-rocmlir passed
-- PR CI passed
-- PR nightly CI looks good, https://ml-ci-internal.amd.com/job/MLIR/job/mlir/view/change-requests/job/PR-2460/6/pipeline-overview/. It stucked on 'create performance report` for a long time but everything else looked good
-- Ran attention sweep with 500 samples. All passed: Passed: 504, Invalid: 35101, Failed: 0

Submission Checklist

• Look over the contributing guidelines at https://github.com/ROCm/ROCm/blob/develop/CONTRIBUTING.md#pull-requests.

[Copilot]

Pull request overview

This PR clarifies and standardizes sliding-window attention semantics across rocMLIR by renaming the KV-cache runtime operand from a “sequence length” concept to an inclusive “last valid KV index”, and renaming the sliding-window parameter to a look-back distance. It updates the driver, Rock dialect/verification, lowering/transforms, performance tooling, and extensive tests to match the inclusive-index + look-back contract.

Changes:

• Rename attention KV-cache operand to lastValidKVIndex and sliding-window parameter to slidingWindowLookBack across tooling, Rock IR, lowering, and tests.
• Make bounds/validation explicit (driver + perf tooling), and keep out-of-range sliding-window masks explicit during matching to avoid invalid Rock IR.
• Update performance reporting/analysis and regression/e2e coverage to reflect the renamed fields and semantics.

Reviewed changes

Copilot reviewed 49 out of 49 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
mlir/utils/performance/reportUtils.py	Renames attention perf-report column to SlidingWindowLookBack.
mlir/utils/performance/perfRunner.py	Renames CLI/config fields and adds validation for last_valid_kv_index and sliding_window_look_back.
mlir/utils/performance/attentionSweeps.py	Updates sweep sampling/config creation to use inclusive KV index + look-back semantics.
mlir/utils/performance/analysis/quickTuningGen.py	Updates attention grouping column to SlidingWindowLookBack and defaulting behavior for legacy TSVs.
mlir/tools/rocmlir-gen/rocmlir-gen.cpp	Renames rocmlir-gen options/operands and enforces new bounds; updates masking and kernel creation plumbing.
mlir/test/rocmlir-gen/problem-key.mlir	Updates tuning-key emission tests for new flag names and disabled sentinel behavior.
mlir/test/rocmlir-gen/options.mlir	Updates rocmlir-gen option validation tests for look-back and last-valid index bounds/count.
mlir/test/rocmlir-gen/attention-splitkv-host-f32-combine.mlir	Updates host combine test to use -last_valid_kv_index.
mlir/test/rocmlir-gen/attention-sliding-window.mlir	Updates sliding-window IR checks to lastValidKVIndex + slidingWindowLookBack, adds endpoint coverage.
mlir/test/rocmlir-gen/attention-kernel-gqa-kvcache.mlir	Updates GQA KV-cache kernel checks to new operand name.
mlir/test/rocmlir-gen/attention-kernel-gqa-kvcache-lse.mlir	Updates LSE KV-cache kernel checks to new operand name.
mlir/test/rocmlir-gen/attention-kernel-gqa-kvcache-lse-splitkv.mlir	Updates split-KV+LSE KV-cache kernel checks to new operand name.
mlir/test/perf-scripts/attention-tuning-db-compat.py	Updates tuning DB compatibility test harness for new optional field name/sentinel and new perfRunner validations.
mlir/test/fusion/pr-e2e/rock-attention-pipeline-early-exit.mlir	Updates e2e attention invocation flags to new KV-index naming.
mlir/test/fusion/pr-e2e/attention/rock-attention-fully-masked.mlir	Updates e2e fully-masked attention invocations to new flags.
mlir/test/fusion/nightly-misc-e2e/mixr-attention/f16/rock-attention-prefix-causal.mlir	Updates nightly prefix-causal attention invocation to new KV-index flag.
mlir/test/fusion/nightly-misc-e2e/mixr-attention/f16/mixr-attention-sliding-window-kvcache-prefix-causal.mlir	Updates folding checks/comments to lastValidKVIndex and slidingWindowLookBack.
mlir/test/fusion/mixr-attention-sliding-window-problem-key.mlir	Updates problem-key checks to new look-back field name and KV-index omission.
mlir/test/e2e/PrAttentionSchedule.toml	Renames e2e suite configs to new flags.
mlir/test/e2e/PrAttentionI8.toml	Renames e2e suite configs to new flags.
mlir/test/e2e/PrAttentionF32.toml	Renames e2e suite configs to new flags.
mlir/test/e2e/PrAttentionF16.toml	Renames e2e suite configs to new flags.
mlir/test/e2e/PrAttentionDirectToLDS.toml	Renames e2e suite configs to new flags.
mlir/test/e2e/PrAttentionBF16.toml	Renames e2e suite configs to new flags.
mlir/test/e2e/AttentionSchedule.toml	Renames schedule suite configs to new flags.
mlir/test/e2e/AttentionNonPowerOfTwoTileSize.toml	Renames non-power-of-two tile e2e configs to new flags and updates comments.
mlir/test/e2e/AttentionDirectToLDS.toml	Renames direct-to-LDS e2e configs to new flags.
mlir/test/Dialect/Rock/ops_error.mlir	Updates Rock op verifier error tests for look-back semantics and operand requirements.
mlir/test/Dialect/Rock/gridwise_attention_accel_lowering.mlir	Updates gridwise lowering checks for lastValidKVIndex and look-back naming/semantics.
mlir/test/Dialect/Rock/gemm_to_gridwise.mlir	Updates gemm-to-gridwise checks to new KV-index operand naming.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-three-mask.mlir	Updates TOSA-to-Rock folding checks to new operand/attr names.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-sliding-window-neg.mlir	Updates negative/edge-case folding tests for renamed semantics and new validity rules.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-prefix-causal-kvcache-gqa.mlir	Updates prefix-causal+KV-cache folding checks to new operand name.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-lse.mlir	Updates LSE attention conversion checks to new operand name.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-kvcache.mlir	Updates KV-cache conversion tests and adds coverage for full-width/out-of-range look-back handling.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-flash-decoding-kvcache.mlir	Updates flash-decoding KV-cache conversion checks to new operand name.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-flash-decoding-kvcache-prefix-causal.mlir	Updates flash-decoding prefix-causal conversion checks to new operand name.
mlir/test/Conversion/TosaToRock/tosa-to-rock-attention-causal.mlir	Updates causal attention conversion checks to new operand name.
mlir/test/Conversion/DetectFlashDecoding/detect-flash-decoding-kvcache.mlir	Updates flash-decoding detection checks to new KV-index operand name.
mlir/test/Conversion/DetectFlashDecoding/detect-flash-decoding-feature-combination.mlir	Updates detection checks for combined features to new KV-index operand name.
mlir/lib/Dialect/Rock/Tuning/RockTuningImpl.cpp	Updates tuning-key string emission to use -sliding_window_look_back.
mlir/lib/Dialect/Rock/Transforms/SortDimensionsMemoryLayout.cpp	Updates attention rewrite to use getLastValidKVIndex() and getSlidingWindowLookBackAttr().
mlir/lib/Dialect/Rock/Transforms/GridwiseGemmToBlockwise.cpp	Updates gridwise attention lowering to use last-valid index and look-back semantics (incl. early-exit math).
mlir/lib/Dialect/Rock/Transforms/GemmToGridwise.cpp	Updates GQA handling and GridwiseAttention op creation to pass renamed operands/attrs.
mlir/lib/Dialect/Rock/Transforms/DetectFlashDecoding.cpp	Updates detect-flash-decoding rewrite to transform the renamed optional batch tensor.
mlir/lib/Dialect/Rock/IR/RockDialect.cpp	Renames/updates verifier logic and diagnostics for look-back and last-valid index operands.
mlir/include/mlir/Dialect/Rock/IR/RockOps.td	Renames operands/attrs in Rock ops and updates documentation/assembly format for the new contract.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[codecov]

Codecov Report

❌ Patch coverage is 91.05431% with 28 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
mlir/lib/Conversion/TosaToRock/TosaToRock.cpp	91.87%	3 Missing and 7 partials ⚠️
mlir/lib/Dialect/Rock/IR/RockDialect.cpp	80.65%	3 Missing and 3 partials ⚠️
...lir/lib/Dialect/Rock/Transforms/GemmToGridwise.cpp	75.00%	3 Missing and 1 partial ⚠️
...ialect/Rock/Transforms/GridwiseGemmToBlockwise.cpp	92.11%	0 Missing and 3 partials ⚠️
...ib/Dialect/Rock/Transforms/DetectFlashDecoding.cpp	84.62%	0 Missing and 2 partials ⚠️
mlir/tools/rocmlir-gen/rocmlir-gen.cpp	97.67%	0 Missing and 2 partials ⚠️
mlir/lib/Dialect/Rock/Tuning/RockTuningImpl.cpp	75.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2460      +/-   ##
===========================================
+ Coverage    82.57%   83.77%   +1.20%     
===========================================
  Files          120      121       +1     
  Lines        42852    43358     +506     
  Branches      7110     7220     +110     
===========================================
+ Hits         35381    36319     +938     
+ Misses        4815     4518     -297     
+ Partials      2656     2521     -135     

Flag	Coverage Δ
gfx120x	83.63% <91.05%> (+1.11%)	⬆️
gfx950	83.50% <91.05%> (+1.15%)	⬆️
mfma	83.53% <91.05%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...ect/Rock/Transforms/SortDimensionsMemoryLayout.cpp	78.42% <100.00%> (+0.40%)	⬆️
mlir/lib/Dialect/Rock/Tuning/RockTuningImpl.cpp	61.05% <75.00%> (+1.73%)	⬆️
...ib/Dialect/Rock/Transforms/DetectFlashDecoding.cpp	77.30% <84.62%> (+0.07%)	⬆️
mlir/tools/rocmlir-gen/rocmlir-gen.cpp	87.23% <97.67%> (+0.68%)	⬆️
...ialect/Rock/Transforms/GridwiseGemmToBlockwise.cpp	89.37% <92.11%> (+0.62%)	⬆️
...lir/lib/Dialect/Rock/Transforms/GemmToGridwise.cpp	82.93% <75.00%> (+0.14%)	⬆️
mlir/lib/Dialect/Rock/IR/RockDialect.cpp	78.28% <80.65%> (+9.62%)	⬆️
mlir/lib/Conversion/TosaToRock/TosaToRock.cpp	79.33% <91.87%> (+0.92%)	⬆️

... and 44 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[umangyadav]

LGTM, make sure to run nightly and PR CI.
If possible also run attentionSweeps locally.