Repository navigation
hw/misc/rk3588_rknpu: support Qwen3-0.6B RKLLM workload - #46
Conversation
The LLM matmul path emits an int8 convolution with a quantized int32 accumulator output. Accept that output precision and validate the 32-bit output layout before selecting the int32 DPU writer. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Add the DPU_RDMA FP16 reshape and notched FP32 matmul paths used by RKLLM prefill. The reshape path copies the RKLLM tile into the DPU output view. The notched path uses the existing softfloat FP16 execution and accepts SURFACE_ADD as the surface step when the programmed DST_SURF_STRIDE is the row step. The Qwen3-0.6B smoke run now completes with RKLLM-RUN ret=0. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aaa2e0c42c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Replace the Qwen-specific shape checks with capability checks for the DPU_RDMA FP16 reshape and notched FP32 matmul paths. The reshape path now accepts any FP16 source and output tile with the same element count, and the notched path accepts any FP16 matmul with a notched FP32 output and a valid SURFACE_ADD surface step. Keep the SURFACE_ADD interpretation scoped to decoded notched layouts so existing DST_SURF_STRIDE checks remain unchanged. Signed-off-by: Chao Liu <chao.liu@processmission.com>
Add a quantized int32 DPU output case and a DPU_RDMA FP16 reshape case. The reshape case verifies the element-for-element copy from the RDMA source tile to the DPU output view; the int32 case verifies that the quantized LLM matmul output reaches the int32 writer with zero input and weights. Signed-off-by: Chao Liu <chao.liu@processmission.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7c749e3ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Multi-token prefill uses a wider RDMA tile with no output notch and no source line notch. Recognize the reshape by the FP16 data format, EW configuration and LUT bypass rather than the single-token tile shape. Signed-off-by: Chao Liu <chao.liu@processmission.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 705c70b560
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed the review findings in d740467:
Validation: The reshape qtest now uses a padded RDMA source row stride to cover the |
|
Additional validation after the review fixes:
Qwen3-0.6B still completes both the single-token English prompt and Latest commits: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5802bf0e8d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4a7ac3a2d8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Validate the DPU core dimensions, CNA storage layout and output geometry before selecting the notched FP32 matmul path, so the shared allocation and indexing checks cannot be bypassed. Handle the 224-row notched output by treating SURFACE_ADD as a row group stride and writing rows across row groups. Account reshape working storage against the host memory budget, validate the RDMA source stride before reading, reject unaligned source channel counts, and only accept the reshape fast path when the BS/BN stages and output conversion are identity. Evaluate the signed RDMA surface notch with sextract32() so a negative notch yields a valid compact source layout. Reconstruct the derived SURFACE_ADD view fields in post_load(). Signed-off-by: Chao Liu <chao.liu@processmission.com>
4a7ac3a to
8233642
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 82336421ce
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Run the existing FP16 BS and BN stage validation before selecting the notched FP32 matmul path. This rejects active RDMA-sourced stages that the execution path cannot service. Signed-off-by: Chao Liu <chao.liu@processmission.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a319b53fbd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Require the notched FP32 matmul path to bypass the DPU LUT. The shared FP16 execution path writes 16-bit LUT results, which is not a valid 32-bit notched output. Signed-off-by: Chao Liu <chao.liu@processmission.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc4a59e476
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The notched FP32 executor initializes out-of-bounds samples to zero and does not apply the programmed CNA pad value. Reject such tasks before dispatching the notched matmul path. Signed-off-by: Chao Liu <chao.liu@processmission.com>
The notched FP32 executor does not apply EW arithmetic. Require the EW stage to be fully bypassed before selecting this mode. Signed-off-by: Chao Liu <chao.liu@processmission.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f25361a709
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Require the reshape fast path to use a fully packed output: valid and storage channel counts must match and be eight-channel aligned. Reject RDMA source padding, which the reshape reader does not synthesize. Signed-off-by: Chao Liu <chao.liu@processmission.com>
|
Latest review fixes:
Review threads are resolved. Current CI state: Local verification: |
Add the RKNPU execution paths needed by the Qwen3-0.6B RKLLM workload.
RKNPU model changes
execution path
DST_SURF_STRIDE is the row step
FP16/FP32 capabilities instead of the initial Qwen-specific shapes
Verification
Qwen3-0.6B RKLLM w8a8 on
rock-5b-plus,rknpu=ongenerates 16 tokensand reaches:
Measured with a timestamped QEMU run:
RKLLM-RUN ret=0: 47.793 sRKLLM-RUN ret=0: 10.134 sThe smaller Qwen2.5-0.5B model was converted with RKLLM-Toolkit 1.3.1
and used to validate the same task shapes before the Qwen3 run.
rk3588-rknpu-testpasses all 117 subtests, including the newquantized int32 output and DPU_RDMA FP16 reshape cases.
Local evidence is under
.oh-my-qemu/rk3588-rknpu-qwen3/.