Repository navigation
Conversation
Hahihula
force-pushed
the
sycle-support
branch
from
September 14, 2026 14:04
03713b9 to
f89260e
Compare
- Add an integer mat-vec kernel (`mmvq_q8`): the activation is quantized to int8 once per call and each weight block is dotted against it in integer arithmetic, matching the CPU `vec_dot` numerics. Both mat-vec kernels now parallelise over (row, chunk of blocks) rather than one work-item per row. - Keep a half-precision pipeline in f16 throughout: the mat-vec reads and writes f16, and prefill dequantizes straight to f16 (`dequantize_f16`) instead of materialising f32 and casting. - Serve `m <= 32` from the mat-vec rather than dequantize-and-GEMM. - Matmul an unquantized GGUF weight (F32/F16/BF16) in its own dtype instead of widening it to F32 at load; `QMatMul::is_quantized` tells the two apart. - Add `get_rows`, so a quantized embedding dequantizes only the gathered rows. - Accept a non-zero `start_offset` on the indexing kernels' operands (what `narrow` produces) instead of rejecting it. - Bound the allocation pool, defer frees behind a synchronize (`sycl::free` does not wait for enqueued work), and drain-and-retry a failed allocation. - Default the UR loader to the Level-Zero V1 adapter; V2 cannot launch kernels from an in-order USM queue on Xe2. - Bake the oneAPI runtime dirs into the library as an rpath, so a binary runs without `setvars.sh`. - Add the `sycl_parity` example, a SYCL-vs-CPU numerical check.
Hahihula
force-pushed
the
sycle-support
branch
2 times, most recently
from
October 2, 2026 05:17
ddab996 to
6ae6c36
Compare
Hahihula
force-pushed
the
sycle-support
branch
2 times, most recently
from
October 20, 2026 13:58
68c5a9e to
6cd5b7e
Compare
Hahihula
force-pushed
the
sycle-support
branch
from
October 21, 2026 11:41
6cd5b7e to
c70a9c9
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a sycl feature that runs candle on Intel GPUs via oneAPI DPC++ / Level Zero.
Design
be built by anyone; the kernel crate is the only thing that needs icpx.
Coverage
gemm_batch).
Status
proof of actual usability: lucasjinreal/Crane#112