[KimHenrikOtte]

Summary

This PR adds an experimental wgpu backend for Candle, enabling portable GPU compute across native and WASM targets and fixes (#344).

Advantages of wgpu compared to CUDA:

• Easier to install — no extra SDK/toolkit required
• Works on older GPUs that don’t meet modern CUDA capability requirements

---

Performance Metrics

Model / Task	wgpu	CPU
llama2c - 15m	378.28 token/s	327.90 token/s
llama2c - 42m	251.88 token/s	68.86 token/s
t5 - 42m	217.23 token/s	98.34 token/s
clip	1.316 s	1.0502352 s
metavoice	36.36 s	too long to measure
stable-diffusion	26.39 s	Error: cannot seed the CPU rng with set_seed

---

Included Changes (High Level)

• Added wgpu backend implementation
• Added wgpu-compute-layer (device setup, shader loading, buffer management, lightweight caches)
• Added candle-wgpu-kernels (source for all Candle wgpu kernels)
• Updated candle-wasm-test (build.rs copies tests and converts sync → async for WASM backend validation)
• Updated candle-wasm-examples (allows selecting wgpu backend)
• Added candle-wasm-examples/xtask (helper to simplify running WASM examples)

---

Notes

If preferred, I can split parts of this into separate PRs (e.g. backend core, kernels, WASM examples, wasm tests).

[vandana-rajan]

Thank you @KimHenrikOtte for this. I have tried your wgpu backend and it works amazing.
One issue I found was that for Q80 models, it throws error saying

Error while parsing WGSL: :132:31 error: mixing '&' and '+' requires parenthesis v_dest[(out_base + 2) & 31u + block_index * 32] = f32(x3_unsigned)*d; ^^^^^^^ :133:31 error: mixing '&' and '+' requires parenthesis v_dest[(out_base + 3) & 31u + block_index * 32] = f32(x4_unsigned)*d;

You can reproduce this for phi example with phi_1_5_Q80 and puffin Q80 models.

Fix was a one line change I have put here - vandana-rajan@caf8bba

I can push this as a PR once your branch gets merged.

[KimHenrikOtte]

Thank you @KimHenrikOtte for this. I have tried your wgpu backend and it works amazing. One issue I found was that for Q80 models, it throws error saying

Error while parsing WGSL: :132:31 error: mixing '&' and '+' requires parenthesis v_dest[(out_base + 2) & 31u + block_index * 32] = f32(x3_unsigned)*d; ^^^^^^^ :133:31 error: mixing '&' and '+' requires parenthesis v_dest[(out_base + 3) & 31u + block_index * 32] = f32(x4_unsigned)*d;

You can reproduce this for phi example with phi_1_5_Q80 and puffin Q80 models.

Fix was a one line change I have put here - vandana-rajan@caf8bba

I can push this as a PR once your branch gets merged.

I tried to reproduce the error using:

cargo run --example phi --features="wgpu" --release -- --model 1.5 --quantized --prompt "A skier slides down a frictionless slope of height 40m and length 80m. What's the skier speed at the bottom?"

This seems to work on my side. I assume you used the candle-wasm-examples/phi example?

In general, the implementation in the browser may be stricter than the native drivers used by wgpu.

Interestingly, I am currently having some issues running:

wasm-pack test --chrome --features="wgpu" --test all

or

cargo build --target wasm32-unknown-unknown --release --features="wgpu"

because candle-core now depends on tokenizers (added in #3245), which appears to pull in onig_sys. That crate requires clang to be installed.

[KimHenrikOtte]

Thank you @vandana-rajan for pointing out the error. Generally, Chrome’s implementation is stricter about the format of the WASM file. I tested this branch in candle-wasm-tests using:

wasm-pack test --chrome --features="wgpu" --test all

In this branch, the build.rs of the project automatically copies the tests from candle-core, candle-nn, and candle-transformers. Unfortunately, I opted out of the quantized_tests in build.rs, so I never tested the quantized tests in the browser.

I fixed the error you mentioned and also found two similar locations where a parenthesis was missing.

[vandana-rajan]

Thats great. Thank you @KimHenrikOtte for the awesome work and prompt response.