Skip to content

Add kev-rust: Kev-0.8B inference example (CPU and CUDA) - #4040

Open
fadhilelrizanda wants to merge 1 commit into
huggingface:mainfrom
fadhilelrizanda:add-kev-rust-example
Open

fadhilelrizanda wants to merge 1 commit into
huggingface:mainfrom
fadhilelrizanda:add-kev-rust-example

Conversation

@fadhilelrizanda

Copy link
Copy Markdown

Adds `experiments/kev-rust`, a standalone crate that runs Jared Palmer's Kev-0.8B (Qwen3.5-0.8B + LoRA + pointer head) with Candle on CPU or NVIDIA CUDA.

  • Reads System One requests as JSONL, returns `choice`, `score` and `noul` answers.
  • Fused CPU DeltaNet and depthwise-conv loops; fused CUDA kernels behind the `cuda` feature.
  • Parity and benchmark results against the official Python implementation in `results/`, `BENCHMARK.md`.
  • `src/qwen3_5.rs` derives from Support for Qwen 3.5 Model Series (Dense and Quantized) #3396 (MIT/Apache-2.0), see `THIRD_PARTY.md`. Kev is Apache-2.0.
  • Weights, build outputs and virtualenvs are git-ignored; no changes outside `experiments/kev-rust`.

Not tested in CI. If you would rather not host this under `experiments/`, I am happy to move it to a separate repo and send only the Qwen3.5 model changes.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@fadhilelrizanda

Copy link
Copy Markdown
Author

Closing for now, will revisit.

@fadhilelrizanda

Copy link
Copy Markdown
Author

Reopening. Standalone version for reference: https://github.com/fadhilelrizanda/kev-rust

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant