FastQueue (fq)

A fast and simple ring-buffer-based single-producer, single-consumer queue with no dependencies. You can use this to write Rust programs with low-latency message passing.

Quickstart

use fq::FastQueue;
use std::thread;

let (mut producer, mut consumer) = FastQueue::<String>::new(2);

let sender = thread::spawn(move || {
    producer.push("Hello, thread".to_owned())
        .expect("Unable to send to queue");
});

let receiver = thread::spawn(move || {
    while let Some(value) = consumer.next() {
        assert_eq!(value, "Hello, thread");
    }
});

sender.join().expect("The sender thread has panicked");
receiver.join().expect("The receiver thread has panicked");

Batch operations

When elements arrive or drain in bursts, the batch APIs move a whole batch for at most one atomic load and one atomic store instead of paying a synchronization cost per element. Use push_slice/pop_slice for Copy types (a straight memcpy into and out of the ring) and push_iter/pop_iter for everything else. Use push/pop when each message needs immediate publication; a batch publishes its progress when that batch finishes.

use fq::FastQueue;

let (mut producer, mut consumer) = FastQueue::<u64>::new(1024);

producer.push_slice(&[1, 2, 3, 4]);
let mut buf = [0; 1024];
let n = consumer.pop_slice(&mut buf);
assert_eq!(&buf[..n], &[1, 2, 3, 4]);

Examples

See the examples directory for more usage examples.

How does it work?

The ring buffer structure allows for a contiguous data structure. The idea is that if we are able to get extreme cache locality, we can improve performance by reducing cache misses. This is also the reason why if you use smart pointers like Box<T>, performance may degrade since cache locality gets degraded. For very large T types, you are more limited by memcpy() performance and less from queue implementations. As such, ring buffers can be considered strongly optimized for data of a few word sizes with some non-linear performance degradation for larger sizes. Additional optimizations are provided for CPUs that support sse and prfchw instructions. As and when Rust std provides more relevant instructions, they will be added. This is simply a high-level explanation of some of the techniques employed by this crate, you can read the code to gain a better understanding of what's happening under the hood.

Profiles

The crate is fully synchronous and runtime-agnostic. We are heavily reliant on std for memory management, so it's unlikely that we will support #[no_std] runtimes anytime soon. You should be using the release or maxperf profiles for optimal performance.

Principles

• This crate will always prioritize message throughput over memory usage.
• This crate will always support generic types.
• This crate will always provide a wait-free and lock-free API.
• This crate will use unsafe Rust where possible for maximal throughput.

CPU Features

On x86 and x86_64 targets, prefetch instructions are used on the stable toolchain. On aarch64, prefetch instructions are only available on the nightly toolchain.

Benchmarks

Benchmarks are strictly difficult due to the nature of the program, it's somewhat simple to do a same-CPU bench but performance will still be affected based on the core type and cache contention. Benchmarks are provided in the benches directory and can be run with cargo bench. Contributions via PRs for additional benchmarks are welcome.

Run throughput and round-trip latency benchmarks separately:

cargo bench --bench bench
cargo bench --bench latency

License

Licensed under either of:

• MIT license (LICENSE-MIT)
• Lesser General Public license v3.0 or later (LICENSE-LGPL) at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the LGPL-3.0 license, shall be dual licensed as above, without any additional terms or conditions.

Acknowledgements

Inspired from the previous works of fastqueue2, rigtorp and rtrb.