Skip to content

INTEGRATION.md

Latest commit

 

History

History
271 lines (209 loc) · 11.8 KB

INTEGRATION.md

File metadata and controls

271 lines (209 loc) · 11.8 KB
Raw

Integration Guide

How to attach sefer-alloc to a project as the global allocator and tune its three operational knobs at compile time: size limit, release period, release trigger.

This guide is a consolidated answer to "how do I use it in production". For the wider documentation map see README.md § Documentation map.

1. Add the dependency

Pick the feature set that matches the workload. The recommended starter for multi-thread / long-running processes is production:

[dependencies]
sefer-alloc = { version = "0.3", features = ["production"] }

production is an alias for `alloc-global + alloc-xthread + alloc-decommit

  • fastbin— the drop-inGlobalAlloc` face, lock-free cross-thread free, M6 decommit (returns empty segments to the OS), and the per-thread fast-bin magazine.

Other valid feature shapes:

Shape features = [...] Use case
Handle store only (default) omit Region<T> / Handle<T> for typed slot storage
no_std + alloc core default-features = false embedded targets
Single-thread allocator ["alloc-global"] single-thread process
Multi-thread allocator ["alloc-global", "alloc-xthread"] multi-thread, no segment recycling (1024-segment ceiling)
Recommended for servers ["production"] long-running multi-thread (DBMS, async runtime)
production + NUMA ["production", "numa-aware"] multi-socket NUMA hardware

Full feature matrix: README.md.

2. Install as #[global_allocator]

One declaration at the crate root (binary or library top-level):

use sefer_alloc::SeferAlloc;

#[global_allocator]
static GLOBAL: SeferAlloc = SeferAlloc::new();

fn main() {
    // Every Vec/Box/String/HashMap allocation in this process — including
    // those made by tokio, async-std, std collections, third-party crates —
    // now goes through sefer-alloc.
    let v: Vec<u8> = (0..1024).collect();
    println!("{}", v.len());
}

That's the full integration. Everything below is optional tuning via the LargeCacheConfig const builder — a code change, no recompile needed.

3. The three compile-time knobs (alloc-decommit)

The large-segment cache is the only piece of the allocator with tunable policy. It governs how aggressively empty large blocks are returned to the OS versus held in a per-shard free-list for reuse. Configuration is via LargeCacheConfig — a Copy + Clone + const fn builder — passed to SeferAlloc::with_config(...). All methods are const fn, so the config lives in a static initialiser and is resolved at compile time (zero runtime overhead, no env reads, no parse errors).

3.1 Size limit — how much memory the cache may hold

Two knobs work together. The cache admits any large free until the budget is hit; headroom is the level below which the decay does not pull memory back to the OS (anti-thrashing floor).

Builder method Default Meaning
.budget_bytes(n) None (unbounded) Per-shard hard ceiling on cached bytes. 0 = cache disabled (every span released to the OS immediately). No admission limit when unset; FIFO eviction fires only when this is set and the new span would exceed it.
.headroom_bytes(n) 256 MiB Floor below which the periodic decay is a no-op. Above this, excess = cached − headroom is the amount eligible for release.

Containers / RSS-sensitive deployments: call .budget_bytes(512 * 1024 * 1024) (or whatever your RSS ceiling is). Without it the cache will retain whatever the workload churned through until the OS or the decay clock pulls it back.

3.2 Release period — how often the cache releases excess

Builder method Default Meaning
.decay_interval_ms(n) 1000 (1 s) Minimum wall-clock ms between two consecutive decay ticks. A tick computes excess = cached − headroom and releases excess × rate back to the OS.
.decay_rate_percent(n) 10 (10 %/tick) Fraction of the excess released on each tick, in integer percent [1, 100]. Values outside the range are clamped.

The model is self-damping exponential decay: each tick removes a constant fraction of the current excess, so the cache approaches headroom aggressively when far above it and gently when near it. No oscillation, no spike. An idle process pays nothing — the tick is gated by the very next alloc/free that happens to be a large one (the "lazy" trigger below).

3.3 Release trigger — event-driven or thread-driven

Builder method Default Meaning
.mode(m) LargeCacheMode::Lazy Selects how the decay tick fires.

  • LargeCacheMode::Lazy (default, fully implemented). Event-driven: each large alloc and free checks whether decay_interval_ms has elapsed since the previous tick; if so, exactly one decay step runs inline on that call. No background thread, no extra syscall on the common path, no allocation. Idle process → no work. This is the mobile/embedded/serverless-friendly mode.

    LargeCacheMode is #[non_exhaustive]; Lazy is currently the only variant. A future background-scavenger mode (e.g. a dedicated thread visiting idle shards on a timer so a quiescent shard still gets decay) may be added as a non-breaking change, but it does not exist today — earlier Background/Both placeholders were never implemented and have been removed from the enum.

Practical recommendation: keep the default Lazy. It already covers DBMS / tokio servers / async runtimes where allocation pressure is continuous, as well as idle workloads (a tick is gated by the next large op, but headroom bounds how much can accumulate before that op fires).

4. Worked example — RSS-bounded server

A containerised tokio server with a 512 MiB RSS ceiling, aggressive trimming every 200 ms:

# Cargo.toml
[dependencies]
sefer-alloc = { version = "0.3", features = ["production"] }
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }
// src/main.rs
use sefer_alloc::{SeferAlloc, LargeCacheConfig, LargeCacheMode};

const CACHE_CONFIG: LargeCacheConfig = LargeCacheConfig::new()
    .budget_bytes(512 * 1024 * 1024)      // hard ceiling: 512 MiB per shard
    .headroom_bytes(64 * 1024 * 1024)     // floor: don't decay below 64 MiB
    .decay_interval_ms(200)               // tick every 200 ms
    .decay_rate_percent(25)               // release 25 % of excess per tick
    .mode(LargeCacheMode::Lazy);          // event-driven (no background thread)

#[global_allocator]
static GLOBAL: SeferAlloc = SeferAlloc::with_config(CACHE_CONFIG);

#[tokio::main(flavor = "multi_thread", worker_threads = 4)]
async fn main() {
    // ... your server ...
}

What this means in practice:

  • Cache will never hold more than 512 MiB of large segments per shard.
  • Below 64 MiB the cache is left alone (anti-thrashing).
  • Every 200 ms, on the next large alloc/free, 25 % of the excess above 64 MiB is returned to the OS.
  • Idle process → no ticks, no work, no syscalls.

For a desktop / dev profile that retains memory more aggressively for throughput, use SeferAlloc::new() (equivalent to SeferAlloc::with_config(LargeCacheConfig::DEFAULT)): the defaults (headroom=256 MiB, interval=1 s, rate=10 %/tick, unbounded budget) are already tuned for the throughput-first case.

5. Verifying the configuration is live

Honest scope: the dbg_* config accessors (dbg_decay_config, dbg_large_cache_mode, dbg_large_cache_used, dbg_large_cache_hits, …) live on AllocCore, the low-level per-thread segment substrate — not on SeferAlloc itself. There is currently no method on SeferAlloc (the type you actually hold as #[global_allocator]) that reaches into its internal per-thread AllocCore to read these back; the accessors are exercised directly against a standalone AllocCore::new_with_config(cfg) instance in the crate's own test suite (see tests/large_cache_config_knobs.rs, tests/large_cache_mode.rs), which is a different, additional AllocCore built purely to assert the config plumbing — not the one servicing your process's actual #[global_allocator] allocations.

If you want to confirm your LargeCacheConfig is wired the way you expect before deploying it as #[global_allocator], build the identical config against a standalone AllocCore in a unit test and read it back the same way the crate's own tests do:

# #[cfg(feature = "alloc-decommit")]
# {
use sefer_alloc::alloc_core::{AllocCore, LargeCacheConfig, LargeCacheMode};

const CONFIG: LargeCacheConfig = LargeCacheConfig::new()
    .budget_bytes(512 * 1024 * 1024)
    .decay_rate_percent(25)
    .mode(LargeCacheMode::Lazy);

let ac = AllocCore::new_with_config(CONFIG).expect("OS reservation failed");
let (rate_bp, _interval_ms, _headroom) = ac.dbg_decay_config();
assert_eq!(rate_bp, 2500); // dbg_decay_config's rate is in basis points (25% = 2500 bp)
assert_eq!(ac.dbg_large_cache_mode(), LargeCacheMode::Lazy);
# }

This verifies the LargeCacheConfig builder produced the values you intended — it does NOT verify the #[global_allocator] process's live per-thread heap, because that heap is not reachable from outside the crate.

Track RSS over time with ps/top/docker stats against the REAL running process instead — this is the only externally-observable signal for the live #[global_allocator] configuration today. The decay profile becomes visible: aggressive settings produce a sawtooth that damps toward headroom; default settings produce a gentle slope. A follow-up improvement (not yet implemented) would be a public SeferAlloc::dbg_* passthrough for exactly this purpose.

6. The sub-crates (optional)

If you only want one of sefer-alloc's building blocks, you can cargo add it independently — these are real crates.io packages, not just internal modules:

Crate What it gives you When to use
sefer-region Region<T> / Handle<T> / SyncRegion<T> typed handle store typed slot storage without an allocator stack
aligned-vmem SEGMENT-aligned mmap / VirtualAlloc + page decommit/recommit building your own allocator on top of a verified OS aperture
numa-shim NUMA detection + binding (mbind / VirtualAllocExNuma, no libnuma) NUMA-aware code without C dependencies
malloc-bench-rs portable GlobalAlloc benchmark harness (larson + mstress) benchmarking your own allocator

sefer-alloc re-exports sefer-region's surface, so existing code using use sefer_alloc::{Region, Handle, SyncRegion}; continues to work unchanged.

7. Common pitfalls

  • Forgetting #[global_allocator]. The dependency builds and the type compiles, but every allocation still goes through the system allocator. The declaration must be on a static at the crate root, not inside a function.

  • Wrong feature set. cargo add sefer-alloc without --features production (or at least --features alloc-global) gives you the handle store only — SeferAlloc is not exported.

  • with_config only available under alloc-decommit. The LargeCacheConfig type and SeferAlloc::with_config are only compiled when the alloc-decommit feature is on (included in production). Without it, use SeferAlloc::new() (the only constructor).

  • Setting budget_bytes smaller than headroom_bytes. Legal but pointless — the cache will be forced into FIFO eviction at the budget ceiling before decay ever kicks in. Keep headroom < budget (or leave budget unset for the throughput-first default).