Locus: Owner-Drained Chunk Mailboxes for KV-Block Recycling in CPU LLM Inference
- authors
- Victor Bona
- status
- independently published
- published
- open pdf
- cite key
- bona2026locus
cat abstract
A technical white paper on Locus, a domain memory pool for CPU LLM serving whose remote-free path is a per-worker lock-free chunk mailbox: a finished request's KV blocks are returned as one atomic push, and the pool owner drains every mailbox off the allocation hot path, so freeing never contends on a shared queue and the design carries zero tuning parameters. Developed under a falsification-first methodology, it is evaluated on LOCUS-EVAL v1, a frozen four-workload suite of deterministic serving-shaped KV traces run against jemalloc, mimalloc, and system malloc on Apple Silicon, where the mailbox ranks first on every workload. A self-critical touch-parity audit corrects the headline: at a one-byte-per-block parity the advantage is 1.6x to 2.7x over mimalloc on the three trace workloads and compresses to about 1.15x over system malloc once full KV writes dominate, with a transient burst-cancellation footprint of up to 1.5x the theoretical peak. The scope is deliberately narrow (a single macOS host, synthetic traces, an allocator-level measurement), with NUMA locality and end-to-end serving integration stated as future work.
ls index-terms/
RustLLM InferenceKV CacheMemory PoolAllocatorsBenchmarks
cite --format=bibtex
@misc{bona2026locus,
author = {Victor Bona},
title = {Locus: Owner-Drained Chunk Mailboxes for KV-Block Recycling in CPU LLM Inference},
year = {2026},
month = {jul},
url = {https://blog.victorbona.dev/papers/locus-whitepaper.pdf},
note = {Independently published}
}