Imagine you run a hotel where guests share rooms by the hour. You track which guest is in which room using a big reservation ledger on the front desk. The rooms themselves are inside a sealed building — you can't walk the hallways to check. If the ledger has a typo — a guest assigned to an occupied room, or a room marked empty when someone's still in it — nobody crashes, nobody screams. The guest just quietly gets the wrong towels, or worse, reads another guest's mail. That's the KV cache in modern LLM serving: a custom GPU-side allocator (PagedAttention) that partitions one CUDA memory region into fixed blocks, mapped by a CPU-side block table. The bugs aren't segfaults. They're silent corruption. The committed claim: KVGuard is the first systematic memory-safety checker for KV cache management in LLM serving systems. It defines five categories of violation — double-free, use-after-free, out-of-bounds access, uninitialized read, and permission violation — that are invisible to existing CUDA-level sanitizers because they operate at the wrong abstraction level. NVIDIA Compute Sanitizer sees raw GPU addresses; it cannot reason about block tables, per-request permissions, or KV entry initialization. KVGuard adds a compact shadow block per physical KV block that records this metadata and checks it at every boundary in the PagedAttention workflow: CPU allocation, block table construction and translation, GPU-side lookup, and KV entry access. The ladder here is straightforward because there's almost nothing on it. Existing tools are NVIDIA Compute Sanitizer (CUDA address-level, 100× overhead, wrong abstraction) and AddressSanitizer/Valgrind (CPU-only, irrelevant to GPU block tables). KVGuard doesn't beat a competitor — it defines a category. The evaluation is against the absence of any prior tool capable of detecting these bug classes. Against Compute Sanitizer on the same workloads, KVGuard is roughly two orders of magnitude cheaper in both time and memory overhead. Architecturally, KVGuard is a shadow-memory sanitizer — the same family as AddressSanitizer and MemorySanitizer from the C/C++ world — but rebuilt for a custom allocator inside a GPU serving pipeline. The key structural move is that shadow metadata lives per-block (not per-byte), which keeps the memory footprint at 0.11% of the KV cache. Each of the five checks fires at the CPU/GPU boundary where the violation was introduced, not downstream where corruption surfaces. This is a debugging-time tool, not a production-hardening layer — the 3-7% overhead is acceptable during development but not free at serving scale. Integrity is strong for a systems paper. The authors collected all 29 historical KV cache safety bugs from the patch histories of vLLM and SGLang — the two dominant open-source LLM serving systems — and demonstrated 100% coverage. They then ran KVGuard on current codebases and found 8 previously unknown violations with zero false positives. The bugs are real (extracted from commit histories), the systems are real (vLLM and SGLang are production-grade), and the overhead numbers are measured on Qwen 2.5 14B. The main integrity caveat: the authors define the bug taxonomy themselves, so the coverage claim is somewhat circular — they detect the categories they defined. But the categories map cleanly to classical memory-safety violations, and the 8 new bugs are a strong signal. The milestone question is about adoption, not algorithmic scaling. The next concrete number: if vLLM or SGLang ships KVGuard as an integrated debug mode (or CI check), the tool moves from academic artifact to infrastructure. The 8 unknown bugs already found suggest real value; the question is whether the 3-7% overhead is low enough for always-on CI testing of KV cache correctness. For the broader field, the real milestone is whether KV cache bugs become a recognized vulnerability class — right now they're treated as implementation quirks, not security surface. The obvious experiment not run: adversarial exploitation. The paper frames these as correctness bugs, but several categories (permission violation, use-after-free reading another request's KV data) are also isolation violations with security implications in multi-tenant serving. A demonstration that a crafted request sequence can reliably read another user's context through a KV cache bug would turn this from a dev-tools paper into a security paper. My read: the authors are saving this for a follow-up, or the exploitation is harder to demonstrate reliably than the detection.
