Imagine you want to bake a cake in someone else's kitchen — but you don't trust them not to peek at your secret recipe. So you seal every ingredient in opaque containers, and the oven somehow bakes the cake without anyone ever seeing what's inside. That's fully homomorphic encryption (FHE) applied to model inference: you send encrypted data to a cloud provider running an LLM, the model computes on ciphertext directly, and the result comes back encrypted — only you can read it. The catch has always been that this "opaque oven" runs catastrophically slowly. Perseus is the first open-source Python library that tackles the entire stack at once and makes it fast enough to start being interesting. The committed claim: Perseus achieves state-of-the-art latency for encrypted inference across multiple models and modalities, notably decoding single tokens on Llama-3-8B 3.96× faster than any prior implementation, all within 3% of plaintext accuracy and perplexity. This isn't a single trick — it's three optimizations layered at different levels of the stack. At the cryptographic backend level, in-kernel regeneration of key-switching keys avoids redundant memory traffic. At the data-packing level, a complex variant of Cachemir packing exploits the structure of complex numbers to double slot utilization. At the model execution level, a bootstrap placer that is both packing-aware and magnitude-aware decides when and where to refresh ciphertext noise budgets — the costliest operation in FHE — with fewer total bootstraps. Where does this sit on the ladder? The paper names specific prior systems and beats them. The 3.96× speedup on Llama-3-8B single-token decoding is the headline number, measured against the fastest previously published implementation. Across other models the speedups vary but Perseus consistently holds the latency crown. Critically, accuracy stays within 3% of plaintext — this isn't speed gained by sacrificing correctness. The baselines are current and named, not straw-men from 2021. That said, the paper is only 4 pages with 5 tables; depth on ablations is thin. We don't get a clean decomposition of how much each of the three optimizations contributes independently. The architecture sits squarely in the CKKS (Cheon-Kim-Song-Song) approximate arithmetic family of FHE schemes, which is the dominant choice for neural-network inference because it handles real-valued computations natively. Perseus builds atop this with a custom backend (the in-kernel key-switching optimization targets GPU memory hierarchy), a packing strategy that is CKKS-specific, and a model-level planner that reasons about the noise budget across an entire computation graph. The hardware lean is GPU — this is not a CPU-only story. Integrity is mixed. The benchmarks are concrete: named models (GPT-2, Llama-3-8B, and others across modalities), named metrics (latency, accuracy, perplexity), and comparison against published numbers from other systems. The code is open-source, which is a strong integrity signal — anyone can reproduce. But there's no pre-registration, no independent replication yet, and the 4-page format means ablation details are sparse. The authors also initiated a "public speedrun" on encrypted GPT-2, which is a clever social mechanism to invite external validation, but it hasn't produced independent results yet. The milestone that matters is real-time encrypted inference. Today, even with a 3.96× speedup, single-token decoding on an 8B-parameter model under FHE is orders of magnitude slower than plaintext. The gap from "state-of-the-art FHE latency" to "usable interactive latency" (say, sub-100ms per token) is still enormous. Perseus moves the needle meaningfully, but the honest framing is: this makes FHE inference go from "completely impractical" to "slow enough that researchers can iterate." The public speedrun is the right move — compounding improvements across groups is the only way to close the remaining gap. The obvious experiments not run: scaling to larger models (70B+), running on real cloud hardware with network latency included, and head-to-head comparison with trusted execution environments (TEEs) or secure multi-party computation (MPC) on the same tasks. The authors likely didn't run these because (a) compute cost for FHE at 70B scale is still prohibitive and (b) comparing against TEEs would reframe the conversation from "fastest FHE" to "fastest private inference," which TEEs currently win on latency by orders of magnitude. That comparison is important but unflattering, and saving it for future work is understandable.
