You know how spell-checkers work? They don't try to understand your sentence — they just need to decide whether 'recieve' is closer to 'receive' or 'relieve.' The actual distance function can be crude as long as the ranking comes out right. This paper applies exactly that logic to encrypted neural networks: stop trying to approximate what ReLU does to every activation value, and instead find the cheapest polynomial that preserves which class wins at the output. A quadratic is enough. The committed claim: for a trained single-hidden-layer ReLU MLP, you can replace ReLU with a per-unit quadratic — degree 2, the minimum that's still genuinely nonlinear — without retraining, while preserving all classification decisions on a calibration set. The authors provide necessary and sufficient conditions for when this works exactly, plus a convex relaxation for when it doesn't (a few misclassified calibration points). This is not 'yet another polynomial approximation for HE-friendly inference.' It reframes the problem from function approximation to decision preservation, which is a categorically different optimization target. The ladder here is Remez-7 — the degree-7 minimax polynomial that's become the standard ReLU surrogate in FHE inference pipelines. On the activation module alone, the quadratic replacement runs 3.7–4.1× faster than Remez-7. End-to-end (including the linear layers that don't change), the speedup is 1.18–1.68×. Crucially, top-1 accuracy under CKKS encryption matches the plaintext network on multiple benchmarks. The quadratic doesn't sacrifice accuracy for speed — it gets both, because it's optimizing the right objective. Architecturally, this lives in the convex geometry / SVM family, not the polynomial-approximation family most HE-friendly inference work belongs to. The key structural move is 'lifting': a quadratic activation on a d-dimensional hidden layer produces a representation in a space of dimension O(d²), and the authors show that preserving decisions in this lifted space reduces to a linear separability problem. When separability fails, they use reduced convex hulls and Lagrangian-dual soft-margin relaxations — classic tools from the SVM era, repurposed. The computation is a set of small convex quadratic programs, not a retraining loop. The method leans on CKKS (approximate homomorphic encryption), where multiplicative depth is the binding constraint and degree-2 polynomials cost exactly one level versus four for degree-7. Integrity is mixed. The theoretical contribution — necessary and sufficient conditions for lossless quadratic replacement — is backed by proof, which is strong. The empirical results run on multiple benchmarks under CKKS and compare directly to Remez-7 with concrete speedup numbers. But the scope is limited to single-hidden-layer MLPs, which is a real constraint. The paper doesn't claim otherwise, but the practical relevance depends entirely on whether this extends to deeper networks. No code availability is mentioned. No independent replication exists yet. The milestone that matters: extending this decision-aware framework to multi-layer networks. Right now, the theory handles one hidden layer. Real FHE inference pipelines use networks with 5–20+ layers, each needing an activation replacement. If the decision-preservation argument composes across layers — or if a layer-by-layer greedy version works empirically — the 3.7–4.1× activation speedup would compound into something transformative for practical encrypted inference. The gap is probably 1–2 papers away, not a hardware problem. The obvious experiment the authors didn't run: multi-layer networks. The entire theoretical framework is built for a single hidden layer. Extending to depth is the natural next step and the paper doesn't attempt it. My read is (c) — they're saving it. The single-layer theory is clean and complete, with both exact conditions and a principled relaxation. Adding depth would muddy the contribution and probably require new machinery. Expect a follow-up within 12 months.
