Imagine you're a sous chef in a restaurant kitchen. The head chef (the big server model) decides what goes on the plate, but you can prep ingredients ahead of time. The old approach was to have one sous chef (a small language model on your phone) guess what the head chef will want next — and the chef either accepts or corrects. HyCo's insight is: what if the sous chef also had a copy of the head chef's own recipe notes? Now the phone runs TWO kinds of prep simultaneously — the head chef's own multi-token prediction head (shipped as hidden states, embeddings, and KV entries from the server) handles the depth of the draft tree, while the SLM handles the breadth by generating alternative branches. One pass each, no repeated NPU cycles for deep trees. The committed claim: by hybridizing the server's native MTP head with a local SLM drafter on a mobile NPU, you can reduce server-side GPU computation per output token by 38.6% on Spec-Bench and 29.2% on 32K-token summarization, relative to server-side MTP self-speculation — while maintaining output quality within noise. This is not a new decoding algorithm; it is a new system-level partitioning of speculative decoding across a device-cloud boundary, specifically designed for the mobile NPU constraint. The architecture story matters here. Speculative decoding is a well-established family: a cheap drafter generates candidate token sequences, and an expensive verifier accepts or rejects them in a single forward pass, preserving the verifier's distribution. HyCo's structural choice is to split the drafter into two roles assigned by tree topology — MTP head for depth (sequential token chains), SLM for breadth (alternative branches). The server transmits its hidden state, input embedding layer, and KV cache entries so the MTP head can run on the NPU without the full model. Quantization, a restricted vocabulary, and a 1K-token context window make both drafters fit the Galaxy S26 Ultra's NPU. An optional ε-threshold relaxation trades exact distribution preservation for higher acceptance rates. On the ladder: the baselines are real and named. Against server-side MTP self-speculation (the natural comparison for MTP-based drafting), HyCo saves 38.6% server GPU time on Spec-Bench. Against an SLM-only drafter under the same acceptance rule, HyCo delivers 9.0 percentage points more server GPU savings and 1.9× delivery speed. Without the ε-relaxation, savings drop to 27.7% and 11.9% — still meaningful but significantly lower, revealing how much of the headline number depends on approximate acceptance. Quality-wise, HyCo differs from Cloud+MTP by +0.2 judge-score points with a 95% CI spanning -0.9 to +1.3 — statistically indistinguishable. Integrity is mixed. The evaluation runs on real hardware (Galaxy S26 Ultra over LTE/5G), which is better than simulation, and Spec-Bench is a community benchmark. But three runs per system for the comparative results is thin — variance estimates will be wide. The 95% CI on quality is reassuringly honest (it includes negative territory), but summarization uses a single 32K-token task. The ε-relaxation is the load-bearing variable: with it, savings are 38.6%; without, 27.7%. That 11-point gap means the headline result is partly an acceptance-rate engineering story, not purely an architecture story. The exploratory translation result favoring a different configuration (SpecEdge-ε) hints at task sensitivity the paper doesn't fully resolve. The milestone question for mobile-cloud speculative decoding is throughput parity: when does the device-side drafting become fast enough that the server is never idle waiting for drafts? HyCo doesn't reach that point — mobile NPU inference is still the bottleneck — but the 1.9× delivery speed improvement over SLM-only drafting suggests the hybrid approach closes the gap. The next concrete number to watch: server GPU savings above 50% on long-context tasks without ε-relaxation. HyCo exact hits 27.7% on Spec-Bench and only 11.9% on 32K summarization. Closing the gap between exact and relaxed modes is the real engineering frontier. The experiment they didn't run: multi-client scaling. All results are single-client. The entire value proposition of reducing server GPU time is that you can serve more clients per GPU — but they never measure actual multi-client throughput or latency under load. My honest read: this is (a) compute/infrastructure limitation. Running a proper multi-client serving benchmark with real mobile devices over real networks is an order of magnitude harder than single-client profiling, and the Samsung collaboration likely prioritized demonstrating the on-device feasibility first. The multi-client story is almost certainly the next paper.
