You know how a good lip-reader doesn't actually watch your lips — they watch your whole face, reading jaw tension, cheek shape, tongue shadow? They're not decoding sound; they're decoding the motor program that produced the sound. This paper asks: if we want a machine to animate a 3D face from speech, which speech representation best captures that motor program? The committed claim: discrete speech tokens — specifically those encoding phonetic classes — produce facial animation quality comparable to richer continuous SSL features, and this shared discrete space can simultaneously drive both speech synthesis and 3D facial motion. The paper evaluates four representation families: self-supervised learning (SSL) features that capture segmental and semantic cues, neural audio codecs optimized for acoustic reconstruction, ASR-style label-based representations, and a fourth family completing the taxonomy. The result is a systematic bake-off across two different facial decoders, using both objective metrics and perceptual evaluation. The ladder here is internal, not external. The paper's contribution is comparative rather than SOTA-chasing: it probes which representation families encode phonetic units and articulatory deformations, then measures how those encodings translate to facial reconstruction quality. The key finding is that phonetic class encoding is beneficial for accurate facial animation, and that semantic-level SSL representations and label-based representations achieve comparable quality. No single named prior system is beaten on a headline number — this is a representation-selection paper, not a leaderboard paper. Architecturally, the work sits in the encoder-decoder family for speech-driven animation, where the encoder is swappable (that's the whole point) and two facial decoders serve as the test bed. The AVTTS pipeline introduced at the end uses discrete tokens as a shared latent space — one tokenization drives two decoders (audio and 3D mesh), which is the architectural novelty. This is a supervised, regression-style approach on the face side, with the speech representations coming from pre-trained models (HuBERT-family SSL, neural codecs, ASR systems). Integrity is reasonable for a conference paper. The authors use both objective metrics and perceptual evaluation, which is the right call for facial animation where numbers alone miss uncanny-valley effects. The probing analyses relating tokens to phonetic units and articulatory deformations add mechanistic depth beyond just reporting reconstruction error. But the abstract doesn't name specific datasets, sample sizes, or the perceptual study's participant count, which limits what we can verify from this summary alone. Accepted at Interspeech 2026 — peer-reviewed, not just a preprint. The milestone trajectory points toward real-time, unified audio-visual speech synthesis. Right now, we have a proof-of-concept AVTTS pipeline using discrete tokens as a shared space. The next concrete target is whether this discrete-token approach can scale to emotionally expressive, real-time 3D avatar animation — think video-call avatars that move naturally from a text prompt. That requires moving from lab-quality mesh animation to production-grade rendering with emotion and prosody control. The obvious experiment not run: end-to-end training where the speech tokenizer is jointly optimized with the facial decoder, rather than using frozen pre-trained representations. The authors chose to probe existing representations — a cleaner scientific question — but joint fine-tuning would likely improve results. My read: this is choice (c), saving it for the next paper. The probing-first approach is the right scientific sequence, and the AVTTS pipeline at the end is already the bridge toward that follow-up.
