
Read the paper
TacInno: Teaching Vision-Language-Action Models the Contact Dynamics Beyond Vision and Action
on Project Page
Imagine you're learning to parallel park with a driving instructor narrating everything you can already see — 'there's a curb on your right, there's a car ahead.' Useless. What you actually need is the information you CAN'T get from your eyes: the moment your tire just barely touches the curb, the subtle resistance when your bumper is an inch from the car behind you. That's TacInno's core insight applied to robot manipulation: strip away every tactile signal the robot could already predict from its camera and motor commands, and train only on the residual — the tactile surprise. The committed claim: by isolating what the authors call 'tactile innovation' — the component of touch that vision and executed actions cannot explain — and using it as an auxiliary training target rather than a direct input, a Vision-Language-Action model learns contact-rich manipulation significantly better than any prior visuo-tactile VLA. The numbers are 87.5% average success across six simulation tasks and 70.5% across four real-world tasks, beating the strongest visuo-tactile baseline by 10.7 and 9.0 percentage points respectively. The mechanism is elegant in its information-theoretic clarity. A TacInno tokenizer is trained to take visual observations and executed actions as conditioning inputs, then compress the future tactile change into a discrete code. Because the tokenizer is conditioned on what vision and action already know, the discrete code captures only the residual — the genuine surprise in the tactile stream. The policy network, TacInno-VLA, learns to predict this code during training as an auxiliary objective. At deployment, the code-prediction head is discarded entirely. No tactile sensor is needed at test time. The robot has internalized a richer model of contact dynamics without any additional hardware burden at inference. This sits in a live debate within embodied AI: does adding tactile sensing to VLA models actually help, or does it just add noise? Prior work (including models like ManipTac and TacSL) has shown frustratingly inconsistent gains from tactile input. The reason, TacInno argues, is that much of the tactile signal — gel deformation during gripper closure, for instance — is already predictable from the visual stream and the robot's own commanded actions. Feeding the full tactile signal into the policy gives the model a shortcut to ignore touch entirely and lean on vision. TacInno's answer is to never give the policy raw touch at all; instead, force it to predict only the unpredictable part. The validation regime is reasonably thorough. Six simulation tasks span diverse contact-rich scenarios, four real-world tasks ground the results on physical hardware, and ablation studies systematically isolate the contributions of visual conditioning, action conditioning, multi-frame encoding, and the auxiliary code-prediction objective. The baselines include recent visuo-tactile VLA models, not just stale comparisons. That said, the real-world task set is still small (four tasks), and the absolute success rate of 70.5% in the real world reminds you these are hard problems with ample room to improve. The architectural choice — training a VQ-VAE-style tokenizer conditioned on non-tactile modalities, then using the discrete codes as auxiliary prediction targets — is a clean instance of information bottleneck thinking applied to multimodal robotics. It belongs to the family of auxiliary self-supervised objectives (like next-token prediction or contrastive learning) used to shape internal representations, but applied specifically to strip a modality down to its unique informational contribution. The key hardware property it leans on is that tactile sensors (GelSight-style) produce high-dimensional signals that are largely redundant with vision during routine manipulation phases. The obvious next experiment is scaling to tasks where tactile innovation is not just helpful but essential — tasks where vision is severely degraded (occlusion, darkness, transparent objects) and where the residual tactile signal carries the majority of task-relevant information. The authors don't run this, and the honest read is a mix of (a) and (c): their current sim and real-world setups don't naturally produce heavy occlusion scenarios, and demonstrating robustness under vision degradation is likely the centerpiece of a follow-up paper.
The analogy
Like a driving instructor who only speaks up when something happens you can't see — the curb touching your tire, the bump you didn't expect — instead of narrating everything already visible through your windshield. TacInno filters robot touch to only the surprise signal, discarding what the camera already knows.
Smart instructor
Only speaks when the student can't see the obstacle — silent during routine maneuvers, vocal at blind-spot contact
TacInno tokenizer
Conditioned on vision and action, compresses tactile change to only the residual surprise — the information neither modality already carries
Honest assessment
TacInno is a clean, well-motivated contribution that correctly diagnoses why naively adding tactile input to VLA models often fails — most of the signal is already in vision. The information-bottleneck approach is principled and the gains are consistent across sim and real. But the real-world validation is thin (four tasks), the absolute success rate of 70.5% shows how far contact-rich manipulation still has to go, and the most interesting regime — where vision is degraded and tactile innovation should dominate — remains untested. Worth watching, not yet load-bearing for deployment.
What Is CLAIMS?
67
/ 100
BedrockNews applies a structured analytical framework called CLAIMS to academic papers and technical research releases.
How CLAIMS worksClaim
3/5
Ladder
4/5
Architecture
4/5
Integrity
3/5
Milestone
2/5
Successor
4/5
What is this paper actually committing to?
Real but not first-of-kind. The core idea — filtering modalities to their unique residual — is established in information theory. The novelty is applying it cleanly to tactile VLA training with a concrete mechanism.
The field fight
Does adding tactile sensing to Vision-Language-Action models genuinely improve contact-rich manipulation, or does it just add redundant noise?
Side A
Raw tactile input helps — more modalities = more information for the policy
Held by: ManipTac, TacSL, early visuo-tactile VLA work
Side B
Raw tactile input is mostly redundant with vision; only the tactile residual matters
Held by: TacInno (this paper)
Milestone tracker
Unlocks: Reliable deployment of tactile-aware manipulation in unstructured real environments without tactile sensors at inference
Current
70.5
% real-world success
TacInno (4 tasks)
Next
85
% real-world success
10+ diverse tasks incl. occluded
~1-2 years; requires standardized contact-rich benchmarks
Verified Facts (6)
TacInno-VLA achieves average success rates of 87.5% across six simulation tasks and 70.5% across four real-world tasks
Outperforms the strongest visuo-tactile baseline by 10.7 and 9.0 percentage points in simulation and real-world respectively
The TacInno tokenizer is conditioned on visual observations and executed actions and compresses future tactile change into a discrete code
The policy adds no module for the code at deployment
Ablation studies validate the contributions of visual and action conditioning, multi-frame encoding, and auxiliary code prediction
11 pages, 4 figures, 4 tables
Practical Implications (3)
Roboticists adding tactile sensors to VLA policies should consider filtering touch to only the vision-unpredictable residual rather than feeding raw tactile streams
The deploy-time simplicity is immediately practical — train with touch, deploy without tactile hardware, reducing sensor cost and failure modes
The VQ-VAE residual-extraction pattern is transferable to other modality-fusion problems where one modality is partially redundant with another
More latest news
Read all