A Negotiation Robot That Reads Your Face and Learns Your Tells in Real Time

Continual learning personalizes facial affect reading during live human-robot negotiation, boosting agent utility and concession rates over static baselines. N=66, within-subject.

Imagine you're playing poker with someone for the first time. At the start, you're reading their face the same way you'd read anyone's — generic tells, textbook micro-expressions. But by the third hand, you start noticing that THIS person's slight jaw clench doesn't mean frustration, it means they're bluffing. You're building a personalized model on the fly. That's the core mechanism here: a negotiation agent that swaps out a frozen, one-size-fits-all facial expression model for one that continually fine-tunes itself to each individual human during the negotiation itself. The committed claim: coupling continual learning for facial expression recognition (CLIFER) with a differential feedback mechanism produces a negotiation agent that achieves significantly higher individual utility and elicits higher human concession rates than the same agent running a static FaceChannel baseline. This was tested in a within-subject design with 66 participants negotiating with a physical Pepper robot — not a screen avatar, a real embodied agent. The architecture sits in the continual learning family, specifically experience replay with a generative imagination component. CLIFER maintains a replay buffer and periodically generates synthetic exemplars to fight catastrophic forgetting as it fine-tunes on each participant's valence-arousal signals. The key trick is an asynchronous update pipeline: while the robot negotiates in real time, CLIFER updates in the background, so inference latency doesn't stall the interaction. The differential feedback module doesn't use raw valence-arousal values — it uses changes in direction and intensity to adjust the agent's target utility, creating a closed loop between perceived affect and bidding strategy. On the ladder: the baseline is FaceChannel, a relatively standard CNN-based affect model that has been used in prior affective computing work but is not the current top of the leaderboard for facial affect recognition (models like EMOCA or more recent transformer-based approaches exist). The CL-enabled agent achieved significantly higher individual utility and concession rates (p-values reported). However, participant utility and Nash distance did not differ significantly between conditions — meaning the agent extracted more without making the human measurably worse off, which is interesting but also raises the question of whether the gains are structural or whether participants simply didn't notice the squeeze. Integrity is mixed. The within-subject, counterbalanced design with 66 participants is solid for HRI research — this is not a 12-person convenience sample. But there's no pre-registration, no independent replication, and the affect model is evaluated against a single baseline rather than a battery of current SOTA affect models. The robot is a Pepper, which constrains generalizability to that specific embodiment. The significant arousal estimation differences are interesting but the valence results are less clear, and the broader output range of CLIFER could reflect genuine personalization or simply model instability. The milestone question is telling. The authors demonstrate personalization over a single negotiation session. The real unlock is whether CLIFER-style approaches can accumulate personalization across multiple sessions with the same person — longitudinal adaptation, not just within-session calibration. That would require 5-10 session longitudinal studies with the same participants, which the authors did not run. My read: (a) resource constraints on participant recruitment and (b) saving longitudinal results for a follow-up paper. The obvious gap is also testing against stronger affect baselines and in adversarial or high-stakes negotiation contexts where facial affect management (i.e., deliberate poker-facing) is part of the strategy. This paper matters most to the HRI and affective computing communities building interactive agents that need to adapt to individual users in real time. It's a clean demonstration that continual learning can be made practical in a live interaction loop. The broader AI negotiation community should note the differential feedback mechanism — using affect deltas rather than absolute values is a design choice that deserves more attention. But the single-baseline comparison and lack of longitudinal data keep this firmly in 'promising demonstration' territory rather than 'settled result.'