The myth of AI emotion recognition: Science or sales pitch?
The myth of AI emotion recognition: Science or sales pitch?
Bold claims from Google and WaveForms AI reignite ethical debates, but the real question is whether the technology works at all.
Last week, Google announced that a new family of models it developed, PaliGemma 2, can recognize and analyze human emotions, partly by detecting tiny facial movements. The tech giant's research team explained in a recently published article that after fine-tuning, the model can identify emotions, provide broader context to the analyzed scene, and generate corresponding subtitles. A few days later, Alexis Conneau, a former OpenAI researcher specializing in voice recognition, unveiled his new venture, WaveForms AI, which he claims will also enable the recognition of human emotions.
These announcements sparked heated discussions about the ethical implications of decoding emotions. Adding fuel to the debate, a UK study released around the same time revealed that some companies have begun deploying emotion-detection tools in workplaces to manage and evaluate employees.
The conversation around the risks of using emotion-recognition tools quickly escalated. Last week, several UK and EU lawmakers announced plans to strengthen protections for employees against such algorithms, introducing the concept of Algorithmic Management to regulate their use.
While protecting employees from arbitrary algorithmic decisions is essential, the broader debate plays into the hands of tech companies and entrepreneurs working on emotion-recognition technology. As we focus on the ethical implications of deploying these tools, we often overlook a fundamental question: Can these technologies even do what their creators claim?
1. Does a smile mean happiness?
The science of emotion recognition has existed for centuries, gaining traction with Charles Darwin, who suggested that muscle movements are involuntary responses that express emotions. He argued these expressions are part of evolutionary mechanisms, implying that emotions are innate, identifiable, and measurable.
In the 19th century, French neurologist Guillaume Duchenne conducted controversial experiments, electrically stimulating facial muscles (without consent) to induce "universal" expressions. His findings laid the groundwork for psychologist Paul Ekman, whose research in Papua New Guinea during the 1960s and 1970s became the foundation for much of today’s emotion-recognition technology.
Ekman identified 40 facial muscle movements (cataloged in the Facial Action Coding System, or FACS) and argued that humans share six universal emotions: happiness, sadness, disgust, fear, anger, and surprise. He claimed these emotions are cross-cultural, innate, and uniform. He noted that these emotions can be objectively identified regardless of cultural, gender, or individual differences—even if the person isn’t aware of their own feelings.
Although Ekman’s work was groundbreaking at the time, his conclusions remain controversial among scientists, including psychologists, anthropologists, and technologists. The core issues lie in defining what emotions are, how they form internally, and how they manifest externally.
A 2019 review of emotion-recognition research found no reliable evidence that emotions can be inferred from facial movements. Neuroscientist Lisa Feldman Barrett noted, “It is not possible to confidently infer happiness from a smile, anger from a scowl, or sadness from a frown, as much of current technology tries to do when applying what are mistakenly believed to be the scientific facts.”
Despite this, Ekman’s theories have spawned a $50 billion industry. Companies like Microsoft, IBM, and Amazon have deployed emotion-recognition tools in workplaces, recruitment processes, and even public spaces, such as airports. While these technologists may not intentionally develop flawed products, the science doesn’t support the claims they make—and yet, people continue to believe in them.
2. The danger of emotion-based discrimination
The primary issue with emotion-recognition technology lies in its conceptual leap: recognizing facial expressions is not the same as recognizing emotions. This distinction has profound implications. For example: An airport passenger misidentified as "fearful" could be flagged as a security threat. Or a job candidate whose facial expressions suggest "anger" might be unfairly labeled as unsuitable.
Such scenarios open the door to discrimination based on perceived emotions that individuals may not even feel.
This isn’t the first time the tech sector has overstated its capabilities. The debate surrounding artificial general intelligence (AGI) follows a similar pattern. Discussions often center on hypothetical scenarios where AI surpasses human intelligence, distracting from immediate, tangible issues. While some, like Elon Musk, propose brain-machine interfaces to keep up with future AI, others, such as philosopher Nick Bostrom, envision humanity evolving into multi-planetary, cloud-based beings after a nuclear holocaust.
In reality, breaking the AGI barrier is highly unlikely anytime soon. Meanwhile, this speculative narrative diverts attention from pressing global issues such as poverty, inequality, or the environmental impact of technology. For instance, in 2023, data centers consumed 4% of all electricity in the U.S., with over half derived from polluting coal energy.
Why focus on realistic problems when we can obsess over the remote possibility of AI destroying humanity?
