Analysis: emotion recognition technologies currently show low levels of accuracy so there’s a way to go before they can read your emotions
A widespread application of machine learning algorithms is to track people’s identity. Just as identity can be recognised through the recording of faces, it is claimed that emotions can be recognised too, though emotion recognition algorithms are not used for global surveillance yet.
Since the development of the first commercial algorithm Facereader in 2005, they have been mainly focused on the validation of marketing content such as TV ads, product packaging or even food taste. With the multiplication of devices able to record peoples’ faces, and with the continuous improvements in the processing power of computers, emotion recognition technologies have found new real-life applications in the automotive industry and, perhaps surprisingly, in job recruitment technologies.
But the performance and accuracy of emotion recognition technologies based on facial expression have come under close scrutiny by AI watchdog, the AI Now Institute. In January 2021, HireVue abandoned emotion recognition technologies as part of their AI recruitment assessment product. The reason for this decision may be related to the growth in research which is invalidating the assumptions about the link between facial expressions and emotions.
Since the 1970s, the most common paradigm in psychology about emotions stated three principles: (i) there is a direct relationship between an emotion felt and its facial expression, (ii) there are six basic emotions and each has a prototypical expression, and (iii) emotions and their expressions are universal. These assumptions were also conveyed in popular culture like in the Disney Pixar movie Inside Out (2015) or in the TV show Lie to Me (2009-2011) and were commonly accepted by companies and organisations.
They were also ideal for the use of machine learning algorithms. Following this model, algorithms claim to identify the emotion felt by someone by categorising facial expressions as one of six basics emotions, and applying the algorithm to everyone without distinction of culture or origin.
In this study, eight commercially available algorithms were compared with human observers in the recognition of 937 videos supposedly expressing an emotion. There was considerable variance in recognition accuracy among the eight algorithms, and their classification accuracy was consistently lower than the observers’ accuracy.
My emotion appears as “Happy” because I am smiling, but this smile was not genuine
The accuracy of algorithms is not the main problem here; the fundamental principle of the relationship between emotions and facial expressions is also called into question. Along with colleagues at the Université Grenoble-Alpes, we have recorded 232 participants watching videos and collected their reported feelings while watching the video. The recordings of their facial expressions were shown to human observers and were processed by a commercial algorithm designed by one of the leaders in the domain of emotion recognition technologies.
The results revealed that the accuracy of the algorithm in identifying the emotion felt from facial expressions was very low, as was the accuracy of human observers. In fact, it appears that there is no scientific evidence showing a reliable correlation between the emotion felt and facial expressions. This tends to indicate that all assumptions upon which emotion recognition technologies were built, are now invalid.
Despite these criticisms, there are some emerging solutions to the frailties in emotion recognition technologies. With the multiplication of devices available, recognition technologies will use not only the face, but also the entire body, and the context in which the expression is produced, to accurately infer emotions.
Author: Dr Damien Dupré, Assistant Professor in Business Research Methods, DCU Business School
This article was first published by RTE Brainstorm on Friday, 19th March 2021
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.