Facial emotion recognition could have broad applications across healthcare, education, marketing, transportation, and entertainment. It might be used to help monitor patients remotely or in over-stretched hospitals or emergency response settings, or patients unable to communicate well for any number of reasons. It could be used to personalize learning, allowing a computerised training system to respond more appropriately to the user. Similarly, such a system could improve customer service and might even be used to create immersive entertainment experiences.
Computer systems that can identify emotions from our facial expressions are in development, but still face man challenges. The earliest systems relied on a single method, such as mapping a person’s face and matching it to a database of annotated expressions. Some approaches based on this simplified method are more accurate than others, but none yet captures all the nuance of human emotion as it is expressed in our faces.
Research in the International Journal of Biometrics introduces a new approach based on machine learning that could address this problem and make an emotion detector viable for a wide range of applications. The biggest issue that is addressed by the new work is that it can extract a complex emotion from real-world situations where environmental factors, incomplete data, or complex emotions might affect the accuracy of the results. However, the new approach brings together facial expression recognition and uses the person’s speech and tone of voice or even what they might be writing to give a more accurate result.
In their experiments, researchers Jian Xie and Dan Chu of Fuyang Normal University in Anhui, China, achieved a recognition accuracy of 98.6% with their approach. The system was particularly adept at identifying happiness or a neutral emotional state when compared with earlier systems. The system could not cope quite as well with the identification of disgust and surprise, however.
Xie, J. and Chu, D. (2025) ‘Character emotion recognition algorithm in small sample video based on multimodal feature fusion’, Int. J. Biometrics, Vol. 17, Nos. 1/2, pp.1–14.