New Research Details More Accurate Approach to Facial Emotion Recognition

A new research paper details an approach to emotion recognition that seems to yield strong results.

Titled “Image-based facial emotion recognition using convolutional neural network on Emognition dataset”, the paper describes an automated system for “facial emotion recognition” (FER) using deep learning models. The researchers aimed to overcome the limitations of existing FER systems, which typically recognize a limited set of basic emotions, by using the Emognition dataset. This dataset includes ten emotions: amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, sadness, and neutral.

The researchers preprocessed the dataset by converting video recordings into image frames, cropping the facial regions, and cleaning the data to retain only images accurately representing the intended emotions. The data was then shuffled, split into training, validation, and test sets, and enhanced using rescaling, resizing, and data augmentation techniques.

Two main approaches are employed to develop the CNN (“Convolutional Neural Networks”) models: transfer learning with fine-tuned pre-trained models (Inception-V3 and MobileNet-V2) and building a CNN model from scratch using the Taguchi method, a statistical approach, to find the best settings (hyperparameters) for the model to improve its performance.

The study’s results demonstrated that the transfer learning model using Inception-V3 achieved the highest performance, with an accuracy rate of 96 percent and an average F1-score of 0.95 on the test data. An F1-score is a measure of a model’s accuracy that considers both precision (how many selected items are relevant) and recall (how many relevant items are selected), providing a single metric that balances these two aspects.

The MobileNet-V2 model also performed well, but with slightly less consistency, achieving an accuracy rate of 89 percent. The model built from scratch, while effective, showed lower performance with an accuracy rate of 87 percent.

The results highlight the superior performance of the transfer learning approach with fine-tuned pre-trained models, particularly Inception-V3, in accurately recognizing a broader range of emotions in facial expressions.

Biometric emotion recognition has become a somewhat controversial technology in recent years. A recent report from Wired detailed the use of this tech, along with other computer vision capabilities, on train travelers in the UK, prompting an outcry from the activist group Big Brother Watch. Concerns have previously been raised about the lack of accuracy in such systems—a problem that could, at least, be addressed, according to this latest research.

Source: Nature

–

June 24, 2024 – by Cass Kennedy and Alex Perala