Facial Expression Analysis Using Multi-label Classification
DOI:
https://doi.org/10.52756/ijerr.2024.v41spl.018Keywords:
Compound emotions, emotion recognition, facial expressions, multilabel classificationAbstract
Emotions form the reflection of an individual's mental state. Automatic expression analysis and its recognition based on their facial landmarks are of critical use, particularly to understand the emotional states of physically disabled individuals like autistic children, deaf, dumb, and bedridden individuals. This approach helps in interpreting human emotions, which is crucial for designing computational models and interfaces that cater to their needs. While traditional research has focused on six basic emotion categories (happiness, surprise, sadness, anger, fear, disgust), your work expands this by exploring compound emotion categories. Compound emotions combine these basic categories to create nuanced emotional states. The research utilizes the CFEE_Database_230 dataset of facial expressions captured for analysis and training purposes. The proposed methodology has the following three steps: Analyze the dataset and extract the region of interest (ROI), Extract various statistical and potentially discriminating features and apply a multi-label classification approach to categorize sets of emotions. This involves comparing the feature values across different emotion classes and assigning appropriate labels, particularly focusing on compound emotion categories. Also, the research employs different classifiers and metrics to evaluate the effectiveness of the model. After applying the classification methods, the results are analyzed using various metrics to assess the accuracy and effectiveness of emotion recognition based on facial landmarks. Based on the findings, the Binary Relevance Method yielded the best performance with Mean Average Precision and hamming Loss of 0.7227±0.0344 and 0.1906±0.0227, respectively. Overall, the work contributes to advancing automatic emotion recognition by considering a broader range of emotional categories beyond the traditional basics. This is particularly beneficial for populations such as the physically disabled and autistic children, where traditional communication methods might be limited or challenging.
References
Beumer, G. M., Tao, Q., Bazen, A. M., & Veldhuis, R. N. (2006). A landmark paper in face recognition. IEEE, In 7th international conference on automatic face and gesture recognition (FGR06), 2006, 6-78. https://doi.org/10.1109/FGR.2006.10
Banerjee, M., Goyal, R., Gupta, P., & Tripathi, A. (2023). Real-Time Face Recognition System with Enhanced Security Features using Deep Learning. Int. J. Exp. Res. Rev., 32, 131-144. https://doi.org/10.52756/ijerr.2023.v32.011
Black, M. J., & Yacoob, Y. (1995). Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. IEEE, In Proceedings of IEEE International Conference on Computer Vision, pp. 374-381.
Dalvi, C., Rathod, M., Patil, S., Gite, S., & Kotecha, K. (2021). A survey of ai-based facial emotion recognition: Features, ml & dl techniques, age-wise datasets and future directions. IEEE Access, 9, 165806-165840. https://doi.org/10.1109/ACCESS.2021.3131733
Ekman, P., & Friesen, W. V. (1978). Facial action coding systems. Consulting Psychologists Press. https://doi.org/10.1037/t27734-000
Jiang, J., & Deng, W. (2023). Improving Multi-Label Facial Expression Recognition with Consistent and Distinct Attentions. IEEE Transactions on Affective Computing, pp. 1-11. https://doi.org/10.1109/TAFFC.2023.3333874
Kollias, D. (2023). Multi-label compound expression recognition: C-expr database & network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5589-5598. https://doi.org/10.1109/CVPR52729.2023.00541
Mehrabian, A., & Ferris, S. R. (1967). Inference of attitudes from nonverbal communication in two channels. Journal of Consulting Psychology, 31(3), 248. https://doi.org/10.1037/h0024648
Mohanaiah, P., Sathyanarayana, P., & GuruKumar, L. (2013). Image texture feature extraction using GLCM approach. International Journal of Scientific and Research Publications, 3(5), 1-5.
Murugappan, M., & Mutawa, A. (2021). Facial geometric feature extraction based emotional expression classification using machine learning algorithms. Plos One, 16(2), e0247131. https://doi.org/10.1371/journal.pone.0247131
Saha, P., Bhattacharjee, D., De, B. K., & Nasipuri, M. (2015). An approach to detect the region of interest of expressive face images. Procedia Computer Science, 46, 1739-1746. https://doi.org/10.1016/j.procs.2015.02.123
Sharma, A., Yaduvanshi, E., Sharma, A., & Saha, P. (2024). Mitigating SAD States and Maladaptive Coping in Law Enforcement: Enhance Emotional Competence. International Journal of Experimental Research and Review, 40(Spl Volume), 132-141. https://doi.org/10.52756/ijerr.2024.v40spl.010
Sharma, R., Mohan, M., & Gupta, S. (2023). Emotions in retail setting: a systematic literature review based on current research. Int. J. Exp. Res. Rev., 30, 416-432. https://doi.org/10.52756/ijerr.2023.v30.039
Tian, Y. L., Kanade, T., & Cohn, J. F. (2000). Recognizing lower face action units for facial expression analysis. IEEE, In Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580), pp. 484-490.
Tsoumakas, G., Spyromitros-Xioufis, E., Vilcek, J., & Vlahavas, I. (2011). Mulan: A Java library for multi-label learning. The Journal of Machine Learning Research, 12, 2411-2414.
Ugail, H., & Al-Dahoud, A. (2018). Is gender encoded in the smile? A computational framework for the analysis of the smile driven dynamic face for gender recognition. The Visual Computer, 34(9), 1243-1254. https://doi.org/10.1007/s00371-018-1494-x
Valstar, M. (2015). Automatic facial expression analysis. Understanding Facial Expressions in Communication: Cross-Cultural and Multidisciplinary Perspectives, pp. 143-172. https://doi.org/10.1007/978-81-322-1934-7_8
Wang, K., Lian, Z., Sun, L., Liu, B., Tao, J., & Fan, Y. (2022). Emotional reaction analysis based on multi-label graph convolutional networks and dynamic facial expression recognition transformer. In Proceedings of the 3rd International on Multimodal Sentiment Analysis Workshop and Challenge, pp. 75-80. https://doi.org/10.1145/3551876.3554810
Zhang, L., Arandjelovic, O., & Hong, X. (2021). Facial action unit detection with local key facial sub-region based multi-label classification for micro-expression analysis. In Proceedings of the 1st Workshop on Facial Micro-Expression: Advanced Techniques for Facial Expressions Generation and Spotting, pp. 11-18. https://doi.org/10.1145/3476100.
Downloads
Published
How to Cite
Issue
Section
