TOWARDS SELF-ADAPTIVE LEARNING PLATFORM IN FUTURE CLASSROOM BASED ON EMOTION COMPUTING

The accurate recognition of the emotion of learners is the basis of harmonious emotional interaction in the intelligent learning environment of the future classroom. This project focuses on the lack of research and practical exploration on harmonious emotional interaction in the future classroom, and constructs the database of learning pictures and expression pictures of learners by employing Convolution Neural Network. Endowing the future classroom with a self-adaptive interaction property at the emotional level by emotion computing can promote to learn effectively. The proposed platform can self-adaptively adjust the visual characteristics of learning picture according to the expression of learners, visual emotion preference and learning picture emotion, and then present the adapted learning picture to learners in real time, so as to realize the adaptive interaction at the emotional level in the future classroom.


Introduction
Future classrooms show information of different dimensions of students analyzed by big data in education field. Students can understand their real abilities, learning habits, learning styles and so on through their own learning portraits. Teachers can inspect the learning portrait of each student in the class, so that they can quickly and accurately grasp the basic situation and learning style of each student. Ultimately, the goal of teaching according to their aptitude could be achieved. Also, managers can acquire learning portraits of learners. Through the presentation of these data, they can more intuitively understand the teaching level of each stage of the classroom. In this way, the future classroom provides data support for educational decision-making for managers. Parents can get learning portraits of their children, so that parents can further understand their children's learning status and provide targeted guidance and intervention to their children.
As the high-end form of digital learning environment, the intelligent learning environment should have enough wisdom to meet the intelligent learning needs of learners. In addition to provide learners with personalized learning content, learning path and self-adaptive interaction at the cognitive level, it should also provide learners with learning portraits in line with their emotional state and visual emotional preferences.
According to the facial expression of learners by response to the learning picture in the intelligent learning environment, the learning platform judges the learners' emotional state. By combining the learning picture emotion with the visual emotional preference of learners about the learning picture, the platform can adjust the visual characteristics of the learning picture, so as to adjust the learning picture emotion. By this way, the platform adapts emotional state of learners and stimulates their learning interest. Finally, the harmonious emotional interaction in the intelligent learning environment is realized.
Self-adaptive learning system can provide learners with personalized learning services, adopt corresponding teaching strategies, and provide learners with adaptive learning content and navigation support functions to meet personalized needs of learners [1]. Its advantage is that it can make up for the defects of network teaching systems such as Moodle [2], Blackboard [3], Sakai [4], WebCT [5] and so on. Brusilovsky and others at the University of Pittsburgh have successively developed adaptive learning systems such as InterBook, Knowledge Sea II, AnnotatEd and ELM-ART [6][7][8][9]. iWeaver, an self-adaptive learning platform has been designed and developed by Wolf of RMIT Australia, provides learners with a learning environment for learning Java programming language [10]. Grockit system [11] is an online social game platform developed by Nivi in 2006. The platform integrates the new concept of social learning and can continuously adjust the learning plan according to the specific progress of students. Knewton system [12][13] is an "adaptive learning" tool founded by Ferreira in 2008. The system can continuously understand students' learning characteristics, provide students with personalized learning resources, and respond to each user's performance and activities in real time on Knewton system. Fluencia [14], a learning tool of spoken Spanish, developed by the founder of spanishdict.com, is a personalized, conversational and intelligent feedback system based on the latest cognitive research.
Although there exists a large body of work in the area of self-adaptive learning environment, surprisingly little has been done with interactive emotion analysis. The existing research on intelligent learning environment seldom considers the role of non-intelligent factors such as emotion, interest, motivation and will in learning activities, and ignores the theoretical and practical research on harmonious emotional interaction in intelligent learning environment, resulting in its lack of emotional adaptability and personalization. Emotional support in the process of intelligent learning is missing.

The proposed method
This project is based on recently emerging information technologies such as artificial intelligence, emotion computing and big data, to name a few. According to learners' facial expression response to learning picture in future classroom, the platform judges the emotional status of learners by combining the emotions of learning picture with the visual emotion preference of learners for the learning picture. Accordingly, the platform could adapt the key visual characteristics of the learning picture, and continuously improve the learner's visual emotional preference model of learning picture to achieve self-adaptive interactions in the intelligence learning environment at the emotion level. The self-adaptive model of learning picture based on learners' facial expression in intelligent learning environment is shown in Fig. 1. The model augments the self-adaptive function of learning picture in the intelligent learning environment dwelling in the future classroom to make learning process more intelligent and friendly. Through the self-adaptation of learning picture, the platform can adapt the learners' learning state and learning emotion, and promote the emergence of learning in-depth. The model is mainly divided into five modules, that is, emotion recognition of learning picture, visual feature extraction, facial expression recognition of learners, self-adaptation of learning picture emotion and the improvement of visual emotion preference of learners as follows.  (1) The module of emotion recognition of learning picture When learners start learning, the module of the emotion recognition of learning picture will receive the learning picture of the screen synchronously transmitted by the teacher through the screen broadcast in real time, or collect the learning picture of the screen independently selected by learners. Then, the trained 9-layer CNN(Convolutional Neural Network shown in Fig. 2) model is used to identify 12 emotions and the corresponding intensity of the learning picture. The recognized emotion data of learning picture will be transmitted to the module of self-adaptation of learning picture.
(2) The module of visual feature extraction of learning picture The emotion of learning picture is related to two factors: one is the intuitive visual features (color, texture, shape, etc.), the other is the implied artistic features (color matching, text format, typesetting layout, etc.), which are largely determined by the artistic features. Visual feature extraction of learning picture mainly extracts intuitive visual features and implicit artistic features. The essence of self-adaptation of learning picture is to adapt the key visual features of learning picture according to the facial expression of learners, the visual emotion preference and the emotion of learning picture, so as to change the visual emotion of learning picture.
(3) The module of facial expression recognition of learners The module consists of two parts: the acquisition of facial expression image and the facial expression recognition of learners. The expression recognition result will directly determine whether and how to adapt the learning picture. It is also a test of the self-adaptation effect of the learning picture. At the time when learners start learning, the front camera of the intelligent learning terminal will be opened synchronously to collect learners' facial expression images in real time. Then, the AdaBoost [15] method based on Haar rectangular [16] feature is used to extract the face region in the image, and the trained 7-layer CNN model is used to identify 7 learning emotions of learners and corresponding intensity. The recognized expression data will be transmitted to the module of self-adaptation of learning picture.
(4) The module of self-adaptation of learning picture According to the data of the module of emotion recognition of learning picture, the module of visual feature extraction of learning picture, the module of facial expression recognition of learners and combined with the visual emotion preference of learners, the platform will automatically adapt the visual features in the learning picture that are closely related to the emotion of learning picture and the emotion of learners, such as the brightness intensity, main body and background color of the learning picture. This enhances the display of key learning contents and background texture. To this end, the platform provides learners with self-adaptive and personalized learning pictures, detects and induces the learning emotion of learners in real time. Finally, learning interest is stimulated and learning effect of learners is improved. The model emphasizes the importance of positive emotional state of learners to cultivate high-order thinking abilities such as innovation ability, problem-solving ability, decision-making ability and critical thinking ability, so as to meet the needs of smart learning of learners.

Results
Through the intelligent transformation of campus equipment, we research on equipment management and intelligent perception ability, optimization and adaptation of environmental parameters, data acquisition and intelligent identification. Also, we develop the ability to support, record and analyze the teaching process, provide teachers and students with teaching services in need, and automatically trace the teaching process. The passing rate of computer programming class is very low before using the future classroom. As a result, students complain about this, and teachers repeat teaching content. To tackle these limitations, the platform intelligently formulates learning plans and after-school review tasks according to the interest points of students. In March 2020, the campus adopted a new measure requiring the completion of homework and final exams on the self-adaptive interactive learning platform. At ordinary times, they study according to the learning plan provided by the platform, and do exercises for unlimited times until the score reaches the standard.
At the same time, teachers also evaluate the advantages and disadvantages of students through the platform. Results at the end of the summer of 2020 shows the passing rate of programming course increased greatly. The most obvious change is that the passing rate of programming courses at intermediate level is only 32.2% in the first half of 2021, and has risen to 81.2% by the autumn of 2021. The self-adaptive interaction platform is highly praised. In future, we will carry out more extensive self-adaptive learning plans for students to make the future classroom more intelligent and efficient by employing the emotion data collection.