The press service of the Ministry of Education and Science of the Russian Federation reported that scientists of the St. Petersburg Federal Research Center of the Russian Academy of Sciences developed a MASAI system that could recognize human emotions by audio and video data. The development was supported by the grant of the Russian Scientific Fund.
The system uses a complex of neural networks to analyze heterogeneous information: speech, facial expressions, gestures and text. The accuracy of recognition reaches 80%, which exceeds the indicators of existing analogues. The program identifies seven basic emotions: joy, sadness, fear, disgust, surprise, anger and calm.
A feature of development is the ability to work in difficult conditions – with a background noise, poor lighting, or when a person does not look into the camera. The system also recognizes irony and sarcasm, when the meaning of the said does not coincide with the intonation or expression of the face.
Neural networks were studied on multilingual databases containing records of people of different ages, gender and nationality. To create a Russian -language base segment, young actors of theater universities were involved.
According to the head of the laboratory of speech and multi -modal interfaces Alexei Karpov, the system can be integrated into various digital assistants.
