Thesis of Benjamin Allaert

Analysis of facial expressions in video flows

Facial expression recognition has attracted great interest over the past decade in wide application areas, such as human behavior analysis, video communication, e-health and marketing. In this thesis we explore a new approach to step forward towards in-the-wild expression recognition, dealing with challenges such as various intensity and various expression activation patterns, illumination variation and head pose variations. Special attention has been paid to encode respectively small/large facial expression amplitudes (micro and macro expressions), and to analyze facial expressions in presence of varying head pose. The first challenge addressed concerns varying facial expression amplitudes. We propose an innovative motion descriptor called local motion patterns (LMP). This descriptor takes into account mechanical facial skin deformation properties (local coherency and local propagation). When extracting motion information from the face, the unified approach deals with inconsistencies and noise, caused by face characteristics (skin smoothness, skin reflect and elasticity). The main originality of our approach is a unified approach for both small and large facial expression recognition, with the same facial recognition framework (descriptor, facial framework, parameters). The second challenge addressed concerns important head pose variations. In facial expression analysis, the face registration step must ensure that minimal deformation appears. Registration techniques must be used with care in presence of unconstrained head pose as facial texture transformations apply. Hence, it is valuable to estimate the impact of alignment-related induced noise on the global recognition performance. For this, we propose a new database, called Simultaneous Natural and Posed 2D facial expression database (SNaP-2DFe), allowing to study the impact of head pose and intra-facial occlusions on expression recognition approaches. We prove that the usage of face registration approach does not seem adequate for preserving the features encoding facial expression deformations

Jury

- Directeur de thèse : Chaabane Djeraba - Rapporteurs : Jenny Benois-Pineau, Monique Noirhomme - Examinateurs : Moncef Gabbouj, Laurence Duchien, Marius Ioan Bilasco

Thesis of the team FOX defended on 08/06/2018