This thesis addresses Dynamic Facial Expression Recognition (DFER), the task of classifying the emotion expressed in a face video. Rather than processing the full image, we adopt a geometric and frugal representation in which the face is reduced to a set of facial landmarks connected by edges to form a spatio-temporal graph. This representation is far lighter than pixels and largely insensitive to appearance variations (lighting, texture, background), which shifts the difficulty toward the quality of landmark extraction and the design of the graph itself, two questions that represent the heart of this work. We first propose a modular preprocessing pipeline that turns a video sequence into a facial graph. Each of its stages (landmark extraction and selection, temporal sampling and extension, coordinate normalization, edge construction) is isolated and evaluated through an ablation study on six datasets ranging from controlled to in- the-wild conditions. We find that no single configuration is optimal everywhere, the best choices depending strongly on the data specificities. We then develop a family of graph architectures that enrich each landmark with four modalities (local appearance, normalized geometry, temporal position, motion) before any message passing, and we compare three topology paradigms, namely a graph dynamically rebuilt from nearest neighbors, a globally learned adjacency, and a differentiable, dynamic topology adapted from the Iterative Deep Graph Learning (IDGL) framework. An empirical convergence analysis shows that this joint learning of the topology yields no systematic gain over simpler constructions for our task, a negative result that we document and interpret. Finally, we study a dual-graph model with two branches, one carrying the geometric landmarks and the other virtual muscles derived from the deformations between landmarks, on which we conduct an interpretability study based on five attribution methods. This study highlights the complementarity of the two branches, the inde pendent convergence of the methods toward the expected expressive regions of the face (mouth, eyebrows, eyes), and a partial correspondence with the Facial Action Coding System (FACS). Our models are evaluated mainly on CK+, Oulu-CASIA and DFEW, complemented by AFEW, MAFW and FERV39K. Under controlled conditions the geometric approach reaches honest and anatomically readable performance, whereas 3Abstract. in the wild it remains behind large whole-image models, a gap that is essentially due to the input modality. The main contributions of this work are: • A modular preprocessing pipeline turning a face sequence into a spatio-temporal facial graph, together with a systematic ablation study of its components across six datasets spanning a wide range of difficulty. • A family of graph architectures with multi-modal representations at the landmark level, and an exploration of three graph-topology learning paradigms along with an analysis of their convergence. • A dual-graph model combining geometric landmarks and virtual muscles, anatomi cally initialized and extended to the in-the-wild regime. • An original interpretability study based on five attribution methods, linking the model’s decisions to facial anatomy and to the FACS.
- M. Chaabane DJERABA Professeur des universités Université de Lille Directeur de thèse, - Mme Jenny BENOIS-PINEAU Professeure des universités Université de Bordeaux Rapporteure, - M. Salvatore-Antoine TABBONE Professeur des universités Université de Lorraine Rapporteur, - M. Akrem SELLAMI Maître de conférences Université de Lille Co-encadrant de thèse, - Mme Clarisse DHAENENS Professeure des universités Université de Lille Examinatrice, - M. Frederic JURIE Professeur des universités Université de Caen Normandie Examinateur.
Thesis of the team FOX defended on 12/11/2026
français
English