Semi- and self-supervised face landmark estimation

Castillo Moreno, Miguel Ángel (2021). Semi- and self-supervised face landmark estimation. Thesis (Master thesis), E.T.S. de Ingenieros Informáticos (UPM).


Title: Semi- and self-supervised face landmark estimation
  • Castillo Moreno, Miguel Ángel
  • Baumela Molina, Luis
  • Prados Torreblanca, Andrés
Item Type: Thesis (Master thesis)
Masters title: Inteligencia Artificial
Date: June 2021
Faculty: E.T.S. de Ingenieros Informáticos (UPM)
Department: Inteligencia Artificial
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (9MB) | Preview


Facial landmark recognition is a relevant supervised problem in computer vision, used as the foundation to solve a multitude of tasks such as pose recognition, video surveillance or biometrics. Most of the current solutions struggle with faces in profile positions, occlusions or certain types of lighting. This project aims to test multiple ways to improve an existing facial landmark prediction model. In the first approach we introduce a self-supervised multi-tasking training scheme. Two versions of the same image with different rotation and translations are used. The prediction of the landmarks should be on the same spot of the face regardless of the transformation, so the predictions for each images are checked against each other. This should improve the consistency of results, forcing the network to learn to be indifferent to these transformations, and can used without labelled data as it works with the predictions. The results of these experiments show how this proposed loss can improve the predictions of the networks. The second one will be to use a GAN to change the attributes of an existing image. In theory, some attribute changes wouldn’t have to change the shape of the face. Therefore, augmentations in this way can be performed to labelled images while keeping the position of the landmarks. We develop a pipeline that allows the usage of StyleFlow for data augmentation, which can be adapted to work with other StyleGAN-based works. However, we demonstrate how it’s impractical to use current state-of-the-art GANs for this purpose, as they are too slow and produce unsatisfactory results.

More information

Item ID: 67932
DC Identifier:
OAI Identifier:
Deposited by: Biblioteca Facultad de Informatica
Deposited on: 23 Jul 2021 07:50
Last Modified: 23 Jul 2021 07:50
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM