Graphical User Interface auto-completion with element constraints

Brückner, Lukas (2020). Graphical User Interface auto-completion with element constraints. Thesis (Master thesis), E.T.S. de Ingenieros Informáticos (UPM).


Title: Graphical User Interface auto-completion with element constraints
  • Brückner, Lukas
Item Type: Thesis (Master thesis)
Masters title: Human Computer Interaction and Design
Date: 25 September 2020
Freetext Keywords: User interfaces; Machine learning; Layouts; Neural networks; Design; Human-computer interaction
Faculty: E.T.S. de Ingenieros Informáticos (UPM)
Department: Lenguajes y Sistemas Informáticos e Ingeniería del Software
Creative Commons Licenses: Recognition - No derivative works - Non commercial

Full text

[thumbnail of TFM_LUKAS_BRÜCKNER.pdf]
PDF - Requires a PDF viewer, such as GSview, Xpdf or Adobe Acrobat Reader
Download (11MB) | Preview


Graphical User Interfaces (GUIs) are often the main touchpoint between a digital product and its users. Consistency is a key usability factor of GUIs that dictates reusing layout patterns throughout the designs of a single system. While visual styles can be easily specified across designs, spatial patterns of related elements often emerge in the creation process. Although these patterns are encoded in existing layouts, a large number is dicult to consider manually. This thesis investigates methods for a design assistance tool that suggests completions for a partial layout. Instead of generating arbitrary completions, we allow the designer to control the next element type and dimensions. The goal of the methods is then to predict the placement of this new element according to the layout patterns of reference designs. Two recently proposed methods for layout completion were evaluated, a Graph Neural Network (GNN) and a Transformer model, as well as a novel approach that leverages a sequence alignment algorithm and a nearest neighbor search (kNN). The methods were tested on handcrafted data sets with explicit layout patterns, as well as larger sets of diverse mobile layouts that lack consistent patterns. The implementation of the GNN mostly fails to predict high-quality results. The transformer model captures general layout structures and works reasonably well for spatial compositions that are close to the training data. The kNN approach achieves the best scores overall. The results suggest that leveraging data sets explicitly via instance-based learning algorithms can outperform neural network approaches for layout design problems. As such, this thesis contributes to establishing smarter design tools for professional designers that increase consistency in the design process.

More information

Item ID: 65263
DC Identifier:
OAI Identifier:
Deposited by: Biblioteca Facultad de Informatica
Deposited on: 05 Nov 2020 13:38
Last Modified: 05 Nov 2020 13:38
  • Logo InvestigaM (UPM)
  • Logo GEOUP4
  • Logo Open Access
  • Open Access
  • Logo Sherpa/Romeo
    Check whether the anglo-saxon journal in which you have published an article allows you to also publish it under open access.
  • Logo Dulcinea
    Check whether the spanish journal in which you have published an article allows you to also publish it under open access.
  • Logo de Recolecta
  • Logo del Observatorio I+D+i UPM
  • Logo de OpenCourseWare UPM