Modeling human behaviour to improve retail efficiency and health monitoring

Abstract

This dissertation presents a model to express the most relevant features of the human behaviour, specifying two personalized instances for retail and health Monitoring domains. These features are divided into static and dynamic. For the static ones facial features are the most relevant since they can express different attributes of the subjects such as age, gender, emotions ... For that purpose landmarks and pose are the most significant points of interest. However, the body pose can also reveal some interesting features of the current state of one person and should also be taken into account. Regarding the dynamic properties, the trajectories described by a subject in the monitored scenario can also express very useful insights. It is a basic task to delimit the area where the subject is located that is mandatory to perform the static inferences. The interactions of the user with the environment are a very relevant feature, and should be modeled as well. Thus, initial model has been evolved for the domains previously mentioned. Retail is a scenario where the shoppers express a large amount of features that should be considered by the retailers. Concepts such as density of the areas of the establishment, focus of attention, interactions with the shelves, queues management or characterization of the visitors are included in the model to improve the efficiency of the physical shops. A completely different field such as health monitoring could lead to a completely different approach. But the trajectories of the patients, their facial properties, their interactions or their body pose are also meaningful in this domain to assist decision-making of the health professionals. In our work we have mainly studied the behaviour of the patients with cognitive diseases. This field requires monitoring in different scenarios such as the home of the patient, the clinic centre or the rehabilitation centre. A scalabe system and its deployment is proposed to provide an accurate monitoring. In this area we shall also include medical features such as the treatment of the patients, their body pose evolution in physiotherapy sessions or measurements such as their body temperature or their heart rate. Stated the architecture of every domain, we have gone beyond in two Computer Vision and Machine Learning tasks: human pedestrian tracking and facial analysis. For the first we have developed two solutions that have been tested on real scenarios and on controlled environments that are suitable for a deeper analysis. Feature descriptors such as HOG or LBP have been tested, and Machine Learning classifiers such as Adaboost or Support Vector Machines have been optimized. We propose the best combination for previously exposed techniques and we show results on the different scenarios proposed to validate our proposal. A 3D visual imaging system is optimized by the proposal of a multi-camera RGB-D system able to capture facial properties at extreme poses. The data collection gathered allow 3D reconstruction of facial areas of 92 subjects captured, and a complete 3D reconstruction pipeline is proposed. Deep learning methods require a large amount of annotated data such as the output of our proposed system. We rely on that insight to perform an innovative data augmentation method for facial landmark detection that has been tested and discussed on two state of the art deep networks for this task. Head pose estimation is another relevant task for this dissertation and relying on 3D reconstruction and facial landmark detection we have implemented a method to validate the alignment of the data captured. Finally a knowledge base model is provided to store structured data, conceptually based on hierarchical and semantic properties of the domains studied. The data models are considered as the output for the proposed fields, and they are built to improve the performance of professionals in both areas.