Deep active learning for instance segmentation

Abstract

One of the main constraints of machine learning is the common lack of annotated data. This constraint becomes even more significant when it comes to deep learning, as its performance is usually proportional to the amount of annotated data available. Active Learning emerged as a way of overcoming this drawback. Its goal is to let the model actively select which samples does it wants to learn from in order to achieve a competent performance with as few annotated data as possible. It has proven to significantly reduce the number of annotated data needed in some fields such as text or image classification. However, its potential has not been explored in a complex task such as Instance Segmentation. The goal of this Master thesis was the application of Active Learning to an Instance Segmentation task through the design and development of different Active Learning frameworks. The experiments were done using two different datasets: Common Objects into Context and the one proposed by the Data Science Bowl 2018. The sampling strategies implemented were divided into classification-based, mask-based, mask+class based and semi-supervised. The classificationbased framework measures the uncertainty of an image using the probability the model gives to each instance when associating a class to them. The mask-based framework measures the uncertainty as the spatial uncertainty a model has with the predicted masks. In the third framework, mask+class-based, the previous uncertainties were combined into one sampling strategy. Finally, the semi-supervised framework includes and additional approach that labels automatically the most certain samples using the masks and classes predicted by the model. This thesis can be considered as a starting point in the study of the application of Active Learning techniques to Instance Segmentation tasks. However, the results obtained were really promising. The different methods managed to achieve a reduction in the number of necessary segmented images. The inclusion of a semi-supervised framework to the Active Learning cycle could reduce the annotation effort even more. Another insight obtained was the importance of the balance of the different classes in the batch and the representativeness of this images with respect to the underlying sampling space.