Deep learning from the crowd. Fundamentals of morphological galaxy classification

Abstract

The objective of this work is to adapt a deep neural network model to perform galaxy morphological classification trained from crowd annotations, considering the training scheme, the agreement between the annotators, and the hierarchy. We used Galaxy Zoo 1 as our experimental test bed and trained a convolutional neural network (CNN) for the automatic classification of galaxies' morphologies. We analyze the impact of the following aspects on the classification accuracy and training efficiency: (i) training only the last layer versus training all the network; (ii) classification with only CNN versus considering the hierarchy; (iii) comparing the models trained with different amounts of data and levels of agreement between the annotators; (iv) training by stages, transferring knowledge from one model to another; and (v) combining several models as an ensemble. From the experiments, we derive the following results: (i) training all the layers in the network significantly improves the accuracy (10% increase in exact match), compared to training only the last layer; (ii) there is a trade-off between the amount of data and the level of agreement between the annotators used for training; (iii) using the hierarchy can improve the accuracy when the amount of training data is reduced; (iv) training by stages through transfer learning (curriculum learning) produces higher accuracy for limited data; (v) ensembles can improve accuracy; (vi) models achieve a low accuracy for the most difficult cases, but, if we consider hierarchical measures, we can derive useful results for upper levels in the hierarchy. An accuracy above 99% is achieved when training all layers of the network and considering a high agreement between the annotators. Training deep learning models from crowd annotations involves additional challenges compared to learning from hard annotations. We analyze the impact of different aspects when training a CNN from the crowd for morphological galaxy classification. The conclusions derived from this work can be useful for more complex galaxy classifications.