Segmentation of cloud organization patterns from satellite images using deep neural networks

Abstract

Climate change is one of the most important challenges that humanity faces now. The essential part of climate models is the movement of clouds, which affects climate parameters dramatically. Shallow clouds play a huge role in determining the Earth's climate. They're also difficult to understand and to represent in climate models. Unfortunately, the exact modeling of clouds movement is notoriously tricky and requires perfect knowledge of underling physical processes and initial states. Boundaries between different types of clouds are usually blurry and difficult to define with rule-based decision systems. Simplification of the segmentation step is crucial and can help researchers to develop better climate models. Convolutional neural networks have been successfully applied in many similar areas, and for cloud segmentation itself, too. However, there is a high cost of good, pixel-level labeled datasets, so the industry often uses coarse-labeled datasets with the either region or image-level labels. In this paper, we propose an end-to-end deep-learning-based method for classification and segmentation of different types of clouds from a single colored satellite image. Here, we propose the multi-task learning approach to cloud segmentation. Additionally to the segmentation model, we introduce a separate classifier that uses features from the middle layer of the segmentation model. The presented method can use coarse, uneven and overlapping masks for clouds. From the experimental results, the proposed method demonstrates stable results and learns good general features from noisy data. As we observed during the experiments, our model finds types of clouds, which are not annotated on the images but seem to be correctly defined. It is ranked in top three percent competing methods on Understanding Clouds from Satellite Images Dataset.