Methodology of neural network compression for multi-sensor transducer network models based on edge computing principles

Abstract

This paper focuses on the development of a methodology to compress neural networks that is based on the mechanism of pruning the hidden layer neurons. The aforementioned neural networks are created in order to process the data generated by numerous sensors present in a transducer network that would be employed in a smart building. The proposed methodology implements a single approach for the compression of both Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) that are used for the tasks of classification and regression. The main principle behind this method is based on the dropout mechanism, which is employed as a regulation mechanism for the neural networks. The idea behind the method proposed consists of selecting optimal exclusion probability of a hidden layer neuron, based on the redundancy of the said neuron. The novelty of this method is the usage of a custom compression network that is based on an RNN, which allows us to determine the redundancy parameter not just in a single hidden layer, but across several layers. The additional novelty aspect consists of an iterative optimization of the networkoptimizer, to have continuous improvement of the redundancy parameter calculator of the input network. For the experimental evaluation of the proposed methodology, the task of image recognition with a low-resolution camera was chosen, the CIFAR10 dataset was used to emulate the scenario. The VGGNet Convolutional Neural Network, that contains convolutional and fully connected layers, was used as the network under test for the purposes of this experiment. The following two methods were taken as the analogous state of the art, the MagBase method, which is based on the sparcification principle as well as the method which is based on rarefied representation by employing the approach of rarefied encoding SFAC. The results of the experiment demonstrated that the amount of parameters in the compressed model is only 2.56 % of the original input model. This has allowed us to reduce the logical output time by 93.7 % and energy consumption by 94.8 %. The proposed method allows to effectively using deep neural networks in transducer networks that utilize the architecture of edge computing. This in turn allows the system to process the data in real time, reduce the energy consumption and logical output time as well as lower the memory and storage requirements of real-world applications.