Accelerating the learning process of a neural network by predicting the weight coefficient

Abstract

The purpose of this study is to analyze and implement the acceleration of the neural network learning process by predicting the weight coefficients. The relevance of accelerating the learning of neural networks is touched upon, as well as the possibility of using prediction models in a wide range of tasks where it is necessary to build fast classifiers. When data is received from the array of sensors of a chemical unit in real time, it is necessary to be able to predict changes and change the operating parameters. After assessment, this should be done as quickly as possible in order to promptly change the current structure and state of the resulting substances.. Work on speeding up classifiers usually focuses on speeding up the applied classifier. The calculation of the predicted values of the weight coefficients is carried out using the calculation of the value using the known prediction models. The possibility of the combined use of prediction models and optimization models was tested to accelerate the learning process of a neural network. The scientific novelty of the study lies in the effectiveness analysis of prediction models use in training neural networks. For the experimental evaluation of the effectiveness of prediction models use, the classification problem was chosen. To solve the experimental problem, the type of neural network “multilayer perceptron” was chosen. The experiment is divided into several stages: initial training of the neural network without a model, and then using prediction models; initial training of a neural network without an optimization method, and then using optimization methods; initial training of the neural network using combinations of prediction models and optimization methods; measuring the relative error of using prediction models, optimization methods and combined use. Models such as “Seasonal Linear Regression”, “Simple Moving Average”, and “Jump” were used in the experiment. The “Jump” model was proposed and developed based on the results of observing the dependence of changes in the values of the weighting coefficient on the epoch. Methods such as “Adagrad”, “Adadelta”, “Adam” were chosen for training neural and subsequent verification of the combined use of prediction models with optimization methods. As a result of the study, the effectiveness of the use of prediction models in predicting the weight coefficients of a neural network has been revealed. The idea is proposed and models are used that can significantly reduce the training time of a neural network. The idea of using prediction models is that the model of the change in the weight coefficient from the epoch is a time series, which in turn tends to a certain value. As a result of the study, it was found that it is possible to combine prediction models and optimization models. Also, prediction models do not interfere with optimization models, since they do not affect the formula of the training itself, as a result of which it is possible to achieve rapid training of the neural network. In the practical part of the work, two known prediction models and the proposed developed model were used. As a result of the experiment, operating conditions were determined using prediction models.