Regression prediction | MATLAB realizes Bayes Gru (Bayes optimized gating cycle unit) multiple input and single output

Regression prediction | MATLAB Realization Bayes-GRU( Bayesian Optimization gated loop unit ) Multi input single output

Basic introduction

The running test environment MATLAB2020b;
The basic task of this prediction is regression , Multivariable input , Univariate output ;

• The main research problems are not limited to traffic prediction 、 Load forecasting 、 Weather forecast 、 Economic forecast, etc .Bayes-GRU Multivariable input and univariate output , Bayesian Optimization gated loop unit .
• Aiming at the problems of too many parameters and easy to fall into local optimization in nonlinear prediction model , A gate control loop unit model in deep learning is proposed , Combined with Bayesian optimization algorithm, the super parameters of door control loop unit are optimized , The model is applied to the deformation prediction of concrete dam . To test the feasibility of the model , Based on measured data , It turns out that : The model has strong generalization ability 、 High operational efficiency , Can effectively predict .

Background review

• The first mock exam has accuracy. 、 Disadvantages of poor robustness and extension , Although the optimal combination of different models has greatly improved the fitting and prediction accuracy , However, there are too many parameters due to machine learning 、 Slow convergence 、 It is easy to fall into local optimality and other problems , Cannot handle height effectively
  Nonlinear problems .
• In recent years , Due to the rapid development of artificial intelligence , Deep learning in image classification 、 speech recognition 、 Medicine and air pollution prediction have been widely applied . Deep learning is a neural network composed of more nonlinear mapping hidden layers , It can better mine the nonlinearity between data .
• In this paper, Bayesian algorithm and gate control loop unit with global optimization ability are introduced into prediction analysis , It can solve the problems of slow convergence and over fitting of the monitoring model , The accuracy and stability of the model can be improved , Finally, it makes the prediction more accurate .

Model is introduced

• In order to improve the prediction accuracy of the model , It is necessary to determine the optimal combination of hyperparameters . And Bayesian optimization ( Bayesian optimization,BO) Is a global optimization algorithm , Be able to effectively deal with GＲU The parameters in the model are optimized , So as to realize the prediction of monitoring data with nonlinearity .
• Select potential evaluation points through collection function xt, Then evaluate the objective function value yt, The probabilistic agent model is constantly updated to find the optimal solution of parameters . Bayesian optimization algorithm can get the approximate solution after a few times of evaluating the objective function , Suitable for solving multi peak problems 、 Unknown complex optimization problem .
  
• Probabilistic agent model and acquisition function are the core of Bayesian optimization . The commonly used probabilistic agent models include Gaussian process 、 Random forests 、 Tree structure Parzen Estimation method, etc . Compared with other probabilistic agent models , Tree structure Parzen The estimation method performs well in high-dimensional space , The convergence speed has been significantly improved . Common collection function generation strategies include promotion based strategies 、 Confidence boundary strategy 、 Information based strategies, etc .
• This paper is based on Bayesian algorithm GRU Number of network layers of the model , Number of units , Learning rate , The regularization rate is 4 A super parameter is optimized . The probability agent model adopts tree structure Parzen Estimation method , The collection function is based on the promotion strategy . The independent variables x Combination of different superparameters , The optimization objective function is mean square error .
  

Programming

%%  Variable initialization  
opt.learningMethod      = 'GRU';
%%  Parameters are defined 
% Maximum number of iterations 
opt.maxEpochs     = 10; 
% Batch size 
opt.miniBatchSize = 16;
% execution environment ： 'cpu' 'gpu' 'auto'
opt.executionEnvironment = 'cpu'; 
% optimization algorithm ： 'sgdm' 'rmsprop' 'adam'
opt.LR                   = 'adam';  
% Training process ： 'training-progress' 'none'
opt.trainingProgress     = 'none';  

%%  Parameter setting                 
opt.isUseDropoutLayer = true;
% Weight loss parameter setting , Prevent over fitting 
opt.DropoutValue      = 0.5;

%  Optimize parameter setting 
opt.optimVars = [
    optimizableVariable('NumOfLayer',[1 4],'Type','integer')
    optimizableVariable('NumOfUnits',[50 200],'Type','integer')
    optimizableVariable('InitialLearnRate',[1e-2 1],'Transform','log')
    optimizableVariable('L2Regularization',[1e-10 1e-2],'Transform','log')];

opt.isUseOptimizer           = true;
opt.MaxOptimizationTime      = 14*60*60;
opt.MaxItrationNumber        = 10;
opt.isDispOptimizationLog    = true;
% Save parameter settings 
opt.isSaveOptimizedValue     = false;       
opt.isSaveBestOptimizedValue = true;       

• Prediction effect
  
  
  
  

Learning summary

• In this paper, Bayesian optimization algorithm and deep learning are combined and introduced into prediction , Built on Bayes-GRU Model , It effectively improves the problem that the model is easy to fall into local optimization , Thus, the accuracy and running speed of the model are improved .
• The model can deal with nonlinear data well , But there are still gradients that disappear 、 The disadvantage of poor memory , How to solve these shortcomings , More in-depth research is needed .

Reference material

[1] https://mianbaoduo.com/o/works/243364
[2] Wu Zhongru , Chen Bo ． Review on the development of dam deformation monitoring model ．
[3] Huang Mengjing , Yang Haibo , Leaf root seedling ． Dam safety monitoring model based on real-time tracking .
[3] Wu Zhongru ． Safety monitoring theory of hydraulic structures and its application .

thank

• Thank you for your subscription and support ！