An Efficient Elman Neural Networks Based on Improved Conjugate Gradient Method with Generalized Armijo Search
Published on Aug 15, 2018 in ICIC (International Conference on Intelligent Computing)
· DOI :10.1007/978-3-319-95930-6_1
Elman neural network is a typical class of recurrent network model. Gradient descent method is the popular strategy to train Elman neural networks. However, the gradient descent method is inefficient owing to its linear convergence property. Based on the Generalized Armijo search technique, we propose a novel conjugate gradient method which speeds up the convergence rate in training Elman networks in this paper. A conjugate gradient coefficient is proposed in the algorithm, which constructs conjugate gradient direction with sufficient descent property. Numerical results demonstrate that this method is more stable and efficient than the existing training methods. In addition, simulation shows that, the error function has a monotonically decreasing property and the gradient norm of the corresponding function tends to zero.