Bellman equation

Quantum mechanics

Hamilton–Jacobi–Bellman equation

Algorithm

Residual

Control (management)

Discrete mathematics

Equivalence (formal languages)

Multi-agent system

Applied mathematics

Gradient descent

Jacobian matrix and determinant

Computer science

Mathematical optimization

Physics

Nonlinear system

Reinforcement learning

Mathematics

Artificial intelligence

Optimal control

Programming language

Control theory (sociology)

Artificial neural network

Identifier

IEEE Transactions on Industrial Electronics

The article proposes an optimized leader-follower formation control using a simplified reinforcement learning (RL) of identifier-critic-actor architecture for a class of nonlinear multiagent systems. In general, optimal control is expected to be obtained by solving Hamilton-Jacobi-Bellman (HJB) equation, but the equation associated with a nonlinear system is difficult to solve by analytical method. Although the difficulty can be effectively overcome by the RL strategy, the existing RL algorithms are very complex because their updating laws are obtained by carrying out gradient descent algorithm to square of the approximated HJB equation (Bellman residual error). For a multiagent system, due to the state coupling problem, these methods will become difficult implementation. In the proposed optimized scheme, the RL updating laws are derived from negative gradient of a simple positive function, which is the equivalence to HJB equation; therefore, the control algorithm is significantly simple. Furthermore, in order to solve the problem of unknown system dynamics, an adaptive identifier is integrated into the control. Finally, the theory and simulation demonstrate that the optimized formation scheme can guarantee the desired control performance.

Optimized Formation Control Using Simplified Reinforcement Learning for a Class of Multiagent Systems With Unknown Dynamics