Bellman equation

Quantum mechanics

Function (biology)

Hamilton–Jacobi–Bellman equation

Control (management)

Dynamic programming

Biology

Multi-agent system

Statistics

Consensus

Computer science

Mathematical optimization

Distributed computing

Evolutionary biology

Physics

Nonlinear system

Mathematics

Discrete time and continuous time

Artificial intelligence

Optimal control

Control theory (sociology)

Artificial neural network

IEEE Transactions on Systems Man and Cybernetics Systems

Conventionally, as the system's dynamics is known, the optimal consensus control problem relies on solving the coupled Hamilton-Jacobi-Bellman (HJB) equations. In this paper, with the unknown system dynamics being considered, a local Q-function-based adaptive dynamic programming method is put forward to deal with the optimal consensus control problem for unknown discrete-time nonlinear multiagent systems by approximating the solutions of the coupled HJB equations. First, a local Q-function is defined, which considers the local consensus error and the actions of the agent and its neighbors. Using the Q-function, it is convenient to get the derivatives with regard to the weights of the consensus control policies, even without the model of system dynamics. Then, with the defined local Q-function, a distributed policy iteration technique is developed, which is theoretically proved to be convergent to the solutions of the coupled HJB equations. An actor-critic neural network framework for implementing the developed model-free optimal consensus control method is constructed to approximate the local Q-functions and the control policies. Finally, the feasibility and effectiveness of the developed method are verified by a series of simulations.

Model-Free Distributed Consensus Control Based on Actor–Critic Framework for Discrete-Time Nonlinear Multiagent Systems