Quantum mechanics

Function (biology)

Algorithm

Banach space

Algebraic Riccati equation

Linguistics

Philosophy

Biology

Discrete mathematics

Statistics

Realization (probability)

Differential equation

Fréchet derivative

Zero (linguistics)

Applied mathematics

Convergence (economics)

Economics

Computer science

Mathematical optimization

Evolutionary biology

Physics

Nonlinear system

Reinforcement learning

Mathematics

Discrete time and continuous time

Nash equilibrium

Mathematical analysis

Artificial intelligence

Economic growth

Zero-sum game

Riccati equation

Newton's method

Q-learning

IEEE Transactions on Cybernetics

In this article, the data-based two-player zero-sum game problem is considered for linear discrete-time systems. This problem theoretically depends on solving the discrete-time game algebraic Riccati equation (DTGARE), while it requires complete system dynamics. To avoid solving the DTGARE, the Q-function is introduced and a data-based policy iteration Q-learning (PIQL) algorithm is developed to learn the optimal Q-function by using data collected from the real system. Writing the Q-function in a quadratic form, it is proved that the PIQL algorithm is equivalent to the Newton iteration method in the Banach space by using the Fréchet derivative. Then, the convergence of the PIQL algorithm can be guaranteed by Kantorovich's theorem. For the realization of the PIQL algorithm, the off-policy learning scheme is proposed using real data rather than the system model. Finally, the efficiency of the developed data-based PIQL method is validated through simulation studies.

Policy Iteration <i>Q</i>-Learning for Data-Based Two-Player Zero-Sum Game of Linear Discrete-Time Systems

Policy Iteration Q-Learning for Data-Based Two-Player Zero-Sum Game of Linear Discrete-Time Systems