MNIST database

Stochastic gradient descent

Convergence (economics)

Artificial intelligence

Economics

Momentum (technical analysis)

Deep learning

Economic growth

Finance

Mathematics

Mathematical optimization

Gradient descent

Computer science

Algorithm

Key (lock)

Rate of convergence

Applied mathematics

Computer security

Artificial neural network

IEEE Transactions on Parallel and Distributed Systems

Federated learning (FL) provides a communication-efficient approach to solve machine learning problems concerning distributed data, without sending raw data to a central server. However, existing works on FL only utilize first-order gradient descent (GD) and do not consider the preceding iterations to gradient update which can potentially accelerate convergence. In this article, we consider momentum term which relates to the last iteration. The proposed momentum federated learning (MFL) uses momentum gradient descent (MGD) in the local update step of FL system. We establish global convergence properties of MFL and derive an upper bound on MFL convergence rate. Comparing the upper bounds on MFL and FL convergence rates, we provide conditions in which MFL accelerates the convergence. For different machine learning models, the convergence performance of MFL is evaluated based on experiments with MNIST and CIFAR-10 datasets. Simulation results confirm that MFL is globally convergent and further reveal significant convergence improvement over FL.

Accelerating Federated Learning via Momentum Gradient Descent