Algorithm

Image compression

Overhead (engineering)

Materials science

Implementation

Computer security

Convergence (economics)

Image processing

Operating system

Economics

Computer science

Physics

MNIST database

Mathematics

Optics

Image (mathematics)

Computer engineering

Rate of convergence

Artificial intelligence

Geometry

Composite material

Data compression

Compression (physics)

Programming language

Deep learning

Economic growth

Data compression ratio

Regular polygon

Machine learning

Convex function

Adaptation (eye)

Key (lock)

Federated Learning (FL) incurs high communication overhead, which can be greatly alleviated by compression for model updates. Yet the tradeoff between compression and model accuracy in the networked environment remains unclear and, for simplicity, most implementations adopt a fixed compression rate only. In this paper, we for the first time systematically examine this tradeoff, identifying the influence of the compression error on the final model accuracy with respect to the learning rate. Specifically, we factor the compression error of each global iteration into the convergence rate analysis under both strongly convex and non-convex loss functions. We then present an adaptation framework to maximize the final model accuracy by strategically adjusting the compression rate in each iteration. We have discussed the key implementation issues of our framework in practical networks with representative compression algorithms. Experiments over the popular MNIST and CIFAR-10 datasets confirm that our solution effectively reduces network traffic yet maintains high model accuracy in FL.

Optimal Rate Adaption in Federated Learning with Compressed Communications