Channel (broadcasting)

Mutual information

Pattern recognition (psychology)

Biology

Pruning

Artificial intelligence

Computer science

Botany

Telecommunications

Convolutional neural network

Channel pruning for light-weighting networks is very effective in reducing memory footprint and computational cost. Many channel pruning methods assume that the magnitude of a particular element corresponding to each channel reflects the importance of the channel. Unfortunately, such an assumption does not always hold. To solve this problem, this paper proposes a new method to measure the importance of channels based on gradients of mutual information. The proposed method computes and measures gradients of mutual information during back-propagation by arranging a module capable of estimating mutual information. By using the measured statistics as the importance of the channel, less important channels can be removed. Finally, the fine-tuning enables robust performance restoration of the pruned model. Experimental results show that the proposed method provides better performance with smaller parameter sizes and FLOPs than the conventional schemes.

Channel Pruning Via Gradient Of Mutual Information For Light-Weight Convolutional Neural Networks