Channel (broadcasting)

Pattern recognition (psychology)

Automotive engineering

Organic chemistry

Internal combustion engine

Agronomy

Data compression

Biology

Data mining

Pruning

Epistemology

Machine learning

Engineering

Philosophy

Artificial intelligence

Composite material

Compression ratio

Compression (physics)

Chemistry

Materials science

Layer (electronics)

Computer science

Simple (philosophy)

Artificial neural network

Telecommunications

Selection (genetic algorithm)

arXiv (Cornell University)

Compression techniques for deep neural networks are important for implementing them on small embedded devices. In particular, channel-pruning is a useful technique for realizing compact networks. However, many conventional methods require manual setting of compression ratios in each layer. It is difficult to analyze the relationships between all layers, especially for deeper models. To address these issues, we propose a simple channel-pruning technique based on attention statistics that enables to evaluate the importance of channels. We improved the method by means of a criterion for automatic channel selection, using a single compression ratio for the entire model in place of per-layer model analysis. The proposed approach achieved superior performance over conventional methods with respect to accuracy and the computational costs for various models and datasets. We provide analysis results for behavior of the proposed criterion on different datasets to demonstrate its favorable properties for channel pruning.

PCAS: Pruning Channels with Attention Statistics for Deep Network Compression