Liu, L;
Zhang, S;
Kuang, Z;
Zhou, A;
Xue, J;
Wang, X;
Chen, Y;
... Zhang, W; + view all
(2021)
Group Fisher Pruning for Practical Network Compression.
In:
Proceedings of the 38 th International Conference on Machine Learning, PMLR 139.
(pp. pp. 7021-7032).
PMLR: Proceedings of Machine Learning Research
Preview |
Text
LiyangLiu-ICML2021-final.pdf - Accepted Version Download (3MB) | Preview |
Abstract
Network compression has been widely studied since it is able to reduce the memory and computation cost during inference. However, previous methods seldom deal with complicated structures like residual connections, group/depth-wise convolution and feature pyramid network, where channels of multiple layers are coupled and need to be pruned simultaneously. In this paper, we present a general channel pruning approach that can be applied to various complicated structures. Particularly, we propose a layer grouping algorithm to find coupled channels automatically. Then we derive a unified metric based on Fisher information to evaluate the importance of a single channel and coupled channels. Moreover, we find that inference speedup on GPUs is more correlated with the reduction of memory rather than FLOPs, and thus we employ the memory reduction of each channel to normalize the importance. Our method can be used to prune any structures including those with coupled channels. We conduct extensive experiments on various backbones, including the classic ResNet and ResNeXt, mobile-friendly MobileNetV2, and the NAS-based RegNet, both on image classification and object detection which is under-explored. Experimental results validate that our method can effectively prune sophisticated networks, boosting inference speed without sacrificing accuracy.
| Type: | Proceedings paper |
|---|---|
| Title: | Group Fisher Pruning for Practical Network Compression |
| Event: | PMLR 139: Thirty-eighth International Conference on Machine Learning |
| Dates: | 18 July 2021 - 24 July 2021 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | http://proceedings.mlr.press/v139/liu21ab.html |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10131008 |
Archive Staff Only
 |
View Item |
