Soft Person Re-identification Network Pruning via Block-wise Adjacent Filter Decaying

Authors: Xiaodong Wang, Zhedong Zheng, Yang He, Fei Yan, Zhiqiang Zeng, Yi Yang

Published in IEEE Transactions on Cybernetics (TCYB), 2022

Recommended citation: Xiaodong Wang, Zhedong Zheng, Yang He, Fei Yan, Zhiqiang Zeng, Yi Yang, "Soft Person Re-identification Network Pruning via Block-wise Adjacent Filter Decaying." IEEE Transactions on Cybernetics, 2022. DOI: 10.1109/TCYB.2021.3130047
Download PDF: https://zdzheng.xyz/files/2022/Wang_Soft.pdf

Code is available at: https://github.com/x-d-wang/Soft-Person-Reidentification-Network-Pruning-via-Blockwise-Adjacent-Filter-Decaying

Abstract: Deep learning has shown significant successes in per- son re-identification (re-id) tasks. However, most existing works focus on discriminative feature learning and impose complex neural networks, suffering from low inference efficiency. In fact, feature extraction time is also crucial for real-world applications and light-weight models are needed. Prevailing pruning methods usually pay attention to compact classification models. However, these methods are sub-optimal for compacting re-id models, which usually produce continuous features and are sensitive to network pruning. The key point of pruning re-id models is how to retain the original filter distribution in continuous features as much as possible. In this work, we propose a block- wise adjacent filter decaying method to fill this gap. Specifically, given a trained model, we first evaluate the redundancy of filters based on the adjacency relationships to preserve the original filter distribution. Second, previous layer-wise pruning methods ignore that discriminative information is enhanced block-by- block. Therefore, we propose a block-wise filter pruning strategy to better utilize the block relations in the pre-trained model. Third, we propose a novel filter decaying policy to progressively reduce the scale of redundant filters. Different from conventional soft filter pruning that directly sets the filter values as zeros, the proposed filter decaying can keep the pre-trained knowledge as much as possible. We evaluate our method on three popular person re-identification datasets, i.e., Market-1501, DukeMTMC- reID, and MSMT17 V1. The proposed method shows superior performance to existing state-of-the-art pruning methods. Af- ter pruning over 91.9 % parameters on DukeMTMC-reID, the Rank-1 accuracy only drop 3.7%, demonstrating its effectiveness for compacting person re-identification.

@article{wang2022soft,
author = "Wang, Xiaodong and Zheng, Zhedong and He, Yang and Yan, Fei and Zeng, Zhiqiang and Yang, Yi",
title = "Soft Person Re-identification Network Pruning via Block-wise Adjacent Filter Decaying",
abstract = "Deep learning has shown significant successes in per- son re-identification (re-id) tasks. However, most existing works focus on discriminative feature learning and impose complex neural networks, suffering from low inference efficiency. In fact, feature extraction time is also crucial for real-world applications and light-weight models are needed. Prevailing pruning methods usually pay attention to compact classification models. However, these methods are sub-optimal for compacting re-id models, which usually produce continuous features and are sensitive to network pruning. The key point of pruning re-id models is how to retain the original filter distribution in continuous features as much as possible. In this work, we propose a block- wise adjacent filter decaying method to fill this gap. Specifically, given a trained model, we first evaluate the redundancy of filters based on the adjacency relationships to preserve the original filter distribution. Second, previous layer-wise pruning methods ignore that discriminative information is enhanced block-by- block. Therefore, we propose a block-wise filter pruning strategy to better utilize the block relations in the pre-trained model. Third, we propose a novel filter decaying policy to progressively reduce the scale of redundant filters. Different from conventional soft filter pruning that directly sets the filter values as zeros, the proposed filter decaying can keep the pre-trained knowledge as much as possible. We evaluate our method on three popular person re-identification datasets, i.e., Market-1501, DukeMTMC- reID, and MSMT17 V1. The proposed method shows superior performance to existing state-of-the-art pruning methods. Af- ter pruning over 91.9 \\% parameters on DukeMTMC-reID, the Rank-1 accuracy only drop 3.7\\%, demonstrating its effectiveness for compacting person re-identification.",
journal = "IEEE Transactions on Cybernetics",
url = "https://zdzheng.xyz/files/2022/Wang\_Soft.pdf",
code = "https://github.com/x-d-wang/Soft-Person-Reidentification-Network-Pruning-via-Blockwise-Adjacent-Filter-Decaying",
doi = "10.1109/TCYB.2021.3130047",
year = "2022" }