KO | EN

GitLite

Minimalist Open Source Repository Search Engine

gitlite — search

#dsh #dsh-plugin #ai-agents #developer-tools #typescript #codex #cli #claude-code #mcp #rust #deepseek-harness #ai-agent

Awesome-Pruning

★ 2,498 Open GitHub ↗

A curated list of neural network pruning resources.

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Awesome Pruning [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)

A curated list of neural network pruning and related resources. Inspired by [awesome-deep-vision](https://github.com/kjw0612/awesome-deep-vision), [awesome-adversarial-machine-learning](https://github.com/yenchenlin/awesome-adversarial-machine-learning), [awesome-deep-learning-papers](https://github.com/terryum/awesome-deep-learning-papers) and [Awesome-NAS](https://github.com/D-X-Y/Awesome-NAS).

Please feel free to [pull requests](https://github.com/he-y/awesome-Pruning/pulls) or [open an issue](https://github.com/he-y/awesome-Pruning/issues) to add papers.

## Table of Contents

- [Type of Pruning](#type-of-pruning)

- [A Survey of Structured Pruning](#a-survey-of-structured-pruning-arxiv-version-and-ieee-t-pami-version)

- [2023 Venues](#2023)

- [2022 Venues](#2022)

- [2021 Venues](#2021)

- [2020 Venues](#2020)

- [2019 Venues](#2019)

- [2018 Venues](#2018)

- [2017 Venues](#2017)

- [2016 Venues](#2016)

- [2015 Venues](#2015)

### Type of Pruning

| Type        | `F`            | `W`            | `S`              | `Other`     |
|:----------- |:--------------:|:--------------:|:----------------:|:-----------:|
| Explanation | Filter pruning | Weight pruning | Special Networks | other types |

### A Survey of Structured Pruning ([arXiv version](https://arxiv.org/abs/2303.00566) and [IEEE T-PAMI version](https://ieeexplore.ieee.org/document/10330640))

Please cite our paper if it's helpful:
```
@article{he2024structured,
  author={He, Yang and Xiao, Lingao},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  title={Structured Pruning for Deep Convolutional Neural Networks: A Survey}, 
  year={2024},
  volume={46},
  number={5},
  pages={2900-2919},
  doi={10.1109/TPAMI.2023.3334614}}
```

The related papers are categorized as below:
![Structured Pruning Taxonomy](https://raw.githubusercontent.com/he-y/Awesome-Pruning/master/Structured_Taxonomy.png)

### 2023
| Title                                                                                                                            | Venue | Type    | Code |
|:-------------------------------------------------------------------------------------------------------------------------------- |:-----:|:-------:|:----:|
| [Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph](https://openreview.net/forum?id=uVcDssQff_)                                                            | ICLR  | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/ramanujan-on-pai)(Releasing)                                |
| [Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?](https://openreview.net/forum?id=xSsW2Am-ukZ)                                                | ICLR  | `W`     | -                                                                                                           |
| [Bit-Pruning: A Sparse Multiplication-Less Dot-Product](https://openreview.net/forum?id=YUDiZcZTI8)                                                                               | ICLR  | `W`     | [Code Deleted](https://github.com/DensoITLab/bitprune)                                                      |
| [NTK-SAP: Improving neural network pruning by aligning training dynamics](https://openreview.net/forum?id=-5EWhW_4qWP)                                                            | ICLR  | `W`     | -                                                                                                           |
| [A Unified Framework for Soft Threshold Pruning](https://openreview.net/forum?id=cCFqcrq0d8)                                                                                      | ICLR  | `W`     | [PyTorch(Author)](https://github.com/Yanqi-Chen/LATS)                                                       |
| [CrAM: A Compression-Aware Minimizer](https://openreview.net/forum?id=_eTZBs-yedr)                                                                                                | ICLR  | `W`     | -                                                                                                           |
| [Trainability Preserving Neural Pruning](https://openreview.net/forum?id=AZFvpnnewr)                                                                                              | ICLR  | `F`     | -                                                                                                           |
| [DFPC: Data flow driven pruning of coupled channels without data](https://openreview.net/forum?id=mhnHqRqcjYU)                                                                    | ICLR  | `F`     | [PyTorch(Author)](https://drive.google.com/drive/folders/18eRYzWnB_6Qq0cYiSzvyOgicqn50g3-m)                 |
| [TVSPrune - Pruning Non-discriminative filters via Total Variation separability of intermediate representations without fine tuning](https://openreview.net/forum?id=sZI1Oj9KBKy) | ICLR  | `F`     | [PyTorch(Author)](https://github.com/tvsprune/TVS_Prune)                                                    |
| [HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers](https://openreview.net/forum?id=D7srTrGhAs)                                                        | ICLR  | `F`     | -                                                                                                           |
| [MECTA: Memory-Economic Continual Test-Time Model Adaptation](https://openreview.net/forum?id=N92hjSf5NNh)                                                                        | ICLR  | `F`     | -                                                                                                           |
| [DepthFL : Depthwise Federated Learning for Heterogeneous Clients](https://openreview.net/forum?id=pf8RIZTMU58)                                                                   | ICLR  | `F`     | -                                                                                                           |
| [OTOv2: Automatic, Generic, User-Friendly](https://openreview.net/forum?id=7ynoX1ojPMt)                                                                                           | ICLR  | `F`     | [PyTorch(Author)](https://github.com/tianyic/only_train_once)                                               |
| [Over-parameterized Model Optimization with Polyak-Lojasiewicz Condition](https://openreview.net/forum?id=aBIpZvMdS56)                                                            | ICLR  | `F`     | -                                                                                                           |
| [Pruning Deep Neural Networks from a Sparsity Perspective](https://openreview.net/forum?id=i-DleYh34BM)                                                                           | ICLR  | `WF`    | [PyTorch(Author)](https://github.com/dem123456789/Pruning-Deep-Neural-Networks-from-a-Sparsity-Perspective) |
| [Holistic Adversarially Robust Pruning](https://openreview.net/forum?id=sAJDi9lD06L)                                                                                              | ICLR  | `WF`    | -                                                                                                           |
| [How I Learned to Stop Worrying and Love Retraining](https://openreview.net/forum?id=_nF5imFKQI)                                                                                  | ICLR  | `WF`    | [PyTorch(Author)](https://github.com/ZIB-IOL/BIMP)                                                          |
| [Symmetric Pruning in Quantum Neural Networks](https://openreview.net/forum?id=K96AogLDT2K)                                                                                       | ICLR  | `S`     | -                                                                                                           |
| [Rethinking Graph Lottery Tickets: Graph Sparsity Matters](https://openreview.net/forum?id=fjh7UGQgOB)                                                                            | ICLR  | `S`     | -                                                                                                           |
| [Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks](https://openreview.net/forum?id=4UldFtZ_CVF)                                                   | ICLR  | `S`     | -                                                                                                           |
| [Searching Lottery Tickets in Graph Neural Networks: A Dual Perspective](https://openreview.net/forum?id=Dvs-a3aymPe)                                                             | ICLR  | `S`     | -                                                                                                           |
| [Diffusion Models for Causal Discovery via Topological Ordering](https://openreview.net/forum?id=Idusfje4-Wq)                                                                     | ICLR  | `S`     | -                                                                                                           |
| [A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesis](https://openreview.net/forum?id=vVJZtlZB9D)                                                    | ICLR  | `Other` | -                                                                                                           |
| [Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!](https://openreview.net/forum?id=J6F3lLg4Kdp)                                                           | ICLR  | `Other` | -                                                                                                           |
| [Minimum Variance Unbiased N:M Sparsity for the Neural Gradients](https://openreview.net/forum?id=vuD2xEtxZcj)                                                                    | ICLR  | `Other` | -                                                                                                           |

### 2022
| Title                                                                                                                            | Venue | Type    | Code |
|:-------------------------------------------------------------------------------------------------------------------------------- |:-----:|:-------:|:----:|
| [Parameter-Efficient Masking Networks](https://openreview.net/forum?id=7rcuQ_V2GFg)                                                                                                   | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/yueb17/PEMN)                            |
| ["Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach](https://openreview.net/forum?id=NaW6T93F34m)                                      | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/Model-Compression/Lossless_Compression) |
| [Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing](https://openreview.net/forum?id=2EUJ4e6H4OX) | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/GATECH-EIC/S3-Router)                   |
| [Models Out of Line: A Fourier Lens on Distribution Shift Robustness](https://openreview.net/forum?id=YZ-N-sejjwO)                                                                    | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/sarafridov/RobustNets)                  |
| [Robust Binary Models by Pruning Randomly-initialized Networks](https://openreview.net/forum?id=5g-h_DILemH)                                                                          | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/IVRL/RobustBinarySubNet)                |
| [Rare Gems: Finding Lottery Tickets at Initialization](https://openreview.net/forum?id=Jpxd93u2vK-)                                                                                   | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/ksreenivasan/pruning_is_enough)         |
| [Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning](https://openreview.net/forum?id=ksVGCOlOEba)                                             | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/IST-DASLab/OBC)                         |
| [Pruning’s Effect on Generalization Through the Lens of Training and Regularization](https://openreview.net/forum?id=OrcLKV9sKWp)                                                     | NeurIPS | `W`     | -                                                                            |
| [Back Razor: Memory-Efficient Transfer Learning by Self-Sparsified Backpropagation](https://openreview.net/forum?id=mTXQIpXPDbh)                                                      | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/BackRazor_Neurips22)         |
| [Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspective](https://openreview.net/forum?id=fbUybomIuE)                                                                | NeurIPS | `W`     | -                                                                            |
| [Sparse Winning Tickets are Data-Efficient Image Recognizers](https://openreview.net/forum?id=wfKbtSjHA6F)                                                                            | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/DataEfficientLTH)            |
| [Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks](https://openreview.net/forum?id=QLPzCpu756J)                                                 | NeurIPS | `W`     | -                                                                            |
| [Weighted Mutual Learning with Diversity-Driven Model Compression](https://openreview.net/forum?id=UQJoGBNRX4)                                                                        | NeurIPS | `F`     | -                                                                            |
| [SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance](https://openreview.net/forum?id=oQIJsMlyaW_)                                                                | NeurIPS | `F`     | -                                                                            |
| [Data-Efficient Structured Pruning via Submodular Optimization](https://openreview.net/forum?id=K2QGzyLwpYG)                                                                          | NeurIPS | `F`     | [PyTorch(Author)](https://github.com/marwash25/subpruning)                   |
| [Structural Pruning via Latency-Saliency Knapsack](https://openreview.net/forum?id=cUOR-_VsavA)                                                                                       | NeurIPS | `F`     | [PyTorch(Author)](https://github.com/NVlabs/HALP)                            |
| [Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm](https://openreview.net/forum?id=5hgYi4r5MDp)                                                        | NeurIPS | `WF`    | -                                                                            |
| [Pruning Neural Networks via Coresets and Convex Geometry: Towards No Assumptions](https://openreview.net/forum?id=btpIaJiRx6z)                                                       | NeurIPS | `WF`    | -                                                                            |
| [Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints](https://openreview.net/forum?id=XUvSYc6TqDF)                       | NeurIPS | `WF`    | [PyTorch(Author)](https://github.com/gallego-posada/constrained_sparsity)    |
| [Advancing Model Pruning via Bi-level Optimization](https://openreview.net/forum?id=t6O08FxvtBY)                                                                                      | NeurIPS | `WF`    | [PyTorch(Author)](https://github.com/OPTML-Group/BiP)                        |
| [Emergence of Hierarchical Layers in a Single Sheet of Self-Organizing Spiking Neurons](https://openreview.net/forum?id=cPVuuk1lZb3)                                                  | NeurIPS | `S`     | -                                                                            |
| [CryptoGCN: Fast and Scalable Homomorphically Encrypted Graph Convolutional Network Inference](https://openreview.net/forum?id=VeQBBm1MmTZ)                                           | NeurIPS | `S`     | [PyTorch(Author)](https://github.com/ranran0523/CryptoGCN)(Releasing)        |
| [Transform Once: Efficient Operator Learning in Frequency Domain](https://openreview.net/forum?id=B2PpZyAAEgV)                                                                        | NeurIPS | `Other` | [PyTorch(Author)](https://github.com/DiffEqML/kairos)(Releasing)             |
| [Most Activation Functions Can Win the Lottery Without Excessive Depth](https://openreview.net/forum?id=NySDKS9SxN)                                                                   | NeurIPS | `Other` | [PyTorch(Author)](https://github.com/RelationalML/LT-existence)              |
| [Pruning has a disparate impact on model accuracy](https://openreview.net/forum?id=11nMVZK0WYM)                                                                                       | NeurIPS | `Other` | -                                                                            |
| [Model Preserving Compression for Neural Networks](https://openreview.net/forum?id=gt-l9Hu2ndd)                                                                                       | NeurIPS | `Other` | [PyTorch(Author)](https://github.com/jerry-chee/ModelPreserveCompressionNN)  |
| [Prune Your Model Before Distill It](https://link.springer.com/10.1007/978-3-031-20083-0_8)                                                                                           | ECCV | `W`     | [PyTorch(Author)](https://https://github.com/ososos888/prune-then-distill)                                       |
| [FedLTN: Federated Learning for Sparse and Personalized Lottery Ticket Networks](https://link.springer.com/10.1007/978-3-031-19775-8_5)                                               | ECCV | `W`     | -                                                                                                                |
| [FairGRAPE: Fairness-Aware GRAdient Pruning mEthod for Face Attribute Classification](https://link.springer.com/10.1007/978-3-031-19778-9_24)                                         | ECCV | `F`     | [PyTorch(Author)](https://github.com/Bernardo1998/FairGRAPE)                                                     |
| [SuperTickets: Drawing Task-Agnostic Lottery Tickets from Supernets via Jointly Architecture Searching and Parameter Pruning](https://link.springer.com/10.1007/978-3-031-20083-0_40) | ECCV | `F`     | [PyTorch(Author)](https://github.com/GATECH-EIC/SuperTickets)                                                    |
| [Ensemble Knowledge Guided Sub-network Search and Fine-Tuning for Filter Pruning](https://link.springer.com/10.1007/978-3-031-20083-0_34)                                             | ECCV | `F`     | [PyTorch(Author)](https://github.com/sseung0703/EKG)                                                             |
| [CPrune: Compiler-Informed Model Pruning for Efficient Target-Aware DNN Execution](https://link.springer.com/10.1007/978-3-031-20044-1_37)                                            | ECCV | `F`     | [PyTorch(Author)](https://github.com/taehokim20/CPrune)                                                          |
| [Soft Masking for Cost-Constrained Channel Pruning](https://link.springer.com/10.1007/978-3-031-20083-0_38)                                                                           | ECCV | `F`     | [PyTorch(Author)](https://github.com/NVlabs/SMCP)                                                                |
| [Filter Pruning via Feature Discrimination in Deep Neural Networks](https://link.springer.com/10.1007/978-3-031-19803-8_15)                                                           | ECCV | `F`     | -                                                                                                                |
| [Disentangled Differentiable Network Pruning](https://link.springer.com/10.1007/978-3-031-20083-0_20)                                                                                 | ECCV | `F`     | -                                                                                                                |
| [Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps](https://link.springer.com/10.1007/978-3-031-19803-8_17)                                                | ECCV | `F`     | [PyTorch(Author)](https://github.com/Alii-Ganjj/InterpretationsSteeredPruning)                                   |
| [Bayesian Optimization with Clustering and Rollback for CNN Auto Pruning](https://link.springer.com/10.1007/978-3-031-20050-2_29)                                                     | ECCV | `F`     | [PyTorch(Author)](https://github.com/fanhanwei/BOCR)                                                             |
| [Multi-granularity Pruning for Model Acceleration on Mobile Devices](https://link.springer.com/10.1007/978-3-031-20083-0_29)                                                          | ECCV | `WF`    | -                                                                                                                |
| [Exploring Lottery Ticket Hypothesis in Spiking Neural Networks](https://link.springer.com/10.1007/978-3-031-19775-8_7)                                                               | ECCV | `S`     | [PyTorch(Author)](https://github.com/Intelligent-Computing-Lab-Yale/Exploring-Lottery-Ticket-Hypothesis-in-SNNs) |
| [Towards Ultra Low Latency Spiking Neural Networks for Vision and Sequential Tasks Using Temporal Pruning](https://link.springer.com/10.1007/978-3-031-20083-0_42)                    | ECCV | `S`     | -                                                                                                                |
| [Recent Advances on Neural Network Pruning at Initialization](https://www.ijcai.org/proceedings/2022/786)                                                                                                                                                                                                                                                  | IJCAI                | `W`     | [PyTorch(Author)](https://github.com/mingsun-tse/smile-pruning)                                |
| [FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server](https://www.ijcai.org/proceedings/2022/385)                                                                                                                                                                                                         | IJCAI                | `F`     | -                                                                                              |
| [On the Channel Pruning using Graph Convolution Network for Convolutional Neural Network Acceleration](https://www.ijcai.org/proceedings/2022/431)                                                                                                                                                                                                         | IJCAI                | `F`     | -                                                                                              |
| [Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization](https://www.ijcai.org/proceedings/2022/449)                                                                                                                                                                                                | IJCAI                | `F`     | -                                                                                              |
| [Neural Network Pruning by Cooperative Coevolution](https://www.ijcai.org/proceedings/2022/667)                                                                                                                                                                                                                                                            | IJCAI                | `F`     | -                                                                                              |
| [SPDY: Accurate Pruning with Speedup Guarantees](https://proceedings.mlr.press/v162/frantar22a.html)                                                                                                                                                                                                                                                       | ICML                 | `W`     | [PyTorch(Author)](https://github.com/IST-DASLab/spdy)                                          |
| [Sparse Double Descent: Where Network Pruning Aggravates Overfitting](https://proceedings.mlr.press/v162/he22d.html)                                                                                                                                                                                                                                       | ICML                 | `W`     | [PyTorch(Author)](https://github.com/hezheug/sparse-double-descent)                            |
| [The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks](https://proceedings.mlr.press/v162/yu22f.html)                                                                                                                                                                                                               | ICML                 | `W`     | [PyTorch(Author)](https://github.com/yuxwind/CBS)                                              |
| [Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness](https://proceedings.mlr.press/v162/chen22af.html)                                                                                                                                                                                                                                | ICML                 | `F`     | [PyTorch(Author)](https://github.com/VITA-Group/Linearity-Grafting)                            |
| [Winning the Lottery Ahead of Time: Efficient Early Network Pruning](https://proceedings.mlr.press/v162/rachwan22a.html)                                                                                                                                                                                                                                   | ICML                 | `F`     | [PyTorch(Author)](https://github.com/johnrachwan123/Early-Cropression-via-Gradient-Flow-Preservation) |
| [Topology-Aware Network Pruning using Multi-stage Graph Embedding and Reinforcement Learning](https://proceedings.mlr.press/v162/yu22e.html)                                                                                                                                                                                                               | ICML                 | `F`     | [PyTorch(Author)](https://github.com/yusx-swapp/GNN-RL-Model-Compression)                      |
| [Fast Lossless Neural Compression with Integer-Only Discrete Flows](https://proceedings.mlr.press/v162/wang22a.html)                                                                                                                                                                                                                                       | ICML                 | `F`     | [PyTorch(Author)](https://github.com/thu-ml/IODF)                                              |
| [DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks](https://proceedings.mlr.press/v162/fu22c.html)                                                                                                                                                                                            | ICML                 | `Other` | [PyTorch(Author)](https://github.com/facebookresearch/DepthShrinker)                           |
| [PAC-Net: A Model Pruning Approach to Inductive Transfer Learning](https://proceedings.mlr.press/v162/myung22a.html)                                                                                                                                                                                                                                       | ICML                 | `Other` | -                                                                                              |
| [Neural Network Pruning Denoises the Features and Makes Local Connectivity Emerge in Visual Tasks](https://proceedings.mlr.press/v162/pellegrini22a.html)                                                                                                                                                                                                  | ICML                 | `Other` | [PyTorch(Author)](https://github.com/phiandark/SiftingFeatures)                                |
| [Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs](https://openaccess.thecvf.com/content/CVPR2022/html/Wimmer_Interspace_Pruning_Using_Adaptive_Filter_Representations_To_Improve_Training_of_CVPR_2022_paper.html)                                                                                            | CVPR                 | `W`     | -                                                                                              |
| [Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network](https://openaccess.thecvf.com/content/CVPR2022/html/Lee_Masking_Adversarial_Damage_Finding_Adversarial_Saliency_for_Robust_and_Sparse_CVPR_2022_paper.html)                                                                                                       | CVPR                 | `W`     | -                                                                                              |
| [When To Prune? A Policy Towards Early Structural Pruning](https://openaccess.thecvf.com/content/CVPR2022/html/Shen_When_To_Prune_A_Policy_Towards_Early_Structural_Pruning_CVPR_2022_paper.html)                                                                                                                                                          | CVPR                 | `F`     | -                                                                                              |
| [Fire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask PredictionFire Together Wire Together: A Dynamic Pruning Approach With Self-Supervised Mask Prediction](https://openaccess.thecvf.com/content/CVPR2022/html/Elkerdawy_Fire_Together_Wire_Together_A_Dynamic_Pruning_Approach_With_Self-Supervised_CVPR_2022_paper.html) | CVPR                 | `F`     | -                                                                                              |
| [Revisiting Random Channel Pruning for Neural Network Compression](https://openaccess.thecvf.com/content/CVPR2022/html/Li_Revisiting_Random_Channel_Pruning_for_Neural_Network_Compression_CVPR_2022_paper.html)                                                                                                                                           | CVPR                 | `F`     | [PyTorch(Author)](https://github.com/ofsoundof/random_channel_pruning)(Releasing)              |
| [Learning Bayesian Sparse Networks With Full Experience Replay for Continual Learning](https://openaccess.thecvf.com/content/CVPR2022/html/Yan_Learning_Bayesian_Sparse_Networks_With_Full_Experience_Replay_for_Continual_CVPR_2022_paper.html)                                                                                                           | CVPR                 | `F`     | -                                                                                              |
| [DECORE: Deep Compression With Reinforcement Learning](https://openaccess.thecvf.com/content/CVPR2022/html/Alwani_DECORE_Deep_Compression_With_Reinforcement_Learning_CVPR_2022_paper.html)                                                                                                                                                                | CVPR                 | `F`     | -                                                                                              |
| [CHEX: CHannel EXploration for CNN Model Compression](https://openaccess.thecvf.com/content/CVPR2022/html/Hou_CHEX_CHannel_EXploration_for_CNN_Model_Compression_CVPR_2022_paper.html)                                                                                                                                                                     | CVPR                 | `F`     | -                                                                                              |
| [Compressing Models With Few Samples: Mimicking Then Replacing](https://openaccess.thecvf.com/content/CVPR2022/html/Wang_Compressing_Models_With_Few_Samples_Mimicking_Then_Replacing_CVPR_2022_paper.html)                                                                                                                                                | CVPR                 | `F`     | [PyTorch(Author)](https://github.com/cjnjuwhy/MiR)(Releasing)                                  |
| [Contrastive Dual Gating: Learning Sparse Features With Contrastive Learning](https://openaccess.thecvf.com/content/CVPR2022/html/Meng_Contrastive_Dual_Gating_Learning_Sparse_Features_With_Contrastive_Learning_CVPR_2022_paper.html)                                                                                                                    | CVPR                 | `WF`    | -                                                                                              |
| [DiSparse: Disentangled Sparsification for Multitask Model Compression](https://openaccess.thecvf.com/content/CVPR2022/html/Sun_DiSparse_Disentangled_Sparsification_for_Multitask_Model_Compression_CVPR_2022_paper.html)                                                                                                                                 | CVPR                 | `Other` | [PyTorch(Author)](https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression)            |
| [Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining](https://openreview.net/forum?id=O1DEtITim__)                                                                                                                                                                                                               | ICLR **(Spotlight)** | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/SFW-Once-for-All-Pruning)                      |
| [On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning](https://openreview.net/forum?id=Fl3Mg_MZR-)                                                                                                                                                                                                                           | ICLR **(Spotlight)** | `W`     | -                                                                                              |
| [An Operator Theoretic View On Pruning Deep Neural Networks](https://openreview.net/forum?id=pWBNOgdeURp)                                                                                                                                                                                                                                                  | ICLR                 | `W`     | [PyTorch(Author)](https://github.com/william-redman/Koopman_pruning)                           |
| [Effective Model Sparsification by Scheduled Grow-and-Prune Methods](https://openreview.net/forum?id=xa6otUDdP2W)                                                                                                                                                                                                                                          | ICLR                 | `W`     | [PyTorch(Author)](https://github.com/boone891214/GaP)                                          |
| [Signing the Supermask: Keep, Hide, Invert](https://openreview.net/forum?id=e0jtGTfPihs)                                                                                                                                                                                                                                                                   | ICLR                 | `W`     | -                                                                                              |
| [How many degrees of freedom do we need to train deep networks: a loss landscape perspective](https://openreview.net/forum?id=ChMLTGRjFcU)                                                                                                                                                                                                                 | ICLR                 | `W`     | [PyTorch(Author)](https://github.com/ganguli-lab/degrees-of-freedom)                           |
| [Dual Lottery Ticket Hypothesis](https://openreview.net/forum?id=fOsN52jn25l)                                                                                                                                                                                                                                                                              | ICLR                 | `W`     | [PyTorch(Author)](https://github.com/yueb17/DLTH)                                              |
| [Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently](https://openreview.net/forum?id=moHCzz6D5H3)                                                                                                                                                                                                  | ICLR                 | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/Peek-a-Boo)                                    |
| [Sparsity Winning Twice: Better Robust Generalization from More Efficient Training](https://openreview.net/forum?id=SYuJXrXq8tw)                                                                                                                                                                                                                           | ICLR                 | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/Sparsity-Win-Robust-Generalization)            |
| [SOSP: Efficiently Capturing Global Correlations by Second-Order Structured Pruning](https://openreview.net/forum?id=t5EmXZ3ZLR)                                                                                                                                                                                                                           | ICLR **(Spotlight)** | `F`     | [PyTorch(Author)](https://github.com/boschresearch/sosp)(Releasing)                            |
| [Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models](https://openreview.net/forum?id=Nfl-iXa-y7R)                                                                                                                                                                                                                         | ICLR **(Spotlight)** | `F`     | [PyTorch(Author)](https://github.com/HazyResearch/pixelfly)                                    |
| [Revisit Kernel Pruning with Lottery Regulated Grouped Convolutions](https://openreview.net/forum?id=LdEhiMG9WLO)                                                                                                                                                                                                                                          | ICLR                 | `F`     | [PyTorch(Author)](https://github.com/choH/lottery_regulated_grouped_kernel_pruning)            |
| [Plant 'n' Seek: Can You Find the Winning Ticket?](https://openreview.net/forum?id=9n9c8sf0xm)                                                                                                                                                                                                                                                             | ICLR                 | `F`     | [PyTorch(Author)](http://www.github.com/RelationalML/PlantNSeek)                               |
| [Proving the Lottery Ticket Hypothesis for Convolutional Neural Networks](https://openreview.net/forum?id=Vjki79-619-)                                                                                                                                                                                                                                     | ICLR                 | `F`     | [PyTorch(Author)](https://github.com/ArthurWalraven/cnnslth)                                   |
| [On the Existence of Universal Lottery Tickets](https://openreview.net/forum?id=SYB4WrJql1n)                                                                                                                                                                                                                                                               | ICLR                 | `F`     | [PyTorch(Author)](https://github.com/RelationalML/UniversalLT)                                 |
| [Training Structured Neural Networks Through Manifold Identification and Variance Reduction](https://openreview.net/forum?id=mdUYT5QV0O)                                                                                                                                                                                                                   | ICLR                 | `F`     | [PyTorch(Author)](https://www.github.com/zihsyuan1214/rmda)                                    |
| [Learning Efficient Image Super-Resolution Networks via Structure-Regularized Pruning](https://openreview.net/forum?id=AjGC97Aofee)                                                                                                                                                                                                                        | ICLR                 | `F`     | [PyTorch(Author)](https://github.com/MingSun-Tse/SRP)                                          |
| [Prospect Pruning: Finding Trainable Weights at Initialization using Meta-Gradients](https://openreview.net/forum?id=AIgn9uwfcD1)                                                                                                                                                                                                                          | ICLR                 | `WF`    | [PyTorch(Author)](https://github.com/mil-ad/prospr)                                            |
| [The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training](https://openreview.net/forum?id=VBZJ_3tz-t)                                                                                                                                                                                                      | ICLR                 | `Other` | [PyTorch(Author)](https://github.com/VITA-Group/Random_Pruning)                                |
| [Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks](https://ojs.aaai.org/index.php/AAAI/article/view/20842)                                                                                                                                                                                                         | AAAI                 | `W`     | -                                                                                              |
| [Prior Gradient Mask Guided Pruning-Aware Fine-Tuning](https://ojs.aaai.org/index.php/AAAI/article/view/19888)                                                                                                                                                                                                                                             | AAAI                 | `F`     | -                                                                                              |
| [Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition](https://ojs.aaai.org/index.php/AAAI/article/view/19958)                                                                                                                                                                                                     | AAAI                 | `Other` | -                                                                                              |

### 2021
| Title                                                                                                                            | Venue | Type    | Code |
|:-------------------------------------------------------------------------------------------------------------------------------- |:-----:|:-------:|:----:|
| [Validating the Lottery Ticket Hypothesis with Inertial Manifold Theory](https://papers.nips.cc/paper/2021/hash/fdc42b6b0ee16a2f866281508ef56730-Abstract.html)                                                                                                        | NeurIPS | `W`     | -                                                                                                  |
| [The Elastic Lottery Ticket Hypothesis](https://papers.nips.cc/paper/2021/hash/dfccdb8b1cc7e4dab6d33db0fef12b88-Abstract.html)                                                                                                                                         | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/ElasticLTH)                                        |
| [Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?](https://papers.nips.cc/paper/2021/hash/6a130f1dc6f0c829f874e92e5458dced-Abstract.html)                                                                                           | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/boone891214/sanity-check-LTH)                                 |
| [Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Sparse Neural Networks](https://papers.nips.cc/paper/2021/hash/15f99f2165aa8c86c9dface16fefd281-Abstract.html)                                                                             | NeurIPS | `W`     | -                                                                                                  |
| [You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership](https://papers.nips.cc/paper/2021/hash/23e582ad8087f2c03a5a31c125123f9a-Abstract.html)                                                                                | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/NO-stealing-LTH)                                   |
| [Pruning Randomly Initialized Neural Networks with Iterative Randomization](https://papers.nips.cc/paper/2021/hash/23e582ad8087f2c03a5a31c125123f9a-Abstract.html)                                                                                                     | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/dchiji-ntt/iterand)                                           |
| [Sparse Training via Boosting Pruning Plasticity with Neuroregeneration](https://papers.nips.cc/paper/2021/hash/5227b6aaf294f5f027273aebf16015f2-Abstract.html)                                                                                                        | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/VITA-Group/GraNet)                                            |
| [AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks](https://papers.nips.cc/paper/2021/hash/48000647b315f6f00f913caa757a70b3-Abstract.html)                                                                                                   | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/IST-DASLab/ACDC)                                              |
| [A Winning Hand: Compressing Deep Networks Can Improve Out-of-Distribution Robustness](https://papers.nips.cc/paper/2021/hash/0607f4c705595b911a4f3e7a127b44e0-Abstract.html)                                                                                          | NeurIPS | `W`     | [PyTorch(Author)](https://github.com/RobustBench/robustbench)                                      |
| [Rethinking the Pruning Criteria for Convolutional Neural Network](https://papers.nips.cc/paper/2021/hash/87ae6fb631f7c8a627e8e28785d9992d-Abstract.html)                                                                                                              | NeurIPS | `F`     | -                                                                                                  |
| [Only Train Once: A One-Shot Neural Network Training And Pruning Framework](https://papers.nips.cc/paper/2021/hash/a376033f78e144f494bfc743c0be3330-Abstract.html)                                                                                                     | NeurIPS | `F`     | [PyTorch(Author)](https://github.com/tianyic/onlytrainonce)                                        |
| [CHIP: CHannel Independence-based Pruning for Compact Neural Networks](https://papers.nips.cc/paper/2021/hash/ce6babd060aa46c61a5777902cca78af-Abstract.html)                                                                                                          | NeurIPS | `F`     | [PyTorch(Author)](https://github.com/Eclipsess/CHIP_NeurIPS2021)                                   |
| [RED : Looking for Redundancies for Data-FreeStructured Compression of Deep Neural Networks](https://papers.nips.cc/paper/2021/hash/ae5e3ce40e0404a45ecacaaf05e5f735-Abstract.html)                                                                                    | NeurIPS | `F`     | -                                                                                                  |
| [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://papers.nips.cc/paper/2021/hash/2adcfc3929e7c03fac3100d3ad51da26-Abstract.html)                                                                                         | NeurIPS | `F`     | [PyTorch(Author)](https://github.com/lucaslie/torchprune)                                          |
| [Sparse Flows: Pruning Continuous-depth Models](https://papers.nips.cc/paper/2021/hash/bf1b2f4b901c21a1d8645018ea9aeb05-Abstract.html)                                                                                                                                 | NeurIPS | `WF`    | [PyTorch(Author)](https://github.com/lucaslie/torchprune)                                          |
| [Scaling Up Exact Neural Network Compression by ReLU Stability](https://papers.nips.cc/paper/2021/hash/e35d7a5768c4b85b4780384d55dc3620-Abstract.html)                                                                                                                 | NeurIPS | `S`     | [PyTorch(Author)](https://github.com/yuxwind/ExactCompression)                                     |
| [Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme](https://papers.nips.cc/paper/2021/hash/effc299a1addb07e7089f9b269c31f2f-Abstract.html)                                                                                    | NeurIPS | `S`     | [PyTorch(Author)](https://github.com/SJLeo/GCC)                                                    |
| [Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks](https://papers.nips.cc/paper/2021/hash/f5c3dd7514bf620a1b85450d2ae374b1-Abstract.html)                                                                                                    | NeurIPS | `Other` | [PyTorch(Author)](https://github.com/mbarsbey/sgd_comp_gen)                                        |
| [ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting](https://openaccess.thecvf.com/content/ICCV2021/html/Ding_ResRep_Lossless_CNN_Pruning_via_Decoupling_Remembering_and_Forgetting_ICCV_2021_paper.html)                                          | ICCV    | `F`     | [PyTorch(Author)](https://github.com/DingXiaoH/ResRep)                                             |
| [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://openaccess.thecvf.com/content/ICCV2021/html/Zhan_Achieving_On-Mobile_Real-Time_Super-Resolution_With_Neural_Architecture_and_Pruning_Search_ICCV_2021_paper.html) | ICCV    | `F`     | -                                                                                                  |
| [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://openaccess.thecvf.com/content/ICCV2021/html/Guo_GDP_Stabilized_Neural_Network_Pruning_via_Gates_With_Differentiable_Polarization_ICCV_2021_paper.html)                     | ICCV    | `F`     | -                                                                                                  |
| [Auto Graph Encoder-Decoder for Neural Network Pruning](https://openaccess.thecvf.com/content/ICCV2021/html/Yu_Auto_Graph_Encoder-Decoder_for_Neural_Network_Pruning_ICCV_2021_paper.html)                                                                             | ICCV    | `F`     | -                                                                                                  |
| [Exploration and Estimation for Model Compression](https://papers.nips.cc/paper/2021/hash/5227b6aaf294f5f027273aebf16015f2-Abstract.html)                                                                                                                              | ICCV    | `F`     | -                                                                                                  |
| [Sub-Bit Neural Networks: Learning To Compress and Accelerate Binary Neural Networks](https://openaccess.thecvf.com/content/ICCV2021/html/Wang_Sub-Bit_Neural_Networks_Learning_To_Compress_and_Accelerate_Binary_Neural_ICCV_2021_paper.html)                         | ICCV    | `Other` | [PyTorch(Author)](https://github.com/yikaiw/SNN)                                                   |
| [On the Predictability of Pruning Across Scales](https://arxiv.org/abs/2006.10621)                                                                                                                                                                                     | ICML    | `W`     | -                                                                                                  |
| [A Probabilistic Approach to Neural Network Pruning](https://arxiv.org/abs/2105.10065)                                                                                                                                                                                 | ICML    | `F`     | -                                                                                                  |
| [Accelerate CNNs from Three Dimensions: A Comprehensive Pruning Framework](https://arxiv.org/abs/2010.04879)                                                                                                                                                           | ICML    | `F`     | -                                                                                                  |
| [Group Fisher Pruning for Practical Network Compression](https://arxiv.org/abs/2108.00708)                                                                                                                                                                             | ICML    | `F`     | [PyTorch(Author)](https://github.com/jshilong/FisherPruning)                                       |
| [Towards Compact CNNs via Collaborative Compression](https://arxiv.org/abs/2105.11228)                                                                                                                                                                                 | CVPR    | `F`     | [PyTorch(Author)](https://github.com/liuguoyou/Towards-Compact-CNNs-via-Collaborative-Compression) |
| [Permute, Quantize, and Fine-tune: Efficient Compression of Neural Networks](https://arxiv.org/abs/2010.15703)                                                                                                                                                         | CVPR    | `F`     | [PyTorch(Author)](https://github.com/uber-research/permute-quantize-finetune)                      |
| [NPAS: A Compiler-aware Framework of Unified Network Pruning andArchitecture Search for Beyond Real-Time Mobile Acceleration](https://arxiv.org/abs/2012.00596)                                                                                                        | CVPR    | `F`     | -                                                                                                  |
| [Network Pruning via Performance Maximization](https://openaccess.thecvf.com/content/CVPR2021/html/Gao_Network_Pruning_via_Performance_Maximization_CVPR_2021_paper.html)                                                                                              | CVPR    | `F`     | -                                                                                                  |
| [Convolutional Neural Network Pruning with Structural Redundancy Reduction](https://arxiv.org/abs/2104.03438)                                                                                                                                                          | CVPR    | `F`     | -                                                                                                  |
| [Manifold Regularized Dynamic Network Pruning](https://arxiv.org/abs/2103.05861)                                                                                                                                                                                       | CVPR    | `F`     | -                                                                                                  |
| [Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation](https://arxiv.org/abs/2105.12971)                                                                                                                                                     | CVPR    | `FO`    | -                                                                                                  |
| [Content-Aware GAN Compression](https://arxiv.org/abs/2104.02244)                                                                                                                                                                                                      | CVPR    | `S`     | [PyTorch(Author)](https://github.com/lychenyoko/content-aware-gan-compression)                     |
| [Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Network](https://openreview.net/forum?id=U_mat0b9iv)                                                                                                    | ICLR    | `W`     | [PyTorch(Author)](https://github.com/chrundle/biprop)                                              |
| [Layer-adaptive Sparsity for the Magnitude-based Pruning](https://openreview.net/forum?id=H6ATjJ0TKdf)                                                                                                                                                                 | ICLR    | `W`     | [PyTorch(Author)](https://github.com/jaeho-lee/layer-adaptive-sparsity)                            |
| [Pruning Neural Networks at Initialization: Why Are We Missing the Mark?](https://openreview.net/forum?id=Ig-VyQc-MLK)                                                                                                                                                 | ICLR    | `W`     | -                                                                                                  |
| [Robust Pruning at Initialization](https://openreview.net/forum?id=vXj_ucZQ4hA)                                                                                                                                                                                        | ICLR    | `W`     | -                                                                                                  |
| [A Gradient Flow Framework For Analyzing Network Pruning](https://openreview.net/forum?id=rumv7QmLUue)                                                                                                                                                                 | ICLR    | `F`     | [PyTorch(Author)](https://github.com/EkdeepSLubana/flowandprune)                                   |
| [Neural Pruning via Growing Regularization](https://openreview.net/forum?id=o966_Is_nPA)                                                                                                                                                                               | ICLR    | `F`     | [PyTorch(Author)](https://github.com/MingSun-Tse/Regularization-Pruning)                           |
| [ChipNet: Budget-Aware Pruning with Heaviside Continuous Approximations](https://openreview.net/forum?id=xCxXwTzx4L1)                                                                                                                                                  | ICLR    | `F`     | [PyTorch(Author)](https://github.com/transmuteAI/ChipNet)                                          |
| [Network Pruning That Matters: A Case Study on Retraining Variants](https://openreview.net/forum?id=Cb54AMqHQFP)                                                                                                                                                       | ICLR    | `F`     | [PyTorch(Author)](https://github.com/lehduong/NPTM)                                                |

### 2020

### 2019

| Title    | Venue       | Type    | Code     |
|:-------|:--------:|:-------:|:-------:|
| [Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask](https://arxiv.org/abs/1905.01067)                                                                                                              | NeurIPS         | `W`     | [TensorFlow(Author)](https://github.com/uber-research/deconstructing-lottery-tickets) |
| [One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers](https://arxiv.org/abs/1906.02773)                                                                       | NeurIPS         | `W`     | -                                                                                     |
| [Global Sparse Momentum SGD for Pruning Very Deep Neural Networks](https://arxiv.org/abs/1909.12778)                                                                                                             | NeurIPS         | `W`     | [PyTorch(Author)](https://github.com/DingXiaoH/GSM-SGD)                               |
| [AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters](https://papers.nips.cc/paper/9521-autoprune-automatic-network-pruning-by-regularizing-auxiliary-parameters)                          | NeurIPS         | `W`     | -                                                                                     |
| [Network Pruning via Transformable Architecture Search](https://arxiv.org/abs/1905.09717)                                                                                                                        | NeurIPS         | `F`     | [PyTorch(Author)](https://github.com/D-X-Y/NAS-Projects)                              |
| [Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks](https://arxiv.org/abs/1909.08174)                                                                             | NeurIPS         | `F`     | [PyTorch(Author)](https://github.com/youzhonghui/gate-decorator-pruning)              |
| [Model Compression with Adversarial Robustness: A Unified Optimization Framework](https://arxiv.org/abs/1902.03538)                                                                                              | NeurIPS         | `Other` | [PyTorch(Author)](https://github.com/TAMU-VITA/ATMC)                                  |
| [Adversarial Robustness vs Model Compression, or Both?](https://arxiv.org/abs/1903.12561)                                                                                                                        | ICCV            | `W`     | [PyTorch(Author)](https://github.com/yeshaokai/Robustness-Aware-Pruning-ADMM)         |
| [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://arxiv.org/abs/1903.10258)                                                                                                      | ICCV            | `F`     | [PyTorch(Author)](https://github.com/liuzechun/MetaPruning)                           |
| [Accelerate CNN via Recursive Bayesian Pruning](https://arxiv.org/abs/1812.00353)                                                                                                                                | ICCV            | `F`     | -                                                                                     |
| [Learning Filter Basis for Convolutional Neural Network Compression](https://arxiv.org/abs/1908.08932)                                                                                                           | ICCV            | `Other` | -                                                                                     |
| [Co-Evolutionary Compression for Unpaired Image Translation](https://arxiv.org/abs/1907.10804)                                                                                                                   | ICCV            | `S`     | -                                                                                     |
| [COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning](https://arxiv.org/abs/1906.10337)                                                                                | IJCAI           | `F`     | [Tensorflow(Author)](https://github.com/ZJULearning/COP)                              |
| [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://arxiv.org/abs/1811.00250)                                                                                      | CVPR **(Oral)** | `F`     | [PyTorch(Author)](https://github.com/he-y/filter-pruning-geometric-median)            |
| [Towards Optimal Structured CNN Pruning via Generative Adversarial Learning](https://arxiv.org/abs/1903.09291)                                                                                                   | CVPR            | `F`     | [PyTorch(Author)](https://github.com/ShaohuiLin/GAL)                                  |
| [Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure](https://arxiv.org/abs/1904.03837)                                                                                      | CVPR            | `F`     | [PyTorch(Author)](https://github.com/ShawnDing1994/Centripetal-SGD)                   |
| [On Implicit Filter Level Sparsity in Convolutional Neural Networks](https://arxiv.org/abs/1811.12495), [Extension1](https://arxiv.org/abs/1905.04967), [Extension2](https://openreview.net/forum?id=rylVvNS3hE) | CVPR            | `F`     | [PyTorch(Author)](https://github.com/mehtadushy/SelecSLS-Pytorch)                     |
| [Structured Pruning of Neural Networks with Budget-Aware Regularization](https://arxiv.org/abs/1811.09332)                                                                                                       | CVPR            | `F`     | -                                                                                     |
| [Importance Estimation for Neural Network Pruning](http://jankautz.com/publications/Importance4NNPruning_CVPR19.pdf)                                                                                             | CVPR            | `F`     | [PyTorch(Author)](https://github.com/NVlabs/Taylor_pruning)                           |
| [OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks](https://arxiv.org/abs/1905.11664)                                                                                               | CVPR            | `F`     | -                                                                                     |
| [Variational Convolutional Neural Network Pruning](https://openaccess.thecvf.com/content_CVPR_2019/html/Zhao_Variational_Convolutional_Neural_Network_Pruning_CVPR_2019_paper.html)                              | CVPR            | `F`     | -                                                                                     |
| [Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search](https://arxiv.org/abs/1903.03777)                                                                                       | CVPR            | `Other` | [TensorFlow(Author)](https://github.com/lixincn2015/Partial-Order-Pruning)            |
| [Collaborative Channel Pruning for Deep Networks](http://proceedings.mlr.press/v97/peng19c.html)                                                                                                                 | ICML            | `F`     | -                                                                                     |
| [Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github](https://arxiv.org/abs/1905.04748)                                                                                             | ICML            | `F`     | -                                                                                     |
| [EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis](https://arxiv.org/abs/1905.05934)                                                                                                         | ICML            | `F`     | [PyTorch(Author)](https://github.com/alecwangcq/EigenDamage-Pytorch)                  |
| [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://arxiv.org/abs/1803.03635)                                                                                                     | ICLR **(Best)** | `W`     | [TensorFlow(Author)](https://github.com/google-research/lottery-ticket-hypothesis)    |
| [SNIP: Single-shot Network Pruning based on Connection Sensitivity](https://arxiv.org/abs/1810.02340)                                                                                                            | ICLR            | `W`     | [TensorFLow(Author)](https://github.com/namhoonlee/snip-public)                       |
| [Dynamic Channel Pruning: Feature Boosting and Suppression](https://arxiv.org/abs/1810.05331)                                                                                                                    | ICLR            | `F`     | [TensorFlow(Author)](https://github.com/deep-fry/mayo)                                |
| [Rethinking the Value of Network Pruning](https://arxiv.org/abs/1810.05270)                                                                                                                                      | ICLR            | `F`     | [PyTorch(Author)](https://github.com/Eric-mingjie/rethinking-network-pruning)         |
| [Dynamic Sparse Graph for Efficient Deep Learning](https://arxiv.org/abs/1810.00859)                                                                                                                             | ICLR            | `F`     | [CUDA(3rd)](https://github.com/mtcrawshaw/dynamic-sparse-graph)                       |

### 2018
| Title                                                                                                                                                                               | Venue   | Type    | Code                                                                                                                                 |
|:-------|:--------:|:-------:|:-------:|
| [Frequency-Domain Dynamic Pruning for Convolutional Neural Networks](https://papers.NeurIPS.cc/paper/7382-frequency-domain-dynamic-pruning-for-convolutional-neural-networks.pdf)   | NeurIPS | `W`     | -                                                                                                                                    |
| [Discrimination-aware Channel Pruning for Deep Neural Networks](https://arxiv.org/abs/1810.11809)                                                                                   | NeurIPS | `F`     | [TensorFlow(Author)](https://github.com/SCUT-AILab/DCP)                                                                              |
| [Learning Sparse Neural Networks via Sensitivity-Driven Regularization](https://arxiv.org/pdf/1810.11764.pdf)                                                                       | NeurIPS | `WF`    | -                                                                                                                                    |
| [Constraint-Aware Deep Neural Network Compression](https://openaccess.thecvf.com/content_ECCV_2018/html/Changan_Chen_Constraints_Matter_in_ECCV_2018_paper.html)                    | ECCV    | `W`     | [SkimCaffe(Author)](https://github.com/ChanganVR/ConstraintAwareCompression)                                                         |
| [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://arxiv.org/abs/1804.03294)                                                     | ECCV    | `W`     | [Caffe(Author)](https://github.com/KaiqiZhang/admm-pruning)                                                                          |
| [Amc: Automl for model compression and acceleration on mobile devices](https://arxiv.org/abs/1802.03494)                                                                            | ECCV    | `F`     | [TensorFlow(3rd)](https://github.com/Tencent/PocketFlow#channel-pruning)                                                             |
| [Data-Driven Sparse Structure Selection for Deep Neural Networks](https://arxiv.org/abs/1707.01213)                                                                                 | ECCV    | `F`     | [MXNet(Author)](https://github.com/TuSimple/sparse-structure-selection)                                                              |
| [Coreset-Based Neural Network Compression](https://arxiv.org/abs/1807.09810)                                                                                                        | ECCV    | `F`     | [PyTorch(Author)](https://github.com/metro-smiles/CNN_Compression)                                                                   |
| [Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks](https://arxiv.org/abs/1808.06866)                                                                         | IJCAI   | `F`     | [PyTorch(Author)](https://github.com/he-y/soft-filter-pruning)                                                                       |
| [Accelerating Convolutional Networks via Global & Dynamic Filter Pruning](https://www.ijcai.org/proceedings/2018/0336.pdf)                                                          | IJCAI   | `F`     | -                                                                                                                                    |
| [Weightless: Lossy weight encoding for deep neural network compression](https://proceedings.mlr.press/v80/reagan18a.html)                                                           | ICML    | `W`     | -                                                                                                                                    |
| [Compressing Neural Networks using the Variational Information Bottleneck](https://proceedings.mlr.press/v80/dai18d.html)                                                           | ICML    | `F`     | [PyTorch(Author)](https://github.com/zhuchen03/VIBNet)                                                                               |
| [Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions](https://proceedings.mlr.press/v80/wu18h.html)                   | ICML    | `Other` | [PyTorch(Author)](https://github.com/VITA-Group/Deep-K-Means-pytorch)                                                                |
| [CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization](https://openaccess.thecvf.com/content_cvpr_2018/html/Tung_CLIP-Q_Deep_Network_CVPR_2018_paper.html) | CVPR    | `W`     | -                                                                                                                                    |
| [“Learning-Compression” Algorithms for Neural Net Pruning](http://faculty.ucmerced.edu/mcarreira-perpinan/papers/cvpr18.pdf)                                                        | CVPR    | `W`     | -                                                                                                                                    |
| [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://arxiv.org/abs/1711.05769)                                                                         | CVPR    | `F`     | [PyTorch(Author)](https://github.com/arunmallya/packnet)                                                                             |
| [NISP: Pruning Networks using Neuron Importance Score Propagation](https://arxiv.org/abs/1711.05908)                                                                                | CVPR    | `F`     | -                                                                                                                                    |
| [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://arxiv.org/abs/1710.01878)                                                              | ICLR    | `W`     | -                                                                                                                                    |
| [Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers](https://arxiv.org/abs/1802.00124)                                                | ICLR    | `F`     | [TensorFlow(Author)](https://github.com/bobye/batchnorm_prune), [PyTorch(3rd)](https://github.com/jack-willturner/batchnorm-pruning) |

### 2017
| Title                                                                                                                                                    | Venue   | Type | Code                                                                                                                  |
|:-------|:--------:|:-------:|:-------:|
| [Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee](https://arxiv.org/abs/1611.05162)                                          | NeurIPS | `W`  | [TensorFlow(Author)](https://github.com/DNNToolBox/Net-Trim-v1)                                                       |
| [Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon](https://arxiv.org/abs/1705.07565)                                          | NeurIPS | `W`  | [PyTorch(Author)](https://github.com/csyhhu/L-OBS)                                                                    |
| [Runtime Neural Pruning](https://papers.NeurIPS.cc/paper/6813-runtime-neural-pruning)                                                                    | NeurIPS | `F`  | -                                                                                                                     |
| [Structured Bayesian Pruning via Log-Normal Multiplicative Noise](https://papers.nips.cc/paper/2017/hash/dab49080d80c724aad5ebf158d63df41-Abstract.html) | NeurIPS | `F`  | -                                                                                                                     |
| [Bayesian Compression for Deep Learning](https://proceedings.neurips.cc/paper/2017/hash/69d1fc78dbda242c43ad6590368912d4-Abstract.html)                  | NeurIPS | `F`  | -                                                                                                                     |
| [ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression](https://arxiv.org/abs/1707.06342)                                            | ICCV    | `F`  | [Caffe(Author)](https://github.com/Roll920/ThiNet), [PyTorch(3rd)](https://github.com/tranorrepository/reprod-thinet) |
| [Channel pruning for accelerating very deep neural networks](https://arxiv.org/abs/1707.06168)                                                           | ICCV    | `F`  | [Caffe(Author)](https://github.com/yihui-he/channel-pruning)                                                          |
| [Learning Efficient Convolutional Networks Through Network Slimming](https://arxiv.org/abs/1708.06519)                                                   | ICCV    | `F`  | [PyTorch(Author)](https://github.com/Eric-mingjie/network-slimming)                                                   |
| [Variational Dropout Sparsifies Deep Neural Networks](http://arxiv.org/abs/1701.05369)                                                                   | ICML    | `W`  | -                                                                                                                     |
| [Combined Group and Exclusive Sparsity for Deep Neural Networks](https://proceedings.mlr.press/v70/yoon17a.html)                                         | ICML    | `WF` | -                                                                                                                     |
| [Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning](https://arxiv.org/abs/1611.05128)                                  | CVPR    | `W`  | -                                                                                                                     |
| [Pruning Filters for Efficient ConvNets](https://arxiv.org/abs/1608.08710)                                                                               | ICLR    | `F`  | [PyTorch(3rd)](https://github.com/Eric-mingjie/rethinking-network-pruning/tree/master/imagenet/l1-norm-pruning)       |
| [Pruning Convolutional Neural Networks for Resource Efficient Inference](https://arxiv.org/abs/1611.06440)                                               | ICLR    | `F`  | [TensorFlow(3rd)](https://github.com/Tencent/PocketFlow#channel-pruning)                                              |

### 2016
| Title                                                                                                                                            | Venue           | Type | Code                                                                 |
|:-------|:--------:|:-------:|:-------:|
| [Dynamic Network Surgery for Efficient DNNs](https://arxiv.org/abs/1608.04493)                                                                   | NeurIPS         | `W`  | [Caffe(Author)](https://github.com/yiwenguo/Dynamic-Network-Surgery) |
| [Learning the Number of Neurons in Deep Networks](https://proceedings.neurips.cc/paper/2016/hash/6e7d2da6d3953058db75714ac400b584-Abstract.html) | NeurIPS         | `F`  | -                                                                    |
| [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://arxiv.org/abs/1510.00149)     | ICLR **(Best)** | `W`  | [Caffe(Author)](https://github.com/songhan/Deep-Compression-AlexNet) |

### 2015

| Title    | Venue       | Type    | Code     |
|:-------|:--------:|:-------:|:-------:|
| [Learning both Weights and Connections for Efficient Neural Networks](https://arxiv.org/abs/1506.02626) | NeurIPS | `W`  | [PyTorch(3rd)](https://github.com/jack-willturner/DeepCompression-PyTorch) |

## Related Repo

[Awesome-model-compression-and-acceleration](https://github.com/memoiry/Awesome-model-compression-and-acceleration)

[EfficientDNNs](https://github.com/MingSun-Tse/EfficientDNNs)

[Embedded-Neural-Network](https://github.com/ZhishengWang/Embedded-Neural-Network)

[awesome-AutoML-and-Lightweight-Models](https://github.com/guan-yuan/awesome-AutoML-and-Lightweight-Models)

[Model-Compression-Papers](https://github.com/chester256/Model-Compression-Papers)

[knowledge-distillation-papers](https://github.com/lhyfst/knowledge-distillation-papers)

[Network-Speed-and-Compression](https://github.com/mrgloom/Network-Speed-and-Compression)

Awesome-Pruning

Explore Similar Repositories

Was this content helpful?

Recent Feedback

Download README