A curated list of neural network pruning resources.

Related tags

Deep Learningawesome-listpruningmodel-compressionmodel-acceleration
Overview

Awesome Pruning Awesome

A curated list of neural network pruning and related resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers and Awesome-NAS.

Please feel free to pull requests or open an issue to add papers.

Table of Contents

Type of Pruning

Type F W Other
Explanation Filter pruning Weight pruning other types

2020

Title Venue Type Code
HYDRA: Pruning Adversarially Robust Neural Networks NeurIPS W PyTorch(Author)
Logarithmic Pruning is All You Need NeurIPS W -
Directional Pruning of Deep Neural Networks NeurIPS W -
Movement Pruning: Adaptive Sparsity by Fine-Tuning NeurIPS W PyTorch(Author)
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot NeurIPS W PyTorch(Author)
Neuron Merging: Compensating for Pruned Neurons NeurIPS F PyTorch(Author)
Neuron-level Structured Pruning using Polarization Regularizer NeurIPS F PyTorch(Author)
SCOP: Scientific Control for Reliable Neural Network Pruning NeurIPS F -
Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning NeurIPS F -
The Generalization-Stability Tradeoff In Neural Network Pruning NeurIPS F PyTorch(Author)
Pruning Filter in Filter NeurIPS Other PyTorch(Author)
Position-based Scaled Gradient for Model Quantization and Pruning NeurIPS Other PyTorch(Author)
Bayesian Bits: Unifying Quantization and Pruning NeurIPS Other -
Pruning neural networks without any data by iteratively conserving synaptic flow NeurIPS Other PyTorch(Author)
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning ECCV (Oral) F PyTorch(Author)
DSA: More Efficient Budgeted Pruning via Differentiable Sparsity Allocation ECCV F -
DHP: Differentiable Meta Pruning via HyperNetworks ECCV F PyTorch(Author)
Meta-Learning with Network Pruning ECCV W -
Accelerating CNN Training by Pruning Activation Gradients ECCV W -
DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search ECCV Other -
Differentiable Joint Pruning and Quantization for Hardware Efficiency ECCV Other -
Channel Pruning via Automatic Structure Search IJCAI F PyTorch(Author)
Adversarial Neural Pruning with Latent Vulnerability Suppression ICML W -
Proving the Lottery Ticket Hypothesis: Pruning is All You Need ICML W -
Soft Threshold Weight Reparameterization for Learnable Sparsity ICML WF Pytorch(Author)
Network Pruning by Greedy Subnetwork Selection ICML F -
Operation-Aware Soft Channel Pruning using Differentiable Masks ICML F -
DropNet: Reducing Neural Network Complexity via Iterative Pruning ICML F -
Towards Efficient Model Compression via Learned Global Ranking CVPR (Oral) F Pytorch(Author)
HRank: Filter Pruning using High-Rank Feature Map CVPR (Oral) F Pytorch(Author)
Neural Network Pruning with Residual-Connections and Limited-Data CVPR (Oral) F -
Multi-Dimensional Pruning: A Unified Framework for Model Compression CVPR (Oral) WF -
DMCP: Differentiable Markov Channel Pruning for Neural Networks CVPR (Oral) F TensorFlow(Author)
Group Sparsity: The Hinge Between Filter Pruning and Decomposition for Network Compression CVPR F PyTorch(Author)
Few Sample Knowledge Distillation for Efficient Network Compression CVPR F -
Discrete Model Compression With Resource Constraint for Deep Neural Networks CVPR F -
Structured Compression by Weight Encryption for Unstructured Pruning and Quantization CVPR W -
Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration CVPR F -
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy CVPR F -
Comparing Rewinding and Fine-tuning in Neural Network Pruning ICLR (Oral) WF TensorFlow(Author)
A Signal Propagation Perspective for Pruning Neural Networks at Initialization ICLR (Spotlight) W -
ProxSGD: Training Structured Neural Networks under Regularization and Constraints ICLR W TF+PT(Author)
One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation ICLR W -
Lookahead: A Far-sighted Alternative of Magnitude-based Pruning ICLR W PyTorch(Author)
Dynamic Model Pruning with Feedback ICLR WF -
Provable Filter Pruning for Efficient Neural Networks ICLR F -
Data-Independent Neural Pruning via Coresets ICLR W -
AutoCompress: An Automatic DNN Structured Pruning Framework for Ultra-High Compression Rates AAAI F -
DARB: A Density-Aware Regular-Block Pruning for Deep Neural Networks AAAI Other -
Pruning from Scratch AAAI Other -

2019

Title Venue Type Code
Network Pruning via Transformable Architecture Search NeurIPS F PyTorch(Author)
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks NeurIPS F PyTorch(Author)
Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask NeurIPS W TensorFlow(Author)
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers NeurIPS W -
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks NeurIPS W PyTorch(Author)
AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters NeurIPS W -
Model Compression with Adversarial Robustness: A Unified Optimization Framework NeurIPS Other PyTorch(Author)
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning ICCV F PyTorch(Author)
Accelerate CNN via Recursive Bayesian Pruning ICCV F -
Adversarial Robustness vs Model Compression, or Both? ICCV W PyTorch(Author)
Learning Filter Basis for Convolutional Neural Network Compression ICCV Other -
Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration CVPR (Oral) F PyTorch(Author)
Towards Optimal Structured CNN Pruning via Generative Adversarial Learning CVPR F PyTorch(Author)
Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure CVPR F PyTorch(Author)
On Implicit Filter Level Sparsity in Convolutional Neural Networks, Extension1, Extension2 CVPR F PyTorch(Author)
Structured Pruning of Neural Networks with Budget-Aware Regularization CVPR F -
Importance Estimation for Neural Network Pruning CVPR F PyTorch(Author)
OICSR: Out-In-Channel Sparsity Regularization for Compact Deep Neural Networks CVPR F -
Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Search CVPR Other TensorFlow(Author)
Variational Convolutional Neural Network Pruning CVPR - -
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks ICLR (Best) W TensorFlow(Author)
Rethinking the Value of Network Pruning ICLR F PyTorch(Author)
Dynamic Channel Pruning: Feature Boosting and Suppression ICLR F TensorFlow(Author)
SNIP: Single-shot Network Pruning based on Connection Sensitivity ICLR W TensorFLow(Author)
Dynamic Sparse Graph for Efficient Deep Learning ICLR F CUDA(3rd)
Collaborative Channel Pruning for Deep Networks ICML F -
Approximated Oracle Filter Pruning for Destructive CNN Width Optimization github ICML F -
EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasis4 ICML W PyTorch(Author)
COP: Customized Deep Model Compression via Regularized Correlation-Based Filter-Level Pruning IJCAI F Tensorflow(Author)

2018

Title Venue Type Code
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers ICLR F TensorFlow(Author), PyTorch(3rd)
To prune, or not to prune: exploring the efficacy of pruning for model compression ICLR W -
Discrimination-aware Channel Pruning for Deep Neural Networks NeurIPS F TensorFlow(Author)
Frequency-Domain Dynamic Pruning for Convolutional Neural Networks NeurIPS W -
Learning Sparse Neural Networks via Sensitivity-Driven Regularization NeurIPS WF -
Amc: Automl for model compression and acceleration on mobile devices ECCV F TensorFlow(3rd)
Data-Driven Sparse Structure Selection for Deep Neural Networks ECCV F MXNet(Author)
Coreset-Based Neural Network Compression ECCV F PyTorch(Author)
Constraint-Aware Deep Neural Network Compression ECCV W SkimCaffe(Author)
A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers ECCV W Caffe(Author)
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning CVPR F PyTorch(Author)
NISP: Pruning Networks using Neuron Importance Score Propagation CVPR F -
CLIP-Q: Deep Network Compression Learning by In-Parallel Pruning-Quantization CVPR W -
“Learning-Compression” Algorithms for Neural Net Pruning CVPR W -
Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks IJCAI F PyTorch(Author)
Accelerating Convolutional Networks via Global & Dynamic Filter Pruning IJCAI F -

2017

Title Venue Type Code
Pruning Filters for Efficient ConvNets ICLR F PyTorch(3rd)
Pruning Convolutional Neural Networks for Resource Efficient Inference ICLR F TensorFlow(3rd)
Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee NeurIPS W TensorFlow(Author)
Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon NeurIPS W PyTorch(Author)
Runtime Neural Pruning NeurIPS F -
Designing Energy-Efficient Convolutional Neural Networks using Energy-Aware Pruning CVPR F -
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression ICCV F Caffe(Author), PyTorch(3rd)
Channel pruning for accelerating very deep neural networks ICCV F Caffe(Author)
Learning Efficient Convolutional Networks Through Network Slimming ICCV F PyTorch(Author)

2016

Title Venue Type Code
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding ICLR (Best) W Caffe(Author)
Dynamic Network Surgery for Efficient DNNs NeurIPS W Caffe(Author)

2015

Title Venue Type Code
Learning both Weights and Connections for Efficient Neural Networks NeurIPS W PyTorch(3rd)

Related Repo

Awesome-model-compression-and-acceleration

EfficientDNNs

Embedded-Neural-Network

awesome-AutoML-and-Lightweight-Models

Model-Compression-Papers

knowledge-distillation-papers

Network-Speed-and-Compression

Owner
Yang He
Ph.D. student at UTS
Yang He
GitHub Repository
RL agent to play μRTS with Stable-Baselines3

Gym-μRTS with Stable-Baselines3/PyTorch This repo contains an attempt to reproduce Gridnet PPO with invalid action masking algorithm to play μRTS usin

Oleksii Kachaiev 24 Nov 11, 2022
Noise Conditional Score Networks (NeurIPS 2019, Oral)

Generative Modeling by Estimating Gradients of the Data Distribution This repo contains the official implementation for the NeurIPS 2019 paper Generat

451 Dec 26, 2022
PyTorch implementation of "Efficient Neural Architecture Search via Parameters Sharing"

Efficient Neural Architecture Search (ENAS) in PyTorch PyTorch implementation of Efficient Neural Architecture Search via Parameters Sharing. ENAS red

Taehoon Kim 2.6k Dec 31, 2022
TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction.

TalkNet 2 [WIP] TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Predictio

Rishikesh (ऋषिकेश) 69 Dec 17, 2022
mlpack: a scalable C++ machine learning library --

a fast, flexible machine learning library Home | Documentation | Doxygen | Community | Help | IRC Chat Download: current stable version (3.4.2) mlpack

mlpack 4.2k Jan 09, 2023
PyTorch implementation of the paper: Long-tail Learning via Logit Adjustment

logit-adj-pytorch PyTorch implementation of the paper: Long-tail Learning via Logit Adjustment This code implements the paper: Long-tail Learning via

Chamuditha Jayanga 53 Dec 23, 2022
This repository is the official implementation of Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models

Using Time-Series Privileged Information for Provably Efficient Learning of Prediction Models Link to paper Abstract We study prediction of future out

Rickard Karlsson 2 Aug 19, 2022
Pytorch implementation AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

AttnGAN Pytorch implementation for reproducing AttnGAN results in the paper AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative

Tao Xu 1.2k Dec 26, 2022
Old Photo Restoration (Official PyTorch Implementation)

Bringing Old Photo Back to Life (CVPR 2020 oral)

Microsoft 11.3k Dec 30, 2022
SAS output to EXCEL converter for Cornell/MIT Language and acquisition lab

CORNELLSASLAB SAS output to EXCEL converter for Cornell/MIT Language and acquisition lab Instructions: This python code can be used to convert SAS out

2 Jan 26, 2022
Implementation of several Bayesian multi-target tracking algorithms, including Poisson multi-Bernoulli mixture filters for sets of targets and sets of trajectories. The repository also includes the GOSPA metric and a metric for sets of trajectories to evaluate performance.

This repository contains the Matlab implementations for the following multi-target filtering/tracking algorithms: - Folder PMBM contains the implemen

Ángel García-Fernández 62 Dec 12, 2022
The 2nd place solution of 2021 google landmark retrieval on kaggle.

Leaderboard, taxonomy, and curated list of few-shot object detection papers.

229 Dec 13, 2022
GT China coal model

GT China coal model The full version of a China coal transport model with a very high spatial reslution. What it does The code works in a few steps: T

0 Dec 13, 2021
(to be released) [NeurIPS'21] Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs

Higher-Order Transformers Kim J, Oh S, Hong S, Transformers Generalize DeepSets and Can be Extended to Graphs and Hypergraphs, NeurIPS 2021. [arxiv] W

Jinwoo Kim 44 Dec 28, 2022
Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation in PyTorch

StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation Implementation of StyleSpace Analysis: Disentangled Controls for StyleGAN Ima

Xuanchi Ren 86 Dec 07, 2022
A list of multi-task learning papers and projects.

This page contains a list of papers on multi-task learning for computer vision. Please create a pull request if you wish to add anything. If you are interested, consider reading our recent survey pap

svandenh 297 Dec 17, 2022
Chinese Mandarin tts text-to-speech 中文 (普通话) 语音 合成 , by fastspeech 2 , implemented in pytorch, using waveglow as vocoder,

Chinese mandarin text to speech based on Fastspeech2 and Unet This is a modification and adpation of fastspeech2 to mandrin(普通话）. Many modifications t

291 Jan 02, 2023
RealTime Emotion Recognizer for Machine Learning Study Jam's demo

Emotion recognizer Table of contents Clone project Dataset Install dependencies Main program Demo 1. Clone project git clone https://github.com/GDSC20

Google Developer Student Club - UIT 1 Oct 05, 2021
Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration

CoGAIL Table of Content Overview Installation Dataset Training Evaluation Trained Checkpoints Acknowledgement Citations License Overview This reposito

Jeremy Wang 29 Dec 24, 2022
git《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》(CVPR 2021) GitHub: [fig8]

FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding (CVPR 2021) This repo contains the implementation of our state-of-the-art fewshot ob

233 Dec 29, 2022