Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Last update: Jun 28, 2022

Related tags

Deep Learning lottery-jackpots

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

Python >= 3.7.4
Pytorch >= 1.6.1
Torchvision >= 0.4.1

Reproduce the Experiment Results

Download the pre-trained models from this link and place them in the pre-train folder.
Select a configuration file in configs to reproduce the experiment results reported in the paper. For example, to find a lottery jackpot with 30 epochs for pruning 95% parameters of ResNet-32 on CIFAR-10, run:

python cifar.py --config configs/resnet32_cifar10/90sparsity30epoch.yaml --gpus 0

To find a lottery jackpot with 30 epochs for pruning 90% parameters of ResNet-50 on ImageNet, run:

python imagenet.py --config configs/resnet50_imagenet/90sparsity30epoch.yaml --gpus 0

Note that the data_path in the yaml file should be changed to the data

Evaluate Our Pruned Models

We provide configuration, training logs, and pruned models reported in the paper. They can be downloaded from the provided links in the following table:

Model	Dataset	Sparsity	Epoch	Top-1 Acc.	Link
VGGNet-19	CIFAR-10	90%	30	93.88%	link
VGGNet-19	CIFAR-10	90%	160	93.94%	link
VGGNet-19	CIFAR-10	95%	30	93.49%	link
VGGNet-19	CIFAR-10	95%	160	93.74%	link
VGGNet-19	CIFAR-100	90%	30	72.59%	link
VGGNet-19	CIFAR-100	90%	160	74.61%	link
VGGNet-19	CIFAR-100	95%	30	71.76%	link
VGGNet-19	CIFAR-100	95%	160	73.35%	link
ResNet-32	CIFAR-10	90%	30	93.70%	link
ResNet-32	CIFAR-10	90%	160	94.39%	link
ResNet-32	CIFAR-10	95%	30	92.90%	link
ResNet-32	CIFAR-10	95%	160	93.41%	link
ResNet-32	CIFAR-100	90%	30	72.22%	link
ResNet-32	CIFAR-100	90%	160	73.43%	link
ResNet-32	CIFAR-100	95%	30	69.38%	link
ResNet-32	CIFAR-100	95%	160	70.31%	link
ResNet-50	ImageNet	80%	30	74.53%	link
ResNet-50	ImageNet	80%	60	75.26%	link
ResNet-50	ImageNet	90%	30	72.17%	link
ResNet-50	ImageNet	90%	60	72.46%	link

To test the our pruned models, download the pruned models and place them in the ckpt folder.

Select a configuration file in configs to test the pruned models. For example, to evaluate a lottery jackpot for pruning ResNet-32 on CIFAR-10, run:

python evaluate.py --config configs/resnet32_cifar10/evaluate.yaml --gpus 0

To evaluate a lottery jackpot for pruning ResNet-50 on ImageNet, run:

python evaluate.py --config configs/resnet50_imagenet/evaluate.yaml --gpus 0

Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Related tags

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

Reproduce the Experiment Results

Evaluate Our Pruned Models

Owner

Yuxin Zhang

Open-Set Recognition: A Good Closed-Set Classifier is All You Need

Defending graph neural networks against adversarial attacks (NeurIPS 2020)

Release of SPLASH: Dataset for semantic parse correction with natural language feedback in the context of text-to-SQL parsing

Reproduces the results of the paper "Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equations".

Human annotated noisy labels for CIFAR-10 and CIFAR-100.

A Pytorch implementation of "LegoNet: Efficient Convolutional Neural Networks with Lego Filters" (ICML 2019).

Training DiffWave using variational method from Variational Diffusion Models.

This repository contains numerical implementation for the paper Intertemporal Pricing under Reference Effects: Integrating Reference Effects and Consumer Heterogeneity.

Implementation of GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation (ICLR 2022).

The trained model and denoising example for paper : Cardiopulmonary Auscultation Enhancement with a Two-Stage Noise Cancellation Approach

Adversarial vulnerability of powerful near out-of-distribution detection

CVPR 2021 Official Pytorch Code for UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

ML for NLP and Computer Vision.

This repository is a series of notebooks that show solutions for the projects at Dataquest.io.

[ICCV 2021] Deep Hough Voting for Robust Global Registration

Self-supervised learning optimally robust representations for domain generalization.

Post-training Quantization for Neural Networks with Provable Guarantees

General Virtual Sketching Framework for Vector Line Art (SIGGRAPH 2021)

Sign Language Transformers (CVPR'20)

This repo contains implementation of different architectures for emotion recognition in conversations.