Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Last update: Jun 28, 2022

Related tags

Deep Learning lottery-jackpots

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

Python >= 3.7.4
Pytorch >= 1.6.1
Torchvision >= 0.4.1

Reproduce the Experiment Results

Download the pre-trained models from this link and place them in the pre-train folder.
Select a configuration file in configs to reproduce the experiment results reported in the paper. For example, to find a lottery jackpot with 30 epochs for pruning 95% parameters of ResNet-32 on CIFAR-10, run:

python cifar.py --config configs/resnet32_cifar10/90sparsity30epoch.yaml --gpus 0

To find a lottery jackpot with 30 epochs for pruning 90% parameters of ResNet-50 on ImageNet, run:

python imagenet.py --config configs/resnet50_imagenet/90sparsity30epoch.yaml --gpus 0

Note that the data_path in the yaml file should be changed to the data

Evaluate Our Pruned Models

We provide configuration, training logs, and pruned models reported in the paper. They can be downloaded from the provided links in the following table:

Model	Dataset	Sparsity	Epoch	Top-1 Acc.	Link
VGGNet-19	CIFAR-10	90%	30	93.88%	link
VGGNet-19	CIFAR-10	90%	160	93.94%	link
VGGNet-19	CIFAR-10	95%	30	93.49%	link
VGGNet-19	CIFAR-10	95%	160	93.74%	link
VGGNet-19	CIFAR-100	90%	30	72.59%	link
VGGNet-19	CIFAR-100	90%	160	74.61%	link
VGGNet-19	CIFAR-100	95%	30	71.76%	link
VGGNet-19	CIFAR-100	95%	160	73.35%	link
ResNet-32	CIFAR-10	90%	30	93.70%	link
ResNet-32	CIFAR-10	90%	160	94.39%	link
ResNet-32	CIFAR-10	95%	30	92.90%	link
ResNet-32	CIFAR-10	95%	160	93.41%	link
ResNet-32	CIFAR-100	90%	30	72.22%	link
ResNet-32	CIFAR-100	90%	160	73.43%	link
ResNet-32	CIFAR-100	95%	30	69.38%	link
ResNet-32	CIFAR-100	95%	160	70.31%	link
ResNet-50	ImageNet	80%	30	74.53%	link
ResNet-50	ImageNet	80%	60	75.26%	link
ResNet-50	ImageNet	90%	30	72.17%	link
ResNet-50	ImageNet	90%	60	72.46%	link

To test the our pruned models, download the pruned models and place them in the ckpt folder.

Select a configuration file in configs to test the pruned models. For example, to evaluate a lottery jackpot for pruning ResNet-32 on CIFAR-10, run:

python evaluate.py --config configs/resnet32_cifar10/evaluate.yaml --gpus 0

To evaluate a lottery jackpot for pruning ResNet-50 on ImageNet, run:

python evaluate.py --config configs/resnet50_imagenet/evaluate.yaml --gpus 0

Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Related tags

Overview

Lottery Jackpots Exist in Pre-trained Models (Paper Link)

Requirements

Reproduce the Experiment Results

Evaluate Our Pruned Models

Owner

Yuxin Zhang

The Face Mask recognition system uses AI technology to detect the person with or without a mask.

This is a model made out of Neural Network specifically a Convolutional Neural Network model

We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

A set of tools for converting a darknet dataset to COCO format working with YOLOX

Source code for our EMNLP'21 paper 《Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning》

Home for cuQuantum Python & NVIDIA cuQuantum SDK C++ samples

Deep Learning Emotion decoding using EEG data from Autism individuals

Autonomous Perception: 3D Object Detection with Complex-YOLO

level1-image-classification-level1-recsys-09 created by GitHub Classroom

Optimising chemical reactions using machine learning

Neural networks applied in recognizing guitar chords using python, AutoML.NET with C# and .NET Core

Official repository for the NeurIPS 2021 paper Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided curriculum Learning Approach

B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search

A modular domain adaptation library written in PyTorch.

PyTorch implementation for OCT-GAN Neural ODE-based Conditional Tabular GANs (WWW 2021)

Multi-Objective Loss Balancing for Physics-Informed Deep Learning

A deep neural networks for images using CNN algorithm.

Code for EMNLP'21 paper "Types of Out-of-Distribution Texts and How to Detect Them"

Pre-trained model, code, and materials from the paper "Impact of Adversarial Examples on Deep Learning Models for Biomedical Image Segmentation" (MICCAI 2019).

Pytorch reimplementation of the Vision Transformer (An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale)