Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search
Pytorch implementation for "Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search".
Requirements
python>=3.7, torch==1.5.0, torchvision==0.6.0, graphviz
Please install all the requirements in requirements.txt.
Datasets
We consider two benchmark classification datsets, including CIFAR-10 and ImageNet.
CIFAR-10 can be automatically downloaded by torchvision.
ImageNet needs to be manually downloaded (preferably to a SSD) following the instructions here.
Training Method
Curriculum search on CIFAR-10
python search.py -o outputs/search
Evaluation Method
- Put the searched architectures in
cnas/model/genotypes.pyas follows.
CNAS = Genotype.from_arch(
normal_arch=[('dil_conv_3x3', 1, 2),
('sep_conv_3x3', 0, 2),
('sep_conv_3x3', 0, 3),
('skip_connect', 1, 3),
('sep_conv_3x3', 1, 4),
('max_pool_3x3', 3, 4),
('sep_conv_3x3', 2, 5),
('sep_conv_3x3', 4, 5)],
normal_concat=[2, 3, 4, 5],
reduced_arch=[('sep_conv_3x3', 0, 2),
('skip_connect', 1, 2),
('dil_conv_5x5', 2, 3),
('skip_connect', 1, 3),
('dil_conv_3x3', 2, 4),
('sep_conv_3x3', 1, 4),
('sep_conv_5x5', 0, 5),
('sep_conv_3x3', 3, 5)],
reduced_concat=[2, 3, 4, 5])
- Evaluate the searched architecture on CIFAR-10 and ImageNet dataset using the following scripts.
Evaluation on CIFAR-10:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch \
--nproc_per_node=2 --master_port=23333 \
eval_arch.py \
--arch CNAS --init_channels 36 --layers 20 \
-o outputs/cifar10
Evaluation on ImageNet:
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch \
--nproc_per_node=4 --master_port=22333 \
eval_arch.py \
--max_epochs 250 --scheduler linear \
--dataset imagenet --data /path/to/imagenet \
--batch_size 64 --no_bias_decay --num_workers 8 \
--arch CNAS --init_channels 48 --layers 14 \
-o outputs/imagenet
Pretrained models
We have released our CNAS ImageNet pretrained model (top-1 accuracy 75.4%, top-5 accuracy 92.6%) on here.
You can use the following codes to load the ptrained models:
from cnas.model.eval import cnas_imagenet
model = cnas_imagenet(pretrained=True)
Citation
If you use any part of our code in your research, please cite our paper:
@InProceedings{guo2020breaking,
title = {Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search},
author = {Guo, Yong and Chen, Yaofo and Zheng, Yin and Zhao, Peilin and Chen, Jian and Huang, Junzhou and Tan, Mingkui},
booktitle = {Proceedings of the 37th International Conference on Machine Learning},
year = {2020},
pages = {3822--3831}
}
3 Jan 7, 2022
79 Dec 19, 2022
21 Dec 13, 2022
14 Sep 30, 2022
232 Dec 25, 2022
57 Dec 15, 2022
21 Dec 27, 2022
41 Dec 12, 2022
43 Oct 22, 2022
1 Jan 13, 2022
2.4k Jan 04, 2023
1 Dec 31, 2021
191 Jan 07, 2023
3 Apr 08, 2022
172 Dec 13, 2022
16 Nov 18, 2022
27 Aug 08, 2022
24 Jul 24, 2022
6 Dec 14, 2022
0 Jan 20, 2022
2 Nov 14, 2022
34 Nov 09, 2022
37 Dec 18, 2022
52 Jan 04, 2023
2 Oct 04, 2021
81 Dec 22, 2022
51 Dec 10, 2022
4 Dec 03, 2021