Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search
Pytorch implementation for "Breaking the Curse of Space Explosion: Towards Effcient NAS with Curriculum Search".
Requirements
python>=3.7, torch==1.5.0, torchvision==0.6.0, graphviz
Please install all the requirements in requirements.txt.
Datasets
We consider two benchmark classification datsets, including CIFAR-10 and ImageNet.
CIFAR-10 can be automatically downloaded by torchvision.
ImageNet needs to be manually downloaded (preferably to a SSD) following the instructions here.
Training Method
Curriculum search on CIFAR-10
python search.py -o outputs/search
Evaluation Method
- Put the searched architectures in
cnas/model/genotypes.pyas follows.
CNAS = Genotype.from_arch(
normal_arch=[('dil_conv_3x3', 1, 2),
('sep_conv_3x3', 0, 2),
('sep_conv_3x3', 0, 3),
('skip_connect', 1, 3),
('sep_conv_3x3', 1, 4),
('max_pool_3x3', 3, 4),
('sep_conv_3x3', 2, 5),
('sep_conv_3x3', 4, 5)],
normal_concat=[2, 3, 4, 5],
reduced_arch=[('sep_conv_3x3', 0, 2),
('skip_connect', 1, 2),
('dil_conv_5x5', 2, 3),
('skip_connect', 1, 3),
('dil_conv_3x3', 2, 4),
('sep_conv_3x3', 1, 4),
('sep_conv_5x5', 0, 5),
('sep_conv_3x3', 3, 5)],
reduced_concat=[2, 3, 4, 5])
- Evaluate the searched architecture on CIFAR-10 and ImageNet dataset using the following scripts.
Evaluation on CIFAR-10:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch \
--nproc_per_node=2 --master_port=23333 \
eval_arch.py \
--arch CNAS --init_channels 36 --layers 20 \
-o outputs/cifar10
Evaluation on ImageNet:
CUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch \
--nproc_per_node=4 --master_port=22333 \
eval_arch.py \
--max_epochs 250 --scheduler linear \
--dataset imagenet --data /path/to/imagenet \
--batch_size 64 --no_bias_decay --num_workers 8 \
--arch CNAS --init_channels 48 --layers 14 \
-o outputs/imagenet
Pretrained models
We have released our CNAS ImageNet pretrained model (top-1 accuracy 75.4%, top-5 accuracy 92.6%) on here.
You can use the following codes to load the ptrained models:
from cnas.model.eval import cnas_imagenet
model = cnas_imagenet(pretrained=True)
Citation
If you use any part of our code in your research, please cite our paper:
@InProceedings{guo2020breaking,
title = {Breaking the Curse of Space Explosion: Towards Efficient NAS with Curriculum Search},
author = {Guo, Yong and Chen, Yaofo and Zheng, Yin and Zhao, Peilin and Chen, Jian and Huang, Junzhou and Tan, Mingkui},
booktitle = {Proceedings of the 37th International Conference on Machine Learning},
year = {2020},
pages = {3822--3831}
}
3 Jan 7, 2022
79 Dec 19, 2022
21 Dec 13, 2022
14 Sep 30, 2022
232 Dec 25, 2022
57 Dec 15, 2022
21 Dec 27, 2022
41 Dec 12, 2022
2 Jan 28, 2022
126 Dec 21, 2022
591 Dec 27, 2022
86 Nov 18, 2022
167 Dec 27, 2022
138 Dec 12, 2022
9 Dec 15, 2022
272 Dec 21, 2022
131 Jan 02, 2023
95 Nov 26, 2021
41 Oct 09, 2022
62 Dec 20, 2022
73 Dec 16, 2022
24 Dec 25, 2022
8 Dec 08, 2022
308 Jan 05, 2023
1 Feb 07, 2022
42 Sep 05, 2022
208 Dec 28, 2022
26 Nov 24, 2022