Deep Networks with Recurrent Layer Aggregation

Related tags

Deep LearningRLANet
Overview

RLA-Net: Recurrent Layer Aggregation

Recurrence along Depth: Deep Networks with Recurrent Layer Aggregation

This is an implementation of RLA-Net (accept by NeurIPS-2021, paper).

RLANet

Introduction

This paper introduces a concept of layer aggregation to describe how information from previous layers can be reused to better extract features at the current layer. While DenseNet is a typical example of the layer aggregation mechanism, its redundancy has been commonly criticized in the literature. This motivates us to propose a very light-weighted module, called recurrent layer aggregation (RLA), by making use of the sequential structure of layers in a deep CNN. Our RLA module is compatible with many mainstream deep CNNs, including ResNets, Xception and MobileNetV2, and its effectiveness is verified by our extensive experiments on image classification, object detection and instance segmentation tasks. Specifically, improvements can be uniformly observed on CIFAR, ImageNet and MS COCO datasets, and the corresponding RLA-Nets can surprisingly boost the performances by 2-3% on the object detection task. This evidences the power of our RLA module in helping main CNNs better learn structural information in images.

RLA module

RLA_module

Changelog

  • 2021/04/06 Upload RLA-ResNet model.
  • 2021/04/16 Upload RLA-MobileNetV2 (depthwise separable conv version) model.
  • 2021/09/29 Upload all the ablation study on ImageNet.
  • 2021/09/30 Upload mmdetection files.
  • 2021/10/01 Upload pretrained weights.

Installation

Requirements

Our environments

  • OS: Linux Red Hat 4.8.5
  • CUDA: 10.2
  • Toolkit: Python 3.8.5, PyTorch 1.7.0, torchvision 0.8.1
  • GPU: Tesla V100

Please refer to get_started.md for more details about installation.

Quick Start

Train with ResNet

- Use single node or multi node with multiple GPUs

Use multi-processing distributed training to launch N processes per node, which has N GPUs. This is the fastest way to use PyTorch for either single node or multi node data parallel training.

python train.py -a {model_name} --b {batch_size} --multiprocessing-distributed --world-size 1 --rank 0 {imagenet-folder with train and val folders}

- Specify single GPU or multiple GPUs

CUDA_VISIBLE_DEVICES={device_ids} python train.py -a {model_name} --b {batch_size} --multiprocessing-distributed --world-size 1 --rank 0 {imagenet-folder with train and val folders}

Testing

To evaluate the best model

python train.py -a {model_name} --b {batch_size} --multiprocessing-distributed --world-size 1 --rank 0 --resume {path to the best model} -e {imagenet-folder with train and val folders}

Visualizing the training result

To generate acc_plot, loss_plot

python eval_visual.py --log-dir {log_folder}

Train with MobileNet_v2

It is same with above ResNet replace train.py by train_light.py.

Compute the parameters and FLOPs

If you have install thop, you can paras_flops.py to compute the parameters and FLOPs of our models. The usage is below:

python paras_flops.py -a {model_name}

More examples are shown in examples.md.

MMDetection

After installing MMDetection (see get_started.md), then do the following steps:

  • put the file resnet_rla.py in the folder './mmdetection/mmdet/models/backbones/', and do not forget to import the model in the init.py file.
  • put the config files (e.g. faster_rcnn_r50rla_fpn.py) in the folder './mmdetection/configs/base/models/'
  • put the config files (e.g. faster_rcnn_r50rla_fpn_1x_coco.py) in the folder './mmdetection/configs/faster_rcnn'

Note that the config files of the latest version of MMDetection are a little different, please modify the config files according to the latest format.

Experiments

ImageNet

Model Param. FLOPs Top-1 err.(%) Top-5 err.(%) BaiduDrive(models) Extract code GoogleDrive
RLA-ResNet50 24.67M 4.17G 22.83 6.58 resnet50_rla_2283 5lf1 resnet50_rla_2283
RLA-ECANet50 24.67M 4.18G 22.15 6.11 ecanet50_rla_2215 xrfo ecanet50_rla_2215
RLA-ResNet101 42.92M 7.79G 21.48 5.80 resnet101_rla_2148 zrv5 resnet101_rla_2148
RLA-ECANet101 42.92M 7.80G 21.00 5.51 ecanet101_rla_2100 vhpy ecanet101_rla_2100
RLA-MobileNetV2 3.46M 351.8M 27.62 9.18 dsrla_mobilenetv2_k32_2762 g1pm dsrla_mobilenetv2_k32_2762
RLA-ECA-MobileNetV2 3.46M 352.4M 27.07 8.89 dsrla_mobilenetv2_k32_eca_2707 9orl dsrla_mobilenetv2_k32_eca_2707

COCO 2017

Model AP AP_50 AP_75 BaiduDrive(models) Extract code GoogleDrive
Fast_R-CNN_resnet50_rla 38.8 59.6 42.0 faster_rcnn_r50rla_fpn_1x_coco_388 q5c8 faster_rcnn_r50rla_fpn_1x_coco_388
Fast_R-CNN_ecanet50_rla 39.8 61.2 43.2 faster_rcnn_r50rlaeca_fpn_1x_coco_398 f5xs faster_rcnn_r50rlaeca_fpn_1x_coco_398
Fast_R-CNN_resnet101_rla 41.2 61.8 44.9 faster_rcnn_r101rla_fpn_1x_coco_412 0ri3 faster_rcnn_r101rla_fpn_1x_coco_412
Fast_R-CNN_ecanet101_rla 42.1 63.3 46.1 faster_rcnn_r101rlaeca_fpn_1x_coco_421 cpug faster_rcnn_r101rlaeca_fpn_1x_coco_421
RetinaNet_resnet50_rla 37.9 57.0 40.8 retinanet_r50rla_fpn_1x_coco_379 lahj retinanet_r50rla_fpn_1x_coco_379
RetinaNet_ecanet50_rla 39.0 58.7 41.7 retinanet_r50rlaeca_fpn_1x_coco_390 adyd retinanet_r50rlaeca_fpn_1x_coco_390
RetinaNet_resnet101_rla 40.3 59.8 43.5 retinanet_r101rla_fpn_1x_coco_403 p8y0 retinanet_r101rla_fpn_1x_coco_403
RetinaNet_ecanet101_rla 41.5 61.6 44.4 retinanet_r101rlaeca_fpn_1x_coco_415 hdqx retinanet_r101rlaeca_fpn_1x_coco_415
Mask_R-CNN_resnet50_rla 39.5 60.1 43.3 mask_rcnn_r50rla_fpn_1x_coco_395 j1x6 mask_rcnn_r50rla_fpn_1x_coco_395
Mask_R-CNN_ecanet50_rla 40.6 61.8 44.0 mask_rcnn_r50rlaeca_fpn_1x_coco_406 c08r mask_rcnn_r50rlaeca_fpn_1x_coco_406
Mask_R-CNN_resnet101_rla 41.8 62.3 46.2 mask_rcnn_r101rla_fpn_1x_coco_418 8bsn mask_rcnn_r101rla_fpn_1x_coco_418
Mask_R-CNN_ecanet101_rla 42.9 63.6 46.9 mask_rcnn_r101rlaeca_fpn_1x_coco_429 3kmz mask_rcnn_r101rlaeca_fpn_1x_coco_429

Citation

@misc{zhao2021recurrence,
      title={Recurrence along Depth: Deep Convolutional Neural Networks with Recurrent Layer Aggregation}, 
      author={Jingyu Zhao and Yanwen Fang and Guodong Li},
      year={2021},
      eprint={2110.11852},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Questions

Please contact '[email protected]' or '[email protected]'.

Owner
Joy Fang
Joy Fang
Official Pytorch Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images.

IAug_CDNet Official Implementation of Adversarial Instance Augmentation for Building Change Detection in Remote Sensing Images. Overview We propose a

53 Dec 02, 2022
Measures input lag without dedicated hardware, performing motion detection on recorded or live video

What is InputLagTimer? This tool can measure input lag by analyzing a video where both the game controller and the game screen can be seen on a webcam

Bruno Gonzalez 4 Aug 18, 2022
[ICCV'21] PlaneTR: Structure-Guided Transformers for 3D Plane Recovery

PlaneTR: Structure-Guided Transformers for 3D Plane Recovery This is the official implementation of our ICCV 2021 paper News There maybe some bugs in

73 Nov 30, 2022
Dataset VSD4K includes 6 popular categories: game, sport, dance, vlog, interview and city.

CaFM-pytorch ICCV ACCEPT Introduction of dataset VSD4K Our dataset VSD4K includes 6 popular categories: game, sport, dance, vlog, interview and city.

96 Jul 05, 2022
A TensorFlow 2.x implementation of Masked Autoencoders Are Scalable Vision Learners

Masked Autoencoders Are Scalable Vision Learners A TensorFlow implementation of Masked Autoencoders Are Scalable Vision Learners [1]. Our implementati

Aritra Roy Gosthipaty 59 Dec 10, 2022
ZEBRA: Zero Evidence Biometric Recognition Assessment

ZEBRA: Zero Evidence Biometric Recognition Assessment license: LGPLv3 - please reference our paper version: 2020-06-11 author: Andreas Nautsch (EURECO

Voice Privacy Challenge 2 Dec 12, 2021
Source code related to the article submitted to the International Conference on Computational Science ICCS 2022 in London

POTHER: Patch-Voted Deep Learning-based Chest X-ray Bias Analysis for COVID-19 Detection Source code related to the article submitted to the Internati

Tomasz SzczepaƄski 1 Apr 29, 2022
The Power of Scale for Parameter-Efficient Prompt Tuning

The Power of Scale for Parameter-Efficient Prompt Tuning Implementation of soft embeddings from https://arxiv.org/abs/2104.08691v1 using Pytorch and H

Kip Parker 208 Dec 30, 2022
Official Pytorch implementation of RePOSE (ICCV2021)

RePOSE: Iterative Rendering and Refinement for 6D Object Detection (ICCV2021) [Link] Abstract We present RePOSE, a fast iterative refinement method fo

Shun Iwase 68 Nov 15, 2022
Caffe models in TensorFlow

Caffe to TensorFlow Convert Caffe models to TensorFlow. Usage Run convert.py to convert an existing Caffe model to TensorFlow. Make sure you're using

Saumitro Dasgupta 2.8k Dec 31, 2022
[ICML 2021] Towards Understanding and Mitigating Social Biases in Language Models

Towards Understanding and Mitigating Social Biases in Language Models This repo contains code and data for evaluating and mitigating bias from generat

Paul Liang 42 Jan 03, 2023
WarpRNNT loss ported in Numba CPU/CUDA for Pytorch

RNNT loss in Pytorch - Numba JIT compiled (warprnnt_numba) Warp RNN Transducer Loss for ASR in Pytorch, ported from HawkAaron/warp-transducer and a re

Somshubra Majumdar 15 Oct 22, 2022
A web-based application for quick, scalable, and automated hyperparameter tuning and stacked ensembling in Python.

Xcessiv Xcessiv is a tool to help you create the biggest, craziest, and most excessive stacked ensembles you can think of. Stacked ensembles are simpl

Reiichiro Nakano 1.3k Nov 17, 2022
Image transformations designed for Scene Text Recognition (STR) data augmentation. Published at ICCV 2021 Workshop on Interactive Labeling and Data Augmentation for Vision.

Data Augmentation for Scene Text Recognition (ICCV 2021 Workshop) (Pronounced as "strog") Paper Arxiv Why it matters? Scene Text Recognition (STR) req

Rowel Atienza 152 Dec 28, 2022
This program will stylize your photos with fast neural style transfer.

Neural Style Transfer (NST) Using TensorFlow Demo TensorFlow TensorFlow is an end-to-end open source platform for machine learning. It has a comprehen

Ismail Boularbah 1 Aug 08, 2022
This project aims to be a handler for input creation and running of multiple RICEWQ simulations.

What is autoRICEWQ? This project aims to be a handler for input creation and running of multiple RICEWQ simulations. What is RICEWQ? From the descript

Yass Fuentes 1 Feb 01, 2022
Repository for scripts and notebooks from the book: Programming PyTorch for Deep Learning

Repository for scripts and notebooks from the book: Programming PyTorch for Deep Learning

Ian Pointer 368 Dec 17, 2022
Official Repository for the paper "Improving Baselines in the Wild".

iWildCam and FMoW baselines (WILDS) This repository was originally forked from the official repository of WILDS datasets (commit 7e103ed) For general

Kazuki Irie 3 Nov 24, 2022
Code for our paper 'Generalized Category Discovery'

Generalized Category Discovery This repo is a placeholder for code for our paper: Generalized Category Discovery Abstract: In this paper, we consider

107 Dec 28, 2022
Random Walk Graph Neural Networks

Random Walk Graph Neural Networks This repository is the official implementation of Random Walk Graph Neural Networks. Requirements Code is written in

Giannis Nikolentzos 38 Jan 02, 2023