A curated list of awesome deep long-tailed learning resources.

Overview

Awesome Long-Tailed Learning

A curated list of awesome deep long-tailed learning resources. We recently released Deep Long-Tailed Learning: A Survey to the community. In this survey, we reviewed recent advances in long-tailed learning based on deep neural networks.

Specifically, existing long-tailed learning studies can be grouped into three main categories (i.e., class re-balancing, information augmentation and module improvement), which can be further classified into nine sub-categories (as shown in the below figure). We also empirically analyzed several state-of-the-art methods by evaluating to what extent they address the issue of class imbalance. We concluded the survey by highlighting important applications of deep long-tailed learning and identifying several promising directions for future research. After completing this survey, we decided to release the collected long-tailed learning resources, hoping to push the development of the community. If you have any questions or suggestions, please feel free to contact us.

1. Type of Long-tailed Learning

Symbol Sampling CSL LA TL Aug
Type Re-sampling Cost-sensitive Learning Logit Adjustment Transfer Learning Data Augmentation
Symbol RL CD DT Ensemble other
Type Representation Learning Classifier Design Decoupled Training Ensemble Learning Other Types

2. Top-tier Conference Papers

2021

Title Venue Year Type Code
Improving contrastive learning on imbalanced seed data via open-world sampling NeurIPS 2021 Sampling,TL, DC Official
Semi-supervised semantic segmentation via adaptive equalization learning NeurIPS 2021 Sampling,CSL,TL, Aug Official
On model calibration for long-tailed object detection and instance segmentation NeurIPS 2021 LA Official
Label-imbalanced and group-sensitive classification under overparameterization NeurIPS 2021 LA
Towards calibrated model for long-tailed visual recognition from prior perspective NeurIPS 2021 Aug, RL Official
Supercharging imbalanced data learning with energy-based contrastive representation transfer NeurIPS 2021 Aug, TL, RL Official
VideoLT: Large-scale long-tailed video recognition ICCV 2021 Sampling Official
Exploring classification equilibrium in long-tailed object detection ICCV 2021 Sampling,CSL Official
GistNet: a geometric structure transfer network for long-tailed recognition ICCV 2021 Sampling,TL, DC
FASA: Feature augmentation and sampling adaptation for long-tailed instance segmentation ICCV 2021 Sampling,CSL
ACE: Ally complementary experts for solving long-tailed recognition in one-shot ICCV 2021 Sampling,Ensemble Official
Influence-Balanced Loss for Imbalanced Visual Classification ICCV 2021 CSL Official
Re-distributing biased pseudo labels for semi-supervised semantic segmentation: A baseline investigation ICCV 2021 TL Official
Self supervision to distillation for long-tailed visual recognition ICCV 2021 TL Official
Distilling virtual examples for long-tailed recognition ICCV 2021 TL
MosaicOS: A simple and effective use of object-centric images for long-tailed object detection ICCV 2021 TL Official
Parametric contrastive learning ICCV 2021 RL Official
Distributional robustness loss for long-tail learning ICCV 2021 RL Official
Learning of visual relations: The devil is in the tails ICCV 2021 DT
Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection ICML 2021 Sampling Official
Delving into deep imbalanced regression ICML 2021 Other Official
Long-tailed multi-label visual recognition by collaborative training on uniform and re-balanced samplings CVPR 2021 Sampling,Ensemble
Equalization loss v2: A new gradient balance approach for long-tailed object detection CVPR 2021 CSL Official
Seesaw loss for long-tailed instance segmentation CVPR 2021 CSL Official
Adaptive class suppression loss for long-tail object detection CVPR 2021 CSL Official
PML: Progressive margin loss for long-tailed age classification CVPR 2021 CSL
Disentangling label distribution for long-tailed visual recognition CVPR 2021 CSL,LA Official
Adversarial robustness under long-tailed distribution CVPR 2021 CSL,LA,CD Official
Distribution alignment: A unified framework for long-tail visual recognition CVPR 2021 CSL,LA,DT Official
Improving calibration for long-tailed recognition CVPR 2021 CSL,Aug,DT Official
CReST: A classrebalancing self-training framework for imbalanced semi-supervised learning CVPR 2021 TL Official
Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts CVPR 2021 TL Official
RSG: A simple but effective module for learning imbalanced datasets CVPR 2021 TL,Aug Official
MetaSAug: Meta semantic augmentation for long-tailed visual recognition CVPR 2021 Aug Official
Contrastive learning based hybrid networks for long-tailed image classification CVPR 2021 RL
Unsupervised discovery of the long-tail in instance segmentation using hierarchical self-supervision CVPR 2021 RL
Long-tail learning via logit adjustment ICLR 2021 LA Official
Long-tailed recognition by routing diverse distribution-aware experts ICLR 2021 TL,Ensemble Official
Exploring balanced feature spaces for representation learning ICLR 2021 RL,DT

2020

Title Venue Year Type Code
Balanced meta-softmax for long-taield visual recognition NeurIPS 2020 Sampling,CSL Official
Posterior recalibration for imbalanced datasets NeurIPS 2020 LA Official
Long-tailed classification by keeping the good and removing the bad momentum causal effect NeurIPS 2020 LA,CD Official
Rethinking the value of labels for improving classimbalanced learning NeurIPS 2020 TL,RA Official
The devil is in classification: A simple framework for long-tail instance segmentation ECCV 2020 Sampling,DT,Ensemble Official
Imbalanced continual learning with partitioning reservoir sampling ECCV 2020 Sampling Official
Distribution-balanced loss for multi-label classification in long-tailed datasets ECCV 2020 CSL Official
Feature space augmentation for long-tailed data ECCV 2020 TL,Aug,DT
Learning from multiple experts: Self-paced knowledge distillation for long-tailed classification ECCV 2020 TL,Ensemble Official
Solving long-tailed recognition with deep realistic taxonomic classifier ECCV 2020 CD Official
Learning to segment the tail CVPR 2020 Sampling,TL Official
BBN: Bilateral-branch network with cumulative learning for long-tailed visual recognition CVPR 2020 Sampling,Ensemble Official
Overcoming classifier imbalance for long-tail object detection with balanced group softmax CVPR 2020 Sampling,Ensemble Official
Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective CVPR 2020 CSL Official
Equalization loss for long-tailed object recognition CVPR 2020 CSL Official
Domain balancing: Face recognition on long-tailed domains CVPR 2020 CSL
M2m: Imbalanced classification via majorto-minor translation CVPR 2020 TL,Aug Official
Deep representation learning on long-tailed data: A learnable embedding augmentation perspective CVPR 2020 TL,Aug,RL
Inflated episodic memory with region self-attention for long-tailed visual recognition CVPR 2020 RL
Decoupling representation and classifier for long-tailed recognition ICLR 2020 Sampling,CSL,RL,CD,DT Official

2019

Title Venue Year Type Code
Meta-weight-net: Learning an explicit mapping for sample weighting NeurIPS 2019 CSL Official
Learning imbalanced datasets with label-distribution-aware margin loss NeurIPS 2019 CSL Official
Dynamic curriculum learning for imbalanced data classification ICCV 2019 Sampling
Class-balanced loss based on effective number of samples CVPR 2019 CSL Official
Striking the right balance with uncertainty CVPR 2019 CSL
Feature transfer learning for face recognition with under-represented data CVPR 2019 TL,Aug
Unequal-training for deep face recognition with long-tailed noisy data CVPR 2019 RL Official
Large-scale long-tailed recognition in an open world CVPR 2019 RL Official

2018

Title Venue Year Type Code
Large scale fine-grained categorization and domain-specific transfer learning CVPR 2018 TL Official

2017

Title Venue Year Type Code
Learning to model the tail NeurIPS 2017 CSL
Focal loss for dense object detection ICCV 2017 CSL
Range loss for deep face recognition with long-tailed training data ICCV 2017 RL
Class rectification hard mining for imbalanced deep learning ICCV 2017 RL

2016

Title Venue Year Type Code
Learning deep representation for imbalanced classification CVPR 2016 Sampling,RL
Factors in finetuning deep model for object detection with long-tail distribution CVPR 2016 CSL,RL

3. Benchmark Datasets

Dataset Long-tailed Task # Class # Training data # Test data
ImageNet-LT Classification 1,000 115,846 50,000
CIFAR100-LT Classification 100 50,000 10,000
Places-LT Classification 365 62,500 36,500
iNaturalist 2018 Classification 8,142 437,513 24,426
LVIS v0.5 Detection and Segmentation 1,230 57,000 20,000
LVIS v1 Detection and Segmentation 1,203 100,000 19,800
VOC-LT Multi-label Classification 20 1,142 4,952
COCO-LT Multi-label Classification 80 1,909 5,000
VideoLT Video Classification 1,004 179,352 25,622

4. Empirical Studies

(1) Long-tailed benchmarking performance

  • We evaluate several state-of-the-art methods on ImageNet-LT to see to what extent they handle class imbalance via new evaluation metrics, i.e., UA (upper bound accuracy) and RA (relative accuracy). We categorize these methods based on class re-balancing (CR), information augmentation (IA) and module improvement (MI).

  • Almost all long-tailed methods perform better than the Softmax baseline in terms of accuracy, which demonstrates the effectiveness of long-tailed learning.
  • Training with 200 epochs leads to better performance for most long-tailed methods, since sufficient training enables deep models to fit data better and learn better image representations.
  • In addition to accuracy, we also evaluate long-tailed methods based on UA and RA. For the methods that have higher UA, the performance gain comes not only from the alleviation of class imbalance, but also from other factors, like data augmentation or better network architectures. Therefore, simply using accuracy for evaluation is not accurate enough, while our proposed RA metric provides a good complement, since it alleviates the influences of factors apart from class imbalance.
  • For example, MiSLAS, based on data mixup, has higher accuracy than Balanced Sofmtax under 90 training epochs, but it also has higher UA. As a result, the relative accuracy of MiSLAS is lower than Balanced Sofmtax, which means that Balanced Sofmtax alleviates class imbalance better than MiSLAS under 90 training epochs.
  • Although some recent high-accuracy methods have lower RA, the overall development trend of long-tailed learning is still positive, as shown in the below figure.

  • The current state-of-the-art long-tailed method in terms of both accuracy and RA is TADE (ensemble-based method).

(2) More discussions on cost-sensitive losses

  • We further evaluate the performance of different cost-sensitive learning losses based on the decoupled training scheme.
  • Decoupled training, compared to joint training, can further improve the overall performance of most cost-sensitive learning methods apart from balanced softmax (BS).
  • Although BS outperofmrs other cost-sensitive losses under one-stage training, they perform comparably under decoupled training. This implies that although these cost-sensitive losses perform differently under joint training, they essentially learn similar quality of feature representations.

5. Citation

If this repository is helpful to you, please cite our survey.

@article{zhang2021deep,
  title={Deep long-tailed learning: A survey},
  author={Zhang, Yifan and Kang, Bingyi and Hooi, Bryan and Yan, Shuicheng and Feng, Jiashi},
  journal={arXiv preprint arXiv:2110.04596},
  year={2021}
}

5. Other Resources

Owner
vanint
vanint
Deep Q-network learning to play flappybird.

AI Plays Flappy Bird I've trained a DQN that learns to play flappy bird on it's own. Try the pre-trained model First install the pip requirements and

Anish Shrestha 3 Mar 01, 2022
This was initially the repo for the project of [email protected] of Asaf Mazar, Millad Kassaie and Georgios Chochlakis named "Powered by the Will? Exploring Lay Theories of Behavior Change through Social Media"

Subreddit Analysis This repo includes tools for Subreddit analysis, originally developed for our class project of PSYC 626 in USC, titled "Powered by

Georgios Chochlakis 1 Dec 17, 2021
POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propagation including diffraction

POPPY: Physical Optics Propagation in Python POPPY (Physical Optics Propagation in Python) is a Python package that simulates physical optical propaga

Space Telescope Science Institute 132 Dec 15, 2022
PyTorch implementation of the REMIND method from our ECCV-2020 paper "REMIND Your Neural Network to Prevent Catastrophic Forgetting"

REMIND Your Neural Network to Prevent Catastrophic Forgetting This is a PyTorch implementation of the REMIND algorithm from our ECCV-2020 paper. An ar

Tyler Hayes 72 Nov 27, 2022
Image processing in Python

scikit-image: Image processing in Python Website (including documentation): https://scikit-image.org/ Mailing list: https://mail.python.org/mailman3/l

Image Processing Toolbox for SciPy 5.2k Dec 31, 2022
Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image

NonCuboidRoom Paper Learning to Reconstruct 3D Non-Cuboid Room Layout from a Single RGB Image Cheng Yang*, Jia Zheng*, Xili Dai, Rui Tang, Yi Ma, Xiao

67 Dec 15, 2022
NeuralForecast is a Python library for time series forecasting with deep learning models

NeuralForecast is a Python library for time series forecasting with deep learning models. It includes benchmark datasets, data-loading utilities, evaluation functions, statistical tests, univariate m

Nixtla 1.1k Jan 03, 2023
以孤立语假设和宽度优先搜索为基础,构建了一种多通道堆叠注意力Transformer结构的斗地主ai

ddz-ai 介绍 斗地主是一种扑克游戏。游戏最少由3个玩家进行,用一副54张牌(连鬼牌),其中一方为地主,其余两家为另一方,双方对战,先出完牌的一方获胜。 ddz-ai以孤立语假设和宽度优先搜索为基础,构建了一种多通道堆叠注意力Transformer结构的系统,使其经过大量训练后,能在实际游戏中获

freefuiiismyname 88 May 15, 2022
Website for D2C paper

D2C This is the repository that contains source code for the D2C Website. If you find D2C useful for your work please cite: @article{sinha2021d2c au

1 Oct 21, 2021
Vehicle Detection Using Deep Learning and YOLO Algorithm

VehicleDetection Vehicle Detection Using Deep Learning and YOLO Algorithm Dataset take or find vehicle images for create a special dataset for fine-tu

Maryam Boneh 96 Jan 05, 2023
ROS Basics and TurtleSim

Waypoint Follower Anna Garverick This package draws given waypoints, then waits for a service call with a start position to send the turtle to each wa

Anna Garverick 1 Dec 13, 2021
[KDD 2021, Research Track] DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neural Networks

DiffMG This repository contains the code for our KDD 2021 Research Track paper: DiffMG: Differentiable Meta Graph Search for Heterogeneous Graph Neura

AutoML Research 24 Nov 29, 2022
Find the Heart simple Python Game

This is a simple Python game for finding a heart emoji. There is a 3 x 3 matrix in which a heart emoji resides. The location of the heart is randomized and is not revealed. The player must guess the

p.katekomol 1 Jan 24, 2022
PyTorch implementation of the TTC algorithm

Trust-the-Critics This repository is a PyTorch implementation of the TTC algorithm and the WGAN misalignment experiments presented in Trust the Critic

0 Nov 29, 2021
covid question answering datasets and fine tuned models

Covid-QA Fine tuned models for question answering on Covid-19 data. Hosted Inference This model has been contributed to huggingface.Click here to see

Abhijith Neil Abraham 19 Sep 09, 2021
"SOLQ: Segmenting Objects by Learning Queries", SOLQ is an end-to-end instance segmentation framework with Transformer.

SOLQ: Segmenting Objects by Learning Queries This repository is an official implementation of the paper SOLQ: Segmenting Objects by Learning Queries.

MEGVII Research 179 Jan 02, 2023
Python based Advanced AI Assistant

Knick is a virtual artificial intelligence project, fully developed in python. The objective of this project is to develop a virtual assistant that can handle our minor, intermediate as well as heavy

19 Nov 15, 2022
Deep Video Matting via Spatio-Temporal Alignment and Aggregation [CVPR2021]

Deep Video Matting via Spatio-Temporal Alignment and Aggregation [CVPR2021] Paper: https://arxiv.org/abs/2104.11208 Introduction Despite the significa

76 Dec 07, 2022
UltraGCN: An Ultra Simplification of Graph Convolutional Networks for Recommendation

UltraGCN This is our Pytorch implementation for our CIKM 2021 paper: Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, Xiuqiang He. UltraGCN: A

XUEPAI 93 Jan 03, 2023
This is the official code for the paper "Ad2Attack: Adaptive Adversarial Attack for Real-Time UAV Tracking".

Ad^2Attack:Adaptive Adversarial Attack on Real-Time UAV Tracking Demo video 📹 Our video on bilibili demonstrates the test results of Ad^2Attack on se

Intelligent Vision for Robotics in Complex Environment 10 Nov 07, 2022