Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

Last update: Nov 07, 2022

Related tags

Overview

Implementation of "Distribution Alignment: A Unified Framework for Long-tail Visual Recognition"(CVPR 2021)

We implement the classification, object detection and instance segmentation tasks based on our cvpods. The users should install cvpods first and run the experiments in this repo.

Changelog

4.23.2021 Update the DisAlign on LVIS v0.5(Mask R-CNN + Res50)
4.12.2021 Update the README

0. How to Use

Step-1: Install the latest cvpods.
Step-2: cd cvpods
Step-3: Prepare dataset for different tasks.
Step-4: git clone https://github.com/Megvii-BaseDetection/DisAlign playground_disalign
Step-5: Enter one folder and run pods_train --num-gpus 8
Step-6: Use pods_test --num-gpus 8 to evaluate the last the checkpoint

1. Image Classification

We support the the following three datasets:

ImageNet-LT Dataset
iNaturalist-2018 Dataset
Place-LT Dataset

We refer the user to CLS_README for more details.

2. Object Detection/Instance Segmentation

We support the two versions of the LVIS dataset:

LVIS v0.5
LVIS v1.0

Highlight

To speedup the evaluation on LVIS dataset, we provide the C++ optimized evaluation api by modifying the coco_eval(C++) in cvpods.

The C++ version lvis_eval API will save ~30% time when calculating the mAP.

We provide support for the metric of AP_fixed and AP_pool proposed in large-vocab-devil
We will support more recent works on long-tail detection in this project(e.g. EQLv2, CenterNet2, etc.) in the future.

We refer the user to DET_README for more details.

3. Semantic Segmentation

We adopt the mmsegmentation as the codebase for runing all experiments of DisAlign. Currently, the user should use DisAlign_Seg for the semantic segmentation experiments. We will add the support for these experiments in cvpods in the future.

Acknowledgement

Thanks for the following projects:

Citing DisAlign

If you are using the DisAlign in your research or with to refer to the baseline results publised in this repo, please use the following BibTex entry.

@inproceedings{zhang2021disalign,
  title={Distribution Alignment: A Unified Framework for Long-tail Visual Recognition.},
  author={Zhang, Songyang and Li, Zeming and Yan, Shipeng and He, Xuming and Sun, Jian},
  booktitle={CVPR},
  year={2021}
}

License

This repo is released under the Apache 2.0 license. Please see the LICENSE file for more information.

Comments

scale in cosine classifier

Hi, thanks for your great work! I notice you use the cosine classifier in many experiments and it can get a better baseline. The formula is as follows

I am wondering the value of s?

opened by L1aoXingyu 5
Is it correct to freeze the weight and bias of the DisAlign Linear Layer as well?

Hello. Thank you for your project! I'm testing your code on my custom dataset. My task is classification. I have a question about your code implementation.

https://github.com/Megvii-BaseDetection/DisAlign/blob/a2fc3500a108cb83e3942293a5675c97ab3a2c6e/classification/imagenetlt/resnext50/resx50.scratch.imagenet_lt.224size.90e.disalign.10e/net.py#L56-L62

From my understanding, in stage 2, remove the linear layer used in stage 1 and add DisAlign Linear Layer. And freeze all parts except for logit_scale, logit_bias, and confidence_layer. At this time, the weight and bias of DisAlignLinear are also frozen. (self.weight, self.bias) Is my understanding correct?

If so, are the weight and bias of DisAlignLinearLayer fixed after the initialization? (The weight and bias of the linear layer in stage 1 are not copied either)

If my understanding is correct, why is the weight of DisAlignLinear also frozen?

I will wait for your reply. thanks!

opened by jeongHwarr 4
Where is the DisAlignLinear module?

Hello. Thank you for your impressive project!

I want to apply DisAlign to classification. However, an error occurs in the import part. https://github.com/Megvii-BaseDetection/DisAlign/blob/a2fc3500a108cb83e3942293a5675c97ab3a2c6e/classification/imagenetlt/resnext50/resx50.scratch.imagenet_lt.224size.90e.disalign.10e/net.py#L7 I coudn't find the DisAlignLinear in cvpods.layers. and there also isn't exist at https://github.com/Megvii-BaseDetection/cvpods/tree/master/cvpods/layers How can I solve this problem?

Thank you!

opened by jeongHwarr 4
Can someone kindly share their codes of Classification task on ImageNet_LT?

I tried to train the proposed method on ImageNet_LT, but I can only get an average testing rate about 49%, which is far from the rate described in the paper (52.9). Some of the details regarding my implementations are given as follows: (1) The feature extractor is ResNexT-50 and the head classifier is a linear classifier. The testing accuracy in Stage-One is 43.9%, which is OK.

(2) The testing accuracy of adopting cRT method in Stage-Two is 49.6%, which is identical to one reported in other papers. (3) When fine-tuning the model in Stage-2, both the feature-extractor and head classifier are frozen, and a DisAliLinear model (which is implemented in CVPODs) is retrained. The testing accuracy can only reach 48.8%, which is far away from the one reported in your paper.

opened by smallcube 4
The code for semantic segmentation is missing

Hi, thank you for the nice work, but the code for semantic segmentation is missing and the URL for it in the README could not be opened. Could you please fix this issue?

opened by curiosity654 3
About the reference Distribution p_r in Eq. (10)

Hi, Thank you for providing your code. Here I was wondering the Equation (10) in your paper (The definition of p_r), which seems not to be a distribution. Since every x_i can only have one label, the reference distribution p_r(y| x_i) will be the distribution like (0, 0, 0,...,w_c, 0, 0,...,0). And the sum of this distribution is w_c, but not 1.

Could you help me understand this equation? Thanks in advance.

opened by Kevinz-code 3
import error

Hi, thanks for the great work. Maybe I missed it, but it seems that the code for this project has been incorporated into cvpods. I couldn't launch any experiments due to ImportErrors like: from cvpods.layers import DisAlignLinear ImportError: cannot import name 'DisAlignLinear' from 'cvpods.layers' Also, I didn't find the corresponding functions in cvpods.

Any help will be appreciated. Thanks.

opened by YUE-FAN 2
about the confidence score σ(x)

In the paper, the σ(x) is implemented as a linear layer followed by a non-linear activation function (e.g., sigmoid function) for all input x. How to understand the input x？the matrix of raw iamge, or the extracted features, even or cls_score? Thank you!

opened by lzed2399 2
exp_reweight = exp_reweight / np.sum(exp_reweight) * num_foreground
Dear author, I have some questions about the code and paper:

exp_reweight = exp_reweight / np.sum(exp_reweight) * num_foreground Why "exp_reweight" is multiplied by the coefficient "num_foreground"? It is not mentioned in the paper.

Is "K" in the empirical class frequencies r = [r1, · · · , rK] on the training set in the paper the same as the class number C of the training set?
opened by Liu-wanbing 2
The DisAlign_Seg page can't open

Hi, thank you for your amazing work. I want to have a try on semantic segmentation problem. But the link https://github.com/Megvii-BaseDetection/DisAlign/blob/main/TODO of DisAlign_Seg cannot open. Could you pls have a look? Thank you.

opened by Kittywyk 1
Do you use validation dataset?

https://github.com/Megvii-BaseDetection/DisAlign/blob/main/classification/imagenetlt/resnext50/resx50.scratch.imagenet_lt.224size.90e.disalign.10e/config.py#L31

It seems that you only use test dataset? What is the reason for that?

opened by qianlanwyd 1
How can I test and augtest the trained semseg DisAlign model?

Thank you for your great work. I met the question of 'AttributeError: 'ConfigDict' object has no attribute 'model' in testing the trained semseg DisAlign model when running the file https://github.com/Megvii-BaseDetection/DisAlign/blob/main/semantic_seg/exps/ade20k_fcn_disalign/disalign_fcn_r50-d8_512x512_160k_ade20k.sh. Is this a version problem of mmcv-full or something else? And can you perfect the explanation of the disalign part of Readme.md in semantic_seg?

opened by jh151170 0
the code question in semantic_seg

Hi, I have a questation about the logit_scale and logit_bias in semantic_seg. The shape of the above parameter is (1, num_classes, 1, 1), why not is (1, num_classes, 512, 512) which is matched the input image size for semantic segmenation.

opened by Ianresearch 8
Value of the learned scale and bias vector?

Hi, did you check the value change of the learned scale and bias vector throughout the training process? I find the value of them change in the first few iterations and remain stable in the rest time on my own classification dataset. I wonder how the learned vectors look like in your paper? Thanks!

opened by Jacobew 1