Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Last update: Dec 17, 2022

Overview

ADE20k Semantic segmentation with MAE

Getting started

Install the mmsegmentation library and some required packages.

pip install mmcv-full==1.3.0 mmsegmentation==0.11.0
pip install scipy timm==0.3.2

Install apex for mixed-precision training

git clone https://github.com/NVIDIA/apex
cd apex
pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Follow the guide in mmseg to prepare the ADE20k dataset.

Fine-tuning for Reproducing Results of MAE ViT-Base

Command:

tools/dist_train.sh configs/mae/upernet_mae_base_12_512_slide_160k_ade20k.py 8 --seed 0  --options model.pretrained=https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth

Expected results log(paper results: 48.1 mIoU):

+--------+-------+-------+-------+
| Scope  | mIoU  | mAcc  | aAcc  |
+--------+-------+-------+-------+
| global | 48.15 | 58.99 | 83.05 |
+--------+-------+-------+-------+

Evaluation

Command format:

tools/dist_test.sh  <CONFIG_PATH> <CHECKPOINT_PATH> <NUM_GPUS> --eval mIoU

Acknowledgment

This code is built using the mmsegmentation library, Timm library, the Swin repository, XCiT, SETR, BEiT and the MAE repository.

Mae segmentation - Reproduction of semantic segmentation using masked autoencoder (mae)

Related tags

Overview

ADE20k Semantic segmentation with MAE

Getting started

Fine-tuning for Reproducing Results of MAE ViT-Base

Evaluation

Acknowledgment

Owner

TACTO: A Fast, Flexible and Open-source Simulator for High-Resolution Vision-based Tactile Sensors

Second Order Optimization and Curvature Estimation with K-FAC in JAX.

This tutorial repository is to introduce the functionality of KGTK to first-time users

[CVPR2021 Oral] UP-DETR: Unsupervised Pre-training for Object Detection with Transformers

Python版OpenCVのTracking APIのサンプルです。DaSiamRPNアルゴリズムまで対応しています。

Bi-level feature alignment for versatile image translation and manipulation (Under submission of TPAMI)

Voice Conversion by CycleGAN (语音克隆/语音转换)：CycleGAN-VC3

PyTorch implementation of Deformable Convolution

InsCLR: Improving Instance Retrieval with Self-Supervision

Hummingbird compiles trained ML models into tensor computation for faster inference.

High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.

PyTorch Implementation of NCSOFT's FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis

Simulating an AI playing 2048 using the Expectimax algorithm

This is the code repository implementing the paper "TreePartNet: Neural Decomposition of Point Clouds for 3D Tree Reconstruction".

Code for "CloudAAE: Learning 6D Object Pose Regression with On-line Data Synthesis on Point Clouds" @ICRA2021

codes for Image Inpainting with External-internal Learning and Monochromic Bottleneck

Exploration of some patients clinical variables.

Python package for visualizing the loss landscape of parameterized quantum algorithms.

Repository accompanying the "Sign Pose-based Transformer for Word-level Sign Language Recognition" paper

Vector Neurons: A General Framework for SO(3)-Equivariant Networks