Model Zoo of BDD100K Dataset

Overview

BDD100K Model Zoo

In this repository, we provide popular models for each task in the BDD100K dataset.

teaser

For each task in the BDD100K dataset, we make publicly available the model weights, evaluation results, predictions, visualizations, as well as scripts to performance evaluation and visualization. The goal is to provide a set of competitive baselines to facilitate research and provide a common benchmark for comparison.

The number of pre-trained models in this zoo is 1️⃣ 1️⃣ 5️⃣ . You can include your models in this repo as well! See contribution instructions.

This repository currently supports the tasks listed below. For more information about each task, click on the task name. We plan to support all tasks in the BDD100K dataset eventually; see the roadmap for our plan and progress.

If you have any questions, please go to the BDD100K discussions.

Roadmap

  • Lane marking
  • Panoptic segmentation
  • Pose estimation

Dataset

Please refer to the dataset preparation instructions for how to prepare and use the BDD100K dataset with the models.

Maintainers

Citation

To cite the BDD100K dataset in your paper,

@InProceedings{bdd100k,
    author = {Yu, Fisher and Chen, Haofeng and Wang, Xin and Xian, Wenqi and Chen,
              Yingying and Liu, Fangchen and Madhavan, Vashisht and Darrell, Trevor},
    title = {BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning},
    booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2020}
}
Comments
  • Using the models to predict on other Images

    Using the models to predict on other Images

    Hi,

    can i use the models under "bdd100k-models/det/" to make predictions on other images ?

    When i followed the "Usage"-Section, it seems that the models can only be used to evaluate the Test/Val Images.

    opened by askppp 5
  • Drivable Segmentation Model inference stuck

    Drivable Segmentation Model inference stuck

    When I am running Deeplabv3+ model by using: python ./test.py configs/drivable/deeplabv3plus_r50-d8_512x1024_40k_drivable_bdd100k.py --format-only --format-dir output It just stuck in around 1490 step image I have tried several different configs, they all have the same issue.

    opened by danielzhangau 4
  • Generate semantic segmentation output as png

    Generate semantic segmentation output as png

    Hello,

    I'm generating semantic segmentation using the following command.

    python ./test.py ~/config.py --show-dir ~/Documents/bdd100k-models/data/bdd100k/labels/seg_track_20/val --opacity 1
    

    This generates the colormaps for the images, however, the output produced is in .jpg format which results in blur within the labels (as shown below.) How can I update the script so that it generates the labels in png format. My input images are from the MOTS 2020 Images dataset, which are in jpg format.

    image

    opened by digvijayad 2
  • Sem_Seg Inference Error - RuntimeError: DataLoader worker is killed by signal: Segmentation fault.

    Sem_Seg Inference Error - RuntimeError: DataLoader worker is killed by signal: Segmentation fault.

    Error when running Sem_seg model inference Command Run: python ./test.py ./configs/sem_seg/deeplabv3+_r50-d8_512x1024_40k_sem_seg_bdd100k.py --format-only --format-dir ./outputs

    ERROR:

    workers per gpu=2
    /home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/mmseg/models/losses/cross_entropy_loss.py:235: UserWarning: Default ``avg_non_ignore`` is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set ``avg_non_ignore=True``.
      warnings.warn(
    load checkpoint from http path: https://dl.cv.ethz.ch/bdd100k/sem_seg/models/deeplabv3+_r50-d8_512x1024_40k_sem_seg_bdd100k.pth
    'CLASSES' not found in meta, use dataset.CLASSES instead
    'PALETTE' not found in meta, use dataset.PALETTE instead
    [                                                  ] 0/1000, elapsed: 0s, ETA:ERROR: Unexpected segmentation fault encountered in worker.
    ERROR: Unexpected segmentation fault encountered in worker.
    ERROR: Unexpected segmentation fault encountered in worker.
    Traceback (most recent call last):
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1011, in _try_get_data
        data = self._data_queue.get(timeout=timeout)
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/queue.py", line 179, in get
        self.not_empty.wait(remaining)
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/threading.py", line 306, in wait
        gotit = waiter.acquire(True, timeout)
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
        _error_if_any_worker_fails()
    RuntimeError: DataLoader worker (pid 15796) is killed by signal: Segmentation fault. 
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "./test.py", line 174, in <module>
        main()
      File "./test.py", line 150, in main
        outputs = single_gpu_test(
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/mmseg/apis/test.py", line 89, in single_gpu_test
        for batch_indices, data in zip(loader_indices, data_loader):
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 530, in __next__
        data = self._next_data()
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1207, in _next_data
        idx, data = self._get_data()
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1163, in _get_data
        success, data = self._try_get_data()
      File "/home/lunet/codsn/.conda/envs/bdd100k-mmseg/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1024, in _try_get_data
        raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
    RuntimeError: DataLoader worker (pid(s) 15796) exited unexpectedly
    
    opened by digvijayad 2
  • red traffic lights

    red traffic lights

    Hello, thanks for your marvelous contribution.I would like to know that the category of red traffic lights is not available on bdd, have you re-labeled it on the bdd dataset?

    opened by liluxing153 1
  • tagging:finetune possibilities

    tagging:finetune possibilities

    hi hi, thanks for your marvelous contribution. I am very impressed. Now I want to apply this pretrain model(tagging road type and weather) on my own dataset, do you have any codebase for finetuning?

    opened by anran1231 1
  • Semantic segmetation ;common settings  MMSegmentation link not working

    Semantic segmetation ;common settings MMSegmentation link not working

    https://github.com/open-mmlab/mmsegmentation/blob/master/docs/model_zoo.md#common-settings The above link is not working

    I would like to know the settings under which the segmentation models are trained , so that i can replicate the results . thank you.

    opened by 100daggers 1
  • Issue in converting the instance segmentation mask encoding from bdd100k to coco

    Issue in converting the instance segmentation mask encoding from bdd100k to coco

    Hello,

    I am trying to convert the bdd100k instance segmentation using this command: python3 -m bdd100k.label.to_coco -m ins_seg --only-mask -i ./bdd100k/labels/ins_seg/bitmasks/val -o ./ins_seg_val_cocofmt_v2.json

    Also, tried this: python3 -m bdd100k.label.to_coco -m ins_seg -i ./bdd100k/labels/ins_seg/polygons/ins_seg_val.json -o ./ins_seg_val_cocofmt_v3.json -mb ./bdd100k/labels/ins_seg/bitmasks/val

    The conversion is successful in both cases and the annotation looks like this

    Screen Shot 2022-01-07 at 11 46 36 AM ** that's not how coco annotations are.

    Now, if you see the segmentation field above there's string encoding of the masks. Now, I am unsure if that's expected or not.

    Further, assuming it's correct, I tried to load the annotations using loader from DETR https://github.com/facebookresearch/detr/blob/091a817eca74b8b97e35e4531c1c39f89fbe38eb/datasets/coco.py#L36

    The line I have mentioned above is supposed to do the conversion but I am getting an error from the pycocotools that it's not expecting a string in the mask. Screen Shot 2022-01-07 at 11 53 51 AM

    So, I am unsure where the problem is? Is the conversion correct to coco then the loader should work? Note: I tried to convert the detections and they worked fine.

    Thank you for any help you can provide.

    opened by sfarkya04 1
  • How to train on my own gpu?

    How to train on my own gpu?

    Hello! thank you for your work~~but i wonder if i could train these models on my own gpu? i wonder if there are som instructions or usages? plz ,thank u!

    opened by StefanYz 1
Releases(v1.1.0)
  • v1.1.0(Dec 2, 2021)

    BDD100K Models 1.1.0 Release

    teaser

    • Highlights
    • New Task: Pose Estimation
    • New Models

    Highlights

    In this release, we provide over 20 pre-trained models for the new pose estimation task in BDD100K, along with evaluation and visualization tools. We also provide over 30 new models for object detection, instance segmentation, semantic segmentation, and drivable area.

    New Task: Pose Estimation

    With the release of 2D human pose estimation data in BDD100K, we provide pre-trained models in this repo.

    • Pose estimation
      • ResNet, MobileNetV2, HRNet, and more.

    New Models

    We provide additional models for previous tasks

    • Object detection
      • Libra R-CNN, HRNet.
    • Instance segmentation
      • GCNet, HRNet.
    • Semantic segmentation / drivable area
      • NLNet, PointRend.
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Oct 29, 2021)

    BDD100K Models 1.0.0 Release

    teaser

    • Highlights
    • Tasks
    • Models
    • Contribution

    Highlights

    The model zoo for BDD100K, the largest driving video dataset, is open for business! It contains more than 100 pre-trained models for 7 tasks. Each model also comes with results and visualization on val and test sets. We also provide documentation for community contribution so that everyone can include their models in this repo.

    Tasks

    We currently support 7 tasks

    • Image Tagging
    • Object Detection
    • Instance Segmentation
    • Semantic Segmentation
    • Drivable Area
    • Multiple Object Tracking (MOT)
    • Multiple Object Tracking and Segmentation (MOTS)

    Each task includes

    • Official evaluation results, model weights, predictions, and visualizations.
    • Detailed instructions for evaluation and visualization.

    Models

    We include popular network models for each task

    • Image tagging
      • VGG, ResNet, and DLA.
    • Object detection
      • Cascade R-CNN, Sparse R-CNN, Deformable ConvNets v2, and more.
    • Instance segmentation
      • Mask R-CNN, Cascade Mask R-CNN, HRNet, and more.
    • Semantic segmentation / drivable area
      • Deeplabv3+, CCNet, DNLNet, and more.
    • Multiple object tracking (MOT)
      • QDTrack.
    • Multiple object tracking and segmentation (MOTS)
      • PCAN.

    Contribution

    We encourage the BDD100K dataset users to contribute their models to this repo, so that all the info can be used for further result reproduction and analysis. The detailed instruction and model submission template are at the contribution page.

    Source code(tar.gz)
    Source code(zip)
Owner
ETH VIS Group
Visual Intelligence and Systems Group at ETH Zürich
ETH VIS Group
This repository contains the code used for the implementation of the paper "Probabilistic Regression with HuberDistributions"

Public_prob_regression_with_huber_distributions This repository contains the code used for the implementation of the paper "Probabilistic Regression w

David Mohlin 1 Dec 04, 2021
A lossless neural compression framework built on top of JAX.

Kompressor Branch CI Coverage main (active) main development A neural compression framework built on top of JAX. Install setup.py assumes a compatible

Rosalind Franklin Institute 2 Mar 14, 2022
🏖 Keras Implementation of Painting outside the box

Keras implementation of Image OutPainting This is an implementation of Painting Outside the Box: Image Outpainting paper from Standford University. So

Bendang 1.1k Dec 10, 2022
Seq2seq - Sequence to Sequence Learning with Keras

Seq2seq Sequence to Sequence Learning with Keras Hi! You have just found Seq2Seq. Seq2Seq is a sequence to sequence learning add-on for the python dee

Fariz Rahman 3.1k Dec 18, 2022
Implementation of CVPR 2020 Dual Super-Resolution Learning for Semantic Segmentation

Dual super-resolution learning for semantic segmentation 2021-01-02 Subpixel Update Happy new year! The 2020-12-29 update of SISR with subpixel conv p

Sam 79 Nov 24, 2022
[CVPR'21] DeepSurfels: Learning Online Appearance Fusion

DeepSurfels: Learning Online Appearance Fusion Paper | Video | Project Page This is the official implementation of the CVPR 2021 submission DeepSurfel

Online Reconstruction 52 Nov 14, 2022
A machine learning library for spiking neural networks. Supports training with both torch and jax pipelines, and deployment to neuromorphic hardware.

Rockpool Rockpool is a Python package for developing signal processing applications with spiking neural networks. Rockpool allows you to build network

SynSense 21 Dec 14, 2022
An Unsupervised Detection Framework for Chinese Jargons in the Darknet

An Unsupervised Detection Framework for Chinese Jargons in the Darknet This repo is the Python 3 implementation of 《An Unsupervised Detection Framewor

7 Nov 08, 2022
Nicholas Lee 3 Jan 09, 2022
The Turing Change Point Detection Benchmark: An Extensive Benchmark Evaluation of Change Point Detection Algorithms on real-world data

Turing Change Point Detection Benchmark Welcome to the repository for the Turing Change Point Detection Benchmark, a benchmark evaluation of change po

The Alan Turing Institute 85 Dec 28, 2022
GBIM(Gesture-Based Interaction map)

手势交互地图 GBIM(Gesture-Based Interaction map),基于视觉深度神经网络的交互地图,通过电脑摄像头观察使用者的手势变化,进而控制地图进行简单的交互。网络使用PaddleX提供的轻量级模型PPYOLO Tiny以及MobileNet V3 small,使得整个模型大小约10MB左右,即使在CPU下也能快速定位和识别手势。

8 Feb 10, 2022
Code Repo for the ACL21 paper "Common Sense Beyond English: Evaluating and Improving Multilingual LMs for Commonsense Reasoning"

Common Sense Beyond English: Evaluating and Improving Multilingual LMs for Commonsense Reasoning This is the Github repository of our paper, "Common S

INK Lab @ USC 19 Nov 30, 2022
Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

Official repository of OFA. Paper: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework

OFA Sys 1.4k Jan 08, 2023
Nicely is a real-time Feedback and Intervention Program Depression is a prevalent issue across all age groups, socioeconomic classes, and cultural identities.

Nicely is a real-time Feedback and Intervention Program Depression is a prevalent issue across all age groups, socioeconomic classes, and cultural identities.

1 Jan 16, 2022
An updated version of virtual model making

Model-Swap-Face v2   这个项目是基于stylegan2 pSp制作的,比v1版本Model-Swap-Face在推理速度和图像质量上有一定提升。主要的功能是将虚拟模特进行环球不同区域的风格转换,目前转换器提供西欧模特、东亚模特和北非模特三种主流的风格样式,可帮我们实现生产资料零成

seeprettyface.com 62 Dec 09, 2022
Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks

Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks This is our Pytorch implementation for the paper: Zirui Zhu, Chen Gao, Xu C

Zirui Zhu 3 Dec 30, 2022
Udacity's CS101: Intro to Computer Science - Building a Search Engine

Udacity's CS101: Intro to Computer Science - Building a Search Engine All soluti

Phillip 0 Feb 26, 2022
Code for the CVPR 2021 paper: Understanding Failures of Deep Networks via Robust Feature Extraction

Welcome to Barlow Barlow is a tool for identifying the failure modes for a given neural network. To achieve this, Barlow first creates a group of imag

Sahil Singla 33 Dec 05, 2022
Rule-based Customer Segmentation

Rule-based Customer Segmentation Business Problem A game company wants to create level-based new customer definitions (personas) by using some feature

Cem Çaluk 2 Jan 03, 2022