Waymo motion prediction challenge 2021: 3rd place solution

Overview

Waymo motion prediction challenge 2021: 3rd place solution

header

Team behind this solution:

  1. Artsiom Sanakoyeu [Homepage] [Twitter] [Telegram Channel] [LinkedIn]
  2. Stepan Konev [LinkedIn]
  3. Kirill Brodt [GitHub]

Dataset

Download datasets uncompressed/tf_example/{training,validation,testing}

Prerender

Change paths to input dataset and output folders

python prerender.py \
    --data /home/data/waymo/training \
    --out ./train
    
python prerender.py \
    --data /home/data/waymo/validation \
    --out ./dev \
    --use-vectorize \
    --n-shards 1
    
python prerender.py \
    --data /home/data/waymo/testing \
    --out ./test \
    --use-vectorize \
    --n-shards 1

Training

MODEL_NAME=xception71
python train.py \
    --train-data ./train \
    --dev-data ./dev \
    --save ./${MODEL_NAME} \
    --model ${MODEL_NAME} \
    --img-res 224 \
    --in-channels 25 \
    --time-limit 80 \
    --n-traj 6 \
    --lr 0.001 \
    --batch-size 48 \
    --n-epochs 120

Submit

python submit.py \
    --test-data ./test/ \
    --model-path ${MODEL_PATH_TO_JIT} \
    --save ${SAVE}

Visualize predictions

python visualize.py \
    --model ${MODEL_PATH_TO_JIT} \
    --data ${DATA_PATH} \
    --save ./viz

Citation

If you find our work useful, please cite it as:

@article{konev2021motioncnn,
  title={MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving},
  author={Konev, Stepan and Brodt, Kirill and Sanakoyeu, Artsiom},
  year={2021}
}

Related repos

Comments
  • Metrics

    Metrics

    Hi! @kbrodt Thanks for sharing this great code!

    Where are the codes of the evaluation metrics (for example: ADE, FDE, minADE, minFDE and so on)? Or where can I find it?

    Looking forward to your reply!

    opened by chx-Github 9
  • Regarding the training epochs

    Regarding the training epochs

    Thanks for sharing this awesome codes!

    Is it necessary to train the model for 120 epochs? Since there are more than 1M training samples. Can you share some performance during the training progress? Such as the performance with 30epochs, 60 epochs, 90 epochs? Since I trained it for several epochs but the loss is still very large.

    To double check the training process, can you share how many training samples for each epoch?

    Thanks so much!

    opened by FutureOpenAI 8
  • Loss

    Loss

    Hi! I tried your method and I observed that in training, l2loss and log_softmax have so large difference. so my network does not learn multimodal tracks, only one best track is fitted. Do you have any solution?

    opened by zsgj-Xxx 7
  • Angle conversion error at prerender.py

    Angle conversion error at prerender.py

    Thank you for amazing works.

    I found the conversion error at prerender.py

    https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L706 https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L707

    These two lines has conversion error.

    In order to convert coordinate properly, it should be changed like this tmp[3] = other_v_yaw - ANGLE tmp[4] = other_bbox_yaw - ANGLE

    opened by KyuhwanYeon 3
  • Question About the waymo dataset lane type

    Question About the waymo dataset lane type

    Hi, @kbrodt,

    Thank you for your great work. I am new to this area, so I have a question regarding to the lane type of waymo dataset. In general, the lanes in waymo dataset, can be broadly categorized into 6: lanecenters, roadlines, stopsign, speedbump, roadedge and crosswalks. So, is the lane centers are just the the center of each line in which the vehicles can drive on? Theoretically, a car's center should be on the lanecenters if this car is staying in this lane. Is my interpretation right? Also, is the lanecenters in waymo the same as the centerlines in Argoverse Dataset? Are you familiar with that dataset? I am looking forward to your reply. Thank you in advance.

    opened by SwagJ 2
  • Vehicles driving on the same lane

    Vehicles driving on the same lane

    Hello!

    Thanks for sharing you work!

    I have one questions: from the motion dataset, is that possible to know if two vehicles are driving on the same lane? I notice that there is a column called "roadgraph_samples/id" but I am not sure whether that means the lane ID.

    Thanks.

    opened by 18627242758 2
  • Congratulations

    Congratulations

    Привет Кирилл,

    Я вижу что ты разбираешься в машинном обучении. Я хотел бы сконнектиться и перенять опыт, если возможно. Как с тобой можно связаться?

    Дмитрий

    opened by Rendok 1
  • I can't download the dataset

    I can't download the dataset

    The infomation is showed as below: Additional permissions required to list objects in this bucket. Ask a bucket owner to grant you 'storage.objects.list' permission.

    opened by fengsky401 1
  • socket.gaierror: [Errno -2] Name or service not known

    socket.gaierror: [Errno -2] Name or service not known

    After running the following command :

    (venvpy37cu10) [[email protected] project]$ python train.py --train-data ./train --dev-data ./dev --save ./xception71 --model xception71 --img-res 224 --in-channels 25 --time-limit 80 --n-traj 6 --lr 0.001 --batch-size 48 --n-epochs 120

    Below error has occurred :

    Screen Shot 2022-07-30 at 10 59 36 PM
    opened by rohansd 1
  • magic_const and shift

    magic_const and shift

    Hi,

    Thanks for your open-source work.

    I don't understand the magic_const and shift in rasterize() function prerender.py.

    Would you please give some explanation?

    opened by ShoufaChen 2
  • Preprocessing issue

    Preprocessing issue

    I tried running with all requirements , it keeps on reading the records but when it tries to write down .....here it fails , please help me to proceed further

    [email protected]:/app/waymo-adas-main/waymo-motion-prediction-2021# python3 prerender.py --data /app/waymo-adas-main/waymo-dataset/original/validation/ --out /app/waymo-adas-main/data/train1
    False
    Namespace(data='/app/waymo-adas-main/waymo-dataset/original/validation/', each=0, n_jobs=20, n_shards=8, no_valid=False, out='/app/waymo-adas-main/data/train1', use_vectorize=False)
    1215it [00:28, 42.50it/s]
      0%|                                                                                                                                                                                   | 0/1215 [00:00<?, ?it/s]
    multiprocessing.pool.RemoteTraceback:
    """
    Traceback (most recent call last):
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "prerender.py", line 731, in merge
        parsed = tf.io.parse_single_example(data, features_description)
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 58, in quick_execute
        tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    """
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "prerender.py", line 836, in <module>
        main()
      File "prerender.py", line 832, in main
        r.get()
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
        raise self._value
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    
    opened by karanveersingh5623 9
  • About preprocessing

    About preprocessing

    Congrats! Actually I have a question which has confused me a few days. When we rasterize the map information, why we need to shift and rotate the local coordinate system that the target agent is located at a specific location? I konw that many papers has used this method like buliding a relative coordinate system and the center is target agent. why? and what is the meaning of "to eliminate the redundant degrees of freedom" in your report? Thank you in advance!

    opened by zyandtom 1
Releases(0.1)
This repository contains a set of codes to run (i.e., train, perform inference with, evaluate) a diarization method called EEND-vector-clustering.

EEND-vector clustering The EEND-vector clustering (End-to-End-Neural-Diarization-vector clustering) is a speaker diarization framework that integrates

45 Dec 26, 2022
The Body Part Regression (BPR) model translates the anatomy in a radiologic volume into a machine-interpretable form.

Copyright © German Cancer Research Center (DKFZ), Division of Medical Image Computing (MIC). Please make sure that your usage of this code is in compl

MIC-DKFZ 40 Dec 18, 2022
FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Detectron is deprecated. Please see detectron2, a ground-up rewrite of Detectron in PyTorch. Detectron Detectron is Facebook AI Research's software sy

Facebook Research 25.5k Jan 07, 2023
Very large and sparse networks appear often in the wild and present unique algorithmic opportunities and challenges for the practitioner

Sparse network learning with snlpy Very large and sparse networks appear often in the wild and present unique algorithmic opportunities and challenges

Andrew Stolman 1 Apr 30, 2021
Music library streaming app written in Flask & VueJS

djtaytay This is a little toy app made to explore Vue, brush up on my Python, and make a remote music collection accessable through a web interface. I

Ryan Tasson 6 May 27, 2022
Optical Character Recognition + Instance Segmentation for russian and english languages

Распознавание рукописного текста в школьных тетрадях Соревнование, проводимое в рамках олимпиады НТО, разработанное Сбером. Платформа ODS. Результаты

Gerasimov Maxim 21 Dec 19, 2022
Paddle pit - Rethinking Spatial Dimensions of Vision Transformers

基于Paddle实现PiT ——Rethinking Spatial Dimensions of Vision Transformers,arxiv 官方原版代

Hongtao Wen 4 Jan 15, 2022
Code for pre-training CharacterBERT models (as well as BERT models).

Pre-training CharacterBERT (and BERT) This is a repository for pre-training BERT and CharacterBERT. DISCLAIMER: The code was largely adapted from an o

Hicham EL BOUKKOURI 31 Dec 05, 2022
A Deep Reinforcement Learning Framework for Stock Market Trading

DQN-Trading This is a framework based on deep reinforcement learning for stock market trading. This project is the implementation code for the two pap

61 Jan 01, 2023
Pytorch port of Google Research's LEAF Audio paper

leaf-audio-pytorch Pytorch port of Google Research's LEAF Audio paper published at ICLR 2021. This port is not completely finished, but the Leaf() fro

Dennis Fedorishin 80 Oct 31, 2022
MoveNetを用いたPythonでの姿勢推定のデモ

MoveNet-Python-Example MoveNetのPythonでの動作サンプルです。 ONNXに変換したモデルも同梱しています。変換自体を試したい方はMoveNet_tf2onnx.ipynbを使用ください。 2021/08/24時点でTensorFlow Hubで提供されている以下モデ

KazuhitoTakahashi 38 Dec 17, 2022
Serverless proxy for Spark cluster

Hydrosphere Mist Hydrosphere Mist is a serverless proxy for Spark cluster. Mist provides a new functional programming framework and deployment model f

hydrosphere.io 317 Dec 01, 2022
Next-gen Rowhammer fuzzer that uses non-uniform, frequency-based patterns.

Blacksmith Rowhammer Fuzzer This repository provides the code accompanying the paper Blacksmith: Scalable Rowhammering in the Frequency Domain that is

Computer Security Group @ ETH Zurich 173 Nov 16, 2022
A PyTorch implementation of EfficientDet.

A PyTorch impl of EfficientDet faithful to the original Google impl w/ ported weights

Ross Wightman 1.4k Jan 07, 2023
Receptive Field Block Net for Accurate and Fast Object Detection, ECCV 2018

Receptive Field Block Net for Accurate and Fast Object Detection By Songtao Liu, Di Huang, Yunhong Wang Updatas (2021/07/23): YOLOX is here!, stronger

Liu Songtao 1.4k Dec 21, 2022
CVPR2022 paper "Dense Learning based Semi-Supervised Object Detection"

[CVPR2022] DSL: Dense Learning based Semi-Supervised Object Detection DSL is the first work on Anchor-Free detector for Semi-Supervised Object Detecti

Bhchen 69 Dec 08, 2022
Simple codebase for flexible neural net training

neural-modular Simple codebase for flexible neural net training. Allows for seamless exchange of models, dataset, and optimizers. Uses hydra for confi

Jannik Kossen 7 Apr 05, 2022
PESTO: Switching Point based Dynamic and Relative Positional Encoding for Code-Mixed Languages

PESTO: Switching Point based Dynamic and Relative Positional Encoding for Code-Mixed Languages Abstract NLP applications for code-mixed (CM) or mix-li

Mohsin Ali, Mohammed 1 Nov 12, 2021
Junction Tree Variational Autoencoder for Molecular Graph Generation (ICML 2018)

Junction Tree Variational Autoencoder for Molecular Graph Generation Official implementation of our Junction Tree Variational Autoencoder https://arxi

Wengong Jin 418 Jan 07, 2023
Unofficial PyTorch implementation of Google AI's VoiceFilter system

VoiceFilter Note from Seung-won (2020.10.25) Hi everyone! It's Seung-won from MINDs Lab, Inc. It's been a long time since I've released this open-sour

MINDs Lab 883 Jan 07, 2023