Waymo motion prediction challenge 2021: 3rd place solution

Overview

Waymo motion prediction challenge 2021: 3rd place solution

header

Team behind this solution:

  1. Artsiom Sanakoyeu [Homepage] [Twitter] [Telegram Channel] [LinkedIn]
  2. Stepan Konev [LinkedIn]
  3. Kirill Brodt [GitHub]

Dataset

Download datasets uncompressed/tf_example/{training,validation,testing}

Prerender

Change paths to input dataset and output folders

python prerender.py \
    --data /home/data/waymo/training \
    --out ./train
    
python prerender.py \
    --data /home/data/waymo/validation \
    --out ./dev \
    --use-vectorize \
    --n-shards 1
    
python prerender.py \
    --data /home/data/waymo/testing \
    --out ./test \
    --use-vectorize \
    --n-shards 1

Training

MODEL_NAME=xception71
python train.py \
    --train-data ./train \
    --dev-data ./dev \
    --save ./${MODEL_NAME} \
    --model ${MODEL_NAME} \
    --img-res 224 \
    --in-channels 25 \
    --time-limit 80 \
    --n-traj 6 \
    --lr 0.001 \
    --batch-size 48 \
    --n-epochs 120

Submit

python submit.py \
    --test-data ./test/ \
    --model-path ${MODEL_PATH_TO_JIT} \
    --save ${SAVE}

Visualize predictions

python visualize.py \
    --model ${MODEL_PATH_TO_JIT} \
    --data ${DATA_PATH} \
    --save ./viz

Citation

If you find our work useful, please cite it as:

@article{konev2021motioncnn,
  title={MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving},
  author={Konev, Stepan and Brodt, Kirill and Sanakoyeu, Artsiom},
  year={2021}
}

Related repos

Comments
  • Metrics

    Metrics

    Hi! @kbrodt Thanks for sharing this great code!

    Where are the codes of the evaluation metrics (for example: ADE, FDE, minADE, minFDE and so on)? Or where can I find it?

    Looking forward to your reply!

    opened by chx-Github 9
  • Regarding the training epochs

    Regarding the training epochs

    Thanks for sharing this awesome codes!

    Is it necessary to train the model for 120 epochs? Since there are more than 1M training samples. Can you share some performance during the training progress? Such as the performance with 30epochs, 60 epochs, 90 epochs? Since I trained it for several epochs but the loss is still very large.

    To double check the training process, can you share how many training samples for each epoch?

    Thanks so much!

    opened by FutureOpenAI 8
  • Loss

    Loss

    Hi! I tried your method and I observed that in training, l2loss and log_softmax have so large difference. so my network does not learn multimodal tracks, only one best track is fitted. Do you have any solution?

    opened by zsgj-Xxx 7
  • Angle conversion error at prerender.py

    Angle conversion error at prerender.py

    Thank you for amazing works.

    I found the conversion error at prerender.py

    https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L706 https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L707

    These two lines has conversion error.

    In order to convert coordinate properly, it should be changed like this tmp[3] = other_v_yaw - ANGLE tmp[4] = other_bbox_yaw - ANGLE

    opened by KyuhwanYeon 3
  • Question About the waymo dataset lane type

    Question About the waymo dataset lane type

    Hi, @kbrodt,

    Thank you for your great work. I am new to this area, so I have a question regarding to the lane type of waymo dataset. In general, the lanes in waymo dataset, can be broadly categorized into 6: lanecenters, roadlines, stopsign, speedbump, roadedge and crosswalks. So, is the lane centers are just the the center of each line in which the vehicles can drive on? Theoretically, a car's center should be on the lanecenters if this car is staying in this lane. Is my interpretation right? Also, is the lanecenters in waymo the same as the centerlines in Argoverse Dataset? Are you familiar with that dataset? I am looking forward to your reply. Thank you in advance.

    opened by SwagJ 2
  • Vehicles driving on the same lane

    Vehicles driving on the same lane

    Hello!

    Thanks for sharing you work!

    I have one questions: from the motion dataset, is that possible to know if two vehicles are driving on the same lane? I notice that there is a column called "roadgraph_samples/id" but I am not sure whether that means the lane ID.

    Thanks.

    opened by 18627242758 2
  • Congratulations

    Congratulations

    Привет Кирилл,

    Я вижу что ты разбираешься в машинном обучении. Я хотел бы сконнектиться и перенять опыт, если возможно. Как с тобой можно связаться?

    Дмитрий

    opened by Rendok 1
  • I can't download the dataset

    I can't download the dataset

    The infomation is showed as below: Additional permissions required to list objects in this bucket. Ask a bucket owner to grant you 'storage.objects.list' permission.

    opened by fengsky401 1
  • socket.gaierror: [Errno -2] Name or service not known

    socket.gaierror: [Errno -2] Name or service not known

    After running the following command :

    (venvpy37cu10) [[email protected] project]$ python train.py --train-data ./train --dev-data ./dev --save ./xception71 --model xception71 --img-res 224 --in-channels 25 --time-limit 80 --n-traj 6 --lr 0.001 --batch-size 48 --n-epochs 120

    Below error has occurred :

    Screen Shot 2022-07-30 at 10 59 36 PM
    opened by rohansd 1
  • magic_const and shift

    magic_const and shift

    Hi,

    Thanks for your open-source work.

    I don't understand the magic_const and shift in rasterize() function prerender.py.

    Would you please give some explanation?

    opened by ShoufaChen 2
  • Preprocessing issue

    Preprocessing issue

    I tried running with all requirements , it keeps on reading the records but when it tries to write down .....here it fails , please help me to proceed further

    [email protected]:/app/waymo-adas-main/waymo-motion-prediction-2021# python3 prerender.py --data /app/waymo-adas-main/waymo-dataset/original/validation/ --out /app/waymo-adas-main/data/train1
    False
    Namespace(data='/app/waymo-adas-main/waymo-dataset/original/validation/', each=0, n_jobs=20, n_shards=8, no_valid=False, out='/app/waymo-adas-main/data/train1', use_vectorize=False)
    1215it [00:28, 42.50it/s]
      0%|                                                                                                                                                                                   | 0/1215 [00:00<?, ?it/s]
    multiprocessing.pool.RemoteTraceback:
    """
    Traceback (most recent call last):
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "prerender.py", line 731, in merge
        parsed = tf.io.parse_single_example(data, features_description)
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 58, in quick_execute
        tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    """
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "prerender.py", line 836, in <module>
        main()
      File "prerender.py", line 832, in main
        r.get()
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
        raise self._value
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    
    opened by karanveersingh5623 9
  • About preprocessing

    About preprocessing

    Congrats! Actually I have a question which has confused me a few days. When we rasterize the map information, why we need to shift and rotate the local coordinate system that the target agent is located at a specific location? I konw that many papers has used this method like buliding a relative coordinate system and the center is target agent. why? and what is the meaning of "to eliminate the redundant degrees of freedom" in your report? Thank you in advance!

    opened by zyandtom 1
Releases(0.1)
External Attention Network

Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks paper : https://arxiv.org/abs/2105.02358 Jittor code will come soon

MenghaoGuo 357 Dec 11, 2022
A Survey on Deep Learning Technique for Video Segmentation

A Survey on Deep Learning Technique for Video Segmentation A Survey on Deep Learning Technique for Video Segmentation Wenguan Wang, Tianfei Zhou, Fati

Tianfei Zhou 112 Dec 12, 2022
SVG Icon processing tool for C++

BAWR This is a tool to automate the icons generation from sets of svg files into fonts and atlases. The main purpose of this tool is to add it to the

Frank David Martínez M 66 Dec 14, 2022
[AI6122] Text Data Management & Processing

[AI6122] Text Data Management & Processing is an elective course of MSAI, SCSE, NTU, Singapore. The repository corresponds to the AI6122 of Semester 1, AY2021-2022, starting from 08/2021. The instruc

HT. Li 1 Jan 17, 2022
This repository contains all data used for writing a research paper Multiple Object Trackers in OpenCV: A Benchmark, presented in ISIE 2021 conference in Kyoto, Japan.

OpenCV-Multiple-Object-Tracking Python is version 3.6.7 to install opencv: pip uninstall opecv-python pip uninstall opencv-contrib-python pip install

6 Dec 19, 2021
A python tutorial on bayesian modeling techniques (PyMC3)

Bayesian Modelling in Python Welcome to "Bayesian Modelling in Python" - a tutorial for those interested in learning how to apply bayesian modelling t

Mark Regan 2.4k Jan 06, 2023
A Strong Baseline for Image Semantic Segmentation

A Strong Baseline for Image Semantic Segmentation Introduction This project is an open source semantic segmentation toolbox based on PyTorch. It is ba

Clark He 49 Sep 20, 2022
This GitHub repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.'

About Repository This repository contains code used for plots in NeurIPS 2021 paper 'Stochastic Multi-Armed Bandits with Control Variates.' About Code

Arun Verma 1 Nov 09, 2021
Implementation of light baking system for ray tracing based on Activision's UberBake

Vulkan Light Bakary MSU Graphics Group Student's Diploma Project Treefonov Andrey [GitHub] [LinkedIn] Project Goal The goal of the project is to imple

Andrey Treefonov 7 Dec 27, 2022
Fully Convolutional DenseNets for semantic segmentation.

Introduction This repo contains the code to train and evaluate FC-DenseNets as described in The One Hundred Layers Tiramisu: Fully Convolutional Dense

485 Nov 26, 2022
toroidal - a lightweight transformer library for PyTorch

toroidal - a lightweight transformer library for PyTorch Toroidal transformers are of smaller size and lower weight than the more common E-I types. Th

MathInf GmbH 64 Jan 07, 2023
Construct a neural network frame by Numpy

本项目的CSDN博客链接:https://blog.csdn.net/weixin_41578567/article/details/111482022 1. 概览 本项目主要用于神经网络的学习,通过基于numpy的实现,了解神经网络底层前向传播、反向传播以及各类优化器的原理。 该项目目前已实现的功

24 Jan 22, 2022
a short visualisation script for pyvideo data

PyVideo Speakers A CLI that visualises repeat speakers from events listed in https://github.com/pyvideo/data Not terribly efficient, but you know. Ins

Katie McLaughlin 3 Nov 24, 2021
Code for Domain Adaptive Video Segmentation via Temporal Consistency Regularization in ICCV 2021

Domain Adaptive Video Segmentation via Temporal Consistency Regularization Updates 08/2021: check out our domain adaptation for sematic segmentation p

36 Dec 12, 2022
Source codes for Improved Few-Shot Visual Classification (CVPR 2020), Enhancing Few-Shot Image Classification with Unlabelled Examples

Source codes for Improved Few-Shot Visual Classification (CVPR 2020), Enhancing Few-Shot Image Classification with Unlabelled Examples (WACV 2022) and Beyond Simple Meta-Learning: Multi-Purpose Model

PLAI Group at UBC 42 Dec 06, 2022
Official PyTorch implementation of PICCOLO: Point-Cloud Centric Omnidirectional Localization (ICCV 2021)

Official PyTorch implementation of PICCOLO: Point-Cloud Centric Omnidirectional Localization (ICCV 2021)

16 Nov 19, 2022
An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks

AnalyticMesh Analytic Marching is an exact meshing solution from neural networks. Compared to standard methods, it completely avoids geometric and top

Karbo 45 Dec 21, 2022
SSD-based Object Detection in PyTorch

SSD-based Object Detection in PyTorch 서강대학교 현대모비스 SW 프로그램에서 진행한 인공지능 프로젝트입니다. Jetson nano를 이용해 pre-trained network를 fine tuning시켜 차량 및 신호등 인식을 구현하였습니다

Haneul Kim 1 Nov 16, 2021
Source code of SIGIR2021 Paper 'One Chatbot Per Person: Creating Personalized Chatbots based on Implicit Profiles'

DHAP Source code of SIGIR2021 Long Paper: One Chatbot Per Person: Creating Personalized Chatbots based on Implicit User Profiles . Preinstallation Fir

ZYMa 32 Dec 06, 2022
smc.covid is an R package related to the paper A sequential Monte Carlo approach to estimate a time varying reproduction number in infectious disease models: the COVID-19 case by Storvik et al

smc.covid smc.covid is an R package related to the paper A sequential Monte Carlo approach to estimate a time varying reproduction number in infectiou

0 Oct 15, 2021