A selection of State Of The Art research papers (and code) on human locomotion (pose + trajectory) prediction (forecasting)

Overview

Awesome-Human-Pose-Prediction

Version Awesome LastUpdated HitCount

A selection of State Of The Art research papers (and code) on human trajectory prediction (forecasting). Papers marked with [W] are workshop papers.

Maintainers: Karttikeya Mangalam

Contributing: Please feel free to pull requests to add new resources or suggest addditions or changes to the list. While proposing a new addition, please keep in mind the following principles:

  • The work has been accepted in a reputable peer reviewed publication venue.
  • An opensource link to the paper pdf is attached (as far as possible).
  • Code for the paper is linked (if made opensource by the authors).

Email: [email protected].{berkeley,stanford).edu

Datasets

  • Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments [Paper]
  • Stanford Drone Dataset (SDD): Learning Social Etiquette: Human Trajectory Understanding in Crowded Scenes [Paper] [Leaderboard]

Papers

As End in Itself

  • From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting [Paper]

  • It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction [Paper]

  • Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data [Paper]

  • Interaction-Based Trajectory Prediction Over a Hybrid Traffic Graph [paper]

  • Map-Adaptive Goal-Based Trajectory Prediction [paper]

  • Interaction-Aware Trajectory Prediction based on a 3D Spatio-Temporal Tensor Representation using Convolutional–Recurrent Neural Networks [paper]

  • DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning [Paper]

  • Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction [Paper]

  • Social-VRNN: One-Shot Multi-modal Trajectory Prediction for Interacting Pedestrians [Paper]

  • Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions [Paper]

  • Social NCE: Contrastive Learning of Socially-aware Motion Representations [Paper]

  • Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach [Paper]

  • Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction [Paper]

  • Deep Learning for Vision-based Prediction: A Survey [Paper]

  • Probabilistic Crowd GAN: Multimodal Pedestrian Trajectory Prediction Using a Graph Vehicle-Pedestrian Attention Network [Paper]

  • Semantics for Robotic Mapping, Perception and Interaction: A Survey [Paper]

  • Benchmark for Evaluating Pedestrian Action Prediction[Paper]

  • Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking [Paper]

  • Pedestrian Behavior Prediction via Multitask Learning and Categorical Interaction Modeling [Paper]

  • Graph-SIM: A Graph-based Spatiotemporal Interaction Modelling for Pedestrian Action Prediction [Paper]

  • Haar Wavelet based Block Autoregressive Flows for Trajectories [Paper]

  • Imitative Planning using Conditional Normalizing Flow [Paper]

  • TNT: Target-driveN Trajectory Prediction [Paper]

  • SimAug: Learning Robust Representations from Simulation for Trajectory Prediction [Paper]

  • SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints [Paper]

  • Social GAN: Socially Acceptable Trajectories With Generative Adversarial Networks [Paper]

  • DESIRE: Distant Future Prediction in Dynamic Scenes With Interacting Agents [Paper]

  • Predicting Whole Body Motion Trajectories using Conditional Neural Movement Primitives [Paper] [W]

  • Anticipating Human Intention for Full-Body Motion Prediction [Paper] [W]

  • Human Motion Prediction With Graph Neural Networks [Paper] [W]

  • Action-Agnostic Human Pose Forecasting [Paper]

  • Human Torso Pose Forecasting in the Real World [Paper]

  • Imitation Learning for Human Pose Prediction [Paper]

  • Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision [Paper]

  • Predicting 3D Human Dynamics from Video [Paper]

  • Recurrent Network Models for Human Dynamics [Paper]

  • Structural-RNN: Deep Learning on Spatio-Temporal Graphs [Paper]

  • Learning Trajectory Dependencies for Human Motion Prediction [Paper]

  • Anticipating many futures: Online human motion prediction and generation for human-robot interaction [Paper]

  • Teaching Robots to Predict Human Motion [Paper]

  • Deep representation learning for human motion prediction and classification [Paper]

  • On human motion prediction using recurrent neural networks [Paper]

  • Few-Shot Human Motion Prediction via Meta-learning [Paper]

  • Efficient convolutional hierarchical autoencoder for human motion prediction [Paper]

  • Learning Human Motion Models for Long-term Predictions [Paper]

  • Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic [Paper]

  • Context-aware Human Motion Prediction [Paper]

  • Adversarial Geometry-Aware Human Motion Prediction [Paper]

  • Convolutional Sequence to Sequence Model for Human Dynamics [Paper]

  • QuaterNet: A Quaternion-based Recurrent Model for Human Motion [Paper]

  • BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN [Paper]

  • Human Motion Modeling using DVGANs [Paper]

  • Human Motion Prediction using Semi-adaptable Neural Networks [Paper]

  • A Neural Temporal Model for Human Motion Prediction [Paper]

  • Modeling Human Motion with Quaternion-based Neural Networks [Paper]

  • Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies [Paper]

  • VRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction [Paper]

  • EAN: Error Attenuation Network for Long-term Human Motion Prediction [Paper]

  • Structured Prediction Helps 3D Human Motion Modelling [Paper]

  • Forecasting Human Dynamics from Static Images [Paper]

  • HP-GAN: Probabilistic 3D human motion prediction via GAN [Paper]

  • Learning Latent Representations of 3D Human Pose with Deep Neural Networks [Paper]

  • A Recurrent Variational Autoencoder for Human Motion Synthesis [Paper]

  • Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling [Paper]

  • Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control [Paper]

  • PISEP2: Pseudo Image Sequence Evolution based 3D Pose Prediction [Paper]

  • Human Motion Prediction via Spatio-Temporal Inpainting [Paper]

  • Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction [Paper]

  • Human Pose Forecasting via Deep Markov Models [Paper]

  • Auto-Conditioned Recurrent Networks For Extended Complex Human Motion Synthesis [Paper]

  • Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network [Paper]

As a Subtask

  • The Pose Knows: Video Forecasting by Generating Pose Futures [Paper]
  • I-Planner: Intention-Aware Motion Planning Using Learning Based Human Motion Prediction [Paper]
  • Language2Pose: Natural Language Grounded Pose Forecasting [Paper]
  • Long-Term Video Generation of Multiple Futures Using Human Poses [Paper]
  • Predicting body movements for person identification under different walking conditions [Paper]
Owner
Karttikeya Manglam
PhD Student in Computer Vision @ BAIR, UC Berkeley.
Karttikeya Manglam
A general 3D Object Detection codebase in PyTorch.

Det3D is the first 3D Object Detection toolbox which provides off the box implementations of many 3D object detection algorithms such as PointPillars, SECOND, PIXOR, etc, as well as state-of-the-art

Benjin Zhu 1.4k Jan 05, 2023
Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021)

Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion (CVPR 2021) This repository is for BAAF-Net introduce

90 Dec 29, 2022
To build a regression model to predict the concrete compressive strength based on the different features in the training data.

Cement-Strength-Prediction Problem Statement To build a regression model to predict the concrete compressive strength based on the different features

Ashish Kumar 4 Jun 11, 2022
catch-22: CAnonical Time-series CHaracteristics

catch22 - CAnonical Time-series CHaracteristics About catch22 is a collection of 22 time-series features coded in C that can be run from Python, R, Ma

Carl H Lubba 229 Oct 21, 2022
A deep learning tabular classification architecture inspired by TabTransformer with integrated gated multilayer perceptron.

The GatedTabTransformer. A deep learning tabular classification architecture inspired by TabTransformer with integrated gated multilayer perceptron. C

Radi Cho 60 Dec 15, 2022
On Out-of-distribution Detection with Energy-based Models

On Out-of-distribution Detection with Energy-based Models This repository contains the code for the experiments conducted in the paper On Out-of-distr

Sven 19 Aug 07, 2022
Code for "Optimizing risk-based breast cancer screening policies with reinforcement learning"

Tempo: Optimizing risk-based breast cancer screening policies with reinforcement learning Introduction This repository was used to develop Tempo, as d

Adam Yala 12 Oct 11, 2022
TensorFlow (Python API) implementation of Neural Style

neural-style-tf This is a TensorFlow implementation of several techniques described in the papers: Image Style Transfer Using Convolutional Neural Net

Cameron 3.1k Jan 02, 2023
RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation (CIKM'17)

RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation This is the implementation of RATE: Overcoming Noise and Spar

Yu Zhang 5 Feb 10, 2022
Keras implementation of Normalizer-Free Networks and SGD - Adaptive Gradient Clipping

Keras implementation of Normalizer-Free Networks and SGD - Adaptive Gradient Clipping

Yam Peleg 63 Sep 21, 2022
Neural Surface Maps

Neural Surface Maps Official implementation of Neural Surface Maps - Luca Morreale, Noam Aigerman, Vladimir Kim, Niloy J. Mitra [Paper] [Project Page]

Luca Morreale 49 Dec 13, 2022
Multi-Objective Reinforced Active Learning

Multi-Objective Reinforced Active Learning Dependencies wandb tqdm pytorch = 1.7.0 numpy = 1.20.0 scipy = 1.1.0 pycolab == 1.2 Weights and Biases O

Markus Peschl 6 Nov 19, 2022
[Preprint] "Chasing Sparsity in Vision Transformers: An End-to-End Exploration" by Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, Zhangyang Wang

Chasing Sparsity in Vision Transformers: An End-to-End Exploration Codes for [Preprint] Chasing Sparsity in Vision Transformers: An End-to-End Explora

VITA 64 Dec 08, 2022
Highway networks implemented in PyTorch.

PyTorch Highway Networks Highway networks implemented in PyTorch. Just the MNIST example from PyTorch hacked to work with Highway layers. Todo Make th

Conner Vercellino 56 Dec 14, 2022
DirectVoxGO reconstructs a scene representation from a set of calibrated images capturing the scene.

DirectVoxGO reconstructs a scene representation from a set of calibrated images capturing the scene. We achieve NeRF-comparable novel-view synthesis quality with super-fast convergence.

sunset 709 Dec 31, 2022
Neural Contours: Learning to Draw Lines from 3D Shapes (CVPR2020)

Neural Contours: Learning to Draw Lines from 3D Shapes This repository contains the PyTorch implementation for CVPR 2020 Paper "Neural Contours: Learn

93 Dec 16, 2022
Set of models for classifcation of 3D volumes

Classification models 3D Zoo - Keras and TF.Keras This repository contains 3D variants of popular CNN models for classification like ResNets, DenseNet

69 Dec 28, 2022
Framework to build and train RL algorithms

RayLink RayLink is a RL framework used to build and train RL algorithms. RayLink was used to build a RL framework, and tested in a large-scale multi-a

Bytedance Inc. 32 Oct 07, 2022
Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks

Bayesian-Torch is a library of neural network layers and utilities extending the core of PyTorch to enable the user to perform stochastic variational inference in Bayesian deep neural networks. Bayes

Intel Labs 210 Jan 04, 2023
Code in PyTorch for the convex combination linear IAF and the Householder Flow, J.M. Tomczak & M. Welling

VAE with Volume-Preserving Flows This is a PyTorch implementation of two volume-preserving flows as described in the following papers: Tomczak, J. M.,

Jakub Tomczak 87 Dec 26, 2022