A selection of State Of The Art research papers (and code) on human locomotion (pose + trajectory) prediction (forecasting)

Overview

Awesome-Human-Pose-Prediction

Version Awesome LastUpdated HitCount

A selection of State Of The Art research papers (and code) on human trajectory prediction (forecasting). Papers marked with [W] are workshop papers.

Maintainers: Karttikeya Mangalam

Contributing: Please feel free to pull requests to add new resources or suggest addditions or changes to the list. While proposing a new addition, please keep in mind the following principles:

  • The work has been accepted in a reputable peer reviewed publication venue.
  • An opensource link to the paper pdf is attached (as far as possible).
  • Code for the paper is linked (if made opensource by the authors).

Email: [email protected].{berkeley,stanford).edu

Datasets

  • Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments [Paper]
  • Stanford Drone Dataset (SDD): Learning Social Etiquette: Human Trajectory Understanding in Crowded Scenes [Paper] [Leaderboard]

Papers

As End in Itself

  • From Goals, Waypoints & Paths To Long Term Human Trajectory Forecasting [Paper]

  • It Is Not the Journey but the Destination: Endpoint Conditioned Trajectory Prediction [Paper]

  • Trajectron++: Dynamically-Feasible Trajectory Forecasting With Heterogeneous Data [Paper]

  • Interaction-Based Trajectory Prediction Over a Hybrid Traffic Graph [paper]

  • Map-Adaptive Goal-Based Trajectory Prediction [paper]

  • Interaction-Aware Trajectory Prediction based on a 3D Spatio-Temporal Tensor Representation using Convolutional–Recurrent Neural Networks [paper]

  • DROGON: A Trajectory Prediction Model based on Intention-Conditioned Behavior Reasoning [Paper]

  • Discrete Residual Flow for Probabilistic Pedestrian Behavior Prediction [Paper]

  • Social-VRNN: One-Shot Multi-modal Trajectory Prediction for Interacting Pedestrians [Paper]

  • Leveraging Neural Network Gradients within Trajectory Optimization for Proactive Human-Robot Interactions [Paper]

  • Social NCE: Contrastive Learning of Socially-aware Motion Representations [Paper]

  • Multimodal Deep Generative Models for Trajectory Prediction: A Conditional Variational Autoencoder Approach [Paper]

  • Risk-Sensitive Sequential Action Control with Multi-Modal Human Trajectory Forecasting for Safe Crowd-Robot Interaction [Paper]

  • Deep Learning for Vision-based Prediction: A Survey [Paper]

  • Probabilistic Crowd GAN: Multimodal Pedestrian Trajectory Prediction Using a Graph Vehicle-Pedestrian Attention Network [Paper]

  • Semantics for Robotic Mapping, Perception and Interaction: A Survey [Paper]

  • Benchmark for Evaluating Pedestrian Action Prediction[Paper]

  • Probabilistic Tracklet Scoring and Inpainting for Multiple Object Tracking [Paper]

  • Pedestrian Behavior Prediction via Multitask Learning and Categorical Interaction Modeling [Paper]

  • Graph-SIM: A Graph-based Spatiotemporal Interaction Modelling for Pedestrian Action Prediction [Paper]

  • Haar Wavelet based Block Autoregressive Flows for Trajectories [Paper]

  • Imitative Planning using Conditional Normalizing Flow [Paper]

  • TNT: Target-driveN Trajectory Prediction [Paper]

  • SimAug: Learning Robust Representations from Simulation for Trajectory Prediction [Paper]

  • SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints [Paper]

  • Social GAN: Socially Acceptable Trajectories With Generative Adversarial Networks [Paper]

  • DESIRE: Distant Future Prediction in Dynamic Scenes With Interacting Agents [Paper]

  • Predicting Whole Body Motion Trajectories using Conditional Neural Movement Primitives [Paper] [W]

  • Anticipating Human Intention for Full-Body Motion Prediction [Paper] [W]

  • Human Motion Prediction With Graph Neural Networks [Paper] [W]

  • Action-Agnostic Human Pose Forecasting [Paper]

  • Human Torso Pose Forecasting in the Real World [Paper]

  • Imitation Learning for Human Pose Prediction [Paper]

  • Disentangling Human Dynamics for Pedestrian Locomotion Forecasting with Noisy Supervision [Paper]

  • Predicting 3D Human Dynamics from Video [Paper]

  • Recurrent Network Models for Human Dynamics [Paper]

  • Structural-RNN: Deep Learning on Spatio-Temporal Graphs [Paper]

  • Learning Trajectory Dependencies for Human Motion Prediction [Paper]

  • Anticipating many futures: Online human motion prediction and generation for human-robot interaction [Paper]

  • Teaching Robots to Predict Human Motion [Paper]

  • Deep representation learning for human motion prediction and classification [Paper]

  • On human motion prediction using recurrent neural networks [Paper]

  • Few-Shot Human Motion Prediction via Meta-learning [Paper]

  • Efficient convolutional hierarchical autoencoder for human motion prediction [Paper]

  • Learning Human Motion Models for Long-term Predictions [Paper]

  • Long-Term Human Motion Prediction by Modeling Motion Context and Enhancing Motion Dynamic [Paper]

  • Context-aware Human Motion Prediction [Paper]

  • Adversarial Geometry-Aware Human Motion Prediction [Paper]

  • Convolutional Sequence to Sequence Model for Human Dynamics [Paper]

  • QuaterNet: A Quaternion-based Recurrent Model for Human Motion [Paper]

  • BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN [Paper]

  • Human Motion Modeling using DVGANs [Paper]

  • Human Motion Prediction using Semi-adaptable Neural Networks [Paper]

  • A Neural Temporal Model for Human Motion Prediction [Paper]

  • Modeling Human Motion with Quaternion-based Neural Networks [Paper]

  • Human Motion Prediction via Learning Local Structure Representations and Temporal Dependencies [Paper]

  • VRED: A Position-Velocity Recurrent Encoder-Decoder for Human Motion Prediction [Paper]

  • EAN: Error Attenuation Network for Long-term Human Motion Prediction [Paper]

  • Structured Prediction Helps 3D Human Motion Modelling [Paper]

  • Forecasting Human Dynamics from Static Images [Paper]

  • HP-GAN: Probabilistic 3D human motion prediction via GAN [Paper]

  • Learning Latent Representations of 3D Human Pose with Deep Neural Networks [Paper]

  • A Recurrent Variational Autoencoder for Human Motion Synthesis [Paper]

  • Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling [Paper]

  • Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control [Paper]

  • PISEP2: Pseudo Image Sequence Evolution based 3D Pose Prediction [Paper]

  • Human Motion Prediction via Spatio-Temporal Inpainting [Paper]

  • Spatiotemporal Co-attention Recurrent Neural Networks for Human-Skeleton Motion Prediction [Paper]

  • Human Pose Forecasting via Deep Markov Models [Paper]

  • Auto-Conditioned Recurrent Networks For Extended Complex Human Motion Synthesis [Paper]

  • Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network [Paper]

As a Subtask

  • The Pose Knows: Video Forecasting by Generating Pose Futures [Paper]
  • I-Planner: Intention-Aware Motion Planning Using Learning Based Human Motion Prediction [Paper]
  • Language2Pose: Natural Language Grounded Pose Forecasting [Paper]
  • Long-Term Video Generation of Multiple Futures Using Human Poses [Paper]
  • Predicting body movements for person identification under different walking conditions [Paper]
Owner
Karttikeya Manglam
PhD Student in Computer Vision @ BAIR, UC Berkeley.
Karttikeya Manglam
Pytorch-Swin-Unet-V2 - a modified version of Swin Unet based on Swin Transfomer V2

Swin Unet V2 Swin Unet V2 is a modified version of Swin Unet arxiv based on Swin

Chenxu Peng 26 Dec 03, 2022
Second Order Optimization and Curvature Estimation with K-FAC in JAX.

KFAC-JAX - Second Order Optimization with Approximate Curvature in JAX Installation | Quickstart | Documentation | Examples | Citing KFAC-JAX KFAC-JAX

DeepMind 90 Dec 22, 2022
Tool for working with Y-chromosome data from YFull and FTDNA

ycomp ycomp is a tool for working with Y-chromosome data from YFull and FTDNA. Run ycomp -h for information on how to use the program. Installation Th

Alexander Regueiro 2 Jun 18, 2022
FEDn is an open-source, modular and ML-framework agnostic framework for Federated Machine Learning

FEDn is an open-source, modular and ML-framework agnostic framework for Federated Machine Learning (FedML) developed and maintained by Scaleout Systems. FEDn enables highly scalable cross-silo and cr

Scaleout 75 Nov 09, 2022
naked is a Python tool which allows you to strip a model and only keep what matters for making predictions.

naked is a Python tool which allows you to strip a model and only keep what matters for making predictions. The result is a pure Python function with no third-party dependencies that you can simply c

Max Halford 24 Dec 20, 2022
DeepGNN is a framework for training machine learning models on large scale graph data.

DeepGNN Overview DeepGNN is a framework for training machine learning models on large scale graph data. DeepGNN contains all the necessary features in

Microsoft 45 Jan 01, 2023
Disentangled Lifespan Face Synthesis

Disentangled Lifespan Face Synthesis Project Page | Paper Demo on Colab Preparation Please follow this github to prepare the environments and dataset.

何森 50 Sep 20, 2022
SciPy fixes and extensions

scipyx SciPy is large library used everywhere in scientific computing. That's why breaking backwards-compatibility comes as a significant cost and is

Nico Schlömer 16 Jul 17, 2022
Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

Ren Tianhe 49 Nov 10, 2022
Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding (AAAI 2020) - PyTorch Implementation

Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding PyTorch implementation for the Scalable Attentive Sentence-Pair Modeling vi

Microsoft 25 Dec 02, 2022
The Adapter-Bot: All-In-One Controllable Conversational Model

The Adapter-Bot: All-In-One Controllable Conversational Model This is the implementation of the paper: The Adapter-Bot: All-In-One Controllable Conver

CAiRE 37 Nov 04, 2022
Painting app using Python machine learning and vision technology.

AI Painting App We are making an app that will track our hand and helps us to draw from that. We will be using the advance knowledge of Machine Learni

Badsha Laskar 3 Oct 03, 2022
EM-POSE 3D Human Pose Estimation from Sparse Electromagnetic Trackers.

EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers This repository contains the code to our paper published at ICCV 2021. For ques

Facebook Research 62 Dec 14, 2022
Adversarial Color Enhancement: Generating Unrestricted Adversarial Images by Optimizing a Color Filter

ACE Please find the preliminary version published at BMVC 2020 in the folder BMVC_version, and its extended journal version in Journal_version. Datase

28 Dec 25, 2022
Official repository of Semantic Image Matting

Semantic Image Matting This is the official repository of Semantic Image Matting (CVPR2021). Overview Natural image matting separates the foreground f

192 Dec 29, 2022
Compact Bidirectional Transformer for Image Captioning

Compact Bidirectional Transformer for Image Captioning Requirements Python 3.8 Pytorch 1.6 lmdb h5py tensorboardX Prepare Data Please use git clone --

YE Zhou 19 Dec 12, 2022
PyTorch implementation of Higher Order Recurrent Space-Time Transformer

Higher Order Recurrent Space-Time Transformer (HORST) This is the official PyTorch implementation of Higher Order Recurrent Space-Time Transformer. Th

13 Oct 18, 2022
DLWP: Deep Learning Weather Prediction

DLWP: Deep Learning Weather Prediction DLWP is a Python project containing data-

Kushal Shingote 3 Aug 14, 2022
This repository provides data for the VAW dataset as described in the CVPR 2021 paper titled "Learning to Predict Visual Attributes in the Wild"

Visual Attributes in the Wild (VAW) This repository provides data for the VAW dataset as described in the CVPR 2021 Paper: Learning to Predict Visual

Adobe Research 36 Dec 30, 2022
SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data Au

14 Nov 28, 2022