Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

PyTorch implementation of Towards Accurate Alignment in Real-time 3D Hand-Mesh Reconstruction (ICCV 2021).

This repo is duplication of jwyang/faster-rcnn.pytorch

PyTorch implementation of Higher Order Recurrent Space-Time Transformer

Codebase for the self-supervised goal reaching benchmark introduced in the LEXA paper

A python library for time-series smoothing and outlier detection in a vectorized way.

Implementation of DocFormer: End-to-End Transformer for Document Understanding, a multi-modal transformer based architecture for the task of Visual Document Understanding (VDU)

A Pytorch implementation of the multi agent deep deterministic policy gradients (MADDPG) algorithm

PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

This repo holds code for TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

Bayesian Meta-Learning Through Variational Gaussian Processes

CLOOB training (JAX) and inference (JAX and PyTorch)

An open source Python package for plasma science that is under development

KE-Dialogue: Injecting knowledge graph into a fully end-to-end dialogue system.

🔥 TensorFlow Code for technical report: "YOLOv3: An Incremental Improvement"

Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems

Weakly Supervised Segmentation with Tensorflow. Implements instance segmentation as described in Simple Does It: Weakly Supervised Instance and Semantic Segmentation, by Khoreva et al. (CVPR 2017).

PyTorch implementation of Munchausen Reinforcement Learning based on DQN and SAC. Handles discrete and continuous action spaces

Source code for the ACL-IJCNLP 2021 paper entitled "T-DNA: Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation" by Shizhe Diao et al.

Machine learning and Deep learning models, deploy on telegram (the best social media)

An end-to-end library for editing and rendering motion of 3D characters with deep learning [SIGGRAPH 2020]