Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

ADSPM: Attribute-Driven Spontaneous Motion in Unpaired Image Translation

Self-Supervised Pre-Training for Transformer-Based Person Re-Identification

PyTorch implementation for Convolutional Networks with Adaptive Inference Graphs

Videocaptioning.pytorch - A simple implementation of video captioning

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Pneumonia Detection using machine learning - with PyTorch

This is project is the implementation of the DeepShift: Towards Multiplication-Less Neural Networks paper

🌎 The Modern Declarative Data Flow Framework for the AI Empowered Generation.

Faune proche - Retrieval of Faune-France data near a google maps location

Tensorflow implementation of the paper "HumanGPS: Geodesic PreServing Feature for Dense Human Correspondences", CVPR 2021.

Code for paper Adaptively Aligned Image Captioning via Adaptive Attention Time

Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically.

Adaptive Dropblock Enhanced GenerativeAdversarial Networks for Hyperspectral Image Classification

This is the official code of our paper "Diversity-based Trajectory and Goal Selection with Hindsight Experience Relay" (PRICAI 2021)

Iris prediction model is used to classify iris species created julia's DecisionTree, DataFrames, JLD2, PlotlyJS and Statistics packages.

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

This is the official pytorch implementation of AutoDebias, an automatic debiasing method for recommendation.

Contenido del curso Bases de datos del DCC PUC versión 2021-2

Classify the disease status of a plant given an image of a passion fruit

Air Quality Prediction Using LSTM