Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color

'Aligned mixture of latent dynamical systems' (amLDS) for stimulus decoding probabilistic manifold alignment across animals. P. Herrero-Vidal et al. NeurIPS 2021 code.

A toolkit for controlling Euro Truck Simulator 2 with python to develop self-driving algorithms.

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Pytorch implementation for "Large-Scale Long-Tailed Recognition in an Open World" (CVPR 2019 ORAL)

A program to recognize fruits on pictures or videos using yolov5

Bayesian Meta-Learning Through Variational Gaussian Processes

Punctuation Restoration using Transformer Models for High-and Low-Resource Languages

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

Official PyTorch Implementation for "Recurrent Video Deblurring with Blur-Invariant Motion Estimation and Pixel Volumes"

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs

FactSeg: Foreground Activation Driven Small Object Semantic Segmentation in Large-Scale Remote Sensing Imagery (TGRS)

Building Ellee — A GPT-3 and Computer Vision Powered Talking Robotic Teddy Bear With Human Level Conversation Intelligence

PyZebrascope - an open-source Python platform for brain-wide neural activity imaging in behaving zebrafish

Phy-Q: A Benchmark for Physical Reasoning

Video Contrastive Learning with Global Context

On Nonlinear Latent Transformations for GAN-based Image Editing - PyTorch implementation

Official PyTorch code for Hierarchical Conditional Flow: A Unified Framework for Image Super-Resolution and Image Rescaling (HCFlow, ICCV2021)

Object detection on multiple datasets with an automatically learned unified label space.