Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Code for the paper "Learning-Augmented Algorithms for Online Steiner Tree"

This is a collection of simple PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations,

Music library streaming app written in Flask & VueJS

Computationally efficient algorithm that identifies boundary points of a point cloud.

An Unpaired Sketch-to-Photo Translation Model

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

SweiNet is an uncertainty-quantifying shear wave speed (SWS) estimator for ultrasound shear wave elasticity (SWE) imaging.

Keras-retinanet - Keras implementation of RetinaNet object detection.

Transformers based fully on MLPs

Spectrum Surveying: Active Radio Map Estimation with Autonomous UAVs

Conservative and Adaptive Penalty for Model-Based Safe Reinforcement Learning

Official Pytorch implementation of Meta Internal Learning

Python PID Tuner - Makes a model of the System from a Process Reaction Curve and calculates PID Gains

The code for SAG-DTA: Prediction of Drug–Target Affinity Using Self-Attention Graph Network.

Re-implementation of the Noise Contrastive Estimation algorithm for pyTorch, following "Noise-contrastive estimation: A new estimation principle for unnormalized statistical models." (Gutmann and Hyvarinen, AISTATS 2010)

Server files for UltimateLabeling

Experimental code for paper: Generative Adversarial Networks as Variational Training of Energy Based Models

Transfer Learning for Pose Estimation of Illustrated Characters

Populating 3D Scenes by Learning Human-Scene Interaction https://posa.is.tue.mpg.de/

Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data - Official PyTorch Implementation (CVPR 2022)