Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Optimal Adaptive Allocation using Deep Reinforcement Learning in a Dose-Response Study

Tensorflow 2.x based implementation of EDSR, WDSR and SRGAN for single image super-resolution

A unified 3D Transformer Pipeline for visual synthesis

TuckER: Tensor Factorization for Knowledge Graph Completion

Official implementation of Pixel-Level Bijective Matching for Video Object Segmentation

Non-Metric Space Library (NMSLIB): An efficient similarity search library and a toolkit for evaluation of k-NN methods for generic non-metric spaces.

A model which classifies reviews as positive or negative.

A minimal implementation of Gaussian process regression in PyTorch

Explainer for black box models that predict molecule properties

Facilitates implementing deep neural-network backbones, data augmentations

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

pcnaDeep integrates cutting-edge detection techniques with tracking and cell cycle resolving models.

PyTorch implementation of GLOM

FLAVR is a fast, flow-free frame interpolation method capable of single shot multi-frame prediction

As-ViT: Auto-scaling Vision Transformers without Training

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs

Pytorch implementation AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

Code for Multiple Instance Active Learning for Object Detection, CVPR 2021

LSTM built using Keras Python package to predict time series steps and sequences. Includes sin wave and stock market data

disentanglement_lib is an open-source library for research on learning disentangled representations.