Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Model Agnostic Interpretability for Multiple Instance Learning

Using Hotel Data to predict High Value And Potential VIP Guests

Second-order Attention Network for Single Image Super-resolution (CVPR-2019)

DEMix Layers for Modular Language Modeling

Implementation of ML models like Decision tree, Naive Bayes, Logistic Regression and many other

Official pytorch code for "APP: Anytime Progressive Pruning"

Quick program made to generate alpha and delta tables for Hidden Markov Models

Improving Object Detection by Estimating Bounding Box Quality Accurately

Implementation of 'X-Linear Attention Networks for Image Captioning' [CVPR 2020]

Proximal Backpropagation - a neural network training algorithm that takes implicit instead of explicit gradient steps

🇰🇷 Text to Image in Korean

ICSS - Interactive Continual Semantic Segmentation

LWCC: A LightWeight Crowd Counting library for Python that includes several pretrained state-of-the-art models.

Repository for "Improving evidential deep learning via multi-task learning," published in AAAI2022

Codebase of deep learning models for inferring stability of mRNA molecules

Code for the paper Task Agnostic Morphology Evolution.

Python package to add text to images, textures and different backgrounds

PyTorch implementation HoroPCA: Hyperbolic Dimensionality Reduction via Horospherical Projections

Learning to Prompt for Continual Learning

TensorFlow (Python API) implementation of Neural Style