Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

A collection of papers about Transformer in the field of medical image analysis.

Create animations for the optimization trajectory of neural nets

Efficient Conformer: Progressive Downsampling and Grouped Attention for Automatic Speech Recognition

Non-Official Pytorch implementation of "Face Identity Disentanglement via Latent Space Mapping" https://arxiv.org/abs/2005.07728 Using StyleGAN2 instead of StyleGAN

Code & Data for the Paper "Time Masking for Temporal Language Models", WSDM 2022

HCQ: Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval

An efficient toolkit for Face Stylization based on the paper "AgileGAN: Stylizing Portraits by Inversion-Consistent Transfer Learning"

An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results

A simple baseline for the 2022 IEEE GRSS Data Fusion Contest (DFC2022)

Practical Blind Denoising via Swin-Conv-UNet and Data Synthesis

Scalable training for dense retrieval models.

The PyTorch implementation of Directed Graph Contrastive Learning (DiGCL), NeurIPS-2021

Fastshap: A fast, approximate shap kernel

Irrigation controller for Home Assistant

Pansharpening by convolutional neural networks in the full resolution framework

Memory Defense: More Robust Classificationvia a Memory-Masking Autoencoder

Bling's Object detection tool

UnpNet - Rethinking 3-D LiDAR Point Cloud Segmentation(IEEE TNNLS)

PyMove is a Python library to simplify queries and visualization of trajectories and other spatial-temporal data