Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Source code for "FastBERT: a Self-distilling BERT with Adaptive Inference Time".

Constraint-based geometry sketcher for blender

ViewFormer: NeRF-free Neural Rendering from Few Images Using Transformers

NeurIPS'21 Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows

Training neural models with structured signals.

AI Toolkit for Healthcare Imaging

MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification

Official implementation of VQ-Diffusion

Reference implementation for Structured Prediction with Deep Value Networks

QueryDet: Cascaded Sparse Query for Accelerating High-Resolution SmallObject Detection

Free like Freedom

Official implementation for "Style Transformer for Image Inversion and Editing" (CVPR 2022)

FACIAL: Synthesizing Dynamic Talking Face With Implicit Attribute Learning. ICCV, 2021.

Python-based Informatics Kit for Analysing Chemical Units

KSAI Lite is a deep learning inference framework of kingsoft, based on tensorflow lite

System Combination for Grammatical Error Correction Based on Integer Programming

[CVPR'21 Oral] Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

Identify the emotion of multiple speakers in an Audio Segment

Implementation of ConvMixer for "Patches Are All You Need? 🤷"

Loopy belief propagation for factor graphs on discrete variables, in JAX!