Robot Action Primitives (RAPS)
This repository is the official implementation of Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives (RAPS).
[Project Website]
Murtaza Dalal, Deepak Pathak*, Ruslan Salakhutdinov*
(* equal advising)
CMU
If you find this work useful in your research, please cite:
@inproceedings{dalal2021raps,
Author = {Dalal, Murtaza and Pathak, Deepak and
Salakhutdinov, Ruslan},
Title = {Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives},
Booktitle = {NeurIPS},
Year = {2021}
}
Requirements
To install dependencies, please run the following commands:
sudo apt-get update
sudo apt-get install curl \
git \
libgl1-mesa-dev \
libgl1-mesa-glx \
libglew-dev \
libosmesa6-dev \
software-properties-common \
net-tools \
unzip \
vim \
virtualenv \
wget \
xpra \
xserver-xorg-dev
sudo apt-get install libglfw3-dev libgles2-mesa-dev patchelf
sudo mkdir /usr/lib/nvidia-000
Please add the following to your bashrc:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.mujoco/mujoco200/bin
export MUJOCO_GL='egl'
export MKL_THREADING_LAYER=GNU
export D4RL_SUPPRESS_IMPORT_ERROR='1'
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/nvidia-000
To install python requirements:
conda create -n raps python=3.7
conda activate raps
./setup_python_env.sh <absolute path to raps>
Training and Evaluation
Kitchen
Prior to running any experiments, make sure to run cd /path/to/raps/rlkit
single task env names:
- microwave
- kettle
- slide_cabinet
- hinge_cabinet
- light_switch
- top_left_burner
multi task env names:
- microwave_kettle_light_top_left_burner //Sequential Multi Task 1
- hinge_slide_bottom_left_burner_light //Sequential Multi Task 2
To train RAPS with Dreamer on any single task kitchen environment, run:
python experiments/kitchen/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with Dreamer on the multi task kitchen environments, run:
python experiments/kitchen/dreamer/dreamer_v2_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with Dreamer on any kitchen environment
python experiments/kitchen/dreamer/dreamer_v2_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with RAD on any single task kitchen environment
python experiments/kitchen/rad/rad_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with RAD on any multi task kitchen environment
python experiments/kitchen/rad/rad_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with RAD on any kitchen environment
python experiments/kitchen/rad/rad_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with PPO on any single task kitchen environment
python experiments/kitchen/ppo/ppo_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train RAPS with PPO on any multi task kitchen environment
python experiments/kitchen/ppo/ppo_multi_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with PPO on any kitchen environment
python experiments/kitchen/ppo/ppo_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
Metaworld
single task env names
- drawer-close-v2
- soccer-v2
- peg-unplug-side-v2
- sweep-into-v2
- assembly-v2
- disassemble-v2
To train RAPS with Dreamer on any metaworld environment
python experiments/metaworld/dreamer/dreamer_v2_single_task_primitives.py --mode here_no_doodad --exp_prefix <> --env <env name>
To train Raw Actions with Dreamer on any metaworld environment
python experiments/metaworld/dreamer/dreamer_v2_single_task_raw_actions.py --mode here_no_doodad --exp_prefix <> --env <env name>
Robosuite
To train RAPS with Dreamer on an Robosuite Lift
python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_lift.py --mode here_no_doodad --exp_prefix <>
To train Raw Actions with Dreamer on an Robosuite Lift
python experiments/robosuite/dreamer/dreamer_v2_single_task_raw_actions_lift.py --mode here_no_doodad --exp_prefix <>
To train RAPS with Dreamer on an Robosuite Door
python experiments/robosuite/dreamer/dreamer_v2_single_task_primitives_door.py --mode here_no_doodad --exp_prefix <>
To train Raw Actions with Dreamer on an Robosuite Door
python experiments/robosuite/dreamer/dreamer_v2_single_task_raw_actions_door.py --mode here_no_doodad --exp_prefix <>
Learning Curve visualization
cd /path/to/raps/rlkit
python ../viskit/viskit/frontend.py data/<exp_prefix> //open localhost:5000 to view
188 Dec 29, 2022
31 Jul 16, 2022
86 Dec 13, 2022
539 Jan 07, 2023
26 Nov 24, 2022
214 Jan 03, 2023
16 Nov 18, 2022
1 Dec 02, 2022
7.3k Dec 26, 2022
158 Dec 21, 2022
118 Jan 06, 2023
155 Jan 08, 2023
4 Dec 14, 2022
239 Jan 02, 2023
27 Dec 16, 2022
1.8k Jan 01, 2023
7 Sep 23, 2022
1 Apr 19, 2022
34 Nov 15, 2022
7 Nov 26, 2022