The CARL Benchmark Library
CARL (context adaptive RL) provides highly configurable contextual extensions to several well-known RL environments. It's designed to test your agent's generalization capabilities in all scenarios where intra-task generalization is important.
Benchmarks include:
-
OpenAI gym classic control suite extended with several physics context features like gravity or friction
-
OpenAI gym Box2D BipedalWalker, LunarLander and CarRacing, each with their own modification possibilities like new vehicles to race
-
All Brax locomotion environments with exposed internal features like joint strength or torso mass
-
Super Mario (TOAD-GAN), a procedurally generated jump'n'run game with control over level similarity
-
RNADesign, an environment for RNA design given structure constraints with structures from different datasets to choose from
Installation
We recommend you use a virtual environment (e.g. Anaconda) to install CARL and its dependencies. We recommend and test with python 3.9 under Linux.
First, clone our repository and install the basic requirements:
git clone https://github.com/automl/CARL.git --recursive
cd CARL
pip install .
This will only install the basic classic control environments, which should run on most operating systems. For the full set of environments, use the install options:
pip install -e .[box2d, brax, rna, mario]
These may not be compatible with Windows systems. Box2D environment may need to be installed via conda on MacOS systems:
conda install -c conda-forge gym-box2d
In general, we test on Linux systems, but aim to keep the benchmark compatible with MacOS as much as possible. Mario at this point, however, will not run on any operation system besides Linux
To install the additional requirements for ToadGAN:
javac src/envs/mario/Mario-AI-Framework/**/*.java
If you want to use the RNA design environment:
cd src/envs/rna/learna
make requirements
make data
In case you want to run our experiments or use our training files, also install the experiment dependencies:
pip install -e .[experiments]
Train an Agent
To get started with CARL, you can use our 'train.py' script. It will train a PPO agent on the environment of your choice with custom context variations that are sampled from a standard deviation.
To use MetaCartPole with variations in gravity and friction by 20% compared to the default, run:
python train.py
--env CARLCartPoleEnv
--context_args gravity friction
--default_sample_std_percentage 0.2
--outdir <result_location>
You can use the plotting scripts in src/eval to view the results.
CARL's Contextual Extension
CARL contextually extends the environment by making the context visible and configurable. During training we therefore can encounter different contexts and train for generalization. We exemplarily show how Brax' Fetch is extended and embedded by CARL. Different instiations can be achieved by setting the context features to different values.
Cite Us
@misc{CARL,
author = {C. Benjamins and
T. Eimer and
F. Schubert and
A. Biedenkapp and
B. Rosenhahn and
F. Hutter and
M. Lindauer},
title = {CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning},
howpublished = {https://github.com/automl/CARL},
year = {2021},
month = aug,
}
References
OpenAI gym, Brockman et al., 2016. arXiv preprint arXiv:1606.01540
TOAD-GAN: Coherent Style Level Generation from a Single Example, Awiszus et al., AIIDE 2020
Learning to Design RNA, Runge et al., ICRL 2019
License
CARL falls under the Apache License 2.0 (see file 'LICENSE') as is permitted by all work that we use. This includes CARLMario, which is not based on the Nintendo Game, but on TOAD-GAN and TOAD-GUI running under an MIT license. They in turn make use of the Mario AI framework (https://github.com/amidos2006/Mario-AI-Framework). This is not the original game but a replica, explicitly built for research purposes and includes a copyright notice (https://github.com/amidos2006/Mario-AI-Framework#copyrights ).
28 Nov 16, 2022
57 Dec 04, 2022
21k Jan 06, 2023
527 Dec 04, 2022
34 Dec 04, 2022
2 Jan 24, 2022
3 Dec 31, 2021
185 Jan 09, 2023
59 Dec 22, 2022
1.7k Jan 04, 2023
106 Jan 09, 2023
180 Dec 19, 2022
536 Dec 20, 2022
560 Dec 31, 2022
779 Jan 05, 2023
1 Jan 31, 2022
39 Oct 31, 2022
61 Dec 19, 2022
181 Dec 27, 2022
6 Sep 21, 2022