Pearl
The Parallel Evolutionary and Reinforcement Learning Library (Pearl) is a pytorch based package with the goal of being excellent for rapid prototyping of new adaptive decision making algorithms in the intersection between reinforcement learning (RL) and evolutionary computation (EC). As such, this is not intended to provide template pre-built algorithms as a baseline, but rather flexible tools to allow the user to quickly build and test their own implementations and ideas. A technical report can be found here.
Main Features
| Features | Pearl |
|---|---|
| RL algorithms (e.g. Actor Critic) | |
| EC algorithms (e.g. Genetic Algorithm) | |
| Hybrid algorithms (e.g. CEM-DDPG) | |
| Multi-agent suppport | |
| Tensorboard integration | |
| Modular and extensible components | |
| Opinionated module settings | |
| Custom callbacks | |
User Guide
Installation
There are two options to install this package:
pip install pearllgit clone [email protected]:LondonNode/Pearl.git
Module Guide
agents: implementations of RL and EC agents where the other modular components are put togetherbuffers: these handle storing and sampling of trajectoriescallbacks: inject logic for every step made in an environment (e.g. save model, early stopping)common: common methods applicable to all other modules (e.g. enumerations) and a mainutils.pyfile with some useful general logicexplorers: action explorers for enhanced exploration by adding noise to actions and random exploration for first n stepsmodels: neural network structures which are structured asencoder->torso->headsignal_processing: signal processing logic for extra modularity (e.g. TD returns, GAE)updaters: update neural networks and adaptive/iterative algorithmssettings.py: settings objects for the above components, can be extended for custom components
Agent Templates
See pearll/agents/templates.py for the templates to create your own agents! For more examples, see specific agent implementations under pearll/agents.
Agent Performance
To see training performance, use the command tensorboard --logdir runs or tensorboard --logdir <tensorboard_log_path> defined in your algorithm class initialization.
Python Scripts
To run these you'll need to go to wherever the library is installed, cd pearll.
demo.py: script to run very basic demos of agents with pre-defined hyperparameters, runpython3 -m pearll.demo -hfor more infoplot.py: script to plot more complex plots that can't be obtained via Tensorboard (e.g. multiple subplots), runpython3 -m pearll.plot -hfor more info
Developer Guide
Scripts
Linux
scripts/setup_dev.sh: setup your virtual environmentscripts/run_tests.sh: run tests
Windows
scripts/windows_setup_dev.bat: setup your virtual environmentscripts/windows_run_tests.bat: run tests
Dependency Management
Pearl uses poetry for dependency management and build release instead of pip. As a quick guide:
- Run
poetry add [package]to add more package dependencies. - Poetry automatically handles the virtual environment used, check
pyproject.tomlfor specifics on the virtual environment setup. - If you want to run something in the poetry virtual environment, add
poetry runas a prefix to the command you want to execute. For example, to run a python file:poetry run python3 script.py.
Credit
Citing Pearl
@misc{tangri2022pearl,
title={Pearl: Parallel Evolutionary and Reinforcement Learning Library},
author={Rohan Tangri and Danilo P. Mandic and Anthony G. Constantinides},
year={2022},
eprint={2201.09568},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
Acknowledgements
Pearl was inspired by Stable Baselines 3 and Tonic
0 Jan 16, 2022
64 Dec 16, 2022
1.8k Dec 28, 2022
29.6k Jan 8, 2023
4.7k Jan 4, 2023
183 Dec 28, 2022
2.1k Jan 4, 2023
4 Dec 31, 2022
4.7k Jan 1, 2023
6 Oct 24, 2022
449 Dec 27, 2022
60 Dec 02, 2022
4 Jun 08, 2022
42 Dec 07, 2022
1 Oct 27, 2021
1 Nov 25, 2022
199 Jan 04, 2023
7 May 19, 2022
49 Dec 27, 2022
16 May 06, 2022
7 Apr 04, 2022
561 Dec 01, 2022
17 Dec 20, 2022
46 Nov 17, 2022
56 Nov 28, 2022
25 Jul 27, 2022
3 Jan 07, 2022
6 Dec 17, 2022
10 Apr 06, 2022