Pearl
The Parallel Evolutionary and Reinforcement Learning Library (Pearl) is a pytorch based package with the goal of being excellent for rapid prototyping of new adaptive decision making algorithms in the intersection between reinforcement learning (RL) and evolutionary computation (EC). As such, this is not intended to provide template pre-built algorithms as a baseline, but rather flexible tools to allow the user to quickly build and test their own implementations and ideas. A technical report can be found here.
Main Features
| Features | Pearl |
|---|---|
| RL algorithms (e.g. Actor Critic) | |
| EC algorithms (e.g. Genetic Algorithm) | |
| Hybrid algorithms (e.g. CEM-DDPG) | |
| Multi-agent suppport | |
| Tensorboard integration | |
| Modular and extensible components | |
| Opinionated module settings | |
| Custom callbacks | |
User Guide
Installation
There are two options to install this package:
pip install pearllgit clone [email protected]:LondonNode/Pearl.git
Module Guide
agents: implementations of RL and EC agents where the other modular components are put togetherbuffers: these handle storing and sampling of trajectoriescallbacks: inject logic for every step made in an environment (e.g. save model, early stopping)common: common methods applicable to all other modules (e.g. enumerations) and a mainutils.pyfile with some useful general logicexplorers: action explorers for enhanced exploration by adding noise to actions and random exploration for first n stepsmodels: neural network structures which are structured asencoder->torso->headsignal_processing: signal processing logic for extra modularity (e.g. TD returns, GAE)updaters: update neural networks and adaptive/iterative algorithmssettings.py: settings objects for the above components, can be extended for custom components
Agent Templates
See pearll/agents/templates.py for the templates to create your own agents! For more examples, see specific agent implementations under pearll/agents.
Agent Performance
To see training performance, use the command tensorboard --logdir runs or tensorboard --logdir <tensorboard_log_path> defined in your algorithm class initialization.
Python Scripts
To run these you'll need to go to wherever the library is installed, cd pearll.
demo.py: script to run very basic demos of agents with pre-defined hyperparameters, runpython3 -m pearll.demo -hfor more infoplot.py: script to plot more complex plots that can't be obtained via Tensorboard (e.g. multiple subplots), runpython3 -m pearll.plot -hfor more info
Developer Guide
Scripts
Linux
scripts/setup_dev.sh: setup your virtual environmentscripts/run_tests.sh: run tests
Windows
scripts/windows_setup_dev.bat: setup your virtual environmentscripts/windows_run_tests.bat: run tests
Dependency Management
Pearl uses poetry for dependency management and build release instead of pip. As a quick guide:
- Run
poetry add [package]to add more package dependencies. - Poetry automatically handles the virtual environment used, check
pyproject.tomlfor specifics on the virtual environment setup. - If you want to run something in the poetry virtual environment, add
poetry runas a prefix to the command you want to execute. For example, to run a python file:poetry run python3 script.py.
Credit
Citing Pearl
@misc{tangri2022pearl,
title={Pearl: Parallel Evolutionary and Reinforcement Learning Library},
author={Rohan Tangri and Danilo P. Mandic and Anthony G. Constantinides},
year={2022},
eprint={2201.09568},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
Acknowledgements
Pearl was inspired by Stable Baselines 3 and Tonic
0 Jan 16, 2022
64 Dec 16, 2022
1.8k Dec 28, 2022
29.6k Jan 8, 2023
4.7k Jan 4, 2023
183 Dec 28, 2022
2.1k Jan 4, 2023
4 Dec 31, 2022
4.7k Jan 1, 2023
1 Dec 05, 2022
542 Jan 01, 2023
21 Nov 14, 2022
13 Dec 10, 2022
5 Nov 13, 2022
8 Nov 20, 2022
2 Jul 05, 2022
65 Nov 28, 2022
280 Nov 24, 2022
25 Sep 21, 2022
105 Dec 20, 2022
15 Dec 16, 2022
11 Nov 30, 2022
103 Nov 25, 2022
190 Dec 01, 2022
92 Dec 30, 2022
820 Dec 26, 2022
23 Nov 07, 2022
430 Jan 04, 2023
1 Jan 27, 2022