Additional environments compatible with OpenAI gym

Last update: Dec 06, 2022

Related tags

Deep Learning quad-swarm-rl

Overview

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

A codebase for training reinforcement learning policies for quadrotor swarms. Includes:

Flight dynamics simulator forked from https://github.com/amolchanov86/gym_art and extended to support swarms of quadrotor drones
Scripts and the necessary wrappers to facilitate training of control policies with Sample Factory https://github.com/alex-petrenko/sample-factory

Paper: https://arxiv.org/abs/2109.07735

Website: https://sites.google.com/view/swarm-rl

Installation

Initialize a Python environment, i.e. with conda (Python versions 3.6-3.8 are supported):

conda create -n swarm-rl python=3.8
conda activate swarm-rl

Clone and install this repo as an editable Pip package:

git clone https://github.com/alex-petrenko/quad-swarm-rl
cd quad-swarm-rl
pip install -e .

This should pull and install all the necessary dependencies, including Sample Factory and PyTorch.

Running experiments

Train

This will run the baseline experiment. Change the number of workers appropriately to match the number of logical CPU cores on your machine, but it is advised that the total number of simulated environments is close to that in the original command:

python -m swarm_rl.train --env=quadrotor_multi --train_for_env_steps=1000000000 --algo=APPO \
--use_rnn=False \
--num_workers=36 --num_envs_per_worker=4 \
--learning_rate=0.0001 --ppo_clip_value=5.0 \
--recurrence=1 --nonlinearity=tanh --actor_critic_share_weights=False \
--policy_initialization=xavier_uniform --adaptive_stddev=False --with_vtrace=False \
--max_policy_lag=100000000 --hidden_size=256 --gae_lambda=1.00 --max_grad_norm=5.0 \
--exploration_loss_coeff=0.0 --rollout=128 --batch_size=1024 --quads_use_numba=True \
--quads_mode=mix --quads_episode_duration=15.0 --quads_formation_size=0.0 \
--encoder_custom=quad_multi_encoder --with_pbt=False --quads_collision_reward=5.0 \
--quads_neighbor_hidden_size=256 --neighbor_obs_type=pos_vel --quads_settle_reward=0.0 \
--quads_collision_hitbox_radius=2.0 --quads_collision_falloff_radius=4.0 --quads_local_obs=6 \
--quads_local_metric=dist --quads_local_coeff=1.0 --quads_num_agents=8 --quads_collision_reward=5.0 \
--quads_collision_smooth_max_penalty=10.0 --quads_neighbor_encoder_type=attention \
--replay_buffer_sample_prob=0.75 --anneal_collision_steps=300000000 --experiment=swarm_rl

Or, even better, you can use the runner scripts in swarm_rl/runs/. Runner scripts (a Sample Factory feature) are Python files that contain experiment parameters, and support features such as evaluation on multiple seeds and gridsearches.

To execute a runner script run the following command:

python -m sample_factory.runner.run --run=swarm_rl.runs.quad_multi_mix_baseline_attn --runner=processes --max_parallel=4 --pause_between=1 --experiments_per_gpu=1 --num_gpus=4

This command will start training four different seeds in parallel on a 4-GPU server. Adjust the parameters accordingly to match your hardware setup.

To monitor the experiments, go to the experiment folder, and run the following command:

tensorboard --logdir=./

Test

To test the trained model, run the following command:

python -m swarm_rl.enjoy --algo=APPO --env=quadrotor_multi --replay_buffer_sample_prob=0 --continuous_actions_sample=False --quads_use_numba=False --train_dir=PATH_TO_PROJECT/swarm_rl/train_dir --experiments_root=EXPERIMENT_ROOT --experiment=EXPERIMENT_NAME

Unit Tests

To run unit tests:

./run_tests.sh

Additional environments compatible with OpenAI gym

Related tags

Overview

Decentralized Control of Quadrotor Swarms with End-to-end Deep Reinforcement Learning

Installation

Running experiments

Train

Test

Unit Tests

Owner

Zhehui Huang

Official tensorflow implementation for CVPR2020 paper “Learning to Cartoonize Using White-box Cartoon Representations”

Learning Dense Representations of Phrases at Scale (Lee et al., 2020)

Code To Tune or Not To Tune? Zero-shot Models for Legal Case Entailment.

CRLT: A Unified Contrastive Learning Toolkit for Unsupervised Text Representation Learning

AirCode: A Robust Object Encoding Method

Code + pre-trained models for the paper Keeping Your Eye on the Ball Trajectory Attention in Video Transformers

An implementation of MobileFormer

Code for our ICASSP 2021 paper: SA-Net: Shuffle Attention for Deep Convolutional Neural Networks

TensorFlow2 Classification Model Zoo playing with TensorFlow2 on the CIFAR-10 dataset.

The implementation of ICASSP 2020 paper "Pixel-level self-paced learning for super-resolution"

Calibrate your listeners! Robust communication-based training for pragmatic speakers. Findings of EMNLP 2021.

Code for the paper "Generative design of breakwaters usign deep convolutional neural network as a surrogate model"

NEO: Non Equilibrium Sampling on the orbit of a deterministic transform

ATAC: Adversarially Trained Actor Critic

[NeurIPS 2021] A weak-shot object detection approach by transferring semantic similarity and mask prior.

✨风纪委员会自动投票脚本，利用Github Action帮你进行裁决操作（为了让其他风纪委员有案件可判，本程序从中午12点才开始运行，有需要请自己修改运行时间）

Implementation of paper "Self-supervised Learning on Graphs:Deep Insights and New Directions"

VQGAN+CLIP Colab Notebook with user-friendly interface.

This is the repository for Learning to Generate Piano Music With Sustain Pedals