Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

Overview

Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

This repository is the official implementation of

  • Seohong Park, Jaekyeom Kim, Gunhee Kim. Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods. In NeurIPS, 2021.

It contains the implementations for SAR, FiGAR-C and base policy gradient algorithms (PPO, TRPO and A2C).

The code is based on Stable Baselines3 (SB3) for PPO and A2C, and Stable Baselines (SB) for TRPO.

Requirements

Run examples

PPO and A2C (based on Stable Baselines3)

To install requirements:

cd sb3
pip install -r requirements.txt
pip install -e .

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

Train FiGAR-C-PPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.01 --max_t=0.05

Train PPO on InvertedPendulum-v2 with the original δ:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg

Train SAR-A2C on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=a2c_rg --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "Action Noise" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=action --anoise_prob=0.05 --anoise_std=3

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "External Force" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=ext_f --anoise_prob=0.05 --anoise_std=300

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "Strong External Force (Perceptible)" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=ext_fpc --anoise_prob=0.05 --anoise_std=1000

TRPO (based on Stable Baselines)

To install requirements:

cd sb
pip install -r requirements.txt
pip install -e .

Train SAR-TRPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

License

This codebase is licensed under the MIT License. See also sb3/LICENSE_SB3 and sb/LICENSE_SB.

Owner
Seohong Park
Seohong Park
DoSer.py - Simple DoSer in Python

DoSer.py - Simple DoSer in Python What is DoSer? DoSer is basically an HTTP Denial of Service attack that affects threaded servers. It works like this

1 Oct 12, 2021
Web3 Pancakeswap Sniper & honeypot detector Take Profit/StopLose bot written in python3, For ANDROID WIN MAC & LINUX

🏆 Pancakeswap BSC Sniper Bot web3 with honeypot detector (ANDROID WINDOWS MAC LINUX) 🥇 ⭐️ ⭐️ ⭐️ First SNIPER BOT for ANDROID & WINDOWS with honeypot

Mayank 12 Jan 07, 2023
Virus-Builder - This tool will generate a virus that can only destroy Windows computer

Virus-Builder - This tool will generate a virus that can only destroy Windows computer. You can also configure to auto run in usb drive

Saad 16 Dec 30, 2022
Übersicht remote command execution 0day exploit

Übersicht RCE 0day Unauthenticated remote command execution 0day exploit for Übersicht. Description Übersicht is a desktop widget application for m

BoofGang 10 Dec 21, 2021
CVE-2021-22005 - VMWare vCenter Server File Upload to RCE

CVE-2021-22005 - VMWare vCenter Server File Upload to RCE Analyze Usage ------------------------------------------------------------- [*] CVE-2021-220

r0cky 224 Aug 05, 2022
CVE-2022-23046 - SQL Injection Vulnerability on PhpIPAM v1.4.4

CVE-2022-23046 PhpIPAM v1.4.4 allows an authenticated admin user to inject SQL s

2 Feb 15, 2022
Official repository for Pyew.

pyew Pyew is a (command line) python tool to analyse malware. It does have support for hexadecimal viewing, disassembly (Intel 16, 32 and 64 bits), PE

Joxean 362 Nov 28, 2022
Find existing email addresses by nickname using API/SMTP checking methods without user notification. Please, don't hesitate to improve cat's job! 🐱🔎 📬

mailcat The only cat who can find existing email addresses by nickname. Usage First install requirements: pip3 install -r requirements.txt Then just

282 Dec 30, 2022
Implementation of an attack on a tropical algebra discrete logarithm based protocol

Implementation of an attack on a tropical algebra discrete logarithm based protocol This code implements the attack detailed in the paper: On the trop

3 Dec 30, 2021
D-810 is an IDA Pro plugin which can be used to deobfuscate code at decompilation time by modifying IDA Pro microcode.

Introduction fork from https://gitlab.com/eshard/d810 What is D-810 D-810 is an IDA Pro plugin which can be used to deobfuscate code at decompilation

Banny 30 Dec 06, 2022
Orthrus is a macOS agent that uses Apple's MDM to backdoor a device using a malicious profile.

Orthrus is a macOS agent that uses Apple's MDM to backdoor a device using a malicious profile. It effectively runs its own MDM server and allows the operator to interface with it using Mythic.

Mythic Agents 37 Dec 06, 2022
Python APK Reverser & Patcher Tool

DTL-X An Advanced Python APK Reverser and Patcher Tool. --rmads1: target=AndroidManifest.xml,replace=com.google.android.gms.ad --rmads2: No Internet (

DedSecTL 10 Oct 31, 2022
A Python & JavaScript Obfuscator made in Python 3.

Python Code Obfuscator A script that converts code into full on random numerical expressions. Simple Scripts: Python Mode... Input: Function that deco

Karim 3 Mar 24, 2022
List of S3 Hacks

s3-leaks List of AWS S3 Leaks Feel free to send in a PR if you know of other leaks Date Description Notes Aug2020 S3 bucket mess up exposed 182GB of s

Nag 291 Dec 28, 2022
This is simple python FTP password craker. To crack FTP login using wordlist based brute force attack

This is simple python FTP password craker. To crack FTP login using wordlist based brute force attack

Varun Jagtap 5 Oct 08, 2022
macOS Initial Access Payload Generator

Mystikal macOS Initial Access Payload Generator Related Blog Post: https://posts.specterops.io/introducing-mystikal-4fbd2f7ae520 Usage: Install Xcode

Leo Pitt 206 Dec 31, 2022
A simple python script to dump remote files through a local file read or local file inclusion web vulnerability.

A simple python script to dump remote files through a local file read or local file inclusion web vulnerability. Features Dump a single file w

Podalirius 48 Dec 03, 2022
Monty Hall Problem simulation written in Python.

Monty Hall Problem Simulation monty_hall_sim is a brute-force method of determining the optimal strategy for the Monty Hall Problem. Usage Set boolean

Xavier D 1 Aug 29, 2022
OpenTOTP is yet another time-based, one-time passwords (OTPs) generator/verifier inspired by RFC 6238.

OpenTOTP is yet another time-based, one-time passwords (OTPs) generator/verifier inspired by RFC 6238. It generates and validates OTPs based

1 Nov 15, 2021