Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

Overview

Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

This repository is the official implementation of

  • Seohong Park, Jaekyeom Kim, Gunhee Kim. Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods. In NeurIPS, 2021.

It contains the implementations for SAR, FiGAR-C and base policy gradient algorithms (PPO, TRPO and A2C).

The code is based on Stable Baselines3 (SB3) for PPO and A2C, and Stable Baselines (SB) for TRPO.

Requirements

Run examples

PPO and A2C (based on Stable Baselines3)

To install requirements:

cd sb3
pip install -r requirements.txt
pip install -e .

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

Train FiGAR-C-PPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.01 --max_t=0.05

Train PPO on InvertedPendulum-v2 with the original δ:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg

Train SAR-A2C on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=a2c_rg --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "Action Noise" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=action --anoise_prob=0.05 --anoise_std=3

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "External Force" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=ext_f --anoise_prob=0.05 --anoise_std=300

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "Strong External Force (Perceptible)" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=ext_fpc --anoise_prob=0.05 --anoise_std=1000

TRPO (based on Stable Baselines)

To install requirements:

cd sb
pip install -r requirements.txt
pip install -e .

Train SAR-TRPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

License

This codebase is licensed under the MIT License. See also sb3/LICENSE_SB3 and sb/LICENSE_SB.

Owner
Seohong Park
Seohong Park
Mass scan for .git repository and .env file exposure

Mass .Git repository and .Env file Scan by Scarmandef Scanner to find .env file and .git repository exposure on multiple hosts Because of the response

8 Jun 23, 2022
Official repository for Pyew.

pyew Pyew is a (command line) python tool to analyse malware. It does have support for hexadecimal viewing, disassembly (Intel 16, 32 and 64 bits), PE

Joxean 362 Nov 28, 2022
Exploiting CVE-2021-44228 in vCenter for remote code execution and more

Log4jCenter Exploiting CVE-2021-44228 in vCenter for remote code execution and more. Blog post detailing exploitation linked below: COMING SOON Why? P

81 Dec 20, 2022
🐎🖥《赛马娘》(ウマ娘: Pretty Derby)辅助脚本

auto-derby 自动化养马 育成结果 Nurturing result 功能 支持客户端 DMM (前台) 实验性 安卓 ADB 连接(后台)开发基于 1080x1920 分辨率 团队赛 (Team race) 有胜利确定奖励时吃帕菲 日常赛 (Daily race) PvP 活动赛 (Cha

NateScarlet 376 Jan 01, 2023
Patching - Interactive Binary Patching for IDA Pro

Patching - Interactive Binary Patching for IDA Pro Overview Patching assembly code to change the behavior of an existing program is not uncommon in ma

589 Dec 30, 2022
PoC for CVE-2021-26855 -Just a checker-

CVE-2021-26855 PoC for CVE-2021-26855 -Just a checker- Usage python3 CVE-2021-26855.py -u https://mail.example.com -c example.burpcollaborator.net # C

Abdullah AlZahrani 17 Dec 22, 2022
WebScan is a web vulnerability Scanning tool, which scans sites for SQL injection and XSS vulnerabilities

WebScan is a web vulnerability Scanning tool, which scans sites for SQL injection and XSS vulnerabilities Which is a great tool for web pentesters. Coded in python3, CLI. WebScan is capable of scanni

AnonyminHack5 12 Dec 02, 2022
ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software.

ClusterFuzz ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software. Google uses ClusterFuzz to fuzz all

Google 4.9k Jan 08, 2023
2021hvv漏洞汇总

清单 披露时间 涉及商家/产品 漏洞描述 2021/04/08 启明星辰天清汉马USG防火墙存在逻辑缺陷漏洞(历史漏洞) CNVD-2021-17391 启明星辰 天清汉马USG防火墙 逻辑缺陷漏洞 CNVD-2021-12793 2021/04/08 禅道项目管理软件11.6 禅道 11.6 sq

555 Jan 02, 2023
This is tools hacking for scan vuln in port web, happy using

Xnuvers007 PortInjection this is tools hacking for scan vuln in port web, happy using view/show python 3.9 solo coder (tangerang) 19 y/o installation

XnuxersXploitXen 6 Dec 24, 2022
PassLock is a medium-security password manager that encrypts passwords using Advanced Encryption Standards (AES)

A medium security python password manager that encrypt passwords using Advanced Encryption Standard (AES) PassLock is a password manager and password

Akshay Vs 44 Nov 18, 2022
一个自动挖掘漏洞的框架,日后会发展成强大的信息收集+漏洞挖掘脚本!

介绍 工具介绍 这是一款致力于将各类优秀脚本集合在一起调用、联动,最终可形成超级渗透脚本的工具。目的是扫描到更全的资产信息,发现更多的漏洞利用。但是这是通过牺牲扫描速度来提升扫描广度的。所以不太适合要进行紧急信息收集和漏洞利用的情况。

Thinking rookie 23 Jul 05, 2022
python写的一款免杀工具(shellcode加载器)BypassAV,国内杀软全过(windows denfend)

python写的一款免杀工具(shellcode加载器)BypassAV,国内杀软全过(windows denfend)

1frame 266 Jan 02, 2023
A simple python code for hacking profile views

This code for hacking profile views. Not recommended to adding profile views in profile. This code is not illegal code. This code is for beginners.

Fayas Noushad 3 Nov 28, 2021
Python exploit for vsftpd 2.3.4 - Backdoor Command Execution

CVE-2011-2523 - vsftpd 2.3.4 Exploit Discription vsftpd, which stands for Very Secure FTP Daemon,is an FTP server for Unix-like systems, including Lin

Padsala Tushal 5 Nov 08, 2022
RCE 0-day for GhostScript 9.50 - Payload generator

RCE-0-day-for-GhostScript-9.50 PoC for RCE 0-day for GhostScript 9.50 - Payload generator The PoC in python generates payload when exploited for a 0-d

534 Dec 14, 2022
CloakifyFactory & the Cloakify Toolset - Data Exfiltration & Infiltration In Plain Sight;

CloakifyFactory CloakifyFactory & the Cloakify Toolset - Data Exfiltration & Infiltration In Plain Sight; Evade DLP/MLS Devices; Social Engineering of

3 Oct 18, 2022
S2-061 的payload,以及对应简单的PoC/Exp

S2-061 脚本皆根据vulhub的struts2-059/061漏洞测试环境来写的,不具普遍性,还望大佬多多指教 struts2-061-poc.py(可执行简单系统命令) 用法:python struts2-061-poc.py http://ip:port command 例子:python

dreamer 46 Oct 20, 2022
SSRF search vulnerabilities exploitation extended.

This tool search for SSRF using predefined settings in different parts of a request (path, host, headers, post and get parameters).

Andri Wahyudi 13 Jul 04, 2021
Fuck - Multi Brute Force 🚶‍♂

f-mbf Fuck - Multi Brute Force 🚶‍♂ Install Script $ pkg update && pkg upgrade $ pkg install python2 $ pkg install git $ pip2 install requests $ pip2

Yumasaa 1 Dec 03, 2021