This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
D-810 is an IDA Pro plugin which can be used to deobfuscate code at decompilation time by modifying IDA Pro microcode.

Introduction fork from https://gitlab.com/eshard/d810 What is D-810 D-810 is an IDA Pro plugin which can be used to deobfuscate code at decompilation

Banny 30 Dec 06, 2022
StarUML cracker - StarUML cracker With Python

StarUML_cracker Usage On Linux Clone the repo. git clone https://github.com/mana

Bibek Manandhar 9 Jun 20, 2022
Python Toolkit containing different Cyber Attacks Tools

Helikopter Python Toolkit containing different Cyber Attacks Tools. Tools in Helikopter Toolkit 1. FattyNigger (PYTHON WORM) 2. Taxes (PYTHON PASS EXT

Saqlain Naqvi 22 Dec 04, 2022
Script for automatic dump and brute-force passwords using Volatility Framework

Volatility-auto-hashdump Script for automatic dump and brute-force passwords using Volatility Framework

whoamins 11 Apr 11, 2022
macOS persistence tool

PoisonApple Command-line tool to perform various persistence mechanism techniques on macOS. This tool was designed to be used by threat hunters for cy

Cyborg Security, Inc 212 Dec 29, 2022
A Burp extension adding a passive scan check to flag parameters whose name or value may indicate a possible insertion point for SSRF or LFI.

BurpParamFlagger A Burp extension adding a passive scan check to flag parameters whose name or value may indicate a possible insertion point for SSRF

Allyson O'Malley 118 Nov 07, 2022
version de mi tool de kali linux para miertuxzzzz digo, termux >:)

Msf-Tool 1.0 Termux apt install git -y apt install python apt install python3 apt install python3-pip apt install metasploit ---- ---- git clone ht

BruhGera 1 Feb 20, 2022
A passive-recon tool that parses through found assets and interacts with the Hackerone API

Hackerone Passive Recon Tool A passive-recon tool that parses through found assets and interacts with the Hackerone API. Setup Simply run setup.sh to

elbee 4 Jan 13, 2022
Natas teaches the basics of serverside web-security.

over-the-wire-natas Natas teaches the basics of serverside web-security. Each level of natas consists of its own website located at http://natasX.nata

Siddhant Chouhan 1 Nov 27, 2021
A Proof-Of-Concept for the recently found CVE-2021-44228 vulnerability

log4j-shell-poc A Proof-Of-Concept for the recently found CVE-2021-44228 vulnerability. Recently there was a new vulnerability in log4j, a java loggin

koz 1.5k Jan 04, 2023
A secure password generator written in python

gruvbox-factory 🏭 "The main focus when developing gruvbox is to keep colors easily distinguishable, contrast enough and still pleasant for the eyes"

Paulo Pacitti 430 Dec 27, 2022
A BurpSuite extension to parse 5GC NF OpenAPI 3.0 files to assess 5G core networks

5GC_API_parse Description 5GC API parse is a BurpSuite extension allowing to assess 5G core network functions, by parsing the OpenAPI 3.0 not supporte

PentHertz 57 Dec 16, 2022
A small Minecraft server to help players detect vulnerability to the Log4Shell exploit 🐚

log4check A small Minecraft server to help players detect vulnerability to the Log4Shell exploit 🐚 Tested to work between Minecraft versions 1.12.2 a

Evan J. Markowitz 4 Dec 23, 2021
Tools ini digunakan untuk krekk pacebuk:v

E-Crack By Aang-XD Fitur Login • Login via token facebook • Login via cookie facebook Install On Termux $ pkg update && pkg upgrade $ pkg install pyth

Aang Ardiansyah-XD 2 Dec 24, 2021
A python script to decrypt media files encrypted using the Android application 'Decrypting 'LOCKED Secret Calculator Vault''. Will identify PIN / pattern.

A python script to decrypt media files encrypted using the Android application 'Decrypting 'LOCKED Secret Calculator Vault''. Will identify PIN / pattern.

3 Sep 26, 2022
Subdomain enumeration,Web scraping and finding usernames automation script written in python

Subdomain enumeration,Web scraping and finding usernames automation script written in python

Syam 12 Nov 22, 2022
A tool combined with the advantages of masscan and nmap

A tool combined with the advantages of masscan and nmap

59 Dec 24, 2022
Ensure secure infrastructure and consistency with the firewall rules

Python Port Scanner This script tries to check if it's possible to make a connection with the specific endpoint port. This is very useful to ensure se

Allan Avelar 7 Feb 26, 2022
Laravel RCE (CVE-2021-3129)

CVE-2021-3129 - Laravel RCE About The script has been made for exploiting the Laravel RCE (CVE-2021-3129) vulnerability. This script allows you to wri

Joshua van der Poll 21 Dec 27, 2022