This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
A high-performance DNS stub resolver for bulk lookups and reconnaissance (subdomain enumeration)

MassDNS A high-performance DNS stub resolver MassDNS is a simple high-performance DNS stub resolver targeting those who seek to resolve a massive amou

B. Blechschmidt 2.5k Jan 07, 2023
The best Python Backdoor👌

Backdoor The best Python Backdoor Files Server file is used in all of cases If client is Windows, the client need execute EXE file If client is Linux,

13 Oct 28, 2022
PortSwigger Burp Plugin for the Log4j (CVE-2021-44228)

yLog4j This is Y-Sec's @PortSwigger Burp Plugin for the Log4j CVE-2021-44228 vulnerability. The focus of yLog4j is to support mass-scanning of the Log

Y-Security 1 Jan 31, 2022
A guide to building basic malware in Python by implementing a keylogger application

Keylogger-Malware-Project A guide to building basic malware in Python by implementing a keylogger application. If you want even more detail on the Pro

Noah Davis 1 Jan 11, 2022
A Python & JavaScript Obfuscator made in Python 3.

Python Code Obfuscator A script that converts code into full on random numerical expressions. Simple Scripts: Python Mode... Input: Function that deco

Karim 3 Mar 24, 2022
WinRemoteEnum is a module-based collection of operations achievable by a low-privileged domain user.

WinRemoteEnum WinRemoteEnum is a module-based collection of operations achievable by a low-privileged domain user, sharing the goal of remotely gather

Simon 9 Nov 09, 2022
Open-source keylogger write in python

Python open-source keylogger Language Python open-source keylogger using pynput module Using Install dependences in archive setup.py or install.sh in

Dio brando 4 Jan 15, 2022
Python exploit code for CVE-2021-4034 (pwnkit)

Python3 code to exploit CVE-2021-4034 (PWNKIT). This was an exercise in "can I make this work in Python?", and not meant as a robust exploit. It Works

Joe Ammond 92 Dec 29, 2022
🔎 Most Advanced Open Source Intelligence (OSINT) Framework for scanning IP Address, Emails, Websites, Organizations.

🔎 Most Advanced Open Source Intelligence (OSINT) Framework for scanning IP Address, Emails, Websites, Organizations.

BhavKaran 1.5k Dec 28, 2022
Fetch Chrome, Firefox, WiFi password and system info

DISCLAIMER : OUR TOOLS ARE FOR EDUCATIONAL PURPOSES ONLY. DON'T USE THEM FOR ILLEGAL ACTIVITIES. YOU ARE THE ONLY RESPONSABLE FOR YOUR ACTIONS! OUR TO

Genos 59 Nov 17, 2022
This repository is one of a few malware collections on the GitHub.

This repository is one of a few malware collections on the GitHub.

Andrew 1.7k Dec 28, 2022
Yet another web fuzzer

yafuzz Yet another web fuzzer Usage This script can run in two modes of operation. Supplying a wordlist -W argument will initiate a multithreaded fuzz

FooBallZ 5 Feb 02, 2022
xray多线程批量扫描工具

Auto_xray xray多线程批量扫描工具 简介 xray社区版貌似没有批量扫描,这就让安服仔使用起来很不方便,扫站得一个个手动添加,非常难受 Auto_xray目录下记得放xray,就跟平时一样的。 选项1:oneforall+xray 输入一个主域名,自动采集子域名然后添加到xray任务列表

1frame 13 Nov 09, 2022
Selamat Datang DiTools Crack-Old, Crack Old Adalah Sebuah Crack Tanpa Login Dan Crack Menggunakan Akun Facebook Tua/Old.

Selamat Datang DiTools Crack-Old, Crack Old Adalah Sebuah Crack Tanpa Login Dan Crack Menggunakan Akun Facebook Tua/Old. ([Welcome to Crack-Old Tools, Old Crack Is A Crack Without Login And Crack Usi

Risky [ Zero Tow ] 7 Dec 25, 2022
Mr.Holmes is a information gathering tool (OSINT)

🔍 Mr.Holmes Mr.Holmes is a information gathering tool (OSINT). Is main purpose is to gain information about domains,username and phone numbers with t

534 Jan 08, 2023
SecurAID securely connects aid organizations directly with individuals in dangerous situations to allow them to discreetly and effectively get the assistance they need.

SecurAID securely connects aid organizations directly with individuals in dangerous situations to allow them to discreetly and effec

Ty K 2 Mar 23, 2022
ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software.

ClusterFuzz ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software. Google uses ClusterFuzz to fuzz all

Google 4.9k Jan 08, 2023
OLOP: One-Line & Obfuscated Python

OLOP: One-Line & Obfuscated Python This repository contains useful python modules for one-line and obfuscated python. pip install olop-ShadowLugia650

1 Jan 09, 2022
This repo created for bypassing Widevine L3 DRM and obtaining keys.

First run: Copy headers (with cookies) of POST license request from browser to headers.py like dictionary. pip install -r requirements.txt # if doesn'

Mikhail 263 Jan 07, 2023
Cve-2021-22005-exp

cve-2021-22005-exp 0x01 漏洞简介 2021年9月21日,VMware发布安全公告,公开披露了vCenter Server中的19个安全漏洞,这些漏洞的CVSSv3评分范围为4.3-9.8。 其中,最为严重的漏洞为vCenter Server 中的任意文件上传漏洞(CVE-20

Jing Ling 146 Dec 31, 2022