This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
A black hole for Internet advertisements

Network-wide ad blocking via your own Linux hardware The Pi-hole® is a DNS sinkhole that protects your devices from unwanted content, without installi

Pi-hole 40.3k Jan 09, 2023
IDA2Obj is a tool to implement SBI (Static Binary Instrumentation).

IDA2Obj IDA2Obj is a tool to implement SBI (Static Binary Instrumentation). The working flow is simple: Dump object files (COFF) directly from one exe

Mickey 94 Dec 13, 2022
A Telegram Bot to force users to join a specific channel before sending messages in a group.

Promoter A Telegram Bot to force users to join a specific channel before sending messages in a group. Introduction A Telegram Bot to force users to jo

Mr. Dynamic 1 Jan 27, 2022
Show apps recorded storage files by jailbreak

0x101 Show registered storage files of apps by jailbreak Legal disclaimer: Usage of insTof for attacking targets without prior mutual consent is illeg

0x 4 Oct 24, 2022
PoC encrypted diary in Python 3

Encrypted diary Sample program to store confidential data. Provides encryption in the form of AES-256 with bcrypt KDF. Does not provide authentication

1 Dec 25, 2021
Using python 3 and Flask an MVC system where the AES 128 CBC and Trivium algorithms

This project was developed using python 3 and Flask, it is an MVC system where the AES 128 CBC and Trivium algorithms can be tested through a communication between the computer and a device such as a

Brandon Israel Camacho Reyes 1 Dec 26, 2021
Unicode fuzzer for various purposes

UnicodeToy Unicode fuzzer for various purposes Unicode based on version 14.0 features Generate the shortest xss domain payload Generate unicode str, u

33 Nov 27, 2022
DependConfusion-X Tool is written in Python3 that scans and monitors list of hosts for Dependency Confusion

DependConfusion-X Tool is written in Python3 which allows security researcher/bug bounty hunter to scan and monitor list of hosts for Dependency Confusion.

Ali Fathi Ali Sawehli 4 Dec 21, 2021
Fast and easy way to rollout on multiple GitLab project file a particular content.

Volatile Fast and easy way to rollout on multiple GitLab project file a particular content. Why ? After looking for a tool to simply enforce a develop

Lujeni 4 Jan 17, 2022
Lazarus analysis tools and research report

Lazarus Research This repository publishes analysis reports and analysis tools for Operation Dream Job and Operation JTrack for Lazarus. Tools Python

JPCERT Coordination Center 50 Sep 13, 2022
Automated tool to find & created Exploit Poc for Clickjacking Vulnerability

ClickJackPoc This tool will help you automate finding Clickjacking Vulnerability by just passing a file containing list of Targets . Once the Target i

Chirag Agrawal 24 Dec 19, 2022
This little tool is to calculate a MurmurHash value of a favicon to hunt phishing websites on the Shodan platform.

MurMurHash This little tool is to calculate a MurmurHash value of a favicon to hunt phishing websites on the Shodan platform. What is MurMurHash? Murm

Viral Maniar 87 Dec 31, 2022
Python program that generates secure passwords.

Python program that generates secure passwords. The user has the option to select the length of the password, amount of passwords,

4 Dec 07, 2021
Lite version of my Gatekeeper backdoor for public use.

Gatekeeper Lite Backdoor Fully functioning bind-type backdoor This backdoor is a fully functioning bind shell and lite version of my full functioning

Joe Helle 56 Mar 25, 2022
A terminal based web shell controller

shell-hack Tribute to Chinese ant sword; A Powerful terminal based webshell controller; Usage : Usage : python3 shell-hack.py --url [URL] --w

s1mple 10 Dec 28, 2021
Ensure secure infrastructure and consistency with the firewall rules

Python Port Scanner This script tries to check if it's possible to make a connection with the specific endpoint port. This is very useful to ensure se

Allan Avelar 7 Feb 26, 2022
This project is all about building an amazing application that will help users manage their passwords and even generate new passwords for them

An amazing application that will help us manage our passwords and even generate new passwords for us.

1 Jan 23, 2022
Uses Sharphound, Bloodhound and Neo4j to produce an actionable list of attack paths for targeted remediation.

GoodHound ______ ____ __ __ / ____/___ ____ ____/ / / / /___ __ ______ ____/ / / / __/ __ \/ __ \/ __

idna 352 Jan 02, 2023
Community Repository for Unofficial Saltbox Add-ons

Saltbox Sandbox Repo Community Repository for Unofficial Saltbox Add-ons Requirements Saltbox Documentation Undetermined Roles List of roles can be fo

Salty Organization 31 Dec 19, 2022
Malware for Discord, designed to steal passwords, tokens, and inject discord folders for long-term use.

Vital What is Vital? Vital is malware primarily used to collect and extract information from the Discord desktop client. While it has other features (

HellSec 59 Dec 01, 2022