This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
Raphael is a vulnerability scanning tool based on Python3.

Raphael Raphael是一款基于Python3开发的插件式漏洞扫描工具。 Raphael is a vulnerability scanning too

b4zinga 5 Mar 21, 2022
Script for automatic dump and brute-force passwords using Volatility Framework

Volatility-auto-hashdump Script for automatic dump and brute-force passwords using Volatility Framework

whoamins 11 Apr 11, 2022
RCE 0-day for GhostScript 9.50 - Payload generator

RCE-0-day-for-GhostScript-9.50 PoC for RCE 0-day for GhostScript 9.50 - Payload generator The PoC in python generates payload when exploited for a 0-d

534 Dec 14, 2022
Acc-Data-Gen - Allows you to generate a password, e-mail & token for your Minecraft Account

Acc-Data-Gen Allows you to generate a password, e-mail & token for your Minecraft Account How to use the generator: Move all the files in a single dir

KarmaBait 2 May 16, 2022
Holehe OSINT - Email to Registered Accounts

holehe allows you to check if the mail is used on different sites like twitter, instagram and will retrieve information on sites with the forgotten password function.

Palenath 3.8k Jan 06, 2023
Brainly-Scrambler - Brainly Scrambler With Python

Brainly-Scrambler Untuk admin brainly jangan lupa pasang captcha mu Note: Kamu

8 Feb 24, 2022
Mert Güvençli 142 Jan 05, 2023
zip-brute Zip File Password Cracking with Using Password List

Zip brute is a python script that cracks zip that are password protected using a wordlist dictionary.

AnonyminHack5 13 Nov 03, 2022
Fast and customizable vulnerability scanner For JIRA written in Python

Fast and customizable vulnerability scanner For JIRA. 🤔 What is this? Jira-Lens 🔍 is a Python Based vulnerability Scanner for JIRA. Jira is a propri

Mayank Pandey 185 Dec 25, 2022
Dependency Combobulator is an Open-Source, modular and extensible framework to detect and prevent dependency confusion leakage and potential attacks.

Dependency Combobulator Dependency Combobulator is an Open-Source, modular and extensible framework to detect and prevent dependency confusion leakage

Apiiro 84 Dec 23, 2022
this keylogger is only for pc not for android but it will only work on those pc who have python installed it is made for all linux,windows and macos

Keylogger this keylogger is only for pc not for android but it will only work on those pc who have python installed it is made for all linux,windows a

Titan_Exodous 1 Nov 04, 2021
Receive notifications/alerts on the most recent disclosed CVE's.

Receive notifications on the most recent disclosed CVE's.

Ameliorate 7 Nov 24, 2022
IDAPatternSearch adds a capability of finding functions according to bit-patterns into the well-known IDA Pro disassembler based on Ghidra’s function patterns format.

IDA Pattern Search by Argus Cyber Security Ltd. The IDA Pattern Search plugin adds a capability of finding functions according to bit-patterns into th

David Lazar 48 Dec 29, 2022
Lightweight and beneficial Dependency Injection plugin for apscheduler

Implementation of dependency injection for apscheduler Prerequisites: apscheduler-di solves the problem since apscheduler doesn't support Dependency I

Glib 11 Dec 07, 2022
WinRemoteEnum is a module-based collection of operations achievable by a low-privileged domain user.

WinRemoteEnum WinRemoteEnum is a module-based collection of operations achievable by a low-privileged domain user, sharing the goal of remotely gather

Simon 9 Nov 09, 2022
NEW FACEBOOK CLONER WITH NEW PASSWORD, TERMUX FB CLONE, FB CLONING COMMAND. M

NEW FACEBOOK CLONER WITH NEW PASSWORD, TERMUX FB CLONE, FB CLONING COMMAND. M

Mr. Error 81 Jan 08, 2023
Arbitrium is a cross-platform, fully undetectable remote access trojan, to control Android, Windows and Linux and doesn't require any firewall exceptions or port forwarding rules

About: Arbitrium is a cross-platform is a remote access trojan (RAT), Fully UnDetectable (FUD), It allows you to control Android, Windows and Linux an

Ayoub 861 Feb 18, 2021
Confluence Server Webwork OGNL injection

CVE-2021-26084 - Confluence Server Webwork OGNL injection An OGNL injection vulnerability exists that would allow an authenticated user and in some in

Fellipe Oliveira 295 Jan 06, 2023
Make files with as many random bytes as you want

Lots o' Bytes 🔣 Make files with as many random bytes as you want! Use case Can be used to package malware that is normally small by making the downlo

Addi 1 Jan 13, 2022
👑 Discovery Header DoD Bug-Bounty

👑 Discovery Header DoD Bug-Bounty Did you know that DoD accepts server headers? 😲 (example: apache"version" , php"version") ? In this code it is pos

KingOfTips 38 Aug 09, 2022