This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
A simple python code for hacking profile views

This code for hacking profile views. Not recommended to adding profile views in profile. This code is not illegal code. This code is for beginners.

Fayas Noushad 3 Nov 28, 2021
Windows Server 2016, 2019, 2022 Extracter & Recovery

Parsing files from Deduplicated volumes. It can also recover deleted files from NTFS Filesystem that were deduplicated. Installation git clone https:/

0 Aug 28, 2022
A script to extract SNESticle from Fight Night Round 2

fn22snesticle.py A script for producing a SNESticle ISO from a Fight Night Round 2 ISO and any SNES ROM. Background Fight Night Round 2 is a boxing ga

Johannes Holmberg 57 Nov 22, 2022
A hack for writing switch statements with type annotations in Python.

py_annotation_switch A hack for writing switch statements in type annotations for Python. Why should I use this? You most definitely should not use th

6 Oct 17, 2021
Argument Injection in Dragonfly Ruby Gem

CVE-2021-33564 PoC Exploit script for CVE-2021-33564 (Argument Injection in Dragonfly Ruby Gem). Usage Arbitrary File Read python3 poc.py -u https://

Michael Tsai 12 Nov 09, 2022
A script to search, scrape and scan for Apache Log4j CVE-2021-44228 affected files using Google dorks

Log4j dork scanner This is an auto script to search, scrape and scan for Apache Log4j CVE-2021-44228 affected files using Google dorks. Installation:

Jagar 5 Dec 27, 2022
威胁情报播报

Threat-Broadcast 威胁情报播报 运行环境 项目介绍 从以下公开的威胁情报来源爬取并整合最新信息: 360:https://cert.360.cn/warning 奇安信:https://ti.qianxin.com/advisory/ 红后:https://redqueen.tj-u

东方有鱼名为咸 148 Nov 09, 2022
Hacktricks - Welcome to the page where you will find each trick/technique/whatever I have learnt in CTFs, real life apps, and reading researches and news.

Hacktricks - Welcome to the page where you will find each trick/technique/whatever I have learnt in CTFs, real life apps, and reading researches and news.

Carlos Polop 5.8k Jan 07, 2023
Web3 Pancakeswap Sniper & honeypot detector Take Profit/StopLose bot written in python3, For ANDROID WIN MAC & LINUX

🏆 Pancakeswap BSC Sniper Bot web3 with honeypot detector (ANDROID WINDOWS MAC LINUX) 🥇 ⭐️ ⭐️ ⭐️ First SNIPER BOT for ANDROID & WINDOWS with honeypot

Mayank 12 Jan 07, 2023
Exploit tool for Adminer 1.0 up to 4.6.2 Arbitrary File Read vulnerability

AdminerRead Exploit tool for Adminer 1.0 up to 4.6.2 Arbitrary File Read vulnerability Installation git clone https://github.com/p0dalirius/AdminerRea

Podalirius 58 Dec 05, 2022
Keystroke logging, often referred to as keylogging or keyboard capturing

Keystroke logging, often referred to as keylogging or keyboard capturing, is the action of recording the keys struck on a keyboard, typically covertly, so that a person using the keyboard is unaware

Harsha G 2 Jan 11, 2022
Installation of hacking tools

Tools-Spartan This is a program that makes it easy for you to download and install tools used in Kali Linux, there are tons of tools available.

1 Nov 10, 2021
PoC for CVE-2021-26855 -Just a checker-

CVE-2021-26855 PoC for CVE-2021-26855 -Just a checker- Usage python3 CVE-2021-26855.py -u https://mail.example.com -c example.burpcollaborator.net # C

Abdullah AlZahrani 17 Dec 22, 2022
Tool for finding PHP source code vulnerabilities.

vulnz Tool for finding php source code vulnerabilities. Scans PHP source code and prints out potentially dangerous lines. This tool is useful for secu

Mateo Hanžek 1 Jan 14, 2022
Brainly-Scrambler - Brainly Scrambler With Python

Brainly-Scrambler Untuk admin brainly jangan lupa pasang captcha mu Note: Kamu

8 Feb 24, 2022
Quickstart resources for the WiFi Nugget, a cat themed WiFi Security platform for beginners.

Quickstart resources for the WiFi Nugget, a cat themed WiFi Security platform for beginners.

HakCat 62 Jan 08, 2023
Deltaspy - an advanced keylogger that can send keylogs and screenshots to gmail

Deltaspy Deltaspy is a advanced keylogger which sends keylogs and screenshot to

Praanesh S 1 Dec 31, 2021
SonicWALL SSL-VPN Web Server Vulnerable Exploit

SonicWALL SSL-VPN Web Server Vulnerable Exploit

44 Nov 15, 2022
Unauthenticated Sqlinjection that leads to dump data base but this one impersonated Admin and drops a interactive shell

Unauthenticated Sqlinjection that leads to dump database but this one impersonated Admin and drops a interactive shell

sam 16 Nov 09, 2022
Official repository for Pyew.

pyew Pyew is a (command line) python tool to analyse malware. It does have support for hexadecimal viewing, disassembly (Intel 16, 32 and 64 bits), PE

Joxean 362 Nov 28, 2022