This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Overview

2022_the_annotated_transformer

Goal

This Repository is an up-to-date version of Harvard nlp's Legacy code and a Refactoring of the jupyter notebook version as a shell script version.

Key points

  • We have re-factored Harvard NLP's Annotated Trasformer into a shell script version.

  • Dataset utilized Multi30K. (The dataset is small, so you can see the results quickly even on computers with low specifications.)

  • We provide the Colab version along with the shell script version, making it easy to modify the model and test the method.

    https://colab.research.google.com/drive/1SrRmC_Ti8IepeHFNBZBjNxl_wkTSJReC?usp=sharing

  • Loss Graph can be drawn.

  • BLEU Score can be measured.

file structure

├── models
│   ├── __init__.py
│   ├── blocks
│   │   ├── __init__.py
│   │   ├── decoder_layer.py
│   │   ├── encoder_layer.py
│   ├── embedding
│   │   ├── __init__.py
│   │   ├── positional_encoding.py
│   │   └── token_embedding.py
│   ├── layers
│   │   ├── __init__.py
│   │   ├── layer_norm.py
│   │   ├── multi_headed_attention.py
│   │   ├── position_wise_feed_forward.py
│   │   └── sublayer_connection.py
│   ├── model
│   │   ├── __init__.py
│   │   ├── decoder.py
│   │   ├── encoder_decoder.py
│   │   ├── encoder.py
│   │   ├── generator.py
│   └── util.py
├── result
│   ├── loss_graph.png
│   ├── train_loss.txt
│   └── valid_loss.txt
├── saved
├── utils
    ├── __init__.py
    ├── batch.py
    ├── batch_size_fn.py
    ├── bleu.py
    ├── data_loader.py
    ├── epoch_time.py
    ├── greedy_decode.py
    ├── label_smoothing.py
    ├── make_model.py
    ├── NoamOpt.py
    ├── run_epoch.py
    ├── simple_loss_compute.py
    └── tokenizer.py
├── README.md
├── test.py
├── train.py
├── config.py
├── data.py
└── graph.py

Training Result

Train Validation loss graph

image

Test set(unseen data) Translation Example

image

Test set(unseen data) BLEU Score Average: 35.870847920953594

Reference

https://nlp.seas.harvard.edu/2018/04/03/attention.html

https://jalammar.github.io/illustrated-transformer/

https://www.facebook.com/groups/TensorFlowKR/permalink/1618169785190740/

https://github.com/hyunwoongko/transformer

Owner
신재욱
신재욱
To explore creating an application that detects available connections at once from wifi and bluetooth

Signalum A Linux Package to detect and analyze existing connections from wifi and bluetooth. Also checkout the Desktop Application. Signalum Installat

BISOHNS 56 Mar 03, 2021
Pupy is an opensource, cross-platform (Windows, Linux, OSX, Android) remote administration and post-exploitation tool mainly written in python

Pupy Installation Installation instructions are on the wiki, in addition to all other documentation. For maximum compatibility, it is recommended to u

7.4k Jan 04, 2023
Cam-Hacker: Ip Cameras hack with python

Cam-Hacker Hack Cameras Mode Of Execution: apt-get install python3 apt-get insta

Error 4 You 9 Dec 17, 2022
Rapidly enumerate subdomains and domains using rapiddns.io.

Description Simple python module (unofficial) allowing you to access data from rapiddns.io. You can also use it as a module. As mentioned on the rapid

27 Dec 31, 2022
Generate your own NFTs and their metadata based on your desired probabilities.

Generate your own NFTs and their metadata based on your desired probabilities. Use your own art assets too! Perfect for use with Candy Machine.

hex 7 Sep 16, 2022
NEW FACEBOOK CLONER WITH NEW PASSWORD, TERMUX FB CLONE, FB CLONING COMMAND. M

NEW FACEBOOK CLONER WITH NEW PASSWORD, TERMUX FB CLONE, FB CLONING COMMAND. M

Mr. Error 81 Jan 08, 2023
Sonoff NSPanel protocol and hacking information. Tasmota Berry driver for NSPanel

NSPanel Hacking Sonoff NSPanel protocol and hacking information and Tasmota Berry driver. NSPanel protocol manual Tasmota driver nspanel.be Installati

blakadder 98 Dec 26, 2022
🐎🖥《赛马娘》(ウマ娘: Pretty Derby)辅助脚本

auto-derby 自动化养马 育成结果 Nurturing result 功能 支持客户端 DMM (前台) 实验性 安卓 ADB 连接(后台)开发基于 1080x1920 分辨率 团队赛 (Team race) 有胜利确定奖励时吃帕菲 日常赛 (Daily race) PvP 活动赛 (Cha

NateScarlet 376 Jan 01, 2023
🍯 16 honeypots in a single pypi package (DNS, HTTP Proxy, HTTP, HTTPS, SSH, POP3, IMAP, STMP, VNC, SMB, SOCKS5, Redis, TELNET, Postgres & MySQL)

Easy to setup customizable honeypots for monitoring network traffic, bots activities and username\password credentials. The current available honeypot

QeeqBox 259 Dec 31, 2022
Vulnerability Exploitation Code Collection Repository

Introduction expbox is an exploit code collection repository List CVE-2021-41349 Exchange XSS PoC = Exchange 2013 update 23 = Exchange 2016 update 2

0x0021h 263 Feb 14, 2022
Anti-Nuke capabilities, powerful moderation features, auto punishments, captcha-verification and more.

Server-Security-Discord-Bot Anti-Nuke capabilities, powerful moderation features, auto punishments, captcha-verification and more. Installation Instal

20 Apr 07, 2022
ShoLister - a tool that collects all available subdomains for specific hostname or organization from Shodan

ShoLister is a tool that collects all available subdomains for specific hostname or organization from Shodan. The tool is designed to be used from Penetration Tester and Bug Bounty Hunters.

Eslam Akl 45 Dec 28, 2022
Enhancing Twin Delayed Deep Deterministic Policy Gradient with Cross-Entropy Method

Enhancing Twin Delayed Deep Deterministic Policy Gradient with Cross-Entropy Method Hieu Trung Nguyen, Khang Tran and Ngoc Hoang Luong Setup Clone thi

Evolutionary Learning & Optimization (ELO) Lab 6 Jun 29, 2022
This is a simple Port Flooder written in Python 3.

This is a simple Port Flooder written in Python 3. Use this tool to quickly stress test your network devices and measure your router's or server's load.

Júlio Carneiro 4 Feb 20, 2022
MD5-CRACKER - A gmail brute force app created with python3

MD5-CRACKER So this is my first app i created with python3 . if you guys downloa

2 Nov 10, 2022
A python script to bypass 403-forbidden.

4nought3 A python script to bypass 403-forbidden. It covers methods like Host-Header Injections, Changing HTTP Requests Methods and URL-Injections. Us

11 Aug 27, 2022
A hashtag check python module

A hashtag check python module

Fayas Noushad 3 Aug 10, 2022
Aviatrix Controller 6.x before 6.5-1804.1922. Unrestricted upload of a file which allows an unauthenticated user to execute arbitrary code via directory traversal

CVE-2021-40870 Exploitation An issue was discovered in Aviatrix Controller 6.x before 6.5-1804.1922. Unrestricted upload of a file with a dangerous ty

Ashraful Islam 16 Nov 15, 2022
A tool to find good RCE From my series: A powerful Burp extension to make bounties rain

A tool to find good RCE From my series: A powerful Burp extension to make bounties rain

52 Dec 16, 2022
Simple python script for generating custom high-secure passwords for securing your social-apps ❤️

Opensource Project Simple Python Password Generator This repository is just for peoples who want to generate strong-passwords for there social-account

K A R T H I K 15 Dec 01, 2022