Injector/automatic translator (using deepL API) for Tsukihime Remake

Overview

deepLuna

Extractor/Editor/Translator/Injector for Tsukihime Remake

About

deepLuna, from "deepL", the machine translation service, and "Luna", the name of the Roman Moon goddess, is a Python script with a GUI interface that works as an extractor/editor/translator (with deepL API)/injector for the text of the Tsukihime Remake game on Switch and possibly on PS4, while taking into account the whole internal structure of the script.

Installation

You just need a regular Python distribution, for example the last one from https://www.python.org/downloads/, and then launch the script deepLuna.py using a Python IDE, for example IDLE that you get by default, or some other (Spyder, Pyzo, etc.).

Usage

Preprocessing steps

  1. Dump/Find the NSP Tsukihime Switch ROM as well as the relevant keys (prod.keys and title.keys)
  2. Extract the NSP and the resulting (biggest) NCA file using for example hactool (https://github.com/SciresM/hactool)
  3. In the romfs folder, copy the script_text.mrg and allscr.mrg files and paste them in the same folder as deepLuna.py.

Extracting the text (first compulsory step)

After launching deepLuna, click on "Extract text" and wait until the extraction finishes.

IMPORTANT NOTE: This might take a bit of time, around a minute, and this is normal if you have the feeling that the graphical interface is lagging. The script is actually working at the command line level, so you can follow what is going on in the python console or in the console of the IDE you're using.

The main editor window should open. If the script complains (an error window will open), make sure that you placed both script files in the same folder as deepLuna and check that their respective names are "script_text.mrg" and "allscr.mrg" (these are the original names of the files).

Editing the text

Once the first extraction has been done, when launching deepLuna simply click on "Open translation". This might also take some seconds.

The main editor window contains three main elements:

  • On the left there is a tree view of the text files of the game, and it is the only active element at the beginning. This section is the organized by heroine and days ("Arcueid" and "Ciel"). The remaining three sections concern the bad end section "Teach Me, Ciel-sensei!", the part that is common to both Arcueid's and Ciel's route called "Common" and finally the section "Hidden text" that shows some text grabbed by the script that is not used ingame. The first thing you have to do is choose one of the scenes that appear as elements of the form "TEXT_WITH_UNDERSCORES_AND_NUMBERS", and then double-click on it. IMPORTANT NOTE: When clicking on one of the element, due to the structure of the script, you might have to wait a few seconds before the next content loads - this is perfectly normal.
  • The second element from the left, a scrolling list, activates. It shows the content of the selected scene in the form "a : b" where a is the page and b is the line on the page a. Double-click on any element. You might also note at this point that the two fields above activated, and will show up the percentage of translation done on the selected scene as well as for the global file, computed both in real time. We will come back to them in the section below.
  • Finally, you have the main editing part: a field that shows the original japanese selected line called "Original text", a field called "Translated text" below where you have to input your translation of the line above by replacing the word "TRANSLATION" that is there, and finally some buttons that will allow you to perform all the relevant actions.

IMPORTANT NOTE: When in some line you meet the pound/hashtag character "#", PLEASE do not delete it and leave as it is in the translation! This character symbolizes the fact that the line is shown ingame with a small break where the "#" is, and from a low-level perspective, symbolizes that the line is cut into two separate strings with altogether different pointers, so erasing it WILL break everything.

Commands

Validate: When you finish inputting the translation for a line, click on this button. The corresponding offset will become green to confirm the translation and the percentage of advancement of translation will update for the scene as well as for the total game.

Save: When you finish your translation, simply click on this button to save everything. IMPORTANT NOTE: This operation might also take a few seconds, this is also normal.

Search: Opens a window to search for some text. Enter the text in the first field, and click on "Search". Move between the results with "Previous"/"Next". If you want to replace some text (works only for the translated text for obvious reasons), simply use the last field in the window and then click on "Replace"/"Replace all" (the number of results will update automatically). This allows you to search for text only in the selected scene, not in the whole game script.

Translating the text

Select an offset or a range of offset (using Shift+double-click), then click on the "Translate" button in the main screen, a new window will open. The first time you execute the script, you'll have to configure three options (after that, they will be saved in a config.ini file created in deepLuna folder after the initial launch and load automatically everytime you launch deepLuna):

a) "deepL API link": Input here your API link(s) from deepL (replace the 0 that is there). If you have several of them, just separate them by a comma - the script will try them successively when a translation request fails. Currently works only for free deepL accounts (possible future update for a pro deepL API link). NOTE: If you copy/paste your API link and it doesn't work, make sure that there is not an additional space character that was copied at the end of the link, this might happen.

b) "Language": Put the initials of the language into which you want to translate the script (e.g. EN for english, FR for french, DE for german, IT for italian, RU for russian, PL for polish, ES for spansih, etc.)

c) "Block translation": If you keep this box unchecked, deepLuna will send each line separately to deepL for translation. If you check the box, deepLuna will glue all the lines from the selected range of offsets together, and send the whole block to deepL before splitting them again. From experience, the latter option might improve the quality of translation since you give more context to deepL, but on the other hand produces sometimes completely unrelated translated sentences in the middle if the block is really big. You have to play around to see what option is the best depending on context and chosen sentences.

IMPORTANT NOTE: The furigana are thrown away during translation at the moment since their structure prevent deepL from giving a consistent translation, and also because their meaning and positioning sometimes doesn't match at all the original word (Nasu writing style is really something...). Maybe I'll try to implement something to deal with this problem in the future, but for the moment it is as it is.

Now, it suffices to click on "OK" and wait until translation finishes. If something fails (too much deepL requests for example), the program will tell you and if you've chosen the line-by-line translation approach, will point you out the line where it stopped.

Injecting the text back

Easy as pie, simply click on the button "Insert" and wait a little bit (it might take up to ~20 seconds on some older computers). deepLuna will generate a new script file of the form "script_text_translatedDATETIME.mrg", with DATETIME being the date and time of insertion (to avoid overwriting files and keeping backups). NOTE: The script inserts everytime the full translation.

NEW: Exporting functions

Since the v2.0, deepLuna makes your life easier if you're working as a group on the translation, which is usually the case. How to keep everything up to date for all the people? Whenever you finish working on some scene, just press on "Export", and deepLuna will generate an update file in the update folder, in the same directory. If you look inside, you will see that this is a simple text file with the same name as the scene you were working on, and if you open it, you'll realise that this is exactly the case - the exported file contains the whole scene in a very nice and readable way, with the original sentences where they were, and the freshly translated sentences where the japanese ones were supposed to be. It suffices now to send this file to your colleagues, and they will simply have to put it in the "update" folder. When they launch deepLuna the next time they want to use it, it will update itself and include the new file in its own database.

If you need for some reason to export your whole personal database, this is also doable - simply click on "Export all", and a new window will open, asking you for confirmation. Note that this operation might take up to 10 mn on some older computers.

On another note, this update function makes it possible to have an alternative use of deepLuna: simply export the scene file, then edit it directly with your favorite text editor, and finally share it with your teammates or use the "Insert" function to inject it in the script yourself.

Additional comments

Feel free to raise an issue if something happens or if you wish to have some specific functionality, I'm always open to new possibilities and improvements! On another note, if you end up using this tool in some way or the other for your translation, please simply credit the script name itself, deepLuna, in some place or the other on your website/patched game/other.

This readme will get a more substantial update in the following days with screenshots and/or video tutorials.

Acknoledgments

I am really thankful to Hintay for his wonderful tools at https://github.com/Hintay/PS-HuneX_Tools. I relied on them heavily (unpack_allsrc.py, prep_tpl.py and the mzx decompression script) and included a modified version of them in deepLuna to make it perfectly operational.

Comments
  • Encoding question

    Encoding question

    Sorry to bother you again, and for the dumb question too; but is there a way to have the encoding supported for other languages? I was testing translations in my native language - which makes a lot of use of commas and accents - however, when executed the letters are always distorted since the format is not supported

    opened by Zac91281 13
  • ModuleNotFoundError (I do have the module installed)

    ModuleNotFoundError (I do have the module installed)

    Traceback (most recent call last): File "C:\Users\arthu\Desktop\deepLuna-main\deepLuna-main\deepLuna.py", line 18, in from PIL import ImageTk, Image ModuleNotFoundError: No module named 'PIL'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:\Users\arthu\Desktop\deepLuna-main\deepLuna-main\deepLuna.py", line 20, in install("Pillow") NameError: name 'install' is not defined

    The problem here is that when I try to run the script on IDE this errors occurs and I just don't know what to do since I already have everything needed installed.

    opened by Arthurfogo7 10
  • Strange Error

    Strange Error

    I've been doing the deepLuna translation in increments and have had minimal issues. I got up to 44% and now whenever I try to translate again, no matter the line or chunk of lines, I'm getting this.

    Exception in Tkinter callback
    Traceback (most recent call last):
      File "C:\Users\Jeremy\AppData\Local\Programs\Python\Python39\lib\tkinter\__init__.py", line 1892, in __call__
        return self.func(*args)
      File "F:\tsukihime\deepLuna-main\deepLuna-main\deepLuna.py", line 910, in translate_game
        if self.i < cs[-1]:
    IndexError: tuple index out of range
    
    opened by jeremoople 7
  • Not opening

    Not opening

    Whenever I try to open deepLuna, python opens and closes it instantly. I'm using the latest version and the files are placed with the needed extracted files from the game. My knowledge is pretty lay, so I don't know what to do.

    opened by Zac91281 5
  • Decode error on new build

    Decode error on new build

    I decide to check new build, and it's give a error when i try to press "Extract Text" or "Open translation". Traceback (most recent call last): File \luna\ui\start_window.py", line 166, in extract_database tl_db = TranslationDb.from_mrg( File \luna\translation_db.py", line 387, in from_mrg strings_by_offset[i] = data.decode('utf-8') UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

    opened by SkyAshpd 4
  • All of the .py files keep immediately closing?

    All of the .py files keep immediately closing?

    So for some reason every time I try to open deepLuna.py it keeps immediately closing with no text or anything at all. I'm quite confused as I have python downloaded.

    opened by Rescuro 2
  • Install error

    Install error

    Every time I try to run deepLuna.py in IDLE or Pyzo, I get this error:

    Traceback (most recent call last):
      File "F:\tsukihime\deepLuna-main\deepLuna-main\deepLuna.py", line 18, in <module>
        from PIL import ImageTk, Image
    ModuleNotFoundError: No module named 'PIL'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "F:\tsuikihime\deepLuna-main\deepLuna-main\deepLuna.py", line 20, in <module>
        install("Pillow")
    NameError: name 'install' is not defined
    >>> 
    

    I'm not used to installing programs using Python as opposed to like, a setup wizard, so I could very well just be making a basic newbie mistake, but still, I don't know where to go from here.

    opened by jeremoople 2
  • Issue with inserting translation

    Issue with inserting translation

    When I press the insert button on the translation window, I get this error in the console window:

    Saving file, please wait...
    Exception in Tkinter callback
    Traceback (most recent call last):
      File "C:\Users\fergu\AppData\Local\Programs\Python\Python38-32\lib\tkinter\__init__.py", line 1883, in __call__
        return self.func(*args)
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1518, in function_insert_translation
        self.enregistrer_fichier()
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1718, in enregistrer_fichier
        self.table_file = self.reinsert_daytable(self.table_scr_file,self.table_file)
      File "C:\Users\fergu\AppData\Roaming\yuzu\load\01001DC01486A000\deepLuna\romFS\deepLuna.py", line 1510, in reinsert_daytable
        if self.splitDayTable[self.i][0] == self.newMainTable[self.j][0] and self.splitDayTable[self.i][2] != 'TRANSLATION':
    TypeError: 'type' object is not subscriptable
    

    Any ideas as to what I can do to fix the issue? I'm currently using python 3.8.5.

    opened by shingle-bells 1
  • Fix cursor accounting for _long_ split lines

    Fix cursor accounting for _long_ split lines

    Previously, when breaking lines, we would take the length of the final section of the broken line and add that to the cursor position, e.g.

    ZM("20 characters of text");
    // Cursor position: 20
    MSAD("20 characters of text")
    // Cursor position: 40
    MSAD("5 characters of text")
    // Cursor position: 45
    -> "20 characters of text20characters of text5 characters of text"
    

    however, this is wrong for lines where a MSAD causes a line break -

    ZM("20 characters of text");
    // Cursor position: 20
    MSAD("40 characters of text")
    // Gets wrapped to 35 chars + 5 chars
    // Cursor position: 20 + 5 (WRONG!)
    
    opened by rschlaikjer 0
  • Injecting Question

    Injecting Question

    Greetings. I know it isn't quite a issue, but I don't know where else to ask. But should the edited script go in some specific folder? Mine is within the DeepLuna folder, already replacing the original file, but the translation is not appearing. image

    opened by Zaldi556 0
Releases(v5.2)
  • v5.2(Jan 16, 2022)

  • v4.6(Jan 9, 2022)

    • deepLuna now uses new 'Literate' import/export script format
    • Internal database structure now JSON-based
    • Various performance improvements
    • Added luna_cli and luna_linter to facilitate scripting of simple workflows
    Source code(tar.gz)
    Source code(zip)
  • v3.1.1(Oct 18, 2021)

    • Major performance improvements, in particular importing update files and loading scenes takes now only split seconds
    • A new comment field has been added (see Readme for more details)
    • A new "swap character" option has been added for languages with special characters (see Readme for more details)
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v3.1.1.zip(224.48 KB)
  • v2.3(Oct 6, 2021)

  • v2.2.1(Sep 14, 2021)

  • v2.2(Sep 10, 2021)

  • v2.1(Sep 7, 2021)

    Major upgrade of deepLuna:

    • Extraction algorithm and main interface completely remodelled
    • Taking now into account the underlying day/heroine structure of the files
    • Proposes a new simple and intuitive function to share your updated translation with you teammates
    • Added many small improvements to the interface that enhances the global user experience
    • Corrected many different bugs
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v2.1.zip(215.62 KB)
  • v1.0(Sep 6, 2021)

    The is the initial release of deepLuna. It contains the following elements:

    • a linear extraction module of the Tsukihime Remake script coming from the script_text.mrg file
    • a graphical interface edition module with some basic usual functions, such as search and replace
    • a machine translation module using deepL API links as a professional translation possible helper
    • an injection module that injects the translated text in the script_text.mrg file
    Source code(tar.gz)
    Source code(zip)
    deepLuna_v1.0.zip(203.09 KB)
A badge generator service to count visitors of your markdown file.

Github Visitors Badge A badge generator service to count visitors of your markdown file. Hello every one! In this post, I will tell you the story of m

Kɪꜱᴀʀᴀ Pᴇꜱᴀɴᴊɪᴛʜ Pᴇʀᴇʀᴀ 〄 1 Feb 06, 2022
Quickly edit your slack posts.

Lightning Edit Quickly edit your Slack posts. Heavily inspired by @KhushrajRathod's LightningDelete. Usage: Note: Before anything, be sure to head ove

Cole Wilson 14 Nov 19, 2021
OGE-2022-na-Python - Solving problems in python for the OGE 2022

OGE-2022-na-Python Решение задачек на питоне для ОГЭ 2022 Тут разобраны разные в

Slava 0 Oct 14, 2022
Бот Telegram для Школы в Капотне (ЦО № 1858)

co1858 Telegram Bot Активно разрабатывался в 2015-2016 году как учебный проект, с целью научиться создавать ботов для Telegram. Бот автоматически парс

Ilya Pavlov 4 Aug 30, 2022
This repo contains a small project i've done using PILLOW module in python

This repo contains a small project i've done using PILLOW module in python. I wrote an automated script which generates more than 5k+ unique nfts with 0 hassle in less time.

SasiVatsal 11 Nov 05, 2022
A tool for creating credentials for accessing S3 buckets

s3-credentials A tool for creating credentials for accessing S3 buckets For project background, see s3-credentials: a tool for creating credentials fo

Simon Willison 138 Jan 06, 2023
It is a useful project for developers that includes useful tools for Instagram

InstagramIG It is a useful project for developers that includes useful tools for Instagram Installation : pip install InstagramIG Logan Usage from In

Sidra ELEzz 14 Mar 14, 2022
Draw your telegram bot in draw.io and generate its code

README Contents: Draw your bot Install requirements Registering a telegram bot Draw bot Start point Message block Single choice block Functions block

DENIS TSITKO 23 Nov 09, 2022
Enables you to execute scripts and perform API requests in MikroTik router

HomeAssistant component: MikroTik API The mikrotik_api platform enables you to execute scripts and perform API requests in MikroTik router To enable M

Pavel S 6 Aug 12, 2022
Telegram forwarder

Telegram Forwarder Quick Start This application using docker, docker-compose to run. So I suppose that you can install these two things. Prepare essen

10 Dec 20, 2022
Microsoft Azure Storage Library for Python

Microsoft Azure Storage Library for Python

Microsoft Azure 329 Dec 16, 2022
procedurally-generated catppuccin wallpapers

Catppuccin Wallpapers I was in need of wallpapers conforming to the catppuccin colorscheme. So, I wrote a simple script to iteratively generate a set

daylin 11 Nov 29, 2022
Leakvertise is a Python open-source project which aims to bypass these fucking annoying captchas and ads from linkvertise, easily

Leakvertise Leakvertise is a Python open-source project which aims to bypass these fucking annoying captchas and ads from linkvertise, easily. You can

Quatrecentquatre 9 Oct 06, 2022
WIOpy - Walmart Affiliate API Python wrapper

WalmartIO Python Wrapper - WIOpy A python wrapper for the Walmart io API. Only s

6 Nov 14, 2022
Pretend to be a discord bot

Pretendabot © Pretend to be a discord bot! About Pretendabot© is an app that lets you become a discord bot!. It uses discord intrigrations(webhooks) a

Advik 3 Apr 24, 2022
A zero-dependency Python library for getting the Kubernetes token of a AWS EKS cluster

tokeks A zero-dependency Python library for getting the Kubernetes token of a AWS EKS cluster. No AWS CLI, third-party client or library (boto3, botoc

Chris Karageorgiou Kaneen 6 Nov 04, 2022
Discord-RAID-Tool - Hacks/tools

How to use Python must be installed run install-config If you dont have python installed, download python 3.7.6 and make sure you click on the 'ADD TO

1 Jan 01, 2022
Bot Telegram per creare e gestire un Babbo Natale Segreto con amici ecc

Babbo Natale Segreto: Telegram Bot Bot Telegram per creare e gestire un Babbo Natale Segreto con amici ecc. Che cos'è? Il Babbo Natale Segreto è un gi

Francesco Ciociola 2 Jul 18, 2022
SUPPORTS 500 GROUPS NO NEED OF BOT 😉

LOVELY RADIO SUPPORTS 500 GROUPS NO NEED OF BOT 😉 Requirements Telegram API_ID , API_HASH and SESSION_NAME HEROKU Get YouTube live stream link instal

6 Nov 24, 2021
all-in-one wrapper for NASA API's

=========== About bowshock is an all-in-one wrapper for NASA API's. Here is a list of currently supported API's : NASA Earth API NASA APOD (Astronomy

Emir Ozer 85 Nov 09, 2022