Optimizes image files by converting them to webp while also updating all references.

Overview

About

Optimizes images by (re-)saving them as webp. For every file it replaced it automatically updates all references. Works on single files as well as on folders.

Installation: Windows

Step 1: Get the exe

Option 1: Download

Download one of the pre-built fvttoptimizer.exe files from Releases

Go to Step 2: Install program

Option 2: Build it yourself

Install python3 (version >= 3.8) and add it to your PATH system environment variable.

You can check the version via CMD or powershell with:

python --version

Download or clone the repo.

Open a CMD instance inside the project root folder and create a virtual environment.

python -m venv .\venv

Activate the venv:

.\venv\Scripts\activate

Install the Pillow package:

pip install pillow==9.1.0

Install pyInstaller package:

pip install pyInstaller==4.10

Download the fvttmv python package (whl file) from here: https://github.com/watermelonwolverine/fvttmv/releases/tag/0.2.4 move it into the folder and install it:

pip install fvttmv-0.2.4-py3-none-any.whl

Run build file:

.\scripts\build_for_windows.cmd

You should now have a fvttoptimizer.exe file under dist. After the build succeeded you can delete the venv you previously created.

Go to Step 2: Install program

Step 2: Install program

Create an empty folder where you want to install the program for example C:\fvttoptimizer

Copy fvttoptimizer.exe into that folder.

Create a fvttoptimizer.conf text file in that folder.

Copy {"absolute_path_to_foundry_data":"INSERT_PATH_HERE"} into fvttoptimizer.conf

Replace INSERT_PATH_HERE with the path to the Data folder inside your foundrydata (Not the foundrydata folder itself!).

IMPORTANT: Escape all \ with \\ in that path.

It should look something like this:

{"absolute_path_to_foundry_data":"C:\\Users\\user\\foundrydata\\Data"}

Add the installation path to your PATH system environment variable.

Uninstallation: Windows

Delete fvttoptimizer.exe and fvttoptimizer.conf files from the installation directory.

Remove the path to the installation directory from the PATH system environment variable.

Installation: Ubuntu 16.04 -20.04

Optional: Install Python

An up-to-date Ubuntu 20.04 should have python >= 3.8. Check it with:

python3 --version

From here on pythonX will be used as placeholder for the python you should use. Depending on your system you need to replace X with 3, 3.8, 3.9 or 3.10 .

Install python version>=3.8 if not yet installed.

sudo apt install pythonX

Step 1: Get the executable

Option 1: Download

Download one of the pre-built fvttoptimizer files from Releases

Go to Step 2: Install the files

Option 2: Build it yourself

Install python if haven't already.

Install Pillow dependencies (read more here https://pillow.readthedocs.io/en/stable/installation.html):

sudo apt install pythonX-dev libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk libharfbuzz-dev libfribidi-dev libxcb1-dev

Download or clone repo.

Create venv inside the repo:

sudo apt install pythonX-venv

pythonX -m venv ./venv

Activate venv:

source venv/bin/activate

Install Pillow (should succeed if you installed all the dependencies):

pythonX -m pip install Pillow==9.1.0

Install pyInstaller:

pythonX -m pip install pyInstaller==4.10

Download the fvttmv python package (whl file) from here: https://github.com/watermelonwolverine/fvttmv/releases/tag/0.2.4 move it into the folder and install it:

pythonX -m pip install fvttmv-0.2.4-py3-none-any.whl

Now run the build script:

./scrips/build_for_ubuntu.sh

Step 2: Install the files

Option 1: Automatic installation

Install python if you haven't already.

Clone the repo if you haven't already.

Go into the project folder.

Build the project if you haven't already. If you downloaded a pre-built executable. Create a folder named dist and move the file there.

Inside the project folder run:

sudo pythonX scripts/install_on_ubuntu.sh

Option 2: Manual installation

Copy the fvttoptimizer file either from dist from where you downloaded it to usr/bin/fvttoptimizer

Make the file executable:

sudo chmod ugo=rx usr/bin/fvttoptimizer

Create a fvttoptimizer.conf file at /etc/

Copy {"absolute_path_to_foundry_data":"INSERT_PATH_HERE"} into fvttoptimizer.conf

Replace INSERT_PATH_HERE with the path to the Data folder inside your foundrydata (Not the foundrydata folder itself!).

It should look something like this:

{"absolute_path_to_foundry_data":"/home/user/foundrydata/Data"}

Uninstallation: Ubuntu

Delete the files /etc/fvttoptimizer.conf and /usr/bin/fvttoptimizer

Usage

IMPORTANT:

Shut down Foundry VTT before optimizing any files.

Use this program at your own risk. It has been tested extensively but there are no guarantees. Always keep backup of your Foundry VTT data for cases where something goes wrong.

Syntax

fvttoptimizer [--verbose-info, --verbose-debug, --version, --help, --quality value, --override-percent value, --skip-webp, --skip-existing, --recursive] target

target: Path to file or directory which should be optimized

Options

--verbose-info: Enables verbose output to console

--verbose-debug: Enables very verbose output to console

--version: Prints version and exits

--help: Display help and exit

--skip-existing: Ignores files of which a webp already exists. For example image.png will not be converted if a image.webp is already in the same folder

--skip-webp: Don't touch webp files at all

--quality: The quality setting for the webp compression. Default is 75

--override-percent: How much smaller the new file needs to be to replace the old one. For example if this value is 25 the file size after optimizing needs to be 25% smaller than the original. Default is 25

--recursive: If the optimization should be done recursively to all sub folders of the target folder.

Examples

Optimizing a file:

Ubuntu: fvttoptimizer path/to/file.png

Windows: fvttoptimizer path\to\file.png

Optimizing a folder and all it's sub folders:

Ubuntu: fvttoptimizer --recursive pathto/folder

Windows: fvttoptimizer.exe --recursive path\to\folder

Use single quotes when moving a file with a space:

Ubuntu: fvttoptimizer 'some folder/some file.jpg'

Windows: fvttoptimizer.exe 'some folder\some file.jpg'

Change parameters:

Ubuntu: fvttoptimizer --quality 50 --override-percent 50 file.jpg

Windows: fvttoptimizer --quality 50 --override-percent 50 file.jpg

Known Issues and Quirks

Windows

The program only works in powershell not in cmd.

When one of the paths has \' at the end, the arguments will get mixed up. This is a problem with how python handles arguments and probably can't be fixed. For example on Windows fvttoptimizer.exe '\folder name with spaces\' will fail but fvttoptimizer.exe '\folder name with spaces' will succeed.

Comments
  • Program fails on special character [ in path

    Program fails on special character [ in path

    First off, thank you so much for making this.

    I am trying to use the following command: fvttoptimizer.exe --recursive 'C:\Users\cnguy\AppData\Local\FoundryVTT\Data\Battle Maps'

    and I am getting this return: An unexpected error occurred: expected str, bytes or os.PathLike object, not NoneType

    Your documentation implies this is supported, as this command works without the "--recursive" flag, so maybe there is an issue with spaces in paths and the recursive function?

    Thank you.

    opened by coboba123 5
  • Trying to create a Docker image - Keep getting 'image.jpg is not in the configured foundry Data folder.'

    Trying to create a Docker image - Keep getting 'image.jpg is not in the configured foundry Data folder.'

    As the title say, I'm trying to get this tool to work in a Docker container, but I'm having an issue that I can't figure out...

    I'm building the image like so:

    FROM python:3.10.0
    
    ARG FVTT_OPTIMIZER_VERSION=0.1.3
    
    RUN curl -L https://github.com/watermelonwolverine/fvttoptimizer/releases/download/$FVTT_OPTIMIZER_VERSION/fvttoptimizer --output usr/bin/fvttoptimizer
    RUN chmod +x usr/bin/fvttoptimizer
    RUN echo '{"absolute_path_to_foundry_data":"/app/Data"}' > /etc/fvttoptimizer.conf
    
    CMD ["fvttoptimizer"]
    

    Which I then configure with docker-compose:

    version: "3.7"
    
    services:
        fvttoptimizer:
            build: python
            volumes:
                - ${FOUNDRY_DATA_VOLUME}:/app
    
    # FOUNDRY_DATA_VOLUME is the directory that contains `Config`, `Data` and `Logs`
    

    However, when ran (docker-compose run --rm fvttoptimizer fvttoptimizer --verbose-debug ./Barney.jpg) I get the error in the title. Running docker-compose run --rm fvttoptimizer ls -la /app/Data, I can confirm the image exists.

    The debug output looks like this:

    root - DEBUG - Got arguments ['fvttoptimizer', '--verbose-debug', '--recursive', './Barney.jpg']
    root - DEBUG - Reading config from: /etc/fvttoptimizer.conf
    root - DEBUG - Running with target_path='./Barney.jpg', config='{'_RunConfig__program_config': {'_ProgramConfigImpl__abs_path_to_foundry_data': '/app/Data'}, 'skip_existing': False, 'skip_webp': False, 'recursive': True, 'quality': 75, 'override_percent': 25}'
    root - ERROR - /Barney.jpg is not in the configured foundry Data folder.
    /Barney.jpg is not in the configured foundry Data folder.
    

    I pushed a branch on my repo where it should be possible to reproduce the issue, or at the very least look at my configuration: https://github.com/gCardinal/foundrty-vtt/blob/feature/use-fvttoptimizer

    support 
    opened by gCardinal 3
  • Missing config file

    Missing config file

    When running on an image file I get the error:

    Missing config file. Could not find /etc/fvttoptimizer.conf

    Tried running all three release versions on Arch Linux with Python 3.10.4 and get the same error every time.

    I'm not seeing any mention of a config file in the readme. Am I missing something?

    support 
    opened by tonydero 2
  • Suggestion: Print which file it is working on during the conversion process.

    Suggestion: Print which file it is working on during the conversion process.

    I think it would be nice to have some kind of progress indicator to show what its working on in the command prompt window. The easiest way to have this would simply just print the name of the file that is being worked on when it gets to it. When working on folders with lots of images, its nice to know its not frozen or hanging on something.

    Thank you!

    opened by coboba123 2
  • getting root - ERROR - [path] is not in the configured foundry Data folder.

    getting root - ERROR - [path] is not in the configured foundry Data folder.

    So I am running on a Google Cloud server, and have manually installed fvttoptimizer with Python 3.8.

    I created the /etc/fvttoptimizer.conf with the following line:

    {"absolute_path_to_foundry_data":"/home/tehjoester/foundrydata/Data"}

    and here is my output with the --verbose-debug option:

    [email protected]:~$ sudo fvttoptimizer --verbose-debug --skip-webp --recursive 'foundrydata/Data/'
    root - DEBUG - Got arguments ['fvttoptimizer', '--verbose-debug', '--skip-webp', '--recursive', 'foundrydata/Data/']
    root - DEBUG - Reading config from: /etc/fvttoptimizer.conf
    root - DEBUG - Running with target_path='foundrydata/Data/', config='{'_RunConfig__program_config': {'_ProgramConfigImpl__abs_path_to_foundry_data': '/home/tehjoester/foundrydata/Data'}, 'skip_existing': False, 'skip_webp': True, 'recursive': True, 'quality': 75, 'override_percent': 25}'
    root - ERROR - /home/tehjoester/foundrydata/Data is not in the configured foundry Data folder.
    /home/tehjoester/foundrydata/Data is not in the configured foundry Data folder.
    
    opened by thejoester 1
  • Argument --skip-webp does not seem to work

    Argument --skip-webp does not seem to work

    It's entirely possible that I am doing something wrong, but when I try to use --skip-webp it still tries to process webp files. Running the optimizer twice in a row shows that it will attempt the same files again even if they've already been converted.

    I am running Windows 11. This is the command I tried to run in Windows Terminal/PowerShell. Variations on this command have not resolved the issue.

    fvttoptimizer.exe --verbose-info --skip-webp --quality 100 --override-percent 10 --recursive 'C:\Games\Foundry Virtual Tabletop Data\Data\worlds\mitera'

    opened by seanms1991 0
  • Some actors are not getting the directory to their new avatar image updated properly

    Some actors are not getting the directory to their new avatar image updated properly

    While most actors have their pictures updated as expected, some are skipped. From what I can tell, it's mostly the avatars of the actors and not the tokens. The images are updated but the reference is not updated in the actors.db. Attached are some examples of actors that have this issue. Their tokens were created with Tokenizer, but those converted successfully and their main picture did not, which was not created with tokenizer. I've uploaded the pictures that were converted, the actor data exported as json files, and the actors.db files they were in both before and after the optimizer processed it.

    I am running Windows 11 and have tried to run this on two separate computers with the same result.

    If you need more information, let me know.

    Example.zip

    opened by seanms1991 0
  • Feature Request

    Feature Request

    First off, this script is amazing. It's greatly reduced the sizes of my images directories.

    It would be really cool though if I could maybe put in the config file the path to personal modules that it will update the file names. I have several shared compendiums in my own module that I move between worlds.

    enhancement 
    opened by Jonwh25 1
  • Program fails when a db file contains a byte UTF-8 can't decode.

    Program fails when a db file contains a byte UTF-8 can't decode.

    I got the following message in powershell:

    `PS C:\FVTTOptimizer> .\fvttoptimizer.exe --verbose-info --skip-webp --recursive "C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\Worlds\bakja" root - INFO - Optimizing directory 'C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja' root - INFO - Optimizing directory 'C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja\Assets' root - INFO - Optimizing directory 'C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja\Assets\Map' root - INFO - Optimizing C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja\Assets\Map\Animal Market Stall.png. Decreased file size by 83% root - ERROR - Traceback (most recent call last): File "cli_wrapper\main.py", line 235, in main File "cli_wrapper\main.py", line 229, in do_run File "cli_wrapper\main.py", line 81, in perform_optimization_with File "fvttoptimizer\optimizer.py", line 42, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 35, in optimize File "fvttoptimizer__file_optimizer.py", line 34, in maybe_optimize File "fvttoptimizer__file_optimizer.py", line 48, in __maybe_optimize2 File "fvttoptimizer__file_optimizer.py", line 68, in __optimize File "fvttoptimizer__file_optimizer.py", line 100, in __replace_file_with_webp File "fvttmv\update\references_updater.py", line 22, in replace_references File "fvttmv\update\references_updater.py", line 34, in _do_replace_references File "fvttmv\update__references_updater_file.py", line 21, in replace_references_in_file File "codecs.py", line 322, in decode UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte

    An internal error occurred: Traceback (most recent call last): File "cli_wrapper\main.py", line 235, in main File "cli_wrapper\main.py", line 229, in do_run File "cli_wrapper\main.py", line 81, in perform_optimization_with File "fvttoptimizer\optimizer.py", line 42, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 35, in optimize File "fvttoptimizer__file_optimizer.py", line 34, in maybe_optimize File "fvttoptimizer__file_optimizer.py", line 48, in __maybe_optimize2 File "fvttoptimizer__file_optimizer.py", line 68, in __optimize File "fvttoptimizer__file_optimizer.py", line 100, in __replace_file_with_webp File "fvttmv\update\references_updater.py", line 22, in replace_references File "fvttmv\update\references_updater.py", line 34, in _do_replace_references File "fvttmv\update__references_updater_file.py", line 21, in replace_references_in_file File "codecs.py", line 322, in decode UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte

    Please file a bug report on https://github.com/watermelonwolverine/fvttoptimizer/issues`

    bug 
    opened by YourDNDPleasesMe 10
Releases(0.1.3)
Owner
Watermelon Wolverine
Watermelon Wolverine
Codes to calculate solar-sensor zenith and azimuth angles directly from hyperspectral images collected by UAV. Works only for UAVs that have high resolution GNSS/IMU unit.

UAV Solar-Sensor Angle Calculation Table of Contents About The Project Built With Getting Started Prerequisites Installation Datasets Contributing Lic

Sourav Bhadra 1 Jan 15, 2022
NeurIPS-2021: Neural Auto-Curricula in Two-Player Zero-Sum Games.

NAC Official PyTorch implementation of NAC from the paper: Neural Auto-Curricula in Two-Player Zero-Sum Games. We release code for: Gradient based ora

Xidong Feng 19 Nov 11, 2022
RM Operation can equivalently convert ResNet to VGG, which is better for pruning; and can help RepVGG perform better when the depth is large.

RM Operation can equivalently convert ResNet to VGG, which is better for pruning; and can help RepVGG perform better when the depth is large.

184 Jan 04, 2023
Sentinel-1 vessel detection model used in the xView3 challenge

sar_vessel_detect Code for the AI2 Skylight team's submission in the xView3 competition (https://iuu.xview.us) for vessel detection in Sentinel-1 SAR

AI2 6 Sep 10, 2022
Implementation of ML models like Decision tree, Naive Bayes, Logistic Regression and many other

ML_Model_implementaion Implementation of ML models like Decision tree, Naive Bayes, Logistic Regression and many other dectree_model: Implementation o

Anshuman Dalai 3 Jan 24, 2022
Here is the implementation of our paper S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations.

S2VC Here is the implementation of our paper S2VC: A Framework for Any-to-Any Voice Conversion with Self-Supervised Pretrained Representations. In thi

81 Dec 15, 2022
本步态识别系统主要基于GaitSet模型进行实现

本步态识别系统主要基于GaitSet模型进行实现。在尝试部署本系统之前,建立理解GaitSet模型的网络结构、训练和推理方法。 系统的实现效果如视频所示: 演示视频 由于模型较大,部分模型文件存储在百度云盘。 链接提取码:33mb 具体部署过程 1.下载代码 2.安装requirements.txt

16 Oct 22, 2022
This package implements THOR: Transformer with Stochastic Experts.

THOR: Transformer with Stochastic Experts This PyTorch package implements Taming Sparsely Activated Transformer with Stochastic Experts. Installation

Microsoft 45 Nov 22, 2022
MMdet2-based reposity about lightweight detection model: Nanodet, PicoDet.

Lightweight-Detection-and-KD MMdet2-based reposity about lightweight detection model: Nanodet, PicoDet. This repo also includes detection knowledge di

Egqawkq 12 Jan 05, 2023
Codes for ACL-IJCNLP 2021 Paper "Zero-shot Fact Verification by Claim Generation"

Zero-shot-Fact-Verification-by-Claim-Generation This repository contains code and models for the paper: Zero-shot Fact Verification by Claim Generatio

Liangming Pan 47 Jan 01, 2023
Code release for "Detecting Twenty-thousand Classes using Image-level Supervision".

Detecting Twenty-thousand Classes using Image-level Supervision Detic: A Detector with image classes that can use image-level labels to easily train d

Meta Research 1.3k Jan 04, 2023
A curated list of awesome resources combining Transformers with Neural Architecture Search

A curated list of awesome resources combining Transformers with Neural Architecture Search

Yash Mehta 173 Jan 03, 2023
YOLOv5 detection interface - PyQt5 implementation

所有代码已上传,直接clone后,运行yolo_win.py即可开启界面。 2021/9/29:加入置信度选择 界面是在ultralytics的yolov5基础上建立的,界面使用pyqt5实现,内容较简单,娱乐而已。 功能: 模型选择 本地文件选择(视频图片均可) 开关摄像头

487 Dec 27, 2022
This repository is the code of the paper "Sparse Spatial Transformers for Few-Shot Learning".

🌟 Sparse Spatial Transformers for Few-Shot Learning This code implements the Sparse Spatial Transformers for Few-Shot Learning(SSFormers). Our code i

chx_nju 38 Dec 13, 2022
This is an official source code for implementation on Extensive Deep Temporal Point Process

Extensive Deep Temporal Point Process This is an official source code for implementation on Extensive Deep Temporal Point Process, which is composed o

Haitao Lin 8 Aug 15, 2022
MixRNet(Using mixup as regularization and tuning hyper-parameters for ResNets)

MixRNet(Using mixup as regularization and tuning hyper-parameters for ResNets) Using mixup data augmentation as reguliraztion and tuning the hyper par

Bhanu 2 Jan 16, 2022
Контрольная работа по математическим методам машинного обучения

ML-MathMethods-Test Контрольная работа по математическим методам машинного обучения. Вычисление основных статистик, диаграмм и графиков, проверка разл

Stas Ivanovskii 1 Jan 06, 2022
ktrain is a Python library that makes deep learning and AI more accessible and easier to apply

Overview | Tutorials | Examples | Installation | FAQ | How to Cite Welcome to ktrain News and Announcements 2020-11-08: ktrain v0.25.x is released and

Arun S. Maiya 1.1k Jan 02, 2023
这是一个yolox-pytorch的源码,可以用于训练自己的模型。

YOLOX:You Only Look Once目标检测模型在Pytorch当中的实现 目录 性能情况 Performance 实现的内容 Achievement 所需环境 Environment 小技巧的设置 TricksSet 文件下载 Download 训练步骤 How2train 预测步骤

Bubbliiiing 613 Jan 05, 2023
PyTorch Implementation of Spatially Consistent Representation Learning(SCRL)

Spatially Consistent Representation Learning (CVPR'21) Official PyTorch implementation of Spatially Consistent Representation Learning (SCRL). This re

Kakao Brain 102 Nov 03, 2022