Optimizes image files by converting them to webp while also updating all references.

Overview

About

Optimizes images by (re-)saving them as webp. For every file it replaced it automatically updates all references. Works on single files as well as on folders.

Installation: Windows

Step 1: Get the exe

Option 1: Download

Download one of the pre-built fvttoptimizer.exe files from Releases

Go to Step 2: Install program

Option 2: Build it yourself

Install python3 (version >= 3.8) and add it to your PATH system environment variable.

You can check the version via CMD or powershell with:

python --version

Download or clone the repo.

Open a CMD instance inside the project root folder and create a virtual environment.

python -m venv .\venv

Activate the venv:

.\venv\Scripts\activate

Install the Pillow package:

pip install pillow==9.1.0

Install pyInstaller package:

pip install pyInstaller==4.10

Download the fvttmv python package (whl file) from here: https://github.com/watermelonwolverine/fvttmv/releases/tag/0.2.4 move it into the folder and install it:

pip install fvttmv-0.2.4-py3-none-any.whl

Run build file:

.\scripts\build_for_windows.cmd

You should now have a fvttoptimizer.exe file under dist. After the build succeeded you can delete the venv you previously created.

Go to Step 2: Install program

Step 2: Install program

Create an empty folder where you want to install the program for example C:\fvttoptimizer

Copy fvttoptimizer.exe into that folder.

Create a fvttoptimizer.conf text file in that folder.

Copy {"absolute_path_to_foundry_data":"INSERT_PATH_HERE"} into fvttoptimizer.conf

Replace INSERT_PATH_HERE with the path to the Data folder inside your foundrydata (Not the foundrydata folder itself!).

IMPORTANT: Escape all \ with \\ in that path.

It should look something like this:

{"absolute_path_to_foundry_data":"C:\\Users\\user\\foundrydata\\Data"}

Add the installation path to your PATH system environment variable.

Uninstallation: Windows

Delete fvttoptimizer.exe and fvttoptimizer.conf files from the installation directory.

Remove the path to the installation directory from the PATH system environment variable.

Installation: Ubuntu 16.04 -20.04

Optional: Install Python

An up-to-date Ubuntu 20.04 should have python >= 3.8. Check it with:

python3 --version

From here on pythonX will be used as placeholder for the python you should use. Depending on your system you need to replace X with 3, 3.8, 3.9 or 3.10 .

Install python version>=3.8 if not yet installed.

sudo apt install pythonX

Step 1: Get the executable

Option 1: Download

Download one of the pre-built fvttoptimizer files from Releases

Go to Step 2: Install the files

Option 2: Build it yourself

Install python if haven't already.

Install Pillow dependencies (read more here https://pillow.readthedocs.io/en/stable/installation.html):

sudo apt install pythonX-dev libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk libharfbuzz-dev libfribidi-dev libxcb1-dev

Download or clone repo.

Create venv inside the repo:

sudo apt install pythonX-venv

pythonX -m venv ./venv

Activate venv:

source venv/bin/activate

Install Pillow (should succeed if you installed all the dependencies):

pythonX -m pip install Pillow==9.1.0

Install pyInstaller:

pythonX -m pip install pyInstaller==4.10

Download the fvttmv python package (whl file) from here: https://github.com/watermelonwolverine/fvttmv/releases/tag/0.2.4 move it into the folder and install it:

pythonX -m pip install fvttmv-0.2.4-py3-none-any.whl

Now run the build script:

./scrips/build_for_ubuntu.sh

Step 2: Install the files

Option 1: Automatic installation

Install python if you haven't already.

Clone the repo if you haven't already.

Go into the project folder.

Build the project if you haven't already. If you downloaded a pre-built executable. Create a folder named dist and move the file there.

Inside the project folder run:

sudo pythonX scripts/install_on_ubuntu.sh

Option 2: Manual installation

Copy the fvttoptimizer file either from dist from where you downloaded it to usr/bin/fvttoptimizer

Make the file executable:

sudo chmod ugo=rx usr/bin/fvttoptimizer

Create a fvttoptimizer.conf file at /etc/

Copy {"absolute_path_to_foundry_data":"INSERT_PATH_HERE"} into fvttoptimizer.conf

Replace INSERT_PATH_HERE with the path to the Data folder inside your foundrydata (Not the foundrydata folder itself!).

It should look something like this:

{"absolute_path_to_foundry_data":"/home/user/foundrydata/Data"}

Uninstallation: Ubuntu

Delete the files /etc/fvttoptimizer.conf and /usr/bin/fvttoptimizer

Usage

IMPORTANT:

Shut down Foundry VTT before optimizing any files.

Use this program at your own risk. It has been tested extensively but there are no guarantees. Always keep backup of your Foundry VTT data for cases where something goes wrong.

Syntax

fvttoptimizer [--verbose-info, --verbose-debug, --version, --help, --quality value, --override-percent value, --skip-webp, --skip-existing, --recursive] target

target: Path to file or directory which should be optimized

Options

--verbose-info: Enables verbose output to console

--verbose-debug: Enables very verbose output to console

--version: Prints version and exits

--help: Display help and exit

--skip-existing: Ignores files of which a webp already exists. For example image.png will not be converted if a image.webp is already in the same folder

--skip-webp: Don't touch webp files at all

--quality: The quality setting for the webp compression. Default is 75

--override-percent: How much smaller the new file needs to be to replace the old one. For example if this value is 25 the file size after optimizing needs to be 25% smaller than the original. Default is 25

--recursive: If the optimization should be done recursively to all sub folders of the target folder.

Examples

Optimizing a file:

Ubuntu: fvttoptimizer path/to/file.png

Windows: fvttoptimizer path\to\file.png

Optimizing a folder and all it's sub folders:

Ubuntu: fvttoptimizer --recursive pathto/folder

Windows: fvttoptimizer.exe --recursive path\to\folder

Use single quotes when moving a file with a space:

Ubuntu: fvttoptimizer 'some folder/some file.jpg'

Windows: fvttoptimizer.exe 'some folder\some file.jpg'

Change parameters:

Ubuntu: fvttoptimizer --quality 50 --override-percent 50 file.jpg

Windows: fvttoptimizer --quality 50 --override-percent 50 file.jpg

Known Issues and Quirks

Windows

The program only works in powershell not in cmd.

When one of the paths has \' at the end, the arguments will get mixed up. This is a problem with how python handles arguments and probably can't be fixed. For example on Windows fvttoptimizer.exe '\folder name with spaces\' will fail but fvttoptimizer.exe '\folder name with spaces' will succeed.

Comments
  • Program fails on special character [ in path

    Program fails on special character [ in path

    First off, thank you so much for making this.

    I am trying to use the following command: fvttoptimizer.exe --recursive 'C:\Users\cnguy\AppData\Local\FoundryVTT\Data\Battle Maps'

    and I am getting this return: An unexpected error occurred: expected str, bytes or os.PathLike object, not NoneType

    Your documentation implies this is supported, as this command works without the "--recursive" flag, so maybe there is an issue with spaces in paths and the recursive function?

    Thank you.

    opened by coboba123 5
  • Trying to create a Docker image - Keep getting 'image.jpg is not in the configured foundry Data folder.'

    Trying to create a Docker image - Keep getting 'image.jpg is not in the configured foundry Data folder.'

    As the title say, I'm trying to get this tool to work in a Docker container, but I'm having an issue that I can't figure out...

    I'm building the image like so:

    FROM python:3.10.0
    
    ARG FVTT_OPTIMIZER_VERSION=0.1.3
    
    RUN curl -L https://github.com/watermelonwolverine/fvttoptimizer/releases/download/$FVTT_OPTIMIZER_VERSION/fvttoptimizer --output usr/bin/fvttoptimizer
    RUN chmod +x usr/bin/fvttoptimizer
    RUN echo '{"absolute_path_to_foundry_data":"/app/Data"}' > /etc/fvttoptimizer.conf
    
    CMD ["fvttoptimizer"]
    

    Which I then configure with docker-compose:

    version: "3.7"
    
    services:
        fvttoptimizer:
            build: python
            volumes:
                - ${FOUNDRY_DATA_VOLUME}:/app
    
    # FOUNDRY_DATA_VOLUME is the directory that contains `Config`, `Data` and `Logs`
    

    However, when ran (docker-compose run --rm fvttoptimizer fvttoptimizer --verbose-debug ./Barney.jpg) I get the error in the title. Running docker-compose run --rm fvttoptimizer ls -la /app/Data, I can confirm the image exists.

    The debug output looks like this:

    root - DEBUG - Got arguments ['fvttoptimizer', '--verbose-debug', '--recursive', './Barney.jpg']
    root - DEBUG - Reading config from: /etc/fvttoptimizer.conf
    root - DEBUG - Running with target_path='./Barney.jpg', config='{'_RunConfig__program_config': {'_ProgramConfigImpl__abs_path_to_foundry_data': '/app/Data'}, 'skip_existing': False, 'skip_webp': False, 'recursive': True, 'quality': 75, 'override_percent': 25}'
    root - ERROR - /Barney.jpg is not in the configured foundry Data folder.
    /Barney.jpg is not in the configured foundry Data folder.
    

    I pushed a branch on my repo where it should be possible to reproduce the issue, or at the very least look at my configuration: https://github.com/gCardinal/foundrty-vtt/blob/feature/use-fvttoptimizer

    support 
    opened by gCardinal 3
  • Missing config file

    Missing config file

    When running on an image file I get the error:

    Missing config file. Could not find /etc/fvttoptimizer.conf

    Tried running all three release versions on Arch Linux with Python 3.10.4 and get the same error every time.

    I'm not seeing any mention of a config file in the readme. Am I missing something?

    support 
    opened by tonydero 2
  • Suggestion: Print which file it is working on during the conversion process.

    Suggestion: Print which file it is working on during the conversion process.

    I think it would be nice to have some kind of progress indicator to show what its working on in the command prompt window. The easiest way to have this would simply just print the name of the file that is being worked on when it gets to it. When working on folders with lots of images, its nice to know its not frozen or hanging on something.

    Thank you!

    opened by coboba123 2
  • getting root - ERROR - [path] is not in the configured foundry Data folder.

    getting root - ERROR - [path] is not in the configured foundry Data folder.

    So I am running on a Google Cloud server, and have manually installed fvttoptimizer with Python 3.8.

    I created the /etc/fvttoptimizer.conf with the following line:

    {"absolute_path_to_foundry_data":"/home/tehjoester/foundrydata/Data"}

    and here is my output with the --verbose-debug option:

    [email protected]:~$ sudo fvttoptimizer --verbose-debug --skip-webp --recursive 'foundrydata/Data/'
    root - DEBUG - Got arguments ['fvttoptimizer', '--verbose-debug', '--skip-webp', '--recursive', 'foundrydata/Data/']
    root - DEBUG - Reading config from: /etc/fvttoptimizer.conf
    root - DEBUG - Running with target_path='foundrydata/Data/', config='{'_RunConfig__program_config': {'_ProgramConfigImpl__abs_path_to_foundry_data': '/home/tehjoester/foundrydata/Data'}, 'skip_existing': False, 'skip_webp': True, 'recursive': True, 'quality': 75, 'override_percent': 25}'
    root - ERROR - /home/tehjoester/foundrydata/Data is not in the configured foundry Data folder.
    /home/tehjoester/foundrydata/Data is not in the configured foundry Data folder.
    
    opened by thejoester 1
  • Argument --skip-webp does not seem to work

    Argument --skip-webp does not seem to work

    It's entirely possible that I am doing something wrong, but when I try to use --skip-webp it still tries to process webp files. Running the optimizer twice in a row shows that it will attempt the same files again even if they've already been converted.

    I am running Windows 11. This is the command I tried to run in Windows Terminal/PowerShell. Variations on this command have not resolved the issue.

    fvttoptimizer.exe --verbose-info --skip-webp --quality 100 --override-percent 10 --recursive 'C:\Games\Foundry Virtual Tabletop Data\Data\worlds\mitera'

    opened by seanms1991 0
  • Some actors are not getting the directory to their new avatar image updated properly

    Some actors are not getting the directory to their new avatar image updated properly

    While most actors have their pictures updated as expected, some are skipped. From what I can tell, it's mostly the avatars of the actors and not the tokens. The images are updated but the reference is not updated in the actors.db. Attached are some examples of actors that have this issue. Their tokens were created with Tokenizer, but those converted successfully and their main picture did not, which was not created with tokenizer. I've uploaded the pictures that were converted, the actor data exported as json files, and the actors.db files they were in both before and after the optimizer processed it.

    I am running Windows 11 and have tried to run this on two separate computers with the same result.

    If you need more information, let me know.

    Example.zip

    opened by seanms1991 0
  • Feature Request

    Feature Request

    First off, this script is amazing. It's greatly reduced the sizes of my images directories.

    It would be really cool though if I could maybe put in the config file the path to personal modules that it will update the file names. I have several shared compendiums in my own module that I move between worlds.

    enhancement 
    opened by Jonwh25 1
  • Program fails when a db file contains a byte UTF-8 can't decode.

    Program fails when a db file contains a byte UTF-8 can't decode.

    I got the following message in powershell:

    `PS C:\FVTTOptimizer> .\fvttoptimizer.exe --verbose-info --skip-webp --recursive "C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\Worlds\bakja" root - INFO - Optimizing directory 'C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja' root - INFO - Optimizing directory 'C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja\Assets' root - INFO - Optimizing directory 'C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja\Assets\Map' root - INFO - Optimizing C:\Users\Scott-Server\AppData\Local\FoundryVTT\Data\worlds\bakja\Assets\Map\Animal Market Stall.png. Decreased file size by 83% root - ERROR - Traceback (most recent call last): File "cli_wrapper\main.py", line 235, in main File "cli_wrapper\main.py", line 229, in do_run File "cli_wrapper\main.py", line 81, in perform_optimization_with File "fvttoptimizer\optimizer.py", line 42, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 35, in optimize File "fvttoptimizer__file_optimizer.py", line 34, in maybe_optimize File "fvttoptimizer__file_optimizer.py", line 48, in __maybe_optimize2 File "fvttoptimizer__file_optimizer.py", line 68, in __optimize File "fvttoptimizer__file_optimizer.py", line 100, in __replace_file_with_webp File "fvttmv\update\references_updater.py", line 22, in replace_references File "fvttmv\update\references_updater.py", line 34, in _do_replace_references File "fvttmv\update__references_updater_file.py", line 21, in replace_references_in_file File "codecs.py", line 322, in decode UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte

    An internal error occurred: Traceback (most recent call last): File "cli_wrapper\main.py", line 235, in main File "cli_wrapper\main.py", line 229, in do_run File "cli_wrapper\main.py", line 81, in perform_optimization_with File "fvttoptimizer\optimizer.py", line 42, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 38, in optimize File "fvttoptimizer__directory_optimizer.py", line 35, in optimize File "fvttoptimizer__file_optimizer.py", line 34, in maybe_optimize File "fvttoptimizer__file_optimizer.py", line 48, in __maybe_optimize2 File "fvttoptimizer__file_optimizer.py", line 68, in __optimize File "fvttoptimizer__file_optimizer.py", line 100, in __replace_file_with_webp File "fvttmv\update\references_updater.py", line 22, in replace_references File "fvttmv\update\references_updater.py", line 34, in _do_replace_references File "fvttmv\update__references_updater_file.py", line 21, in replace_references_in_file File "codecs.py", line 322, in decode UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 0: invalid continuation byte

    Please file a bug report on https://github.com/watermelonwolverine/fvttoptimizer/issues`

    bug 
    opened by YourDNDPleasesMe 10
Releases(0.1.3)
Owner
Watermelon Wolverine
Watermelon Wolverine
Learning hierarchical attention for weakly-supervised chest X-ray abnormality localization and diagnosis

Hierarchical Attention Mining (HAM) for weakly-supervised abnormality localization This is the official PyTorch implementation for the HAM method. Pap

Xi Ouyang 22 Jan 02, 2023
Random-Afg - Afghanistan Random Old Idz Cloner Tools

AFGHANISTAN RANDOM OLD IDZ CLONER TOOLS Install $ apt update $ apt upgrade $ apt

MAHADI HASAN AFRIDI 5 Jan 26, 2022
Pytorch implementation of ProjectedGAN

ProjectedGAN-pytorch Pytorch implementation of ProjectedGAN (https://arxiv.org/abs/2111.01007) Note: this repository is still under developement. @InP

Dominic Rampas 17 Dec 14, 2022
FastyAPI is a Stack boilerplate optimised for heavy loads.

FastyAPI A FastAPI based Stack boilerplate for heavy loads. Explore the docs » View Demo · Report Bug · Request Feature Table of Contents About The Pr

Ali Chaayb 47 Dec 27, 2022
PyTorch implementation of paper: HPNet: Deep Primitive Segmentation Using Hybrid Representations.

HPNet This repository contains the PyTorch implementation of paper: HPNet: Deep Primitive Segmentation Using Hybrid Representations. Installation The

Siming Yan 42 Dec 07, 2022
A Research-oriented Federated Learning Library and Benchmark Platform for Graph Neural Networks. Accepted to ICLR'2021 - DPML and MLSys'21 - GNNSys workshops.

FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks A Research-oriented Federated Learning Library and Benchmark Platform

FedML-AI 175 Dec 01, 2022
Demo project for real time anomaly detection using kafka and python

kafkaml-anomaly-detection Project for real time anomaly detection using kafka and python It's assumed that zookeeper and kafka are running in the loca

Rodrigo Arenas 36 Dec 12, 2022
Point-NeRF: Point-based Neural Radiance Fields

Point-NeRF: Point-based Neural Radiance Fields Project Sites | Paper | Primary c

Qiangeng Xu 662 Jan 01, 2023
Keras Model Implementation Walkthrough

Keras Model Implementation Walkthrough

Luke Wood 17 Sep 27, 2022
Provide partial dates and retain the date precision through processing

Prefix date parser This is a helper class to parse dates with varied degrees of precision. For example, a data source might state a date as 2001, 2001

Friedrich Lindenberg 13 Dec 14, 2022
[CVPR 2022 Oral] MixFormer: End-to-End Tracking with Iterative Mixed Attention

MixFormer The official implementation of the CVPR 2022 paper MixFormer: End-to-End Tracking with Iterative Mixed Attention [Models and Raw results] (G

Multimedia Computing Group, Nanjing University 235 Jan 03, 2023
University of Rochester 2021 Summer REU focusing on music sentiment transfer using CycleGAN

Music-Sentiment-Transfer University of Rochester 2021 Summer REU focusing on music sentiment transfer using CycleGAN Poster: Music Sentiment Transfer

Miles Sigel 2 Jan 24, 2022
Sound Source Localization for AI Grand Challenge 2021

Sound-Source-Localization Sound Source Localization study for AI Grand Challenge 2021 (sponsored by NC Soft Vision Lab) Preparation 1. Place the data-

sanghoon 19 Mar 29, 2022
Paper: De-rendering Stylized Texts

Paper: De-rendering Stylized Texts Wataru Shimoda1, Daichi Haraguchi2, Seiichi Uchida2, Kota Yamaguchi1 1CyberAgent.Inc, 2 Kyushu University Accepted

CyberAgent AI Lab 55 Dec 18, 2022
Paddle implementation for "Highly Efficient Knowledge Graph Embedding Learning with Closed-Form Orthogonal Procrustes Analysis" (NAACL 2021)

ProcrustEs-KGE Paddle implementation for Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis 🙈 A more detailed re

Lincedo Lab 4 Jun 09, 2021
NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling @ INTERSPEECH 2021 Accepted

NU-Wave — Official PyTorch Implementation NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling Junhyeok Lee, Seungu Han @ MINDsLab Inc

MINDs Lab 242 Dec 23, 2022
Official implementation of "SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers"

SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Figure 1: Performance of SegFormer-B0 to SegFormer-B5. Project page

NVIDIA Research Projects 1.4k Dec 31, 2022
PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks

Code for the paper "PICK: Processing Key Information Extraction from Documents using Improved Graph Learning-Convolutional Networks" (ICPR 2020)

Wenwen Yu 498 Dec 24, 2022
Graph parsing approach to structured sentiment analysis.

Fine-grained Sentiment Analysis as Dependency Graph Parsing This repository contains the code and datasets described in following paper: Fine-grained

Jeremy Barnes 36 Dec 12, 2022
Single-Stage 6D Object Pose Estimation, CVPR 2020

Overview This repository contains the code for the paper Single-Stage 6D Object Pose Estimation. Yinlin Hu, Pascal Fua, Wei Wang and Mathieu Salzmann.

CVLAB @ EPFL 89 Dec 26, 2022