A basic animation modding workflow for FFXIV

Overview

AnimAssist

Provides a quick and easy way to mod animations in FFXIV.

You will need:

  • Before anything, the VC++2012 32-bit Redist from here. Havok will not launch in any way without it. You need VSU_4\vcredist_x86.exe from the link.
  • The executables from this repository. Click releases, download the latest one, and extract the folder somewhere on your system.
  • One of:
    • Python. I wrote anim.py on 3.8.10.
    • BeautifulSoup. pip install bs4 at a terminal or command line.
  • OR:
    • anim.exe, packaged in the release.
  • 3DS Max. No other editors are supported with this method.
  • The HavokMax plugin by PredatorCZ: https://github.com/PredatorCZ/HavokMax
    • You can place this plugin in your 3DS Max plugins folder.
  • FFXIV Explorer: https://github.com/goaaats/ffxiv-explorer-fork
    • OR: Some other method of obtaining raw files from the game. They must be raw files, and not converted in any way.
  • Godbert: https://github.com/xivapi/SaintCoinach

Ok, how do I do this?

Warning: this is fairly technical. It's command line. It's not as easy as using something like Textools or the quick launcher.

Obtaining the files you need

FFXIV's files are only browsable with third party tools, like FFXIV Explorer above, which is what I use. However, it's not always easy to tell which file you need. This is where tools like Godbert come in handy, since there aren't really any easy to use tools for identifying animations or skeletons. I'm going to target an emote for now. I used the Fist Bump emote back when I wrote my C++ converter, and I've been using it again, so we'll use that here. I'll be using Godbert to view game sheets.

Popping open the emote sheet, we can see emote names, their text commands, and a list of "ActionTimeline" columns in the row. sheet

As you can see, not all ActionTimeline entries are filled out. The main one we care about is the first entry - emote/fistbump. While this is actually referring to the timeline, the animation for a timeline is almost always located at the same folder, but for the skeleton that the animation is for. A lot of animations in XIV are built for male Hyur (skeleton 0101) and other races just use that. Hard to tell without using a live path logger.

Anyways, let's just mess with male Hyur's fist bump. We know where it is now. All animation related things, barring cutscene-specific animations, are located in 040000. Pop open FFXIV Explorer, navigate to your game folder, and pop open 040000.win32.index. Let's grab the male Hyur skeleton first:

male hyur skele

FFXIV keeps human skeletons in chara/human/c{skele_id}/skeleton/base/b0001/skl_c{skele_id}b0001.sklb. So, we went there, and we have an sklb file. Extract this as "raw" using File > Extract Raw. Save it somewhere nice. Note that FFXIV Explorer will recreate the FFXIV folder paths. If you select C:\Users\User\Desktop\tmp, it will save the file in C:\Users\User\Desktop\tmp\chara\human\c0101\skeleton\base\b0001\skl_c0101b0001.sklb. That's just the way it is.

Anyways, since we're still on male Hyur, go to chara/human/c0101/animation/bt_common/emote and do the same thing as above, but with fistbump.pap. This is your animation file. Place them both somewhere save, as you'll need both of them later to put the animation back into the game.

Using AnimAssist

AnimAssist requires two files: anim.exe and animassist.exe. Or, if you're just running the Python script itself, anim.py and animassist.exe. I'll be using anim.exe rather than using the python script directly.

The command for turning the skeleton and animation files into something you can edit is:

anim.exe extract -s skeleton_file.sklb -p pap_file.pap -o output_file.hkx

where output_file.hkx will be a Havok file that you can open in 3DS Max using HavokMax.

This is what it looks like when I run this command on the files we just obtained: command

Note the skeleton warning. Skeletons mismatch don't produce an error, and will break literally everything ever.

Opening the file properly

Open 3DS Max and check the Plug-in Manager to ensure HavokMax is properly installed.

From the Max menu, click Import and select your output_file.hkx from the previous step to open it in 3DS Max. Select disable scale, invert top, and set Back: Top. Click import, and you'll get a wonderful fistbumping skeleton, read directly from what was once game files.

skeleton

Saving the file properly

You have some sweet changes to some animation, but now we need to put it back into the game.

From the Max menu, click Export and type any filename you want, and save it in a cool, dry place. In the "Save as type" drop-down, select "Havok Export (*.HKA, *.HKX, *.,HKT)". We will be saving an hkx file once more. Export your file, again, selecting disable scale, invert top, and set Back: Top. You must select those, as well as Export animation and include skeleton. Optimize tracks is optional.

Using AnimAssist (cont)

You have a file containing a skeleton and an animation. But it's Havok. We need AnimAssist to turn it into an animation file for the game.

That command is:

anim.exe pack -s original_skeleton_file.sklb -p original_pap_file.pap -a modified_animation.hkx -o new_ffxiv_animation.pap

where new_ffxiv_animation.pap is an animation file, ready to be imported into the game.

pack The output will look like this. Again, note the skeleton warning.

That's great. How do I put it into the game? What?

There are two methods.

FFXIV Textools

The love hate relationship we all have. Assuming you wanted to replace the animation you originally exported (in my case, chara/human/c0101/animation/a0001/bt_common/emote/fistbump.pap),

Steps:

  • Open Textools
  • Select Tools > Raw File Operations > Import Raw File
  • Enter the path to the game's file in the box. Note: You can right-click on files in FFXIV Explorer to copy their path.
  • Select the new pap file from the file selector button TexTools should import the file. If it doesn't, that kinda sucks.

Penumbra

I use Penumbra to develop things like that modify game files because it's far easier. Penumbra installation will not be covered here, but it is assumed you have a mod directory set up and other Penumbra mods work properly. Steps:

  • Create a new mod using the + button in the bottom left of the Installed Mods menu in Penumbra.
  • Name the mod.
  • Navigate to the mod's folder.
  • Recreate the directory structure up to the expected file. If you exported with FFXIV Explorer, you can copy that folder structure, but be careful you don't overwrite your new pap file.
  • Place your new pap file in the expected location by the game. For example, when I used AnimAssist to create my new pap file, I named it fistbump_new.pap, but when I copy it into the mod folder, I need to name it fistbump.pap because that's what the game is expecting.
  • Go to Settings, and click Rediscover Mods.
  • Go back to Installed Mods, and your replacement file should appear in the list.
  • Click Enable on the mod's checkbox.
  • You may have to swap races or jobs to get the game to reload an animation file live.

Congrats. You have an animation.

comparison

Known Issues

Pap files with multiple animations will ask which animation to extract, but I didn't test this. I am not going to be adding support for recreating a pap file with multiple animations in it.

This has been barely tested, and was developed over 3 days. I was still fixing stuff when I wrote this README.

Technical Details

I was originally writing these tools to be used with the Havok content tools for asset merging and conversion manually, as well as manual XML editing. But I was like, I could just automate that too.

Python handles:

  • Parsing XIV sklb (skeleton) and pap (animation) files
  • Parsing Havok XML packfiles for the skeleton remapping functionality
  • Calling the C++ executable

C++ handles:

  • Conversion between Havok proprietary filetypes

The overall flow goes as follows:

  • Skeleton and animation are obtained from the game in the form of the sklb and pap files.
    • Both file types have a header, and some optional info in the beginning before the Havok data.
    • This Havok data is directly readable by just about any Havok tool. They are Havok Binary Tagfiles, which means they aren't specific to any platform.
  • The following occurs when "extracting":
    • The sklb and pap are read, and their Havok data written to temporary files.
    • Those temporary files are combined in C++ by invoking animassist.exe.
    • A Havok Binary Packfile is written out with both the skeleton and animation present. HavokMax does not support binary or XML tagfiles, so this is how it had to be handled.
  • The following occurs when "packing":
    • The original sklb and pap are read, with the sklb Havok data being written to a temporary file.
    • The temporary sklb is converted in C++ to a Havok XML Packfile. (The output of HavokMax is the same type)
    • Python parses both XMLs with beautifulsoup to determine how to remap the bone indices, since HavokMax rewrites the transform track order differently from how the skeleton actually is, and we cannot include a skeleton as XIV already has it loaded.
    • With the transform tracks remapped properly, that text is written out, and converted to a Havok Binary Tagfile using C++ again, but this time, it removes the skeleton from the file. The game is expecting a Binary Tagfile, so I just didn't want to rock the boat too much. Other formats might work.
    • Python then reads the data preceding the Havok data and the data after the Havok data in the original pap, fixing the offset to the data after the Havok data based on the size of the new Havok data.
    • Python writes pre-Havok data, new Havok data, and post-Havok data to a new pap file. Done.

Building and Contributions

Note

I feel bad that this workflow was thought up, designed, and then kept a secret. That's not what the XIV developer community is about. While there are things people don't share for certain reasons - be it legal, be it automation, monetization should not be one of them. That's my belief. Note that this project's license is WTFPL.

Contributions

I would prefer forks rather than contributions to this repository. Please do whatever you want to make modding more fun and easy for others, but I ask you make it public.

This repository would not have been possible without PredatorCZ's immense work on HavokLib and HavokMax.

Building

Building animassist.exe requires the Havok 2014 SDK and an env var of HAVOK_SDK_ROOT set to the directory.

You can find the Havok SDK to compile with in the description of this video. Please note that is NOT a download I control, just a random one from online.

Comments
  • Anim not... working?

    Anim not... working?

    I'm trying to convert a skeleton and a animation file to .hkx but I got this error everytime I try to run the code. Am I missing something?

    With the file "anim.py" on the folder: Screenshot_4

    Without the file "anim.py" on the folder: Screenshot_5

    opened by jovemlex 3
  • Can not use anim to pack the hxk file

    Can not use anim to pack the hxk file

    QQ图片20220413075102 Hello I have some problems using it. when I use anim to pack the hxk it show me this message. I don't know much about python. Can you help me figure out how to solve it?

    opened by yangjimzero 0
  • Error with Edge of Shadow

    Error with Edge of Shadow

    Edge of Shadow and also Oka both cause 3ds max to crash and fail on import. Any change on how to fix this?

    If anyone has the XML animation file for edge of shadow I'd greatly appreciate it.

    opened by XCZA 0
  • Animations not working on male viera

    Animations not working on male viera

    Male viera potentially different from other skeletons. Animations that work on other races have issues on Male Viera where the skeleton goes all over the place.

    Any ideas on a fix? Male Viera uses c1701 however file explorer doesn't seem to have that.

    opened by TetsunoWondel 0
  • Animation not importing correctly

    Animation not importing correctly

    Using max 2020 and the latest v1.10 import plugin(AnimAssist). Q1: 01 Animation files: \chara\human\c0101\animation\a0001\bt_common\ability\2sw_dark\abl029.pap abl029.zip

    Q2: Honeycam 2022-03-22 10-25-10 Animation files: \chara\human\c0101\animation\a0001\bt_2sw_emp\resident\idle.pap idle.zip

    opened by xyunhe0729 0
Releases(1.0.1)
Owner
liam
I think C# is a good language
liam
用于红队成员初步快速攻击的全自动化工具。

关于 Author:m0sway Mail:[email protected] Github:https://www.github.com/m0sway/Jud JuD是

m0sway 46 Jul 21, 2022
This repository contains a lot of short scripting programs implemented both in Python (Flask) and TypeScript (NodeJS).

fast-scripts This repository contains a lot of short scripting programs implemented both in Python (Flask) and TypeScript (NodeJS). In python These wi

Nahum Maurice 3 Dec 10, 2022
A Linux program to create a Windows USB stick installer from a real Windows DVD or image.

WoeUSB-ng A Linux program to create a Windows USB stick installer from a real Windows DVD or image. This package contains two programs: woeusb: A comm

Longinus 1 Nov 19, 2021
Play tic-tac-toe in PowerPoint

The presentation has around 6,000 slides representing every possible game state (and some impossible ones, since I didn't check for wins or ties). You play by clicking on the squares, which are hyper

Jesse Li 3 Dec 18, 2021
Attempt at a Windows version of the plotman Chia Plot Manager system

windows plotman: an attempt to get plotman to work on windows THIS IS A BETA. Not ready for production use just yet. Almost, but not quite there yet.

59 May 11, 2022
0xFalcon - 0xFalcon Tool For Python

0xFalcone Installation Install 0xFalcone Tool: apt install git git clone https:/

Alharb7 6 Sep 24, 2022
A python package template that can be adapted for RAP projects

Warning - this repository is a snapshot of a repository internal to NHS Digital. This means that links to videos and some URLs may not work. Repositor

NHS Digital 3 Nov 08, 2022
"Hacking" the (Telekom) Zyxel GPON SFP module (PMG3000-D20B)

"Hacking" the (Telekom) Zyxel GPON SFP module (PMG3000-D20B) The SFP can be sour

Matthias Riegler 52 Jan 03, 2023
Mixtaper - Web app to make mixtapes

Mixtaper A web app which allows you to input songs in the form of youtube links

suryansh 1 Feb 14, 2022
Your missing PO formatter and linter

pofmt Your missing PO formatter and linter Features Wrap msgid and msgstr with a constant max width. Can act as a pre-commit hook. Display lint errors

Frost Ming 5 Mar 22, 2022
Set up a sidechain for the XRPL quickly and easily

Sidechain Launch Kit Introduction This directory contains python scripts to tests and explore side chains. This document walks through the steps to se

Xpring Engineering 15 Dec 08, 2022
This is a repository containing the backend and the frontend of a simple pokédex.

Pokémon This is a repository containing the backend and the frontend of a simple pokédex. This is a work in progress project! Project Structure 🗂 pok

André Rato 1 Nov 28, 2021
Batch generate asset browser previews

When dealing with hundreds of library files it becomes tedious to mark their contents as assets. Using python to automate the process is a perfect fit

54 Dec 24, 2022
Stop python warnings, no matter what!

SHUTUP - Stop python warnings, no matter what! Sometimes you just can't mute python warnings. Use this library to solve this. Installation pip install

80 Jan 04, 2023
Built with Python programming language and QT library and Guess the number in three easy, medium and hard rolls

password-generator Built with Python programming language and QT library and Guess the number in three easy, medium and hard rolls Password generator

Amir Hussein Sharifnezhad 3 Oct 09, 2021
pyshell is a Linux subprocess module

pyshell A Linux subprocess module, An easier way to interact with the Linux shell pyshell should be cross platform but has only been tested with linux

4 Mar 02, 2022
Xkcd.py - Script to generate wallpapers based on XKCD comics

xkcd.py Script to generate wallpapers based on XKCD comics Usage python3 xkcd.py

Gideon Wolfe 11 Sep 06, 2022
A visidata plugin for parsing f5 ltm/gtm/audit logs

F5 Log Visidata Plugin This plugin supports the default log format for: /var/log/ltm* /var/log/gtm* /var/log/apm* /var/log/audit* It extracts common l

James Deucker 1 Jan 06, 2022
Mata kuliah Bahasa Pemrograman

praktikum2 MENGHITUNG LUAS DAN KELILING LINGKARAN FLOWCHART : OUTPUT PROGRAM : PENJELASAN : Tetapkan nilai pada variabel sesuai inputan dari user :

2 Nov 09, 2021
A simple program to recolour simple png icon-like pictures with just one colour + transparent or white background. Resulting images all have transparent background and a new colour.

A simple program to recolour simple png icon-like pictures with just one colour + transparent or white background. Resulting images all have transparent background and a new colour.

Anna Tůmová 0 Jan 30, 2022