A basic animation modding workflow for FFXIV

Overview

AnimAssist

Provides a quick and easy way to mod animations in FFXIV.

You will need:

  • Before anything, the VC++2012 32-bit Redist from here. Havok will not launch in any way without it. You need VSU_4\vcredist_x86.exe from the link.
  • The executables from this repository. Click releases, download the latest one, and extract the folder somewhere on your system.
  • One of:
    • Python. I wrote anim.py on 3.8.10.
    • BeautifulSoup. pip install bs4 at a terminal or command line.
  • OR:
    • anim.exe, packaged in the release.
  • 3DS Max. No other editors are supported with this method.
  • The HavokMax plugin by PredatorCZ: https://github.com/PredatorCZ/HavokMax
    • You can place this plugin in your 3DS Max plugins folder.
  • FFXIV Explorer: https://github.com/goaaats/ffxiv-explorer-fork
    • OR: Some other method of obtaining raw files from the game. They must be raw files, and not converted in any way.
  • Godbert: https://github.com/xivapi/SaintCoinach

Ok, how do I do this?

Warning: this is fairly technical. It's command line. It's not as easy as using something like Textools or the quick launcher.

Obtaining the files you need

FFXIV's files are only browsable with third party tools, like FFXIV Explorer above, which is what I use. However, it's not always easy to tell which file you need. This is where tools like Godbert come in handy, since there aren't really any easy to use tools for identifying animations or skeletons. I'm going to target an emote for now. I used the Fist Bump emote back when I wrote my C++ converter, and I've been using it again, so we'll use that here. I'll be using Godbert to view game sheets.

Popping open the emote sheet, we can see emote names, their text commands, and a list of "ActionTimeline" columns in the row. sheet

As you can see, not all ActionTimeline entries are filled out. The main one we care about is the first entry - emote/fistbump. While this is actually referring to the timeline, the animation for a timeline is almost always located at the same folder, but for the skeleton that the animation is for. A lot of animations in XIV are built for male Hyur (skeleton 0101) and other races just use that. Hard to tell without using a live path logger.

Anyways, let's just mess with male Hyur's fist bump. We know where it is now. All animation related things, barring cutscene-specific animations, are located in 040000. Pop open FFXIV Explorer, navigate to your game folder, and pop open 040000.win32.index. Let's grab the male Hyur skeleton first:

male hyur skele

FFXIV keeps human skeletons in chara/human/c{skele_id}/skeleton/base/b0001/skl_c{skele_id}b0001.sklb. So, we went there, and we have an sklb file. Extract this as "raw" using File > Extract Raw. Save it somewhere nice. Note that FFXIV Explorer will recreate the FFXIV folder paths. If you select C:\Users\User\Desktop\tmp, it will save the file in C:\Users\User\Desktop\tmp\chara\human\c0101\skeleton\base\b0001\skl_c0101b0001.sklb. That's just the way it is.

Anyways, since we're still on male Hyur, go to chara/human/c0101/animation/bt_common/emote and do the same thing as above, but with fistbump.pap. This is your animation file. Place them both somewhere save, as you'll need both of them later to put the animation back into the game.

Using AnimAssist

AnimAssist requires two files: anim.exe and animassist.exe. Or, if you're just running the Python script itself, anim.py and animassist.exe. I'll be using anim.exe rather than using the python script directly.

The command for turning the skeleton and animation files into something you can edit is:

anim.exe extract -s skeleton_file.sklb -p pap_file.pap -o output_file.hkx

where output_file.hkx will be a Havok file that you can open in 3DS Max using HavokMax.

This is what it looks like when I run this command on the files we just obtained: command

Note the skeleton warning. Skeletons mismatch don't produce an error, and will break literally everything ever.

Opening the file properly

Open 3DS Max and check the Plug-in Manager to ensure HavokMax is properly installed.

From the Max menu, click Import and select your output_file.hkx from the previous step to open it in 3DS Max. Select disable scale, invert top, and set Back: Top. Click import, and you'll get a wonderful fistbumping skeleton, read directly from what was once game files.

skeleton

Saving the file properly

You have some sweet changes to some animation, but now we need to put it back into the game.

From the Max menu, click Export and type any filename you want, and save it in a cool, dry place. In the "Save as type" drop-down, select "Havok Export (*.HKA, *.HKX, *.,HKT)". We will be saving an hkx file once more. Export your file, again, selecting disable scale, invert top, and set Back: Top. You must select those, as well as Export animation and include skeleton. Optimize tracks is optional.

Using AnimAssist (cont)

You have a file containing a skeleton and an animation. But it's Havok. We need AnimAssist to turn it into an animation file for the game.

That command is:

anim.exe pack -s original_skeleton_file.sklb -p original_pap_file.pap -a modified_animation.hkx -o new_ffxiv_animation.pap

where new_ffxiv_animation.pap is an animation file, ready to be imported into the game.

pack The output will look like this. Again, note the skeleton warning.

That's great. How do I put it into the game? What?

There are two methods.

FFXIV Textools

The love hate relationship we all have. Assuming you wanted to replace the animation you originally exported (in my case, chara/human/c0101/animation/a0001/bt_common/emote/fistbump.pap),

Steps:

  • Open Textools
  • Select Tools > Raw File Operations > Import Raw File
  • Enter the path to the game's file in the box. Note: You can right-click on files in FFXIV Explorer to copy their path.
  • Select the new pap file from the file selector button TexTools should import the file. If it doesn't, that kinda sucks.

Penumbra

I use Penumbra to develop things like that modify game files because it's far easier. Penumbra installation will not be covered here, but it is assumed you have a mod directory set up and other Penumbra mods work properly. Steps:

  • Create a new mod using the + button in the bottom left of the Installed Mods menu in Penumbra.
  • Name the mod.
  • Navigate to the mod's folder.
  • Recreate the directory structure up to the expected file. If you exported with FFXIV Explorer, you can copy that folder structure, but be careful you don't overwrite your new pap file.
  • Place your new pap file in the expected location by the game. For example, when I used AnimAssist to create my new pap file, I named it fistbump_new.pap, but when I copy it into the mod folder, I need to name it fistbump.pap because that's what the game is expecting.
  • Go to Settings, and click Rediscover Mods.
  • Go back to Installed Mods, and your replacement file should appear in the list.
  • Click Enable on the mod's checkbox.
  • You may have to swap races or jobs to get the game to reload an animation file live.

Congrats. You have an animation.

comparison

Known Issues

Pap files with multiple animations will ask which animation to extract, but I didn't test this. I am not going to be adding support for recreating a pap file with multiple animations in it.

This has been barely tested, and was developed over 3 days. I was still fixing stuff when I wrote this README.

Technical Details

I was originally writing these tools to be used with the Havok content tools for asset merging and conversion manually, as well as manual XML editing. But I was like, I could just automate that too.

Python handles:

  • Parsing XIV sklb (skeleton) and pap (animation) files
  • Parsing Havok XML packfiles for the skeleton remapping functionality
  • Calling the C++ executable

C++ handles:

  • Conversion between Havok proprietary filetypes

The overall flow goes as follows:

  • Skeleton and animation are obtained from the game in the form of the sklb and pap files.
    • Both file types have a header, and some optional info in the beginning before the Havok data.
    • This Havok data is directly readable by just about any Havok tool. They are Havok Binary Tagfiles, which means they aren't specific to any platform.
  • The following occurs when "extracting":
    • The sklb and pap are read, and their Havok data written to temporary files.
    • Those temporary files are combined in C++ by invoking animassist.exe.
    • A Havok Binary Packfile is written out with both the skeleton and animation present. HavokMax does not support binary or XML tagfiles, so this is how it had to be handled.
  • The following occurs when "packing":
    • The original sklb and pap are read, with the sklb Havok data being written to a temporary file.
    • The temporary sklb is converted in C++ to a Havok XML Packfile. (The output of HavokMax is the same type)
    • Python parses both XMLs with beautifulsoup to determine how to remap the bone indices, since HavokMax rewrites the transform track order differently from how the skeleton actually is, and we cannot include a skeleton as XIV already has it loaded.
    • With the transform tracks remapped properly, that text is written out, and converted to a Havok Binary Tagfile using C++ again, but this time, it removes the skeleton from the file. The game is expecting a Binary Tagfile, so I just didn't want to rock the boat too much. Other formats might work.
    • Python then reads the data preceding the Havok data and the data after the Havok data in the original pap, fixing the offset to the data after the Havok data based on the size of the new Havok data.
    • Python writes pre-Havok data, new Havok data, and post-Havok data to a new pap file. Done.

Building and Contributions

Note

I feel bad that this workflow was thought up, designed, and then kept a secret. That's not what the XIV developer community is about. While there are things people don't share for certain reasons - be it legal, be it automation, monetization should not be one of them. That's my belief. Note that this project's license is WTFPL.

Contributions

I would prefer forks rather than contributions to this repository. Please do whatever you want to make modding more fun and easy for others, but I ask you make it public.

This repository would not have been possible without PredatorCZ's immense work on HavokLib and HavokMax.

Building

Building animassist.exe requires the Havok 2014 SDK and an env var of HAVOK_SDK_ROOT set to the directory.

You can find the Havok SDK to compile with in the description of this video. Please note that is NOT a download I control, just a random one from online.

Comments
  • Anim not... working?

    Anim not... working?

    I'm trying to convert a skeleton and a animation file to .hkx but I got this error everytime I try to run the code. Am I missing something?

    With the file "anim.py" on the folder: Screenshot_4

    Without the file "anim.py" on the folder: Screenshot_5

    opened by jovemlex 3
  • Can not use anim to pack the hxk file

    Can not use anim to pack the hxk file

    QQ图片20220413075102 Hello I have some problems using it. when I use anim to pack the hxk it show me this message. I don't know much about python. Can you help me figure out how to solve it?

    opened by yangjimzero 0
  • Error with Edge of Shadow

    Error with Edge of Shadow

    Edge of Shadow and also Oka both cause 3ds max to crash and fail on import. Any change on how to fix this?

    If anyone has the XML animation file for edge of shadow I'd greatly appreciate it.

    opened by XCZA 0
  • Animations not working on male viera

    Animations not working on male viera

    Male viera potentially different from other skeletons. Animations that work on other races have issues on Male Viera where the skeleton goes all over the place.

    Any ideas on a fix? Male Viera uses c1701 however file explorer doesn't seem to have that.

    opened by TetsunoWondel 0
  • Animation not importing correctly

    Animation not importing correctly

    Using max 2020 and the latest v1.10 import plugin(AnimAssist). Q1: 01 Animation files: \chara\human\c0101\animation\a0001\bt_common\ability\2sw_dark\abl029.pap abl029.zip

    Q2: Honeycam 2022-03-22 10-25-10 Animation files: \chara\human\c0101\animation\a0001\bt_2sw_emp\resident\idle.pap idle.zip

    opened by xyunhe0729 0
Releases(1.0.1)
Owner
liam
I think C# is a good language
liam
Async timeit - Async version of python's timeit

Async Timeit Replica of default python timeit module with small changes to allow

Raghava G Dhanya 3 Apr 13, 2022
The Python agent for Apache SkyWalking

SkyWalking Python Agent SkyWalking-Python: The Python Agent for Apache SkyWalking, which provides the native tracing abilities for Python project. Sky

The Apache Software Foundation 149 Dec 12, 2022
A professional version for LBS

呐 Yuki Pro~ 懒兵服御用版本,yuki小姐觉得没必要单独造一个仓库,但懒兵觉得有必要并强制执行 将na-yuki框架抽象为模块,功能拆分为独立脚本,使用脚本注释器使其作为py运行 文件结构: na_yuki_pro_example.py 是一个说明脚本,用来直观展示na,yuki! Pro

1 Dec 21, 2021
A type based dependency injection framework for Python 3.9+

Alluka A type based dependency injection framework for Python 3.9+. Installation You can install Alluka from PyPI using the following command in any P

Lucina 16 Dec 15, 2022
Really bad lisp implementation. Fun with pattern matching.

Lisp-py This is a horrible, ugly interpreter for a trivial lisp. Don't use it. It was written as an excuse to mess around with the new pattern matchin

Erik Derohanian 1 Nov 23, 2021
Simple Kahoot Botter.

Kahoot A simple Botter made in Python 3 for Kahoot.com. Also sorry for the shitty code lol. How to Run You need Python 3 installed on your device. Aft

7 Jun 29, 2022
💻 Algo-Phantoms-Backend is an Application that provides pathways and quizzes along with a code editor to help you towards your DSA journey.📰🔥 This repository contains the REST APIs of the application.✨

Algo-Phantom-Backend 💻 Algo-Phantoms-Backend is an Application that provides pathways and quizzes along with a code editor to help you towards your D

Algo Phantoms 44 Nov 15, 2022
A simple wrapper for joy library

Joy CodeGround A simple wrapper for joy library to render joy sketches in browser using vs code, (or in other words, for those who are allergic to Jup

rijfas 9 Sep 08, 2022
Utility to play with ADCS, allows to request tickets and collect information about related objects

certi Utility to play with ADCS, allows to request tickets and collect information about related objects. Basically, it's the impacket copy of Certify

Eloy 185 Dec 29, 2022
Slotscheck - Find mistakes in your slots definitions

🎰 Slotscheck Adding __slots__ to a class in Python is a great way to reduce mem

Arie Bovenberg 67 Dec 31, 2022
Purge all transformation orientations addon for Blender 2.8 and newer versions

CTO Purge This add-on adds a new button to Blender's Transformation Orientation panel which empowers the user to purge all of his/her custom transform

MMMrqs 10 Dec 29, 2022
WildHack 2021 solution by Nuclear Foxes team (public version).

WildHack 2021 Nuclear Foxes Team This repo contains our project for the Wildberries Hackathon 2021. Task 2: Searching tags Implement an algorithm of r

Sergey Zakharov 1 Apr 18, 2022
🐍 A Python lib for (de)serializing Python objects to/from JSON

Turn Python objects into dicts or (json)strings and back No changes required to your objects Easily customizable and extendable Works with dataclasses

Ramon Hagenaars 253 Dec 14, 2022
My solutions for the 2021's Advent of Code

Advent of Code 2021 My solutions for Advent of Code 2021. This year I am practicing Python 🐍 and also trying to develop my own language, Chocolate 🍫

Jakob Erzar 2 Dec 15, 2021
Feapder的管道扩展

FEAPDER 管道扩展 简介 此模块为feapder的pipelines扩展,感谢广大开发者对feapder的贡献 随着feapder支持的pipelines越来越多,为减少feapder的体积,特将pipelines提出,使用者可按需安装 管道 PostgreSQL 贡献者:沈瑞祥 联系方式:r

boris 9 Dec 07, 2022
A simple python script to convert Rubber Ducky payloads into AutoHotKey scripts

AHKDuckyReplacer A simple python script to convert Rubber Ducky payloads into AutoHotKey scripts. I have also added a sample payload for testing. I wi

Krizsan0596 5 Sep 28, 2022
「📖」Tool created to extract metadata from a domain

Metafind is an OSINT tool created with the aim of automating the search for metadata of a particular domain from the search engine known as Google.

9 Dec 28, 2022
Synthetik Python Mod - A save editor tool for the game Synthetik written in python

Synthetik_Python_Mod A save editor tool for the game Synthetik written in python

2 Sep 10, 2022
Archive, organize, and watch for changes to publicly available information.

0. Overview The Trapper Keeper is a collection of scripts that support archiving information from around the web to make it easier to study and use. I

Bill Fitzgerald 9 Oct 26, 2022
Tindicators is a Python library to calculate the values of various technical indicators

Tindicators is a Python library to calculate the values of various technical indicators

omar 3 Mar 03, 2022