[Raylib 6.0] Model Animation Discussion

[JeffM2501]

Point of Discussion

There has been some discussion about what we can do to make raylib's 3d animation system more robust. The current API and model structure are heavily focused around CPU based animation and there being one animation instance per model. With the addition of GPU animation support and advanced needs of raylib users, this API is proving to be weak.

Several people have different ideas on how to enhance the API, but in different ways, and this leads to confusion and large PR "code drops" that people don't understand.
This discussion is to talk about the needs of raylib and come up with a design before talk of any code.

Design of raylib

Raylib is designed first and foremost as a library to help people get into gamedev and learn how lower level systems are made. As has been said many times before, raylib is not an engine, and along that line, we should not entertain complex systems that are not easy for new users to pickup and user. Raylib APIs are meant to be simple and meant to be extended. We should focus on the lowest level APIs possible to do what is desired, and leave higher level systems up to external/drop in code (such as what is hosted on raylib-extras).

GPU vs CPU animation

Raylib has traditionally only supported CPU based animation, with GPU support just added in 5.5, because of this much of the API and structure is based around the model and mesh structure storing the current state of the transformed vertices. This is not always the best case for GPU based animation.

Splitting the current pose out from the model

Currently the animation data and animation pose cache is stored spread over several areas, some in the model, some in the animation, and some in the mesh.
I don't feel that storing any animation data in the mesh is correct, as there are no raylib APIs that use the data directly on the mesh, it's always used in the context of a model. Thus I propose that we remove the animation data from the mesh itself, DrawMesh doesn't need it, so why should it be there?

We should make a new structure called ModelAnimationPose, and it should contain a list of MeshAnimationPoses, in the same order as the meshes in a model. This structure would contain the animation related data that is currently in the mesh, most importantly the boneMatrices list. The boneMatrices is a cache of the current bone transforms for a pose. It is what the base vertices are transformed by for CPU animation, and are what is uploaded to the shader for GPU animation.

The model structure itself should hold one of these pose structures, giving the model the same storage for a 'current pose' that it has now, allowing all the existing animation functions to continue to work.

Any new APIs that change the pose on a model would take a ModelAnimationPose as input and modify it to the new post. Again, this lets all existing animation functions work as expected.

We should make low level functions that apply a type of animation to a pose, such as
SetModelAnimationPoseToFrame(Model*, ModelAnimation*, ModelAnimationPose*, int frame);
InterpolateModelAnimationPoseToFrame(Model*, ModelAnimation*, ModelAnimationPose*, int frame1, int frame2, float paramater);
These functions would simply update the pose passed in.

GPU based animations would then just upload the matrix cache for a mesh from the post as they draw the model.

For CPU animation we should have a function that applies a pose to the vertices of a model, by transforming the verts and uploading them to the GPU
ApplyPoseToModel(Model*, ModelAnimationPose*);

Pose Per Instance

Now that we have the animation system working on a self contained pose structure, we can add functions to extend this to add more features.

We can add a function to clone a pose.
ModelAnimationPose CopyAnimationPose(ModelAnimationPose*);
This will allow games that reuse the same model with different animation states to cache a pose per instance and update them as necessary. Right now if I have 10 copies of a mesh in different parts of the animation, I have to constantly rewrite the boneMatrices of the meshes before I draw anything. I would much rather maintain my own cache of poses and update them as needed (possibly in a thread), but the current system prevents that.

We can then also look into adding additional functions, internal, or external to update a pose in different ways such as doing animation blending, or programmatic modification of bones for features such as mouse control or inverse kinematics.

The separation of pose and model gives us a large range of possibilities for the future, with no real downside on the raylib side.

Scope

I think we should start with this low level of changes so that we can have a good foundation for storing and applying animation states as poses. This will then people expand as needed.
These low level functions will be used to reimplement the current high level animation functions so they work the same as they always have.

Then we can prototype/discuss further high level functions after this low level framework is in place.

[Kirandeep-Singh-Khehra]

IMO Using ModelAnimationPose as name might not be a good idea. Because across raylib the term "pose" is generally represented by Transform *. Like bindPose in Model is Transform * and framePoses in ModelAnimation is of Transform **(Array of poses). So, it might create two definitions of word Pose. I would suggest words similar to Rig(not good name in animation domain), Skeleton(way too broad) or Armature(good but not used a lot in game development but in sounds good in modeling) (<-- Please don't consider these as suggestions for functionality but name and a rough base for new Name).

1. Model.bindPose reference
   

   raylib/src/raylib.h

   Line 417 in 79facde

   Transform *bindPose; // Bones base transformation (pose)

2. ModelAnimation.framePoses reference

   raylib/src/raylib.h

   Line 425 in 79facde

   Transform **framePoses; // Poses array by frame

[Kirandeep-Singh-Khehra]

As per the requirements discussed i am trying to implement a drop in system here https://github.com/Kirandeep-Singh-Khehra/raylib-3d-anim-system. It is using layer based animation approach with Skeleton and Pose and needs no change in raylib code. Here are types defined:

typedef Transform* Pose;

typedef struct Skeleton {
  /* Extracted from Model */
  int boneCount;         // Number of bones
  BoneInfo *bones;       // Bones information (skeleton)
  Pose bindPose;   // Bones base transformation (pose)

  /* Extracted from Mesh */
  Matrix *boneMatrices;  // Bones animated transformation matrices (not used yet)

  /* Custom defined */
  Pose pose;      // Current pose
} Skeleton;

And here is a basic usage example.
https://github.com/Kirandeep-Singh-Khehra/raylib-3d-anim-system/blob/main/examples/skeleton/skeleton_animation_blending.c

Primary intent for this repo is to act as a playground or external base and then add some contributions (step by step) of basic commonly used and simple functions to raylib when it will be good enough to call it stable. Currently naming convention is poor and some redundant functions are present.

Please do give it a look(and little review if possible). And if anything in it looks good and can be added safely to raylib now or anytime in future then do let me know.