[rmodels] Added implementation of UpdateModelAnimationBonesWithBlending() function

[Kirandeep-Singh-Khehra]

This PR adds a function to blend between two different frames of different or same ModelAnimation. It can be used to blend between two animation or smoothly transition between two animation by interpolating between exiting and entering poses. Updates for bones only and need GPU skinning

Params

1. Model model: Target model to update bones.
2. ModelAnimation animA: First animation.
3. int frameA: Frame number of first animation.
4. ModelAnimation animB: Second animation.
5. int frameB: Frame number of second animation
6. float blendFactor: Ratio of blending between animA's frameA and animB's frameB. Value must be from 0.0f to 1.0f. Value 0.0f means use frameA of animA and 1.0f means use frameB of animB. And other values correspond to other values in between.

animA           animB
0.0f    <--->    1.0f

Example usage(not-complete code)

// Utilities
#define min(x, y) ((x < y) ? x : y)
#define abs(x) ((x < 0) ? -x : x)

// Main code
motion = Vector2Normalize((Vector2){IsKeyDown(KEY_W) - IsKeyDown(KEY_S),
                                    IsKeyDown(KEY_A) - IsKeyDown(KEY_D)}); // Direction of motion on XZ plane

// Choose animation index between running forward and backward
int indexX = 0;
if (motion.x) {
  indexX = (motion.x < 0) ? RunBack : Run;
}

// Choose animation index between running left and right
int indexY = 0;
if (motion.y) {
  indexY = (motion.y < 0) ? RunRight : RunLeft;
}

// Update animation frame counter
animFrameCounter++;
animFrameCounter %= min(anims[indexX].frameCount, anims[indexY].frameCount);

// Function usage
UpdateModelAnimationBonesWithBlending(model,
                                      anims[indexX], animFrameCounter,
                                      anims[indexY], animFrameCounter,
                                      abs(motion.y));

TODOs

• Add an example.
• Add blending without need of GPU skinning Dropped for now Adding it see this comment

I tested on a model downloaded from Mixamo and animated using Mixamo. Its in glb format. Not sure if it is acceptable as asset. If yes then i'll push that example. Otherwise i need to create a custom model.

Demo uses above example with 5 animations. Idle, Run, RunBack, RunLeft and RunRight. Idle is played by default and blending is done for diagonal motion.

raylib_blend_demo_0.mp4

[RicoP]

nice!

[Kirandeep-Singh-Khehra]

Added example to show blending using robot.glb model.

anim_blending_example-2024-12-07_22.58.26.mp4

[raysan5]

@Kirandeep-Singh-Khehra This is a great addition! I find the function name UpdateModelAnimationBonesWithBlending() quite long, maybe it can be shortened a bit? I'm thinking about UpdateModelAnimationPro(), that, despite not being so clear, it follows raylib naming conventions for advance versions of some functions (UpdateCameraPro(), DrawTexturePro(), DrawTextPro()...).

In the same line, UpdateModelAnimationBones() is the GPU alternative to UpdateModelAnimation(), maybe it can be simplified to just one function and enable GPU-skinning with a compile flag? Not sure how useful is having both methods available.

That way, UpdateModelAnimationPro() could implement blending internally in CPU/GPU using the same compile flag.

[JeffM2501]

While I agree that we need interpolation/blending support, I feel that this simple API does not go far enough.

1. I would like to see the ability to update the bones separate from the model. I may have many instances of a model at different animation states, I want to keep my own set of 'current' bones with my instance structure and not with the model. So to me an optimal function would take the output bone matrices as an argument.
2. I would like to be able to pick what I apply the interpolated frame to, the vertices or upload them to the GPU.

I don't mind having a basic function that just updates the model's internal bone state, but I would like the lower level functions to be accessible for people who need more than a simple single model use case.

[Kirandeep-Singh-Khehra]

@raysan5 @JeffM2501
Thanks guys for sharing your feedback and concerns,

When i started using raylib i was hoping for something like simple utility to work with skeleton and forward kinematics. And both these things were way complex. And i was unable to make a good looking 3d game without adding something to animations.

But as Ray asked to unify animation functions. I discarded/upgraded this code and tried to create a one stop solution and i implemented most of it. Just minor adjustments are needed.

I completely agree that the idea discussed below is lot complex(but code will be readable and simple to debug, i assure). But it will add things like blending, interpolation, split body animation. single call function for more complex animations(like add all animations in single call and user only need calculate weights(discussed below))
But if we consider it to be a function at low level. Then it can do a lot on animation side.(see the last code block in this comment)

The idea is to create function UpdateModelAnimationPro(). But for now lets consider only bones UpdateModelAnimationBonesPro(Model, BoneMask, int, int, .../*va_args*/). with paramers including:

1. Model model: Model to use.
2. BoneMask mask: Just a fancy name for bitmap. Which stores list of 0 or 1. Useful for model splitting. Like upper body split and lower body split.
3. int flag: Any config flags. Like ANIM_PRO_INVERT_MASK(to invert the mask before using it) and other flags etc etc. Intended use is like ANIM_PRO_INVERT_MASK | ANIM_PRO_FLAG2 | ANIM_PRO_FLAG3. Not yet planned what could be added.
4. int animCount: Number of animations range from 1 to N.
5. ...: va_arg list consisting of multiple ModelAnimation anim, int frame, double weight(va_list promotes float to double and may introduce some issues).

Example i used:

UpdateModelAnimationBonesPro(model, fullBodyMask, 0, 2,
                              /* Anim       |  Frame         |   Weight            */
                              anims[indexX], animFrameCounter, (double)abs(motion.x),
                              anims[indexY], animFrameCounter, (double)abs(motion.y)
                              /* More lines here */
                              );

BoneMask is implemented but not yet used in function.

I tested passing 1,2 and 3 animations and it passed with flying colors. (As far as i can see on screen).

We can also pass ModelAnimation array then int array for frames and double or float array for weights. This will make usage more complex but using va_list will make no-compile time warnings or linting or auto completion.

Then we can replace/redefine UpdateModelAnimationBones() and UpdateModelAnimationBonesWithBlending() with

#define UpdateModelAnimationBones(model, anim, frame) UpdateModelAnimationBonesPro(model, FullBodyMask(), 0, 1, anim, frame, 1)

#define UpdateModelAnimationBonesWithMask(model, animA, frameA, animB, frameB, blendFactor) UpdateModelAnimationBonesPro(model, FullBodyMask(), 0, 2, animA, frameA, (1-blendFactor), animB, frameB, blendFactor)

// This one will be interesting
void UpdateModelAnimationBonesWithTime(model, anim, time) {
  int frameA = time / GLTF_ANIMDELAY;
  int frameB = time / GLTF_ANIMDELAY;

  float blendFactor = (time % GLTF_ANIMDELAY) / GLTF_ANIMDELAY;
  UpdateModelAnimationBonesPro(model, FullBodyMask(), 0, 2, 
                               anim, frameA, (1 - blendFactor),
                               anim, frameB, blendFactor)
}

// ... or many more such animation stuff can be done with this one function.

If its not possible to have this then ... (Just let me know) .Or Do we have plans for official plugin system (separate repo for bunch of simple header files) to add such advanced functionality to raylib.

I am also planning to work on state machine after this for animation but State machine will be lot less useful when added to raylib. So, it must be separate repo. But that's the story of another day.

Looking forward for some feedback.

[JeffM2501]

That seems very complicated.
I'm not a fan of the bitmask thing, as I don't think people will get it.
I am also no a fan of combining interpolation and blending in the same base API.

This makes me wonder if a full animation system should be made external to raylib as a drop in.

I would take blending out of this feature to start with. While I get that it's similar, I think it complicates something that should be a simple API (prevent feature creep).

My thoughts are that we should add the following new API functions to start with.

void UpdateModelAnimationBonesLerp(Model model, ModelAnimation anim, int frame1, int frame2, float param);  // computes a new set of bone matricies between two frames.

void UpdateModelVertsToCurrentBones(Model model); // takes the current set of bone matrices and applies to them to mesh verts (CPU Animation)

These would give us basic interpolation using a simple API that works for both GPU and CPU users without code duplication.

Someone doing GPU animation would just call UpdateModelAnimationBonesLerp and then upload the bones to the shader.
Someone doing CPU animation would need to also call UpdateModelVertsToCurrentBones to apply the bones to the verts.

I think it's important for people doing CPU animation to know that they are modifying vertex data.

I don't think there is any benefit to locking in a 'pro' version of any function right now. I fear we may 'burn' the pro name on something that's not useful just to 'get something in'. We do not need to rush this.

Blending is a big subject with a lot of requirements. I think to start for blending we should open a discussion about what those requirement and use cases are before we design any APIs. I think that for blending we are really going to need a way to define a model skeleton outside of the model structure and update it separately, then apply it back to the meshes, and that's a bigger design, something I feel that is outside the scope of a simple single PR. My fear is that a purely internal solution will be lacking in some regards, so an external drop in solution may be best (with API changes to support the access it needs). But we for sure need to design the API properly first, not just throw code around.

[Kirandeep-Singh-Khehra]

Updated the PR with functions suggested by JeffM2501. Example is also updated. To have both CPU and GPU skinning along with comments that user viewer can follow to make it use GPU skinning.

Modified UpdateModelAnimation() (It had the code to update verts) to use UpdateModelVertsToCurrentBones().

Tested GPU and CPU version and other examples using older UpdateModelAnimation function.

void UpdateModelAnimationBonesLerp(Model model, ModelAnimation anim1, int frame1, ModelAnimation anim2, int frame2, float param);  // computes a new set of bone matricies between two frames.

void UpdateModelVertsToCurrentBones(Model model); // takes the current set of bone matrices and applies to them to mesh verts (CPU Animation)