SKEL model instead of SMPL

[abhishekmonogram]

Hi, Thank you so much for sharing this repository. I wanted to check if a SKEL body model could be used in place of SMPL? Incase the answer is yes, what would I have to do to use SKEL here?

[saidwivedi]

I am not sure if I can answer that. Maybe Marilyn, lead author of SKEL, can answer.

[MarilynKeller]

Hi,
Token HMR was trained on images with ground truth SMPL fits, so it cannot directly output SKEL parameters.

However, you can obtain SKEL parameters by fitting SKEL to the SMPL mesh output from Token HMR using this script. The idea is to optimize the SKEL pose q to minimize the vertex to vertex distances ∣∣SKEL(β,q)skin − SMPL(β,θ)∣∣.

For direct regression, you could also retrain Token HMR on SKEL. To do this, you would need to fit SKEL to the entire training dataset SMPL meshes and then use the pose parameter of SKEL (q) instead of SMPL's θ in the architecture. But this would be a very consequent work.