Have you tried to use manopytorch?

[mks0601]

Hi,

I'm trying to use your dataset, but I met some problems.
I visualized the mesh using manopth using provided groundtruth mano parameters, but got very strange result. You can see the attached images with visualized mesh. I manually rotated the mesh in meshlab, so global rotation might be wrong. Although ignoring the global rotation, overall hand pose is absolutely wrong.

Have you faced this problem? Have you used the manopth to inspect your mano parameters are correct ones?
What I did is just split the 61-dimensional mano parameter (from training_mano.json) to [:48] as the pose parameter and [48:58] and the shape parameter. The splitted pose and shape parameters are feed to the manopth layer. I checked the training_xyz.json gives good joint coordinates.

[zimmerm]

Hi,

I didn't use manopth for visualization, but I imported the related Obman dataset to be visualized by my script at some point, which seems like the inverse problem to me.

There are two things worth checking:

1. Make sure you are using the right MANO version of the dataset. See here. If you downloaded the dataset some time go (before Oct 2019) then it could be that you have the MANO v1 version of FreiHAND, which you are trying to visualize using MANO v2. In that case download the updated version of the dataset.

2. Another problem could be the hand pose mean. When I imported Obman to be aligned with FreiHAND I used the following conversion function:

import pickle
import numpy as np
g_pca_mean = None
g_pca_comps = None
def _convert_pca_spaces(pca_obman):
    """
        Convert Obman poses to FreiHAND poses. 
        Obman uses poses in pca space, that are not relative to the mean.
    """
    global g_pca_comps, g_pca_mean
    if g_pca_comps is None:
        with open('./data/MANO_RIGHT.pkl', 'r') as fi:
            data = pickle.load(fi)
        g_pca_comps = np.array(data['hands_components'], dtype=np.float32)
        g_pca_mean = np.array(data['hands_mean'], dtype=np.float32)

    pose = np.zeros_like(g_pca_mean)

    for i, a in enumerate(pca_obman):
        pose = pose + a * g_pca_comps[i, :]
    pose -= g_pca_mean
    return pose

What I call pca_obman here is what is called hand_pca in Obman (compare).

So what you probably would need to do is the inversion of this. I'd try subtracting the mean from the provided FreiHAND parameters and feed that to manopth. Let me know if that works.

Cheers Chris

[mks0601]

1. I downloaded the dataset 2019/12/31.
2. The function seems convert pose in PCA space to axis-angle representation. But I turned off the use_pca feature of manopth like your codes. So I think I do not have to do inverse thing? Could you try to get hand mesh from manopth?

[TerenceCYJ]

Hi, @mks0601. That is also my question! It seems that the mano parameters here are hard to align with that in manopth. I think it might be that here the output hand joints and vertices are in Camera Coordinate System, by they are in Hand-centric Coordinate System in manopth?
I am not sure, and still trying to solve it :)

[mks0601]

I don't think so, because that kind of coordinate system difference can be solely handled by root joint rotation. What I'm saying is not global coordinate system difference, but local hand pose.

[adrianofragomeni]

I have the opposite problem, I want to align the FreiHAND dataset with the obman dataset. I tried to subtract the mean from the provided FreiHAND parameters, but it does not work. how can I do this?

[hassony2]

Hi @mks0601,

Looking at your mesh, it looks like it might be a problem with the flat_hand_mean parameter
https://github.com/hassony2/manopth/blob/c1873746b8659ea96d9537475996e5fcba1ff2db/manopth/manolayer.py#L21
Can you try to switch the flag ? (In your case, probably set flat_hand_mean=False)

Best !

Yana

[mks0601]

@hassony2

Works like charm! Thanks!

[hassony2]

@mks0601 good to hear that it worked !

Have a great day :)

[moranli19]

Is there any guidance to apply the mano parameters of the FreiHAND dataset to MANO layers (implemented by hassony2)?
After feeding mano parameters to the MANO layer, the joints generated by MANO layers (with flat_hand_mean=False, center_idx=9, side='right', use_pca=False) are different from the 3D joints annotation of the FreiHAND dataset (with subtracting root joints as well).

[liwenssss]

SAME question, it seems you set use_pac=False, what if I want use your hands parameters in the original SMPL-H full human body model ?

[liwenssss]

SAME question, it seems you set use_pac=False, what if I want use your hands parameters in the original SMPL-H full human body model ?

In the other words, have you tried to align your parameters to the original MANO model which have not set use_pac.I find when I set the use_pac=True when I use manopth, I got the same result when I use the original version under the same parameters.

[Ishan-Kumar2]

Hi @moranli19 did you get an answer for how to get the PyTorch MANO output joints to be same as the ground truth joints 3d?

[mks0601]

For me, ManoLayer(mano_root=osp.join(cfg.mano_path, 'mano', 'models'), flat_hand_mean=False, use_pca=False) worked.

[liumc14]

Can I have a look at your pred.py file? Thank you！

[adamqian111]

I have the opposite problem, I want to align the FreiHAND dataset with the obman dataset. I tried to subtract the mean from the provided FreiHAND parameters, but it does not work. how can I do this?

Hi, I have the same problem, I want to convert axis-angle to pca parameters, did you solve this problem?