Asset with skinned mesh not at root, and various transforms

[javagl]

(@emackey @lexaknyazev as discussed during the call)

There should be a test asset with a skinned mesh node that is not the root node, and where the skinned mesh node or its parent contain some transform. The goal of that would be to verify that viewers do not apply these transforms, because neither the parent transform nor the local transform should affect the skinned mesh.

I took the existing SimpleSkin example, and added the respective nodes:

• The skinned mesh node has a parent with a translation of (1,0,0)
• The skinned mesh node itself has a translation of (0,1,0)

I also added some "floor" (a unit square) to make it visually obvious when a viewer displays the wrong thing.

The wrong way of displaying it would be

(here, showing both the x- and the y-translation)

The right way of displaying it is this:

The warnings that are generated by the validator are

NODE_SKINNED_MESH_NON_ROOT: Node with a skinned mesh is not root. Parent transforms will not affect a skinned mesh. /nodes/1
NODE_SKINNED_MESH_LOCAL_TRANSFORMS: Local transforms will not affect a skinned mesh. /nodes/1

as expected.

(I checked it in https://gltf-viewer.donmccurdy.com/ and https://sandbox.babylonjs.com/ , and they both display it correctly)

The embedded version of the asset is shown here:

{
  "accessors" : [ {
    "bufferView" : 0,
    "byteOffset" : 0,
    "componentType" : 5123,
    "count" : 24,
    "type" : "SCALAR",
    "max" : [ 9 ],
    "min" : [ 0 ]
  }, {
    "bufferView" : 1,
    "byteOffset" : 0,
    "componentType" : 5126,
    "count" : 10,
    "type" : "VEC3",
    "max" : [ 0.5, 2.0, 0.0 ],
    "min" : [ -0.5, 0.0, 0.0 ]
  }, {
    "bufferView" : 1,
    "byteOffset" : 160,
    "componentType" : 5123,
    "count" : 10,
    "type" : "VEC4",
    "max" : [ 0, 1, 0, 0 ],
    "min" : [ 0, 0, 0, 0 ]
  }, {
    "bufferView" : 1,
    "byteOffset" : 320,
    "componentType" : 5126,
    "count" : 10,
    "type" : "VEC4",
    "max" : [ 1.0, 1.0, 0.0, 0.0 ],
    "min" : [ 0.0, 0.0, 0.0, 0.0 ]
  }, {
    "bufferView" : 2,
    "byteOffset" : 0,
    "componentType" : 5123,
    "count" : 6,
    "type" : "SCALAR",
    "max" : [ 3 ],
    "min" : [ 0 ]
  }, {
    "bufferView" : 3,
    "byteOffset" : 0,
    "componentType" : 5126,
    "count" : 4,
    "type" : "VEC3",
    "max" : [ 0.5, 0.0, 0.5 ],
    "min" : [ -0.5, 0.0, -0.5 ]
  }, {
    "bufferView" : 3,
    "byteOffset" : 48,
    "componentType" : 5126,
    "count" : 4,
    "type" : "VEC3",
    "max" : [ 0.0, -1.0, 0.0 ],
    "min" : [ 0.0, -1.0, 0.0 ]
  }, {
    "bufferView" : 4,
    "byteOffset" : 0,
    "componentType" : 5126,
    "count" : 12,
    "type" : "SCALAR",
    "max" : [ 5.5 ],
    "min" : [ 0.0 ]
  }, {
    "bufferView" : 4,
    "byteOffset" : 48,
    "componentType" : 5126,
    "count" : 12,
    "type" : "VEC4",
    "max" : [ 0.0, 0.0, 0.707, 1.0 ],
    "min" : [ 0.0, 0.0, -0.707, 0.707 ]
  }, {
    "bufferView" : 5,
    "byteOffset" : 0,
    "componentType" : 5126,
    "count" : 2,
    "type" : "MAT4",
    "max" : [ 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0 ],
    "min" : [ 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, -1.0, 0.0, 1.0 ]
  } ],
  "animations" : [ {
    "channels" : [ {
      "sampler" : 0,
      "target" : {
        "node" : 3,
        "path" : "rotation"
      }
    } ],
    "samplers" : [ {
      "input" : 7,
      "interpolation" : "LINEAR",
      "output" : 8
    } ]
  } ],
  "asset" : {
    "generator" : "JglTF from https://github.com/javagl/JglTF",
    "version" : "2.0"
  },
  "buffers" : [ {
    "uri" : "data:application/gltf-buffer;base64,AAABAAMAAAADAAIAAgADAAUAAgAFAAQABAAFAAcABAAHAAYABgAHAAkABgAJAAgAAAAAvwAAAAAAAAAAAAAAAAAAAD8AAAAAAAAAAAAAAAAAAAC/AAAAPwAAAAAAAAAAAAAAPwAAAD8AAAAAAAAAAAAAAL8AAIA/AAAAAAAAAAAAAAA/AACAPwAAAAAAAAAAAAAAvwAAwD8AAAAAAAAAAAAAAD8AAMA/AAAAAAAAAAAAAAC/AAAAQAAAAAAAAAAAAAAAPwAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAAAAAAAQD8AAIA+AAAAAAAAAAAAAEA/AACAPgAAAAAAAAAAAAAAPwAAAD8AAAAAAAAAAAAAAD8AAAA/AAAAAAAAAAAAAIA+AABAPwAAAAAAAAAAAACAPgAAQD8AAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAABAAIAAQADAAIAAAAAvwAAAAAAAAC/AAAAPwAAAAAAAAC/AAAAvwAAAAAAAAA/AAAAPwAAAAAAAAA/AAAAAAAAgL8AAAAAAAAAAAAAgL8AAAAAAAAAAAAAgL8AAAAAAAAAAAAAgL8AAAAAAAAAAAAAAD8AAIA/AADAPwAAAEAAACBAAABAQAAAYEAAAIBAAACQQAAAoEAAALBAAAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAkxjEPkSLbD8AAAAAAAAAAPT9ND/0/TQ/AAAAAAAAAAD0/TQ/9P00PwAAAAAAAAAAkxjEPkSLbD8AAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAkxjEvkSLbD8AAAAAAAAAAPT9NL/0/TQ/AAAAAAAAAAD0/TS/9P00PwAAAAAAAAAAkxjEvkSLbD8AAAAAAAAAAAAAAAAAAIA/AACAPwAAAAAAAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAAAAAAAAAAAAgD8AAAAAAAAAAAAAAAAAAAAAAACAPwAAgD8AAAAAAAAAAAAAAAAAAAAAAACAPwAAAAAAAAAAAAAAAAAAAAAAAIA/AAAAAAAAAAAAAIC/AAAAAAAAgD8=",
    "byteLength" : 1004
  } ],
  "bufferViews" : [ {
    "buffer" : 0,
    "byteOffset" : 0,
    "byteLength" : 48,
    "target" : 34963
  }, {
    "buffer" : 0,
    "byteOffset" : 48,
    "byteLength" : 480,
    "byteStride" : 16,
    "target" : 34962
  }, {
    "buffer" : 0,
    "byteOffset" : 528,
    "byteLength" : 12,
    "target" : 34963
  }, {
    "buffer" : 0,
    "byteOffset" : 540,
    "byteLength" : 96,
    "byteStride" : 12,
    "target" : 34962
  }, {
    "buffer" : 0,
    "byteOffset" : 636,
    "byteLength" : 240
  }, {
    "buffer" : 0,
    "byteOffset" : 876,
    "byteLength" : 128
  } ],
  "materials" : [ {
    "pbrMetallicRoughness" : {
      "baseColorFactor" : [ 0.0, 0.5, 0.0, 1.0 ],
      "metallicFactor" : 0.0,
      "roughnessFactor" : 1.0
    },
    "doubleSided" : true
  }, {
    "pbrMetallicRoughness" : {
      "baseColorFactor" : [ 0.0, 0.0, 1.0, 1.0 ],
      "metallicFactor" : 0.0,
      "roughnessFactor" : 1.0
    },
    "doubleSided" : true
  } ],
  "meshes" : [ {
    "primitives" : [ {
      "attributes" : {
        "POSITION" : 1,
        "JOINTS_0" : 2,
        "WEIGHTS_0" : 3
      },
      "indices" : 0,
      "material" : 0,
      "mode" : 4
    } ]
  }, {
    "primitives" : [ {
      "attributes" : {
        "POSITION" : 5,
        "NORMAL" : 6
      },
      "indices" : 4,
      "material" : 1,
      "mode" : 4
    } ]
  } ],
  "nodes" : [ {
    "children" : [ 1 ],
    "translation" : [ 1.0, 0.0, 0.0 ]
  }, {
    "skin" : 0,
    "mesh" : 0,
    "translation" : [ 0.0, 1.0, 0.0 ]
  }, {
    "children" : [ 3 ]
  }, {
    "rotation" : [ 0.0, 0.0, 0.0, 1.0 ],
    "translation" : [ 0.0, 1.0, 0.0 ]
  }, {
    "mesh" : 1
  } ],
  "scene" : 0,
  "scenes" : [ {
    "nodes" : [ 0, 2, 4 ]
  } ],
  "skins" : [ {
    "inverseBindMatrices" : 9,
    "joints" : [ 2, 3 ]
  } ]
}

If this is considered to be a suitable test asset, I'll create a PR.

[bghgary]

Maybe use this as reference. We spent a lot of time in the past making sure edge cases are covered in this list.

https://github.com/KhronosGroup/glTF-Asset-Generator/blob/main/Output/Positive/Animation_Skin/README.md

[javagl]

@bghgary Indeed, entry 02 in that list looks like it is testing what is covered here.
@emackey Maybe that long-running task was not a task after all 🙂

I think that it could still make sense to have an asset where the correctness can be seen. From the rendering, it's not obvious that the transforms are ignored:

But no strong opinion on that for now.

[bghgary]

We didn't spend much time to make the rendering obvious as that wasn't a goal. I agree that would be nice.

[emackey]

Indeed, entry 02 in that list looks like it is testing what is covered here.

I think Alexey pointed out the issue with 02: Because most viewers automatically center the entire asset, there is no way to visually see whether any root transform has been applied or not. Including a "floor polygon" like @javagl's proposed asset here would fix the issue. I'm in favor of adding the new sample asset to this repo (unless someone plans to update Asset Generator, but that seems less likely in the short term).

[bghgary]

My comment was not intended to slow or stop the addition of such an asset to this repo. I just wanted to point out the other edge cases.

[create3000]

What about the RecursiveSkeleton example (https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Assets/master/Models/RecursiveSkeletons/glTF/RecursiveSkeletons.gltf), wondering if all the transforms above all the skinned meshes are not considered?

[emackey]

wondering if all the transforms above all the skinned meshes are not considered?

In glTF, each individual vertex in a skinned mesh has some number of joints that influence its location. This influence from multiple joints is required by spec to sum up to 100% exactly for each vertex. That means the skinned mesh's own local node transform has 0% influence, and the skeleton as a whole gets 100% influence over the location of any one vertex in the skin. For simple cases, that means the skin's local node transform is irrelevant to the visual appearance.

That said, we're looking into some edge cases where for example a skinned mesh might have non-skinned children, and the children might be influenced by the transforms on the parent mesh which normally would not have influence on the skin. Also we want to ensure that various implementations are indeed following the rules as laid out in the spec.

[lexaknyazev]

That means the skinned mesh's own local node transform has 0% influence, and the skeleton as a whole gets 100% influence over the location of any one vertex in the skin.

While that's certainly a way to describe it, one could say that node's local/global transform could be applied separately as if the whole skeleton had it as its parent. Practical implementation of that would be very convoluted (glTF 1.0 tried to support that) so the glTF 2.0 spec requires mesh node transforms to be ignored.