Skinning inverse world matrix

[helikopterodaktyl]

In this code from the skinning tutorial:

 update(node) {
    const globalWorldInverse = m4.inverse(node.worldMatrix);
    // go through each joint and get its current worldMatrix
    // apply the inverse bind matrices and store the
    // entire result in the texture
    for (let j = 0; j < this.joints.length; ++j) {
      const joint = this.joints[j];
      const dst = this.jointMatrices[j];
      m4.multiply(globalWorldInverse, joint.worldMatrix, dst);
      m4.multiply(dst, this.inverseBindMatrices[j], dst);
    }

I understand joint.worldMatrix holds the joint global pose matrix. inverse bind matrix is self explanatory. But what about globalWorldInverse? Where is this matrix coming from and why do we need to multiply it.

[greggman]

Sorry if it wasn't clear. From the article

<script id="skinVS" type="notjs">
attribute vec4 a_POSITION;
attribute vec3 a_NORMAL;
attribute vec4 a_WEIGHTS_0;
attribute vec4 a_JOINTS_0;

uniform mat4 u_projection;
uniform mat4 u_view;
uniform mat4 u_world;
uniform sampler2D u_jointTexture;
uniform float u_numJoints;

varying vec3 v_normal;

// these offsets assume the texture is 4 pixels across
#define ROW0_U ((0.5 + 0.0) / 4.)
#define ROW1_U ((0.5 + 1.0) / 4.)
#define ROW2_U ((0.5 + 2.0) / 4.)
#define ROW3_U ((0.5 + 3.0) / 4.)
 
mat4 getBoneMatrix(float jointNdx) {
  float v = (jointNdx + 0.5) / u_numJoints;
  return mat4(
    texture2D(u_jointTexture, vec2(ROW0_U, v)),
    texture2D(u_jointTexture, vec2(ROW1_U, v)),
    texture2D(u_jointTexture, vec2(ROW2_U, v)),
    texture2D(u_jointTexture, vec2(ROW3_U, v)));
}
 
void main() {
  mat4 skinMatrix = getBoneMatrix(a_JOINTS_0[0]) * a_WEIGHTS_0[0] +
                    getBoneMatrix(a_JOINTS_0[1]) * a_WEIGHTS_0[1] +
                    getBoneMatrix(a_JOINTS_0[2]) * a_WEIGHTS_0[2] +
                    getBoneMatrix(a_JOINTS_0[3]) * a_WEIGHTS_0[3];
  mat4 world = u_world * skinMatrix;
  gl_Position = u_projection * u_view * world * a_POSITION;
  v_normal = mat3(world) * a_NORMAL;
}
</script>

...

Also notice we multiply in the u_world matrix here. We subtracted it out in Skin.update with these lines

const globalWorldInverse = m4.inverse(node.worldMatrix);
// go through each joint and get its current worldMatrix
// apply the inverse bind matrices and store the
// entire result in the texture
for (let j = 0; j < this.joints.length; ++j) {
  const joint = this.joints[j];
  const dst = this.jointMatrices[j];
  m4.multiply(globalWorldInverse, joint.worldMatrix, dst);

Whether you do that or not is up to you. The reason to do it is it lets you instance the skin. In other words you can render the skinned mesh in the exact same pose at more than one place in the same frame. The idea being that if there are lots of joints then doing all the matrix math for a skinned mesh is slow so you do that math once and then you can display that skinned mesh in different places just by re-rendering with a different world matrix.

Inside the shader we multiply by u_world. The world matrix is already in joint.worldMatrix so if we didn't remove it by multiplying by globalWorldInverse then we'd be applying the world matrix twice

We have several options. 3 off the top of my head

1. Get rid of u_world in the shader

   Then we don't need to multiply by globalWorldInverse

2. Multiply by globalWorldInverse

   And leave u_world in the shader. Like it says above, this lets us draw the skinned mesh multiple times in the same pose at different locations. In other words.

   gl.drawArrays(....) // draw the mesh somewhere
   gl.uniformMatrix4fv(u_worldLoc, false, someOtherMatrix);
   gl.drawArrays(....) // draw the mesh somewhere else in the same pose
   

   With option 1, if we wanted to draw the skinned mesh in another location we'd have to recompute all the joint matrixes

3. Change the code that computes joint.worldMatrix so it computes a root joint relative matrix

   So the worldMatrix for the character isn't in their in the first place. This has tradeoffs as well. Often you want to put something at a bone's location. For example of a character is carrying a flashlight then you can just get the worldMatrix of one of the hand's of that character to position the flashlight. If the hand's matrix is already a world matrix (options 1 and 2 above) then you just use it. If instead to change the code so joint.worldMatrix computes a matrix relative to the root of the character then you'd need to add in world matrix again. Of course if you're instancing them (drawing more than one in the asme post) (option 2) then you'll have to multiply by that instances word matrix anyway so maybe this option 3 is best? It's a judgement call. You'll have to decide it on your own. In our case, we have a scenegraph and the joints are in that scenegraph so they get world matrices and you can easily add more things to the same scenegraph

Let me add, drawing a skinned mesh in the same pose is uncommon. Take a typical video game using skinned meshes for characters. Usually every character is doing something different and so in a different pose (joints oriented in different ways) so there's no reason to support drawing something twice in different locations with the same pose. The only reason to do that is if you want to portray a crowd of people, like a marching band, you might compute 1 pose and draw 10 of say 50 characters with that pose, then compute another pose and draw another 10 characters with that pose, repeat 3 more times. This would save you computing all the matrices for all 50 characters.