logo

Module 3

Understanding External Body Mesh
Module 2 mentioned garment deforming, Body Mesh and External Body Mesh. Let’s dig into what External Body Mesh means, how it relates to garment deforming, and why you should take the time to create one.

What’s External Body Mesh?

It is an extension of the Body Mesh feature that allows you to fit meshes, such as clothes, onto the tracked user body. On runtime (in real time), external meshes will automatically deform/re-shape to conform to tracked bodies and move without requiring rigging.

Why should you create an External Body Mesh?

Snapchatters come in all shapes and sizes and you want your Lens to be body inclusive! Using External Body Mesh, you can create one 3D asset that will fit every Snapchatter by automatically adjusting to their body. If you are familiar with Body Tracking, you might be asking yourself why can’t I just use Body Tracking? You absolutely can. However, we recommend using External Body Mesh to get the closest fit of the garment to each user. Examples below help visualize how using External Body Mesh helps garments to dynamically deform to a user.

    What is required from a 3D Asset?

    Lens Studio provides documentation that covers how to prepare an External Body Mesh when using digital content creation tools that we listed in the Getting Started section. The guide will get you started on preparing your asset, but here are a few things to keep in mind:

    Model Fit

    The asset model has to fit to the base mesh as tightly as possible. Even though the reference model is not gender-specific, your garment needs to be fitted to it. As a result, your garment’s mesh will automatically deformed to match any body type. Any properties of your model that aren't fitted, in some cases, could become disproportionate. See examples below and pay attention to the hip area of the dress.

    Would any garment work with External Body Mesh?

    Every garment and accessory will perform differently in Lens Studio based on their mesh topology, which will affect the matching process (more below). Looser garments based on the design of the garment may need additional modifications to their setup to move less rigidly with the body. For example, a flowy dress may need to use Cloth Simulation in order to look more realistic in AR and showcase the fabric’s movement. We encourage you to experiment with different garments and accessories until you achieve desired results.

    How does External Body Mesh work?

    Once attached as an External Body Mesh, Lens studio will estimate a match between a garment mesh and the reference Body Mesh. Mesh matching is what drives the source mesh to move based on the Body Mesh deformations and on-demand re-shaping. It can be done in two ways:

    Proximity Matching

    Proximity Matching tries to match the source and target mesh based on the closest body area to each polygon of the source mesh. For tighter garments this may be a more reliable task, but the further a polygon is from the body, the harder it is to guess the matching.

    You can direct the matching by adjusting the Max Distance and Max Count parameters.

    Max Distance

    Controls the maximum distance under which a Body Mesh vertex can match to an external mesh vertex.

    Max Count

    Controls the maximum number of vertices on the base mesh that can match to an external mesh vertex.
    Example: a tight garment can use lower max distance values to keep matching strictly to local areas.

    Example: Loose fitted garment has more polygons further from the body skin and may require higher max distance. It can also benefit from higher Max Count values to soften the deformation effect around bending joints. Review the below examples of a person moving their arms with three different counts and distance applied. You can see the first animation where the sleeves are stiff and angular. As you increase the distance and count, you can see the animation smooth out the shirt.

    Part Submeshes

    Selectively enable or disable submeshes for the Body, Left Hand, Right Hand, and Head. You can experiment with removing the head or hands matching if your source mesh doesn’t have parts around those areas.
    Tip: Try disabling the hands for garments with long sleeves to prevent hands from affecting the external mesh.
    Hint: You should experiment with some combinations of distance and count values with different poses to see what works best for your garment mesh. 
    To set up a mesh for proximity matching, it must follow the guidelines described in Prepare an External Mesh. However, if you are using one of the 3D garments from Lens Studio’s Asset Library, you do not need to worry about preparing an External Mesh. Asset Library Items already follow these guidelines.

    Manual UV Matching

    If you find that no matter what you do, you can’t match the meshes using proximity matching, you can always manually define the matching yourself using UV matching:
    • Open your mesh in a 3D software such as Maya or Blender. 
    • A good place to start is placing the mesh on top of the reference mesh and remap the UV onto your mesh from the reference.
    • You can adjust the UV map of your mesh to reflect when parts of the reference mesh should be driving every part of the mesh.