Module 3

Understanding External Body Mesh

Module 2 mentioned garment deforming, Body Mesh and External Body Mesh. Let’s dig into what External Body Mesh means, how it relates to garment deforming, and why you should take the time to create one.

It is an extension of the Body Mesh feature that allows you to fit meshes, such as clothes, onto the tracked user body. On runtime (in real time), external meshes will automatically deform/re-shape to conform to tracked bodies and move without requiring rigging.

Snapchatters come in all shapes and sizes and you want your Lens to be body inclusive! Using External Body Mesh, you can create one 3D asset that will fit every Snapchatter by automatically adjusting to their body. If you are familiar with Body Tracking, you might be asking yourself why can’t I just use Body Tracking? You absolutely can. However, we recommend using External Body Mesh to get the closest fit of the garment to each user. Examples below help visualize how using External Body Mesh helps garments to dynamically deform to a user.

What is required from a 3D Asset?

Lens Studio provides documentation that covers how to prepare an External Body Mesh when using digital content creation tools that we listed in the Getting Started section. The guide will get you started on preparing your asset, but here are a few things to keep in mind:

The asset model has to fit to the base mesh as tightly as possible. Even though the reference model is not gender-specific, your garment needs to be fitted to it. As a result, your garment’s mesh will automatically deformed to match any body type. Any properties of your model that aren't fitted, in some cases, could become disproportionate. See examples below and pay attention to the hip area of the dress.

If your garment is very different in shape from the reference mesh (i.e. very loose fit, or has special geometry), consider using UV matching (see below).

Make sure to remove any inner facing polygons (i.e. any polygons that you don’t see once worn on the body). This can create undesired artifacts and incorrect matching.

T-Pose vs A-Pose. Lens Studio supports both standard poses. To the right is an example of a T-Pose on the left, and an A-Pose on the right. In a T-pose, the arms are at a 90 degree angle, providing the largest range of shoulder deformation. In the A-pose, the arms are at a 45 degree angle, which typically results in better deformation in most poses. A-pose uses a more relaxed body pose that helps reduce stretching of the clothes around the shoulders and legs, providing a better fit of the garment.

If you already have an asset and it's using either pose, do not worry about adjusting it and add the asset to your Lens Studio project. If you are creating an asset, we recommend using A-pose to achieve a more optimized fit of the garment.

Be mindful of the mesh topology for the garment you make and use a clean quad mesh. This will result in a garment that better deforms in more poses. To illustrate what a clean quad mesh means, take a look at the example below. The mesh topology similar to the sample on the right will result in better garment deformation.

Every garment and accessory will perform differently in Lens Studio based on their mesh topology, which will affect the matching process (more below). Looser garments based on the design of the garment may need additional modifications to their setup to move less rigidly with the body. For example, a flowy dress may need to use Cloth Simulation in order to look more realistic in AR and showcase the fabric’s movement. We encourage you to experiment with different garments and accessories until you achieve desired results.

How does External Body Mesh work?

Once attached as an External Body Mesh, Lens studio will estimate a match between a garment mesh and the reference Body Mesh. Mesh matching is what drives the source mesh to move based on the Body Mesh deformations and on-demand re-shaping. It can be done in two ways:

Proximity Matching tries to match the source and target mesh based on the closest body area to each polygon of the source mesh. For tighter garments this may be a more reliable task, but the further a polygon is from the body, the harder it is to guess the matching.

You can direct the matching by adjusting the Max Distance and Max Count parameters.

Controls the maximum distance under which a Body Mesh vertex can match to an external mesh vertex.

Controls the maximum number of vertices on the base mesh that can match to an external mesh vertex.

Example: a tight garment can use lower max distance values to keep matching strictly to local areas.

Example: Loose fitted garment has more polygons further from the body skin and may require higher max distance. It can also benefit from higher Max Count values to soften the deformation effect around bending joints. Review the below examples of a person moving their arms with three different counts and distance applied. You can see the first animation where the sleeves are stiff and angular. As you increase the distance and count, you can see the animation smooth out the shirt.

Selectively enable or disable submeshes for the Body, Left Hand, Right Hand, and Head. You can experiment with removing the head or hands matching if your source mesh doesn’t have parts around those areas.

Tip: Try disabling the hands for garments with long sleeves to prevent hands from affecting the external mesh.

Hint: You should experiment with some combinations of distance and count values with different poses to see what works best for your garment mesh.

To set up a mesh for proximity matching, it must follow the guidelines described in Prepare an External Mesh. However, if you are using one of the 3D garments from Lens Studio’s Asset Library, you do not need to worry about preparing an External Mesh. Asset Library Items already follow these guidelines.

If you find that no matter what you do, you can’t match the meshes using proximity matching, you can always manually define the matching yourself using UV matching:

Open your mesh in a 3D software such as Maya or Blender.
A good place to start is placing the mesh on top of the reference mesh and remap the UV onto your mesh from the reference.
You can adjust the UV map of your mesh to reflect when parts of the reference mesh should be driving every part of the mesh.

What's Next?

Module 4

Questions?

Forums

Documentations

Twitter

YouTube

Module 3

Module 2 mentioned garment deforming, Body Mesh and External Body Mesh. Let’s dig into what External Body Mesh means, how it relates to garment deforming, and why you should take the time to create one.

What’s External Body Mesh?

Why should you create an External Body Mesh?

What is required from a 3D Asset?

Lens Studio provides documentation that covers how to prepare an External Body Mesh when using digital content creation tools that we listed in the Getting Started section. The guide will get you started on preparing your asset, but here are a few things to keep in mind:

Model Fit

Garment Shape

Garment Polygons

Asset Pose

Quad Mesh

Would any garment work with External Body Mesh?

How does External Body Mesh work?

Once attached as an External Body Mesh, Lens studio will estimate a match between a garment mesh and the reference Body Mesh. Mesh matching is what drives the source mesh to move based on the Body Mesh deformations and on-demand re-shaping. It can be done in two ways:

Proximity Matching

Max Distance

Max Count

Part Submeshes

Manual UV Matching

What's Next?

Questions?