Introducing Image-to-Image Translation in Lens Studio
Introducing Image-to-Image Translation in Lens Studio
At Snap, we love nothing more than fueling your creativity. Whether it’s through new features, tools, or insights, we want to make sure our creative community is getting the most of their Snap AR experience, creating game-changing Lenses, and pushing their craft to new heights. Machine learning is one way we’re developing more advanced AR experiences — and that's why we're excited to announce a brand new SnapML template focused on Image-to-Image Translation machine learning model use cases.
Until now, using image-to-image networks in Lens Studio was just out of reach. Models were simply too large for mobile deployment, which prevented creators from getting the most out of their Lenses.
Now we’ve introduced a new method that demonstrates how to train and compress image-to-image networks — like CycleGAN and Pix2Pix — for better deployment on your mobile devices. By changing the way you prepare the dataset for your image output, we’ve reduced and optimized file size for your Snap Lenses. This new method is based and developed on a paper written by our own Snap Research Team, CAT, which you can read in full here.
This update makes it even easier for developers to build out immersive Lens experiences using Image-to-Image Translation. Effects such as collection style transfer, object transfiguration, season transfer, and photo enhancement are now easily compressed under 1 megabyte for use in mobile devices. Also, the new model is templated so you can drop it directly into Lens Studio and run it in real-time across different devices.
We’re not only excited about the new Image-to-Image Translation model — we’re thrilled by the opportunities it presents for creators like you. Transition effects like day-to-night, black-to-white, summer-to-winter, and painting-to-photo are just a few of the incredible ways the Snap AR community can use machine learning to their creative advantage.
This new model is a perfect example of how we're empowering our community to use SnapML to develop their own models for creating incredible Lenses. If you’d like to learn more about Image-to-Image Translation, as well as get training code for testing it out yourself, click here. Or, to see other ML Lens Templates, including Style Transfer, which this latest release utilized, click here.