Google's: sCaRy Masks A/R Project
Halloween 2020 — Transforming Masked Faces into Spooky Fun
During the height of COVID-19, with lockdowns, health concerns, and financial stress weighing heavily on families, the joy of Halloween seemed out of reach—especially for children trying to understand these changes. To help preserve the magic, we created an advanced Augmented Reality project that transformed children’s faces into friendly, animated Halloween characters using Instagram's facial detection technology. This free, tech-driven solution helped to bring families together and delivered a much-needed dose of fun and creativity during a challenging time.
The Goal: Utilized the power of AR to create a safe, fun, and interactive experience for children wearing masks, providing a playful distraction during a challenging time.
The Result: It was a fun and immersive distraction for children and parents in socially distanced Halloween celebrations.
How it Works
There were four characters for the children to choose from, each featuring a distinctive lower portion of a monster's face as a drawing template. The children used this template to draw their own designs directly onto medical face masks, which everyone was wearing at the time.
Once the children put on the masks, parents could use Instagram's AI face swap feature and the phone's camera to scan the drawing. The recognition and tracking software would then replace the child’s head with an animated 3D monster head, bringing their artwork to life.
Parents could capture video recordings and photos of the experience, creating fun keepsakes to save or share.
My Role:
In collaboration with Inphantry, I was responsible for delivering all the 3D content creation for this project. Ensuring the project was both visually stunning and technically optimized for Instagrams AI face swap feature.
My contributions included:
-
Sculpting four unique Halloween characters and retargeting them into efficient, game-ready meshes for seamless engine performance.
-
Crafting advanced PBR materials and sourcing high-quality textures using Substance Painter to achieve a polished and spooky look.
-
Providing daily updates and implementing client feedback to refine content.
-
Hiring and managing contractors to rig and animate the characters in Maya, while I simultaneously create the next character.
-
Integrating finalized assets into Instagram’s Spark AR engine, setting up the project features, lighting, cameras, and facial recognition plugins to deliver an immersive and interactive functionality
The Deadline:
All 3D creative development was started and completed within an incredibly tight 12-day timeframe. This included modeling, texturing, rigging, animating, and in app setup. All delivered to meet Instagram's approval process in time for Halloween.
Tools and Workflow:
-
ZBrush for rapid digital sculpting and initial paint details.
-
Maya: Used for character setup, asset development, animation, and export of .FBX to Spark Engine.
-
Modo: Utilized for creating the initial low-polygon models and UV layouts.
-
Substance Painter: Texture development and transferring high-quality AO and Normal maps
This streamlined process allowed for efficient asset creation and timely delivery under challenging conditions.
2D Character Design
Bat Concept Design
Monster Concept Design
Although I didn’t have the opportunity to work directly with these talented artists, but it was truly an honor to bring some of their 2D designs to life in 3D.
Monster character concepts by:
Invisible Creature Studio.
Pumpkin Concept Design
Cat Concept Design