Pinning down a lip color might have just gotten a little easier.
Pinterest revealed a new feature on Tuesday that allows users to virtually try lipstick. With Try On, users in the U.S. can test lip colors using augmented reality, courtesy of the selfie view of their Pinterest camera. The feature maps the onscreen looks to real world products from bareMinerals, Estée Lauder, Sephora, Neutrogena and L’Oréal’s NYX Professional Makeup, YSL Beauté, Lancôme and Urban Decay.
To give it a whirl, users can tap “Try On” in the Pinterest camera or check search results, where items that are available in AR show up with a “Try It” button. People can save the looks for later reference or purchase, or explore different looks with similar lip shades.
The feature was more than a year in the making, and debuts as an extension of Pinterest Lens — a computer-vision tool that can search the platform for ideas by using the camera to identify objects, from buildings to outfits, in the real world.
“We actually saw a lot of people taking selfies with Lens. But when you’re looking at yourself, the best experience isn’t necessarily to show you other people that look kind of similar to you, or to show celebrities that are similar,” Jeff Harris, Pinterest’s head of visual search, told WWD. “What people often want to do is figure out what kind of inspirational stuff they can do in their own looks. And this try-on feature is really an answer to that.”
The other basis for Try On was Pinterest’s work on skin tone detection, a tool that launched a year ago to address diverse users. Now the AR can account for more than lighting conditions and colors, but also coverage and finish — like matte or sheer — on a range of different complexions.
In practical application, Pinterest’s beauty AR works similarly to apps such as YouCam, Sephora and others. But there are some key differences.
While branded apps often try to create a social aspect to their experience, Pinterest is already a destination for more than 320 million users who come to the app to get ideas from one another.
Another distinction: The company made a conscious choice not to use filters or other beautifying techniques, opting for realism instead.
“A lot of AR experiences that you see will be things like skin smoothing or face thinning, and for us — and this is sort of a core philosophy for Pinterest — we want to help you feel confident exactly as you are. We don’t want to build tools that change the way you look,” Harris said. “There isn’t any kind of beautification.”
Pinterest’s focus on getting the realism right prompted it to order lipsticks, apply them to subjects and compare them to the virtual shades for literal reality checks. Then the team hand-tuned from there to ensure fidelity.
“This was stuff that, as a man, I was unfortunately quite ignorant about until we started working on this feature,” Harris continued. “But the way that it’s specified and the way that we work with partners — they’re not just telling us the color of the product, they’re telling us the desired capacity. They’re telling us whether it’s sparkly. There are a bunch of different ways that they customize the look, so it can be as lifelike and realistic as possible.”
It also helps that the launch brands have experience with augmented reality technology. Harris believes it simplifies things, since companies that have been in the space already have their product catalogs and AR parameters.
One of them was ModiFace, the AR beauty tech firm owned by L’Oréal.
“We worked with them in a deeper capacity,” Harris explained. “ModiFace was sort of one of the initial vanguards in this space, so a lot of the way that people create AR technology — like the parameters that they use to actually digitize it and map real-world products to something digital — a lot of that ModiFace trailblazed.” Pinterest incorporated elements from ModiFace tech into its existing features and visual search technology.
“We worked with them really closely to make sure that the products that we got, that were already ModiFace-compatible, would render really nicely in our experience,” he said.
Given the social media company’s obsession with computer vision and camera technology, and the massive beauty interest among its user base — according to Harris, some 52 million people search for beauty every month on the platform — it’s a wonder that it took so long to come out with an AR try-on feature.
Then again, perhaps not. Pinterest seems to defy tech company stereotypes of playing fast and loose with development. Early investor and FirstMark Capital director Rick Heitzmann once described cofounder and chief executive officer Ben Silbermann as someone who “measures twice, cuts once.” The thinking still apparently permeates the culture.
“We have a philosophy with both AI [artificial intelligence] and AR in general that we don’t want to launch stuff that we think is a good demo, but doesn’t fit seamlessly into people’s day-to-day experiences,” Harris said. “So at this point, we feel like lipstick is kind of there, and we were seeing enough people engaging with Lens and searching for beauty products on Pinterest.”
Lip colors are just a starting point. The company is interested in other aspects of beauty, like eye makeup and other categories, so more expansions are on the way.
If AR proves popular on the platform, it couldn’t come soon enough.
After redesigning its app last year, Pinterest missed analysts’ estimates for the third quarter of 2019, causing its shares to sink at the time. The shares have bounced back in recent days, after the platform edged out Snapchat in U.S. users during the quarter, at 82.4 million versus 80.2 million. A hit feature could potentially put more distance between the two.
So it makes some sense that the goal, at least for Try On, is not about immediate revenue, but user engagement. People interested in purchasing lipstick — or anything else — can buy, but not directly from Pinterest. With the same framework that undergirds its Shoppable Pins, a swipe up on the screen opens an e-commerce page from the participating brand.
In other words, the company isn’t angling to become the next Amazon.
“We don’t want Pinterest to primarily be about navigating between products. We want you to be mostly engaging with inspiration content,” Harris added. “So a lot of the technical investment that we did was mapping products and product colors into all of the looks that people are already engaging with.
“I think that that is a special part of the experience,” he said.