
AR Virtual Try-On With Face Tracking for an Omnichannel Eyewear Retailer
Let's Connect
Overview
What we built
An eyewear retailer with 28 stores was losing margin to fit-related returns because online shoppers could not see how frames would actually look on their own face. We built an AR try-on that lets them see it before they buy.
In plain terms: buying glasses online meant choosing frames from a flat photo and a size guide nobody read, so shoppers had no real sense of how a pair would sit on their own face. Online frame purchases came back at nearly twice the rate of in-store purchases, each one eating into margin, and store teams had no idea what a customer had already browsed online before walking in for an appointment, so every fitting started from zero.
We built an AR virtual try-on that renders frames on the shopper's own face in real time, using camera-based face tracking in the iOS app and directly in the mobile browser, so no separate app download was needed to try it. A 3D asset pipeline turned the frame catalogue into try-on-ready models, and saved favourites sync to store appointments so opticians start every fitting already knowing the customer's shortlist. Conversion on frame product pages rose 28% for shoppers who used try-on, fit-related returns fell from 24% to 15%, 41% of mobile sessions on frame pages engaged the try-on within 3 months of launch, and shortlists now attach to 1 in 5 in-store appointments booked through the site.
The Problem
Frames bought blind online
Across the retailer's 28 stores and its growing e-commerce arm, buying glasses online and buying them in person were two very different experiences. In store, a customer picks up a pair and looks in a mirror. Online, they chose from a flat product photo and a size guide that, in practice, went largely unread.
That gap showed up directly in returns. Online frame purchases came back at nearly twice the rate of in-store purchases, and the reason was consistent: shoppers could not judge how a frame actually sat on their own face until the physical pair arrived. Every one of those returns was margin the retailer had already spent to acquire and fulfil the order.
The disconnect extended into the stores themselves. When a customer who had browsed online came in for an appointment, the optician had no view of what they had looked at or shortlisted beforehand. Every in-store fitting started from a blank page, even for customers who had already done real research online.
No way to judge fit online
Shoppers chose frames from a flat product photo with no sense of how a pair would sit on their own face before ordering.
Unread size guides
Size guides intended to help shoppers judge fit went largely unread, leaving online buyers to guess at how frames would look.
Returns eating into margin
Online frame purchases were returned at nearly twice the in-store rate, with fit-related returns absorbing margin on every affected order.
Blind in-store fittings
Store teams had no view of what a customer had browsed online, so every in-store appointment started without knowing the customer's shortlist.
What it was costing them
Every fit-related return cost the retailer the margin on that order plus the cost of processing it back, and with returns running at nearly twice the in-store rate, that added up across all 28 stores and the growing online arm together. Unread size guides meant shoppers had no real substitute for trying frames on, and opticians starting every appointment blind meant even engaged, research-minded customers got a slower, less informed in-store experience.
The Solution
Face-tracked virtual try-on
We built an AR virtual try-on that puts frames on the shopper's own face using camera-based face tracking, so the moment a size guide used to try and fail to convey now happens visually, in seconds. It runs in the iOS app and directly in the mobile browser, so shoppers who never download an app still get the same experience.
A 3D asset pipeline converted the retailer's frame catalogue into try-on-ready models, so every frame a shopper could browse online was also one they could see rendered on their own face. This closed the biggest gap between the online and in-store experience without asking the retailer to rebuild its catalogue.
Saved favourites now sync to store appointments, so when a customer who has browsed and tried frames on online books a fitting, the optician opens the appointment already knowing their shortlist. The online and in-store journeys became one continuous experience rather than two disconnected ones.
Key decisions
Track faces in real time
Camera-based face tracking renders frames on the shopper's own face as they move, giving an immediate answer to the question a size guide never could.
Support app and mobile browser
We built the try-on for the iOS app and directly in the mobile browser, so shoppers get the experience whether or not they have installed the app.
Convert the whole catalogue
A 3D asset pipeline turned the frame catalogue into try-on-ready models, so try-on covers what shoppers can already browse rather than a limited selection.
Connect favourites to appointments
Saved favourites sync directly to store appointments, giving opticians the customer's shortlist before the fitting starts rather than after.
Close the online-to-store gap
We designed try-on and appointment syncing as one flow, so browsing online and visiting a store became connected steps rather than separate journeys.
Measurable Impact
What changed after launch
Shoppers who used the try-on converted more: product page conversion rose 28% for that group compared with those who did not use it. And the biggest problem, fit-related returns, moved directly: the return rate on online frame orders fell from 24% to 15% within two quarters of launch.
Adoption came quickly too. 41% of mobile sessions on frame pages engaged the try-on within 3 months of launch, and the connection between online browsing and in-store visits started showing up in bookings, with try-on shortlists now attached to 1 in 5 in-store appointments made through the site.
Frame page conversion
Baseline conversion without try-on
Up 28% for shoppers who used try-on
Fit-related returns
24% of online frame orders
Down to 15% within two quarters
Try-on adoption
No try-on available online
41% of mobile sessions engaged within 3 months
Appointment shortlists
Opticians had no view of online browsing
Attached to 1 in 5 appointments booked online
Headline results
Conversion on frame product pages up 28% for shoppers who used virtual try-on versus those who did not
Fit-related return rate on online frame orders down from 24% to 15% within two quarters
41% of mobile sessions on frame pages engaged the try-on within 3 months of launch
Try-on shortlists attached to 1 in 5 in-store appointments booked through the site
Tech & Tools Used
What powered the build
Every tool below earned its place in this engagement. Here is the part each one played.
Swift (ARKit)
Powers the native try-on experience inside the iOS app, using face tracking to render frames on the shopper's face in real time.
MediaPipe Face Mesh
Provides the facial landmark tracking that positions each frame model accurately on the shopper's face, in both the app and the mobile browser.
TensorFlow.js
Runs face tracking directly in the mobile browser, giving shoppers the try-on experience without needing the iOS app installed.
Three.js
Renders the 3D frame models over the camera feed for the mobile browser version of the try-on.
Blender
Used in the 3D asset pipeline to prepare and refine the frame catalogue into try-on-ready models.
Next.js
Delivers the browser-based try-on experience and the shopper-facing pages where saved favourites are managed.
Node.js
Runs the services connecting saved favourites, the frame catalogue and store appointment booking.
PostgreSQL
Stores the frame catalogue's try-on-ready model references, saved favourites and their links to booked store appointments.
AWS S3 + CloudFront
Hosts and delivers the 3D frame models and try-on assets quickly to shoppers browsing on mobile.
Ready to Build your Retail & E-Commerce Business with AR & Visual Simulation
Ask Byte
Ask Byte
Typically replies instantly
just Now
Hi! I'm OrganByte's assistant. How can I help you today?
AI-generated content may be incorrect

