AI perception of hand movement is a vital component in significantly improving the user experience. For example, it can increase user engagement by superimposing digital content on the physical world in augmented reality. However, implementing real-time AR hand tracking or wrist perception is an absolutely challenging computer vision task because hands often occlude themselves and lack high-contrast patterns. Based on AI deep learning, Kivisense has broken through the challenges of real-time tracking to ensure the perfect virtual try-on experience and drive omnichannel customer engagement for brands in the Metaverse.
Significant Breakthroughs in AI Wrist Tracking Algorithms
For real-time AR hand tracking, the 3D models have to work across a variety of hand sizes with a large scale span relative to the image frame, and be able to recognize wrist movements and occluded hands. Although faces have high-contrast patterns, such as in the eye and mouth areas, the lack of such features in hands makes it relatively difficult to reliably detect them from visual features alone. Kivisense addressed this with self-developed AI wrist tracking algorithms. Check out the following breakthroughs:
- Extract the feature points of the palm to ensure accurate recognition even with palm movements
- Perceive the shape and movement of the wrist, and track it in real-time for a stable AR wrist tracking
- Make the watch, bracelet, bracelace, etc., fit your wrist better with real-time hand mask
- Calculate the width of your fingers and wrists relatively through AI measurement for more precise AR try-on effects
Detailed Comparison of Real-time AR Try-on and Image-based Try-on
In addition to real-time AR try-on, Kivisense also supports image-based try-on, which is a static experience from a single angle on the back of your hand. Both algorithms support multiple product categories like watch, bracelet, bracelace, ring, etc., and are available on various platforms, including the Web, App, Shopify, Social media, Boutique store, etc.
In terms of Network support, Kivisense has updated the server nodes in the United States and Europe, which supports AWS and Alibaba Cloud. Through the global CDN and GDPR standard, partners can easily experience fast, efficient, and secure delivery of content.
Embedding image-based try-on saves costs for partners compared to that of real-time try-on. Here is a detailed comparison of them:
| Comparisons | Image-based Try-on | AR Try-on in Real-time |
| Experience | A static try-on experience on the back of your hand | Freely rotate or move for perfect try-on effects in real-time |
| Response Rate | Process cloud computing within 2 seconds after taking photos | Real-time processing without any delay |
| Material Type | Both 2D & 3D Lower cost for 2D mass production | 3D More costly for 3D mass production |
| 3D Engagement | Pay extra to create a 3D engagement for 2D material | FREE |
| Product Category | watch, bracelet, bracelace, necklace, ring, etc. | watch, bracelet, bracelace, necklace, ring, etc. |
AR Plays a Great Role in Watches Digitalization
Generally, wearable devices might be returned for many reasons, such as incorrect size, improper fit, inconsistent look, etc. To reduce returns, one of the biggest challenges e-commerce businesses have to overcome is the inability of trying on products online before buying. Many innovation-oriented brands are addressing this with digital strategies like AR watch try-on.
Moreover, the role of Gen Z consumers has shifted from passive obedience to empowered dominance. They want to interact, belong and be the brands they buy from, and digital technology has significantly impacted the way they interact with luxury brands. The widespread usage of AR campaigns is inevitable for how brands position, market, and sell wearables tomorrow and beyond. Through Kivisense AR try-on engagement, various brands achieved a great performance in 2021.
- 8 times in average per user to re-try-on in a day
- Average 20 clicks per user in 3D & AR try-on engagement
- Purchase button hit increased by 310%
Impressive AR Try-on & 3D Engagement for Watch, Bracelet, and Ring in 2021
Kivisense AR try-on algorithms are available for any category you want, such as shoes, eyewear, handbag, watch, accessories, fragrance, etc. Here we show you the best AR try-on solutions with hand tracking.
AR Watch Try-on
The feasible Kivisense AR watch try-on technology can work on the Web, App, EC SaaS Platform like Shopify, Social media, WeChat Mini-program, Boutique store, etc. With AI wrist tracking in real-time, consumers can freely rotate or move their wrist to view perfect try-on effects in 360 degrees. You wouldn’t believe it’s just a 3D model made with high-precision 3D rendering, because it is as realistic as what you’d see it in a real boutique store.
Luxury brands such as Gucci, IWC, Rolex, etc., are at the forefront of releasing AR watch try-on, aiming to reshape the shopping experience and increase sales conversion rates. Cartier also introduced image-based try-on for its new watches on the Mini-program, which greatly improves consumer engagement.
Redmi X Wang Yibo launched an AR experience of pop-up stores on the App, WeChat, and Weibo, which supports AR try-on for Redmi Note 11 & watch, and 3D engagement for Note 10 & watch. Once consumers find the perfect one, they can complete the purchase with one click. Sharing the try-on photos via social media platforms is also available, greatly increasing brand awareness. Within seven days, it achieved 1 million UV growth and 310% sales. That’s amazing.
AR Bracelet Try-on
AR retail experiences for bracelets will capture hand movements and create an immersive shopping experience. Gucci and Cartier have both released hyper-realistic, ultra-smooth AR bracelet try-on to enhance online shopping experience. They modernize the luxury shopping experience in an ultra-personalized approach through the seamless AR engagement on the bare wrists. The highly-realistic textures, material reflections, and lighting simulations provide consumers with perfect virtual try-on effects, saving brands unnecessary return costs and driving significant sales growth.
AR Ring Try-on
Purchasing a ring is usually an emotional experience, especially during an engagement or wedding. Selecting the right one is quite time-consuming and stressful, so Pomellato augments the online shopping experience by introducing AR jewelry try-on on its Website, App, and Mini-program. It allows consumers to see how the ring looks and fits without stepping into a boutique store. The materials and gemstones are highly replicated through 3D rendering technology, which greatly increases user engagement.
Conclusion
Implementing accurate AR hand tracking might be a complicated concept for some developers, but Kivisense AR algorithms can accurately detect your hands, fingers, palm, wrist, etc., and track the movements in real-time. With end-to-end support, they will provide you with more precise hand tracking, including lightweight forms and more efficient data procession. Through continuous improvement and optimization, Kivisense helps partners easily integrate the virtual try-on experience into the Web, App, Social media, Shopify, Boutique store, etc., and drive omnichannel sales in the Metaverse.
