Apple Vision Pro has a super powered Visual Search feature — here’s what it could do

The Apple Vision Pro headset will let wearers get information about various items by simply looking at them, essentially making an augmented reality version of the Visual Lookup feature found in iPhones and iPads. 

Dubbed “Visual Search” this feature was discovered by MacRumors contributor Steve Moser, who took a deep dive into the Xcode beta of visionOS, Apple’s augmented and reality operating system that’ll power the Vision Pro headset. 

Reportedly, Visual Search will let Vision Pro users get information about objects in the real world and serve up text about those items in front of their eyes. From there, they can copy and paste the text into various apps, as well as translate that text into 17 different languages. 

This text will apparently include useful information such as webpages, contact details, unit conversions and more. MacRumors cites an example of being able to pull data from a real-world pamphlet and then share that information with others or save it for later. 

We can see such a function being handy in creative companies, say letting advertising designers better highlight the information contained in physical items like posters and flyers. It could also be a way for artists to serve up information and context around their art installations by providing people with the Vision Pro to have an augmented art experience. 

Equally, MacRumors noted that the ability to carry out live translation would be a boon for people traveling abroad, quickly getting translations and information about what they are seeing in front of their very eyes. The slight catch would be wearing such a piece of tech in say a busy market or historical site might make you stand out like a sore thumb. 

As mentioned, the ability to get information about the real world by simply looking at things already exists in smartphones. Google Lens, as found in the Google Pixel 7 Pro and other Android phones, is particularly good at this. But the logical next steps for such tech would be in augmented reality wearables. I’m not sure the Vision Pro will be quite the headset for that given its size and battery life, but I could see it working for slimmer augmented reality glasses, say the long-rumored Apple Glasses. 

More from Tom’s Guide