Google Gemini can now describe the world around you
What's the story
Google has launched a new feature called Guided Vision, which uses artificial intelligence (AI) to provide real-time audio descriptions of anything you point your phone's camera at. The feature is available in Gemini Live on compatible Android devices. By sharing their camera with Gemini Live, users can have Google's AI assist them in reading small text, describing their surroundings, identifying objects around them and providing details about specific items.
Accessibility feature
Guided Vision is similar to Apple's VoiceOver Live Recognition
Guided Vision is similar to Apple's VoiceOver Live Recognition feature, which was added to iPhone and Vision Pro.
The new Google feature is specifically designed for people who are blind or have low vision, as well as those who may need assistance in certain situations.
Apart from the Gemini app, Guided Vision can also be accessed through Google TalkBack or by setting up an accessibility shortcut in the Settings app on devices running Android 9 and above.
User interaction
Users can ask follow-up questions about what they're looking at
The Guided Vision feature also lets users ask follow-up questions about whatever they're looking at. For instance, you can ask Gemini to read off the expiration date on an item it helped you find.
Google says that Gemini will provide audio cues to help line up your camera if you ask about something that's not in the shot.
However, it's important to note that Google warns against using Guided Vision for navigation or obstacle detection purposes.