Google's 'universal AI assistant' can now perform tasks without prompts
What's the story
Google's ambitious AI project, Project Astra, is taking giant leaps toward becoming a "universal AI assistant."
Launched at Google I/O 2024, the prototype is now being tested by a select group of users.
The end goal of Project Astra is to integrate into daily life seamlessly without causing any disturbances.
Greg Wayne from Google DeepMind describes it as "the concept car of a universal AI assistant."
Integration
Project Astra's features are being integrated into Gemini
The successful features from Project Astra will eventually be integrated into Gemini and other apps. So far, this includes improvements in voice output, memory, and basic computer-use capabilities.
As these features go mainstream, the Astra team continues to explore new areas for development. This way, Google's AI tech is always evolving and improving based on user feedback and testing results.
Proactivity
Astra can now perform tasks without user prompts
At this year's I/O developer conference, Google unveiled new features for Project Astra, showcasing its potential as an intelligent assistant.
Beyond answering queries and using your phone's camera to locate misplaced items, Astra can now perform tasks on its own.
Wayne states that "Astra can choose when to talk based on events it sees." This is a shift from reactive assistance to proactive engagement in users' activities.
Challenges
DeepMind CEO highlights challenges of teaching proactive behavior
DeepMind CEO Demis Hassabis admited that teaching Astra to act on its own isn't easy.
He called it "reading the room," noting it's harder than it sounds.
The problem is knowing when to step in, what tone to adopt, how to best assist users, and when not to butt in.
These are subtleties humans deal with easily but are hard for AI systems like Astra.
Information access
Google plans to enhance Astra's access to user information
For Astra to work as a proactive assistant, it needs to know a lot about its users.
To make this happen, Google plans to improve Astra's access to information from the web and other Google products.
This could include checking calendars for scheduling reminders, or scanning emails for confirmation numbers.
However, making these features work consistently and reliably will take a lot of time and effort from Google's development team.
Device integration
Astra is learning to control Android phones
Project Astra is also progressing toward learning how to deal with Android phones.
In a demo, product manager Bibo Xiu demonstrated that after recognizing a pair of headphones, Astra could locate their manual and explain how to pair them with her phone.
When Xiu commanded Astra to open settings and pair the headphones, it did so on its own.
This level of interaction is what Apple aims for with its next-gen Siri assistant.