Microsoft Copilot just gained 'vision' to become your AI-powered tutor
What's the story
Microsoft has launched a new feature, Copilot Vision, for its AI assistant on Windows.
The innovative tool lets users share their screen or apps with Copilot, which can then guide them through using different features of programs like Adobe Photoshop.
It can also analyze photos and webpages in real time.
The company first started testing this feature earlier this year and is now making it available for free in the US to all Windows 10 and 11 users.
Feature details
How to share your screen with Copilot
Copilot Vision works within the Copilot app and can be activated through a glasses icon.
This lets you select a browser window or app to share with the AI, similar to how you would share your screen during a Microsoft Teams meeting.
However, unlike Recall which continuously takes snapshots of your screen once enabled, this is an opt-in experience where users have to specifically share their apps for it to work.
User control
Real-time analysis of shared content
Once you share an app window, Copilot can highlight parts of your screen to guide you and answer questions.
Microsoft has assured users that they are in control with this tool as it only analyzes what is shared with it, ensuring no unauthorized access to other content on the screen.
Enhanced functionality
Highlights feature for in-app guidance
Copilot Vision also comes with an option called Highlights, which lets you ask the AI how to do something in a specific app.
It will then guide you through the steps within that app.
For instance, if you're stuck at a certain level in a game, you can ask Copilot Vision for tips on how to proceed and it will nudge you along.
App integration
Connect apps for a more holistic view
You can even share two apps or files at a time with Copilot Vision, which will then connect the dots between them.
For instance, if you share your calendar and a webpage of interesting events, the AI can find a suitable date for you to attend those events.