Microsoft Copilot can now browse web, analyze videos for you
What's the story
As part of its 50th anniversary celebrations, Microsoft is supercharging its AI-powered chatbot, Copilot. The updated version, with the Copilot Vision feature, can now take actions on most websites, from booking tickets to making restaurant reservations. It has also learned to remember certain user preferences like favorite food and movies. The bot's new capability also allows it to analyze real-time video from your phone camera and answer questions based on what it "sees."
Web actions
Copilot's new web-browsing capabilities
Microsoft has partnered with a number of online platforms for Copilot's improved web-browsing capabilities. These include 1-800-Flowers.com, Booking.com, Expedia, Kayak, OpenTable, Priceline, Tripadvisor, Skyscanner, Viator, and Vrbo. Now, you can give prompts like "send a bouquet to my partner," and Copilot will try to complete these tasks on the web. It can also track online deals for you by alerting you about price drops/sales on items of interest.
Multimedia capabilities
Podcast generation and visual analysis
The upgraded Copilot can generate "podcasts" like Google's NotebookLM Audio Overviews. Given a website or study, it can create a dialog between two synthetic hosts that you can interrupt with questions, and they'll acknowledge it and respond. On Android and iOS devices, the revamped Copilot app can now analyze real-time video from your phone camera or photo gallery. It even provides information about what's visible in the phone's camera viewfinder or photo gallery.
Advanced functionalities
Desktop and project management
On Windows, the revamped Copilot app can see what's on a user's desktop screen to search, change settings, organize files, and more. The feature will be available for members of the Windows Insiders program initially starting next week. Notably, Copilot also offers a project-consolidating Pages function that organizes notes and research into a canvas that can be turned into a document with its help. Pages is heavily inspired by ChatGPT's Canvas and Anthropic's Claude Artifacts tool.
Information synthesis
Deep Research and memory features
Complementary to Pages, Copilot's new Deep Research feature finds, analyzes, and combines information from online sources, documents, and images to answer complex queries. Similar to ChatGPT and Gemini's Deep Research, it can remember more about you over time. However, Microsoft assures that you have control over what types of information Copilot remembers about you or the option to opt out entirely. This is in line with the growing trend toward AI systems that adapt based on user interactions.