Copilot Vision brings Microsoft’s screen-watching AI to everyday Windows tasks

AI overload: Microsoft wants Copilot to be your digital sidekick, always watching and ready to help – whether you need it or not. The latest feature, Vision, turns the AI into a screen-reading assistant that talks back, offering tips, tasks, or tech-induced headaches.

Microsoft announced that Copilot Vision is now available to users in the US, with plans to expand to more non-European countries soon. This update significantly enhances Copilot, which is evolving into more of a digital “companion” for everyday tasks – hallucinations and all. For those wary of intrusive AI “solutions” chasing problems that don’t exist, Copilot Vision remains an opt-in feature – for now.

Copilot Vision sees what users see and helps them get things done, Microsoft said on the Copilot blog. Available through the Copilot app, it can navigate multiple application windows at once. Users can ask Copilot to perform specific tasks or offer suggestions, and the AI will either comply or invent a convincing-sounding solution. It’s glorious – if you’ve already accepted living in an AI-slop-filled world.

Copilot Vision also includes a new “Highlights” feature that shows specific steps to complete tasks. It can assist with playing games, viewing photos, and adjusting lighting to enhance colors. Copilot can talk too, conversing in real-time with a synthetic voice, just like the classic dystopian movies that shaped our vision of the future.

Never mind that many daily computer tasks require little actual “intelligence,” or that most Windows users are perfectly capable of searching the web on their own. Microsoft insists Copilot makes everything better – and this is how far it’s willing to go to embed large language models deeply into Windows.

Microsoft began testing Copilot Vision earlier this year, and the feature is now available on Windows 10 and 11 through the Copilot Labs program – no Copilot Pro subscription required. Users are said to remain in control of the new app-sharing option and can supposedly stop the AI from peeking at their personal data at any time.

Copilot doesn’t capture desktop screenshots every few seconds like Recall does, so it poses less of a threat to privacy and data confidentiality – at least for now. Still, any tool that watches the screen and processes sensitive information in real-time is bound to raise security concerns. Given Microsoft’s recent track record, users have every reason to remain cautious.