Google Lens will now let people search the internet via video as well as still images thanks to a set of new features just announced.
Video search lets users point their camera at an object, record it, ask a question about it, and the app will bring up search results. Google Lens is a separate app on Android while iOS users can access it via the Google app.
The features use a customized version of the Gemini AI model which uses computer vision to identify what is in the video and has the ability to answer pertinent questions.
“Let’s say you want to learn more about some interesting fish,” Lou Wanlg, director of product management, explains to the press via Tech Crunch. “Lens will produce an overview that explains why they’re swimming in a circle, along with more resources and helpful information.”
Users wanting to gain access to Lens’ new video search feature must sign up for Google’s Search Labs program and opt into the “AI Overviews and more” experimental features in Labs. While in the Google app, users can hold down the smartphone camera shutter button which will activate Lens’ video capture mode.
With the video recording, the user can ask a question and Lens will link out to an answer supplied by AI Overviews, people will be familiar with AI overviews that have started appearing at the top of Google search results.
Wang explains that the AI model powering Lens determines which frames from the video are interesting and which frames will help answer the questions being asked by the person.
“All this comes from an observation of how people are trying to use things like Lens right now,” Wang says. “If you lower the barrier of asking these questions and helping people satisfy their curiosity, people are going to pick this up pretty naturally.”
The ability to ask a question about still images has also been rolled out too. Users can now take a photo by holding down the shutter button and ask a question which will return a relevant search result.
Another update to Lens is e-commerce-specific functionality. Now when Lens recognizes a product it will display more detailed information about it, such as how much it costs and any relevant deals, reviews, and stock. It is limited to select countries and specific shopping categories like electronics, toys, and beauty.
“Let’s say you saw a backpack, and you like it,” Wang says. “You can use Lens to identify that product and you’ll be able to instantly see details you might be wondering about.”