At Google I/O, Google senior vice president Prabhakar Raghavan announced new improvements to its Lens Multisearch tool, which allows you to perform a search with just an image and a few words. A new mode, called ‘near me’, allows users to take a photo of an object and then find results locally.
As Raghavan explained, you can take a picture of a dish and then search for restaurants that serve that particular food. Google will then show you a list of relevant restaurants in your area. To enable this feature, Google scans photos from the web and compares them to the photo you’ve uploaded. “Near Me” will be available in English to begin with and will be expanded to more languages ”over time”.
Google is also rolling out something called scene exploration. This allows users to pan their camera and then enter a search term about the objects in front of them. Raghavan used the example of looking for a chocolate bar without nuts. You can scan an entire shelf of chocolate bars and then see overlays that provide more information about each object.
The search giant first introduced multisearch in April, but its primary purpose was to shop and search for instructions. For example, you can take a screenshot or photo of a dress you like and then type in the name of a color. Google’s multisearch then shows a list of results with similar dresses in that color. Or you can take a picture of a particular type of plant and look up “care instructions” to find relevant information.
But, as Lou Wang, Google’s director of product management, hinted during release, Google’s multisearch feature can be used for much more, and it looks like that extra functionality is starting to arrive.
To use multisearch, open the Google app on Android or iOS, then click the Lens icon to the right of the search bar. From there, you can upload a photo or screenshot from your gallery — or snap a photo of an object for you — to begin your search. Then swipe up on the screen and select “Add to your search” to search for the photo next to a relevant word or phrase.