Google's AI Mode search can now answer questions about images
Briefly

In March 2024, Google advanced its search capabilities by introducing AI Mode, which leverages a custom version of the Gemini large language model. This feature allows users to conduct searches not only via text but also through images, enhancing the search experience. The updated Gemini model, supported by Google Lens, interprets image content to inform search results more contextually and accurately. As part of this user-centric approach, the AI Mode also utilizes a fan-out technique to delve deeper into user queries and maximize the relevance of suggestions, demonstrating Google's commitment to evolving search technology.
Google's AI Mode marks a significant shift in search functionality, transforming it from a simple list of links to a multimodal experience integrating images.
With AI Mode's release, Google shows users it’s possible to upload images directly in search, fundamentally changing how they interact with its search engine.
This new capability, powered by the Gemini LLM, allows for more nuanced searches as the model utilizes image recognition from Google Lens to enhance result relevance.
The fan-out technique enables AI Mode to make multi-layered queries based on image content, thereby providing tailored suggestions for users' inquiries.
Read at Ars Technica
[
|
]