You’ll Soon Be Able to Use Gemini to Search Your Google Photos

At I/O 2024, Google announced a great new AI feature for Google Photos, simply called Ask Photos. With Ask Photos, you can treat the app like a chatbot, say, Gemini or ChatGPT: You can request a specific photo in your library, or ask the app a general question about your photos, and the AI will sift through your entire library to both find the photos and the answers to your queries.

How does Ask Photos work?

When you ask Ask Photos a question, the bot will make a detailed search of your library on your behalf: It first identifies relevant keywords in your query, such as locations, people, and dates, as well as longer phrases, such as “summer hike in Maine.”

After that, Ask Photos will study the search results, and decide which ones are most relevant to your original query. Gemini’s multimodal abilities allow it to process the elements of each photo, including text, subjects, and action, which helps it decide whether that image is pertinent to the search. Once Ask Photos picks the relevant photos and videos for your query, it combines them into a helpful response.

Google says your personal data in Google Photos is never used for ads and human reviewers won’t see the conversions and personal data in Ask Photos, except, “in rare cases to address abuse or harm.” The company also said they don’t train their other AI products with this Google Photos data, including other Gemini models and services.

What can you do with Ask Photos?

Of course, Ask Photos is an ideal way to quickly find specific photos you’re looking for. You could ask, “Show me the best photos from my trip to Spain last year,” and Google Photos will pull up all your photos from that vacation, along with a text summary of its results. You can use the feature to arrange these photos in a new album, or generate captions for a social media post.

However, the more interesting use here is for finding answers to questions contained in your photos without having to scroll through those photos yourself. Google shared a great example during its presentation: If you ask the app, “What is my license plate number?” it will identify your car out of all the photos of cars in your library. It will not only return a picture of your car with your license plate, but will answer the original question itself. If you’re offering advice to a friend about the best restaurants to try in a city you’ve been to, you could ask, “What restaurants did we go to in New York last year?” and Ask Photos will return both the images of the restaurants in your library, as well as a list you can share.

When will Ask Photos be available?

Google says the experimental feature is rolling out in the coming months, but no specific timeframe was given.