![](https://cdn.vox-cdn.com/thumbor/7QQuPDiZvyw_4NUVHaGxUiJngYo=/0x0:5000x3572/1200x628/filters:focal(2500x1786:2501x1787)/cdn.vox-cdn.com/uploads/chorus_asset/file/25443180/Google_IO.png)
Google I/O just ended — and it was packed with AI announcements. As expected, the event focused heavily on Google’s Gemini AI models, along with the ways they’re being integrated into apps like Workspace and Chrome.
If you didn’t get to tune in to the event live, you can catch up with all the latest from Google in the roundup below.
Google Lens now lets you search by recording a video
Google Lens already lets you search for something based on images, but now Google’s taking things a step further with the ability to search with a video. That means you can take a video of something you want to search for, ask a question during the video, and Google’s AI will attempt to pull up relevant answers from the web.
Gemini will answer questions about your photo library
Google is rolling out a new feature this summer that could be a boon for just about anyone with years — or even more than a decade — of photos to sift through. “Ask Photos” lets Gemini pore over your Google Photos library in response to your questions, and the feature goes beyond just pulling up pictures of dogs and cats. CEO Sundar Pichai demonstrated by asking Gemini what his license plate number is. The response was the number itself, followed by a picture of it so he could make sure that was right.
Google’s flagship Gemini model gets faster and more capable
Google has introduced a new AI model to its lineup: Gemini 1.5 Flash. The new multimodal model is just as powerful as Gemini 1.5 Pro, but it’s optimized for “narrow, high-frequency, low-latency tasks.” That makes it better at generating fast responses. Google also made some changes to Gemini 1.5 that it says will improve its ability to translate, reason, and code. Also, Google says it has doubled Gemini 1.5 Pro’s context window (how much information it can take in) from 1 million to 2 million tokens.
Gemini joins users in Workspace
Google is rolling its latest mainstream language model, Gemini 1.5 Pro, into the sidebar for Docs, Sheets, Slides, Drive, and Gmail. When it rolls out to paid subscribers next month, it will turn into more of a general-purpose assistant within Workspace that can fetch info from any and all of the content from your Drive, no matter where you are. It will also be able to do things for you, like write emails that incorporate info from a document you’re currently looking at or remind you later to respond to an email you’re perusing. Some early testers already have access to these features, but Google says it’s rolling it out to all paid Gemini subscribers next month.
Project Astra is Google’s Star Trek AI future
Google’s Project Astra is a multimodal AI assistant that the company hopes will become a do-everything virtual assistant that can watch and understand what it sees through your device’s camera, remember where your things are, and do things for you. It’s powering many of the most impressive demos from I/O this year, and the company’s aim for it is to be an honest-to-goodness AI agent that can’t just talk to you but also actually do things on your behalf.
Veo brings Sora-style video generation to creators
Google’s answer to OpenAI’s Sora is a new generative AI model that can output 1080p video based on text, image, and video-based prompts. Videos can be produced in a variety of styles, like aerial shots or timelapses, and can be tweaked with more prompts. The company is already offering Veo to some creators for use in YouTube videos but is also pitching it to Hollywood for use in films.
Gems bring custom chatbot creation to Gemini
Google is rolling out a custom chatbot creator called Gems. Just like OpenAI’s GPTs, Gems lets users give instructions to Gemini to customize how it will respond and what it specializes in. If you want it to be a positive and insistent running coach with daily motivations and running plans — aka my worst nightmare — you’ll be able to do that soon (if you’re a Gemini Advanced subscriber).
Gemini becomes a better conversation partner
The new Gemini Live feature aims to make voice chats with Gemini feel more natural. The chatbot’s voice will be updated with some extra personality, and users will be able to interrupt it mid-sentence or ask it to watch through their smartphone camera and give information about what it sees in real time. Gemini is also getting new integrations that let it update or draw info from Google Calendar, Tasks, and Keep, using multimodal features to do so (like adding details from a flyer to your personal calendar).
Circle to Search can help solve math problems now
If you’re on an Android phone or tablet, you can now circle a math problem on your screen and get help solving it. Google’s AI won’t solve the problem for you — so it won’t help students cheat on their homework — but it will break it down into steps that should make it easier to complete.
Google Search gets an AI overhaul
Google will roll out “AI Overviews” — formerly known as “Search Generative Experience,” a mouthful — to everyone in the US this week. Now, a “specialized” Gemini model will design and populate results pages with summarized answers from the web (similar to what you see in AI search tools like Perplexity or Arc Search).
Android gets AI-powered scam detection
Using on-device Gemini Nano AI smarts, Google says Android phones will be able to help you avoid scam calls by looking out for red flags, like common scammer conversation patterns, and then popping up real-time warnings like the one above. The company promises to offer more details on the feature later in the year.
Android devices are about to get smarter AI
Google says that Gemini will soon be able to let users ask questions about videos on-screen, and it will answer based on automatic captions. For paid Gemini Advanced users, it can also ingest PDFs and offer information. Those and other multimodal updates for Gemini on Android are coming over the next few months.
Google Chrome is getting an AI assistant
Google announced that it’s adding Gemini Nano, the lightweight version of its Gemini model, to Chrome on desktop. The built-in assistant will use on-device AI to help you generate text for social media posts, product reviews, and more from directly within Google Chrome.
Google upgrades its SynthID AI watermarking
Google says it’s expanding what SynthID can do — the company says it will embed watermarking into content created with its new Veo video generator and that it can now also detect AI-generated videos.
Update, May 14th: Added “Ask Photos.”
Subscribe to our Newsletter
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Ut elit tellus, luctus nec ullamcorper mattis, pulvinar dapibus leo.
Share this post with your friends
![](https://pigtrotters.com/wp-content/uploads/2024/09/google-and-doj-return-for-round-two-of-their-antitrust-fight-this-time-about-ads-300x157.jpg)
Google and DOJ return for round two of their antitrust fight — this time about ads
Google and the Justice Department are set for a rematch of sorts on Monday when they return to court to argue about Google’s alleged monopolistic
![](https://pigtrotters.com/wp-content/uploads/2024/09/telegram-ceo-says-hell-moderate-better-after-getting-arrested-in-france-300x157.jpg)
Telegram CEO says he’ll moderate better after getting arrested in France
❤️ Thanks everyone for your support and love! Last month I got interviewed by police for 4 days after arriving in Paris. I was told I
![](https://pigtrotters.com/wp-content/uploads/2024/09/boost-your-productivity-with-a-powerful-suite-of-ai-powered-tools-from-wondershare-pdfelement-11-300x169.jpg)
Boost your productivity with a powerful suite of AI-powered tools from Wondershare PDFelement 11
We’ve all been there. Staring at a computer screen with a fresh PDF file sent to us via a message or email, and no way
![](https://pigtrotters.com/wp-content/uploads/2024/09/how-the-wayback-machine-is-trying-to-solve-the-webs-growing-linkrot-problem-300x157.jpg)
How the Wayback Machine is trying to solve the web’s growing linkrot problem
We’ve been talking a lot about the future of the web on Decoder and across The Verge lately, and one big problem keeps coming up:
![](https://pigtrotters.com/wp-content/uploads/2024/09/use-whatsapp-on-your-mac-you-should-download-this-major-upgrade-now-before-the-old-one-stops-working-300x200.jpg)
Use WhatsApp on your Mac? You should download this major upgrade now before the old one stops working
If you use WhatsApp on macOS, you may find that you have to redownload the app on your device very soon, as while the older
![](https://pigtrotters.com/wp-content/uploads/2024/08/anandtech-shuts-down-after-27-years-300x157.jpg)
AnandTech shuts down after 27 years
Hardware enthusiast site AnandTech is shutting down after nearly three decades of covering computers. AnandTech’s final editor-in-chief, Ryan Smith, announced the news in a farewell