Hands-On: Gemini Live’s New Video And Screen-Sharing Features Make It Way More Useful

Mishaal Rahman / Android Authority

When Google first unveiled Gemini Live, the more conversational version of its Gemini assistant, it was only able to respond to your voice, making it quite limited in what it could do. In recent updates, Google made Gemini Live more useful by allowing it to answer questions about files, images, and YouTube videos. Now, Google is slowly rolling out yet another update to Gemini Live that allows it to answer questions based on what it sees from your phone’s camera feed or screen. We managed to take it for a spin.

Google announced earlier this month that it would bring some Project Astra capabilities to Gemini Live, starting with live video and screen-sharing. Project Astra is Google’s experimental, next-gen AI assistant that can react to your surroundings in real-time. Google demoed Astra at last year’s I/O, illustrating its capabilities by showing a person using a smartphone and smart glasses to ask questions about their surroundings. Nearly a year later, this is now possible with Gemini Live on Android phones.

Once this update reaches your device, you’ll see a new “share screen with Live” button appear in the Gemini assistant overlay. If you tap this button, the Google app will ask to record your screen so it can share it with Gemini Live. Next, you can ask Gemini any question about what’s on screen, and it will answer to the best of its ability. As usual, you can interrupt Gemini at any time to ask follow-up questions that may or may not be related to what’s on screen. To end the screen-sharing session, just pull down the notifications panel, expand the “sharing your screen with Gemini Live” notification, and tap the “stop sharing” button.

To ask Gemini about objects in the real-world, you can enable the new live video mode. Open the Gemini Live interface and then tap the camera button in the bottom left. This will open a viewfinder in the middle of the screen that shows what Gemini Live is seeing. At any point, you can ask Gemini questions about what your phone’s camera is showing, and it will answer using its general knowledge capabilities. By default, Gemini Live opens the rear camera, but you can have it open the selfie camera instead by tapping the switch button in the bottom right of the viewfinder.

Gemini Live’s new live video and screen-sharing capabilities appeared on my Xiaomi phone the other day, but it seems the features have not yet widely rolled out. Google stated these features would be available to Gemini Advanced subscribers this month, so a wider rollout is likely imminent.

Have you received the new live video and screen-sharing features in Gemini Live? If so, what have you tried them on, and how well did it work? Let us know in the comments below!

Got a tip? Talk to us! Email our staff at [email protected]. You can stay anonymous or get credit for the info, it’s your choice.