The Best Ways To Use Google’s AI Studio’s Stream Realtime

AI companies are all scrambling to “Build A Better Mousetrap”, so to speak. They’re trying to craft the AI models that will push humanity forward (and push more people to invest in them). Companies want to craft the perfect AI assistant that will make life easier for them, and most people don’t know about the amazing tool that is Stream Realtime from Google’s AI Studio.

Android has the Gemini app, but what about your computer? Sure, there are tools that bring AI to the PC like the ChatGPT desktop client and Microsoft’s Copilot, but what does Google have? Google has some tools that you can use on your computer, but Stream Realtime in AI Studio is the closest thing that the company has to a full-fledged AI assistant that can see your computer screen.

So, what is this feature and what are the best ways to use this AI tool? Let’s find out!

 What is Stream Realtime in AI Studio?

AI Studio is an entire platform that allows you to build your own AI models based on Gemini. If you’re into building a model, you can get your API key and start crafting it. However, we’re here to use one of this platform’s features.

First things first, go to the official AI Studio site and make sure you’re signed in if you aren’t. Once signed in, you’ll want to open the side panel on the left side of the screen. Click on the Stream Realtime button, and you’ll see the screen pop up with three options. You can either talk to Gemini, much like Gemini Live, turn on your webcam to let Gemini see the world, or let Gemini see your screen. In order to use this feature, you’ll need to make sure that you give your browser permission to access your microphone and camera.

If you want to share your screen, you’ll need to choose the third option and choose which screen you want to share. You can share the current browser window you’re using, but can also click on other windows and other applications. When you’re using this feature, it will tap into your microphone and you’ll be able to have a conversation with it. It’s like having Gemini Live on your computer.

One thing to note is that, at the time of writing this, the tool will use Gemini 2.0 Experimental. So, you should expect it to use more powerful models as time goes on. Another thing to note is that Stream Realtime can view separate windows on your computer, but it can’t view your desktop. It can view programs other than your browser, however.

How Stream Realtime can make life easier

So, you have a powerful AI tool at your fingertips, but what next? Here are some ways that Stream Realtime can help make your life easier.

Navigating your browser

This is a feature that might work better if you’re using Chrome. Gemini has a pretty deep knowledge of Chrome, but you don’t need to use Chrome. We also tested this out with Microsoft Edge. Gemini will be able to see the entire browser window, so it will see all of the tabs you have open along with your bookmarks.

You can ask it questions like “How can I organize my bookmarks?” or “Can I have some help organizing my tabs?” Along with that, you can ask it where certain menus are if you forget or how to access certain parts of the interface. This can also work for other browsers; however, you’ll want to make sure that you’re using a more popular browser like Edge or Firefox.

Your homework buddy

When you’re working on some sort of schoolwork, you can summon the AI Studio and choose the screen-sharing option. It will be able to see what problems you have on your screen. Since it uses your microphone, you’ll be able to have a back-and-forth conversation about the problem you’re trying to solve. The AI will be able to understand what’s going on in the problem, so it will walk you through it.

The thing with modern generative AI, you’ll be able to speak to it in a natural tone, so don’t be afraid to say something like “I’m trying to find out how to calculate this problem, but I forgot some of the first steps I have to take. Help me out, but don’t give me the answer.” Be sure to tell it not to give you the answer if you don’t want it because, in our testing, it would bulldoze through the problem and give you the answer.

There’s something to note; you’ll need to double-check your work. There were a few times when Gemini flat-out got the answers to the questions wrong. It was presented with a problem of 537 + 426, and it answered 1,063. It’s better to ask it to help you get to the right answer rather than write the answer to your question.

It’s not only capable of doing math problems. Gemini has advanced throughout last year to become rather good at problem-solving, so you can expect it to be able to help you with your schoolwork no matter what subject it is. It can be a great tutor to help you get through.

Your co-author

Since Gemini has a live view of your screen, you can use it to help bounce some story ideas off of. You’re having a 1-on-1 conversation with it as though it’s another human being. We don’t advise you to have Gemini generate a story for you. However, you can always use it to augment your writing.

For example, you can ask for some ideas on how to advance your story or what suggestions it has. Again, you can talk with Gemini in a very conversational way, so you can relax and pretend that you’re just chatting with your best friend. While testing this, we created a pretty cheesy fairytale about a mean princess who gets cured by a witch to have skin as cold as ice. She must find someone to fall in love with to break the spell. That story, aside from begging for a lawsuit from Disney, came purely from spit-balling and tossing ideas back and forth.

The beauty of this is that Gemini can see what’s on your screen. Say, you want to pull up a draft of a story. Gemini can look at it and gain some context. Just know that it will only be able to see what’s on the screen. So, if your story is several pages long, it won’t be as effective. However, it will see you typing live, so it will always have the most up-to-date information.

Help with photo editing

Having an AI that can look at your computer screen is (among other things…) helpful when it comes to editing photos. You’re able to ask the AI questions about how you want to edit the image. The AI will give you advice and tell you what sorts of tools you could use.

You can say things like “I want to make this picture more dramatic” for example. Then, it will work with you until you’re satisfied with your image. Since it processes in real-time, it will help you as you edit the photo.

Stream Realtime is a great tool

While Stream Realtime isn’t the kind of assistant that can dig into your computer’s software and perform actions on your behalf, it’s still a great tool to use if you want some assistance with your day-to-day tasks. It gives you a taste of the kind of things Google and other companies want to be able to do with AI.