Gemini In Cars Might Be The Most Impressive AI Integration Yet [Video]

No place is safe from the expansion of Gemini, and while some integrations might raise eyebrows, it’s at home in your car, almost like nowhere else. Here’s just what it’s like to use, and what you can expect when it eventually rolls out.

Voice assistants in cars are nothing new. We’ve had the Google Assistant on Android Auto and Automotive for almost a decade now. While it’s been useful for accessing car controls, it has many limitations and drawbacks that have become exacerbated as the years pass.

From a pure safety perspective, having conversational control is a safer way to navigate, get information, and use the increasingly touch-focused in-car interfaces. The days of dials and knobs are done in modern vehicles, Gemini feels like an obvious way to still keep your eyes on the road while controlling the major head unit options. The thing is, that doesn’t do the system justice. We spent time with both systems to ascertain why this could be a big deal for drivers in the not-too-distant future.

New and improved UI

Because Android Auto and Automotive work differently with your vehicle, there have been some very important revisions in how and why specific layouts and interaction methods are utilized.

Cars that have Android Automotive or “Google Built-in” work slightly differently with Gemini than those that support Android Auto. Firstly, those cars need a big system update. It might come as an OTA release or you may need to go to your local dealership to have them install it for you. Around 50 vehicles on the road right now will be eligible for this. That’s too many to list here, but Google has a page detailing which will get it and how to do so. Android Auto has over 250 million users globally, so this is going to be a big component of the experience very soon, too.

When available, you get a Gemini app on your homescreen. Unlike on your phone, it doesn’t pull up the chatbot interface that you’re probably used to. Instead, it’s just a pop-up text prompt bar that looks similar to the Assistant in cars right now. This is just the voice recognition kicking in and a basic visual to indicate things are working.

Because your interactions or chats get logged to your account, you should be able to view the transcript on your phone, tablet, or desktop right away. So you can continue a conversation later if you get into the weeds with things or want to check the recipes you might have found on the way home. We weren’t able to see this before or after our hands-on demo, but were told it should work exactly like how it does with conversations with Gemini across your devices today. You might also spot the microphone indicator on the top of the central dash, showing you that the microphone is live once Gemini is active.

The pop-up bar is also unobtrusive and doesn’t distract from other on-screen information.

For all intents and purposes, Gemini in cars with Android Auto works the same. However, it has a marginally different interface. The scalability of Android Auto’s dual-column layout and various head unit shapes means it might not be like-for-like from car to car – especially based on head unit orientation.

Depending on the steering column’s positioning or region, the voice recognition prompt bar might be on the left or right side of the screen. It’s still visually similar. The glowing violet pulsating bar replaces the Assistant’s colorful glow either way.

Before I get into the nitty gritty of what it’s like to use and why it could be a genuine gamechanger, Gemini Live is treated very differently in Android Automotive and Android Auto. In the latter, likely because it’s using your phone, it pulls up a mini window with the visuals you are used to seeing when activating on your smartphone. So it works like a phone call UI. There are the pause and end Live session buttons with the glowing animations behind. Minimal is arguably the best way to describe this floating pill.

On Android Automotive vehicles, after using the “let’s talk” command, it’s vastly more subtle. Here, it has a floating pill that indicates that Gemini is conversing with you. Everything is squashed down, and depending on screen size, it’s much less visible in your eyeliner. You still have the same buttons to pause or end a Live session.

That said, the use cases are the same, but the minor differences are essential.

Gemini Live is right at home on the road

As I noted, you can ask individual queries and one-off questions. That’s absolutely fine, and it’s very good at doing so, but it’s Live where the real potential power lies with Gemini in cars.

Thanks to integrations with other Google services and the ability to decipher natural language almost flawlessly, it’s immediately obvious that Gemini Live could be one of the most natural places I have ever seen the tech implemented.

We were shown a fairly on-rails demo, but we asked for more unique options here and there, and Gemini was able to roll with the punches. That itself isn’t all that impressive given how capable most AI services have become in recent years, but the conversational flow means you do not need to be so specific. If you’re a driver who gets distracted by things going on outside of your control, you can understand why this seems like such a big deal.

So when mapping out a journey, you can ask to find a place nearby to get some good gluten-free snacks or ask questions about nearby locations, including getting an overview of recent reviews. You can do that without so much as glancing at your screen. Route-related queries are practically endless in that regard and go way beyond anything we’ve seen from any other voice assistants in vehicles before.

For instance, if you have your Google and Gmail accounts linked, you can even ask for specifics like the location of an event you might have saved here or to your calendar. You can ask things like “where is the best local Indian cuisine?” and get results based on your dietary preferences. That’s just the tip of the iceberg. It doesn’t take much to see how useful this might be to busy parents or those who aren’t always the most organized.

Integrations are going to be key to the Gemini system in cars and vehicles, but what we were shown looks promising even at this stage. Gemini has developed, and it can do way more than it could back at launch which is important because it’s being released out into the wider world as so it can’t be lacking in functions.

Being able to add things to a Google Keep list looks particularly useful, as you can basically do anything with Gemini Live that you can on your phone or tablet right now. Google’s example had the “driver” asking about what recipes they could make with the contents of their fridge and then adding the extras to their shopping list while mapping out directions to the nearest supermarket. This “almost autonomy” smokes the rigid Assistant experience we’ve had for so long.

A particularly fun demo and showcase for the flexibility of Gemini in cars is the ability to have it work out what you want. For example, you can ask for a specific song to be played, but you don’t even need to know the name of it. Asking for “the song played at the start of The O.C.,” and it’ll find it and queue it up on YouTube Music. This isn’t available on your phone right now either, making it even more impressive and useful.

If you didn’t get this right away: Natural language and conversational flow are the killer features of Gemini in your car.

Initial impressions: The most convenient route

I have used both in-car Android builds for a couple of years now, and while initially I loved the integration with the Google Assistant, I am genuinely excited to no longer need to use distinct commands to do certain things. When you’re driving, it’s hard enough staying focused and alert to various on-the-road hazards without then adding a layer of complexity courtesy of a finicky voice assistant that requires very specific or precise phrases to work correctly, all without a hint of flexibility.

The Google Assistant has always been a little too specific in what it requires before it is capable of doing things – which applies to smart speakers too. Gemini looks like a breath of fresh air in that regard, because you can use natural language that often is miles away from the very on-rails commands that the Assistant can process.

We didn’t even see every single ability or usability case. There are bound to be edge cases where things aren’t perfect, but so long as integrations keep progressing, I’m genuinely excited to start conversing with Gemini in my car. It’s passive in all the right ways, and I can’t wait for it to…literally…roll out.

Add 9to5Google to your Google News feed.