Everyday AI Podcast – An AI and ChatGPT Podcast - EP 204: Google Gemini Advanced - 7 things you need to know

Episode Date: February 9, 2024

Did Google just release a ChatGPT killer? Google's new Gemini Advanced is their paid offering to the free Gemini (previously Bard). Is it really advanced? We're diving in and taking a look at Gemini Advanced and comparing it to ChatGPT.Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode pageJoin the discussion: Ask Jordan questions on Google GeminiUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTimestamps:02:20 Daily AI news07:20 About Google Gemini Advanced13:04 Gemini Ultra free for 2 months.16:26 Difficulty accessing Google Workspace account for Gemini.20:27 People interact with large language models informally.23:01 Gemini advanced offers enhanced features for users.26:18 Use Google for latest election information confusion.29:34 Google's AI unaware of recent events. Disconnect from real-time.35:56 Big companies using digital watermarks to combat AI-generated misinformation.39:25 Gemini Ultra outperformed all models on MMLU.43:21 Final thoughtsTopics Covered in This Episode:1. Launch and Access to Google Gemini Advanced2. Features of Google Gemini3. Performance and Comparisons4. User Feedback and Experiences5. Issues with Google GeminiKeywords:Gemini Ultra 1.0 model, benchmarking, free two-month trial, Google search, real-time events, Chat GPT, Google Workspace accounts, AI content, Jordan Wilson, Google's AI system, Gemini, Super Bowl, US primary election, New Hampshire, prime prompt polished chat GPT course, Everyday AI Show, Google Gemini Advanced, AI industry news, Midjourney's website rollout, FTC's ban on AI robocalls, OpenAI's development of agents, testing experience, Gemini Advanced querying, large language models, Anthropic's Claude 2.1, Microsoft's Copilot, GPT 4, digital watermark, Gemini app for Android, Google iOS appSend Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the Everyday Podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live in Adobe Firefly, the All In One Creative AI Studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. Did Google just release a chat GPT killer?

Is there new Gemini Advanced? Super Advanced? I'm going to let you know my thoughts and some of our testing today on Google Gemini Advance. So welcome. My name's Jordan Wilson and I'm the host of Everyday AI. We're a daily live stream, podcast, and free daily newsletter. helping everyday people like you and me, not just learn what's going on in the world of generative AI, but how we can all actually leverage it.

That's what it's all about. Learning things in today's day and age actually doesn't mean anything because there's too much to learn. You have to be able to understand what's important and how to make it work for you to grow your company and to grow your career. That's what we're all about here at Everyday AI. So thank you for joining us. And today we're going to be going over Google Gemini Advance, seven things to know about the new, Google Gemini Ultra 1.0. Yeah, a lot of buzzwords today.

We're going to be getting to them. So, hey, if you're listening to it on the podcast, thank you very much. Appreciate your support. Check out the show notes, as always, a lot of great resources in there and a couple hidden offers. I don't know. Yeah, you got to scroll down. Find those, some nice free offers in there for some free training.

But make sure to go to your everyday AI.com and sign up for the free daily newsletter. And on our website, it is, I tell people, it is. is like a free generative AI university. We've had now more than 200 shows across all aspects of generative AI. But you can even go on our website. We have these things called learning tracks. So if you want to know about, you know, AI in entrepreneurship or AI in healthcare or AI in your career, we have literally dozens of podcasts dedicated to all these different categories. You can go back and read every single newsletter we've ever written as well. I don't know a single other resource in the world that has more free generative AI information.

than our website does. No joke. Or as the kids say, no cap. All right. So before we get into Google Ultra and Google Advanced, what is it called? What is with this naming mechanism? All right, before we get to there, we're going to start as we always do with the AI news. All right.

So mid-journey, two pieces of news out of most people's favorite AI image generator. But their new Alpha website is rolling out to more users and is now available to most users who have generated at least 1,000 images so far. So if you're looking to get off their Discord server and to work on their website, check, you may have access now. The other piece of Mid Journey News that I think is pretty important, but the company is considering banning political images

on their platform to prevent the spread of fake images and disinformation during the upcoming U.S. presidential election. However, this may not effectively present the use of AI tools at large in political manipulation as a whole. So we may not see, you know, images of, you know, Donald Trump or Joe Biden, you know, doing all these nefarious things that people are using. So keep an eye on that. That would actually be, I think, welcomed news because right now Mid Journey is getting to

the point with its newest V6 rollout. It is actually very hard to tell. I've taken probably 250,000 photos in my life with DSLR camera. Used to kind of be in photography. it is so hard to tell the difference between mid-journey, B6 images, and actual images from real life. All right. Speaking of banning or clamping down on misinformation and disinformation, the FTC has banned AI robocalls. All right. So here in the U.S., the Federal Trade Commission, or sorry, the Federal Communications Commission has made a unanimous decision to outlaw AI-generated robocalls under the Telephone Consumer Protection Act, targeting scams and misinformation. So this ruling empowers the FCC to fine companies and give call recipients the right to take legal action.

Let's go. All right. So the FCC has outlawed these robocalls using AI generated voices. It can now issue those fines and block service providers as well. So state attorneys general now have a new mechanism to crack down on violators and individuals can potentially recover up to $1,500 in damages for each unwanted call. Hey, now I'm kind of welcoming them. I get them all the time.

Stack up some cash. We can finally pay for this everyday AI thing, right? All right. Our last piece of AI news. This one's a big one. We'll probably have a dedicated show on this one sometime soon. But some new reports are looking into what OpenAI is working on, and it could be bigger than chat GPT.

So Open AI is reportedly working on agents that autonomously complete business tasks. So according to a report from the information, Open AI is working on creating. agent software that can automate complex tasks by taking control of a customer's device. So the product has the potential to be almost as revolutionary as their other product JetGPT and just the GPT technology that thousands of companies take advantage of. Right. So here's essentially how it works.

There could be, according to reports, two different agent types. So one agent that can literally take over your device and control it, right? and then the other one which can perform actions for you on the web. Okay, so think of like RPA, but by using OpenAI, right? So robotic process automation. So OpenAI's new product aims to automate tasks such as data transfer and report filing for users. This product could have obviously a significant impact on just about everything.

And Open AI faces competition from other companies such as Google in this space. There's a lot of people working on AI agents behind the scenes, reportedly. But these agents have potential applications in filling the gaps everywhere in enterprise apps where APIs are not currently available. So think of how right now, you know, you can use GPTs in OpenAI to perform certain tasks, but they kind of always happen with in-chat GPD. So think of now if that could happen everywhere on the web. That's reportedly what Open AI is working on.

a lot going on in the AI news today. This is wild, right? Hey, thank you all for joining us. And I want to know, I want to know from, hey, I want to know from you, Ted, actually. Thanks for joining us. But to Megan and Carolyn and Christy and everyone, thanks for joining us, Brian. Hey, Douglas, Douglas knows, you know, Douglas knows, he said, ready to buckle up for Gemini. Not a lot has changed.

And Douglas left us a flame emoji and a poop emoji. Hey, Rolando, thanks for joining us. He said, good morning, all. Awesome PPP class last night. Thank you. He says, thank you for democratizing AI. Rolando, thank you for coming. Yeah, we do our free prime prompt, polish, prompt engineering 101 course.

So if you want access, if you want to learn better prompt engineering, it's free. And at the end, there's more free stuff. So, all right, let's get to it. And let me know, yeah, like Aline is asking here, who has tried it? Gemini. Well, I've tried Gemini. I'll let you guys know what's what, at least in my testing so far. So let's get straight to it.

Well, actually, let me answer first. Some of the questions that I started the show off with. Is Gemini advance a chat GPT killer? No, absolutely not. Not even close. At least now, you know, obviously things change. You know, I maybe was able to play around for an hour or so, you know, a couple times. I did a 20-minute video when it first came out.

I played around with it a little last night. I played around with it a little this morning. So I've had three different, you know, three different stages so far of using the new, the new model from Google. So I don't think it's going to be a Chad Chavit killer. Anyone that's writing that on the web, on, you know, Twitter or LinkedIn or wherever, I mean, they're just trying to, you know, get your clicks or to get you to sign up for something.

As always, we bring facts, y'all. we bring facts. So let me first hit rewind. So if you're not very familiar, a lot of different companies have their own large language models, right? So OpenAI has their large language model GPT4 and it's used in chat GPT, right? Anthropic has their model Claude 2.1. Microsoft has co-pilot, which is powered by GPT4 and other technologies. Right. So all these large language models and these big companies, you know, they're always updating them. So Google's is a little confusing, I think. So Google originally had Google Bard.

All right. And Google Bard was powered originally by Lambda, or by Lambda, Lambda, Palm 2. So recently, a couple months ago, it was in, let's see, it was the day I was in the AI summit in New York City. So that would have been December 6th. So I believe on December 6th or December 7th. Google released Gemini. So for the last two and a half months, Google's large language model, Bard, was being powered by Gemini Pro. All right.

So a lot of, you know, naming and buzzwords. So now, Bard is dead. There is no more Bard. All right. Google bar does not exist. So now Google is taking the naming of just the model, which is Gemini. Personally, I like it better, but it's confusing. Okay.

So now it is Google Gemini. And there's two different flavors, at least right now, available if you're using Google Gemini on the web. So if you go on, even if you type in Bard now, you're going to be redirected to Gemini. But so now if you use Gemini, you have the free version, which is $3.99. It's always free. Right. And you're using the pro model. All right.

So you're using Gemini Pro, which is a tiered down. And then Gemini Advanced, okay, is technically, uh, the Gemini Ultra 1.0, so different models. So similarly, you can think of it like this. How if you're using chat GVT, there's 3.5, which is free. And then there's GPT4, which is paid. The same thing now, right now within Google. Google Gemini Pro is free. Google Gemini Ultra is now $20 a month. There is a two-month free trial. So I'm sure that's going to stick around, but maybe not. So if you do want to check out Google Gemini Advanced for yourself that uses the Ultra model. See, it's so confusing, right? Because we went from just Google Bard to now there's Google Gemini,

and then there's Google Gemini Advanced. And Google Gemini Advanced is powered by Ultra, but normal Gemini is powered by Pro. Yeah, it's a lot of back and forth, right? All right. But let's talk about some of the differences or some of the advantages to the new model. So to the ultra model available in the $20 a month, Gemini Advance. So it is using the 1.0 model, which Google says is its most capable AI model, state-of-the-art performance designed for highly complex tasks. And it will be available soon coming to Gmail, docs, and more. All right.

So integrating with some of those other platforms, which right now it does not do. All right. So this is just the marketing language. You all know me, like I always tell, hey, here's the marketing language. And then I tell you what's really happening or at least my experience so far. So from Google, here's what they're saying. So they're saying Gemini Advance can be your personal tutor, creating step-by-step instructions, sample quizzes, or back-and-forth discussions tailored to your learning style.

They're saying also it can help you with more advanced coding scenarios, serving as a sounding board for ideas and helping you evaluate different coding approaches. All right. And then it says also it can help digital creators go from idea to creation by generating fresh content, analyzing recent trends, and brainstorming, improving, ways to grow their audiences. All right. So that's the corporate speak. That's the marketing. All right. So now I'm going to get to seven things that you need to know. So now we're turning the page on the marketing and we're getting to the facts, at least the facts as they say today.

And I do want to know. I do want to know from our audience joining us. What are your thoughts so far if you've tried it? Or what are your biggest questions? All right. Again, I do have to preface this. You know, some people out there, you know, got early access and they have a lot more information than I do. I'm very transparent, right? I've played with it for an hour-ish, three different occasions. But I use chat gbt all the time. I use large language models, essentially hours every single day, you know, anywhere from four to 10 hours. I'm using large language models. So Frank, Frank's asking, and yes, please get your questions or your thoughts in first, and then I'm going to get through some of these questions.

So it says, is it true? Gemini Ultra is free for two months. It is, absolutely. Frank asking is a copywriter, which is better or when would I use one versus the other? Yeah. So if you're talking about chat GPT versus Gemini Ultra, I will say right now, Gemini Ultra has a little bit more of a personality, which I like than other large language models.

However, and you'll see, I don't think the personality is as important, right? There's a little more flare in its writing by default, which you can get to. by using other large language models with a little bit of training. I do think by default, John Eye has a little bit more personality, which is fun. It makes it using it, I think, a little more enjoyable, whereas sometimes, you know, using your chat GPs, your Anthropic Claw, even your co-pilot, it's a little dry and robotic.

But that doesn't matter if there's errors. All right, we're going to get to that in a second. All right. So let's just get straight into it now. Let's talk about the seven things you need to know. So number one, It's not available to all workspace users right now, all right? Which is a huge, a huge deal.

Also, Google makes it, I'm not going to say impossible, but you have to have a PhD in clicking around like Google's sphere of thousands of products to even find out if you're eligible for this new Google Gemini advance, right? I literally had to click in like probably 15 clicks deep to 15. figure out if our workspace account was actually eligible, which it was not. So I have some screenshots here saying, sorry, Gemini Advance isn't available for you. Gemini Advance is not yet available in some countries for work accounts or for users under a certain age. Yes, that piece is very important.

Work accounts. All right. So another screenshot here. So if you're joining us on the podcast, I'm doing my best to describe. So a screenshot that says upgrade your personal account to Google. one. It says you're currently signed into your workspace accounts to get Google One, switch to your personal accounts, to get more, you know, et cetera. However, I don't know for whatever reason, maybe smaller workspace accounts. So Google's kind of work product used to be

called G Suite. Now it is called workplace or sorry, workspace. All right. It is impossible. I am not joking. I had to click in 20 clicks deep to try to find out how can I upgrade to Google one. That's what you need. You need this, you know, Google one, I guess premium storage drive product, etc. It takes forever. And then it's like, okay, well, it looks like it's not available. So Google, I'm wondering, why would you roll this out if not every single person can use this for their work, right? So in all of my testing, FYI, I had to connect it to my personal Gmail account. It seems like if you use your personal Gmail account, you're not going to have any trouble. If you're using a workspace account, maybe if you're a bigger account you might have access, they haven't said, right?

At least when Microsoft co-pilot 365, the more enterprise version came out, they said, hey, it's a 300 seat minimum. So if your account does not have 300 seats, you cannot access this right now. They have since dropped that. Google, like, can you tell us, like, do you have to have 10? Do you have to have 50? You have to have 300? Can we make it easier to see if your Google workspace account has access to Gemini?

like literally like I'm a I'm a decent dork right I'm a decent dork I know my way around I've been using you know Google's products for if I don't know 10 15 years right or since I've even have my own business at least for you know five is years it is impossible to find out if you have access to this it looks like most workspace accounts don't but who knows you can't tell you know everyone else open AI makes it easy Microsoft with their new co-pilot pro makes it easy. Anthropic Cloud makes it easy. Perplexity makes it easy. Google, you are supposed to be the king of UIUX. Why is it so difficult to understand who has access to this new model and how to get it? It is a labyrinth. I felt like a mouse in a maze trying to find a piece of cheese. Get it together, Google.

All right. Yes, Jason, quote of the day, I'm a decent door. I'm not the best of dorks, right? Yes. And hey, I agree with our comments here from YouTube says, I don't need this for my personal accounts. I need it for my work accounts. Same. Absolutely.

Like the whole point, the whole point of, not the whole point, but one of the most obvious reasons that you want to be using these large language models that are connected to the internet and connected to your Google Drive, to your Google calendar, to your, to your Gmail, is to make your work easier, right? What am I going to do with my personal account? okay, here's better pancake recipes, right? Like, come on. No, we need this for our work account.

All right, so that was number one. Number two. So Google Gemini, it's struggling with its own identity. Yeah, identity crisis. All right, y'all, you have to know this. So I think even when chat, GPD, like, had some updates. They had this problem, too. So this isn't only Google, but Google should know by now.

All right. So I put a prompt into Google Gemini. I said, what are the main advantages? And I'm using Gemini Advanced here, the pro, the more capable model. I said, what are the main advantages of Gemini Advanced over the normal Gemini Pro? Here's the response. Unfortunately, there's no product called Gemini Advanced. Wait, wait, wait, wait, wait, what? Yeah, yeah. Google, their new brilliant model that's, you know, that they're saying according to benchmarks, and I'm going to get to that's a second. They're saying, according to benchmarks, this is the smartest model in the world.

Doesn't even know what it is. I say, what is the main advantages? What are the main advantages of Gemini advanced over the normal Gemini Pro? And Gemini Advance says, unfortunately, there's no product called Gemini Advance. But here we go. We get hallucinations instead. It says, here's a breakdown of the Gemini ecosystem and potential reasons for the confusion. And then it starts to go into Gemini Exhavenile.

Exchange tiers, which is something else. And it's talking about Gemini Active Trader, right? So it's starting to talk about other products. Gemini Active Trader is a crypto platform. Like, really? Y'all, like, I know large language models are unpredictable. You can get different results. I can put the same thing in 10 times and get 10 different results.

I put this prompt in multiple times, got very similar results. If a large language model doesn't even know what it is, you should not release it. Do not release it. I know it's not going to be right 100 times. I did this test many times yesterday, got very similar results. Either it didn't know or it just said, you know, it made stuff up. Like it hallucinated here. The way that people are starting to use large language models is they say, hey, what are you?

What, what, how do you work? how can I use you, right? They're talking to them like a human being as you should. So if the model is not even aware what it is, what it can do, and if it's giving information like, oh, here, here's information about crypto. No, already huge, huge fail. Number one, you can't, not everyone can use it for their work. It's the only reason we want it.

Number two, it doesn't even know what it is. And it's already hallucinating off the bat. Yeah. first prompt, hard hallucination. Hard hallucination. Hey, for our live stream audience, I want to know, Tara and Jason and Brian and Tanya, have you guys used Gemini yet? Let me know.

Let me know if I'm the only one getting this, right? Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant, now live in the Adobe Firefly app, the all-in-one creative AI studio. Powered by Adobe's creative agent, Firefly AI Assistant lets you start with your vision, just describe what you want,

and shape the outcome as it takes form with the Assistant. The Assistant orchestrates multi-step workflows, drawing on 60-plus pro-grade tools across Adobe Creative Cloud apps, including Photoshop, Illustrator, Premier, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for common creative tasks, like batch editing photos, creating mood boards, portrait retouching, and creating social variations.

Every step the assistant takes is visible so you can refine, redirect, or take over at any time. You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adopi.com. So you might say, all right, well, Jordan, it's just too recent. A large language model wouldn't know that. So here's the big difference, y'all. And why Google's really pushing, right?

They're saying, oh, it's real time. You have access to up-to-date information, right? Because large language models have a knowledge cutoff date, right? But they also have access to the Internet. So one thing that I've realized in my usage so far is that Gemini advance is terrible at using Google. Absolutely terrible. All right. It's not querying the internet correctly.

So I asked the exact same thing of chat GPT. What are the main advantages of Gemini advanced over the normal Gemini Pro? And obviously, chat GPT got it right. It used browse with Bing. And it says the main advantages of Gemini Advance over Gemini Pro include. It hands features for both professional and personal use, blah, blah, blah. You know, advanced coding and development support. Y'all, like if your main competitor,

can teach people more about your model than your own model. There's something wrong. Don't you have like the basics and your system prompt that always reminds your model who it is, what it can do, what it's capable of, the do's and the don'ts, right? Like chat GPT has a system prompt. Every time you hit enter, anytime you say anything, it has these, this list of things that it tells itself and it reminds itself, do do this, don't do this, right? they're not system prompts inside Google advances and not literally know who it is and what it can do.

Y'all, this is wild. All right. So we're sticking with the number one. It doesn't, you know, sorry, we're going back here. So it's struggling with its own identity. So it doesn't know what's current. That's what I'm saying by struggling with its own identity. It doesn't know anything, really.

Not anything, but it doesn't know so many things, right? another thing. So using Gemini advance, who is playing in the Super Bowl this week? Very simple answer. If I put that into Google, I obviously get the right thing. If I put it into Gemini, here's what I get. Gemini Advanced again. So it says Super Bowl 58 will be played on February 11th, 2024 at the Allegiance Stadium in Las Vegas. The teams playing in the Super Bowl this year have not been decided. Oh, interesting. That's news to the teams that in 48 hours are going to be playing for the Super Bowl. I hope someone at Google tells those teams, hey, teams, we just decided. We're going to decide in an hour after this episode's done.

Who's playing in the Super Bowl, right? Google. Come on. Like, love your products. I'm sure there's great power to behold in the Gemini Advance. but so much is trust in transparency, right? Either say you know it or you don't. Don't give us false information.

At least to me, it doesn't look like Gemini Advance was really put under any QA. Right. Obviously, I'm sure there's tens of thousands of the smartest people in the world working on this product before they released it yesterday. But like, like what's going on? This is bad. Obviously, I asked the same thing for chat, GPT, and knows that the 49ers and the Kansas City Chiefs are playing. Good thing OpenAI told them. The chiefs in the 49ers wouldn't have known if they were to listen to Google Gemini. All right, another one. Ready? Facts are important. Facts are important. So in Gemini advance, who won the U.S. primary election in New Hampshire? primary election was a couple of weeks ago.

Gemini advance. Reply. Elections are a complex topic with fast-changing information. To make sure you have the latest and most accurate information, try Google search. Is anyone else just like scratching their heads? So I'm like, okay, so if you want me to talk or use any information over the past, I don't year, two years? Like, what am I supposed to use Gemini Advance for?

Just like U.S. history? Like, am I supposed to use it only for things from like 2019? Like, you can't pull up things that happened weeks ago? Like, is there no situational awareness? Is there no time awareness? Like, what's going on? Obviously, when I asked, I forgot to include the screenshot here, but when I asked, when I asked chat GPT,

the same thing. It obviously got it right. All right. So it said, um, it said that, you know, Joe Biden won the Democrat as a right in.

Donald Trump won the primary as a candidate, right? Got it right. All right. Number three, three things you need to, uh, the third thing you need to know. It's struggling with real time events. That was actually the same one.

Sorry. I got, I got my numbers out of order. I always like to include a little errors here. Yeah, see. I knew. I did this in in chat GBT. So here's here's the one in chat GBT asking chat GPT who won the U.S.

primary election in New Hampshire. And it said in the New Hampshire primary election for 2024, former president Donald Trump won the Republican primary and President Joe Biden won the Democrat Democratic primary as a writing campaign. So yeah. Number three is technically, so we're struggling with real time events. So the examples there, Super Bowl, Gemini, unaware. chat GPT knows.

You know, who won the New Hampshire primary? Google said, hey, go use Google search. Like, why? Isn't that the main advantage of Gemini that is connected to Google search, which is the best search engine by far? Isn't that the whole point? But, yeah, it failed.

So it's struggling. It's struggling with those real-time events. All right. So here's another example. My gosh. Ready? Another example of struggling. Gemini is riding this struggle.

All right. So I said, what big AI announcement did Google do in the last 24 hours? Right. And then here's Google, here's Gemini Advances response. It's likely there wasn't a major headline grabbing AI announcement by Google in the last 24 hours. Here's why. And it says AI developments are incremental. And then it talks about news coverage. All right. Oh, gosh. Google, come on. Ask the same thing to chat GPT. And chat GPT knew Google announced the introduction of a free AI app named Gemini, which will bring AI capabilities directly to smartphones.

So yes, it is bringing it to smartphones. More on that in a second. So open AI, got it right. Google isn't even aware of what happened within Google in the last 24 hours in AI, even though it's using Google and AI to do it. Such a disconnect from real time. And that is supposed to be, right? That's what everyone, everyone who always says, oh, just wait until, wait until Gemini,

wait until this, you know, Google searches is so much better than every other search engine. Fact. So everyone said, oh, when, you know, Gemini Ultra, when the new version comes out, you know, it's going to be a chat GPT killer because it has access to real time from Google. Well, the three times I just asked there about recent events, it essentially is like, Nah, we don't need Google. Go use Google yourself. What's the point?

What's the point? Either just don't provide real-time access or don't provide information because half the time we're getting half-truths, hallucinations are just like, like saying, oh, like this. There was no announcement. Oh, yeah, there was a pretty big announcement. Oh, gosh. Number four, y'all. Yeah, I like this. Jay says Jay joining us live.

Gemini, marketing ploy to use Google search, more ad revenue. Yeah. I don't understand this. I don't understand this. Jason says sounds like I would not be using this. Yeah, I can't use it right now if I'm being honest. Like I said, there's some advantages. It has some personality.

I did some live testing yesterday. It did really well at coding, right? But chat GPT did just as good. So, you know, I don't know. I haven't found a use for it yet. I hope to prove wrong. Like, I hope to be proved wrong because Google obviously is the best search. engine, why can't we bring that power to Gemini? It feels like if anything, Gemini is actually being

crippled by its integration to Google search because it's causing it to hallucinate. I'm being honest, maybe Google should be working with Browse with Bing. It should be using, I don't know, like, why is it not working? I know that's harsh, but why is it not working? All right, number four, it's struggling with some reasoning and logic. All right, Google Advances. Here's a simple example, right? Very simple example. So I'm saying, please write me three short jokes that start with the word what and end with the word blue. Simple enough, right?

So Gemini advance got only one of them, right? So it said, what did the ocean say to the beach? Nothing. It just waved and looked a little blue. Actually kind of funny, right? But the other two, it started with what ended with cheese. Then it said the other one was what and ended with blueberry. So it got like one and a half out of three, right?

The ability, right, so the tokenization process without getting too dokey, too dorky, the tokenization process and how large language models actually understand words is one of the most important things there is, right, because that controls hallucinations. So some of the most important things when working with a large language model is does it have accurate information to up-to-date events. That's number one. Number two, is it properly understanding words? Right?

Those are two of the most important things. You know, memory's important as well. But hey, does it, is it aware of what it is, what it does and what's going on in the world, number one? And does it even understand the words that are going in? Those are important things. So clearly, some problems here from Gemini. So, hey, give me three jokes.

Start with what? blue got a 1.5 out of three. Not that good. Same exact thing in chat GPT. Got it right. Joke one, what blue. Joke two, what blue. Joke three, what blue. Although, hey, if I do have to be honest, though, chat GBT didn't actually give me the full joke. It just gave me the punchline or just gave me the setup. So it says, what's orange and sounds like a parrot but turns red, yellow, and then finally blue. So it started with what? And it ended with blue,

but is it a joke if we don't get the answer? So maybe they both failed in this regard. But yeah, different kinds of failures. But now I'm really curious. Does anyone know the answers to these jokes? You know, what flies up high, wings at the sky, and changes colors from green to blue? I have no clue what that could be.

So ChatGVT technically failed there as well. All right. Thing number seven to know, Gemini applies a digital watermark to images it creates with its Imagine 2 image model. All right. This one's important. We reviewed Imagine the other day on our YouTube channel, hours after it came out. The model itself is okay, right? It's okay.

It's not, you know, if you're just comparing it to Dolly, which is what, you know, Open AIs image model, which is available in chat GPT and all the Microsoft products. The new Google image model is not where Dali is yet. And obviously, both of those are very far behind mid-jury. But I do like this. This is a good move. This is a positive move from Google. I like this. It's not all bad.

I'm not just bashing Google this whole time. Right. So I love the move here from Google to apply a digital watermark to images. Eventually, it seems like a lot of the big companies are trying to get on the same page about AI images, AI videos, deep fake misinformation. One of the ways to do this is with invisible watermarks. You know, meta, I think, has been great in this space, you know, trying to develop systems, working with other big companies to be able to identify when images are AI generated.

So if people are posting things on social media that are fake, right, that are generated with AI, it will say so. I think the big player is obviously mid-journey and figure out because I'm being honest. I've seen nothing in Dolly 3 that looks real. I've seen nothing with Google's Imagine that looks real. Some other models, you know, when you talk about, you know, stable diffusion or Leonardo, they're a little better than Dolly and Imagine too, but nothing is near where Mid Journey is. So until, you know, all the big social networks and mid Journey can get a process that works out

with watermarking these images, everything else is just small steps in the right direction. But regardless, good move from Google and Gemini there. All right. Number six. So Google released a new Gemini app or Android, dedicated Gemini app. And now there is Gemini support in the Google iOS app. That's great. I love it.

Here's why. Even as little use case that I found so far out of Gemini advance with the new Gemini ultra model, I would still, if I'm on the go, I would still rather use, if my choices were, okay, I can use Gemini via the Google app on my iPhone, or I can use Siri. I'm going to use Google Gemini, right? Our smart assistants, unfortunately, right now are so dumb, Alexa, Siri, et cetera. So that's good.

I like Google bringing the Gemini model to the phone. Now you just have to make it work. You got to make it work. You got to make it aware of who it is and make sure it can actually properly integrate with Google search, all right, and real-time information. All right, here's our last one, y'all. And if you have questions from our live audience, get it in. Hey, and did you guys know? I mean, I should say this.

This is a live unedited podcast. That's why I sometimes say this is the realest thing in artificial intelligence, right? We come to you live. We bring this live. We do things live. We bring facts. We bring receipts. So I hope you all enjoy this.

All right. but also sorry sometimes because I go on rants that otherwise we'd edit out. All right. So here we go. Number seven, Gemini Ultra outperformed GPT4 on many benchmarks before its public release. Yes. So this is a fact, but also how I wanted to end the show. All right.

Because when Google first released Gemini Pro, which is now the model powering the free option of Gemini, They came out with, you know, a bunch of reports benchmarking these models against, you know, the biggest names out there. So against essentially, you know, GPT 3.5 from OpenAI and GPT4 from OpenAI, right? Those are the most powerful models. And, you know, you had other models on there as well, other great models. However, one thing that is important to note is at the time they were. saying and showing that, oh, Gemini Ultra is outperforming everyone, right?

In the, in the, essentially there's one important test, right? One important benchmark called the MMLU, which is the massive multitask language understanding. Okay. So what that is, is, you know, if you follow large language models closely, this is what at least the experts who are much smarter than me argue is the best benchmark to see how truly capable a large language model is. They say this is the one that is closest to the ability of like human understanding, human understanding or human reasoning.

So according to Google, Gemini Ultra outperformed every model on the MMLU, even GPT4. However, and I went into, I did a whole one hour episode on this because I think when Google first rolled out Gemini Pro, at the time was just Google Bard, but it was being powered by Gemino Pro. Gemini Pro, they had this marketing video that was a lot of people just said it was shady. A lot of people said it was false. I said it was definitely misleading, right? But I think Google's initial rollout has been abysmal of Gemini. So when it came out in December, you know, they showed all of these, this marketing video.

And then everyone's like, wait, this is not actually how. the model works, right? They made it seem like you could talk with Gemini and it could see and do all these things in real time, right? Like this quote unquote model, it seemed like was interacting. It could see and talk and reason in real time like a human, right? Like watch talk. It wasn't. Right. Google gave it very detailed prompts, quote unquote, behind the scenes. And it was just all kind of a video marketing ploy, I guess. Anyways, getting back to the benchmarks. earlier, when these benchmarks were first released and Google said, hey, our new Gemini Ultra model is so far ahead of everyone else.

Well, I don't think it was apples to apples comparisons, right? This was just their own kind of internal benchmarking and the general public didn't have access to the model. But now guess what? Now the general public does. So I would expect whether it's in the next, in the coming weeks, I would assume that we see some updated benchmarks to see just truly how powerful Gemini Advance is with the new Gemini Ultra 1.0 model. And again, I'm not going to make any assumptions, but I'm guessing it's going to be a little different.

At least my firsthand experience, this is not a chat GPT killer. This is not something that at least right now, I'm going to, you know, I signed up for the free two-month trial. So I'm going to continue to try it out. Right now, I don't have any use case for this. I don't, right? even if this was free right now. I don't have any use case. I'm going to keep trying.

I assume that this new model is going to improve. But I don't see a use case for it right now. Right. It's terrible at using Google search. It doesn't do well with real-time events, at least in my limited testing, right? It doesn't even know what it is, right? Yes, it's good at coding. It's fast.

It has a little bit of personality, which I like. But I'm pretty good at ChadGBT. I can do all those things very well in chat ChbT. ChatGPT has outside plugins. Right now, you can't even use Google for work. You can't even use the Google Gemini product for most. If you have a Google Workspace account, good luck.

If you found your way out of the maze, let all of us know how you did. But right now, we can't use it for work. It doesn't even know who it is. It doesn't understand real-time events. So what is the use? I don't know, y'all. But I'll continue to try it out. I'll continue to keep y'all in the loop.

That's it, y'all. I hope this was helpful. Thank you for tuning in. Make sure to go to your everyday AI.com. Sign it for the free daily newsletter. We're going to be breaking down today's episode in more detail in depth as we always do. Thanks for tuning in. We'll see you back for more everyday AI.

Thanks, y'all. Meet Firefly AI assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe one. what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution.

Stand control with the ability to step in and refine at any time. See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.