Everyday AI Podcast – An AI and ChatGPT Podcast - EP 465: Deep Research Throwdown - Perplexity vs. Google vs. OpenAI

Episode Date: February 19, 2025

Deep Research tools are gonna be the second wave of GenAI. Granted, they'll be a MUCH smaller wave than the ChatGPT moment of 2022. Yet, before AI Agents take off flying, we're going to see a mass adoption to these new Deep Research Tools. But which ones are good? Is Google's the winner? Is OpenAI's in-depth researcher untouchable? Or is Perplexity's free and speedy option the way to go? We'll go in depth on each. Newsletter: Sign up for our free daily newsletterMore on this Episode: Episode PageJoin the discussion: Ask Jordan questions on Deep ResearchUpcoming Episodes: Check out the upcoming Everyday AI Livestream lineupWebsite: YourEverydayAI.comEmail The Show: [e-mail] with Jordan on LinkedInTopics Covered in This Episode:1. Breakdown of Deep Research Tools2. Challenges in Using the Internet for Research3. Tasks Suitable for Deep Research Tools4. Comparison of Deep Research Tools5. Practical Demonstration and Live TestingTimestamps:00:00 Deep Research AI Tool Showdown05:30 OpenAI's Advantage in AI Research07:21 "Disruptive Deep Research Tools"11:49 Essential AI Tools for Research14:20 Adapting Work to Connectivity Constraints17:34 OpenAI's Enhanced O3 Model Overview23:10 Google Deep Research Explained24:05 Web Crawling: Google vs. OpenAI29:28 "LLM Development Interests Clarification"31:14 "Optimizing Deep Research Methodology"35:11 Tool Comparison and Pause Announcement38:06 AI Accuracy and Hallucinations42:27 Misreported Interests: From Music to Marketing46:01 Choosing a Deep Research Service47:49 OpenAI Dominates Deep Research ChoicesKeywords:AI research tools, deep research tools, large language models, perplexity deep research, Google deep research, OpenAI deep research, AI-powered tools, AI news, AI predictions, business use cases, knowledge workers, AI in business, ROI on GenAI, strategies for deep research, Internet research, research synthesis, AI deep dive, professional research, AI hallucinations, SEO and AI, AI-powered market analysis, AI in education, AI-generated reports, AI in competitive analysis, advanced research methodologies, AI and disruption, deep learning models, transformative AI use cases, factual accuracy in AI, AI pricing and access, AI model reasoning.Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) Start Here ▶️Not sure where to start when it comes to AI? Start with our Start Here Series. You can listen to the first drop -- Episode 691 -- or get free access to our Inner Cricle community and all episodes: StartHereSeries.com Also, here's a link to the entire series on a Spotify playlist. 

This is the Everyday AI Show, the everyday podcast where we simplify AI and bring its power to your fingertips. Listen daily for practical advice to boost your career, business, and everyday life. Meet Firefly AI Assistant, now live and Adobe Firefly, the all-in-one creative AI studio. Just describe what you want to create and the assistant handles the rest, orchestrating multi-step workflows across Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome. The assistant accelerates execution. One of the biggest no-brainer use cases for AI right now is deep research tools.

But there's also a problem, as obvious as it is, that we should be using these AI-powered deep research tools. It is equally as confusing over which one to use, because now deep research is just a type of AI tool. and there's already three of the biggest players in the large language model space that have a tool literally called deep research. So today we're going to be doing a somewhat deep dive on deep research in our deep research throwdown, looking at perplexity, Google, and Open AIs version of deep research. All right, I'm excited to go deep. I hope you are too. What's going on, y'all?

my name is Jordan Wilson and I'm the host of Everyday AI. That's what you're listening to. This is going to be, I think, one of those episodes. If you listen to it, if you watch it, you're going to save a lot of time. So if you're listening on the podcast, this might be one of those ones where you check out your show notes, show notes and come watch the video. We're going to have that on our website. That is Your EverydayAI.com.

So on that website, you can sign up for the free daily newsletter. We're going to be recapping today's episode and also in today's newsletter, keep you up to date with everything else that you need to be the smartest person in AI at your company. So, thank you for listening. If you are on the podcast, please make sure to subscribe, follow us, leave us a rating, all that good stuff. We appreciate it. Live stream audience, love to see you here. I'm going to keep reminding you all. This is so relevant. Go on our website. Go listen to episode 443 to 447. It's all free. They're short episodes about 25 minutes. These are our 2025 AI predictions.

and Roadmap series. Trust me, you need to listen to them. All right, we're going to have the AI news for today in the newsletter. If you normally tune in, this is going to be a longer show. I'm actually going to challenge myself to keep it concise because I want to give you all a pretty deep dive into the three different kind of deep research tools. So go check out our daily news on the newsletter. All right, let's get into it.

Perplexity, Google, Open AI. I'm curious, live stream audience, which one of these deep research tools are you all using? So I use them all, right? And we just did a show yesterday, specifically on the perplexity deep research, right? So there's is the newest of the three challengers in this new deep research space. So I'm curious, live stream audience, let me know which one are you using. Are you using them all? Are you just using one?

Are you using two? Which one are you using and which one are you liking? All right. So I'm going to say at the end, which one I'm using the most and which one I'm liking the most. And I think I'm going to use them all for different purposes. All right. So also, if this is helpful at the end, hopefully I've earned your repost. So if you're listening on LinkedIn, please, this is helpful. Repost this. And I'm going to send you our guide that is 10 business use cases for deep research. All right. So like I said, there's three big players right now in the deep research game. And I wouldn't be surprised if by next week, maybe there's another. And I know it's confusing. They're all using the same name deep research. So perplexity is the newest in their deep research, right? Just released within the last couple of days, February. In January, Open AI released their deep research. The month before that, in December, Google released there. So Google was the first company to have a deep research product to market.

Although previously, I talked about this yesterday, it was actually open AI had been tied to this deep research tool in reporting from, I believe, May of last year. So a lot of people are like, oh, you know, Google was first. Everyone else is copying. I don't know about that. I'm not saying open AI was first, right? But open AI had been tied to a deep research tool as of last year. It doesn't matter who was first. I think it matters who is best, not just who is best, but who is best for your company

and your specific use case. So originally when I was planning out this episode, I said, I'm going to do a bunch of live demos, right? But then I thought our biggest audience is actually the podcast audience. And I thought I would be all over the freaking place. If I do multiple, I was going to do a series of three to five. deep researches across these three different platforms, score them up, add them up. So I actually kind of just did that offline right before this.

So I can tell you kind of the results or just, you know, to put everything kind of in a nice chart. And I thought that might be a little more helpful for you all because I know that this is important, right, especially now since I think this is one of the best use cases for getting a positive ROI on Gen A.I. Right. I've said since the very beginning. And this is why I've always been, you know, pretty big on chat GPT and, you know, kind of,

lukewarm on some of the others, right? I think Google has gotten so much better. But the ability to query the Internet is so important. All right. If you're using a large language model, number one, you should never copy and paste anything. Number two, human in the loop is important. And I think the human in the loop responsibility is changing. And I think we're going to see that specifically here with deep research.

But the ability to be connected to the internet. internet is so important because a lot of the information that goes into the training data of these large language models can sometimes be very old, right? So that's why I think OpenAI and chat GPT have been a leader in this space because I think they've provided the best ability, right? So even before they had chat GPT search, which I believe they released in October, they had Browse with Bing, which, you know, when it first came out, was actually kind of bad and there was better GPs that connected to the internet. But Brows with Bing actually became amazing before they replaced it, you know, at least officially with chat GPD search. All right. So let's let's just quickly talk

about what the heck is deep research in general. Because like I said, I just laid out the timeline. So we've had December, Google, January, Open AI, February perplexity. So what the heck is a deep research tool? Well, it does the most basic human research work. And they work in different ways. but essentially these three tools, one thing that they share in common, they go to multiple websites and they try to take your input and they try to answer it in the best way possible.

Oftentimes it's in the form of a long report. The quality of these reports vary. The process in which these three different tools actually do the deep research is much different. Right, but this is something, And this is why I think it's going to change a lot of industries, right? Maybe not two of these tools, but I think one of these tools is going to be incredibly disruptive. Presumably, they're all going to continue to get some good updates.

But this is what so many of us humans do on a day-to-day basis. If you are a knowledge worker, right, especially here in the U.S., that means you get paid to sit in front of a computer and use your brain and create value for a company. so much of what you're probably doing is you're probably doing a lot of research, right? So what does that mean? Well, you're probably going to multiple websites. You're reading, you're synthesizing, sometimes you're summarizing, and you are trying to extract information and use that information to create new business value or to better understand something.

All right? And that's what these deep research tools do, right? Again, this is never something, even though they create these. really long reports and they're thorough. A lot of people just think, oh, now I can just, you know, copy and paste these and use these. That's not what I think they're necessarily good for. I think these are to help us research better, to help us come to a point where we can

use our strategic minds as humans, our creative minds as human, as humans a little faster, right? So this is where like, and this hits me, right? Because I used to, I worked at a nonprofit for 10 years. we essentially just became a marketing and activation agency for Nike and Jordan Brand. And I spent so much of my time. I would say I spent probably at least 60% of my time most of those years doing a combination of internet research, grabbing information from dozens of websites and compiling it into

reports, right? Whether these were reports to send to a partner like Nike and Jordan Brand or maybe it was to work on an RFP, right? We had to gather all this information, this data, this statistical trends forecasting for a proposal, right? But this is something I've spent, honestly, thousands of hours of my professional career, right? I was previously a journalist, you know,

even in the earlier days of the internet, right? This is what I would do. If I was working on a big story, I would do as much internet research as I possibly could. And, you know, let's be real here. We have to talk about SEO because it's terrible. Using the internet is terrible. You know, not only is the internet made to distract you, but it is just distracting.

And because of AI in large language models, the internet has become more distracting. Adobe just introduced an entirely new way to create, bringing the power and precision of its creative suite into one conversational experience. Meet Firefly AI Assistant, now live in the Adobe Firefly app, the all-in-one creative AI studio. Powered by Adobe's Creative Agent, Firefly AI Assistant lets you start with your vision, just describe what you want, and shape the outcome as it takes form with the Assistant. The Assistant orchestrates multi-step workflows, drawing on 60-plus pro-grade tools across Adobe Creative Cloud apps, including Photoshop, Illustrator, Premiere, Lightroom Express, and more to help bring your ideas to life. You can also get started with creative skills, a growing library of pre-built workflows for

common creative tasks, like batch editing photos, creating mood boards, portrait retouching, and creating social variations. Every step the assistant takes is visible so you can refine, redirect, or take over at any time. You stay in the driver's seat as the creative director. Adobe Firefly AI assistant now in public beta. See it today at firefly.adobie.com. Right. When was the last time you went through an entire workday? Be honest with yourself. When was the last time you went through an entire workday? You researched a bunch of in-depth topics, stayed on track, didn't go down any rabbit

holes, didn't accidentally have 882 tabs open, right? Doesn't happen. The internet is made to be distracting and it distracts us, right? And there's just a bunch of garbage out there now, right? Because it's a lot of SEO stuff, right? So a lot of these companies have figured something out. And they have a blog post from 10 years ago and they just update it once every year. And it actually might be bad and not very good information.

Right. But sometimes they trick the algorithm, right? You trick a Google algorithm into thinking that you're an authoritative source on something when it's actually, you're not. And that's what a lot of times as a human when you're researching something, that's what you're trying to do. You're trying to look at all these web pages and you're like, all right, what's real? What's not? What's good? What's garbage?

right? And that's kind of the initial or the 10% promise of these tools. So who should use them? Well, who shouldn't? Right. Let's be honest. Like I said, if you are an everyday professional, right, if you are a knowledge worker, especially here in the U.S. where, you know, it's the Wild West and there's no real rules on AI, you need to be using these tools, whether it's perplexity, deep research, Google deep research, Open AI deep research, you need to be using them. So a lot of people are thinking, oh, these are great for students. It is. It's great for students to learn things.

It's great for researchers. It's great if you are working in a highly technical field as well that would generally require you to look at many, many, many long documents, right? But if I'm being honest, when I think who should use deep research, I see more use cases for who should use it versus who shouldn't. I honestly don't know anyone that sits in front of a computer all day, uses the internet, that shouldn't be using one of these three tools. They are, I think, one of the biggest places of impact that will see AI have in 2025.

Personally, I think the management consultant industry, if they're not using like this as their starting point, they're screwed, right? or they're just going to be over billing their clients and those clients are going to end up going with someone else that, you know, is essentially honest, right? Hey, we used to bill you, you know, for 50 hours of research. Now we can do it in five, right? Or two, right? So I do think this is really going to change, especially some of those high price services that require just such an, like such a high level understanding to be able to absorb and make use and synthesize, you know, hundreds or

thousands of pages of data. That's what these tools do. So what types of tasks are they best for? So number one, learning new topics. It's a great thing to do, right? Instead of, like I said, getting lost. Because not only do you have to go to all of those web pages, right, five, 10, 20, 30 web pages to learn this new topic. You also have to be able to synthesize it. You have to be able to connect the dots between the different points and these different points might happen across different points on your journey of trying to learn it. Right. So even even the general way of doing this, using a deep research tool is so much better because

not only does it, you know, give you points A, B, and C, but it builds the bridge and it segues and it creates a narrative that helps explain those missing points, right? Because otherwise, again, that's just something you just have to hope that there's a resource out there, one all-encompassing resource that does it all. But there's usually not, right? Because everyone's learning experiences is different. Everyone needs something different out of their research, right? So learning new topics, number one, great use case. Market analysis, obviously doing competitive research, competitive analysis, right? That's huge. Keeping up with recent news and trends and how that impacts your type of work.

problem solving and discovery. I mean, think, think like any, like I always, like I always think it's like, okay, if you're on an airplane, right, and maybe there's not going to be good Wi-Fi, you have a type of work that you can do, right? Deep writing, you know, deep reading, right? And then it's like, oh, okay, well, hey, when I get back to Wi-Fi, I need to go research these things, right? There's almost that, that list of things that you're like, oh, man, I got to go do that.

That's what these deep research tools are great at. All right. So podcast audience, this is one of those. You might just want to just check out the little screen that I have here. All right. So just this is from, I've been using, right, all these tools since the day they came out, literally. Since the day they came out, I've been using Google deep research, perplexity deep research, and open AI deep research.

So perplexity is the newest one, but I've used it for many hours already, even though it's only been out a couple of days. Same thing. Open AI deep research, I use this extensively, extensively. Luckily, I haven't hit my 100 a month limit, although I think I'm almost there. In Google deep research, I'm using it nonstop. So these are all three tools that I use a lot, and I've been kind of keeping track on how they all work and kind of how they're built, because that's important. So now let's start the differentiation process across these three different tools. All right. So I have them tiered off.

So free users, that's number one. So right now, Open AI, nope, no free users, although in the future you'll get two queries a month, a month. Yikes. All right. Anyways, yeah, right now, nothing free for Open AI deep research or Google deep research. But with perplexity deep research, free users. have five queries a day. So automatically, there's a big plus in the column for perplexity. So here, let's look at our normal paid plan, what you have to pay to use these different

deep research tools. So right now, the only tier it's available on for Open AI deep research is the pro plan. That's $200 a month for 100 deep research queries. All right. Again, Open A.I. I did say that they'll be rolling this out to the normal chat GPT Plus plan. And I believe that will be 10 queries a month. But that's not out yet. Right now, only on that $200 a month pro plan. All right. So for perplexity, if you're a paid user, it's unlimited usage.

I think they do a cap at like 500 a day, right? But no one's getting that. So it's essentially unlimited usage if you are on a paid perplexity account. Same thing for Google Deep Research. It's essentially unlimited, although you can only run two queries at a time. I haven't run into any maximum number on the others. All right, model used. This is important.

So I'm actually going to layer these two together. Model use and reasoning because Open AI uses the most powerful model on this list, which is Open AIs 03. So from what OpenAI said, this is not the same 03 that we have access to in chat GBT. This is a fine-tuned version of the full version of 03. All right now, even if you're on the chat GPT pro plan, you only have access right now to 03 mini and 03 mini high. So this is a fine-tuned version of the full version of 03, which is according to all Benchety, It's scary, good, right?

So it's a reasoning model. Perplexity, deep research, they have a reasoning model as well that powers this. However, they did not disclose what the model is. So hopefully they do pretty soon, but there's no information on what reasoning model they're actually using, right? Are they using R1 from Deepseek? Are they using O3 Mini? are they using a reasoning version of their sonar? I don't know.

No one knows. They didn't release it. So not a huge fan of that. And then with Google Deep Research, they're using Gemini 1.5 Pro, which is not a reasoning model. So again, we talk about this on the show a lot.

Right now, there's kind of two different families of models, right? And they actually might all be merging in the future. So we'll have a story on that sometime soon. But essentially, you have your quote unquote, old school LLM, transformer models, and then you have your reasoner model. So your reasoning models, they kind of show this step-by-step logic, right? It kind of reasons under the hood.

So that's why, you know, I think we'll see some examples here. It's two different approaches, right? Because I think just because Google deep research isn't powered by a reasoning model doesn't make it less powerful necessarily for some people, depending on your use case, because they take a different, approach, which I kind of like. All right. So file uploads. Open AI. Yes. Perplexity. Yes. Google Deep research. No. So why does that matter? Well, maybe you have a huge report, right? And you want to do some research, right? Maybe this is a report from fourth quarter of 2024. Maybe you need to do a bunch of

research and find all the new laws, regulations, competitors, everything they're doing related to this big report. Well, you can just upload it. Say, hey, this is the report from quarter four, 24, you know, do all research on any new laws, regulations, competitor movement in anything that this report touches as an example, right? Boom, done. So big advantage there for open AI and perplexity. No go right now for Google deep research. Again, this is all as of today, right?

This could all change tomorrow next week. Follow-up questions. Okay. So this is the model asking you follow-up questions. Not yet. You obviously, once deep research is done, you can keep talking to that chat and get more out of it, refine it, et cetera. But this is the mode, does the deep research mode ask you questions? And you might be saying, like, why does that matter? Well, some of these take many minutes, right? So you might put in a prompt, and let's be honest, y'all, the weakest link of artificial intelligence is the human in the loop. The human is always the one that doesn't do a good enough job.

Because if you do a good enough job with AI, let's be honest, it's way smarter than any human out there right now. You can't argue with benchmarks and science. You can't. All right. I don't care who you are. You're not smarter than in 03 deep research. You're not, period, right? At least not across multiple categories.

You're not. So it's important to have that opportunity to clarify and refine the query before one of these deep research models goes and spends five, 10, 30 minutes on it. So that's extremely important. So Open AI does ask a full follow-up. Perplexity does absolutely nothing. And Google deep research essentially outlines its plan ahead of time and then has you approve the plan. Okay. So different. Let's talk about sources use. and this is where I think Google Deep Research goes in a little different path because this is more of the research methodology. So for the most part, both open AI in perplexity use this reasoning and logic.

So what that means is they might have a certain path set, right? But they might start out on that path. Let's just say, hey, we're going to go to the, you know, this is going to be a 10-step research plan. Let's just say. And then they get to step three and they see something that they didn't anticipate or they didn't know. So open AI deep research and perplexity deep research because they use reasoning step by step between all of these steps of research, they can pivot, which is huge. That is huge, right?

But what that means, and that's what you get when it's a reasoning model, what that means is it can take fewer sources, right? Because they're making hopefully educated decisions, and they usually, start first with a broad or high-level source, where Google Deep Research takes a completely different approach. Google Deep Research, because it is not powered by a reasoning model, so presumably it doesn't cost them as much, and Google essentially serves you up cached versions of these pages. So for Open AI deep research, I've seen it usually go between 15 to 30 sources. perplexity deep research varies wildly.

I'd say like 20 to 200. And then Google Deep Research, I'll say 50 to 500, right? I've had Google Deep Research go to like $1,300 before. But it uses cached pages. So as far as I know, and Google friends, I know there's some of you out there listening from everything I understand and people I've talked to, it essentially is not visiting, quote, visiting these web pages in real time per se. It's working with a cached, right?

Or Google essentially crawls websites usually multiple times a day. So it essentially just synthesizes all of that information, whereas Open AI is actually visiting the pages. So it takes way longer. But Google's is still takes longer as well. Because when you look at speed, Open AI deep research is five to 30 minutes. Perplexity is one to three. Plexity is crazy fast.

And it does go. So perplexity in terms of like pages per minute, I would say it's usually the most, right? But quality, you got to talk about that here in a second, right? So sources use Google's the most time. Open AI deep research is the slowest. Perplexity is the fastest. Google deep research is in the middle. Then what does it give you, right, an output?

So all the outputs, it's essentially a combination of, you know, intros, breaking it down by sections, and essentially gives you a pretty long report. I've seen in different instances where, you know, it'll usually give you a comparison chart. If you're asking for comparisons on something, which is really nice. So it'll put together a very comprehensive, helpful, insightful report for you to use and even give you information that you might not have even thought of, right? Like breaking things down in a chart chronologically, competitors, etc. etc. So even it's going to usually go a little above and beyond what you think it may deliver.

So the output, the number of words. So open AI about 1,000 to 2,000 words. Again, it varies. I've obviously had, you know, plenty that are less than 1,000, plenty that are more than 2,000, right? But I'd say for the most part, you're in that 1,000 to 2,000 words. Perplexity, pretty short, 400 to 800. Again, I've had plenty come in at more than 800, but it seems most. are in that range. And then Google Deep Research are generally the longest, I'd say about 1,500 to 3,000 words. But this is where it's important. All right. We have to talk about hallucinations because that's what this is ultimately all about. Because what we're doing when we are using a deep research tool, whether you are using it for personal purposes or you're using it for business use

cases. You are essentially putting your trust in a deep research AI tool to go research things and tell you the truth, right? And this is one of those instances where accuracy matters. This is one of those instances where it's better to be more factual than first, right? I'm fine with using open AI deep research, even though it's the slowest, because in my experience, it is the most factual. All right. So these are my arbitrary kind of rankings, and I have an actual example that I'm going to share with you all on the screen here. Hallucinations, I'll say open AIT research, very low, very low. I don't see hallucinations a lot.

All right. Google deep research, I'm saying low. Some hallucinations, not a ton. It's very, very, very accurate. Perplexity is perplexing. It is concerning in some of my use cases and some of my examples. How many hallucinations there are, right? So, I mean, you might look at it on paper and be like, oh, wow, perplexity is free.

It's fast. It still uses a bunch of sources. This is amazing. Well, you got to know your stuff. I wouldn't be copying and pasting anything for Plexity, deep research, unless it's something you already have an intimate understanding of. It's concerning. I'm going to show you examples. So the example I'm going to show next, and then we're going to jump in.

We're going to begin a deep research quick. Actually, no, let's just go ahead and, uh, do a deep research query first. We'll start it, and then we'll look at this example of some of the hallucinations or accuracy rate first. All right. So let's go ahead. We're going to do a simple one here.

So I'm saying, what are the latest LLM large language model developments this week? And what will the impact be on knowledge workers in the U.S. and how can businesses take advantage? Okay. So again, I am starting off in OpenAI. This is the one that takes the longest. I do have the pro plan. So I have 100 queries of this.

So to use deep research, it's a little confusing, right? Because you can technically access it in almost any mode or model. And so like right now I'm in chat GPT-O-1 Pro, and I can click the deep research button. That does not mean I'm using O1 Pro. That just means I'm using deep research, which is the fine-tuned version of O3. So it was a little confusing, and I wish Open AI kind of fix this. All right. So anyways, I'm going to go ahead and start.

and what you're going to see here, if I share my screen correctly, there we go. All right. So I started it. Here's my query. And it's asking me questions. It says, could you specify which areas of LLM development you are most interested in? For example, are you looking for updates on new models, advancements, and training techniques, ethical concerns, regulatory changes, or applications in business?

Also, do you want insights on specific industries regarding the impact of knowledge, workers, and business strategy? So I'm going to answer that. I'm going to say mainly interested in updates on new models and their applications in business. And then I'm going to say focus, anything that impacts U.S. knowledge workers. All right. So now once I click enter, then open AI, deep research is like, all right, cool. And then it's going to say starting research.

and then there's this little box here. I'm going to give it a second to propagate here, and I'll be able to click on it. And if I want to, I can kind of watch it work step by step. All right. Let's just go ahead. We're going to jump over into perplexity, deep research, exact same thing. Make sure that you have the deep research.

If you are using perplexity, right, and it is free. So anyone can go run out there right now and try this. Make sure that you have the deep research mode on. And I'm going to click. the enter button. And you'll see right away, it's not going to ask me anything. It's just preparing the research plan on its own. It's running off by itself. Generally, so, okay. And I said this yesterday in my perplexity deep research episode, which, you know, if you are interested in this one in particular, make sure to go go watch or listen to that. One thing you need to be doing is reading the chain of

thought. Not only is it pretty fascinating, but it's going to make you better at using these tools and it's going to ultimately give you better outputs. Also, never, never, ever, ever do a deep research query once, period. Always do it once first. Go look at the chain of thought, then look at the output and then improve it. All right. So as an example, here's the way that perplexity's tackling it. It says, I need to search the web to find the latest developments in large language models for February 2025. Good job. Perplexity. Getting that month in there is extremely important because what happens, essentially these models, they break it down step by step, and then they run different queries at different points in this step by step process. So perplexity

got this right. The first thing they're doing is they're just Googling or they're searching LLM developments February 2025, getting that date in year. extremely important because I'm asking about the latest LLM developments. So I don't want anything from January. I don't want a 2024 year in review recap. So even though perplexity didn't ask me a follow-up question, it's at least starting down the right path. All right.

So let's quickly jump back over into OpenAI, deep research. And now you'll see I can click on this sources thing and it's giving me a progress bar. And then I can go see the activity and the, sources. So we're going to go visit that here in a second. So unfortunately, I have to have a different instance of Google Chrome open to share Gemini because it's not accessible via my workspace account. All right. So now same thing. I'm asking the same thing. And I am on the pro plan of Gemini Advance. All right. So this is Gemini 1.5 Pro with deep research. I'm asking the exact same prompt. What are the latest LLM developments this week and what will

the impact on knowledge workers in the USB and how can businesses take advantage? So you'll see what Google does is it gave me a plan. So it said, you know, kind of gave me a, let's see, eight step plan. I'm not going to read all eight of these steps, but it's finding articles, then it's going to find research papers, then it's looking at, you know, how businesses are using large language models, et cetera. So these are kind of general, right, whereas perplexity and open AI just went much more specific.

So it says it's giving me the option, then I can either edit the plan or start the research. So I'm going to click to start the research. All right. So now let's jump back because I am guessing that we are going to be done with perplexity, because perplexity is pretty fast. All right. I should probably go, wait, I think I navigated away from that page. Give me a second.

That's the bad part of doing these live. So unfortunately, I don't know what happened. It looks like it just timed out. Yeah, that's strange. So, yeah, the perplexity one just timed out and started over in the middle of that. So I'm not sure what happened there. So we're going to restart this perplexity one. And hopefully this time it doesn't just randomly time out.

So I don't know why I did that. It's the first time that's happened to me. But all right, it's going down the same path here. So now what, now here's what we're going to do. We're going to hit pause on this. So we have the three different tools that take anywhere between, you know, two minutes to 30 minutes. This is a simpler query.

So this, none of these should, I don't think, take more than like 10 minutes. And it looks like some of them are, it looks like Open AI is almost done. and it looks like Google Gemini is going to take a little bit longer here. So let's go back to this comparison chart that I had up here because I talked about the hallucination rate. And then I did an example and I essentially asked all three of these to do a deep research on myself and everyday AI. All right. So here's the actual, here's the actual prompt that I used. All right, let's see.

Here it is. I said, tell me everything about Jordan Wilson, who does everyday AI, from birth until today, make it creepy in depth. All right. All right. So let's look at how these different ones did, because I literally wrote down the number of hallucinations that I found per platform. All right. Open AI deep research, zero. Google Deep Research 2, which I think is still pretty impressive,

perplexity got the majority of everything wrong. It just hallucinated off the chain. I have no clue why. But you can go back. I actually cover this yesterday in the perplexity deep research, the dedicated episode on that. So if you want to see more information about where it went off the rails, you can go do that.

Essentially, it brought in way too many sources. It brought in like 300 web pages. and I would say more than half of them were irrelevant, were not relevant to either myself, Jordan Wilson, or everyday AI. It was a bunch of random Reddit threads, and it just started pulling information. None of that was related. So some problems with perplexity.

They didn't say that they were trying to improve the accuracy here. All right, but let's just go ahead and look at these examples. Okay, so now, so live. Live stream audience, I'm sharing the reports that each of these created with that simple prompt, right? I said, give me your report on Jordan Wilson from Everyday AI from beginning, the end, right? So this is the Google one, which did pretty, pretty well. So live stream audience, anything that I thought was, you know, impressive or, you know, noteworthy in a good way, I highlighted in green. Anything that was a hallucination I highlighted in red. So one hallucination that

Google got, it said I co-founded everyday AI with Chris Ball. No clue who Chris Ball is. But, and then it cited that as well to a random page that had nothing to do with Chris Ball. So one hallucination, not sure where that came from. But Google did a pretty good job. It got some of the details right, some things that were only on like one place on the internet. So it did a pretty good job of looking at all the nooks and crannies. You know, looked at every page on my old business website, my current business website,

interviews I had done in the past. So did a pretty good job. Pulled out some key highlights there. There's also another hallucination here that says I was a college professor. So where that hallucination came from is it was from an interview I did with an actual college professor. And I'm guessing it got mixed up in the transcript. Each speaker was labeled. So for whatever reason,

It thought I was the college professor. I was not. Although, hey, coming to a university near you working on it. All right. Going through the rest of the Google, it did a really good job. So Google deep research, aside from those two hallucinations, which, I mean, hallucinations aren't good. It really did a good job.

Formatted everything kind of throughout my career, right? Nonprofit works, strategic partnerships, starting my other company, Accelerant AI, then starting everyday AI, a little bit on different topics that I touch on and my viewpoints on these topics. So, I mean, overall, it did a pretty good job. It even found something that I forgot this was on the internet. Like I'm an avid basketball player, having met a key mentor during a game. Yeah, that was like 20 years ago. So Google Deep Research did a really good job.

So now let's look at, here we go. So this is Open AIs Deep Research. So the formatting didn't really work very well when I copy and paste it because it inserted all of these citations in here in the middle. So not great from a copy and paste perspective, but from the actual report inside Chad Chubit looks great. It did really, really well. Got my birth year, my birthplace, when I started working in, when I started delivering newspapers, right? I started delivering newspapers as a 13-year-old. I don't even know where it got that.

I forgot, but it was correct. Then I started working as a sports writer at my hometown daily paper by the time I was 17, got some information about the Pulitzer Fellowship, you know, that I traveled abroad. None of the other ones got that. Yeah. So just a lot. It did a really good job at pulling out specific details that I didn't see really very much in the other two reports. You know, something that even found.

on the podcast. I think this is, I did another podcast interview. Someone interviewed me about the podcast. And, you know, I mentioned that, you know, community interaction is a big piece of everyday AI. And this was the only deep research that talked about that. So I'm like, okay, that's, that's pretty unique. Because I only really shared that in one place. So yeah, I mean, overall, I mean, a lot of information here that I'm scrolling through on the screen. But you'll notice zero red. No hallucinations. I read this thing top to bottom twice. Nothing is incorrect. And it did a pretty freaky good job of pulling out details from me all over the web. So even though it didn't go to 500 websites, I think this ended up only going to like 12. It did a good job of logically going

step by step, starting broad, right, making those connections. It said, oh, Jordan, everyday AI, okay, let me go do a little bit more research on Jordan. Oh, he has another company. This is the right, Jordan, right? And then it's kind of working its way backwards. But you can go through and see step by step its journey, right? Which having a reasoning model and being able to look at it is extremely important. So perplexity.

Yikes. Okay. So it did find a couple things that were good, right? It said by 13, I built my first website, a GeoCities page, which is correct. But then it just, I mean, analyzing cereal box marketing tactics, although that's something right up my alley. That's not correct. That's not what I did on my first website.

It was actually, what was it? It was like covering the best, like rap and hip hop music or something in the 1990s. It wasn't anything about marketing tactics for cereal boxes. which is honestly something I love, but that's not correct. But if you're looking at my screen right now, I'm not going to take the time in poo-poo on perplexity, but the majority of this report is red. It is concerning.

It made up some of the craziest things ever, right? So it's saying everyday AI 2021 through present, which is wrong. It's 2023. It says content cadence, 538. AM Central Standard Time live stream, wrong. It says the 1734 phenomenon. Listeners noted eerie vocal modulation shifts in episodes around the 17-minute mark, correlating with spikes in chat GPT API usage.

Wilson dismisses this as audio compression artifact, right? It made up some of the weirdest things I had ever seen, right? Look at this one. It says, Chicago AI Week, 2024. Yes, I presented there at Chicago AI Week. But then it says, keynote speech included subliminal messaging in the form of steganographic QR codes hidden in slide decks. Scan codes led to an unlisted YouTube video of Wilson reading passages from parable of the sower backwards. Like, perplexity, where did you come up with this?

How? Well, like you saw the prompt that I gave everyone. Everyone else gave me pretty much factual report. Perplexity went nuts. I don't understand it. All right. Yeah, I mean, you can see. Everything's, everything's in red here.

Everything's in red. Here we go. Projected 2026. Leaked internal roadmaps hint at everyday AI v4.0, a neural lace interface bypassing vocal cords to implant thoughts directly into listeners, Broca's areas? I don't know what this is, y'all. This is wild. This is wild.

I don't know what perplexity did. It hallucinated off the chain. Now, I will say this. This is the one that it hallucinated in the most. I've actually run this two times since this one, but I had to share this one because this is the first round that I did, and I was comparing it first round, first take with everyone. it's actually gotten a little bit better,

but even today when I run it, there's still a lot of hallucination. So it's not giving me the same high hallucination rate when I ask about other things. And maybe it did it because I'm like, yo, take it to a creepy in-depth level. So maybe it thought,

I'm just going to make a bunch of stuff up because none of that stuff even existed on those 200 irrelevant sources either. It was just all a bunch of made-up garbage. All right. So, we're going to wrap this up quickly. So I have that screen back on here.

So I encourage you if you're listening on the podcast, check your show notes. I made a nice little chart comparing everything because you need to be able to make the right decision for you. And here's kind of what I boiled it down to. Which deep research service should I choose? So Open AI deep research offers the lowest hallucination rates, and it has a fine-tuned O3 model, but it is slower and way more expensive. Google Deep Research provides extensive sources, way more sources, pretty low hallucination, but it does require a paid plan.

So Open AI and Google right now, no freebies, right? I think Google's like right there in the middle. Perplexity, there's tons of promise in perplexity. If they can get the hallucination thing in check, I think they got a banger. But the last year, perplexity has been going in my use cases, perplexity's been making stuff up at a wildly unacceptable rate, wildly unacceptable, right? So it is the fastest.

It has the best limits. It's free, right? Five a day is a great free plan. But the hallucinations, I can't honestly recommend anyone use it. That's my experience. But what I say, do the exact same thing I just did, right? Have it run a creepily in-depth biography of yourself or of your department if you work at a big company, right?

Something that you know better than anyone. Have it run a report on that. See if it's factually accurate. So maybe my experience is more of a, you know, fringe use case. But I followed it up. I'm not seeing accuracy doesn't look like is a strong suit right now of perplexity deep research. Open AI deep research crushes it. This is the one I use on a daily basis.

I use Google deep research in open AI deep research nonstop. I'm not going to use perplexity deep research much right now because I can't personally trust the outputs because what it's shown me, it goes off the rail sometimes, which you can't have, right? So I hope this was helpful doing a deep research throwdown in a deep dive. So I'm going to probably follow up in this maybe in like six months because I assume they're not the only three players that are going to get into this. So I'm sure we're going to see something in the near future from meta or Microsoft or maybe a startup out of nowhere that does this exact same thing. And I do think, you know, there's going to be open source versions of this. there's going to be improvements from these three big players,

but I will say this, if you're still not sure on where to start with generative AI, and you just get confused, oh, all these large language models, all these features, all these things, right? Start here. People are always like, where do I start?

And I'm always like, well, find the right use case, you know, depending on what type of work you do. You know, here's the different large, no, start here because every single person, pretty much if you get paid to sit in front of a computer and create business value, which means you're a knowledge worker, you use the internet, which is guessing 95% of the people listening to this podcast, this is one of your best use cases. You're spending way too much time researching the internet.

The internet stinks. The internet has become an eyesore the past two years. Mainly it's large language models fault, right? They probably didn't really have the legal access to go to scrape all this. information anyways, but that's going to be, you know, adjudicated in the courts in the years and decades to come. But, you know, right now, the internet stinks. It is so hard to find good quality information at scale on the internet. This is something, if you do it right, follow my advice, always follow it up a second time, make sure you test it out, keep human in the

loop. But I think, I think, if you do that, you're going to do that, you're going to be a second time, make sure. going to be blown away right away by the power of these deep research tools. All right. If this was helpful, I hope it was. Please repost this show on LinkedIn or Twitter. Tag me if you do that. We have a guide that we put together 10 business use cases for deep research. All right.

So go click that repost button. If this was helpful, I hope it was. Thank you for listening. Please, if you haven't already, go to your everydayaI.com. Sign up for the free daily newslet. We're going to be recapping this show, as well as keeping you up to date with everything else that you need to know. Thank you for tuning in. Hope to see back tomorrow and every day for more everyday AI.

Thanks y'all. Meet Firefly AI Assistant. Now live in Adobe Firefly, the Allman One Creative AI Studio. Just describe what you want to create in your own words and the assistant handles the rest, orchestrating multi-step workflows across Adobe Creative Cloud apps, including Photoshop, Premiere Express, and more in one conversational interface. You direct the outcome while the assistant accelerates execution. Stand control with the ability to step in and refine at any time.

See it today at firefly.adobie.com. And that's a wrap for today's edition of Everyday AI. Thanks for joining us. If you enjoyed this episode, please subscribe and leave us a rating. It helps keep us going. For a little more AI magic, visit Your EverydayAI.com and sign up to our daily newsletter so you don't get left behind. Go break some barriers and we'll see you next time.