This Week in Startups - AI Demos: Gatekeep, Deepgram, Claude 3 Haiku and more! | E1920

Episode Date: March 26, 2024

This Week in Startups is brought to you by… LinkedIn Ads: To redeem a $100 LinkedIn ad credit and launch your first campaign, go to http://www.linkedin.com/thisweekinstartups Coda: A new doc that brings words, tables and teams together. All your valuable data, plans, objectives, and strategies in one place. Go to https://coda.io/twist to get a $1,000 credit! Wistia is THE all-in-one video platform for business, with tools that help you create, manage, and measure the impact of your videos. Try Wistia for free at http://www.wistia.com/startups * Todays show: Sunny joins Jason to dive into AI news and demos, including a new report about AI and Enterprises (12:28), Gatekeep AI (23:34), Deepgram API Playground (39:55), and more! * Timestamps: (0:00) Sunny joins Jason. (4:33) What the resignation of the CEO of Stability AI signifies. (11:01) LinkedIn Ads - Get a $100 LinkedIn ad credit at http://www.linkedin.com/thisweekinstartups (12:28) Exploring the a16z report about AI and Enterprises. (20:42) Coda - Get a $1,000 startup credit at https://coda.io/twist (23:34) Sunny demoes Gatekeep AI and get surprising results when Jason asks it to explain a “keystone”. (31:27) Wistia - Try Wistia for free at wistia.com/startups. (39:55) Sunny demoes Deepgram API Playground with a fun clip from a young Jason asking Steve Jobs a question. (50:19 ) Sunny demoes the Claude model Haiku from Anthropic with an image from a famous Michael Jordan dunk. * Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp * LINKS: Report: 16 Changes to the Way Enterprises Are Building and Buying Generative AI: https://a16z.com/generative-ai-enterprise-2024/ Check out Gatekeep AI: https://app.gatekeep.ai/ Check out Deepgram API Playground: https://playground.deepgram.com/ Check out Antrhopic’s Claude Haiku: https://console.anthropic.com/workbench/01fd2f39-ed95-40e1-9d11-c92f1a5ef26f Claude 3 Haiku announcement: https://www.anthropic.com/news/claude-3-haikuplayground: Article about Stability AI CEO resigning: https://techcrunch.com/2024/03/22/stability-ai-ceo-resigns-because-youre-not-going-to-beat-centralized-ai-with-more-centralized-ai/ * Follow Sunny: X: https://twitter.com/sundeep LinkedIn: https://www.linkedin.com/in/sundeepm * Follow Jason: X: https://twitter.com/Jason LinkedIn: https://www.linkedin.com/in/jasoncalacanis * Thank you to our partners: (11:01) LinkedIn Ads - Get a $100 LinkedIn ad credit at http://www.linkedin.com/thisweekinstartups (20:42) Coda - Get a $1,000 startup credit at https://coda.io/twist (31:27) Wistia - Try Wistia for free at wistia.com/startups. * Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland * Check out Jason’s suite of newsletters: https://substack.com/@calacanis * Follow TWiST: Substack: https://twistartups.substack.com Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups * Subscribe to the Founder University Podcast: https://www.founder.university/podcast

So this is from our friends over at DeepGram. I don't know if you remember this. I'm just play a little snippet. Steve, there's a thousand bloggers at Microsoft and bloggers that people at Apple, I understand, are not allowed to blog. You remember this, right? Yeah. DeepGram has released its new tool where if you give it an audio file, they can do an intelligence

analysis of it. So you can basically have it break out the intense of that conversation. So 25% they felt was about requesting clarification. about blogging. Yep. 9% was about the RSS feed. 11% was about inquiring about podcasts. And then the last bit was the Zigger about the email.

Oh, asked for Zinger email. I love it. That's hilarious. It got the singer. I made the joke in the audience left. Wow. This Week in Startups is brought to you by LinkedIn ads. To redeem a $100 LinkedIn ad credit and launch your first campaign,

go to LinkedIn.com slash this week in start. startups. Coda, a new doc that brings words, tables, and teams together. All your valuable data, plans, objectives, and strategies in one place. Go to coda.io slash twist to get a $1,000 credit. And Wistia is the all-in-one video platform for business with tools that help you create, manage, and measure the impact of your videos. Try Wistia for free at Wistia.com slash Startups. All right, everybody. Welcome back to this

weekend startups. I'm here with my guy, Sondi Maudra. He works at a company called GROC. Yeah, definitive intelligence was acquired by GROQ.

GROQ. Q. Q is your GROQ. Yes. Hey, is the GROC that Elon spun out from Twitter. So, just so we're clear.

How's your two weeks into the acquisition, how's it going so far? I see a number of developers using the GROC platform has gone parabolic or it's doubling every couple weeks. It's going really, really fast. You know, we are over 60,000 developers, over 15,000 apps. That's awesome. Yeah, we are rocking and rolling. So it has been

really fun. We're adding capacity really fast. It is a bit of cat and mouse for us, because every time we add capacity, people consume it as fast as we can. And so, you know, love all the support we're getting. I was actually at a hackathon yesterday that we we kind of co-sponsored with the mistral team. Oh, yeah. I saw that. Was that in the city? Yeah. It was. It's a great space there. I don't know if you've been there. Shaq 15. Wow. It's in the fair. Yeah, it's the second level of the ferry building. And I didn't even know this thing existed. It's a little bit like a we work slash soulhouse. So you buy a membership. I did hear about this. Yeah. And here's the thing. like there's so much space available when you go through the boom bus cycle.

And I've seen this before during the dot com and the Great Recession, people after the dot com bust or after the Great Recession were like, you know, there's tons of office space for like $1,200 bucks a month. The five of us could be in a cool office. Maybe we'll spend 15 grand a year having an office. It'll be fun, right? Then it became, oh my God, you know, my we work for like five desks, like $9,000 or something. It was like, okay, we're spending $100,000 a year and this just makes no sense. What are we doing? Yeah.

And that boom bus cycle is now, we're in that trow where people are like, oh, there's space available and it's free basically or close to free or minimum missing cost. Oh, yeah, might as well get a space. And so that's really interesting when you think about space. People get inspired by space too. They see a space or like, wouldn't it be cool if a bunch of people came here and did something, like a restaurant? This space was a 10 out of 10. It was a 10 out of 10. It's the best space I've ever seen in the city for like startup slash kind of, you know, community vibes.

Yeah. So I think we're going to start to see a lot of that, you know, because why not all this empty space in the city? We should do it. Oh, yeah, we should totally do it. Yeah, absolutely easy to do. We just pick a day and then we just put a bunch of speakers and we're done. Great idea. Some demos too.

So we'll get that gone. Okay. Yeah. Actually, that's what we do. It's just demo or die kind of situation. A couple of new. There's items, J-Kow. Yeah.

Oh, there's news. Yeah. Okay. So, news and then demo. So quick news in the AI world. Yeah. And so I would say there's like one interesting thing, which is the CEO and founder of Stability Diffusion, he basically stepped down from the role.

And, you know, that's one thing in of itself. Let's not speculate too much on that. I think he said something interesting in his comment in stepping down, which is he feels as though there is, yeah, and exactly this is it, because you're not going to be centralized AI with more centralized AI. And so I find that to be a really interesting comment in terms of, you know, stepping down. And, you know, they're open source focus. So I'm not fully sure, well, I can speculate on what he's trying to mean here, but I want to see what you have to think when you read that comment. Because they're open source first.

I met him, the gentleman in question briefly when I was in the Middle East. He was in Dubai and we were out of a salon together. It seemed like an interesting cat. But yeah, he's doing an open sourced image creator, right? And what is the market for these things, I guess is going to become the question. So I think we're past the honeymoon phase and we're now getting into brass tax. And so what investors are starting to ask, I believe, is, okay, how many Do consumers and businesses need to AI? Okay, if you are using Notion, I'm paying extra for AI. We're using Zoom right now. We've got the AI summary, AI companion. I pay for everybody on our team to have OpenAI. I paid also for Po. And, you know, now I'm like, I'm pretty promiscuous in trying all these things.

But I do think businesses are going to, and investors are going to start to say, okay, what what's essential? What do we need? What are we going to use? And maybe the idea is not many people need image generation. Maybe that's like a niche thing. You know, how often are you creating images? Maybe for fun, you and I could sit there and, you know, go crazy all day.

But I think maybe we only need two of these in the world, three of these in the world. Maybe we don't need 20 people making image creators. Maybe there's just not that much work. And so the one built into Adobe's suite, right? They have one that's legal and built into there. Then there's Dolly. you know, unstable diffusion, there just doesn't need to be that many of them.

And that's my read into his comments, which is we can't compete with Open AI. That would be my read. Open AI plus Microsoft plus Google Gemini, or maybe the Google Apple Alliance, which seems to be emerging, we talked about last week. So if you have these two competing factions,

Open AI plus Microsoft, and then over here you have Google and Apple, and then the open source community and the Google app, and then you have the meta open source community. Maybe there's enough going on here that the world just doesn't need number four or five. And maybe they, he believed they would be number five.

And, you know, that's like this, I watch this happen with Uber, Lyft, DoorDash, Just Eats, hostmates, sidecar. There was a long tale of people going after that space and at a certain point, consolidation happens. And this has been a rapidly evolving space. Maybe we're witnessing consolidation happening or capitulation. You know what? That was a very similar read to mine, which is the big, folks are already here. You know, it wasn't hard for them to get there from a technology standpoint. And it's very difficult to be, you know, like you said, third, you know, maybe not even third,

but fourth, fifth, sixth player, because you've got the consumer folks covering it off. You've got the original pro folks covering it off with Adobe. And then you have the up and coming new folks covering it off with mid-jurney. And so really like, you know, to your point that it's, it's difficult to win in that space. And maybe, you know, he'll come at this from a different point of view as he rethinks what he's going to do next. Yeah. Or there could be something else to him leaving the company.

Maybe people didn't want to work for him. We don't know, right? You sort of speculated who knows why he's leaving the company. When CEOs leave a company, there could be burnout, personal issues that they've got. Could be, you know, bad news inside the company. Maybe people don't want to work. work for them. Maybe they're not a great manager. It could be any of those things. And so sometimes when people leave the playground, they take their ball with him. So he could be leaving the playground or being

pushed out of the playground. It's like, I'm going to take my ball with me. This space sucks, you know, but it might just be he couldn't compete. Maybe he just didn't have the energy to fight this fight for the next five or 10 years, which would be a mature thing to do. If you don't think you can fight a dog fight for five or 10 years to just be honest about it and just be like, it's hard. What's the point? You know, and sometimes you have to face that as an entrepreneur. I've had to face it in my career many times where like you're playing a certain game on the field, the game changes. I did a print magazine and then the world didn't want print magazines. And I did blogs and the world didn't want blogs and I did podcasts.

Maybe someday the world doesn't want podcasts and want shorts. Actually, the world wanted blogs. You sold to her elite there. Yeah. You know, I had a little bit of scar tissue from not selling too late the first time. So that was like my 18 months in, I'm selling this thing and I'm getting my first check. So I don't have to sweat. The world wanted Baha'u.

Just Google didn't want you to be it. And now we have a version of it with AI. And perplexity, yeah. And the perplexity is using the same concept we're using. Just AI wasn't ready 12 years ago when I thought, you know, these 12 blue links would be better if it was like content plus services, plus Q&A. I mean, I had the right idea. I just was way too early.

And I think that's like you could forgive yourself as an entrepreneur. I think is some important message here. Sometimes you're too early. sometimes you sell too early sometimes you're too early to the party sometimes you're too late to the party and who knows what's happening here but it does seem like

maybe there just wasn't an opportunity maybe they didn't perceive there was a huge opportunity and also with the sale of to Microsoft of hi what was the parent name towards the last week inflection that feels like okay and we said this like how many

large models do there need to be how many assistants do there need to be Yeah. You know, it does seem to me like verticalized services in AI will win the day. And then there'll be three or four of these foundational models. And that's exactly what's happening, right? Navigating the B2B maze can feel really tough, huh? You're trying to hit the mark with all those top-tier executives.

You want them to pay attention to your enterprise product. But where can you find all those big fish, the whales? The ones who call the shots and make the buying decisions for corporations, for startups, and everybody in between. Well, here's where LinkedIn ads is going to solve that problem for you. And I've used this. It is one of my secret weapons. LinkedIn means business.

Business equals LinkedIn in people's minds. When you're on LinkedIn, you're in the business mindset. So you're going to really be thinking about business products and services. You're open to those opportunities. And LinkedIn recently passed a billion users. 180 million of those billion are senior executives, 18%. But, hey, we all know about the 1%. 10 million C-suite executives. That's your CFO, CTO, CIO. These are the people who are always looking for a new product or service to make their organization run better. But they are on LinkedIn. That's why LinkedIn's ad platform delivers two to five times greater return on investment compared to other social media platforms. So easy to understand why this is because this is where all the business people are and they're in that business mindset. Super easy call to action. Make your B2B marketing everything it can be and get a $100 credit on your next campaign. Go to LinkedIn. Go to LinkedIn.

com slash this week in startups to claim your credit. That's LinkedIn.com slash this week in startups, no spaces, no dashes. Terms and conditions apply because they're giving you a hundred. Jayca, you always did this magic to do it. A great segue into just the last news piece, because I do want to touch on it, which is A16Z, at least a really good report. And I just want to touch on a couple things for all the builders out there, because we do this for a lot of builders. And yesterday, a lot of folks came up at the hackathon and they were like, oh, it was so awesome. You guys. guys do the demos, so they're really cool to see it. But you should read the whole report, but I'm going to touch on three interesting bullets from this report. And it's generally

value in the enterprise. First, you can see here, the amount of average spend for LLMs is going from $7 million last year to $18 million. This is for enterprise companies. So large companies are saying their average spend in 2023, the average spend across foundation model APIs, self-hosting, and fine-tuning models was 7 million of, of course, dozens of companies we spoke to. So they spoke to a couple of dozen companies. Yeah, it does look like the average spend for enterprise. Yes. It went from 7 million to 18 million is the estimate.

So the actual was seven. So they're two and a half timesing it. What would this be the equivalent of the cloud computing revolution? Even faster, right? Like, you know, this is a kegger, which is like, you know, growing really, really fast, right? And I think it's faster than anything we've seen before. And look, like, you know, they call it's like anywhere from, two to five X, right? And more workflows going into production. So this is good for folks that are

building. Okay. I thought this was interesting in terms of like where the allocation. Yeah, where the money is for genera, where is the money for genera of AI coming from? So they asked 25 enterprises, I'm guessing here. It says dozens. They didn't say two dozens. They said dozens. Dozens of companies, okay. Oh, dozens of companies. Yeah. So when you say dozens of companies, that would mean like 24, 36 companies probably. So in the second one, where do they allocate this from or reallocate?

So they're saying the budget doesn't exist. One company cited saving $6 million for each call served, $6, which calls served by their LLM power customer service for a total of 90% cost savings. That's a reason to increase their investment in Gen. I eatfold. Okay. So they're using one anecdotal story there. Got it. But what I would say here, JCal, is when you're building in the earliest phases, and I've seen this in, you know, two previous lives, I've seen it when I had a mobile business and I had a cloud business.

A lot of the early budget comes from the innovation teams because they're the ones within the enterprise that, you know, are tasked with looking at this. But to see that we're already at a place where 75% of the budget is not coming from innovation, this is really good. Got it. So this isn't the innovation budget where like experimental stuff goes. Okay, you know, Vision Pro comes out, AR, VR, okay, we'll put a little bit of money into that. Okay. Yeah. Here, 24% innovation, 76% other business unit budgets, 21% is the next highest. That means somebody in a business unit, like customer support, said, hey, I want to allocate my budget towards this and away from something else. Reallocated from the IT group 15% new net new IT budget for. LLM, so that's 19%. So somebody said, hey, I need to increase our budget.

Yes. Give me some money for this. Reallocated from other AI, 13%, R&D product development 8%. So yeah, it's definitely immediately having an impact on enterprise is your point. I can't argue with that here. And I have one more that I want to pull up. And then we'll get into our demos here, which is interestingly showing, you know, which models are enterprises using.

And so let me bring the share back here. By the way, I found it in the first paragraph here. Over the past couple of months, we've spoken with dozens of Fortune 500 and top enterprise leaders and surveyed 70 more to understand how they're using buying and budgeting generative AI. We were shocked by how significant the resourcing and attitudes towards Gen. I.I. had changed over the last six months. Though these leaders still have some reservations about deploying generative AI, they're also nearly tripling their budgets, expanding the number of use cases that are deployed on smaller open source models and transitioning more workloads. from early experimentation into production, that says to me that they know it's not ready for prime time, but they're trying to make it ready for prime time. So there's some recognition that it's hallucinating. How would you interpret that? I would say that they are ready to push into production.

And these next two charts, which will be the last ones, and we won't go over the whole report, but this is really fascinating because you can basically see which model providers or enterprises using. And you can see here that, you know, overall, you know, Open AI, still the clear leader, which is, you know, which is 100% of the enterprises are trade. Open AI. Are open AI is that. That's impressive.

So 70 of the 70, yes. In fact, 70 companies or 70 leaders said they were using open eye. And 63% said they were using Microsoft. No, no. I'm sorry, Google. Google. Yes. Yeah.

And 50% of it was testing, 13% of it was production. Yes. Lama, 30% testing, 11% production, anthropic, 34% we're using, but not in production. Mistral, 17%. Oh, here, 17%. So basically, yeah, people are testing everything and solely putting stuff into market. Yes. And then the last one, and we'll get into it.

demos, which is right here. Open source is booming. And you can see here where the shift is really happening across this little pie chart. So we have a pie chart here with three sections. The three choices are one.

For those people who are listening. Yeah. The three sections are no plans to increase open source. That's 18%. Then 41% say they will switch to open source when the performance matches. and another 41% saying will increase or already use majority of open source.

This is really important. So this means 82% of enterprises want open source. They don't want a closed model. Yes. Wow. You know, this is so interesting because you remember in the early days of open source 20 years ago, everybody was like, yeah, nobody wants to use it. now enterprises are basically defaulting to it.

Yes. It's incredible what open source has done, and this means you would be long, llama, anthropic. No, anthropics not open. Oh, no, no. You'd be long llama. Lama, you'd be long, one of the mistral models, mixed trial. Yeah.

Right? And then you would be short. And then the open source Google ones. And then there's an open source Google one, yeah. It makes sense. I think this technology is so power. that putting your entire company in the hands of Microsoft and OpenA or Google and Google Cloud, some folks might feel uncomfortable with them having so much say in what they're doing, right?

That's how I would read it. And they want to be able to not have that dependency, right? So that's what I think I would read in here is open source means not dependent, not taking a huge risk, which is so funny when you think about it because we used to have this expression. Nobody ever got fired for selecting Microsoft or IBM and then Microsoft. So, you know, now it's kind of the opposite. Nobody would ever got fired for picking an open source project. Because with an open source project, you know that you can't get rug pulled, you can't get price gouging.

You know, you're just fascinating, which makes it even more perplexing why Open AI went closed. Because the statistics here are showing don't go closed. Right? That's what the statistics are saying. The statistics are saying the market, at least the enterprise market wants this solution in a certain way and people are just ignoring them. So there you have it, folks. The great ignoring of the consumer base. Okay, listen, I got a lot on my plate. I've got to do a couple podcasts. I got Founding University and Launch Accelerator and all my personal life and things I want to do. So there's just a lot. But I'm able to manage it all with an amazing piece of software called Coda. Coda is the all in one

platform that combines the best of documents, spreadsheets, and apps. Here's an example. We use Coda to run the Founder University. Every week, we ask all of our founders to submit a progress update. And then all of that goes into Coda as a database. And then we can sort through all of those weekly updates and look for trends. And Coda also allows us to send automated reminders to all those founders to send in their updates. And then we track week over week growth by generating charts. So if we see strong growth, we'll reach out and we'll invest in that company. And it's all done through this beautiful product called CODA. Here's your call to action. You can use this software to solve any business problem. It is extraordinary. And if you want that platform to empower your startup to strategize plan

and track goals effectively, you can get started for free. Coda wants to support founders. So they're going to give you $1,000 in credits at coda.io slash twist. What a generous offer. It's a limited time offer. So I want you to get it right now. Cota.io slash twist. That means you can begin planning right now to make your startup just really tight Codda.io slash twist to get started for free and get that $1,000 credit. Can't beat the price. I love Cota and you will too. You know, there's this notion that is, you know, kind of cross-pollinated between enterprise and consumer, I believe. Definitely in Silicon Valley more than other places, which is moat. And, you know, obviously in consumer, that makes it's important. But I think in enterprise, the notion of moat being tied to something proprietary

or closed is a bad idea, but folks maybe tend to go in that direction versus something that is like, you know, performance or price. So this lock-in thing, I think Silicon Valley and SAS and everything that's happened in the last 10 plus years are still working through this. Yeah, makes sense. All right, let's do some. Let's do it. Yeah, of course. Let's go. Well, I mean, we're starting to see where the industry is going to wind up. And it's, it's, you know, which if you dovetail the two stories, like stable diffusion then is an open source project with a closing project. That would argue that they were more valuable because enterprises would want the open source solution. So there's something with the founder leaving that's a little bit.

Wasn't that the same founder who allegedly screwed his co-founder out of their shares and bought a bunch of the shares back from them? There was a little bit of dicingness. There was some story related to, yeah, getting rid of a co-founder or something. And then getting all their shares. So, okay, let's get started. Okay, first one today. I really like this one. They are under a lot of pressure today because it's been going viral.

What it is, it's called gatekeep. com. And what they do is you generate something. I did this already because it's been slammed today. but you ask it for a concept. They have some pre-built ones here. I did one myself, which I'll share here, which is tell me about put options,

because I know, you know, I know how much J. trading loves, you know, trying to. I don't do options trading. I know, I know. I know. Nancy Pelosi level here of taking more risks and bets. Just follow her trades.

There you don't even just follow her trades. So what I did was I just said, tell me about put options. And what this did was, and I really liked this a lot. It created a little short video. I'll play it in a second. It gave me a chat about it, and then it gave me a little summary down here in a transcript.

I'm just going to play this for me 30 seconds. Put options are financial contracts that give the owner the right, but not the obligation, to sell an underlying asset at a specified price known as the strike price within a specific time frame. Let's visualize how a put option works. Imagine we have a stock whose price is decreasing over time. Okay. I'm not going to play the whole video. I think this is awesome. It made this explainer video.

The graphics, the voiceover and the coffee. The graphics, the voiceover, and gave me a little chat. So imagine, you know, you're trying to work through, trying to understand something, or you're doing something with your kids when they're trying to learn, you know. This is a real combination of the video, the chat experience. It's Khan Academy in a box. It's like a Khan Academy. Oh, a good way to put it.

Yeah. It's AI Khan Academy. Let's type. I want to try one. Explain to me how a keystone in an arch works. To me, in an archworks. It does take them a few minutes. Yeah, yeah, that's all good.

Well, because it's generating a video. This is generating a how-to video that you could then publish on YouTube. And now YouTube's going to be flooded with these. And hopefully they're correct. Because if they suck, then it's wrong information, that's not helping. So this is where synthetic data or the moment where data can be, to generate data revolution is going to pollute the hell out of the internet. So it's almost like you want to have the human internet ends at 2023. And going forward, YouTube is going to be filled with this

stuff. Now, is this stuff been checked? Is it correct or not? We're going to have to create a whole cleanup crew because, as we know, it gets things wrong all the time. And so what if this is wrong? What if it explains the pathagorean theorem wrong? What if it explains how, you know, the Keystone works incorrectly? And now we start flooding the internet. My replies are frequently on X Twitter generated by AI. I can tell somebody's got an AI. Oh, really?

Reply bot. And it's like, oh, you enjoy beef ribs in Texas. Beef ribs are a nice thing to enjoy. I hope you enjoyed your beef ribs. Continue enjoying your beef ribs from Terry Blacks. And I'm just like, what's going on here? Yeah. Okay. And it's obvious like we're going to pollute the hell out of the internet with this garbage.

And so I'm just worried about YouTube. The incentives are really messed up right now. So we had content farms, which Mahalo was definitely not part of, but there was like EHOW and other people who were doing content farming, which was just throw anything on a page, get SEO, link it together, grab the ads. And if you don't answer the question or if it's incorrect, that's even better because people click on the ads. So if it's a frustrating page, what are you do when you land on a frustrating page that's supposedly going to answer your question? Yeah. You either hit the back key and go back to Google or you click one of the Google or the ads on the page. Yeah.

And so that's what led to the whole Panda update and Google not wanting to link to content sites that were scaling up content. And it killed Mahalo and a bunch of other sites along with it. If it was accurate, what was their issue with that? Their issue was that there were a group of people who had figured out SEO and had to build a large number of content pages. and that people would start going direct to those. And they just didn't want these content networks to take over the top ranks. And so they just said, this is the maximum amount of traffic that can go to each site. And they wouldn't tell you what was going on.

So we went from having like the no, as but one silly example, how to cook salmon we had the number one result on. And for some period of time, Mahalo just had, you know, like literally 25,000 people a day, day and day out going to that page. So we just started investing more on that page, more on that page than we did, how to cook cod. Of course. We did like little how to content and then links to the best stuff on the internet. But they started to consider this like a little bit competitive. And then they started taking from other sites, content and then putting it in the one box.

So their vision of the world was don't go to these pages. We'll just put the information on the Google search results. So they started getting competitive. And you see that today where Google is competing heads up with a lot of the sites that they used to link to. So you used to get stock quotes, sports scores. recipes, shopping at other websites, those got built into the Google search results, right? If you ask for a stock price, Google just gives it to you.

If you ask for sports score, it just gives it to you. If you ask for a recipe, it just gives it to you. Or if you're asking what temperature should medium rare salmon be, it just gives it to you. And so, yeah, you know, this is the pollution of the internet is going to be crazy in the coming years. And that's where figuring out if this is in fact, AI generated or human generated.

And then as long as it's correct, then it doesn't matter. But that's the question I have for you is, when do we think we can start trusting the results from AI and in what sectors? It's just the amount of wrong information is the challenge I have right now with fully embracing a lot of this. Let's do a real-time test. It just got generated. This is the one that you asked. How a keystone and an arch works. Here's the summary.

Okay. The keystone is a wedge-shaped stone placed at the top of an arch. It's rolled, you should be a little bit faster here. Let's play faster. Here we go. Welcome to this video explaining the importance of the keystone in an arch. An arch is a curved structure that supports weight above an opening, and it has been a crucial architectural element for centuries.

Let's first look at an arch without a keystone. The stones in the arch push against each other and rely on friction to stay in place. It's not an arch. Now let's introduce the keystone. The keystone is the final wedge-shaped stone placed at the top of the arch. When the keystone is inserted, it locks the other stones in place by distributing the weight evenly. No, the keystone prevents the other stones from slight outward, providing stability to the arch.

A bunch of bricks on top of each other. Totally bundled. I think it didn't get the drawing right, but its concept is correct. It's concept is correct. So proving my point, it's crucial for maintaining the integrity of the arch. It's got this completely utterly

and sure it's strength and stability. In terms of drawing it. It got the other parts correct. Yeah. Yeah. I mean, anybody does it now. If you think of an arch, there's a keystone.

That's the middle stone. all the stones lean into that and all the pressure goes into that stone and keeps the arch up. And for those of you who are listening and not watching, this video just showed a stack of squares with a triangle laid in the middle of them. It's exactly the opposite of how it works. So gatekeep.a.i, great, but I guess garbage in, garbage out or good data in, garbage out. It's almost like we have a problem here where you get good information into the, these LLMs, and then they screw it up. So it's still work to be done here. I think it got the information

from Wikipedia relatively well, so the copy wasn't bad, but the drawing was terrible and wrong. So I don't trust any of this stuff right now. Okay, so your boss tells you you got to make a product tutorial and a customer support video. And I know what you're thinking. I need a video producer. I need an editor. And, oh man, I need a new boss. Well, think again. With Wistia, anyone can make professional quality videos right in their web browsing. Do you need a script? Just use Wistia's AI script generator and it will work magic. Need to be on camera? Use Wistia's built and recorded to capture your face, your screen, or both at the same time. And when it comes to editing, Wistie's text-based editor makes things a breeze. Just highlight the mistakes and hit delete,

take out those mistakes, the ums, the ahs, the hiccups, whatever you got. You can edit it in real time and export your video. You can even add music with Wistia's smart tracks, and these are composed specifically for business videos, so they're not going to be corny. Most importantly, they're royalty-free, and they automatically adjust to the length of your video. Seriously, if you need to make polished videos fast, Wistia is your go-to because they make amazing products for startups. So, I want you to go to Wistia.com slash startups. Even if you've got a big company, they're great for big companies too. W-I-S-T-I-A dot com slash startups. Yeah, I mean, look, conceptually, the way you put it, which is like kind of an on-demand Khan Academy, awesome.

The idea is very powerful. I mean, this is just, yeah, it's got to be right. So I think this is an interesting spot where there could be a convergence of human, you know, what do they call them on Twitter or X now? The people that do those. Oh, the community notes. Yeah, community notes. Yeah, like Wikipedia editors. So, you need to have teachers, authority.

You need to have humans who can say this is correct getting into this model. I give this an F for the output and an A for the concept. Okay, okay. So what is the average out to? I give it a D as in it's dumb. It's a dumb answer. I give it a D. Okay.

As in dumb. Well, the answer was right. No, the answer was right. The image was wrong. It didn't. Okay. And learning an arc, there's plenty of videos on YouTube right now where you can learn it. So this is D for dumb.

I give it a D. Okay. You'll make you dumb. No, no. I think the concept's in A. Okay. I love the concept that it would, you know, make me a unique video of anything I'm trying to understand. It's just, it's worthless if it's not correct.

It's actually harmful. I might go to an F now. I'm talking myself back into one out. Oh, my God. I think you're so kind It's no It's hard to build these things

Shake out You know Maybe I'm frustrated on a Monday Yeah I know Exactly You know

For me It's a C Because You know If there's not a way You know For the system To tell you

This might not be right And so they should Maybe provide Some Some type of score On that one To say We're not really sure

Because Like I said I was playing around with the pre-built ones and the stock option one was pretty good. But that one wasn't. And so, but that's a, that's an easy thing for the team to iterate on is provide a confidence score or something like that. And look, they did do one thing. And when I refresh the site, they said they did change their model because they were getting overloaded. Maybe their older model is not running as well. They might have gone to a cheaper one since everybody came there to make their,

anyway, I'm giving you guys and gals, whoever's over there, say them, the whole group. That's unit. It could be one person. It could be 100. But I'm going to give... I just want to share it. It says right up here on this. You see that, right? Yeah, yeah. Our servers are overloaded. So anyway, if it was correct, I would have given this like a B plus, an A minus. Like, it's really a great, compelling concept. So I, but I give it a D on the execution. But I think the concept is wonderful. So I grind it out and get the results to equal the vision like anything else. Listen, first Tesla was a little rough too, man. Yeah.

The roadster was a, was a bit of a rough experience. And, you know, no charging network, none of that stuff, right? You were just, no supercharger, you know,

200 and something range. It was a little rough, you know, when you hit a bump bottle in the roadster, you'd feel it in your teeth. You know, it was a hard experience. You know,

I drive that car sometimes. And then I drive my model Y. And it's like, oh my God, you want to talk about going from like a D to an A plus. Like, yeah,

there you go. You know, the roadster was the D. Like it was, it was, I mean, but it was also delightful in that. It was like so peppy or whatever, but you do have to make it comfortable for everybody and it has to be accurate and safe. And yeah, here we go. So you're on your journey. I think it's good. How long before these things in your mind start to give enough consistent good answers that you would trust it to teach your kids? So I would- When would you trust it to teach your kids for the SATs? So here's my issue. I think if you asked me the question, three months ago, I would have given you my really conservative, or so my aggressive answer, like, oh, within six months, this will be solved.

There's a weird thing that's happening where, and I'm probably not like uniquely positioned to say I'm being fair about it because of, you know, what I've been working on for the last couple years. Yeah. Where so many people just blindly rely on whether it's chat, GPT or which LLM of your choice or something like this, that it's, there's like this virtuous, cycle of people saying, oh, you know what, let me just count on the LLM to produce whatever it does. And I think now there's so many of these things out there that do that, it's been stretched out.

So I would have said by the end of this year, we'll be in a spot where you can trust these things and it'll be grounded. Now I think that's going to be harder because there's so much being created that we have to go through the first arc of many people kind of dealing with that. And it's a unique thing for computers because we never really count on computers to be fundamentally probabilistic, right? You don't break out a calculator and go like, that might not be all the time, you know. Yeah, no. I mean, the whole, this is the conundrum of this technology is we're so used to that if you put numbers in a spreadsheet, there is no way for the spreadsheet to say, you know, cell A1 plus cell B1 equal cell C1. Like, the formula can't be wrong. It's just, it's not possible for the computer to do a wrong formula in Excel or for spell check not to work.

Like, we just fundamentally when you hit the, you know, when you type the word the, we expect it to be correct. And, you know, if it's not correct, like your keyboard's broken, like, something is something really wrong that's easy to identify. It's like, you get these hallucinations or whatever. It's like, it's almost like a drunk human, you know, like if you were to had a great, let's say you had a great history professor and they got hammered. Yeah. So like you were listening to them, like they gave a great talk. Like, and then that night they got bombed and you're at, you know, the kitchen table and they're drunk out of their mind telling you history and it's not accurate. Yeah.

But during the day they told you really accurate stuff. Like that's what we're trying to deal with is is Jekyll and Hyde. I think we got a lot of work to do here. year. I don't know if it's two more jumps or three more jumps, but we're not being, I think, harsh enough on accuracy. And so this needs to be the year of accuracy. I'm going to dub this the year of accuracy. I'm looking at every single demo going forward. Accuracy is my number one thing. I'm not wowed by it anymore. I'm not wowed by what I just saw. I like the, I'm wow by the vision, but I'm judging things on accuracy going forward. I'm just,

just letting people know, starting today, March 25th, 20, 24. Accuracy matters. When it comes to building quickly and, like, you know, putting things out there, when it comes to accuracy, that's something you really should. Like, you want to go fast, but I think on accuracy, it's important. I'm for accuracy to play a role here. No more gifts of, you know, A pluses, but you got not accurate. I want to see a little accuracy.

Okay, let's go to another one. Okay. Okay. So the next one, you know, hopefully. get a kick out of this one, J-Cal. This is a good one. So this is from our friends over at DeepGram. And I don't know if you remember this.

I'm just play a little snippet. Steve, there's a thousand bloggers at Microsoft and bloggers that people at Apple, I understand, are not allowed to blog. Okay. You remember this, right? You're basically asking about blogging at Apple, right? And what was going to happen? So DeepGram has released its new tool where if you give it an audio.

file, what they can do is they can do basically an intelligence analysis of it. You can summarize it, topic detection, intent detection, and sentiment. So I took that video and basically, obviously, obviously you have the summary, which is the speaker discusses the issue of blogging at Apple, the need for employees not to spread false information, ironically. Yep. They also ask the other speaker if they can add a feeder into their podcast. Like I think you were talking about something about OpenState.

I was talking about iTunes supporting. Podcasts back 10 plus years ago. Exactly. So now what they have this tool is so you can basically have it break out the intense of that conversation. So 25% they felt was about requesting clarification about blogging. Yep. 9% was about the RSS feed. 11% was about inquiring about podcasts.

And then the last bit was the Zigger about the email. Oh, asked for Zinger email. I love it. That's hilarious. You remember at the end of that one? I made the joke. audience left. Wow. Yeah. This is really scary, how interesting this is. And then you have word level sentiment.

Wow. Yes. So negative, positive. That's interesting. Yeah. So this is, I think, really important. This is really cool. Yeah, because, you know, you could take an entire podcast. Yes. And run it through this and, you know, start to understand what I, and people are doing this for sales calls, obviously, with things like gong or whatever. Yeah. So, you know, I feel like this is another one of these really cool pools for you to understand how you're being perceived. And I could see this working really well.

We do 60 or 70 calls with founders per week and we invest in, on average, two companies a week, right? Okay. So if you're doing 70 calls, you know, and we're doing two investments a week, you know, it's like one in 35 eventually wind up getting. an investment. Now, they had to beat out a bunch of people to even get on that phone call, yada, yada. We have the accelerator. But we now are recording these calls and transcribing them, et cetera. So you're really good to have the associates do this. And we could see, you know,

both the associates at our firm, we call them RAAs, researchers, analysts, associates. Those are the three levels. You start as a researcher out of school. You become an analyst after 500 falls and then you become an associate after a thousand calls or something like that. All right. Is the program created at launch. So I would like to actually- Jobsatlaunch.com. There you go.

I would like to actually have run some of my RAAs through this and see how they are perceived, if they're perceived as smart or not. And then the way I hire people now is I have them, I have somebody on our team pitch, do a mock pitch. And I pay people a hundred bucks who are considering hiring to do a call. record it where they're given the script of how you take a pitch call, right, an introductory call. So we pay them for that, but I would write it through this now.

So I'm going to use it. I give this an A. This is A for Awesome. Yeah. And it goes an A for awesome. Yeah. A for awesome. Okay, great.

And you can see. And it's awesome. And it's and it is accurate. Yeah. And, you know, that one's particularly interesting because it shows how the sentiment changed over time throughout the, you know, very short conversation. It was like a minute long. Yeah, Q&A at a conference.

You know, a bit aggressive. And then obviously, you know, the, Steve says, hey, you'll be able to add whatever you want. We're working with open standards. And yes. Ends with, you know, the email comment. So I thought this was really, really great, really cool. There's a lot of other things you can define custom intents that you want it to detect as well.

So if there's, in your case, if you're, there's like a certain type of intent. like question or, you know, things that you're focusing on with your employees. Very, very cool. I mean, I want to keep the calls positive. And I want to keep them moving quickly, right? And keep them positive and be respectful of people's time. And also make sure that we under,

my most important thing is making sure that the associates and researchers at our firm understand the product. So the founders feel that we understand their vision. And so I have them say at the end of the recording, okay, can I repeat back to you your vision for definitive intelligence? I just want to make sure I understand it. Let me know if I got it right. And by adding that to the script,

I got rated higher by founders on our calls because sometimes founders were like, yeah, I don't know if J-Cal understood our vision. Now, if you don't invest in somebody, they're like, they didn't invest because they don't understand our vision and how genius we are. I get it. Like, I'm an entrepreneur, too. But if I say at the end of the call,

hey, just to be clear, Sundee, your vision for definitive intelligence is to use AI to sort through enterprise companies and their big data sets and help them leverage that to achieve their business goals and you charge them a fee. And the fee's not cheap. It's a couple hundred thousand dollars a year.

You would feel like I understood what definitive intelligence did. Yes. And so my custom thing here would, could we tell sentiment of feeling understood, feeling heard? Oh. And I don't know. So you would put a custom intent in there. My custom intent would.

I want the founders to feel that we understand them or they feel heard. So I don't know how to make a founder. You're high on the empathy these days. Well, you know, your reputation in venture is really important. Yeah. And, you know, you have moments in this job where you're, you have to be a bit of a hard ass, you know, like people want to sell their company and they put like 90% of the deal value in an employment track to the screw the investors. Like we deal with all kinds of weird stuff that occurs, you know, people rent themselves an apartment for 20K month and you know, you're like, what are you doing?

Like, this is not ethical or it's, you know, would be perceived as like impropriety. And so when we don't have to, when we, we don't have to do with those situations, I'm trying to. to have a very high level of empathy and connection with the founders. Yeah. I know that sounds groovy and hippie-dippy, but, you know, they're unique individuals in the world founders, right? They are. And they're unique.

You know, we, but feeling heard, like having been on all sides of it, right, having been a multiple time entrepreneur, having an investor, having been, you know, selling companies, acquired companies. I think that little trick that you're talking about is really important. I've felt that frustration many times. I felt it selling a product to someone, not feeling like the person I'm selling to does not understanding. Felt it during a pitch to people.

So I like that. I'll give this one. I call these devices. And when you have a device like this, I try to make devices like we're doing here when we give it a rating. A device, you know, can in a process or a system,

these kind of devices in a system or a process. can really manifest like a certain brand, right? So if you even have the concept of an introductory call, we frame it as an introductory call because it's not designed to be the be all end all. It's just to introduce you to our team and what your vision is, and then for you to understand who we are

and how we invest. Oh, we're not Series A leads. We're seed. Okay, we have programs, right? So sometimes we need to, you know, help with them, understanding what we do, right? So that's why I created the concept of the introductory call and the device of, let me repeat back to you the vision. So I make sure we understand it.

I do it myself now. It's like slowing down to speed up in a way. You slow down and you reflect back to the person what they're saying, right? It's also a good way to be a good partner to a spouse or to be a good friend. It is. The other thing is reflection is a big thing in AI as well. And I'll explain why. When an LLM starts to produce output, right? It's predicting X tokens. But one of the things it doesn't have is a backspace key. So once it starts down its journey,

and if in its prediction tree, it makes a prediction that's incorrect, it just has to continue. But when you ask it, are you sure? Are you sure? It gets a chance to run it again and take a different path. And so the same thing works with LLMs.

Yeah, so that's interesting. So confirming, are you sure, like if we did that when we did the Keystone thing, in the training, if we said, are you sure that's how a Keystone works? And we're like, let me go check. But yeah, in the inference, like, yeah, let me check. Because I kind of made a mistake and I just have to continue with it because I was, and there's no way to stop because it's just streaming the tokens at you. So it's an important concept.

It's reflection. There's some good papers written about it. It will drop a link to everyone. Yeah. Awesome. All right. Let's do one more demo here. One more demo first.

Yeah, I do. Okay, I'm going to give that one an A as well, though. So that's a A for both of us. Yeah. I mean, it's so awesome. I'm a sucker for these kind of tools that help you understand humans more. And it seemed, I give it the A for accuracy because I know that was me talking. I know Steve Jobs replying.

I know. Yeah. You know, it was Walt Moss. You know, teeing it off to me. Like, I've seen that clip a couple of times. I know it's accurate. So it felt very accurate. So credit to them.

Okay. We are going to close out with our friends over at Anthropic. Okay. And so, you know, I have this queued up. And here we go. And so I basically have a famous picture of,

you know, Michael Jordan. Oh, the dunk. Yeah, it's a famous dunk of his where he takes off at the free throw line. Yeah. Sam Dunn competition.

You know, we've all seen it. And basically, the team over at Anthropic have launched a new model or a variant of a model. In this particular one, it's called Cloud 3 Haiku, so it's smaller and faster. And it's multimodal. So in this case, for folks that are listening, and I can run this again, just real time here, and you can see it's going to come back pretty fast. You said, describe this image, and it was instant.

The image depicts a professional basketball player in midair, dunking a basketball through the hoop during a game. The player is wearing a red jersey appears to be in the mist. of an impressive dunk and with the crowd in the background visible and cheering the court and basketball arena suggests this is a high level basketball competition likely an NBA game, the dynamic action and energy captured in the image convey the excitement and athleticism of the sport. And so what we can see here is, you know, what we said at the end of last year that multimodal AI is going to become really important. And, you know, these folks have a model. It's fast. It's accurate.

and it's incredibly useful for where things are going now. And what I wanted to really talk about with respect to this, because we've seen a few of these now, this is one of the ones that's available, like hosted, along with like GPTV. I want entrepreneurs who are building to think about this in the following way. If you were to, oh, let me actually find another image really quickly and pull it up. Well, let me also ask you a question while you're doing this. If you were to increase temperature and the max tokens to sample, there's two bars there.

Explain to the audience what that would do here. Yeah. So temperature is going to increase the variance in its response. It's going to basically, it's going to make it much more, I would say, I'm just going to describe it. When it's going down its path of probabilities, it's going to make it a little bit more random. Sometimes it won't always pick the highest probability. is what I would say. Yeah.

And so if we run that again, and then I will pull up another image here as well. So we'll run this. And so... At a point six. It's kind of very similar. But I'm going to pull up an image. Yeah, 0.6.

And this model, Haiku, is the... This is their cheapest, fastest, smallest, right? Like, this is there. You want to grab an image and describe it real fast. This is the way to do it. This is the way you want to do it. This is exactly the way you want to do it.

Okay. So what we're going to do is... And I want really... entrepreneurs to think about this particular example. I've got this image of a nice, healthy food bowl. That is a healthy food bowl. Okay. And what I've done is I'm saying, describe this image and return the results in JSON

format, include the type of food, the size, and calorie count. And we're going to do this. And why this is really important, this one knows, okay, I want more details, okay. So it simplified. Oh, let me turn the temperature down. This is probably why as well. Let's take this here. Let's run this again.

Why this is very powerful, I'm going to do this one more time. Hold on. Okay. I scribe each ingredient. And for people don't know, JSON is an open standard file format. Yeah. Human readable text to store and transmit data in arrays. Right.

So here's why this is important. If you were to try to do this a year ago, you would have to have had something take the image, then process the image, and then go through an individual element on each thing that was there, and then cross-reference that against another database with food information. Now, all of that's been collapsed into a single model, which is basically saying, telling me there's about a cup of roasted potatoes in there, sweet potatoes. Calorie count. It's telling me there's about a half a cup of sauteed mushrooms.

There's some of, you know, Friedberg's favorite. It's a half a cup of quinoa in there. There's half an avocado, some roasted tomatoes, and some turmeric roasted cauliflower, about half a cup. I didn't know it's turmeric because of the color. Yes, because of the color. Yeah. So why this is amazing is if you're now developing something, you don't have to use a whole bunch of services inside of AWS or GCP or Azure to decompose a problem.

You can basically have the LLM decompose a problem for you. And so if you wanted to make like the next generation calorie counter app, it's literally maybe five lines of code. to pull that off. Amazing. And, you know, if you also wanted to take, I mentioned this like every Charlie Rose episode ever, you know, and understand, like, who's a film director, who's an author, who's a journalist, and then, you know, start to understand what topics they're talking about,

then looking for those topics, you know, in history, in journalism, like, stitching this stuff together is getting faster and smoother. You don't need to do multiple choices. Like a Quentin Tarantino interview on Charlie Rose in 2002 would include these films, but it wouldn't include the following films. A 2010 interview would include those. And so just understanding, you know, the 17 seminal Charlie Rose interviews with Quentin Tarantino or just Quentin Tarantino generally, putting those all together and then understanding the arc of what he's talking about, what topics over time and being. to pin those to his filmography, etc. All of that's starting to be included in these models, yeah?

Yeah. And what I think for developers, and I saw this at the hackathon I was at yesterday to close out, is what you can pull off in a short amount of time has been fundamentally rewritten. I think I'm excited to do, we should do, like we said, a demo hackathon thing together, J-Cal. And I really want, you know,

folks that are out there that are building, And I know many already are, but if the folks that haven't made that jump yet, it's easier than ever to do these things. You're going to be able to really solve a lot of problems very quickly as these platforms, you know, just keep extending the data and the reasoning, right? I mean, the reasoning part of this is super interesting because you could ask it. Once you have this information, you could say, what should be removed from this and what should be added to make it healthier? Yeah. Right? So you could say like, hey, optimize this salad for me to reduce the amount of calories in it, but increase the amount of fiber.

You didn't ask it for fiber, but we could get fiber in there. And you could say, I need to, you know, get more fiber in my diet and I need to reduce, you know, the number of calories. So what's a way to do that? Oh, you know, get rid of the quinoa and go with Chanadal, you know, like here's a different. This would be the easiest way to solve that problem, right? This is just absolutely astounding and how fast it is and how cheap it is. Which means, I think, on a venture basis, we were talking about this $40 billion fund that the PIF was considering doing an AI and how we might allocate it. It does seem like the fundamental language model is like they're getting cheaper and cheaper.

I'm wondering how much money is to be made there. I mean, I like your business, the hardware business, the hosting business. Running them. But the models themselves, if they're open source and they're going to trend towards free, right? Like, how much is it going to cost to do this? I think, you know, the way the allocation was being put out was accurate. I think it's like just give developers access to these models and, you know, let them build. And I think the market will stabilize itself when it comes to who's running them and who can run them the cheapest and fastest.

It's going to be amazing. I mean, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the, the. cost plummeting and the quality increasing means this accuracy issue, I think it's solved. I don't know when, but I think today's theme is accuracy. And yeah, so I'll give this one. That seemed highly accurate to me. I'm going to get this one a B plus. Yeah, just having not tested it across many things, but I just feel like, yeah, significant.

I think they've done a good job. They've made it fast. They've made it accessible. And now we just want to see a lot more people building with it. I think that'll be really exciting. Keep building with it. Keep building. And the number of developers is going to greatly increase, I think,

with these tools being able to teach you to be a developer or be a maestro amongst a bunch of agents that can write code. So I'm really excited for that. I know that's going to take a little while, but I really want to see the number of people who can write code or push code. I just say push code because they're not writing it, but who can oversee code production. I want to see that increase, you know,

and the velocity of new products increase. All right, man, it's been another great episode. We'll see you all next time. Follow X.com slash Sundeepe, X.com slash Jason. You get us this week in startups.com slash bets to see the bets we've placed on our AI future. And this week and startups. com slash AI to get the AI playlist. Go to this week in startups on YouTube so you can watch the video and see the Keystone not get placed in the right spot if you want to.

Great job, Sunny. And we'll see you all next time. Bye-bye. Awesome. Bye.