This Week in Startups - Creating the future of search and competing vs Google with Perplexity AI’s Aravind Srinivas | E1770

Episode Date: June 28, 2023

This Week in Startups is presented by: Crowdbotics. Great ideas can change the world, and Crowdbotics is the fastest way to turn those ideas into code. Get a free scoping session for your next big app idea at crowdbotics.com/twist Vanta. Compliance and security shouldn't be a deal-breaker for startups to win new business. Vanta makes it easy for companies to get a SOC 2 report fast. TWiST listeners can get $1,000 off for a limited time at vanta.com/twist OpenPhone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. TWiST listeners can get an extra 20% off any plan for your first 6 months at openphone.com/twist * Today’s show: Perplexity’s Aravind Srinivas joins Jason to discuss competing with the major players in the generative search / AI chatbot market (1:25), designing an AI-powered search engine (4:49), and much more. * Check out Perplexity: https://www.perplexity.ai/ Follow Aravind: https://twitter.com/AravSrinivas * Time stamps: (00:00) Perplexity CEO Aravind Srinivas joins Jason (1:25) Competing in the generative search / AI chatbot market  (4:49) How Perplexity's AI model formulates answers (11:18) Crowdbotics - Get a free scoping session for your next big app idea at https://crowdbotics.com/twist (12:46) How Perplexity is citing sources (20:20) Incorporating advertising into AI chatbots (25:32) Vanta - Get $1000 off your SOC 2 at https://vanta.com/twist (26:40) How Perplexity recruits talent and Aravind's time at OpenAI (32:04) The future of AI technology and overcoming overlap (38:26) OpenPhone - Get 20% off your first six months at ⁠https://openphone.com/twist (39:53) Meta's LLaMa model being leaked * Read LAUNCH Fund 4 Deal Memo: https://www.launch.co/four Apply for Funding: https://www.launch.co/apply Buy ANGEL: https://www.angelthebook.com Great recent interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland, PrayingForExits, Jenny Lefcourt Check out Jason’s suite of newsletters: https://substack.com/@calacanis * Follow Jason: Twitter: https://twitter.com/jason Instagram: https://www.instagram.com/jason LinkedIn: https://www.linkedin.com/in/jasoncalacanis * Follow TWiST: Substack: https://twistartups.substack.com Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin * Subscribe to the Founder University Podcast: https://www.founder.university/podcast

Your interviews with Brian Chesky, I learned a lot from those episodes, actually. Me too. Also, I liked his idea that, like, they were going to only ship the number of features he could keep in his brain. And that his brain would be the maximum, you know, size of the canvas. So if one person can't keep all these changes in their brain, let's put those changes into the next six-month cycle. I thought that was pretty awesome as well. I actually borrowed a heuristic from there, adapted it for our company, which was, if the person, in building the feature doesn't know how to write the code for it.

They're very good programmers, but if they're finding it a hard time to break it down and actually implement it, then it's not worth shipping. This week in startups is brought to you by CrowdBotics. Great ideas can change the world. And CrowdBotics is the fastest way to turn those ideas into code. Get a free scoping session for your next big app idea at CrowdBotics.com slash twist, Vanta. Compliance and security shouldn't be a deal breaker for social. startups to win new business. Vanta makes it easy for companies to get a SOC2 report fast.

Twist listeners can get $1,000 off for limited time at Vanta.com slash twist. And Open Phone brings your team's business calls, texts, and contacts into one delightful app that works anywhere. Get 20% off your first six months at openphone.com slash twist. All right, everybody. Welcome back to the program. We have been having an amazing array of founders who are taking on the challenges of implementing AI in the real world. And today will be no different. We have R. Unvind Shrin Ivas on the program.

He's the founder and CEO of Perplexity AI. Welcome to the program. Arnvind. Do you have a nickname or you go by Arravind? Arvind's good. Thank you for having me, or Jason. Great to have you. And listen, you, this is a name.

There's a big battle going on between chat, GPT, and Bard. And you're right in the middle of that. You are doing Perplexity AI. And you are trying to compete with these two giant software developers. Tell us a little bit about how that's going and how you see Perplexity. That AI, which you can go check out right now. The interface will look familiar. How do you plan on competing with them and how is that going?

Yeah. So firstly, we started off. A week after Chat ChaptiPT came out, we put it out. And there was a lot of difference at that time, which is we were just a search bar and we gave direct answers with citations. Whereas Chat Chapti was this entertaining hallucinatory bot that was not just, you know, like correct many times, but it was also equally wrong many times and its mistakes were also entertaining, right? So we focused a lot more on revamping search, realizing that 10 years from now, no one's going to be asking for 10 blue links.

You're going to ask for answers. So we might as well start it today. And the technology for that was ready. Both Chad GPT and us were basically being powered by GPD 3.5. That was the fundamental breakthrough. And then after that, GPT4, even better than that. So that's kind of how it started And we were seen pretty different

It's like oh, you know If Chachapiti lies or makes up Things There's this other side called perplexity You go there and like It's going to be this boring Educated Uncle kind of product But useful

And you can trust it And that's kind of how we grew And then BART came out I believe BART still hasn't solved The fake news problem completely it does hallucinate and it doesn't actually have like real citations sometimes. So we are better than barred in that in the context of search.

But Google's also rolling out this thing called Magi or they call it search generative experience to the public, but the Wall Street Journal called it Magi. So they're trying to do something pretty similar to us. And so far, the experience, at least from what I've seen and myself used that and other people who have used that is, it's not very different from the way they used to extract text from the top link and put it in space at the top. It's not very different from what they've already done before.

And they cannot afford to use really powerful models. So like the search traffic that they have, and if they actually want to really get it right, in the actual search bar itself, outside of bar, they're going to lose a ton of revenue. Yeah. have two challenges there. And so have you done a

crawl of the web because you are giving citations and you do have a language model behind this? So tell us what is underneath the hood here because I have been saying, hey, if you're going to get a bunch of information and present it to me in a beautiful

answer with bullet points and numbers like perplexity just did for me. I asked it, hey, what are activities I should do with my seven-year-olds and they like cities. and the outdoors, and it gave me four popular destinations for cities and for popular destinations outside. Really good suggestions, really tightly summarized.

And then at the bottom, it said, hey, and then here are three citations, trip advisor, U.S. News, family vacationist, and today, the Today Show. So what's underneath the hood here? Yeah. How is it generating the answer? Yeah. So L-LMs are these great reasoning engines. You throw a lot of texts at them and tell them what to do with it,

and they'll do it for you. And then there's the other part that's great, which is having a good index and a ranked version of the index, which is a traditional search engine. And what we do, where we come in is we combine the two together. We say, hey, like, LMs are great.

We'll figure out what content to throw at them for a given query, and we'll instruct them on how to actually take all the text that's thrown at them in the context of the query and get the needles from the haystack and present it in the right format of the user. So they're doing more of the reasoning job, they're not actually doing, pulling up actual facts that's been stored in the LLM itself,

because some of them could be right, some of them could be wrong, the real, actual facts are in your web pages. So that's the content that we want to take. And we have, like, our own index and also, like, we rely on other index providers. And we collate from multiple different indexes, multiple different crawls of the web, and pull up the relevant links. And then we asked the LLM to do all the reasoning on top. and then we give you the answer.

Now, the magic is that all this happens so fast. We've put out the product in December, and back then the latency used to be like five to six seconds per query. In fact, one of our investors, Daniel Gross, he used to joke to me saying, you should call it submit a job and not a submitted query. It's that slow. And now it's like almost as fast as Google.

Like you're hardly waiting. The summary is like really generated really quickly. And we still have like, you know, so much more room to improve there. And I think at some point, you're just going to take answers as the de facto search experience. That's kind of what we want to bring together.

And our primary like superiority over the existing products is the speed at which we deliver the really accurate, well-collated answers from so many different sources. And so, but you are built off of today, chat GPD4, correct? Yes, we heavily use

chat GPD 3.5 and 4. And we also use a little bit of our own LLMs for many other things. Every question you ask on our site, you see a few related questions that are being popped up, right? That's actually one of the favorite parts of the product from many of our users because they like asking more. And that is sort of generated with our own LLM, for example. So there are some parts of the product that we use our LLMs, but I would say like most of the heavy lifting is being done with opening as LLM's right now. And so you added right now, does that mean your plan on building your own? Because it does seem like you're directly in competition with Bing.

Bing has the partnership with ChatGP4. So it's almost like you're both using the same underlying technology. They already have some scale. So that would be a difficult race there. So how do you look at ChatGPT's 4's relationship or OpenAI's relationship and Microsoft's access to it? I think we just need to win by building a superior product. There's just no other way. And I believe so far we have done that.

We have not won against them, but they still have a lot of distribution through Windows devices. So a lot of people just go to Edge and they can start using BingChat. But people have, despite that, won against Microsoft in the past. They Google, everyone went in search for Chrome as the first search query on, like, Internet Explorer, to install it. We all did that despite the friction they added. So there's only one way to win against the person who has much more distribution than you, which is a superior product. Now, about using the same underlying technology, it is the case today.

The reason is they have the best models and there's still a lot of differentiation you can have and how you harness the power of these models. These are so general purpose machines. It's almost like you buy the engine from somewhere, but you're building a whole car with a lot of different parts, and you can still build a better car. And if it is the case that Open AI is just going to be the number one place by far, and you want to give the best product to your users,

you don't need to use their model. Like, there is no... Like, you can say, yeah, I'm going to use my own model because I don't want to use someone else's. But then if the search experience is pretty shit there compared to what you have with Open AI, you're not going to get users. And then you build your own modes of differentiation in other ways that just the person

owning the LLM cannot build as good competitive product as yours. So if just the LLM is the only reason this is working, we don't have a chance. But that's not the case here. There's so many other things needed to be done to give you this experience where there's real-time facts being pulled up and presented in the right manner, super fast, reliable, and make the product engaging. So all that also matters. So for example, people have done comparisons between us and Bing.

And, you know, we have like much better accuracy in terms of how correctly we cite things. A lot of academic research has been done there. People spend on an average like two minutes more on our site than Bing. So that engagement is much better there. So our bounce rates are much lower than Bing. So basically we only lack in one thing, which is number of views on the site. But that can only be addressed if we're given sufficient. time to grow and make people aware of us.

All right, we all know the one thing that separates great startups from the good ones is product velocity. What does it mean? Product velocity. Fancy term, right? You've got your product and you have velocity. Speed. The speed in which your product improves.

So can you ship updates? Can you release new features? Can you do bug fixes? Can you iterate on the interface? Can you solve problems for your customers? And can you do it quickly? because you're not alone. You have competitors and your customers have choices.

They may solve their problems by writing their own custom code or they might use your solution. This is what startups are about. How fast can you get that product velocity going? And so, you know, how do you supercharge it? Everybody says, okay, yeah, we want to go faster, but you got to go faster intelligently. And crowdbotics is going to help you do that. They're your CTO as a service. Basically, they provide you with the most optimal architecture to get your product to market as fast

as possible. You'll have access to an on-demand product manager and developer talent, and they will help get your app into production 10 times faster than conventional development. CrowdBotics can work with your in-house dev team, or you can just have them work independently. And you own all the IP, you own all the source code. Let the folks at CrowdBotics supercharge your product velocity today. No more waiting. Get a free build plan at CrowdBotics.com slash Twist. That's a $499 value just for the Twist listeners. You get that for free. That's C-R-O-W-D, B-O-T-I-C-E. dot com slash twist for a free build plan. How do you get the citations?

If you were asking this query I just did about like, hey, what cities, should I take my seven years old to and then what outdoor locations? How do you actually get the citations? Because chat GPT4, they don't provide citations? Yeah. Or do they? They have this thing called a browser plugin,

which is basically powered by Bing. But people hate that experience in the sense it's really slow and clunky. Yeah, it is slow and funky, yeah. And so how do we do the citations? We basically pull up the relevant links to your query from a search index. And then we combine that and tell the LLMs to write the answer. We basically ask the LLMs to go read all those links.

And then pull up the relevant paragraphs from each of those links and then make an answer out of whatever you thought was relevant. But write down the answer as if an academic or a journalist would write it, where each part of the answer has the corresponding citation like Wikipedia. You basically say, hey, like, I want you to do the job of what a human does on Wikipedia

where when they're writing something about a new person or a new phenomenon or a new city, this is basically going and like picking up a lot of web links about that, sifting through them and reading them and then coming back and writing an essay on it, right? So that whole human labor, intelligence needed to do that,

is being automated now. All of this happening in like seconds, right? That's worth like powers of human labor. And that's the value we are actually adding to everybody. Got it. And so you collect all those links, give all those articles, and then give the summary of them.

They basically instruct the LLM to like, hey, behave like a Wikipedia person. Just write it like this. So the core of this is prompt engineering and knowing how to prompt engineer for different types of query. because different queries might require a Wikipedia editor, other ones might need a more of a sensibility of a journalist.

And the LLM knows the difference between those things? You need to make it know. That's the skill there. And you're right, prompt engineering is a big part of it. But prompt, just because somebody might have your prom doesn't change much, actually. Like, prompts can leak. So it's all about orchestrating the back end, making it work with the right sources to.

So there's a Steve Jobs movie with Kate Winslet in it where there's a scene between Wozniak and Jobs where Wozniak's like, I'm the guy writing all the code and I'm the code, you don't write code, you don't do design. Why do you, why does everybody know you and not me? And he says, I play the orchestra. So that's basically where anyone who aims to build a long-lasting company on top of LLMs,

the thing you need to be really good at is, playing the orchestra. Like having so many things work together reliably and efficiently and correctly and super fast. So one of the pieces is searching the webbing, finding the right articles, the next piece is knowing how to write the answer. Right. What are the other pieces here?

Thinking the relevant parts from each article too. Like article has a ton of content in it. You only need a few for the query you ask. making sure that you write the answer in the most accessible way. Initially, we just started off with just putting text with citations. Then people were like, hey, I want neatly formatted answers. I want markdown in it. I want, like, code to be rendered in a specific way.

I want images in it. I want, like, videos in it. I might want to customize it for kind of the domain I'm searching in. And then people keep asking for more, and you learn more about the second part of Google's mission, right, making it universally accessible and useful. So the first part is organized those information. The second part is basically

where LLMs are adding tremendous value. And how do you deal with specific verticals of data that are more siloed? I see one of your co-founders or one of your founding team members was from Quora. You of course have the Reddit dataset. Great for conversations.

You have Twitter, great for debates and funny one-liners and breaking news. You have Yelp, you have Google local. You've got all these silos of data. I asked, hey, what are some great Greek restaurants? Did a pretty good job of telling me Greek restaurants in the Bay Area. And so how do you think about those silos of data and are you intercepting searches and saying,

hey, this search is about local businesses and restaurants. This search is about something that the Reddit data set would do better with. How do you think about that? Yeah. So the part of our data, like, you know, the access and things like that, it's, it's, It's an ongoing debate and I don't have like, you know, very strong opinions on what each person should do. Ideally, if there's a need for us to pay any party for their data access, we'll do it. As for how we do it, like, what links we know to use for which query, we do, like, take your query and figure out, like, which category it is and, like, try to use that information to give you the right sources.

It's pretty hard, actually. Google does a tremendous job at this. And we are also doing some things. called focus searches where in the search bar, instead of using all of the internet, you can go and pick academic, or you can pick YouTube, you can pick Reddit, Wikipedia,

and you can just, yeah. Yeah, so you can, there is a drop down called all. Yeah. And I could just pick YouTube. And then YouTube, you have access to the corpus of all the transcripts or just the metadata, I guess, and titles. For now, we use metadata

and titles, but that's already amazing. Sometimes I can't find some videos on YouTube directly, but these LLMs are so good at, like, doing the relevance ranking, that's much better than the YouTube search algorithm. The language models do better than Google's native search algorithm. Wow. Sometimes. Not always. Got it. Most of the times it's equal, but sometimes it's just really good at, like, these fine grain. I was trying to find a video of, like, oh, so there's a scene in this movie I want to find, for watching for

inspiration or something. And then I couldn't find out YouTube, and I come here and I get it. It's very useful for Reddit. Like I want to like learn about like, you know, the nothing phone. Like, you know, who's even using it? Who are those million people? And then I don't have time to go to the subreddit nothing phone and like score over all these like links. That's very useful there. People use it a lot for Wikipedia.

Like if they just want to focus on one thing. Like I was talking to the founder of Wikipedia, Jimmy Wales. And he literally just asked for this feature. Like, hey, I just want to do search over Wikipedia with an LLM. I was a great idea Yeah I think they're building it now within Wikipedia itself

Hmm Interestingly I did a search For interviews With the CEO of Airbnb Mine didn't come up But other ones did

But then it came up with I did once from the past year And man That was kind of a bingo It kind of nailed it Which is a kind of a nice feeling I really think that's a creative idea And I can see how

what you're talking about is got some there is some point to this which is if you narrow the scope or you build some interesting prompt engineering or narrowing and thoughtfulness

you can get to a better answer so what's going to be your business model here you talked before about how Google is not going to be able to make it work with advertising there's a group of people who believe that the chat interface

will cannibalize their existing business. So do you agree that this chat GPT style interface or just the chat interface, let's leave the GPD out of it? Nobody owns a chat interface. But is the chat interface anti-advertising,

or could advertising be integrated into it? Because on All In, a lot of, I think three out of four besties thought, hey, advertising's not going to work. And I thought, I think advertising is going to work great inside of this. You have your citations, but you could put right in, embedded in the discussion, you know, all kinds of

interesting things. So if you were asking about places to travel with your kids and I'm Disneyland and you didn't make it, I could put in there, hey, and if you're thinking about outdoor stuff, Disneyland also has this adventure park and they do the safari and I could have like a really AI generated answer at the bottom. So it gives me the correct answer or what it thinks is the correct answer, but then it also gives an ad engine's answer to it. So am I right or are my other three besties right?

You decide. I'm more of a two here. Oh, you are? Okay. So firstly, I think relevance can be even more targeted now than ever before. What is the purpose of Google? It's just bringing two parties together, the advertiser and the consumer. And they help you connect these two parties together,

where their query and link matching, right? At the end of the day, the advertiser wants to get their content to the consumer or the content. And LLM can give you that needles in the haystack even better. Like, it's even more targeted, honestly. That if I were an advertiser,

I would just kind of focus on selling myself really well, writing even better marketing copies with LLMs, cater to the person. I'm trying to sell to. And we introduced this thing called AI profile on perplexity, where you can just write about yourself. Ah, yes, I saw that. And that way you, the results are even more catered to you.

And then if you're an advertiser, you can say, I want to target people who are of, like, having all these attributes in their profiles. And then the ranking will automatically take care of that. So in some sense, you're creating, way more relevant and targeted ads than ever before. I don't know if you use Instagram, but my experience in Instagram

is that the ads on Instagram are even more relevant than than Google. Often is that is the case. Here, take a look at this. Can you see my screen? Here's the query I did based on our little back and forth here. Will LLMs, will the chat

interface be accommodating to advertising. Well, I put in here, you're the CEO of Disney Parks, pitch me on why I should take my seven-year-olds to one of your parks. And so imagine this got appended to my previous search, which, hey, what should I do with my seven-year-olds? In a city or outdoors. And I says, oh, thank you for considering one of our parks

for seven-year-olds. Here are reasons we believe you'll have an unforgettable experience. Number one, a place where everyone is welcome. Two, more value and flexibility. Three, disability access service. That's kind of weird. Number four, new attractions and experiences. That's really good. Memorable music, that's good too, actually. Park reservation system, that's great.

We hope you'll consider this. And then here it could have bookings and would you like to talk to an agent? Do you have further questions? And you could just hijack somebody's chat stream for your own purposes. They could be thinking they want to go to Europe for the summer, and then you could sell them on going on a European Disney cruise or something. And I think that this kind of style of advertising where a company CEO

starts a discussion with you in chat GPT and in a chat interface is going to be magical. Yeah. And like you said, you know, I can give you an answer that's sort of neutral and unbiased and it's not targeted at you. And I can also say, by the way, in case you were actually looking for something

very much to you. And if you already share that information with us, fully transparent and you're in control, we're not going to do it in a creepy way like Facebook. Then we should be able to give you the answer. We should be able to help the advertiser sell to you even better, right? So I think basically I'm going even more abstract first principle in thinking that it's not clear how you do it in the product

and how you build a business model, but at an abstract level, the point of advertising is to reach the right person to sell to, and this can help you do that even better than the current system. So therefore, you should be able to figure out something at a level below this. If you're a SaaS or services company that stores customer data in the cloud, then you need to be, uh, SAC2 compliant. You knew that from a third party,

and you need that third party to close big deals. And if you want to get compliant, easier and faster you need to use. V-A-N-T-A. V-A-V-A-V-A-V-A-V-A-V-A-V-A-V-A-V-A-V-A-V-A-V-A-V-T-A-V-T-C-A-V-T-C-T-C-T-Compin in just two to four weeks. Prepare that to three to five months without Vanta. And Vanta can save you hundreds of hours of manual work and up to 85% of compliance costs. This is a total no-brainer. And Vanta does more than just SOC2 compliance. They also automate up to 90% compliance for GDPR, PIPA, and more. You can't afford to lose out our major customers.

We all know that. Listen, it's a hard year. Last year was hard. You can't lose those major customers because you don't have your compliance dialed in. Just work with Vanta. Get your compliance automated and tight and tight is right. Lock down those big deals. Here's the best part.

Vanta's going to give you $1,000 off. That's $1,000 off at vanta.com slash twist. That's vanta.com. Slice for $1,000 off your sock, too. So you've raised some money and you're currently trying to grow the company. Tell me a little bit about what it's like to try to compete in this area for talent. People are raising, you've raised a lot of money, but people have raised even more. And there's a massive talent battle going on right now.

Is it better to just hire great developers and have them learn? Because you're not building the fundamental model. You're building something on top of it. What's your strategy for talent here? Yeah. So we don't waste time trying to hire people that, say, on when they'll be hiring anyway. It's very,

it's, you cannot compete. They are way more cash, way more, like, and they can give way less percent of the company because they have a way bigger valuation. So what we do is go for these people who are still trying to get into AI, very talented engineers who haven't done AI before.

And want to be part of an amazing product that's growing and they want to feel the dopamine from shipping every week and want to see their stuff actually being. put up. And there is quite a lot of people, there are quite a lot of people who are like that, like who haven't done AI before, very talented generalist programmers. That's another thing that I look for, which is, are they generalists?

Can they do back and can they do front and can they strategize for the product? Can they do prompt engineering? Because all these are new skills. Like prompt engineering is not like a, you know, you don't need to, there's, but you cannot ask ask for years of experience there. It's like a few month old skill.

So you just need to be somebody who's like pretty logical and like pretty good at like getting things done. You were at Open AI for a while. I was at Open AI, yeah. Yeah, how long were you there and what did you work on? So I yeah, I worked on like diffusion models and like conversational models for like chat bot. Like not exactly like chat tributy, but more like trying to get another modality into like conversations. So that's kind of what I was focusing on.

But the reason I started this company was because ever since I came to U.S. for grad school in Berkeley. I was always interested in starting a company. And I was trying to look for people who were like me before, who were like PhD students who started a company. And I could only find one example from the past that I really resonated with was Larry and Sergey. So Larry is my entrepreneurial hero.

Like he's the only reason I kind of wanted to do a company. And in fact, in a book he's written like he'd either do a professor or he would do a company. he would never work for anybody else. I had more constraints in my, like, you know, immigration and other stuff like that to have to, like, sort of work for a bit, get some money and, like, learn more skills. But that was sort of always there. And it's not planned, but it's just more like a coincidence, happy coincidence that I'm working on search too. But, yeah, being at OpenEI, it was really helpful. Back then, there was no chat GPT, so I didn't foresee the future where Open AI is so successful.

but there was GPD 3.5 and was pretty good and like, you know, we knew like a lot of things were happening. Nobody knew that, like, if you put out these models in the chat UI, the world would go crazy. That was very unknown. So the fact that people are so used to the modality of chat, because they live in it all day long, this was the breakout moment for AI.

Because AI stuff had existed. People were using it in the backends to serve you up your for you page on TikTok or filling your search query or giving you a couple of words ahead in Gmail and finish your sentence. All that stuff, it was happening. Yeah. But it needed the interface to make it work. Yeah.

Fascinating. The generality of the models was also amazing, but it was all, if you remember, opening, I had a playground where you could go and enter a prompt. And in green text, you would see the completions. But nobody cared about the average person in the world did not care about it. And then you put it. into a chat UI and then the world goes crazy, right? Makes you wonder if there's another thing that you could do that make the world go even

crazier. And I got to think, and I'm interested in your thoughts on this, were Siri and Alexa, just far too early. They had the ability to understand what you were saying. Yeah. They just didn't have the ability to give you the right answer or any answer. Exactly. I mean, you could barely call, you know, you'd be like, oh, okay, call my mom.

And it would be like, calling Mother Teresa. saying you're like, no, no, no, no, no. It's not what I want. And just even getting it to play the right song took three tries. Now with chat GPT and all these language models and Bard and Poe and what you're doing at perplexity, it feels like talking to the computer would work. And I don't know why this doesn't exist yet. It's going to happen. It's going to happen.

Yeah, like, if I had perplexity as running in the background on my phone in my ear pieces, and I could just whisper to it and say, hey, hey, perplexity. what are some Greek restaurants near me that have a lamb and that are over four stars and it just gave me the answer back and started talking to me and I could take out my phone that would be so magical

and just using the language models as your interface but using voice and having to talk back to you would be incredible. Yeah. Still doesn't exist. In five years I think

what's going to happen is we'll talk. We'll all wear glasses, we'll talk and then we'll see the answer render in our glasses and then or it can speak back to us and we we can listen via the glasses or whatever okay why doesn't it exist today like as we speak uh i think you you can stitch together a demo with a speech recognition model and lLM and then a text speech model right yeah um the latency wouldn't be enjoyable like um that it's mostly on the lLM side not not even on the speed side uh you can make these

ASR and DTS work pretty fast. But if you had to wait for two to three seconds, it's a bit like talking to a socially awkward person. Like they would be like staring at you for like two seconds and then giving you back the answer, right? Yeah, yeah. So that's the experience you would get. It might not be very enjoyable, like how you and I are talking right now.

I think for that you need even smaller or even faster LLMs. And... Ah, so it's not... It wouldn't have the response time that people would find not annoying. It would quickly become annoying to have it giving those pauses.

Yeah, I find it quite charming now when my chat GPT interface takes a second or Bard is kind of skipping around and it stutters and then it plays and I'm like, wait a second. And then Bard now just gives you the answer straight away. Boom. It doesn't do the typing.

But I've got, I think the OpenAI Apple app, iOS app, has like kind of haptics in it where it's like typing. I think it's kind of a gimmick, right? It's not. We chose not to do it. But there is this thing where you stream the output tokens, token by token.

The reason we did that is because you perceive the latency as lower. Like, if I waited in my backend to generate the full answer and then displayed it like in the bard style, you might just be like, oh, what the hell? Like, I don't want to wait, you know? And then you just, I just bombard you with a huge paragraph. It may not be as fun as like anticipating, like, you're, like, you're. reading along with the model generating the tokens, that's a different kind of UX. I like opening eyes choice here,

but we didn't do the haptic thing because I found it pretty annoying to use when we were beta testing it and so did the others in our company. So that said, you know, like, here's the thing with TTS. Like, you have to generate the full answer before feeding it into the Texas speed system. If it's just going to read it word by word

as the LLMD goes the answer, it's not going to get the tone of the sentence completely about the same thing. right. So if there's an exclamation mark at the end you started reading the sentence. Yeah, that's a fair point.

It's not going to know that. I'm curious, you also were a researcher at Google in Deep Mind. Before going to Open AI and before launching your own. A lot of these language models were based on

seminal papers on tensors and whatever. And a lot of the code base was open source or open source is. I guess in Facebook's it was leaked in the case of opening eye the original models were open source.

How much overlap is there in the fundamental technology at this point and how much is different? If we were to take, you know, the top five language models, how much shared DNA do they actually have? How different

are they at their cores? So, everything is a transformer, which is the architecture built by Google in 2017. and everything is generatively pre-trained with language models. So all of that is the same. The difference comes to what data is being trained on,

where Open AI puts in a lot of effort there compared to other organizations. The reason methods, Lama models were actually really good, despite not being as-because Open AI's models, is because the research is there put a lot of effort into curating the right data. Ah. Well, explain what? what that means to lay people here who are wondering what you're talking about. Yeah.

So how these intelligent language models are built is you have this giant neural network and you download a lot of data from the internet, terabytes of data. And you make these neural networks predict the next word given the previous words. You basically train them to be great auto-complete machines. And by virtue of doing that, they become really good at reasoning and things like that. Now, that doesn't mean that if you just keep scrolling the web and scraping every page and then creating the dataset, you're going to keep getting smarter and smarter. In fact, you get smarter by not training on junk and actually training on good quality data. And now, like, it also turns out that if you train a lot on coding, like GitHub and other datasets, you develop these reasoning capabilities to an even higher level.

than not training on coding. It's kind of like thinking about, like let's see you have a kid, you send the kid to coding or math competitions, even if they may not become the, you know, the I am a medalist, they might end up being great analytical

and logical thinkers in their life and that might help them in their life. So that's sort of what happens with these LMs. And so if you pay a lot of attention to what data they are trained on, that helps you a lot in terms of what you can achieve with them later. So the base core IQ of these models will be much higher if you put a lot more effort into like curating the training data more carefully. And Open AI was ahead of everybody else there.

Google has all the data in the world, but they didn't pay enough attention to this. And now like people have caught up, they've understood, you know, this is where they need to pay attention on. As for like who's really ahead right now, I think it's Open AI like with GPD4. Yeah, much far ahead. Who can likely catch up? There's one more organization called Anthropic. Sure. And they are the closest number two.

And both these organizations were more or less the same people. Like the people who trained GPD3 were the guys who had then started Anthropic later. Are you still using your personal phone number at work at your startup in 2023? Stop! Such a common mistake founders make. But open phone has totally rethought every detail of what a business phone should look like in 2023. Open phone makes it so easy to do this and so affordable that you have no excuse. And you really don't want your team using their personal phones for business. Why?

Well, it could get creepy. People start texting people on your team. It could be that they leave your company and the salesperson has all of these text threads going with all your clients and they bring them to your competitor. Do you want to deal with this nonsense? You don't. I can tell you open phone is amazing because we use it. Our sales team, our ops teams. We use it daily. We also started using open phone for Angel Summit communications. It's rated number one on G2 for customer satisfaction. And let me tell you, those G2 rankings, those are dogged battles. If you win that, you really have to be the best.

Twist listeners love open phone. My sales team uses it. Ops team uses it. Customer support uses it. And you know what's great about it? You can create a shared phone number like we did for the Angel Summit. With multiple employees being able to field those calls and text and keep it all sorted. It's affordable at just $13 per user per month.

But Twist users are going to get 20% off that already ridiculously affordable price for six months at openphone.com slash twist. And if you got an existing number, Openphone will port it over at no extra cost. Head to openphone.com slash twist to start your free trial and get 20% off. So when you look at the open source community, they seem to be really moving fast now. Correct. Meta's Lama models were leaked, maybe, or maybe leaked on purpose. Yeah. You think that you think that story is true, that it was leaked on purpose to jumpstart the open source community?

I wouldn't be surprised, but, you know. Yeah. It was accidentally leaked. Accidentally on purpose. There's some parallels with the COVID leaks there. I don't know. Yeah. It was, yeah, it was an accidental leak, but they might have leaked it because, yeah.

But this was actually good. Like, it was good for the world. That this anthropic got leaked. I'm sorry. that Lama got leaked? Yeah, Lama leaking was actually really good for the world. You know, I think, I think,

I think it gave more power to the rest of the world in terms of what they can do with LLM's outside of Open AI or Google or Anthropic. So that's my question. These open source models, you've got a lot of people working on them.

Yeah. And a lot of people are not happy with how closed open AI has become. Even I've started referring to it as closed AI. So if they're super closed, and Open tends to win, if we're sitting here in five years, who do you think wins?

Open source or, you know, Google and OpenAI with closed models? Who do you think is going to win? Yeah, it's, if you pattern match, Open tends to win, that's kind of correct. But there's like a catch here, which is the next big wins

are not necessarily going to come from whoever is going to continue. continue to train more. You need some algorithmic efficiencies to make use of compute even better. And you need really good researchers for that. And the best researchers are sort of like NBA players and like they're taken by these organizations who pay them millions of dollars a year. And then if these guys who are building the tricks for making these models even better

are in the closed organizations, then they'll always stay ahead of the open. Right. So then, and if these organizations stop publishing these techniques, and these guys to stay in these organizations are paid to stay there forever, it's kind of like closing the walls. So the only way in which the open source world can catch up is, like, there are amazing researchers who kind of work in organizations that are actively open source models.

And I think right now there's only one big org that wants to do that, which is meta. And so as long as meta is in the game, I think there's a chance for open source to sort of stay there and like, you know, win in the long run. Every other organization doesn't want to publish anymore. That's a problem.

Nobody publishing. Except for meta. Except for meta. And I guess that Google feels like they made a mistake publishing all this stuff and giving a to some moment. I'm sure they do. Like, they missed out on the whole revolution.

It's fascinating. And I didn't ask you about the paid version. If I, if I choose to pay, what do I get? So there is this thing called co-pilot. That's more like an interactive search companion. Ah. That it does the equivalent of hundreds of search queries for you, not just one. So you can ask it really complex queries, like,

go pull me all of Jason's investments and all startups such is done. And like, you know, at what valuation he's done. like prepare a table for me and get it back to me. If the information is there, put in public, for example, I could only find the valuation you invested in Uber,

but not on Robin Hood. Ah. So then it'll come back to me and give me that information. Or you can say, like, give me the year-by-year revenue of AWS ever since its inception.

I want to track it and growth percentage year over year, and it's going to come back to you with information. So it's almost like you're having a researcher at your disposal. Oh, wow. That's wild.

And when you say it's a co-pilot, is it something that lives in my system? try and Mac or Windows or at Chrome? No, it's on the browser. The co-pilot is just meant to be like a companion, like the word, the user of the word is just a companion for search. And it's going to help you plan, travel, buy products, prepare meal plans according to your preferences.

And if you integrate your AI profile with it, it's going to give you, like, much more detail, recommendations, travel itineries, web research. Like, I wanted to know a lot about, like, when did read off and start making money in LinkedIn? like you know, they took a while to like start making revenue. What was the hypothesis in lit scaling? All these kind of things that you're like not coding as well.

Like write me a piece of code for pulling up all Elon Musk tweets to where he tagged Jeff Vezos in it. And like you get the Twitter API, we do code. You can copy paste that and like go and execute it. So it's, it can read documentation pages. So that way it's more factual than what code you get from chat chagip. So all these kind of things, it's very powerful. So what we offer in this.

the paid version is unlimited usage of that. Not fully, like technically unlimited. It's more like 300 queries a day, which is practically unlimited for most people. And then everything else is free. So the way we're thinking about it is the free version grows enough that we can do advertising there.

And the paid version is for power users who want to like use it for work or very complex queries that they seek. But the free users get like 25 queries a day even on the co-pilot version. So you don't have to pay if you don't want to. We just want regular daily users

to stop using Google and use our product. I will be one of them. I'm just signing up for the paid version as we wrap up the episode here. You're hiring, so where can people learn more

about what you're hiring for? We're hiring for iOS and Android, mainly right now. So iOS engineers, if you want to come and help build our mobile experiences, please join us. That's the most important time here.

And I think you can go to perplexity. slash about and you'll learn more. All right, we'll see you all next time on this week and start. Bye bye. On behalf of the producers and the partnership team, thank you for listening to episode 1770. We'd like to take one more time to thank our partners,

Crowbotics. Get a free scoping session for your next big app idea at crowbotics.com slash twist. Vanta. Get a thousand dollars off your sock too at vanta.com slash twist. and Openphone. Get 20% off your first six months at openphone.com slash twist.

If you're looking to become a partner of this week in startups, you can email Hanna at Hannaatlaunch.co. That's Hanna atlaunch.com. Thanks for listening.