This Week in Startups - The future of enterprise search and AI-powered work productivity with Glean’s Arvind Jain | E1916

Episode Date: March 20, 2024

This Week in Startups is brought to you by… OpenPhone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. TWiST listeners can get an extra 20% off any plan for your first 6 months at http://www.openphone.com/twist Lemon.io - Hire pre-vetted remote developers, get 15% off your first 4 weeks of developer time at https://Lemon.io/twist Gelt. It’s time to take control over your taxes. Discover how Gelt can help you to manage and optimize both your personal and business taxes. Visit joingelt.com/twist now. * Todays show: Glean’s Arvind Jain joins Jason to discuss how his startup is building the future of AI-Powered enterprise search (2:35). The two also discuss Google Gemini (42:45), Open-Source vs Closed models (45:17), and much more! * Timestamps: (00:00) Glean’s Arvind Jain joins Jason (2:35) Glean’s beachhead market and primary focus (5:13) Glean’s approach to data confidentiality (11:57) OpenPhone - Get 20% off your first six months at http://www.openphone.com/twist (13:13) Competition against native tools and building language models in SaaS companies (17:00) Glean’s approach to permissions (24:20) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist (25:23) Compliance and the "CEO God Mode" feature (29:28) Potential of productivity software, and people analytics in tech businesses (38:50) Gelt - It’s time to take control over your taxes. https://joingelt.com/twist now (40:04) Impact of AI on business models (42:45) Arvind’s thoughts on Google Gemini (45:17) Open source vs closed models in AI development and the progress of OpenAI (47:49) AGI and the limitations and potential of AI as an assistant * Check out: https://cybernews.com/security/chatgpt-samsung-leak-explained-lessons/#google_vignette * Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp * Follow Arvind: X: https://twitter.com/jainarvind LinkedIn: https://www.linkedin.com/in/arvind-jain-5935161 * Follow Jason: X: https://twitter.com/Jason LinkedIn: https://www.linkedin.com/in/jasoncalacanis * Thank you to our partners: (11:57) OpenPhone - Get 20% off your first six months at http://www.openphone.com/twist (24:20) Lemon.io - Get 15% off your first 4 weeks of developer time at https://Lemon.io/twist (38:50) Gelt - It’s time to take control over your taxes. Visit joingelt.com/twist now * Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland * Check out Jason’s suite of newsletters: https://substack.com/@calacanis * Follow TWiST: Substack: https://twistartups.substack.com Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups * Subscribe to the Founder University Podcast: https://www.founder.university/podcast

Who's going to win long term? Who will have the best models and which will be the most market share? These closed models or open source models? I think in the future, majority of AI work is going to be based on open source models. I would say 80% of all, like, you know, AI inferencing or like, you know, people building AI applications is going to be based on open domain models. And like some of those will be like fully like open domain. Some of them could be open domain, which are sort of.

supported by enterprises, you know, and, you know, that's sort of like really how the industry has progressed over the last like two decades. It's just really hard to beat open source. This week in startups is brought to you by Open Phone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. Twist listeners can get an extra 20% off any plan for your first six months at Openphone.com slash Twitch Lemon.io. Need to speed up your product development without draining your budget? Hire vetted engineers from Europe at lemon.io. Go to lemon.io slash twist to get 15% off for the first four weeks. And, GELT. It's time to take control over your taxes.

Discover how GELT can help you to manage and optimize both your personal and business taxes. visit join gelt.com slash twist now. All right, everybody, welcome back to the program. Today on the program, we've got Arvin Jane. He is from a company called Glean. What is Glein doing? We're going to find out today. They're trying to get corporations, enterprises, to use AI to help them sort, make sense of,

and search their data. We'll hear all about it from Arvin. Arvin, welcome to the program. Thank you for having me. So tell me, what are you building and why is it important? So think of Glean like Google or chat GPD, but inside your company. It's a product where people can go and ask any questions they have, and Glein will use all of your company's knowledge and data and information to answer those questions to you.

That's what our product is. We are a AI-powered search engine. We are an AI-powered assistant that helps people get more work done. Got it. And so you're not asking the chat GPD. for example, to answer questions or make a marketing plan. This is specifically to search the data inside your enterprise. That's right.

And ask questions against it. So I see on the site, you mention every department that any company could have sales and marketing, etc. Which categories? What's the beachhead market? Where are you being the most effective for your customers? So typically, Glean gets deployed company-wide. Typically, Glean will sell to CIOs.

Our top users do tend to be. engineers, support people, folks in sales, like those are the three biggest user populations that we have. But in general, this is a product that actually is useful to every single employee in a business and therefore we don't go and sell the product to individual departments. We typically go and sell through the CIA. So the CIO is evaluating new technologies and saying this is going to go across all

the verticals in the organization. We need a solution to ask questions. in every department. That's right. Well, that means you're going up against, I assume, like, Intercom, HubSpot, Zendesk for support then. So, or are you sitting on top of those systems? We typically sit on top of those systems. Think of, like, Glean as, you know, an assistant.

Like, it's a layer, it's a connective tissue that connects your knowledge across all of different systems. So while you may be using Intercom or Zendesk as your customer CRM, and your support people use that as a system of record, but when they have a case that has come to them, so they'll open the case in Zendesk or Intercom, and they need to actually resolve it. To resolve that, we're going to actually help them, you know,

find the right answers. And, you know, sometimes those answers may be in knowledge articles in Zendesk, but sometimes they may be, you know, answers that are in some Slack conversations or in some internal Jira issue. And sometimes, you know, the answers are with people, like, you know, that you can actually go on. So Glean will actually help you find those people or that knowledge that sits outside of those

systems and help you answer those questions. So it's sort of like, you know, what's the best example? Well, I mean, like, let's say that, you know, there is a support agent, you know, somebody files a request that my product has stopped working for some reason. And like, you know, what would have happened is that like maybe there's a needed release that got rolled out and there's a bug in the. that and right now people inside the company are actively discussing, you know, that issue in some Slack channel.

It has not made it, it made its way into your knowledge articles here. So when you get a, you know, requests, you know, from your customers, you know, will actually quickly help you tap into like, you know, hey, it's like, has other people run into it and like other conversations, you know, inside the company that would help you sort of figure out a quick answer, you know, back for your customer. How do you deal with the fact that a lot of this data is confidential and maybe not everybody in the organization should see it. There's permissions in each of these systems,

but you're going to index the whole thing. So somebody could ask about salaries in the company or, you know, different things. The language model has been trained on all this data. I assume you're training your language model and all this. What LLM are you using? Yeah. So first of all, like, you know, we are LLM agnostic.

So we can work with GPD4 or Gemini or, you know, which one are you using right now? We use all of them. So, use all of them. Like, typically we'll let customers make a choice, like, what language models, you know, they would like to use. Which one do they pick most often? Actually, like, most of the times,

they will, you know, give back the choice to us. So, like, you know, we get to choose. I think right now we started out with, you know, I think majority of our deployments right now are using GPD4. With GPD4, when you put that data in, how do you know if the model is, the GPT4 model by OpenAI assures you that,

that the customer data does not go in there, or do you have it off-prem? How do you manage that issue with the language models? Because a lot of CIOs and CEOs are really concerned about giving Open AI their data to train it on. Yeah. So see, like, you're absolutely right. Like, if you think about using AI in the enterprise,

first of all, your data inside the company, like has, first of all, it's private to you as a company. But second, within the company, you know, there's governance on that data. Like, not every employee can actually use, you know, all the information that exists. within the company. So Glean actually solves both of those problems. So number one, you know, we're not actually training or fine-tuning models like GPD4. We're actually using

them only as summarization and synthesis engines. The way our product works is that, you know, when you come and ask a question and you're one of the employees in a company, what we will do is, you know, first, you know, based on that question, we're going to use our core search technology and we'll assemble that right pieces of knowledge and information that we think is going to be able to, you know, sort of answer that question that you have. And we will actually restrict you. So, like, we know who you are and what content you have permissions for. So we'll only let you use the information that you are actually individually authorized to use. Now, once you've actually gathered, you know, this information safely, now we will actually take the snippets of

this information and ask an LLM like GPD4 to summarize that information. Do you trust opening AI? So we actually work with Azure, you know, to use GPD4. And the way we work with these model providers is that we have a contract with them where our customers are guaranteed full privacy for their data. Like their data has never logged outside of their own clean environments. And Azure or Google don't have the ability to actually go and train any models on that data. So yes, our customers get full assurance. So you trust them with that function because a lot of CIOs have been a little bit concerned watching some of these. You may have seen, do you see the viral video of the CTO of opening I talking about SORA last week?

And she couldn't answer the question of like what training data was there and she wasn't sure. And it kind of felt like she was lying. I think based most people's thing there. So it does seem to me like the big challenge here, you tell me if I'm wrong, is that using these third party models, even on Azure or Google Cloud, people are nervous, are people nervous about that and they want to move to having, say, an open source one on-prem or just, you know, in their own cloud? Well, see, like, you know, different customers are at different level of sort of both paranoia and security requirements. A lot of the companies today, a lot of enterprises are now comfortable

with storing their enterprise information in the big cloud vendors like Google or Microsoft of a DWS, a lot of like, you know, business technology and systems run in these systems. And, you know, so the trust level, you know, for these, you know, the big three cloud providers is actually quite high. And if you think about it, like, you know, like, see, AI is a new thing, you know,

first of all, already like my business data is, you know, is in these systems, right? And so now, you know, I'm also using some additional, like AI models, again hosted, you know, within Google or Microsoft. So as long as I get those VPC controls, I'm actually comfortable with that. So that's sort of like most of the customers feel that way.

And if you don't press Google or Microsoft, then of course, like, you know, then you typically are running everything on-premises. And we, like, as a company, like we support, like, also like hosting models ourselves. So if a customer wants to use an open domain-based model as, the core LLM, you know, that's, you know, in their clean experience, you know, we also allow them to do that. Yeah, I mean, there was a big instance of a bunch of Samsung employees, I guess, using chat GPT4, and then their source code and other information was then trained into chat

GPT4. I'm sure you've seen that, and I'm guessing that comes up with CIOs. Can you explain what happened there? Yeah. So in that particular instance, the employees in the company, they were actually, like, you know, using the standard chat GPD product, and they were actually pasting, you know, their code inside of that, like, you know,

code or sensitive, you know, documents within the company. You were posting that, you know, within that chat GPD interface and then asking, you know, chat GPD to do some work on it. And so when you actually use these services directly, like these are meant to be consumer services, if you use them directly, you don't have any controls. Like, you know, every, you know, data that you actually put in that system, like, you know, like OpenAI, you know, has, you know, like, you know, is allowed to actually go and train their future models, you know, using that information.

On the public interactions, right? On the public interactions. And that's what happened there. The way to sort of, you know, make sure that, you know, you are not, you know, exposing your private data as a CIO, like, you know, to ensure that your employees are not sending information, you know, to these public, consumer-based products. And that's when you use a product like clean. Because if you use clean, you know, now you have a very safe and secure environment, you

where people can still go and ask questions. And we will make sure that, like, you know, any information that is being sent, you know, to Azure or to Google, like, it's actually following, you know, that contract and that security agreement that you have, you know, from them that they're not going to use that information to train. Yeah. Juggling multiple devices and apps to run your business is a mess. Open phone is here to make it simple by simplifying your business communications with one easy-to-use app. phone has rethought every detail of what a modern business phone should be. And here's the magic.

It works through a beautiful, elegant app on your phone or you can just use it on your desktop, making it super easy to get a business phone number for your entire team. And you know how brilliant open phone is? My teams use it every single day. My sales team loves it. My ops team, they use it all day long. And here's the features that we love. You can create a shared phone number like customer support with multiple employees fielding all the calls and all the takes. text to that one number. At my investment firm launch, we pride ourselves on replying to every single call or email instantly. And Open Phone is the number one rated business phone on G2 for customer satisfaction. So here's your call to action. Super easy. Open Phone is already affordable.

Starts at just 13 bucks a month. But Twist listeners get an extra 20% off any plan for the first six months at Openphone.com slash twist. And if you have existing numbers with other services, no problem. Open phone's going to port them over easy, peasy, lemon squeasy, no extra cost. Head over. to openphone.com slash twist to start your free trial and get 20% off. How do you compete against the native tools, you know, getting more and more robust? So you can't possibly write an LLM that's going to work as good as, you know, the one that's built into, you know, Salesforce eventually. Does Salesforce build in a language model yet or no? So, so typically like, you know, companies like, you know, SaaS companies like Salesforce or at

or any of these systems, they're not building language models. Language models are built by... No, no, no, but they're building language models into their product. And you're not building language models either, right? Well, so it's complicated. So we build language models. We build smaller models, you know, to sort of, you know, build semantic understanding of your company knowledge.

But we, as well as Salesforce and, you know, like other product, SaaS product companies, All of us use APIs to these large language model providers like GPT4 or Gemini to do some work. So typically in that model what happens is that you're basically sending prompts to these models and having these models do some work on that prompt and return back a response. And you sort of create these AI experiences within your application. So now to think about like if you think about Salesforce, they're going to actually have some AI features within their product.

You know, Koda will have some features within theirs. Jira will have some. So, yes, so you're right that, you know, every application in the future, you can imagine that they will have some AI smarts. They might even move to the chat interface, right? You know, and yeah, they may have a chat interface in addition to actually do some of the work that, you know, people do with those products. For example, you know, today, like if you want to create a new issue in GRA, like you

like typically will go in that app and click a button and then fill a form, but you can imagine that, yeah, they may have a chat interface, you know, that allows you to sort of go and, you know, create that, you know, new issue using a natural language interface. So, yeah, so those things may happen in the future, but, but that's not like, you know, what Glean is actually solving part. Like, you know, Glean is, you know, solving a different problem, which is that if you think about your work inside a company,

it happens, you know, across many different systems. I'll give you one user journey as an example. As an engineer, let's have to actually go and build this new technical component. And so that journey for me is going to start with first talking to people. I'm going to be actually having some conversations like this one. Like on Zoom, I'm going to be talking to some other engineers, talking about design choices. I have some conversations in Slack, you know, we'll be talking about like, hey, like, what about, you know, this approach versus that approach? There's actually a, you know, a Jira which actually tracks why am I even building this technical component.

and what were the problems that we're trying to solve. So, you know, first, like, you have all of these different things. Then, like, at some point, I'm going to actually write a design document, like, maybe in Google Drive to sort of describe my design. And then later on, I'm going to actually write code, and that's going to go and GitHub. And so if you think about, like, this whole journey, all the information about this project, like, it actually spanned all of these different systems. So now think about, like, you know, six months down the line.

Somebody comes and asks a question that, hey, why did we use, you know, this programming language, you know, to build this component? Where is the answer? It's not in one place. You get that answer by actually consulting all the knowledge that sits in like those five or six different systems. And so that's where the power of Glean comes in. Like if you think about your work, we are tying together knowledge from all of these different systems in one place. And we give you as a user, like we remove that burden from you.

Like, you know, where should you go and find things? Where should you go ask questions? It's a great place to start. if I was joining the company or I'm the CFO or the chief operating officer and somebody tells me about Project Bluebird and I don't know what's going on Project Bluebird. Hey, give me an overview of project over at Bluebird and it gives me all those. But if I'm not, the question I have, the next question I have there, because that's kind of a cool feature to be able to go across the entire enterprise. So I totally get that. But there's a discussion going on in a Slack room that I don't have permission to, permission for. So how do you deal with that? Maybe there's a Bluebird, Project Bluebird. Project Bluebird. Bluebird Slack room. It's got 20 people in it. But I am the COO, I don't have access to that room. I was never invited. But you have that in the LLM. So how do you let me know that there's a conversation there that I don't have the rights to see because of the way Slack works or Giro works? Like, the CEO doesn't even have a Gira account. They're not having a get up account. So how do you deal with

permissions? Yeah. So two questions. First of all, like, you know, the way our system works, there's no data, none of your enterprise data is actually in the LLM. The LLM is basically just the standard, you know, like language model that is trained on the world's public knowledge. We're not using it to store knowledge. But now, like, the way Glean works is that when we actually connect with all of these different systems, we have actually built an understanding of how permission works in those systems.

So, for example, when Glean connects with CLEAN works. It knows the concept of channels. It knows that certain channels are private, some of them are public. If it is private, it knows who are the members in those channels. So now, you know, we're going to index every single message or conversation and we know exactly who are the people who have access to it. And this is all of this information is stored in our search index. Same for a document on Google Drive.

You know, we know, like, you know, who are the collaborators on that doc? And we store this permission. So now, and so our, and this is unique about the Glean technology, so it's fully permissions aware. Now, when you come in and ask a question in Glean, you have to be first of all signed and you have to present to us your identity, who you are. And we are able to now, you know, retrieve documents from the index, but only work on the documents that, you know, we already know that you have permissions for. So, so building that, like, you know, that's part of our core technology is to understand, you know, like these authentication and, permission models in each one of these individual labs and sort of, and make them, make them work.

So you know, Jason, the COO, can't see Project Bluebird. I'm not in that group. So when I do a search, it won't show you anything. You won't show me anything. Yeah. We won't even tell you, we won't even tell you that, hey, there's some useful information, but we're not, like, we can't share it with you, talk to somebody else. Because sometimes even that is dangerous, like, even letting you know.

Like, let's say, let's say you did a search. Yeah. Who's on a performance improvement plan? That's right. It's like there are seven conversations about performance improvement plans happening with these people in them with these names. You were like, uh, wait a second. Who are the seven people who are in a just the fact that there is a file in and of itself is information. And do these, you know, Slack has pretty robust AI coming on board.

Notion and Coda also have AI built in. Have they built APIs into it or are you just having to rebuild all of that from scratch against their services? or are you doing the search in Slack as if I was logged in? No, we actually returns. We do like our, we do search natively in our system. So the way our system works is that we bring content from those systems and index them in Glee. So as content is being produced, as somebody sends a new message, we get a notification and we'll then use, you know, take that message and index that in our system.

So this is, and this is continuous. This is done in real time all the time. And now when you come into a search, like that search is entirely served from within our system. And that's important because for two reasons. One, like, you know, like you need to be fast. Like you can't actually, in an enterprise, you know, you have 1,000 systems, you know, that you're using.

You can't actually, you know, when an user comes and asks a question to you, you can't actually send message to all, you know, hundreds of them and wait for responses to come back. Yeah, no, search. Search usage correlates directly with the speed of returns, right? That was Google's big lesson, right? That's right. Exactly.

If you make it faster, people use it more. Yeah, in fact, that's one of the things I worked on at Google, you know, I was actually making it fast. But the second thing is, the second thing is also, like, search is a hard problem. It's sort of like magic. Like, you know, you come in, you type tool of words, and, like, I need to sort of now figure out from those 10 million documents,

the one that exactly you're looking for. So there's a lot of work that needs to be done to build a great search. And I think, like, what we assume typically is that, like, each one of these individual SaaS products, like, you know, they don't have so many resources to put on search. Like, we have hundreds of engineers and to actually make search better. It's such a good point, right? Search is like an afterthought for them, or they made you some open source library.

They never update their search. Even like, I've been complaining about Twitter search since Twitter was born. And there was a third party called Summise that they bought to make their search a little bit better. And that was 10 years ago. Because you think about like enterprise software companies, they don't win customers that way. like in Jira and Asana are competing, they're competing on features, not by saying that, hey, my search is better than yours.

So I think that's another thing. So like, you know, to really solve the search problem, you have to do a lot more work. And I'll say just one more thing on this, like, you know, why, you know, thinking about search in the way we think is important. It's really important to sort of take all of your enterprise context. And that sometimes gives you signals on, like, you know, both, you know, what information is

actually relevant and important and to whom. So one example. Let's say that somebody writes a, there is a document that, you know, that talks about benefits. But whenever somebody asks a question in Slack that, hey, like, you know, where's our benefits policy? Like, you know, somebody in HR shares that document with you. So there's a lot of, you know, like these exchanges that are happening in Slack,

which tells you that, hey, this particular document is authoritative for that answer. So Slack's great training data in that way. Yeah. So and then Slack just being one example. Like, you know, like every, if you think about there are these interconnections between, like, you know, how a G-Di issue is created and how it's referenced in Slack conversation. So when you think about your enterprise knowledge, it's like, it's a graph. Like, you know, there's knowledge. There's a lot of different pieces of knowledge and they have all interconnections with them.

And similarly, there are people and, you know, and like people, like, of course, you know, they're like, you know, you're talking about, like, you know, there are engineers that are support people, they're salespeople. and we build these learnings that, you know, engineers are actually, you know, clicking on this, you know, or using this document a lot more than salespeople and vice versa. So all of those learnings, you know, is sort of what enables you to find out what is more relevant information for whom. And that's the core of like, you know, what glean is. And that's why it's so important to actually have that full enterprise-wide view of your people as well as your knowledge.

Right now, startups have to do more with less. We all know that. it's rough out there, folks. So if you need great tech talent, but you don't have the time to interview dozens and dozens of candidates, you need to check out lemon.io. Lemmon.io has thousands of on-demand developers to choose from. And these devs are vetted experience result-oriented, and they charge competitive rates. Great developers can be incredibly hard to find, and when you do find them, it can be hard to integrate them into your team. Lemon.io handles all of that for you. Startups choose Lemon.io because they only offer handpicked developers with three or more years of experience

and strong portfolios. In fact, only 1% of candidates who apply get in. And if something ever goes wrong, lemon.io will get you a replacement ASAP. You know what? A bunch of our launch founders have worked with lemon.io. And they've had great experiences, which is always good to hear. Go to lemon.com slash twist and find your perfect developer or tech team in 48 hours or less. Go to lemon.

com. Or even a tech team in 48 hours or less. And twist listeners get 15% off their first four weeks. What a deal. Stop burning money. hire developer smarter, visit lemon.io slash twist. What's really great as a byproduct, and you can tell me if any of your customers are asking for this,

there is a concept of compliance and legal reviews. So, for example, people think, like, their DMs on Slack or their emails, even if you delete your emails, those things are stored if you have the settings done properly, like in Google Docs or Microsoft Teams. And so because you ingest everything, you do have the ability to do a god, mode where the compliance could say, hey, did anybody say this? Let's say it's insider trading. Did anybody share Project Bluebird?

Let's say Project Bluebird was an acquisition. Did anybody say the word Bluebird? And you could actually see across all documents. Does that exist like a Super God compliance mode? There is a compliance mode which is highly restricted and it's available only to your governance and legal teams like for exactly the use cases like that. you know, e-discovery or, you know, also like for privacy compliance, like, for example, you know, a big use case, you know, that that's out there is, you know, you have a, you know,

a, you know, a previous, like an ex-customer or an ex-employee, you know, who comes and tells you that, hey, you know, delete all my data, right? You know, and then you have to sort of, you know, there are laws that actually required to do that, like, you know, and so how, but how do you even figure out, like, nowhere is all that information. Yeah, where is that data? Facebook had this issue because Facebook had been doing backups of backups of backups. And that's why when you delete your account, people are like, oh, you know, they say it takes 30 days. I think that's because they have all kinds of mirrors and mirror images of data so many different places.

They want to be thoughtful and thorough about it. It would seem like a CEO God mode would be one of the great features of being able to look across this whole amount of data. If you're working at a company, you should just know by default, anything you do on your last. App top is your companies and every email you send never use corporate devices. Yeah. To order from Amazon or do private communications. Gosh, it's 2024. I don't know why I have to say this to people, but I'm shocked that people will,

because I'm on the board of so many companies or investments, some story will come up that people were sending things to each other on Slack or teams or doing something on a corporate device that is completely insane that you should not be doing. That's right, yeah. And I think from our perspective, like, you know, we, like, you know, we help companies, you know, from a compliance perspective there. But Glean is actually not a system of record. So that's not like, you know, another system that you do to worry about, like in the sense of when you need to delete data for somebody, like, you know, you don't have to actually go and explicitly go and delete that in Glean. Well, you do have to get rid of the data and Glean if it's indexed it, right?

Because you have to be indexed? We stay in sync with the actual system. So if deleted in Slack, it's automatically deleted in our system. Wow, that's super complicated to take care of all that, huh? Yeah, that's where the complexity is. But actually, it's an interesting thing. Like, you know, you talk about AI, you know, like everybody wants AI. And this is one of the key problems that businesses are running into, which is, like, look, you know, we have all this information in our company.

And yes, like, you know, we've set some rules, you know, permissions, but you don't always get it right. Like, you know, oftentimes, you know, there will be some documents somewhere that, like, you know, sensitive and the person in HR. like they didn't know how to set correct permissions. They made it open to everybody in the company. And you start living with that, like, because nobody can find anything anyways. Like, who cares? Like, you know, those talks somewhere that, you know.

Let me ask you. Yeah. We know, get finished. I was just going to say that. So that's a big issue today. Like, you know, with like AI, because now AI does all of that work for you. Like, you know, like in this new world,

you just get to ask questions, right? And there's this AI, you know, like, for example, our product is connected with all of your company information. And it's going to actually answer questions back for me. So it sort of makes, you know, these governance gaps, you know, like, you're going to pay for it now. Like, they're going to become a big problem, problem. And that's one of the things that we hear a lot from, you know, CIOs, you know, they feel AI is powerful, but they're also scared of it. When you see, I mean, you must have seen the Devon demo last week, the AI coder going out and, like, doing jobs on its own.

Did you see that last week? I didn't see it, but I've sort of, you know, seen, like, things like that and heard discussions about it. Yeah. So, you know, now that you're indexing the whole company, you're watching all this data and code and customer support tickets and sales, all occurring. It would seem to me that you understand, you know, what a salesperson does all day, what a coder does all day and all their activity buzzing around. Yeah. So that's great. I mean, you understand who the most productive employees are on a certain level. You could tell me who's working, who's making the most commits, and this exists already in general. But, you know, you could tell me, hey, this person's work hours, they work three hours today according to the data we've seen.

So is there some idea here of looking at productivity? There are certain apps that people are using to monitor their own productivity. Then there's like people tracking their hours. But it does seem you could tell me, hey, you know, this person hasn't done anything for four days. I guess they're on vacation or, hey, this person is putting in, they're dropping, you know, data into all these different resources 12 hours a day. They're working 50% harder than the average person. Well, I mean, so, you know, we have, you know, we didn't start our company, you know, with the goal of sort of building these, you know, analytics and, you know,

or like some people analytics in some sense, you know, our goal, you know, has always been to help people, like, get work done faster, like make them more productive. But people are analytics is a really fascinating topic. So it is. And so the data is there. Like, you know, say with or without us, you know, like that data is there. And you write that, like, you know, when you bring it all together, like, you know, how you bring it in clean, you know, like, you know, somebody can actually run those

analytics on our platform and get and gain insights faster. But I would say, like, you know, like, we haven't really seen, like, people talk about it, but I think, like, you know, I think they, like, we haven't seen, like, you know, actual, like, attempts, you know, where somebody's tried to actually build reporting like that, you know, using the data on our platform. You know, the negative interpretation of it is employee monitoring. So you can, you can see employee monitoring. And then there's employee productivity. That's right. And, you know, they're just, if you're doing, if you're running a call center, you really need to monitor it because people might say something stupid to a customer and somebody who's on a call center all day, they expect all of those interactions to be monitored. Now, a higher level knowledge worker, a sales executive developer, they don't expect it, but they might very much want to be productive.

And so I know people who run productivity software and you have it on your iPhone, right? It tells you which were the most popular apps. And I've looked at it a bunch of times and I'm just like, I want this. I know like six people in my organization want it, but I bet there's like 10 who are absolutely paranoid about like that day to being there. But what's important for people to understand is with AI, with a system like Lean or any other system, the byproduct is your collective work is going to be

in a database somewhere, which means you can really study it and figure out, what is this person doing in our organization? Do they need to be here or do they need a raise? Is this job need to be eliminated, or do we need 10 more of these people? Or do we need to study this person? That people analytics to me is incredible.

Yeah, I think that's really powerful. And actually, like, you know, but I would say one more thing, which is, you know, today, you know, part of it is that, you know, you can have, like, you know, a few people in the company that could sort of do these analytics on an organization-wide basis. But part of it is, like, what about you yourself? Like, you know, like, you know, you can go and clean today and say, like, hey, tell me,

you know, where I spent my time last week. Right. I was going to tell you, like, you know, like, you know, like, if you're meeting a lot, like, you know, like, for example, I can ask and clean, like, how many hours of meetings did I have last week? And it has access to that information has been answered that back to you. So part of it is that, like, like, you know, like, you know, like, you know, like, how can we help you as an individual sort of have more insights into your own work.

And so we think more about that, like, you know, from that perspective. And like, so like one of like one of the, one of the very popular use cases or like popular questions and, you know, the people ask in Gleen is like every Monday morning they will ask for like summarize, you know, all the work I did last week, you know, because they actually need to share, because they need to share that information with their manager or with their team, like, you know, posted in like, you know, whatever their scrum, you know, notes. So you can do that and clean and go through your Jiras and your Guitors and your Guitous and your Slack conversations and sort of give you like a really nice summary of what you did last week. So there are the analytics or summarization that you can actually bring to each individual for themselves.

And you know, like you sort of start to like, you know, bubble up like, you know, as a manager of a team, you can ask the same question, what did my team do last week? And it will actually do it for you. Like as a manager, you'll be able to sort of get that summary, but only with. information that you as the manager actually have access to. So you could actually, like, so if there was an employee doing something, you know, writing, you know, working on a dog that they're not shared yet with the team,

you know, the manager won't get to see that. But, but yeah, so there are use cases like those. We are actually, you know, going from the angle of like helping each individual with their work and, you know, with their own sort of productivity. We haven't seen that much of, like, you know, the, like what you mentioned,

which is that they could be. I mean, I have my own little ways of doing it. Sometimes I go into Notion and it will show me, I think I'm the administrator is why it shows it to me all of the changes in the database. Yeah. And I click on it and I see Bianca, Andre, Heidi, like coming up over and over again. The three people on my investment team. And I'm like, wow, they're super productive inside of Notion all day long. And I noticed that.

Like, oh, wow, they're really taking good notes. And sometimes I'll just take a look at the document. Now, all those documents are public. Anybody could look at them. But it's really nice to see the part. pulse of the company, right? Yeah. And then there was this really cool reporting that I got just by opening up Slack's admin

to add somebody. It'll show you how active each person was in the last 30 days. So I just told you like how many messages they sent, how many they got? And then how many days out of the less 30 they logged in? And I was like, shout out to these, you know, 30% of the company that logged in, you know, 28, 29 or 30 out of 30 days. Like you can take a day off from it. I would never not check my Slack.

that to me would be crazy as the CEO. Yeah. That's the one I actually like a lot myself. I think it does tell you a lot about the company. Like, you know, when you sort of look at these data. Or that you look at the bottom and you're like, like one time I was like, oh, my God, this person was logged in like 14 out of 30 days. And I was like, oh, they took two weeks off.

They had a honeymoon or something. Totally fine. That's the time you want to turn it off, right? But then other times is like, should that person have turned off their slack for two weeks in them or whatever number of days? It might be time to have a conversation about that. Let me ask you about search. You are at Google.

In five years, will people be doing search engine searches or will they be doing chat GPT searches or like chat interface searches? I'll take open AI out and I'll take Google out since you work there. Search engine versus chat interface and just having a conversation, which will be the majority of, you know, users searching for knowledge, which will be the majority in five or ten Well, I think in five or ten years, there won't be two different interfaces. There's only one. You know, because, like, ultimately, like, you know, what are you doing?

Like, you know, you have a question. You need an answer. Sometimes, you know, sometimes your question is about research. Like, you know, you want to read a document actually, like, in response to, like, what you're looking for. Sometimes, you know, you're looking for a one line. So it will just move to a chat interface. No. We won't be on this, like, 10 blue links.

No, that's not what I said. Like, what I'm saying is the, what I'm saying is that there's not. only one interface, but that interface, you know, is, you know, is adaptive, is rich. Like, it, you know, takes, you know, what's the kind of question that's coming in and, like, you know, appropriately give you the right answers to that. If you think about, I think, like, I think that there isn't actually this dichotomy, like, you know, that, you know, we make off right now. Even in Google, for example, well before, well before, like, this whole generative answers

and, like, the conversational interface, that you could go and ask, you know, in Google, like, five years, you know, from, you know, like, five years, you know, back. you could ask the question, hey, what's the temperature, like in, you know, what's the weather like in the Sontas today? Sports, or weather, time, yeah. Currency exchange, stock ticker price, and it just gives you the answer right. It will give you the answers and it will actually also tell you,

like, you know, other interesting questions you may actually ask. So this has been a progression, right? You know, where I think the search interfaces will sort of be like that, where, you know, you're going to understand the intent of the user and what they're trying to look for, sometimes you can actually give them, like, you know, resources, links, you know, to go on, you know, they should go and read more details.

Sometimes you're going to see summaries or, like, you know, quick answers on it. Are you grinding hard to grow your business? I bet you are. You're listening to this week in startups. Of course you are. But don't let your hard-armed profits slip away because of overpaying on taxes. You need to check out gel.

GELT is the secret tax weapon trusted by savvy founders and CEOs. Their elite solutions will transatl, form. How you handle your taxes. You can integrate your personal and business tax planning with one provider. They have tech-driven efficiency that simplifies tax management and in-house advisors with startup and business expertise. This means no more overpaying your taxes. You're going to save time and these expert opinions will boost your financial health and help you grow your business. Geld helps to stay compliant so you can focus on your startup's mission with year-round tax advice, personalized strategy sessions, and a comprehensive.

tax library for continuous education. When you optimize your tax strategy, you optimize your competitive edge. So here is your call to action. It's time to bring Gelt's elite tax team into your business. Visit join gelt.com slash twist and get 15% off your first year. What a generous offer. Thanks, team Gelt. Join GELT.com slash twist to get 15% off, transform your taxes from a liability to an asset. That's 15% off your first year at join gelt.com slash twist. They had the one box snippets, all the stuff. But it's going to just move to an answer. So what does that do to the cost per click business model of the internet? Because right now, if a search engine, I'll just say any search engine, it could be any of the popular ones, gives you, does it answer your question?

And it forces you to click. Some number of people will click the ads because the ads are generally answers to the question. You know, how much does the latest Volvo have, you know, is there a convertible Volvo? And it might be an ad for convertible volvos or use convertible Volvo's. But if we're just going to just answer people, hey, yeah, the Volvo, you know, made four different convertible models. There are one active and these are the three historical ones. Okay, I'm done. I don't click on any of the ads. So what's going to happen to the cost per click model over the next 10 years as AI just answers everybody's question? My view is that I don't think, I don't think like, you know, the models sort of disappear. Already in Google, like, you know,

there's a concept of cost per click. Google always also like, you know, we'll talk about like cost per conversion. There are all these different sort of, you know, degrees of like, you know, like how you're actually ultimately driving a sale and like, you know, you as, as, you know, the sort of facilitator in that, like, you know, what is your cut off, you know, that transaction. So there is, you know, there are cost per impression, there's cost per click, there's cost per conversion. And so I think, like, what will happen is that, like, in the future, like, when you ask a question that has commercial intent and there is, there are four different commercial parties, you know, that could actually, you know, all provide you with an answer.

They're going to compete and, you know, like, you know, the search engine may show like an answer coming from one of them and potentially like, you know, there is like further follow-ups, you know, that you take you to those sites and, and then you get like, you know, higher, higher. So there could be a different type of funnel or modality for monetizing answer. So you give the answer about this Volvo. Yeah. And at the end it says, would you or, you know, follow-up questions, where can I, buy a Volvo. Where's the closest Volvo dealer? Are there any incentives for buying a Volvo?

Who can lease me a Volvo? Or they use volvos? And all those, if you click them, could include either cost per click links or it could put you into a conversation. You know, that's the ultimate. Hey, yeah, what kind of car are you looking for? What's your budget? And then give that lead to yeah, I think we're to summarize it like, you know, very, very simply, I think like Google is getting paid, you know, because, you know, that's where users are going and seeking those answers. So as long as that stays, that means, you know, they should be getting, you know, they're cut off, you know, that. How long were you at Google for? Was there for? I was there from, like, end of 2003 till 2014. So what? Like, wow. You were there

during the early days. You, you haven't been there for 10 years. So what do you think of all this, you know, brouhaha? this Donnybrook around the Gemini project and all this woke DEI stuff that was included in it. How does something like that happen at a big organization? And what do you think Google's chances are of kind of being able to release product faster? Like, how did it get to this point? Because it did seem like Google was so efficient in the period you were there in just giving us products that solved on problems as consumers.

And now it seems like they're doing something completely different. Well, my take on just the AI models first, like, you know, from Google, is that I personally feel like, you know, they're actually in a strong position. Like, you know, whatever goes wrong in the model, like, you know, they get more attention than anybody else. But if you think about, if you think about Gemini, like actually, you know, it works, you know, it works really well, like as an AI model. You know, they also have, I mean, like, you know, if you think about Google, like, you know, they're the best set of engineers, the most AI talent, like, by far. even now, you know, they have the world's biggest data centers. They got all the machines. They got all the money.

So I think the, I think the calls for like, you know, the, the doom, like, you know, scenario. I think, like, it's in my opinion, you know, it's sort of, like, I think it's, like, it's fun for people to talk about. But I think, like, I feel like the company is in a really strong position. Yeah, you think they can still win. Yeah. I think so.

Yeah. Yeah. Yeah. It just seems like maybe they've got. maybe too much process. Like, it used to move much faster, right, when it was a smaller org? Part of it is, yes, like, you know,

they need to organizationally make improvements. But part of it is also, like, you know, the burden of, like, success. I mean, I think about, like, you know, they could not, like, as a company, you know, like, whose core business is to help, you know, people find information and correct information. Like, you know, like, they were sort of rightfully cautious

about, like, not putting these models, you know, that hallucinate, like, in front of people. Yeah, giving the wrong answer is really anti-Google's mission. And it does seem like this is why Apple hasn't released a ton of AI features is because they also like to have a lot of fit and finish and polish on their products. And so... And like in an upstart, like, you know, they can launch whatever. And that is sort of like, you know, like, you know, in reality,

like sort of what is like, you know, like, you know, cause them to be a little bit on this backseat. What do you think is going to win? Open source. Elon just open source grok over the weekend. obviously Facebook and meta, all their models are open source. Apple is working on an open source image editor, generative product,

and even Open AI started as open and then when closed, who's going to win long term? Who will have the best models and which will be the most market share? These closed models or open source models? I think in the future, majority of AI work is going to be based on open source models. I would say 80% of all, like, you know, AI inferencing or, like, you know, people building AI applications is going to be based on open domain models. And, like, some of those will be, like, fully like open domain.

Some of them could be open domain, which are sort of supported by enterprises, you know. And, you know, that's sort of like really how the industry, you know, has progressed over the last, like, two decades. Like, I think it's just really hard to beat open source on any. on any technology, like the momentum you get, you know, with it. So that's sort of what I feel like, you know, it's going to happen from a who's going to happen. How far ahead is opening eye, if at all? Do you think opening eyes 4.0 is much better, 10% better?

How much, how big is there a lead if you were to say in the number of months or quarters? And then how soon before open source and, you know, everybody else catches up or exceeds them? Yeah. Yeah. So on text-based models, like I think right now they are testing internally. Like, you know, it feels like, you know, there's still ahead, but the gap has been closing every quarter. It's actually not significant right now. Like, it's not significant in the sense that I think now, like our team, for example, is, you know, continuously thinking about, like, you know, we need to actually use, you know, the smaller, faster models.

because they're faster, because they're cheaper, and because, you know, like, the, you know, it's sort of like, you know, how you design, like, you know, sometimes you can, like, you know, like if you make 10 requests and like sort of triangulate to interesting things, you can actually get a better response and like making one like costly request to a costly model. So there's already in that, in that domain where it's not straightforward anymore,

like, you know, to decide like what's the right model. So like the things are getting been quite close. how do you define AGI? You must talk about this and think about it. General intelligence, what's the test that you put on it? I mean, obviously, we have, you know, all kinds of the classic tests, but what do you think would be a reasonable definition of artificial general intelligence that we could all agree on or you might agree on?

Well, like, you know, in an enterprise, like, you know, when you feel like, you know, there is a person today, you know, they have a role, you know, to perform, and, and that role is now

completely taken over by an AI agent. And, and, and I think that's sort of what I, you know, what's our definition, like,

you know, within our context, but I would actually also tell you, like, you know, we, we talk about big things, and I think we're,

we're far behind, like, in terms of, like, you know, where real technology today is, you know, people talk about having co-pilots. I, you know,

I feel like, you know, that's a big bar, like, you know, as a word, you know, to describe the technology that we actually have in front of us today. I mean, like, you know, there's, this, this is really powerful, but there's a lot of work to be done. I mean, like, you know, it's a co-pilot. We're in the co-pilot phase. And the next phase will be. I feel like it's not the, we're not in the co-pilot phase.

Maybe just developers are. I think the, like, you know, you are getting like, maybe, you know, 10% of what an assistant would do for you. Like, you're not getting co-pilot is actually a lot more, more. like a lot more stronger than, like in an assistant, like you think about your own personal life, you can have an assistant,

you can actually have somebody who can replace you as a co-pilot. I think the AI technology is actually, like not even at a place where they can do a better job than your EA right now. Interesting. So, yeah, I would agree with that.

Yeah, it feels like, I like your definition. One of the employees at work gets replaced and you don't know it's an AI. I like that. Yeah, pick a random person in your organization, replace them with an AI. And when you talk to them in Slack or you talk to them in, you know, GitHub or whatever,

you talk to them in a Google Doc, yeah, you can't tell the difference. That would be a pretty good one. Yeah. I feels like we're making, you know, steady progress there. But yeah, it feels like we're in the co-pilot era. But yeah, you're right. I never thought about it that way. You wouldn't give them, you wouldn't hand them control of the plane right now.

You wouldn't go to the bathroom and let them fly the plane. That's right. She'd be like, I'm going to say here and watch you fly the plane. I'm not quite sure. I trust you. Yeah. But at the same time, like, there's this real value. Like, you know, I think, you know, even with Clean, like, you know, we, we want to be that

assistant, you know, for everybody who works. And I think we're bringing, you know, like a great deal of assistance. But it's a long, you know, it's a long road. Like, you know, like, you're going to be working on. How do you charge for it? Is it per seat? Is it by Datasaurus? Or do you just look at, like, per seat?

Per seat, 10 bucks a month or something, 20 bucks a month? Yeah, a little bit more. A little more. Oh, okay. Yum, yum, yeah. But that's the model. Like, you can connect that, yeah. Got it.

So if a thousand people, a couple hundred dollars a year per person, it's probably, and that's what you're going after. Mid-sized organizations need this. It can't be like a 50-person company, maybe not worth the choose ain't worth the squeeze, or are you going after the mid-size? We are focusing on companies from like a few hundred people all the way to the largest enterprises of the world.

The need is quite universal, but like, yeah, from a focus perspective, Like, you know, we are, like, majority of our business is actually an enterprise. And it takes a while to get all these services into the database, right? It's got to take a couple of weeks or a month to tweak everything and get it all plugged in, right? No, Glean is actually very turnkey. That's actually one of the big requirements for when we started our company was, like, you know, we can bring, you know, glean to, let's say, 2,000-person enterprise, you know, the, you know, and like, you know, it's up and running, you know, within, within a day. Oh, that's pretty great.

And, yeah, because I think, like, the, I think one thing that helped in that has help is that, like, you know, like the new, like, you know, SaaS-based IT environments are actually quite accessible. And you can be up and running pretty quickly. All right, listen, everybody check out glean. You have glean.com. Yeah, glean.com. All right. Good domain. Pretty great domain.

It's a million dollar. Well, maybe half million dollar demand right there in my estimation. It's in the dictionary. Great job. And everybody check out glean. time, bye-bye. Thank you so much.