a16z Podcast - When Will AI Hit the Enterprise? Ben Horowitz and Ali Ghodsi Discuss

Episode Date: October 6, 2023

Today’s episode continues our coverage from a16z’s recent AI Revolution event. You’ll hear directly from a16z cofounder Ben Horowitz and Databricks cofounder and CEO, Ali Ghodsi as they answer questions around AI and the enterprise, plus their perspectives on open source, whether benchmarks are BS, and the scramble of universities to take part in the very wave they kicked off decades ago.If you’d like to access all the talks from AI Revolution in full, visit a16z.com/airevolution. Resources:Find Ali on Twitter: https://twitter.com/alighodsiFind Ben on Twitter: https://twitter.com/bhorowitzCheck out Databricks: https://twitter.com/databricks Stay Updated: Find a16z on Twitter: https://twitter.com/a16zFind a16z on LinkedIn: https://www.linkedin.com/company/a16zSubscribe on your favorite podcast app: https://a16z.simplecast.com/Follow our host: https://twitter.com/stephsmithioPlease note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

The AI revolution is here. But adoption is not always evenly distributed. While we see consumers jump to try the latest AI apps... We haven't seen anybody with any traction in the enterprise. But awareness is not the issue. In fact, the Financial Times reported that nearly 40% of S&P companies mentioned AI in their earnings last quarter. So...

Why is it so hard for enterprises to adopt? up generative AI. As companies wake up to the value of their proprietary data sets, a whole new set of questions emerge. Are the enterprises right about not wanting to give their data? Like, is that a correct fear? Can they build a better model? Do they really need it to be accurate?

Plus, with OpenAI recently dropping ChatGBTGBT Enterprise, will all of this change? Today, you'll hear directly from A16Z co-founder Ben Horowitz and Databricks co-founder and CEO Ali Gottzi as they answer these questions and more, including their perspectives on open source, whether benchmarks are BS, and the scramble of universities to take part in the very wave that they kicked off decades ago. Plus, you'll get to hear firsthand how Databricks' recent acquisition of Mosaic ML fits into all of this. This episode continues our coverage from A6CZ's exclusive AI Revolution event from just a few weeks ago, where we house some of the most influential builders across the ecosystem, including the founders of OpenAI, Anthropic, Character AI, Roblox, and much more. Be sure to check out the full package, including all the talks, also in full, at A6CZ.com slash AI Revolution.

As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16c.com slash disclosures. All right, so going to Generative AI, one of the things that's been interesting for us as a VC is we see all the kinds of companies, some with amazing traction. But every company that has traction is in a category like selling to developers or selling to consumers or maybe selling to like small kinds of, you know, law firms or these kinds of things. But we haven't seen anybody with any traction in the enterprise. Why is it so hard for enterprises to adopt generative AI? Yeah, so look, enterprises move slow.

That's one in general, right? Which is the beauty, which means if you crack the code and you get in, it's harder for them to throw you out. So you're going to have an awesome business. If you do crack the code and you're in, it's more robust. You're not going to lose it overnight. So that's one. They just move slower.

Second, they're super freaked out about their data, privacy, security of their data. But then in general also, I just realized, everybody isn't talking about data for 10, 15, 20 years. I just realized how valuable my data actually is. So maybe I'm actually, I'm sitting on a treasure trove and I'm going to be super successful. So I'm going to be very careful with this.

Like now I finally realize how valuable this data set I have is. So I definitely don't want to give it to you or you or you. And should be careful about this. And then there's all these reports about it leaking, you know, data leakage. Like, oh, you know, suddenly the LLM is spitting out your code. So they're freaked out about that as well. All of these things are slowing it down and they're kind of thinking through it. That's like just one set of challenges that enterprises have.

Second challenges enterprises have is that, hey, for a lot of the use cases, we need the data to be accurate, we need to be exact. So there's a lot of use cases where... Are they right about that? Do they really need it to be accurate? I think it depends on the use case. They're just being cautious and they're being slow as they are in the big enterprise. And then there's the last aspect which people don't talk about,

which is there's like a food fight internally at the large enterprise, which is... Who's fighting? I own generative AI, not Ben. And then you go around say, hey, I own generative AI. And it's like, no, no, no, my team is building genera. So there's this, you know, food fight internally of who owns it. And then they slow each other down.

So I say, hey, don't just Ben, because he's not handling data the right way, but I'm building my gen AI. And it's unclear. Is it IT that owns Gen AI? Is it the product line? Is it the business line? So there's like huge politics going on inside the large enterprise. They want to do it, but there's all these hurdles in the way. And the prize is huge.

Whoever can crack the code on that is going to create an amazing company. Are the enterprises right about? not wanting to give their data to Open AI or Anthropic or BART or whoever. Like, is that a correct fear or are they being silly and they could get so much value by putting their data in a big model? They can, but I think also a lot of the leaders, you know, by the way, I get to talk these days to the CEOs of these big companies who previously were not interested in what I'm doing.

I would be talking to the CIO, but now suddenly they want to talk. Like, hey, I want this generator of AI. I want to talk strategy of my company. Let's talk. And we have this data set. it's super valuable, like, you know, we've got to do something with it. And it's generative AI seems interesting. What do you want to do with it?

And one of the things that's really interesting that's happened in the sort of brains of the CEOs and the boards is that they realize maybe I can beat my competition. Maybe this is the kryptonite that will help me kill my enemy. I have the data with generative AI. I can actually go ahead and do that. So then they're thinking, well, but then I have to build it myself. Yeah.

I have to own that, right? I have to own the IP of that. I can't just give away that IP to enteropic, Open AI, anyone. Like, it has to be completely proprietary. I want to do that myself. By the way, I have a whole bunch of people here that are lined out outside of my office

in different departments that are saying they actually will do it and they can do it. So we're trying to figure out which of them I should give it to. So this is what's happening internally right now. Interesting. And from a strategy standpoint, when you think about it, let's say you had a big data set, be it like a healthcare dataset

or some kind of security data set or Nielsen's dataset, can they build a better model themselves for that with their data or if they took their data and put it in one of the large models, would that always beat what they're doing? Yeah, so this is why we did the acquisition of Mosaic. Yes.

You can. It's hard. It requires a lot of GPUs. And the Mosaic guys just figured out how to do that at scale for others. You want to build your own LLM from scratch? Come to me, I know all the sort of, you know, landmines and so on. It just will work, trust me. And so they can do it.

And yeah, they've done it for large customers. They can do it. Still, it's not for the faint of heart. Still, it requires a lot of GPUs, it costs a lot of money. And it depends on your data sets and your use cases. But they're having a lot of success doing it for, you know, really large enterprises. They'll train it from scratch for them, and it just works. And the result that they get with Mosaic, so I'm doing it.

So the good news is it's all mine. Nobody can touch it. It's my data, screw off, competitor. But is the bigger model such a bigger brain anyway? that I could get a better answer if I put that same data in the big model, or is a kind of mosaic-tuned enterprise-specific

dataset-specific model going to perform better? Like, how do you think about that? For specific use cases, you don't need the big one. First of all, you can build the big one with mosaic and with data works. Just how much money do you have?

We're happy to train you an under-billion-parameter model if you want, but, A, it's going to cost more to use it. Even if you have all the money to train it, it costs you a lot to use it. So when you're using it, then you're doing inference as it's going to cost you more. And how do you think about the diminishing returns on kind of like a data set against like how many parameters versus how much data do you have? Does like a bigger model just start to be diminishing returns,

both in terms of latency, expense, everything? Yeah, I mean, there's a scaling law. You need to scale. If you're scaling the parameters up, you kind of have to scale the data with it. Right. You know, so you just have to do that. So if you don't have that, then, you know, just scaling it. you're not going to get the bang for the buck.

You still get improvement if you increase the parameters or if you increase just the data in any one of these dimensions. But you're going to pay. You're going to pay. It becomes inefficient. Yeah, it's no longer perito optimal, so to say. But look, what I'm saying is this. Four enterprises that have specific use cases, which they all have.

When they come to us, they don't say, hey, I would love to have an LLM that could, like, kind of answer anything under the sun, you know? They're saying, hey, this is what I want to do. Like, I want to classify this particular, you know, defect in the manufacturing process from these pictures really well. And there the accuracy matters. Like every ounce of accuracy that you can give me matters. And there you're better off if you have a good data set to train.

You can train a smaller model. The latency will be faster to use it later. And it would cheaper to use it later. And yes, you can have absolutely accuracy that beats the really large model. But that very model that you built can't also entertain you on the weekend and answer physics question and help your kids do their homework. Why do you think it's important for you, Databricks, to build a very large So the bigger models, if you follow the scaling laws, are more intelligent, assuming if you're okay with paying the price, and you're okay with, you know, you have the GPUs, and if you can crack the code on how to fine-tune the bigger model, which is kind of the holy grail right now that everybody's looking at in the research community and in the field and the companies and all that.

And when you say fine-tune, kind of get more specific. Yeah. So take an existing really awesome foundation model that exists and just modify it a little bit to be able to become really good at some. other task. And there are many different techniques to use to do that. But right now, nobody has really cracked the code on how you can do that without modifying the whole model itself, which is pretty costly, especially when you want to serve it, when you want to use it later. Right. Because you have to go through all the... Yeah, if you have thousands, if you made a thousand versions of it, that's good at thousand different things. If you have to load all of each of those thousand into the GPUs and, you

know, serve them, becomes very expensive. The big, I would say, holy grail right now that everybody's looking for, there are techniques where you can just do small modifications where you can get really good results and you can just stack on a little bit of additional, you know. He's just that part of the brain. Exactly. Just add this thing and there are lots of techniques. There's like prefix tuning, there's Laura, Q Laura, so on and so forth. The result, none of them really are slam dunk. It's like awesome, we found it, but someone will. Once you have that, then it seems in the future in a few years, the ideal would be a really big foundation model, that's pretty smart,

and then you can like sort of stack on these kind of additional tuned sort of brains that are really good at this specific classification task for manufacturing errors and this other, you know, translation tasks. And they'll be compute efficient and energy efficient for just dealing with that task at that point. Exactly. And then you could also, you can load up your GPUs with that one intelligent brain, that one giant model, and then you could specialize it. Yeah. But to be clear, no one's really done this yet.

That's what I think a lot of people are hoping to do. And it might not be easy to do that. And meanwhile, we're having lots of lots of customers who want to have specialized models that are cheaper, smaller, and that have really high accuracy and performance on that task. Yes. I can just say it. Like at Databricks, so we bought Mosaic.

I did not unleash our Salesforce and go to market of 3,000 people to sell the thing that we bought because we just can't satisfy the demand. Like, there's not enough GPUs. So you won't even let all your guys sell it. No, I'm not even letting all the customers buy this thing. Because we don't have the GPUs and we don't have... If we unleash up, every company wants to do this. Everyone wants to, okay, okay, I have a thousand things I want to build.

Can you help me do that? In this context, sort of how much do you think these use cases will fragment? So you talked about, okay, I want it to be good at doing my kids' homework. I wanted to be my girlfriend. So how much do you think the use cases, the very specific use cases, will fragment? And kind of within that, like one of the things that we're finding is getting the model to do what you want is kind of where the data advantage is from the users in that if I want it to draw me

a certain kind of picture, that's a lot of conversations to do that. And so whoever is drawing those kinds of pictures will be good at that, but then there may be another model that wants to draw memes, but that thing that's drawing the pretty pictures can't draw the memes because that involves words and all this other stuff that it hasn't, it just hasn't learned to get that out of the humans and map it into its model.

So how much do you think we're going to get tons of specialization versus no, no, no, once the brain gets big enough and we do these fine tunings, that's going to be it. It'll be like AWS, GCP, Azure. I think the answer is closer to the latter. There's going to have lots of specialization. But having said that, it's not a dichotomy in the sense that maybe they're all using some base models that are underneath

common to many of them. You're not starting from scratch every time. But you're tuning it up a certain way. Yeah, look, I think in some sense, the industry, like right now there's, people are looking at the wrong thing. Right now, it's a little bit like 2000, and the Internet is about to take over everything, and everybody's super excited. And there's one company called Cisco, they build these routers. Obviously, that's like the biggest thing. And the most important thing is whoever can build the best routers is going to dominate all of Internet forever.

Yeah. Right, it's like that's the thing. The future of mankind is going to be determined by who builds the best routers. And right now, this company, Cisco, is the best one by far. It's obvious, what I'm saying. Cisco in 2000, I think, was worth half a trillion dollars at its peak. And people were talking about it's going to be a trillion-dollar company. It was worth more than Microsoft.

So I think it's a little bit right now like that. Who has the largest LLM? Obviously, whoever can build it the largest one that can train it the most, obviously will own all of AI and all the future of humanity. But just like the Internet, someone will show up later and think about Uber rides and cab driving. And someone else showed up and thought about, hey, I want to check out my friends on the Facebook and so on.

And those end up being huge businesses. So there's these applications, which many of them are obvious. Like, you know, Mark talked about it in his, you know, AI will save the world. You know, the lawyer, the teacher, they're like, there's lots of use cases. Everybody knows. Probably there's going to be a lot of value in those. And no, it's not just going to be one model that Open AI or Databricks or Anthropical or someone builds. And that model will dominate all these use cases.

No, it's a lot of things will need to go into building the doctor that you trust that will be able to tell you, you know, how to cure you. your loved ones. So I think that those are the companies that we will build in the future. And I think there's going to be a lot of value in those, obviously. And yeah, there's a place for the Cisco router still, for the LLM and so on. And Cisco still is a pretty valuable company. Yeah. Not that. But I think that's this overfocus right now. Yeah. Interesting. So then how do you think about open source? Because a lot of the large model providers are literally going in and saying stop open source now. You've got to outlaw. So how do you think about that? Why are they saying that? Do they

a legitimate gripe. And then, you know, coming from Databricks perspective, how are you all thinking about open source, both with respect to Mosaic and then with the other, you know, things like Lama? If the original Lama was never released, what would the state of the world and our view of AIB right now? We would be way further behind, right? And A, it was a big model, you know, by what existed in open source, and it was open sourced. And both of those things completely changed everything that's happening in AI right now. Size kind of mattered, and the fact that it was open source also kind of mattered.

It doesn't stop there. It's going to continue. It's also really hard to block any of this, because if you just check out the source code for Lama, it's like a couple pages. Yeah, but you have to have the weights too. Yeah, but, you know, the weights leaked, and people will leak the weights,

and they will get out, and people will keep turning them. And there's ways to also, you know, distillation techniques where you can take the weights from a, can just take an output of a model and train smaller ones, and train other ones and so on. So people are going to continue pushing the boundary of this. So I think open source will continue to do better and better and better.

And I think more and more techniques, because there's scarcity, they don't have GPUs, they'll come up with techniques in which they can do things more efficiently, like the fast transformer and so on. At the same time, I also think that anyone that trains a really gigantic model that's really, really good, typically will not have the incentive to release it. So it's the usual thing we see that open source kind of lags the proprietary ones and the proprietary thing is way ahead, and it's way better.

And in some rare cases, like Linux and so on, it bypasses, you know. And in that case, that would be game-changing. And will that happen? It's hard to predict that. Right now, it just seems that you need a lot of GPUs to do this. But how about when GPUs become abundant? Yeah. That's going to happen.

I mean, almost certainly. GPUs become abundant or certain tweaks to the transformer that lets you train at higher learning rate and, you know, have less issues with it. So like, you know, this. Right, because they're super inefficient now. Like, they can be more inefficient. Yes. And so, then there will be released.

They will be released. And the universities are just chomping at the bit, right? Because what has happened right now is that the universities kind of feel a little bit that. They're aced out. They're not really even in the game anymore, right? Look, this was my game. I was playing it.

I was inventment. And now you threw me out. Yeah. And I can't even participate because I don't have GPUs. I don't have the funding. The universities are having a huge sort of crisis internally with the research. Plus see you hired all my guys. Yeah, I was like, so no,

there are guys are leaving. and their gals are leaving because they want to work close where they can train the models and do this kind of stuff and what the data is? And at the universities, there's none of this. So then what are the universities doing? They're, of course, looking at, okay, how could we crack the code on this? How could we make it much easier, cheaper, and how can we release it?

So there's going to be innovation there. So I think this sort of race will continue between open source and proprietary, and eventually open source kind of catches up. So, you know, I think it's going to be diminishing returns. I think we're going to hit walls with scaling laws, and you just move down those. You know, you go to the right on the x-axis, and you know, and you move the parieter curve to the right,

and eventually you get the AGI. Yeah. And it's just happening. It's guaranteed. It's going to happen. I think we're going to hit the mission returns on walls that kind of... So you think we'll get stuck before we get to AGI in a fundamental... We'll need an actual breakthrough as opposed to just more size.

That, and I also think that almost in all the use cases where you seriously try to use this, like for medicine or for like anything where you're really, for lawyers and so on, it quickly becomes clear that you need a huge. in the loop. Unit augments with the human loop. There's no way you can just let this thing loose right now. It's stupid. It does mistakes and so on.

And maybe that can get better and better and better. But it does better on the medical exams than like doctors do. This is a funny thing. I kind of think all the benchmarks are bullshit. And so all these LLM benchmarks, here's how it works. Imagine in all our universities we said, we're going to give you the exam the night before. Okay?

And you can look at answers. And then the next day we're going to bring you in and you answer them. And then we'll score, you know, how you did. how you did. Suddenly everybody would be acing their exams too, right? Yeah. Like for instance, MMLU is what a lot of people benchmark these models on. MMLU is just a multi-choice question that's on the web. Ask a question, is the answer A, B, C, and then it says what the right answer is. Yeah. And it's on the web. You can deliberately train on it and create an LLM that crushes it on that. Or you can inadvertently, by mistake,

in the pile or whatever you used to train your model, happen to see some of those questions that happened to be elsewhere. So the benchmarks are a little bit, yes. Well, they're benchmarks for taking the test, but presumably the test correlates with being able to make a medical diagnosis a decision. Yeah, but they memorized all these, you know, they memorized. Yes, yes. There's no transfer learning from the memorizing the exam to actually diagnosing. No one really knows the answer to this. Everybody's playing the benchmarking game this way right now. Yeah, I would love it if, you know, a whole bunch of researchers. It's like the old fake database benchmarks when it's like, look how fast their databases,

but it's only good at the actual benchmark. Yeah, I would love it if there was like a bunch of doctors that get together and come up with a benchmark that's super secretive and they don't show it to you. And you give your model to them and they'll run their questions on that and then they'll come back and tell you how you scored. But that's not how it works right now. So then let me go to the question that you dodged,

which is, okay, what are the ethics of the large models versus open source or just in general? What is the responsibility? How big is the threat? is open source an ethical threat? Yeah, look, I don't have all the answers. There's, like, different categories. There's, like, the jobs are going to go away kind of category.

We've been doing that for 300 years. And the nations that are doing the best, highest GDP, they're the ones that automated the most. And the ones that weren't able to... And they have the most jobs in the highest... So that's happening anyway. There are ways to deal with that problem. And the ways to deal with it is not to just stop all progress.

That's stupid. You know, the nations that win are the ones that are doing well on automation, not just AI, in general, efficiency, improvements, right? That's like, economics is about efficiency. So anyway, so that's that category. Then there's, like, bad things that humans can do deliberately because they're malicious, which is the one I think Mark was the most worried about. But I would just say, look, ever since, like, the invention of the hammer,

we started misusing technology that, you know, in a bad way. You know, like, so that's going to be. When you have a hammer, your head looks like a nail. Exactly. Right? So that's happening all the time with every technological improvement, especially the internet. So there's a really big question

that I think, kind of like Mark a little bit maybe dodged in his essay, which is, are we going to get this super AGI that decides to destroy us? And I don't agree. But the side part is the part where I got a little lost, right? Because, like, free will

is not something we're on the path for. Yeah. For machines. Yeah. Like, a machine doing many, many, many computations. Yeah. Which, you know, we never had machines do this many computations.

in the history of humanity, that is amazing, but it's very different than, like, no LLM has ever decided to do anything. Like, that's not what they do. And so it does seem like, okay, now they've got free will. Well, maybe they don't have free will. Maybe you're just in my way, and I need to kill you all, right? It's like, and that's just what I'm going to do unemotionally without any.

I don't even reasonable. I don't have consciousness or anything. I'm just doing stuff. Yeah, the paper clip. Yeah, kind of. So I do think, like, those hypotheticals, if you had something, this is a big if. Yeah. If you have that thing that has that level of intelligence and can control things and so on,

then I do think that's a big risk. I just don't think that's going to happen very soon. Here's why. There's several things that people are kind of not looking at. So I don't agree with, like, Mark, when he says, oh, it's just like a toaster. It's just like your toaster will not decide to kill you. I don't believe that. That's not true.

If this thing is pretty smart, it has reasoning capability. if you connect it to robots and give it a bunch of like, it can start doing it. And let it run free with those safetys. Run free and say go do it, then it can do a lot of damage. The reason I'm not too worried about the scenario is the following. One is it's very costly and very expensive

and hard to get your hands on, you know, GPUs and have the money to train a new model. If that comes down and that takes like 10 minutes to train a new model, that's as good as the largest best models that we have, then we're kind of fucked. Because then some asshole will say auto-GPT, connect it, write a bunch of versions of yourself, just try it out in peril, do a million of these in peril, and then figure out if you get smarter and smarter and smarter and just do this.

They'll have a best one. And then before you know it, after maybe let's call it 12 months, we find a slightly better version of the transformer that is a little bit more efficient. Now that 10 minutes goes to like two minutes, and then you're like on this race, and then eventually you'll get into this loop where it can create itself. But right now, it's extremely expensive and really hard to train a new large, giant model, much harder than actually just asking questions from it, unlike the human brain, where I can memorize new things and update my brain quickly,

and I can also just read things from my memory and tell you things. Right now, it's huge asymmetry. Secondly, we really haven't cracked the code on machines reproducing themselves biologically, kind of like humans. So reproduction is not in the game yet. So once you have reproduction and, you know, the building of new ones automatically. Once you crack the code on that loop,

yes, then I think we're fucked. But we're very far away from that. Like, nobody's really doing that, right? Just moving the scaling laws and getting these things to be better and better at reasoning doesn't solve the problems that I mentioned. So that's, I think, what's kind of saving us right now. That's my belief.

All right, well, on that happy note. Well, conclude, I'd like to thank Ali for joining us today. If you liked this episode, if you made it this far, help us grow the show. Share with a friend, or if you're feeling really ambitious, you can leave us a review at rate thispodcast.com slash A16c. You know, candidly, producing a podcast can sometimes feel like you're just talking into a void. And so if you did like this episode, if you liked any of our episodes, please let us know. I'll see you next time. Thank you.