a16z Podcast - Can Open Source Keep AI Power From Concentrating?

Episode Date: September 7, 2026

MTS host Sophia Dew visits the Open Source AI Summit in San Francisco to ask researchers and founders across the AI stack a central question: can open source prevent AI power from concentrating in the hands of a few companies?Lukasz Kaiser, co-author of Attention Is All You Need, argues that today’s concentration may be a feature of the current technological paradigm rather than a permanent feature of AI. Transformers reward enormous amounts of data and compute, but future breakthroughs could make smaller, more specialized models far more capable.Across conversations with researchers and builders working on open models, infrastructure, and applications, Sophia explores why China has taken the lead in open-weight models, whether the U.S. needs more open-model startups, what it means for companies to own their own intelligence, and where openness alone falls short, particularly when access to compute remains concentrated. Resources:Follow Lukasz Kaiser on X: https://x.com/lukaszkaiserFollow Sophia Dew on X: https://x.com/sophiadewFollow MTS on X: https://x.com/mtslive  Stay Updated:Find a16z on YouTube: YouTubeFind a16z on XFind a16z on LinkedInListen to the a16z Show on SpotifyListen to the a16z Show on Apple PodcastsFollow our host: https://twitter.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

AI is becoming more powerful, but the resources needed to build it are increasingly concentrated. Does it have to stay that way? MTS host Sophia Doe heads to the Open Source AI summit in San Francisco to ask researchers and founders across the AI stack whether open source can create a more distributed future for AI. Lucas Kaiser, co-author of the landmark, Attention is All You Need Paper, argues that today's concentration may be a property of our current technology, not an inevitable feature of AI.

Transformers thrive on enormous amounts of data and compute, but they're less than a decade old, and the next breakthrough could change those economics. From open models and access to compute to companies owning their own intelligence, Sophia explores where AI power is actually concentrating, what openness can change, and what it still leaves unsolved.

Joining me is Lucas Kaiser, an AI researcher, and the co-author of Attention is All You Neat. the paper that introduced the transformer that completely changed modern day AI. Where is modern day AI and where is power currently concentrating? There was a very important moment sometime around end of last year,

start of this year, where the coding agents started to work. The nature of this AI is that you need to train it on very expensive data centers and on a lot of data. So there's big companies that do this and then they charge for it, right? That concentrates it

in the sense to get the, like, to me, even the current models are not smart enough. So there's still, they could be much smarter, right? There may be some research breakthroughs that are needed to make them smarter with less data. But since research breakthroughs, you know, sometimes they come, sometimes they don't. They're not like a business proposition. But the big companies are like, okay, but you can do things without research breakthrough, which is to go bigger, bigger, bigger.

Now that is very concentrating. You go bigger, bigger, you need billions of dollars. you need to scrape data from every corner of the internet. So the current state of the technology is a bit concentrating, but we should remember that it's just the current state. Sure, transformers are great, but they're not even 10 years old. I'm sure will be other things. Maybe we'll make it possible to train with less data, more diverse model.

Like, transformers are really good if you train them on the whole internet, and then they're reasonably smart. But if you try to say, you know, you just learn about this, then they're just stupid, right? It doesn't work that well. But it doesn't work because maybe we need a research breakthrough that will make it work. But the transformer needs to be everything. And that's a problem, but it's a research problem that hopefully will get solved.

With the transformer, architecturally, you need a lot of resources. And the more resources you have as a company, as a model, the more powerful you can be. But do you think there is a type of algorithmic breakthrough or some type of research breakthrough that will allow smaller players to really compete? Well, I mean, we know it exists. We are the proof, right? Humans are not experts in everything. But in the field, there are experts. They're very good, sometimes better than our super huge models. So it exists. I don't think we have found it technically. It may, you know, it may even have attention layers somewhere, right? It may be, because it's not just the model. It's also the loss and the data and how it's trained. There's a lot of factors. So we don't really know exactly where to look. That's the thing in research. But on the other hand, I, I, I, think more and more people are looking, I hope. I think the labs have a little bit, you know, Open A1A when I joined was a very research. Now it's a research lab a bit, but it's also a very big company like that needs to do products

and stuff. So it has less focus on researching. But this gives chance to, that's a chance to the open source movement. That's a chance for academia to universities. I recently started doing things on my own also. and it's funny, so I bought a 5090 RTX GPU. It has more power than the eight GPU machines we used as a team to do the Transformers research. So you can buy and have one GPU that has all the power we used to design transformer.

So you know, so you can do things with it. There is, like you can't train a big LLM, but you can research and experiment on things. And I think many people should. I believe there is still much more in machine learning research than. My last question for you is, what makes you feel optimistic that the future of AI is going in a direction that will be truly open and broadly empowering for everyone? If you look at the world, humans are the most amazing computers in the world, right? our brains are still unmatched by the models and we are not generalists

and there's not like one big brain of humanity. I think fundamentally it might be that given a fixed amount of data, the best way to learn from it is to have a lot of distributed models strong on its own but even stronger when they're... And yeah, if you look at some basic research,

there are actually reasons to believe it. like I was just talking about ensembles and things like that. So, yes, transformers are great when you feed them all of the internet, but we know there is an algorithm that's even better and it can learn from much smaller data and be amazing at the things it's learning. We are the proof. We have not found it yet like how to do this in computers very well, but we will. I mean, you know, progress in machine learning has not stopped in 2017.

Of course. Since because it's so good, everyone has focused on the big data. But now that it's so expensive, maybe we will focus back more on more fundamental research. And yeah, I'm absolutely convinced it can work and it just needs research and effort. I think many people sometimes look at the current state of AI and get a little pessimistic. That it's like, you know, it's just big companies, big data centers. Everything's so big. subscription to something you cannot do much about.

And to me, it's like, okay, but that's just the current state. You know, it's just this year, next year. It's a step in the technology. But it's a step towards something much more fun where you can have, everyone can have their own model maybe. We'll learn from much smaller data. The models will be experts in different domains like we are. And maybe, you know, maybe one will be making jokes this way and another, another way.

It's like, currently when you ask a big language, model about a joke, it's always the same, right? Something about atoms. Like, this will change because it's an outcome of a technology, and the technology will change when the research catches up. And also for researchers, there's like just huge opportunity to catch up with this. So, you know, like, just don't worry about, okay, it got concentrated because that was a convenient thing to do.

And it showed that great things can be done. good, but there is think ahead that we just need to just need some research breakthroughs. Thanks for listening to this episode of the A16Z podcast. If you like this episode, be sure to like,

comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcast, and Spotify. Follow us on X at A16Z and subscribe to our substack at A16Z. substack.com. Thanks again for listening and I'll see you in the next episode.

This information is for educational purposes only and is not a recommendation to buy, hold, or sell any investment or financial product. This podcast has been produced by a third party and may include pay promotional advertisements, other company references, and individuals unaffiliated with A16Z. Such advertisements, companies, and individuals are not endorsed by AH Capital Management LLC, A16Z, or any of its affiliates. Information is from sources deemed reliable on the date of publication. but A16Z does not guarantee its accuracy.