This Week in Startups - Liquid AI's Ramin Hasani on liquid neural networks, AI advancement, the race to AGI & more! | E1928

Episode Date: April 9, 2024

This Week in Startups is brought to you by… LinkedIn Jobs. A business is only as strong as its people, and every hire matters. Go to LinkedIn.com/TWIST to post your first job for free. Terms and conditions apply. Experimentation is how generation-defining companies win. Accelerate your experimentation velocity with Eppo. Visit https://geteppo.com/twist Attio - A radically new CRM for the next era of companies. Head to attio.com/twist to get 15% off for your first year. * Todays show: Liquid AI’s Ramin Hasani joins Jason to discuss the mission and the concept of Liquid AI's liquid neural networks (1:09). They dive into liquid neural networks’ applications (16:07), transition from theory to execution (21:37), their efficiency on small devices (27:30), and more! * Timestamps: (0:00) Liquid AI CEO and co-founder Ramin Hasani joins Jason (1:09) Liquid AI's mission and concept of liquid neural networks (7:06) LinkedIn Jobs - Post your first job for free at https://linkedin.com/twist (8:34) Demo of Liquid AI: traditional vs. Liquid neural networks in autonomous driving (16:07) Practical applications of Liquid AI (20:07) Eppo. Accelerate your experimentation velocity with Eppo. Visit  https://geteppo.com/twist (21:37) Commercializing worm-inspired AI systems, building a team, and solving problems across various sectors (27:30) Efficiency of liquid neural networks in compact devices like the Raspberry Pi and the transformative potential of AI modeled after worms. (34:15) Attio - Head to https://attio.com/twist to get 15% off for your first year. (35:24) Data ownership in AI and incentivizing data providers (41:18) Role of AI in real-world applications (43:57) Societal impact of AI, job displacement, and the optimistic view on AI's potential (50:25) Explanation of physical models vs statistical models in AI and the challenge of understanding black box AI systems (57:01) Speculations about AGI's market cap, who might achieve AGI first, and the potential of using AI systems to build various applications (1:00:12) Comparison between open-source and closed-source models in AI and the trend of open-source moves in the AI industry * Mentioned on the show: https://www.raspberrypi.com/products/raspberry-pi-5 https://www.capgemini.com/us-en https://www.ctc-g.co.jp/en https://www.accenture.com https://www.ey.com/en_gl https://www.cnn.com/videos/media/2024/04/02/the-daily-show-jon-stewart-ai-work-force-jobs-orig.cnn * Source of C. elegans worm footage: https://www.youtube.com/watch?v=zjqLwPgLnV0&t=1s * Follow Ramin: X: https://twitter.com/ramin_m_h LinkedIn: https://www.linkedin.com/in/raminhasani Check out Liquid AI: https://www.liquid.ai * Follow Jason: X: https://twitter.com/Jason LinkedIn: https://www.linkedin.com/in/jasoncalacanis * Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp * Thank you to our partners: (7:06) LinkedIn Jobs - Post your first job for free at https://linkedin.com/twist (20:07) Eppo. Accelerate your experimentation velocity with Eppo. Visit https://geteppo.com/twist (34:15) Attio - Head to https://attio.com/twist to get 15% off for your first year. * Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland * Check out Jason’s suite of newsletters: https://substack.com/@calacanis * Follow TWiST: Substack: https://twistartups.substack.com Twitter: https://twitter.com/TWiStartups YouTube: https://www.youtube.com/thisweekin Instagram: https://www.instagram.com/thisweekinstartups TikTok: https://www.tiktok.com/@thisweekinstartups * Subscribe to the Founder University Podcast: https://www.founder.university/podcast

If you have AGI, as you said, like, you can solve the energy problem. You can solve once you solve the energy problem, like, well, I mean, you are basically the most valuable company on Earth. You know, think about that. I mean, if you can solve economy, like, if you can solve politics, basically, like the structure of governments, you know, this is the thing that we're hoping to get. And there's a race to getting there. Do you think everybody gets there at the same time? Like, AGI feels, no, you don't. You feel some people will get to AGI first?

First, yes. Yes. Yes. This Week in Startups is brought to you by LinkedIn Jobs. A business is only as strong as its people and every hire matters. Go to LinkedIn.com slash Twist to post your first job for free. Terms and conditions apply. Epo.

Experimentation is how generation-defining companies win. Accelerate your experimentation velocity with Epo. Visit getepo.com slash twist. and Adio, a radically new CRM for the next era of companies. Head to adio.com slash twist to get 15% off for your first year. All right, everybody, welcome back to this week and startups. We've got a great guest for you today with a great idea. Ramin Hassani is the CEO and co-founder of Liquid AI. And we're going to hear all about what Liquid AI is doing in a moment, but they're kind of headed in a new direction, trying to make smaller and more efficient. language models. Welcome to the program. Ramin. Thank you for having me.

Maybe, you know, just by way of introduction here, explain to me what the mission of Liquid AI is, and then let's get into, you know, sort of language models and, you know, the size of models and making them more efficient. Yeah, definitely. So I started a company to design basically, like from first principles, systems that we can understand from scratch on a completely new base for artificial intelligence that is rooted in biology and physics. So you started looking into brains and see how we can get inspirations from there to design kind of a new math that we can understand and we can scale, basically. And that kind of became kind of a liquid neural network technology that I invented during my PhD program.

Okay, liquid neural networks. What does it mean compared to, say, a traditional AI model, large language model? So what's the different? What is a liquid neural network? Let's explain that. And is that a term you came up with or is this an industry term?

Yes, that's something that I came up with. So believe it or not, like about seven years ago, I started looking into the brain of a littered worm. The worm is called sea elegance. The worm is, is one of the, like in the tree of evolution is one of our fathers. Okay. So it's basically nervous systems and cellular kind of organization and everything is evolved

from this animal. This worm has already won four Nobel prices for us because it shares 75% of its genes with humans. So, and its entire genome is actually sequenced. It's one of the only animals on Earth. We have actually two animals now. that its entire nervous system is mapped. That means, like, we know exactly how anatomically,

like how each part of the nervous system is actually connected to each other, right? So I thought another nice behavior of this biological organism, is the fact that its nervous system is differentiable. What does that mean? Today's AI systems, as you know them, they are basically a set of neurons in a layer-wise architecture next to each other's, and they're connected through synapses or weights of the neural network, and they become like a giant neural network that can do what chat chabit can do today.

We scale those kind of neural networks into this kind of regime. Now, neural networks, the way we train these systems on massive amount of data is with the technology called back propagation. Back propagation of errors. The underlying mathematics of the systems is differentiable. That means you can propagate errors. without interruption inside the neural network, inside this huge kind of gigantic kind of functional form of neural networks.

This property doesn't exist in the human brain. In the human brain, neurons spike. So you have seen like, I don't know, EEG kind of signals and stuff. Like you can see that there are spiking neural networks. Spikes, we haven't understood yet, like from nervous systems. We don't know why spikes work. We have no idea. We still don't know what's the purpose of the spike.

I mean, some people say they translate an analog to digital kind of conversion to propagate information much faster. We know a little bit about the learning theory around that. Like Geoffrey Hinton is actually like, he was working on some forward algorithms, you know, non-back propagation based kind of methods and stuff. So there are local kind of learning rules and stuff that we figured out. But there is still so much that we don't know how to bring. brain actually does learning.

But when we go back into animals, until we arrive at this worm, we don't have any nervous system that doesn't spike. So that's why I liked this worm because, you know, the nervous system is something that is very similar to the mathematics that we design artificial intelligence with. So I started like basically modeling the behavior of cells inside this worm. And then this system became a new type of learning system. This learning system is flexible in its behavior, meaning that when you train it on data,

the system still stays adaptable to incoming inputs. This is not the case with artificial intelligence systems. When you train them, they become kind of a fixed system. So when you train the weights of a neural network, let's say, in the case of, let's say, GPD4, GPT4 has 1.8 trillion parameters. each parameter, this corresponds to 1.8 trillion weights in the system. These weights of the system are already trained and they are fixed.

Now that they're fixed, now it's now became an intelligent system. You can input information in there and then take output information. But the system is fixed. Liquid neural networks, on the other hand, they're not fixed. There are systems that you can have, they can stay adaptable to the input in That's the major kind of difference between the two. And that's an advantage because it will make the answers more dynamic or more real-time. What's the advantage to...

Or more robust. Okay, let me cut to the chase right now because I know you're busy and everyone is hiring right now. And, you know, it's a lot of competition for the best candidates, right? Every position counts. Markets starting to come back. You need to get the perfect person. You want a barraiser in your organization. somebody who will raise the bar for the entire team,

and LinkedIn is giving you your first job posting for free to go find that barraiser. LinkedIn.com slash twist. And if you want to build a great company, you're going to need a great team. It's as simple as that. LinkedIn Jobs is here to make it quick and easy to hire these elite team members. And I know it's crazy, right? Lincoln has more than a billion users. We all watch this happen when it was tens of millions, that hundreds of millions,

and now a billion people using the service. This means that you're going to get access to active and passive job seekers. active job seekers, they're out there looking. Passive job seekers, they got a job, but it's not as good as the job you're offering them. So you want to get in front of both of those people. Maybe somebody got laid off wasn't their fault and they're an ideal candidate. Get that active job seeker. And LinkedIn also knows that small businesses are wearing so many hats right now and you might

not have the time or resources to devote to hiring. So let LinkedIn make it automatic for you. Go post an open job role. You get that purple hiring ring on your profile. You start posting interesting content and you watch the qualified candidates. They just roll in. And guess what? First ones on us.

Call to action. Very simple. LinkedIn.com slash T-W-I-S-T. LinkedIn.com slash twist. That'll get you your first job posting for free on your boy J-Cal. Terms and conditions do apply. A demonstration would be great here because this all sounds quite theoretical. So maybe we could walk through your product demo or your PowerPoint on how this all works.

Yeah, definitely, definitely. So we are talking about still the science of things, like how this became. liquid AI and this is all based on a worm what worm is it it's a worm called sea elegance it's a two millimeter meter long worm it's a very very tiny worm got it but it's a very popular worm let me show you what would happen when you train a liquid neural network versus a typical neural network okay and for those of you not watching you can go to this week and start-ups on youtube and find this episode just look at the recent videos tab yes what i'm showing is this is basically a dashboard of an autonomous driving system.

It's a neural network, what I'm showing here in the middle. You see layers of neural networks, a stack to each other. And then they receive camera inputs and they make a final decision. They make a driving decision, basically. Now, this system has been trained on massive amount of driving data. This is just a lane keeping task because we've done that at MIT during our research, basically. So what we see here, this is actually an actual car that is getting driven by this neural network. In the camera on the top left, what you see is the camera view.

And on the bottom left, what you see is an attention map of this neural network. That means, where does this neural network is paying attention to when it is taking driving decisions? This neural networks has 500,000 parameters. It's a rather small neural network. Now, as you see, there is also a little bit of noise on top of the image. You know, like on the camera, you see like we put a little bit of noise so that we can disturb and see how robust the decision-making of this system is. Okay.

And as we see, in a typical kind of neural network that you see in the middle, I put all of these dots that are glowing. There are basically single neurons that are getting activated and deactivated, basically. it is very hard to say what this neural network is doing, right? Because there's a lot of them and there's a lot of 500,000 parameters. How can I actually say what each individual of these systems is doing, you know, in this task? But again, if in an abstract way, if I bring it back to this image on the bottom left, what you see is the attention map.

The lighter regions are the regions where the network is paying attention to when it's taking a driving decision. God. And that would be the road, I guess. Exactly. It has to be the road. Yeah. It has to be the road, but it is basically like outside of the road. In this case, as you see, the attention is kind of outside and is kind of affected by the noise that we put at the input, you know.

So that's why it is not that much reliable. This is how a typical artificial neural network works. Now, let me change that to a liquid neural network. All we did, we switched the parameter heavy part of this neural network. We kept the eyes of the network. which is this conv-conv kind of layers, as you see, like convolutional layers, basically. But we replace basically the parameter-heavy part of the system with 19 neurons, 19 liquid neurons, neurons that are modeled after the worm's brain.

And then we basically, you know, like the synapses also like the connectivity, you know, it looks like a little bit more scattered, this kind of recurrent kind of connections. You can see a lot of kind of unstructured kind of connections in this system. But this system has 19 neurons and around 1,000 parameters as opposed to the previous system that I showed you that had 500,000 parameters. The system became very small. Yeah, it becomes much smaller. And does that make it more accurate? Or does it make it faster at decisions or both?

Or we just don't know? Both, actually. So now let's look at the bottom left again, like the attention map of the system. As you see in the attention map, now the focus is on the road and on the sides of the road. So the system, without any prior,

it actually figured out how to perform decision-making without being like, you know, like disturbed by anything else. Now, not only this system is much smaller than a transformer architecture, but it's also, it can give you basically

much more robust representation very similar to how biological systems perform decision-making. So net net, a worm is a better driver than a human brain. Yes. Is what you're telling us. That seems counterintuitive. Aren't human brains better than worm brains? And is this because the silicon that these things are run on and the cameras aren't

able to process fast enough in real time like a human brain? So actually a worm brain might be a little bit simpler and easier to run on today's silicon. Is that what I'm reading into this as? I mean, yeah, to some extent. But the fact that these are just modeled after how nervous systems perform computation in the brain of the worm, now we can take those mathematical inspirations and then build machine learning systems that are not just like they're not just mimicking to be a worm or anything. they're just like basically the fundamentals of computation in nervous systems.

Now, the reason why I told you the worm in the tree of evolution is one of our fathers is the fact that these principles actually scale. That means if nature actually evolved worms into humans, we can take these inspirations from neural computations and even go beyond that. So there's an opportunity to build AI systems powered by how nature designed nervous systems. Okay, so the worm system is less robust and narrower than humans, but you could scale it up. And if a worm had a billion neurons or a million neurons, I don't know how many it has. Actually, the worm has 302 neurons.

It's a very tiny worm. Okay, so it's got 302. How many neurons does it? Human has 100 billions of neurons. Got it. Okay. So there's a big gap between those two, but the worms are... simpler and easier to define or easier to emulate than a human because humans are much more complex with 300 billion?

Yes, yes, we can understand this one. We can understand the brain of this worm much better than we can understand the brain of a human being. Because we still have a lot of questions. Even we don't understand mice. We still don't fully understand monkeys. We don't understand small fruit fly, you know? So that's why we need to start from summer. So I wanted to take a step back and start as a, you know, computer scientist, basically, wanted to see like how these kind of systems, how can we look at the origin of these nervous systems and where can we find, basically, principles that we can at least confirm that exist in biology and then now take the systems and build new type of learning systems.

Okay. Got it. Okay. Just an inspiration, basically. I understand. Yes. Okay. So this is pretty trippy, but I think I'm following.

So let's keep going. Yes. Yes. So then since then, we managed to drive cars autonomously, like with these small nervous systems. We showed that you can fly drones with them. Okay. You can recently, United States Air Force actually showed that you can also fly a full-blown F-16 jets with them. This type of warm-inspired systems can actually, you know, do a lot more than just, you know, like now.

navigating warps. So it might not be able to handle the existential crisis or making a season of the Sopranos and something complex like that and creative, but it might be able to do something incredibly simple and basic, like stay in the middle of this road, you know, dead center or, you know, keep this drone in the sky, not crashing into something. Exactly. That's what we thought at the beginning, right? That this is going to be the property of this learning system.

But then we started to see that you can also do much more complex kind of tasks way better than how artificial intelligence systems performing that. For example, what? Predictive models for financial markets. Predictive models for biological signals. Let's say, like, if you want to predict the mortality rate of people in ICU based on their biomarkers, you know. And then if you want to do predictive tasks like that, you can see that, models are really good at doing that. In principle, we figured out that this type of new type of technology is really good at modeling time series data. Data, it could be video data, it could be audio

data, it could be text, it could be user behavior, it could be financial time series, medical time series. So it is basically a general purpose computer, the type of models that we develop. These type of models we've applied them and we checked it the last seven years actually we have seen that these systems are really good at performing these kind of sequential decision-making processes, right? And that became basically the point where we thought that, okay, so now it's time to start

maybe making larger and larger systems off of these general-purpose computers that we can you know, like we can change this spectrum of how AI is done today. Because today, we are working with a base called transformer architecture, right? We're basically changing that transformers and GPTs, generative pretrained transformers, into a new foundation, which is called liquid foundation models,

which is called LFMs, basically. So it's a new thing that is coming, basically. Okay. And a time series, just so people know, when you say time series, it's very simple. it's the series of a similar data point, but over time. So perfect would be a stock price over every minute on the stock exchange. Or as you talked about driving, it would be the steering wheels alignment or the speed of the vehicle over every second or millisecond. That's a time series.

And these are particularly good at studying a time series is what you're saying. Yes, yes. and also this podcast, you know, the audio signal that you're hearing is basically a time series. The video that you're seeing is also a time series. So all video data, I mean, in some sense, if you think about it, that's a time series. You know, audio is a time series, video is a time series. But then language is a little bit different than that. Language is also a kind of sequential kind of data, but it's not the time element is different.

It's basically just a sequence of words coming after each other. So you could also technically apply liquid neural networks to those kind of problems as well. Are you tired of slow A-B testing? I'm sure you are. Do you have any trouble trusting your experiment results? I know I do sometimes. We'll get ready to 10x your experiment velocity with Epo. That's EPP.

Whether you're a scrappy startup, a tech giant, or anybody in between, their future management platform will turn your risky launches into clear, cut experiments. Data teams, of course, love Epo, and so will your product growth and machine learning teams? The executors are going to love it too because they're going to love the results and the discipline that comes from defining really important product experiments and then executing on them really well, because it gives data teams better coordination and faster innovation. Online marketplace Inventa has cut down experiment time by 60%. And click up, the project management company has cut down analyst time by over 12 hours. And Epo's class class.

based system is all about empowerment. You get instant access to your experiments from anywhere in the world, boosting flexibility and teamwork. And you get a system that grows with your needs, easily scaling up to handle more tests as your business grows. With EPO, the daunting becomes doable. I love that. So your team can rely on the experiment results and make faster decisions. Here's your call to action. Experimentation is how generation-defining companies win. Accelerate your experimentation velocity with EPO. Visit getepo.com slash twist. Just visit Get EPP.com. dot com slash twist and let's get some experiments running let's get that product market fit and thanks to epo for supporting independent media like this week in startups and all the startups who are

listening well done all complex uh where are you at in terms of this being theory versus execution so we see chat chp t4 we see fSD 12 where is your company at in terms of you know commercializing this and did this all come out of mit i heard you mentioned mit earlier so you went to MIT, you studied this. And, you know, this jet fighter that, you know, was AI-based. Is that your software or they also studied this? So explain us to us where you're at with this company and maybe some demos of the product. Yeah, definitely, definitely.

So we started exactly maybe one year ago, one year and three days ago, actually the company. So the company has four co-founders, myself and all MIT. So we, it's myself, it's Matthias Lechner, who's another CTO. We have actually invented, co-invented the technology together. And then we have Alexander Amini, another PhD student from MIT. And he's graduated now. And then the director of computer science and artificial intelligence lab at MIT, who is Daniela Rousse basically is also a co-founder of our team.

We started this company on this new technology because we've seen a lot of, like, you know that our lab at MIT was, focus on real world applications of AI, you know, like we really wanted to design AI systems that can go into the real world and solve real world problems, you know, and that's why like we always had our AI systems deployed in the society, like they were always deployed in an environment doing a task, you know, this would be an autonomous car, this could be also an, you know, manipulation of a robotic arm, you know, this could be any kind of task, a humanoid robot kind of control. Do they refer to that as C-sail at MIT, the computer science, artificial intelligence

laboratory, which is known, correct me if I'm wrong, for a lot of robotics that we see in the world. Absolutely. So the Rumba and some of those projects came out of that, yeah? 100%. Yes. Yes. So a lot of the fingerprints on robotics come out of this. Yes. MIT's C-Salle Lab. And you were part of that. Now, where are you out in terms of providing this as like a product? Are you, is there an API or people starting to use this? Are you, yet, how old is the company? How much have you raised?

Tell me a little bit about, you know, now that we got the background on the science behind this and the science is worms. We get it. Yes. Super interesting. Let's talk about the application and like making the startup reality because going from theoretical and spinning something out of a university and then making it reality. That's a jump that very first. few companies are able to make. So explain to me where you're out with that big jump.

Yeah, definitely, definitely. So we started last year, 30th of March, the company. It's very fresh. It has been like 12 months now. We raised the substantial amount of kind of seed money. I think we first had a seed round of $5 million at $50 million valuation. And then we actually did like a C2 basically. And that C2 was also, I think eventually became 37 million. And so overall, we raised like $42 million in C-value at a $300 million valuation. The reason for raising the money was basically building the superstar team, which is one of the things that we have, because if you're building something completely different than 99% of the companies, because every company in the generative AI space and AI space is working on top of a

technology called Transformers. Now we're changing that foundation. So you need to have like people, like-minded people from all over the world. I actually gathered them mostly from MIT and Stanford and some of the students of Yoshio Benjou as well. So we gathered this team of people, brilliant people, they have all invented new technology for efficient alternatives to machine learning systems, people that have worked on explainability of AI systems.

We have all sort of kind of capabilities in the team. With the purpose of wrapping basically this technology of ours, like building on top of the core technology, which is liquid neural networks, for enterprise kind of solutions with a horizontal kind of look to the market. So we are basically going after verticals because as I told you, it's a generally system. I can solve financial problems for banks, for large banks. I can solve also problems in the space of biotech. I can solve problems in the space of autonomy, right?

So it's a horizontal play. Now, as a startup, it's always like the weird way to actually go after all. I want to solve all of them. But we're talking about AI. So, yeah, are you a platform that you're going to provide an API to people? Or are you going to go after one of these verticals, I guess, is the question everybody has. Yes, yes. So we are building an AI infrastructure in which you can train, fine-tune, and play around and use liquid foundation models.

This product is an enterprise-facing product. It comes with a developer package where we actually give it to enterprises. Enterprises are basically can use this technology and actually enjoy its performance. They can see the efficiency of the models. Mostly you can develop models on the edge. We have today language models that run on a Raspberry Pi. Raspberry Pi, just so people know, is the smallest computing unit, essentially, in the open source hardware community, these Raspberry Pies go for $10, $25, it has a certain amount of power to it.

So you're telling me, you're going to be. running this on this neural network on a Raspberry Pi, which is like running it on like a thumb drive, basically. Exactly. People can imagine that. Yeah. That's like one of the beauties of the technology. So the technology can be running on a very, very tiny, they're very energy efficient, you know,

depending on their, they can be small, but they can be very powerful. Now, in terms of like how we are going to market and how we are actually commercial and how we are managing to be the AI platform for all the verticals, we have established some contracts across the globe, actually with some of the system integrators in the world. So in Europe, we have a contract with Cap Gemini, which is one of the largest system integrators actually in Europe. In Japan, we are working with Itochu CTC, which is basically the Accenture of Japan, you know. In United States, we are signing up with EY and conversations with Accenture, basically. So the target is that system integrators would take the platform as basically being able to

integrate it in the verticals that they're interested in. So you don't have to worry about the commercialization of this. You have to provide the people who do commercialization and license to them. So this seems incredibly disruptive. If you are able to do this for a fraction of the cost, what does this do? And the fraction of the hardware, if you're successful, what does this do to Nvidia? What does this do to Open AI? They're putting together billions of dollars, tens of billions of dollars

in supercomputers to train these models you're claiming you're going to be able to do this because it's with the worm brain and it's a much more efficient process with a fraction of the hardware model, the hardware footprint. So, you know, head to head what's going to happen

to, you know, big iron in AI if you're successful. Yeah, definitely. So there are two costs on developing AI systems. One cost is like designing the AI systems. The other cost is basically usage of AI systems, right? Like you can now my AI's inference, basically, right?

So now on inference side, as I told you, we can be between 10 to 1,000 times more efficient than the models that are available today. that's basically the energy footprint of the models. On the training side, we can be between 10 to 20 times more efficient than the transformer models. That means if I train, let's say, a 10 billion parameter liquid model,

it's going to cost me, depending on how much information it can process, which we call context lengths, right? Depending on the context lengths that they have, it can be between 10 to 20 times much more efficient to actually develop this kind of system. So that means instead of requiring $10 billion, like $10 billion basically to develop GPD4 quality models, you would need a fraction of that, basically. Yeah, maybe $500 million or something, or $100 million.

A serious fraction. What does that mean for, you know, somebody like Open AI, Microsoft, some of these cloud computing platforms that are, are they building? all this extra hardware and focused on the wrong problem, you know, that hardware is not the problem. It's the architecture and the framework and the paradigm under which they're building this, and they're just building under a much less efficient paradigm. Is that your claim here?

I would say, you know, the beauty of the transformer architecture and what Open AI and everybody else is after is the fact that this system is scaled really nicely. You can scale them into larger amounts of data. and also larger model sizes. So what motivates the community on generative AI is the fact that the larger you make the systems, the more powerful they become. Now, if you look at where we are today

with the state of the art, we have a Claude Opus, basically, which is the most powerful model. I expect this model to be in the order of like three to five times bigger than GPT4. That means, this model is, I would say, in the range of maybe 10, trillion parameter model.

Now, they haven't released, Claude. Anthropic hasn't released what that model is, but it is number one on hugging phase now with the ELO ratings of a chat interface. 100%. It's the number one kind of performing kind of AI system in the world right now. Okay.

Like there's now, Anthropic is talking about 10xing the size of the models every year that goes forward. That means we can expect by the end of next year to have a hundred trillion parameter transformer model. The reason why they're doing that is because when the models are actually getting larger, they become better and better. And maybe we can get into AI and generally this kind of AI systems by enlarging kind of the architecture.

And the focus is just that. There are two companies in the world that I think the absolute focus of the companies are building AI, is open AI as an antropic right now. So there are kind of gotsy moves like what we are doing basically. We are basically changing the fundamental architecture. We're building new scaling laws, basically on top of this thing. The scaling laws, let's see if we can make liquid neural networks also scale. That means if I have one trillion parameter liquid model, it might actually be as performant

as a 50 or 20 billion parameter transformer model. The other way of it is also true. If I have a hundred trillion parameter liquid model, it might be better than a 20 trillion parameter than a 20x larger transformer-based model. So that means these are basically the kind of moves that we want to make. I mean, so far we have been- If you're successful, when will Anthropic move over

to your platform, do you think? Or are you a competitor to them? I think, I mean, right now, like, we are going through another fundraising, like Series A of Liquid. And I think after this round, we are basically getting prepared to actually train very, very large models. So these models are going to be, I mean, after the release of those models, probably by the end of the year, I would say.

Then the community is going to see that there are alternative kind of models that they can come in and disrupt the way transformers are actually disrupting. And they can scale the way transformers scale, basically. What hardware are you going to use? What platform are you using? Right now we are using Nvidia GPUs as well. Like it's very similar. It's just that the number, the amount of GPUs that. we consume is about 10 to 20 times less than how.

Got it. 5%, 10% of them what they're using. Startups and small businesses, listen up. You want a CRM that neatly organizes all your customer data so that you can avoid missed opportunities and you can deliver a personalized service. R rigid CRMs can adapt to your fast-growing needs and that's where Adio comes in. A-T-T-I-O delivers the goods. It's a custom CRM that's flexible and deeply intuitive.

A-T-O is built for the modern. and company headed into the next era of businesses. It connects your data sources, adjust easily to your specific setup, and suits any business approach, whether it's self-served or sales-driven. Adio automatically enriches all your contacts. Think about that. You might be missing a first name, a last name, an email, an address, all of that stuff. It's going to sync your emails and calendars.

It's going to enrich those contacts, and it's going to give you powerful reports. It's also going to let you quickly build Zapier-style automations. if this, then that type of automations. The next generation deserves more than a one-size-fits-all CRM. Join 11 labs, replicate, modal, and more, and get ready to scale your startup to the next level. Head to adio.com slash twist, and you'll get 15% off your first year. That's ATTIO.com slash twist. And so talk to me about data, because it does seem like this is the next big shooter drop, licensing data,

Balkanization of data. Hey, maybe Reddit is available to Gemini, but not Open AI. Twitter now is, you know, closing up access or X.com is closed up access for people. And the New York Times is in a lawsuit with Open AI, which obviously trained on their data without permission. How do you see all of this resolving itself? Because obviously, people are rightfully saying, hey, I own this data. I have the archive of the New York Times. Or I'm Disney.

I own this archive of IP from Star Wars to Marvel, where I'm an author and I have these books. How do you see all this shaping up in the coming years? Because is that going to be the limitation, the data you have access to, or is it going to be synthetic data rules the day and you're going to be able to just make your own data to train on? How do you see all this unfolding?

Yeah, definitely. I believe, like, at the end of the day, I think the data providers, They should be incentivized to provide their data, and they should know, they should know that their data is being used. Basically, like, you need to have a payment scheme, basically, for people that you're using their data in your systems. What should that be in your mind? How would that work? Do you have any ideas?

We haven't, we haven't gotten there yet. Like, I think, I think this would be like a challenge to think about. But at the moment, what we're trying to do is basically the way everybody does. Like, we are basically purchasing data, purchasing data. Right. But you're basically paying for the data that you use in order to be able to, you know, like to leave. And you believe this is a good idea because it will keep people making data. So journalists, artists, writers, thinkers, you believe, hey, this is a fair deal here, some sort of licensing arrangement where they get paid some reasonable fee to train your models or train Plaud's models or Open AIs models or Google's models. Yeah.

100%. the reason being, say, for example, a content creator on YouTube, right? So if people come and look at their content, basically, you know, like, and they get inspired to build something off of that. You see, so AI is also like basically doing the same thing, right? It's looking at the data that is basically available and it's getting inspired by that data if it's not directly the copy of that data, right? And that scheme of how you're doing it through like, let's say, social media kind of channels, right? It has to happen like very similar ways that we can, we can incentivize users of social medias or users of AI or providers of data for AI systems to also like have this understanding of this is basically the same thing. The same kind of scheme can actually apply here.

They might be analogous here. But again, like you really have to be systematically going after this problem, which is one of the one of the main main issues like as we're thinking about the scaling our company. It does feel like it's fair if somebody's put a lot of work in. to it, that if an AI was built on top of the New York Times corpus, that they would have permission to do that because it is something that you could partner with the New York Times as opposed to open AI and build this with them and monetize it with them. And it's their opportunity to create an AI based on the New York Times data, not open AIs or Geminized. Everybody should have the ability to opt into these things. I feel like that's a pretty smart approach that

you're taking. How long before people will be able to use your platform? and swap out Gemini or swap out Clod or swap out Open AI for yours. Yeah. So, I mean, the first batch of products that are coming is basically already in use with some of the clients. It's a developer package, as I told you, for solving AI problems. Like, this could be, let's say, like, you have a predictive task, where you have, like, video data from surgical kind of processes.

And at the output, you want to predict basically, what phase of surgery we are in, for example. That's a kind of case study where a developer can take our package and then basically use our system in that kind of real-word application to solve that task. This is already ready and it's available to some of the enterprises through our system integrator contracts and through directly. Some of them we are already working like in the financial sector, in the medical sector, in the healthcare and biotech.

We have been like very active and automotive. Okay, this is already available. What's your definition of AGI? How do you determine that a system is generally artificially intelligent? Do you have, I mean, you must have heard a million of these different ones that when you're at MIT and there's a big debate around it. But what do you think? I think for AGI, I think that I just want to stick to something that we can actually still understand and talk about. For example, a system that is beyond human capable can perform beyond.

human capabilities given the same resources. That means if I'm provided that the same kind of resources is provided to the to the human and to it to the AI system, the AI system is being able to perform that task better or orders of magnitude better than humans. Got it. So given the same resources, we both have access to the internet. We both have broadband. Can I beat this system at chess? No.

Okay. But it would be at a new game. that just came out today, could it beat me? I guess is the question. Exactly. And in the age, I can exist in a virtual world as well.

Like as you were mentioning, these are possibilities that are inside a virtual kind of existence. It's going to be existing in an internet kind of system. But in real work, you need to have also embodiment. So that's why a lot of work is actually going towards, you know, like the humanoid kind of movements of robots.

We are building humanoid kind of robots an open AI figure I mean the new works that are going on like there's so many I mean at MIT there are many many people working on humanoid kind of research

and also like other types of AI systems that you can integrate in the society you know when do you think is yeah so the point is you know there's virtual we know that those are creeping up

like getting an answer to a legal question or making a marketing plan or writing something you know and obviously chess and verticalize games go it's crushing humans, but it's got to be able to translate into the real world, and if it's

going to be doing, picking strawberries, we're going to need a robotic arm, we're going to need computer vision, but all those things seem to be aligning. So a robot, we had a company called Rude AI, which I think actually had some of its origins at MIT as well with the robotic hands, being able to pick strawberries in the real world better than a human, faster, pick the right ones, not crush them, put them in a box. I think we're kind of there today, or pretty close to it. For those kind of applications, yeah. Yeah.

But think about, think about for application of play, I want, I want to have like a robotic soccer team or a basketball team. Oh, boy. Can we have like those kind of things, right? That's a level of fine motor skill. Probably not. Yes. Yeah.

So when do you think we hit AGI in your definition that it's able to be the human at any task? Could be basketball, could be cooking. I think the next two to five years is going to be very, very exciting. And I think we are going to see like leaps in performance of these models as the size of the models are growing. I would say we might actually see, you know, first versions of it like very soon. I would say maybe after 100 trillions of parameters, this is where in terms of number of capacity, in terms of number of parameters, we would be equivalent to a human kind of the amount that is available to humans.

So what is that two more boosts of 10x? So we have like two more boost of anthropic training there, Cloud 4 and 5 probably. So yeah, somewhere around Claude 5 or ChatGPT, 6, something in that range of jumps, two more jumps, which might take another, you said, two to five years. We get some, what feels like smarter than any human on the planet. That was mine. Like smarter than any human on the planet. able to be any human on the planet at any test.

Now, robotics might be hard because you do have some physical fine motor skills that basketball and soccer seem out there. But, you know, to work in a factory or to cook, maybe it does work pretty quickly. Yeah. How do you think about job destruction, societal changes? You know, this is always something that folks in your career and coming out of MIT, you know, debate late at night when you're having. and drinks or whatever you're imbibing, whatever the vibes are.

What do you, when you're sort of off-duty talking with people who are building this stuff, how do you think about retiring a whole swath of jobs that are arduous and painful, but that also do provide meaning and purpose to some degree or employment generally for humans, working in a factory, picking strawberries, writing marketing copy, all this stuff seems to be at risk. So how do you think about job destruction? What's the back channel on this? Is it coming fast and furious?

Or do you think we're going to be able to manage it as a species in a society? I think we can manage it. Like any technology that comes in, I would say it's going to be disruptive. Like you can think about the evolution of technology in all the things that are in our hands. And it changed the type of the jobs that you would be actually having. But it's not going to like replace because right now you can use these systems as an assistant. In some sense, I think I think that these AI revolution, this one, one in particular, is helping us to evolve into a better versions of ourselves.

Like every kind of application that today you see in a generative AI enables is like in the productivity space, right? So it's increased productivity. We can do things faster. We can build things faster because of AI. And I feel like this is going to be the trend, you know. And we're going to frame basically AI systems for basically helping us to become the better versions of ourselves and get things done faster.

for me, the moment that I'm dreaming of happening is the fact that when AIs can actually solve new physics and new mathematics, new science, right? Like if AI can discover new science, I want to give an AI system, basically, the Einstein's equation, Maxwell's equation, and the theory of everything that, you know, cosmologists are working on. If I want to give them there and tell the AI system, hey, continue from here and go figure out what's next thing that is going to happen. Now, if you solve physics, then you can solve the basically the, you know, the way we built structures, like the way we do science. If you solve mathematics,

you can solve the economy of the world, you know, if you solve humanitarian sciences, like the conflicts that we would have, you know, we might actually have AI helping governments basically solve conflicts, you know. There might be so many use cases of AI enabling like new opportunities for work. But this is how I see AI helping us as an assistant, as an elevator of the way we live. Yeah, this is, I think, the most positive spin on it, which is, hey, yeah, you might get rid of some arduous jobs, just like we got rid of being a phone operator. Like, people used to have that job. People used to work in the mailroom. I remember when I was starting my career in the 90s. Working in the mailroom or being a bike messenger was like a major career.

Like there were many jobs that you could do and you get paid really well. Passengers got paid a sick amount of money in New York to run documents back and forth for law firms from Wall Street to Midtown. And they don't exist anymore for all intents and purposes. You don't have to run documents because you obviously, the fax machine and email changed that forever. But yeah, you're right. Like, you know, what if we could actually solve existential problems or, you know, science problems around clean energy, around farming, around calories, around health? you know, maybe we just live with massive abundance. And I think that's what people have to keep in mind.

It was like this short term look at it on the John Stewart show. I don't know if you saw that trending. What was your take on the John Stewart take that like, oh my God, we're just doing job destruction here. I got a little cameo in there because I was interviewing Brian from Airbnb. And he was talking about like, hey, we just don't, we're not going to need a bunch of customer support people answering repetitive questions, which I don't know if that's a great career or not. I don't know if people, and there's some people who love being a customer because they're like interacting with people, but maybe it's not a great job long term.

I think we just get better choices, like as a species. You would basically have a choice to interact more with humans, right? And because let's say for a customer support job, right, why a person would be interested in that job? I would say the human aspect of it, right? Yeah, I like to talk to people. I like to interact with humans. you can do that in the presence of AI

just in a different way. It might actually be less involved than how you have to do it or you're forced to basically do it for that kind of human interaction. I would say AI and intelligence in general is giving us choice. Choices like what is an important kind of element

of human civilization as well. Like the way we evolved actually became like this kind of the most powerful species in the world is by the fact that we have a lot of choice like choices are integrated in our site and the ability to have choice. I think again, as I always say,

like I'm going back to this. Of course, AI would have like, you know, like downsides and upside. It's not all green and everything. Yeah. But I think that the right version of AI is going to be extremely useful. What do I mean by the right version of AI?

One of the things that this, concerning is making this today's AI systems larger and larger as black boxes. If you don't understand what you're doing with the system, that system is no matter how much control, like you're losing control, basically. You're not, you're not going to have like a lot of control in the system. The fact that everybody, like, Anthropic is actually putting like 20% of their workforce on, on explainability, right? So explain what this means for people who don't understand because this is a topic that I

thing is super important and underreported on. Understanding what the machine is doing, it's hard for people to believe that people don't actually understand what the neural networks are doing. So take a minute to explain this to folks. Yeah, definitely. So let's first define like, what do I mean by explanation? Okay, what do I mean when I say I can explain a system, okay? I tell you the equation that I think most of your audience would actually be able to relate

to. E is equal to mc square. That's the Einstein's equation, right? May I ask you this, like, do you think this equation is explainable? That means what? That means like if I have an object, and I know the mass of this object. Yep. And if we know that if this object is moving with the speed of light,

then you can compute the energy that it would dissipate at that kind of fruit. Yeah, you can explain this. Yes. You can explain it in full. It's explainable across time. It's basically like at any given point in time, if I just give you this equation, this is called a physics equation or physical model. Okay? This is the best type of modeling framework that scientists has ever designed.

A physical model is a model that is completely 100% explainable and it explains a kind of real. that you can relate to. Right. On the other side of this. In reality. That's it. Exactly. On the other side of the spectrum, you have statistical models.

I said physical models, and now we have a statistical models. Statistical models are not 100% explain the behavior of a system, but they observe data. And from data, they infer what is basically the construct of this topic that modeling. Let's say a chat GPT. Chat GPT is a statistical model. Okay. It's guessing the next word. It's figuring out what the next thing in this thread should be. Just by observing data, right? Yes. Because E is equal to MC squared, it doesn't need data anymore. It's explainable. You just need to plug in your data and it will always give you like the answer, you know. But if you were to say the quick brown fox jumped over the lazy dog.

This is something that you get. Probabilistic kind of thing. You know, like you have to see whether or do I, this is a statistical model. Okay. Chat GPT and systems like that are statistical models. Now, now scale this statistical models into billions of parameters as well. This becomes today's AI systems, right? Today's AI systems are black boxes because of the fact that we cannot really understand

why if there is an input coming in and an output is getting generated, why this output is getting generated? There is no explanation to why this input output. No citation to a source. Exactly. What's the source material? Explain your work is or show your work is what people tend to do in PhDs, right? And in graduate school, you have to show your work.

How did you come to this conclusion? You can't just solve the math equation. you've got to show us how you solved it. So we get an idea of that. And in these neural networks, people have not been doing that. Exactly. And now we are basically hopelessly, basically trying. There is a term called mechanistic interpretability.

Mechanistic interpretability tries to point into a part of a system, a gigantic system, and tries to say, based on this interaction here, I suspect that this method is basically doing what? You know, this part of the system is, you know, responsible for biases in my system or something. Now, in the middle of these two spectrums that I plotted for you, okay? So I told you there's a statistical models and physical models. In the middle, there is a set of models, which we call causal models. Okay.

Causation. Yeah. Exactly. That means like X implies Y. And if X X implies Y, then what? You know, like basically like more structure into the. the way you're designing learning systems. What I understood from the liquid neural network kind of thing,

and actually I proved theories around this thing, like in my PhD, T says that liquid neural networks are dynamic causal models. There are one step ahead of the statistical models. That means you can understand, to some extent, the behavior of what goes in and comes out, and you can explain a little bit about the cause and effect of tasks inside the system, not 100%, but to a really good extent compared to this statistical. Because they're simple or they're more basic.

Exactly. They are more basic. And the math itself is kind of tractable. The mass itself is like something that you can, you know, as a technical person, you would be able to understand the machine. Now, when I was telling you that we want to do that. the mission of liquid AI is basically is to design AI systems that we can understand and efficiently deploy in our society because we understand the math behind our systems.

It's not like a transformer architecture that I just take it and scale it. Because it scales, it gives your eyes to like very nice kind of capabilities as a black box. But now we are designing systems that are kind of white boxes that at every step of the go, we have a lot more control into how these AI systems are doing decision-making. Yeah. Exactly. Exactly. And this is where, like, I think there are some weird incentives to take the time to slow down.

If you're open AI, Anthropic or Gemini, you're working on some big project, to slow down and say, hey, we don't want to make this model bigger until we understand it a little bit better. There's a perverse incentives here in capitalism and in this race to see who can get to AGI-I for, or who can monetize this first and get their next version out, opening I five, six, seven, you know, Claude version four, five, six, Gemini, whatever. Is there not an incentive to not slow down and not understand it? Like, why put engineers, if you're putting 20% of them, why not put zero percent of them on explanations and, you know, explainability?

Why do explainability when you could just, you know, put more servers on and get more data and and beat everybody else. That's the perverse issue here, right? The alignment of incentives. But I know what's their incentive? Like, let's say, what's the capital for AGI? The market cap of AGI. It's 600 trillion.

Yeah, I mean, it would be the market cap of human existence. It's the world. That's the world. That's the market. So that's it. So that's where these companies are heading at, you know. So the market, like, if you have AGI, as you said, like you can solve the energy problem.

you can solve once you solve the energy problem like what i mean you are basically the most valuable company on earth you know like think about that i mean if you can solve economy like if you can solve politics basically like the structure of governments you know this is the thing that we're hoping to get and there's a race to get everybody gets there at the same time like aGI feels no you don't you feel some people will get to aGI first first yes yes of course of course like a lot of people have i mean of course like i would say open ai and anthropic would be the first bets that I would say, both of them. I don't know which one first, but I think they have a head-to-start,

and they have a lot of kind of information in-house to get there. I don't know about Google. I don't know where their priorities are, but I think the two companies that are focused on really scaling AI systems into more and more kind of powerful beings, I would say at this point, I think it's going to be antropic and opening. But they both have the, they're both, taking the approach that you can just use their system to build whatever you want on top of it.

Of course. If it's open, it's not open, but it's available to people to pay for it. So then if there was the ability to, I don't know, figure out which stock's going to go up, you might have a thousand developers realize Claude and opening higher grade at this. I'm going to make the best trader in the world to go trade stocks. That's true. But that's why the release of those kind of huge models is still, It by itself is actually like a huge challenge.

I would say today we haven't seen those kind of systems yet, but the systems that are coming in the next two years, as I was saying, those systems, even the release of those systems to public, it has to be a rollout. It has to be like a trial and error. We really have to see how, what's the reaction? Internally, like these systems getting massively tested, you know? Like it's not like they're just,

today they get it and then tomorrow they enable it to, to everybody to get access to, right? Like, you need to do a lot of testing of the system and see the capabilities how they come about. Do you think that's why there was that chaos at Open AI is that maybe they felt like that next version was getting close?

And that's why there was sort of chaos because there was that whole sort of speculation. Like maybe they did feel like this thing was getting, you know, AGIS, let's say. Do you think they're close? I don't know. I don't know. I seriously don't know, like, because, I mean, it's all behind closed doors. I really just don't know.

The only thing I would say is, like, it might look like more of a, more of a conflict, like, just on mission, I would say. Yeah. As opposed to, like, how, if the AGI has been achieved or not, you know? Yeah. And this is, the open source models seem to be doing pretty strong as well. Do you think open source wins the day, or do you think open source can keep up with the closed systems or no? The unfortunate answer is no, because the closed models are usually like a lot of resources are going in.

A lot of concentrated resources is thrown at close source months. That's like, that's just the simple allocation kind of task, you know. Just think about like resource allocation, like the massive concentration of resources and in the hand of like Open AI and Nvidia itself, like, you know, Google, Microsoft, all of these companies, right? So that alone is also slows down open source. Open source is going to always play the catch-up. And the gap between the closed-source capabilities and open-source actually grows as well.

You know, that's also another thing. So I don't think the gap is shrinking. So unless there's going to be an open-source move on, you know, like a more... Facebook, you know, to their credit is moving, you know, has moved to open-source models. Apple's doing open-source models, so it's going to be really interesting to see if either of these can catch heat. Delayed open source. Think about how Lama 2. Lama 2 came out.

Delayed open source is again the same story, right? The Lama 2 came out as a commercial license first, right? And then they decided to open sources. Now, let's see how Lama 3 is getting released. So it is important also to think about timing on the open source, like, moves. You know, it is true that some companies are just putting out, like, for example, Mr. Al also played like an amazing role in the open source kind of community, right? Like they put a model out.

But then immediately they put the more powerful models like behind the paywall, right? Got it. So you always have to think about like what is happening in the game. And I would say the unfortunate truth is the fact that closer smalls are really. Amazing. All right. So I think you're hiring and things are going pretty well for the firm. If people want to join the firm, where can they learn more?

And come join the Liquid team. Yeah. Liquid. AI, basically. Like there's a get involved. section where you can today we have like around 25 smartest people on earth

I would say it's a really crazy concentration of people we have people with Olympiad medalists in the team like we are people that are solving literally like really complex problems for us we have on the team like inventors of very important AI technologies

and we have good philosophers also in house we have Joshua Bach also like was part of our organization. And, you know, like,

it's, it's always like, it's a privilege for myself to work with such an amazing team of talent because this is, this has been the power of liquidity. I have been like very good at, uh,

bringing in like a key players into into the space to build like something from scratch, a kind of white box kind of intelligence and then hopefully scale it into something that is meaningful. And again, we are, we are obviously hiring as well. And,

uh, we'll continue success with it and thanks for sharing this crazy vision and uh you know be thoughtful about releasing this stuff let's not end the world let's make life awesome for everybody and we'll see you all next time on this week and service bye bye great job so much