This Week in Startups - How 3 CEOs Use AI to Run $10B in Companies | This Week in AI

Episode Date: April 2, 2026

This Week in AI, JCal sits down with three CEOs building the infrastructure, intelligence, and interfaces for the next era of AI: Jeremy Fraenkel (CEO, Fundamental), Victor Riparbelli (CEO, Synthesia)..., and Nick Harris (CEO, Lightmatter). We break down what's actually happening beneath the AI hype: the data modality LLMs completely missed, why copper is the real bottleneck in AI data centers, OpenAI shutting down Sora, the build vs. buy debate for AI tools, and how close we really are to AGI.AI's Biggest Blind Spot, Tabular Data: LLMs transformed text, images, and code, but 70-80% of enterprise data lives in rows and columns.Copper Can't Keep Up: Nick explains why AI data centers are hitting a wall. GPUs compute faster than they can communicate. Lightmatter's photonic chips push 1.6 terabits per fiber and can 3x training speed.Why OpenAI Killed Sora & Anthropic's Focus is Winning: Victor breaks down why even OpenAI had to learn the lesson of focus, and why Claude Code has every founder talking.Vibe Coding Your Own CRM vs. Buying Salesforce: Jeremy reveals Fundamental built their own internal CRM using vibe coding. The panel debates when building beats buying and when it's a distraction.The Omnipresent CEO: Jason shares how he's using AI agents for root access to Slack, Gmail, and Notion, resurrecting former employees as AI personas, automating SDR workflows, and summarizing employee inboxes while they're on vacation.Are We Already at AGI?: Nick says the rate of progress is a double exponential. Jeremy argues AGI is a moving goalpost. Victor warns of "Future Shock" and societal disruption.🔗 Learn more about Fundamental: https://fundamental.tech🔗 Learn more about Synthesia: https://www.synthesia.io🔗 Learn more about Lightmatter: https://lightmatter.coThis Week In AI is made possible by:PayPal Open - One Platform for all Business: paypalopen.comTimestamps:00:00 Welcome & intro to Jeremy Fraenkel, Victor Riparbelli, and Nick Harris01:47 What is Fundamental? Large tabular models explained07:01 Victor Riparbelli on Synthesia & why OpenAI killed Sora11:09 Claude Code dominance & the Lightspeed founder retreat12:08 Nick Harris on Lightmatter, photonics & the new Moore's Law14:38 Copper vs. fiber: why AI data centers are hitting a wall18:44 Reinventing video: interactive, real-time, personalized21:32 The economics of a custom AI movie23:55 Why Amazon, Google & Meta are building their own chips28:06 Tables have a bandwidth problem too32:27 When will compute be as cheap as storage?36:10 The future of software: every company gets a custom stack38:06 Vibe coding your own CRM vs. buying Salesforce45:57 Jason's quest for root access to Slack50:18 The omnipresent CEO: Doctor Manhattan meets Jesus CEO52:13 Resurrecting former employees as AI personas53:25 Victor's executive changelog for a 650-person company55:07 Whisper Flow & the Plaud Pin1:00:03 AGI: is it already here?1:03:37 Jeremy: we've only solved half the brain1:06:30 70% of Americans fear AI will impact jobs1:08:47 Future Shock & keeping the rope tight*Mentioned in the show:*Wisper Flow: https://wisperflow.aiPlaud Pin: https://www.plaud.aiAthena Executive Assistants: https://www.athenawow.comWHOOP: https://www.whoop.com"Future Shock" by Alvin Toffler: https://www.amazon.com/Future-ShockAlvin-Toffler/dp/0394425863Victor on TWiST, E1776: https://youtu.be/jxET4fq_2eANick on TWiST, E1787: https://youtu.be/FPW2nnEqfMsSubscribe to This Week in AI on Apple: https://thisweekinai.ai/spotifySubscribe to This Week in AI on Spotify: https://thisweekinai.ai/appleThanks for watching!🤖 If you want to stay ahead of the curve on all things AI, make sure to join our community across all platforms:📩 Get the Weekly Newsletter: https://thisweekinai.ai/📺 Subscribe on YouTube: https://www.youtube.com/@ThisWeekinAIPodcast📸 Instagram: https://www.instagram.com/thisweekinaipodcast

Transcript
Discussion (0)
Starting point is 00:00:00 Hey, it's Oliver from This Week in AI, the brand new podcast from the team at Twist. We're dropping a sneak peek right here in your feed to show you what we've been building. If you enjoy it, join the community at thisweekinAI.com, or find us on Spotify, Apple Podcasts, or YouTube. Like, I was talking to a friend of mine. She's an accountant, and she told me accounting is never going to be replaced by automation. I'm like, what are you talking about? It's the first time we're really automating cognition as opposed to just automating the physical part of a job. 70% of them think they'll have a decrease in job opportunities.
Starting point is 00:00:30 Only 30% of Americans are worried in the same poll about themselves, so they all think it's happening to somebody else. I do think that humans will want to play status games. I think we'll find other jobs. I think we'll probably be doing less numerical and logical jobs. It feels like something very big is coming. The world doesn't appreciate that it's happening because most people are not very good at asking questions.
Starting point is 00:00:50 You can taste the singularity at this point. I can't even imagine the end of this year is going to be shocking. Thanks to our friends at PayPal, the exclusive sponsor for This Week in AI. Try the payment and growth platform that's trusted by millions of customers worldwide. PayPal Open. Open.com. All right, everybody, welcome back to not this week in startups, not All In. This is a new roundtable I'm doing.
Starting point is 00:01:16 It's called This Week in AI. It's in the name, folks. Every week, three amazing CEOs, just like on All In or the VC roundtable we do over a twist. three amazing CEOs who are actually building the future and me, an investor in the space and an entrepreneur, talk about the weeks issues and sometimes the bigger picture issues. You can find out more about the podcast this week in AI.a.i. Or if you want to see the YouTube channel this week in AI.com.com. And this is our seventh episode. It's March 31st, 2026. Three amazing guests with us today. Jeremy Frankel is here. He's the
Starting point is 00:01:56 CEO and co-founder of Fundamental. They're building large tabular models for enterprises. They emerged from stealth as a unicorn just 16 months after founding, 255 million Series A, led by Oak with participation from Valor Battery, Salesforce, and more welcome to the program, Jeremy Frankel. So Jeremy, explain what your company is doing and how it's going so far. Things are going great. So what we are doing is we build a foundation model for tabula. data. So what does that mean? So when you know, when people think about, you know, the AI boom or the AI revolution, everyone is thinking about, you know, LLMs. And, you know, for a good reason, right, like, you know, chat GPT, like, you know, a crater breakthrough. Like, you know, you can now,
Starting point is 00:02:41 you pre-trained one model on the entire internet and you understand language and you can use it to power thousands of use cases. But what's less appreciated is that LLMs really mostly solves on structural data issues, such as, you know, text, audio, video, images, coding, but they really didn't impact structural data. And structural data means everything that comes in rows and columns. So think about spreadsheets, databases, CRMs, ERPs, it's all rows and columns, and that's the vast majority of useful data for enterprises. And that part of the enterprise data has never had its tragedy moment.
Starting point is 00:03:22 And so this is the modality we're going after. And for a variety of reasons, it's the one modality that acts very differently than others. And what we're building is really the tragicity moment for the data. So if I can repeat it back to you to make sure I understand this vision, is what I always like to do with founders, see if I can repeat it back. You have large language models. Those are built, as we all know, like guess the next word and transformers. It's based on massive corpuses.
Starting point is 00:03:52 of text-based data, generally speaking, you're building an LLM to focus specifically on tables and tabular data structures that we all know as a, you know, might experience an Excel sheet, or a database, a notion database, a, you know, a SQL database. Am I correct? So it's not an LLM, it's a large tablo model. So it has a very different architecture than LLMs.
Starting point is 00:04:19 Basically, if you look at the LLMs, they're, as you said, that they are built on your being next token predictors, right? It's an auto-aggressive model. The problem with that is that if you look at the way transformers, for example, are applied to allelms, they have a positional encoding as part of it. So like the order of the sentence matter, right? Like if you change the orders of your sentence
Starting point is 00:04:39 in cloud or charging a P, you can get a different output. But with tables, you actually don't want that. And the reason why is imagine if you have a table with a million patients and you try to predict which one of them have cancer. And your first column has the weight of the patient, and your second column has the heart rate of the patients. If you switch the order of those columns
Starting point is 00:04:59 and you first leave the heart rate and then the weight, you shouldn't expect or you wouldn't want to expect a different output. But with LLMs, if you change the order of the data, you get a different output. And that's fine when you're writing an email. It's not fine when you look at the deterministic outputs. And so that's what we've focused on. Got it.
Starting point is 00:05:15 So large tabular model. And LTM is how they actually your first. Are you the first people, to do this, or is this like a known alternative to an LLM? It's very nascent. There are a few smaller companies working on it as well, more in an academics environment and academic setting, but we are the first, like, large company doing that at an enterprise scale.
Starting point is 00:05:36 What is the benefit here? What's the, is it because you'll have a better fidelity, better results, more trustable results, than the problem we have with hallucinations in large language models? Would that be the reason to do this? It's very different use cases. So if you'd look at everything in the economy, for example, every time you swipe your credit card, one of the credit card providers has to make a split-second decision
Starting point is 00:06:01 of whether a transaction is fraudulent or not. When you work with retail, for example, forecasting demand, all of those, or like for example, if you order an Uber, Uber has to make a prediction on the ETA of your driver, each one of those stars is tabular by nature and it's predictive. But when you look at the way those predictions are being made, they still rely on traditional machine-earning algorithms that predate LLMs. And those algorithms still do better than most LLMs at making those predictions.
Starting point is 00:06:31 And so what we've built is a model that can essentially unify all of those use cases into one model to allow you to make much more accurate predictions than what you would otherwise be able to make. Genius. So the company is named Fundamental, and your flagship model, the LTM, large tabular model, is called Nexus. Am I correct there? Correct. Correct. All right.
Starting point is 00:06:53 And we just had Perplexity, CEO, Arvind, who's really crushing it. He's one of your angels, huh? Correct. Awesome. Victor, Riparbelly. I got your name, correct. I hope. We did our belly.
Starting point is 00:07:04 Welcome back to the program. You're with synesthesia. Syngsthesia. Syng-N-T-H-E-S-I-A. You're an AI video platform for business. 90% of Fortune 100 companies are your customers already. over 100 million in ARR, and you've raised over 500 million, $4 billion evaluation. And we're seeing an incredible demo here.
Starting point is 00:07:29 Explain to us how you're different nanobanana, the free services out there, chat GPTs, image generation, and why you exist as a company dedicated to just, you know, working on video models. So I think it goes, we start the company 2017, way before any AI video tech actually worked. So we've been quite a bunch of different companies all the way up to 2026. What we decided on like five years ago was we're kind of looking at the early iterations of AI video, right, which is the models that we're seeing right now, like VO, SORA, which has just been discontinued, our models.
Starting point is 00:08:06 Obviously, very high fidelity, fairly inexpensive to run, assuming you're just like creating single videos and really getting to the point where you actually can't tell the difference, which unlocks a whole bunch of new use cases. But what we figured out five years ago was that the first iteration of this technology was not ready for prime time. They could definitely not make a Hollywood film. It would definitely not make performance marketing ads.
Starting point is 00:08:26 But there was a very real use case in taking all the world's PowerPoint users and enabling them to communicate in video as opposed to slide decks or documents, which in 2022, when we launched the first product was what everybody wants. People want to watch and listen to their content. They don't want to read that much anymore.
Starting point is 00:08:44 And we essentially provide a way for PowerPoint creators to very easily switch to making video and stuff. And that has worked really, really well. And today, you know, we both have our own models. We build voice models. We build video models. We have interactive models. So, Adortage, we can actually talk to in real time.
Starting point is 00:09:00 That's launching very soon. And then we also use a mix of, like, the big models from some of the bigger providers to solve some workforce for our customers. And for folks who want to hear from you three years ago, We had you on this week in startups as part of our next unicorn series before you were a unicorn episode 1776. What a great episode number to have. Why did chat GPT open AIS shut down SORA and why is Elon doubling down on video? He's been very vocal just this week talking about how video is the most important thing.
Starting point is 00:09:39 Take us through your take on that. Obviously you were in video years before chat GPT was even launched. I mean, I think it's kind of interesting that you've been accompanied like Open AI found by like some of the smartest people in the valley, right? Like so those accomplished people still had to learn the lesson of like focus. You know, it's kind of like, I feel like it's almost like unteachable lessons, but they had to learn the hard way. I think it's very obvious to anyone looking at the way that I'm frobic is ripping right now,
Starting point is 00:10:06 that code gen is probably the most valuable near-term use case for all these technologies. And I think open AI probably had a little bit of flying through close to the sun moment, but it decided to do absolutely everything all at once, right? Which often in the PowerPoint, it sounds doable, but in reality, I think anyone who's run a business knows that doing too many things at once is rarely a really good idea. And Froepping focused on no voice models, no video models,
Starting point is 00:10:31 just like code gen, B2B, no freemium, and that is clearly paid off really well for them. So I think my take on what's going on in opening eyes is that they're like, let's cut all the side quests and focus on the market that's really, really going to matter. I think that's going to be probably screwing more towards B2B and it's going to be heavier on code gen and powering just as like campaign explosion of a product that we're seeing being built with the bike coding right now.
Starting point is 00:10:56 I guess Claude's got people shaken. They've done, or specifically it's got open AI shaken. They've added so much revenue. They've become such a darling. Jeremy, I see you smiling about this. This has become notable in the industry. Yeah, Jeremy? No, correct.
Starting point is 00:11:11 I mean, it's funny. It was at the founder retreat a few weeks ago, and everyone was just talking about ClaudecotechardCode. No one was mentioning anything else, the CloudCode. What founder retreat was this? It was a light speed event. Ah, light speed. Okay, the venture calvert firm had it. And that's fascinating.
Starting point is 00:11:27 Nick Harris is back in this weekend. Family you were on this week and start of, I remember this discussion, August of 20203 as well, episode 1787. And so we got it right having you guys on early. And we were talking about, and you were predicting just how important, you know, these data centers were. And that photonics, using light instead of electricity to connect AI chips, would be critically important. And you explained to me, pack on that program, Nick, that energy and data centers were going to be a major, major issue. And here we are three years later, energy is the bottleneck, isn't it, Nick? Yeah, it's exactly the bottleneck.
Starting point is 00:12:11 You know, as a company, we've been focused on driving the future of computing. The central challenge that we're solving at Light Matter is around how do you create a new roadmap for Moore's Law for DeNard's scaling? These are rules that drove computing progress for, you know, our entire lives. I think about being a kid in the 90s in every 18 months. You get an incredible new chip, more performance, all these things. That's over now. And there's only two ways that computers get better at this point. One is big computer chips.
Starting point is 00:12:40 You put more chips in a package. Computer chips are getting to the point where, you know, Nvidia sells nearly $100,000 chips. So those chips are getting really big, and the size of the chip is going to keep growing. And the other way is that at any given time, there's a biggest chip you can build. So networking them together is the other piece. So big chips network together. This is the future of computing.
Starting point is 00:13:02 It's the new Moore's Law. And we power both of these with our product passage. And we also, since we spoke last, started building lasers, which I never thought we would get into. But when you look at the Photonics Revolution, what's interesting is that you've got this device that powers all of the communication for these AI supercomputers, and it relies on the laser.
Starting point is 00:13:24 It's kind of like batteries for electric vehicles. It's a really fundamental, huge part of the bomb, and it drives all the progress and how these computers are going to connect. One of the cool things to tie into the software piece of this is with Photonic technology, we've shown in research and so on that you can actually 3x time to train. If you guys are watching Anthropic and their incredible tear,
Starting point is 00:13:48 I'm hearing about the new model mythos that's coming out, we can actually 3x with 3x faster time to train. Imagine if you have PE to the RT where you've got R times 3 now, so the rate of takeoff is going to go up like crazy. The first companies that adopt this photonic technology for linking up GPUs and AI data centers, the foundation companies, they're going to have an enormous advantage. Got it. And for the audience, again, my typical technique of repeating back, so we all understand
Starting point is 00:14:19 what you're doing, Nick, at light matter. Ethernet cables, that's how we connect computers typically. Obviously, people are consumers using Wi-Fi. You'd never use that because it's very limited bandwidth. But Ethernet, which is typically copper wrapped in plastic versus photonics, which would be made of glass, I'm assuming here in fiber optics, yes, and the throughput is radically different. Maybe you could explain and give us a bit of a primer and then you can sportscast what we are seeing on the screen. Right now, the way that AI supercomputers are built, you think about NVL-72 from Nvidia, they take 72 GPUs and they link them all together in a very high bandwidth domain. So here we're talking about petabit per second bandwidths within a rack. That's all linked together in copper.
Starting point is 00:15:05 What's really interesting about copper is it can't go very far. The cables have to be quite short. And so what you're seeing is the racks are getting packed as tight as possible. Now people are building racks that are a megawatt. So you have a rack that's a megawatt. You have to reinforce the concrete below it because it's so heavy that it's actually a load on the infrastructure. And you're building these custom racks that are just for delivering the cooling to these systems. That's kind of where people are at today.
Starting point is 00:15:31 And the reason they're there is you have to bring the density because the copper can't reach very far. So just to give you an example of the Delta and what we do, we just announced a chip with Qualcomm where with each glass fiber, we're packing 16 wavelengths of light, and we're pushing 1.6 terabits over a single optical fiber. That bandwidth is crazy. It's like 1,600 houses worth of internet. A normal house has one gig internet, so 1,600 houses. So copper really doesn't have very much reach. And the reason it matters is when you're building these AI supercomputers, if you want to have great performance, you want to link as many GPUs as you can tightly together. If you have optics like what we do, you don't need to put it all in one rack. You can separate
Starting point is 00:16:18 it by a kilometer. It travels at the speed of light. There's very little loss in the optical fiber. And you can build giant systems that act like a single brain rather than a bunch of mini brains with 72 GPUs talking in parallel. You could have thousands of GPUs working together on a workload. It drives interactivity, drives time to train. Both inference and training get a huge benefit out of switching from copper to optics. And we're kind of the leader in performance in this space. And just so people can conceive of a pet a bit,
Starting point is 00:16:50 you're talking about thousands of 4K movies from, you know, coming from Netflix every second. So this and probably a hundred million high-res photos from your camera, your library per second. So if you had a hundred, if we as consumers had a hundred million photos somehow in our photo libraries, Victor, you could be sending just, you know, hundreds of people's photo libraries. The entire, you could send the entire corpus of Netflix movies in 10 seconds. Yes, Nick? Yeah, exactly. And there's a kind of a cool analogy here.
Starting point is 00:17:29 We have the chip M1,000 that we announced last year. That chip is 114 terabit per second. So that's 114,000 gigabit per second. That's 114,000 houses worth of bandwidth. And a more interesting comparison is that that is about the bandwidth of the cables that connect North America to Europe for the internet, the undersea optical cables. So we're building chips that have just an obscene amount of bandwidth. And it's all needed to drive AI scaling.
Starting point is 00:17:57 or Victor, in terms of your data usage when you hear about light matter and the impact it could have, what goes through your mind, Victor, because obviously you're working in video and your data centers, I'm not sure what your standard platform is and where you host, but maybe thinking ahead to the future, how do you think about what Nick is building? I think it's super exciting. So there's kind of two big ideas we found out Synthesia. The first one was like, as AI increasingly can generate data, the marginal cost of creating video, audio, also other content has got dropped to zero, right?
Starting point is 00:18:33 Both the dollars, both very much in time required and skills required. And then we're in the middle of that right now. But this is still video, as we know it today. It's a broadcast medium. You make one video, you put it on YouTube, and everybody watches exactly the same version of it. The second part of our thesis was always around, when we invent new technologies as humans,
Starting point is 00:18:51 we always invent new media formats that are native to those technologies, like they have a podcast or TikTok, video wouldn't exist without modern technology. And for us, the big question is, like, what does video look like if you were to reinvent it in 2026 with all the new primes we have around us, right? We have LLMs with essentially intelligence on tap. We have offline video models that can create extremely high-quality content. We have real-time video models that you can interact with, advertise you can talk to, canvass
Starting point is 00:19:18 that can be drawn in real-time, and a whole bunch of other, like, cool technologies around it. And so what we're building for and what we're actually launching, it's in private beta right now, launching in a couple of months, is real-time video, which is the idea that if you, to take one of our use cases, if you're a salesperson and you're doing a bunch of training to understand like the competitive landscape or new product that you are launching, instead of just like receiving a video that you sit down and then you watch it and then, you hope you understand it. It's got to be an interactive experience. It's going to be maybe first you could consume some content. You go into an agent, you roll play with it. It pretends to be a customer. You have to answer questions, overcome objections. Then you go to another thing where we actually in real-time draw a diagram of a customer's potential tech stack, how you've got to work with this, how you're going to integrate it. That's a very different type of video, which is almost closer to maybe like a game or like a website, something like that. But one of the bottlenecks here, right, is of course that if we're actually going to do this with video
Starting point is 00:20:16 and we're going to do it with like avatar models and we're going to draw things in real time, that's going to take up a lot more bandwidth. it's also going to have much high inference costs. And so the more we can reduce these, the more accessible this becomes. So I think in the next couple of years, we'll see this becoming a new type of interface that's going to emerge. But for it to really take off and just be, you know, every interact we have with the computer could be done with technology like this. We need the cost of serving that content to drop like very significantly.
Starting point is 00:20:43 I think that's the core of what the problem Nick's working on. So I think it's very exciting. Nick, when we have this ready for Victor to, experience it and like it's we could probably do a deal right now that it could be one of your beta customers because it would be amazing for Disney. I mean, I'm thinking in a consumer mind frame to sort of help the audience follow along here. But imagine, you know, Disney releases Mandalorian and they had done a deal with SORA to try to get the IP to work. Now imagine with Light Matter being able to enable Victor's company to be able to make a short film with Grogu and the Mandalorian.
Starting point is 00:21:21 and you're talking to them in real time, and it's making that in real time. That's just... Yeah, that's absolutely... To put some economics on that, right, I think, you know, if you were to do that today, let's say you were to, like,
Starting point is 00:21:32 personalize, like, a one-hour movie for a kid from Disney, that would cost you, like, a lot of money, right? If you say, like, an eight-second clip with, like, a state-of-that video model costs, like, one or two dollars today, you can add that up for, like, an hour of eight-second clips, right? That's not going to be sustainable
Starting point is 00:21:47 within, like, a $15 per month. It would be, yeah, If it was $6 a minute, if we just made it like $6 a minute, 120 minute, you're talking about $700 for a custom movie that you can live in. Exactly. And we're not that many years ahead of like, I remember when I was a kid, right? And I had to like call my dad and ask him if I could download like a 10 megabyte file because it was like ADSL and you were like mirror how much you would download. The idea of doing a video call for an hour with someone across the world. Like that's an after the doodicrous idea, right?
Starting point is 00:22:15 But probably in like X amount of years, this is going to be like completely. completely normal. We're going to be just generating content in real time in front of people, and we're going to be able to offer that, like, you know, within the subscriptions that these services charged today. Nick, you were going to add to this. Yeah, Jason. Yeah, we're actually busy building chips for a bunch of companies. We typically work with hyperscalers to build their own chips. Think about like the Google, Amazon, Microsoft meta type companies who are building their own hardware to do both training and inference. And then we also work with semiconductor companies, both GPU companies, as well as networking
Starting point is 00:22:50 companies. So those are the people we build for. We're building a ton of ships right now. So I would say in the next year and two years, you're going to start running on light matter hardware. These will be in the new data centers. Think about like the Texas stuff. Yeah. Core weave. What's the one? Not Star Bay. Stargate. Another great film. Speaking of. Yes. Excellent film. Yeah. And so there's a picture of I think that's Stargate. And what you see in the middle is that plus I think is, I think I was talking to Jensen or the CEO of Corweave about this. Somebody on my team will tell me, I believe this is Corweaves Data Center. You have the cooling there, that bottom line that looks like memory chips in a motherboard.
Starting point is 00:23:33 These are starting these data centers are starting to look like giant motherboards. But I think those are the cooling apparatus where the contained water system. What do they call a closed loop water system? Yeah, closed loop water system. So that was Crusoe's data center. I remember the CEO was walking me through it on a pre-exam. previous episode. That's the closed loop water system and the data exchange. Pretty compelling stuff. Jeremy, when you look at all this, oh, by the way, Nick, Amazon making their own
Starting point is 00:24:05 chips. And I don't know if they're a customer, if they were, you could say so, but you gave us the sort of like. Amazon's are called Traneum for training AI models and inferentia for running inference models. Somebody at Amazon needs to go to branding school. That's a little too on the nose. Training many for endure. I mean, what did they do? They ask chat GPT to come up with names. But these are going to be
Starting point is 00:24:30 dedicated chips. And I think Broadcom generally builds people their chips. Is that typically what happens? So if we explain it to the audience, you have Nvidia. They work with TSMC. They're making their own chipsets. They're the leader of the pack. Every single other
Starting point is 00:24:46 hyper-scaler tensors from Amazon, tensors from Google. Then you have these infrancia Traneum. And MTIA from Meta. MTIA from Meta. So explain to the audience
Starting point is 00:25:01 why people are doing two different supply chains, Nick, and then we'll get to you, Jeremy, on your thoughts on this next wave. Well, I mean, how that'll affect your business. If you look at the incredible spend, I mean, they're over a hundred billion year.
Starting point is 00:25:15 They're like 180 billion year, I think, is what Google announced they spend. I think Amazon was over $200 billion for the year. When you're spending that kind of money, developing your own custom silicon is a little bit of a rounding error. So I think that they're really looking at these costs. They're trying to figure out how they can optimize costs,
Starting point is 00:25:33 and they think they can build their own solutions. Now, building a chip is one thing, but building all the software and the ecosystem around it is another. And that's where Nvidia has had decades of experience building out the moat there with CUDA and everything. But everyone's trying to build these chips. chips. And the reason is that it's a race on the infrastructure point. People are trying to get power. They're investing in these micro, you know, nuclear reactors to go power the data centers,
Starting point is 00:25:59 100 megawatts each. So you get 10 of those and you've got a gigawatt data center. They're working on that power delivery. They're working on building the chips. They do their own, you know, these hyperscalers are becoming like very heavy duty infrastructure players from cement to energy, all the way to chips. And obviously the software stuff on top. There's just so much money in this space that they're all kind of making the bet on doing it themselves. And it's all in service of powering technologies like Jeremy and Victor's. It's really about the apps that run on top of this and getting the cost to the right point because the AI models are incredible. But we've got to keep driving down the token cost and driving up the inference rate.
Starting point is 00:26:39 And then we'll be able to keep unlocking incredible things like, you know, custom movies. And you're going to need blazingly fast interconnect for that. Jeremy, I'm assuming you're building on CUDA, which is the proprietary layer for coding and sending jobs to Nvidia hardware, correct? Correct. And if you were to consider other platforms, other hardware platforms that were non-NVIDIA, have you considered that? And is there a path for you? Or would you have to maintain, you know, CUDA plus some other open source software? I guess there's some abstraction layers for CUDA now to get on AMD processors.
Starting point is 00:27:20 So as the CEO, how do you think about where to spend your energy? Is it just too much to even consider other platforms? Or are you like Amazon, Meta and Google, saying, hey, we need to have two swings at bat? I very much think that, you know, we are in the process of exploring different chips as well. You mentioned Traneum is one of the chips who are in the process of exploring. And the idea here is that, you know, we we don't want to just be dependent on one, you know, hardware, one type of chip. Of course, it comes with, you know, like Qaeda gives you a lot of advantages. And, you know, it's not easy to switch away from QDAR, but it's definitely something that's when the process of exploring.
Starting point is 00:28:00 And, you know, what Nick is doing is really, it's really exciting, right? Because, like, you know, the funny thing about everyone knows, but, you know, with video, like, you know about the amount of data that is being moved from one place or another. but people also don't realize that you have the same problem with tables. If you think about a table with 10 million rows and 100 columns, which is not that big of a table, right? It's like, what? You have like a billion cents. That's orders of magnitude more than the context,
Starting point is 00:28:27 the largest LLMs can even take in, right? The largest LLMs can maybe take 100,000 rows. But when you're working, for example, with banks on fraud detection, you're working with billions of rows. So, like, you just need and, you know, and they're like, you know, millisecond matters, right? Like you make a decision, like when you strike a crackout,
Starting point is 00:28:44 you don't want to be waiting for 10 minutes before you get an answer. You just want to get an answer right away. And so the amount of data out there in tables is just massive. Like we've been, you know, talking to a few companies where like they, like, think about every IoT sensor,
Starting point is 00:28:59 every time, like, you know, you get some data. That comes in some form of structure full. And like the amount of data that they, that they are dealing with is like petabytes of data. And so being able to move data much faster and having a lower latency, and as Nick said, you know, also lower cost will really be essential. Yeah, I mean, if you were to think about, I have these air things. I don't know if you guys care about error quality in your homes,
Starting point is 00:29:25 but it turns out like in your office in your home, like CO2 and radon, all this stuff, very important for health, very important for like cognitive function, especially CO2. So I have these air things. It's taking recordings in six rooms in three different houses for me. The amount of data that just one person consumes, or let alone your whoop or your FitFit, like how many heartbeats is whoop? And shout out to whoop.
Starting point is 00:29:49 They just raise money at a $10 billion valuation. Like what's the data processing there when they have to do my sleep and my recovery and my run and my heartbeat? I mean, my Lord, it's huge amount of data. Yeah, exactly. It's unfathed from the amounts of data. Join the community at this weekendai.a.i. or find us on Spotify, Apple Podcasts, or YouTube.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.