This Week in Startups - E1005: Scale AI CEO & Co-founder Alexandr Wang creates training data for all AI applications to improve machine learning, shares insights on the future of autonomous vehicles, China’s AI advantages over US, importance of humans focusing on higher-value work & next major trends in AI

Episode Date: November 29, 2019

1:04 Jason intros Alexandr 2:19 Alexandr shares his personal startup history 5:17 How & why did Scale start? 8:26 What is the best example of Scale in practice? What problem are they solving? 10:44 Video demo of Scale's platform 15:34 Acquiring the scale.com domain name & insights on the unique spelling of Alexandr 17:31 How does Scale deal with data-sharing between customers? 21:34 LIDAR vs. non-LIDAR... or both? 32:29 When will we have capable self-driving vehicles from Palo Alto to San Francisco? Over/under 2030? How will gov't regulations affect self-driving? 36:03 China vs. US in the race of self-driving 41:22 Explainability in ML 47:26 Does it matter that we sometimes don't know the answer to ML systems? 51:39 Should explainability have to be proven in ML? 55:13 How should inherently biased data-sets (like US justice system) be handled via ML? 1:00:00 Importance of focusing on higher-value work 1:02:41 Are dangers of AI overblown? 1:08:50 Will "General AI" happen in our lifetime? 1:12:26 What's the next major AI trend after self-driving? 1:23:46 Does Alexandr remember a time before the Internet? 1:26:35 Jason plays "good tweet/bad tweet" with Alexandr

This Week in Startups is brought to you by Calm. Sees the day and Sleep the Night with the help of Calm, the number one app for sleep. This Week in Startups listeners get 25% off a Calm premium subscription at Calm.com slash Twist. That's C-A-L-M-com slash Twist. LinkedIn.

A business is only as strong as its people, and every hire matters. Go to LinkedIn.com slash Twist and get a $50 credit towards your first job post. And Cabbage. Get the money you need to run your small business today. Go to cabbage.com and use code twist to get a $100 credit on your first loan statement. That's K-A-B-B-A-G-E.com and use promo code Twist.

Terms and conditions apply. Offer ends November 30th, 2019. Apply for the next launch accelerator cohort. Applications are due December 23rd. Learn more and apply at launch accelerator.co. Hey, everybody, welcome to this week in startups. I'm your host, Jason Calcanus. And this is the podcast where we talk to founders about their vision for how they want to change the world.

And today, I've got an interesting cat on the program. His name is Alexander Wang. He is the CEO and co-founder of Scale AI. You got the domain name, scale.com. That's like a million-dollar domain name. We'll find out what he paid for it later. And I guess the thing that most of the domain. most people would think is remarkable, is candidly, that you've raised over $100 million in your

last round of funding. That's a lot of money. It's quite a bit. It's quite a bit of money from Founders Fund and that I think are 22 years old. 22. Yeah. 22. So that's annoying to be young and successful because then every interview starts with your age. A little bit. It's annoying. I had it happen to me, but it was like 23-year-old publisher. of Cyber Server, a 25-year-old publisher of Silicon I'm reporter.

And I was like, why does my age matter? Now, I tell you, when you hit about 35, 40, they don't mention it anymore. Because they're like, well, you're 40. You should be doing interesting things or be successful in the world. But you've been running this company since you were how old? 19. 19. Now, were you a Teal Fellow or something?

How did you get into the game? No. So I have a fun little history. I grew up in, well, Salamos, New Mexico. So both of my parents are physicists. and they worked at the National Lab in Los Alamos. Yeah. I tell people about that lab.

It was the lab where the atomic bomb was originally built. So the Manhattan Project started in Los Alamos. It was very secretive at that time. Where do they call that lab? Los Alamos National Lab. Los Alamos National Lab. It's pretty boring. It's pretty boring. Yeah, yeah.

And then I... It's a government sponsored lab. Exactly. Yeah, totally government funded. And then growing up in high school, I did a bunch of programming. I did all these coding competitions. And I was getting recruiter inbounds in high school. So after high school, I actually came out here to work.

I worked at this company Quora for a couple of years. Yeah, we know it. They do the Q&A site. Yeah, Q&A site. How did you get that job? You just applied and they saw your code and they were like, okay. They recruiter inbounds because I was on these, I was an anonymous person on these coding competitions. Ah.

And then you could just go into a coding competition. nobody knows your age. Yeah. And then do you teach yourself how to code? I, uh, well, the internet taught me how to code. Uh, so, uh, I guess the internet. You just looked it up. You found courses online on YouTube or?

Yeah. Uh, I don't, it's hard to remember. I think I just Googled around. Anyway, so, so I, uh, I worked at, I worked at Cora for a couple of years doing engineering, infrastructure, et cetera. And then, um, no college. No, well, then I went to college after that. Ah.

Uh, I went to MIT. Um, and then got. basically bored after a year and started scale. So you left? I left, yeah. So your parents are heartbroken. How about you leaving MIT at that time? For now, yeah, for now.

They're still heartbroken, even after raise $100 million? My God, these parents have high standards. Yeah, yeah, I mean, it's a meme, but it's true. Yeah. All right, listen, mom and dad, he's going to build a building at MIT with your name on it, the family name on it, so cut him a break. It'll be okay. You'll be a professor emeritus at some point.

Yeah, well, one can hope. One can hope. No, it wouldn't be Professor Emeritus. That would be somebody who left. You'd be like an honorary professor. Yeah, that would probably, that would be the way I'd appease my parents. Yeah, because you went to, so you went to work for a couple years, then you went to MIT. That is not the way to do it because you're going to be sitting there and everything's going so slow and everything's theoretical.

Yes, that's exactly what happened. Yeah, that's... So you went from like running fast, you're driving race cars, and then they like put you in the pit in the go-carts. and they're like, here's some go-carts. Yeah, and if I'm being honest, I think the, I think the speed, the slower speed of school is sort of what got me agitated enough to eventually start the company. I think there's an alternate world where I continued working at companies after Quora for many, many years.

So what was the vision for scale? How did you get the idea? When did you have the idea? So our mission is to accelerate the development of AI applications. I think we fundamentally view AI and machine learning as kind of a once-in-a-generation shift in technology. It might be once-in-a-species, by the way. Yeah, I mean, it's a- Depending on how this goes. Yeah, we'll see. We'll see.

It's a, I mean, it's obviously very hyped, but I think we hold that belief quite strongly. Do you think it's as big as the internet or bigger? The internet itself took billions of people and connected them for the first time. Yeah, I... Bigger than the internet and bigger than the silicon chip being, you know, CPU is being created? I think it's more comparable to

to the advent of computing than it is to the advent of the internet. Because, because it's an enabler of all these things that previously had to be done by humans and now can be done by machines. Got it. Okay. So for you to rank them AI, computer, internet,

or maybe computer AI internet? We'll see what happens between AI and computer. We'll watch, intently over the coming decades. Because computers did change our day-to-day lives pretty significantly. Yeah, well, AI will as well. Yeah. With autonomous vehicles and all these assistance on your phone.

And I think it'll go on and on and on. There's a lot more applications. All right. So that's the backdrop. And then you have some insight on what was holding back AI or something? Yeah. So the big reason I went back to school, actually, was to study machine learning. So I was at Quora.

It was a very machine learning driven company. but I didn't have that strong in academic backing. And so I went back to MIT to really study this more deeply. And then I had all these ideas of products I wanted to build, but there was sort of an elephant in the room problem, which was how are you supposed to get the data to build these machine learning models that you could integrate into a product? So give me an example of that.

And then define for the audience that's not super familiar with the term machine learning, what is the difference between saying the word machine learning and AI, for people who hear them together. How would you explain what each one is? Yeah. So machine learning is a subset of AI. It's sort of a particular kind of AI where in particular we're training these, you sort of are writing programs that are able to do various things that humans normally do

or various tasks that traditionally require human judgment. And they're able to do that by feeding them lots of data and sort of a particular brand of AI, if you will. So we pick a task that humans have done with their brains, which is some combination of logic, intuition. Who knows what human brains are how they're making decisions? Exactly. A lot of debate about that. So you feed a bunch of data to a machine learning algorithm. Yep.

And then it gives you an answer that it thinks would approximate a human's answer or the best answer. It thinks it would approximate the best answer. But the only way it's going to know what the best answer is, is through all this data that humans have created. Got it. Let's come up with the most illustrative example. What's an example that when you gave this example to VCs, they threw money at you? Well, the one that has really captivated the world's attention is autonomous vehicles, right?

Sure. And it's a compelling example because, first, nobody likes driving, but also driving is very unsafe. And there's a lot of risk in driving. Sure. And so the... A lot of state. Yeah, exactly. And so the captivating sort of machine learning model is one that can take in all of the camera data and other sensor data from the vehicle, understand everything that's going on around it, something that's very easy for URI, but currently, or at least before machine learning was very difficult for machines, and then can determine the best path to take and figure out how to drive on its own, basically. Got it. So we see the lane markers, double yellow markers, double white markers on the highway. We know keep the car between those two lines as smoothly as possible. Yeah. We see somebody deviate from their lane into ours. We know to slow down, give them some room, maybe they're drunk. Machine doesn't know that inherently. We have to teach at that. Exactly.

And what does scale.com do that Tesla and Waymo don't already do? Yeah, exactly. To solve that problem. Because they're solving that problem. Do they use your software and do they need to? Yeah. So that's a great question. So the core problem, as you just laid out, is that machines don't know what to do unless

they have data that actually tells them what they're supposed to be doing, right? And so what that means is one of the huge bottlenecks for machine learning is data. It ends up being like data that tells these algorithms, tell these models what they're supposed to be doing. And that's where that's where scale comes in. What we are is sort of this data refinery, if you will. We accept a bunch of raw data from our customers. We go through and process it and we sort of, we tell the machine what it should be doing. For example, given an image taken by a self-driving car, we would outline these are where the people are.

These are where the cars are. These are the lane markings, et cetera, so that over time these algorithms can learn those things. You have a video of that you can show right here. So here's a video of. Pacific Street in You have a better eye than I, but yeah. I'm going to take a guess. That's one of those streets and it's one of those streets and I see you are highlighting cars,

you're highlighting people. Exactly. And the machine is figuring out, okay, that's the approximate shape of a Dodge pickup truck, that's a Toyota Prius and these look like the silhouettes of people. But that's a human telling the machine that's what it is for now. Yeah, exactly. So the core way that our whole pipeline, works is that a lot of work is done behind the scenes by machines and our own AI models originally.

And then humans basically give input and correct mistakes to make sure that the end data is extremely accurate. Because that ultimately is what's important for the safety of these systems and for low bias, etc. All these things that are needed imperative for machine learning to be performing. So you would go to a customer, is Waymo or Uber a customer? Yeah, exactly. They're both customers? They're both customers. Got it. And you can say that. It's public knowledge. Yes. Got it. Okay. So they're both customers. So they would give you videos of their cars driving and then you would annotate it for them and put that data into a database somehow. That's exactly right. So for example, if they gave us a video like this, you'll see originally the first step was a human drawing a box. Yep. And then a machine learning model that's already pre-processed through all this data has determined the path of that vehicle over time.

time. Right. And then and then we confirm that all this is correct and then send that data over to the customer and they train machine learning malls on top of it. Got it. And this is how I guess one of the cars got fooled. Somebody drew on the ground like an arrow turning and a car followed the arrow, which how human would do too. But they basically drew a turning signal to see if like it would fool a self-driving car. And of course it did. I didn't see this news, but I would believe that that's what would happen, basically. Yeah. And that's what would happen to a human, by the way. So I thought that was the stupidest prank ever.

They're like, look, we can fool a machine that's driving cars to make a wrong turn. It's like you would also fool a human to make the same right turn. It's like taking the do not enter sign off of the off ramp and putting on ramp on it. Like, congratulations, like Fight Club. You just did like some crazy stupid prank. Yeah, exactly right. I mean, in a lot of ways, they will have some of the same challenges that humans have been driving. All right.

When we get back, I want to understand if you are storing all this data and annotating it for one company, or is this some sort of grand plan to have it go across multiple companies so everybody doesn't have to recreate the wheel when we get back on this week in startups, so to speak. Are you struggling to sleep while you're not alone? One in three U.S. adults does not get the sleep that they need. and not sleeping enough, that affects all your cognitive function. Think about it, like learning and problem-solving and decision-making, all these things we do as founders every day. Sleeplessness causes people to also be prone to more accidents, weight gain, and depression.

But when we sleep and we get that great night's sleep, you know what I'm talking about. Then you're more focused and relaxed, and you're actually happier. So that's why we're partnering with com.com.com, the number one app, for sleep. You're going to get a library of programs from Calm that are designed to help you get the sleep, your brain and body needs, like soundscapes and over 100 sleep stories. And I do these with my kids and they love it and I do it myself. And it is amazing. And here's my associate, Prech, who has been having trouble sleep because his boss is too intense. And he goes through it and he finds some nonfiction and he's looking, ooh, a cruise on the Nile, some Matthew McConaughey. What? Matthew McConaughey, reading some sleep stories. He looks at the AS SMR, painting, Beauty and the Beast. And then he looks at sleep. He does lullaby to the stars. Ah, so relaxing.

So here's your call to action. This week in startup listeners, we'll get 25% off a com premium subscription at com.com. slash twist. That's right, com.com. slash twist. C-A-L-M.com slash T-W-I-S-T.

40 million people have downloaded a com, and it was Apple's 2017 app of the year. Find out why at com. slash twist. Thanks again tocom.com. I'm an investor in the company. I love the company

and I'm so proud of the work the team over there is doing. It's just such an amazing app and such a great story. Okay, let's get back to this amazing episode. All right, Alexander Wang is here. Blah, blah, blah,

20-something year old. Who cares? He's young, he's smart. Don't worry. I'll get old eventually. You'll get old eventually and people will stop saying and that you're 22 and whatever.

And he is building scale.com. That's like a six or seven figure domain name? No comment. No comment on the price of that domain. It's not cheap. It's in the dictionary and it's less than six letters, so it's not cheap. It was not, it was not inexpensive. No. Not inexpensive, but boy, a good domain name does help the branding. Does it not? Well, I guess, I guess we'll see. That'll happen in the out years. We'll see what happens. It's baller. Everybody's like, oh, it's your email. I was like, Alexander at scale.com. Just leave out an E. pick the right E to leave out

So wait, your name is Alexander but you're missing the E between the D and the R. Yeah, that's exactly right. It's my... Typo on your birth certificate? My given name, intentional, very intentional. So like a joke by your program or parents

in some way? My parents wanted eight letters in my first name because they're Chinese. And that's good luck, eight. Eight is very good luck. Yes. Extremely good luck, yeah.

And they literally did that for good luck. And it worked. Yeah, it worked. I mean... I have a friend I play poker with. He's extremely superstitious in his Chinese. Yeah. Bies in when we play poker if he's not playing well.

If he's losing, he buys in for $88,000. That's a chunk of change. Does not look at his cards and pushes the chips in blind in poker, Texas Holden. And then two or three of us will just call him if we have an ace or whatever. And then he wins every time. Oh, wow. For 88. thousand dollars i've seen him do it five times in a row he's a legend in los angeles yeah that's all i can

say he sounds really good yeah i i love to play poker with him it's pretty it's pretty amazing because when you think about the worst hand in poker is like typically 80 20 or something like that so you even have a chance but yeah it's it's quite a thing to see um so before we left for the break i wanted to know if you're doing all this data are all these companies programming their algorithms and data sets in a silo over and over and over again. There's no sharing across these companies. Yeah, that's exactly right. That's crazy. Because they all want to own the, they all want to win in the end. So they're all not sharing their data. Well, I think, I think this is what it, what kind of comes down to is what is IP in, in the machine learning context. And I think it, it, it's intellectual

property. Exactly. And it would be equivalently crazy if, uh, if every company in the valley were to just develop their code out in the open. I think it's like... Like open source. Exactly. There's a little bit of open source, but... Yeah. But the underlying technology, people do open source,

but Google's not giving their algorithm away. Yeah, exactly. Yeah, or open sourcing that. And so ultimately, it's not that crazy. I think it means that it means that there's an incentive to do things that are novel and interesting and produce value. Okay. So you get the variability of 10 different people, trying 10 different data sets,

but you lose the... efficiency of 10 different people working off a common data set. Yeah, I mean, the core, by the way, the core of your stance on this, it's like, it's a very similar thing to whether or not you believe in just free market economics in general. You have a lot of people running around, running in the same direction, running in different directions. And it's a like, and if you believe in that approach versus a sort of planned economy where

there's high efficiencies, but maybe low variance and low, low chaos, then, then I think it's really fine. Yeah, but because you have 10 different people competing with 10 different data sets and there are big prizes at the end, like whoever solves self-driving wins $100 billion or a trillion dollars. Yeah, you've incentivized the large groups of influential pools of capital to pursue it. Yeah, exactly. There is an open source company, isn't there? There's somebody doing an open source company. Do you know about this company? They're going to open source the data and do exactly what I'm talking about. I forgot the name of it. It is not a new idea. And in In fact, in the research community, people open source of data quite commonly.

Really? Yeah. So a lot of people say the start of all of the machine learning, part of the deep learning life cycle was this large dataset called ImageNet, which was published by this Stanford professor, Fei-Fei Lee, who basically produced this large data set, and then it really kicked off the sort of machine learning, deep learning hype. Wow. So because he got all of those open source Creative Commons images up there and then had everybody train them, you had a trained set? Yes, exactly. So she had a, she published this large, she and her lab published this large, this extremely large data set of millions of images classified with what was in those images.

This is an orange, is an apple. Yeah, there's an orange as an apple. There's a cat. There's some rare ones. Like this is a rare kind of fish, et cetera. This is a cat eating an orange. Not quite that detailed. So this is a this is this was the beginning, mind you. Yeah. And basically that that created this open source data set that then sort of the whole world could work on top of. So that would be the example of centralization and open source being perhaps something between, let's say a socialist communist singular government approach versus a democratic capitalist approach.

there was open source, which kind of sits somewhere between the two, doesn't it? Yeah, ultimately the, the, actually, maybe it's socialist. I'm trying to figure it out. Yeah, I do think that open source is, I'm not going to comment exactly how to align in these economic situations. But I think very much so, like in general, the trend of providing some, like, core underlying infrastructure for a large group of people or a large community of people who are all iterating or building on top of that infrastructure is very valuable.

In self-driving, is it the video of what's on the road, or is it some other way of recording it that is the most effective? So we have LIDAR, Google bet the farm on LIDAR, Elon bet the farm with Tesla on video cameras. Everybody thought Elon was an idiot. Turns out, I'm hearing now that people are starting to think the cameras are getting so good and the data sets getting so good. that cameras will win the day and LiDAR will be ridiculously unnecessary. So we actually... Which is true. So my personal opinion is that I do think both sensors have different advantages, and fundamentally

they're both very good in different scenarios. So if you explain. So we actually, we published this blog post about this because we obviously see a lot of LiDAR data, a lot of image data. What's the name of the post? Do you remember the title? LIDAR versus cameras, Elon versus... Larry? I think it's called

LiDAR versus cameras or something like that. We'll search for it on scale.com. But yeah, I think there's different scenarios where both are good, right? So LiD is very good. First of all, at giving you a 3D map of everything around you. That turns out to be very valuable if you're planning very

careful maneuvers. And it's very reliable in giving you that 3D map. Yes. It makes a map that is incredibly well refined. Here it is. Is Elon wrong about LiDar? Is Elon wrong about LiDar? Exactly. There you go.

It's also very good in dark scenarios because the LIDARs create its own light. So you know exactly what's going on around you. But it's bad in other scenarios. It's bad when there's a lot of snow or it's bad when there's a lot of fog, et cetera. Why is LIDAR bad in snow and fog? Because it's shooting out these little lasers. And snow and fog are both very reflective and basically screw with how these laser. Got it.

So it makes an imperfect model in those situations. Exactly, an imperfect 3D model, or at least one that you'd care about. So, for example, if there's a plume of smoke, a LIDAR will catch the plume of smoke, but you're actually, it's fine if you drive through a plume of smoke. Right. Yeah. Okay. And so can machine learning now, is it able to reconcile when both systems are on effectively

to know, hey, the LIDAR is built this perfect model, but the LIDAR is hitting something that could be smoke, and the camera's like, or it could be a brick wall that just suddenly appeared. And the camera's like, no, it's smoke. We can tell because cameras are better at detecting smoke and snow. Yeah, that's why fundamentally you want both, right? So both is the best system. Both is definitely the best system.

Because so the place where the camera stuff breaks down is that right now, if you were to look at sort of the state-of-the-art computer vision models that work on cameras, they're accurate maybe like 99% of the time, which sounds like a lot, but not 99.9% of the time. 99 sounds like a lot until you drive 100th hours. Yeah, exactly. The 100th hour is not, or you drive 100 seconds and the 100 second is not, and that happens to be the second where a boulder rolls into the street.

Exactly. Yeah. So it's pretty important that you get like these asymptotically difficult levels of quality. And you can do that. You can actually do that if you have multiple sensors that have different strengths and different weaknesses and can sort of play off of one another when you need it. Which is why you want both. You really want both.

It's the software doing that today currently, or are people just picking one system and going with it? No, no, no. They very much so work together, like on a Waymo vehicle or on a cruise vehicle or whatnot. They very much work together. So in particular scenarios, you'll pay more attention to what the camera tells you. And others, you'll pay more attention to them. But the camera's the default now, right? That's not true.

No? Okay. I think a lot of, a lot of these cars still drive very much influenced by the LIDAR. Really? Yeah. It's a really good sensor. If we had LIDARs on our phones, life would be great. Why?

It gives you, again, it gives you a very accurate 3D map of the world around you. So you can basically do a lot more with your surroundings. And that is, I thought Google was starting to put that kind of depth sensing in there. They're not doing it with LIDAR. They're doing that with some other sensor. Yeah, there's a structured light sensor on the front of your phone. Now most of these phones. haven't.

Yeah, that does face ID or whatnot. Ah, structured light sensor. Yeah. And that can do depth. And so it knows the depth of my nose, my eyes, or all that kind of stuff, so it knows it's me, not you. Exactly. And that's why a flat photo doesn't work because it would be pretty hard to fake. Yeah, exactly.

Exactly. Although I heard Asian faces, the original versions didn't work for. Or like, people who looked similar who were Asian when the white guys created the algorithm. at Google or Apple, it didn't actually, Asian people could unlock each other's phones. Did you see that? I saw the articles. Is that true or not?

This gets back to the core of the issue, which is machine learning is really hard because it's all about the data. So who knows what was going on in the underlying data that trained those algorithms? It was some white guys camera roll. He's like, here we go. Well, either way,

this is why it's really important to have really good data in algorithms, because otherwise they'll do weird things. Well, it would make sense, right? Because if the algorithms were built off of a database in China, let's say the flicker of China and 99.99% of the photos were of Chinese descent, he would be optimizing for that dataset. And if you did it in America and whatever percent was white,

and the percentage of Asian might be whatever, two, three, four percent, let's say, it's not going to be as refined. That's exactly right. Yeah, this is why, I mean, this is why when a lot of, of our customers and a lot of, I think companies doing machine learning today think about it. It's really about how do we constantly improve with more and more data that sort of fills in the gaps and makes the whole system holistically more robust over time. And you guys build which piece of this, the data storage, the algorithms, I'm still unclear

as to which piece you build. Yeah. So what we do is all this, so all this data that comes in, let's say it's camera images. It's just images. It's simpler to think about. Talk about petabytes of data. These petabytes of images come in and prima facie, you have no idea what's going on in these images. Right.

Right. And so what you need is you need to figure out where are the people, where are the cars, where are the stop signs, where are the cats, etc. And you'd have figured that out for every single one of these images so that you can train a machine learning model on top of that. Got it. So what we do is we build this pipeline where most of the work is done by. machines on our end. We have classical computer vision algorithms. So it's like you do a first scrub of the data. Exactly. So somebody like Waymo could say, hey, here are, here's 10 million miles of driving, have at it. You say, okay, here's what we think. These are all the minivans. These are all

the pickup trucks. These are all the cats. These are the bouncing balls, etc. Yep. And then we also do, we also have a large team of smart, well-trained humans who can basically go through and spot errors at these that are made. And then the...

That's a second level of scrubbing, which is humans looking at things that computers have a low degree of certainty of? Yes. So if the computer is 99% certain, you just go with the computer? Well, we have more sophisticated filters

than that, but more or less, yeah. And then basically this highly accurate data goes back to the customer, and they feel great about it. They retrain their machine learning models. It's a wonderful cycle. So they don't have to worry about building a team or to do this basic level.

It's almost like you're just really good at getting that data set scrubbed and clean for them and normalizing it in some way so that they can work on the higher level stuff, like what to do with the minivan or what to do with a minivan turned on that side. Oh, it rolled over something's going wrong here, right? That's exactly right. And the way that we think about this in general is we're really providing this sort of this infrastructure layer for machine learning globally or AI globally, where in AI in general, there's like there's one big problem, which is getting all the data. And there's another big problem, which is like, how do you improve, how do you build these models and how do you improve the models?

Yeah. And we're trying to take the first problem off of people's plates. And we get back from this break, I want to know when you believe, based on your seat, which is very close to the data. in fact, you're sitting on top of the data. You're soaking in it. I want to know when you, Alexander Wang, with no E, eight-letter characters in that first name, I want to know when Alexander Wang thinks, we will not need a steering wheel on cars in San Francisco, driving from Palo Alto to San Francisco.

When do you think that'll be legal without a steering wheel when we get back on this weekend startups? All right, listen, there's 600 million people on LinkedIn, including me and you and the person sitting next to you and the three people you just email. And you have to hire people. But where are all those potential candidates? Well, they probably have a job right now. And so you've got to get in front of them because they're passive job searchers.

If they see something interesting, they might just click it. Well, how do you do that? Well, here's how you do it. Watch my associate press. He's going to go on LinkedIn Talent Solutions and he's going to post a new job for our client success manager in our Toronto office.

He quickly selects the skills that are needed, writes a quick description, adds additional screening, questions, which I love. And then he sets a deli budget that's, you know, reasonable. And he's on his way to find him the perfect candidate, whether they're looking for a job or not. LinkedIn is going to get you in front of hundreds of millions of job seekers who are not actively seeking a job. So with LinkedIn jobs, you can pay what you want and the first $50 is on them. What? Yes, that's right. LinkedIn Talent Solutions is going to give a 50-0-0 for you. Right now, all you, all you,

have to do is go to LinkedIn.com slash twist. LinkedIn.com slash twist. I don't know how long this will last for, but it's $50 for you right now for free from our friends at LinkedIn. And by the way, a hire is made every eight seconds on LinkedIn. And you know that that's true because how many people do you know who found a job or a candidate on LinkedIn? It is the central repository of talent. And now with LinkedIn talent solutions, you can leverage that massive network. Okay, let's get back to this amazing episode. All right. Alexander Wang is here. You heard what they do. And how many people you got working over there now?

I think now the team is about 150 folks. Mostly here in San Francisco, Bay Area? Yep. Okay. Hey, when I left our hero, that's you, Alexander. I was wondering, when do you think we'll have a self-driving car in a major city like San Francisco driving a major route, let's say from Palo Alto onto the highway, off the highway, and, you know, into, you know, Kolkari, the great Greek restaurant here?

Have you been? I've not. Kokari, you can write that down. Kolkari is the best Greek restaurant. Some people will consider the best restaurant in San Francisco. Get the octopus and the Saginaaki. So you get him there. You leave your house in Palo Alto.

You get out, you order the Saginaaki. What year would this be possible? 2030 and over or under 2030? When do you think we'd first see this? Ballpark. Would you pick under 2030 or above 2030? So this is over under. This is the million dollar question, which is when or...

It's actually a trillion dollar question. A trillion dollar question. Let's be real. But where we're at, like fundamentally, the technology is getting better and better every year. How much better on a percentage? Do we double, triple? Well, it's hard based on what metric you're measuring. But the algorithms that perceive their environments are getting a lot better, like asymptotrically better every year.

And then the algorithms that figure out what the car is supposed to do, the planning hours and etc are also getting better so it really is it's sort of only a matter of time before we get to the point where this these kinds of routes are possible and we'll live in a safer world got it so that seems to me that you're thinking less than 10 years from now this will be happening with regulation all that counted in yeah i think um i don't think regulation is necessarily going to be that that tricky why uh well i think there um there is press for, like if you think of when autopilots first came about, I think there's precedent for how to think about a lot of these things. You mean autopilot in the airplane sense or in the Elon sense? In the airplane sense, yeah. Got it. Sorry. Yeah. So the FAA and whatever regulatory bodies were like, okay, we get it. Autopilot works better than a human. Yeah, exactly. It's pretty obvious. Yeah. Plenty of room up there to operate when you're up at 30,000 feet. A lot less room to operate when you're going through the tenderloin. There's six people in the middle of the street, though. The challenges are a bit. different. But I think there's precedent for how to think about these things. I think the technology,

once it gets good enough, will be clearly extremely good. And so I don't think it'll be that big of a deal. So you think 10 years may be less? Yeah. I'm excited for the self-driving future. Got it. Wow. Look at that. You won't make a commitment. I like it. Why is that? You want to be neutral because of all your partners, your customers, or all the driving? You don't want to speak on their behalf or something or you just don't like the idea of gambling well uh i was i didn't put any money on this but i'll put the kokari lunch on it if you want i've i've i've no problems with gambling uh i'm a risk risk seeking guy what do you like you blackjack guy you're poker guy what do you i play poker yeah well really i would plo or hold them a little bit a little bit a little bit of both so you play a

a mixed game um yeah i would i would love to do you have a regular game uh there are some games i play at in san francisco yeah yeah yeah i i you probably have you probably have bigger games You want to play the bigger game. I don't have to play the bigger games. You may watch. You may watch. Yeah. One day, one day.

Well, stop getting there. Post-IPO, I'll hit you up. Well, I mean, there's always secondary shares, and, you know, common shares play. That's why I always tell founders. Common shares play. I would settle up a 50K balance with some common shares in scale.com. So let me ask you about the race between China and America for this. China is really putting a lot of effort into this.

Yeah. And do you have any customers in China currently? We don't have any customers in China. We work with some U.S. arms of Chinese companies, so if I do, for example. Got it. Bidu's got a self-driving. They have a U.S. arm that works on self-driving and other machine learning efforts. What is the state of affairs in China?

Because they seem to be doing a Manhattan project. Back to your parents at Los Alamos. They're kind of doing a Manhattan project. Does that mean they're going to get a lot further? than us, you think? I think it's definitely true that China is innovating very much in machine learning and AI. And a lot of it is, it's very clear that it's a very concerted effort on the government's part, on a lot of these large tech companies in China's part. Like there's a lot of investment where I think if you were to look five years back, it would be, it would be kind of shocking at how

much progress they've made today. Yeah. So it's very, something is definitely happening. And I think to be clear, I think machine learning, a lot of the machine learning is definitely very good in the U.S. and for most things is much better. But there are certain, they're certainly making a lot of progress. And there's a lot of data sets that they have access to that we don't necessarily have access to.

And so like CCTV cameras, they have cameras everywhere. they can find somebody already on facial recognition. It was a 60 minutes piece. I guess they found people in like less than five minutes. Yeah. Pretty scary. Yeah. Yeah, it's very, um, uh, oh, look at that.

Sayan, yeah. Oh, Kai Fu. I know Kai Fu. He likes to play Parker too. Yeah. He does, actually. Um, but yeah, it's basically, it's very, um, there's a lot of concerted effort. There's a lot less, um, there's a lot fewer questions as this, as this technology gets

productionized in China versus, the U.S. I think we're... You mean moral, ethical, regulatory issues? They're going to just go faster there. As Sian, the angel investor, points out on Twitter, Kai Fuli said that China would get further because they don't have the same issues around human casualties. That's interesting.

Yeah, they're more accepting of risks, et cetera. And I think, I mean, I do think move, fast, break things lets you move faster, right? So I think there is, there is a real, there in the two approaches. I've been pretty excited that in America, we haven't had a panic over self-driving. So when the horrible Uber accident in Arizona happened where the driver, the safety driver, wasn't paying attention. And I believe they were playing Candy Crush or on their SMS.

Did you see the actual video of them in the? Yeah, they were watching a video or something. They were on their phone. Yeah, they were looking down at a video. Yes. They knew they were on camera, and they were told they're being videotaped the whole time, they still couldn't keep their eyes on the road. And they killed a poor homeless person who was walking across the street in the middle of a dark road.

Yeah, I believe it was a man with a bike, but yeah. Oh, it was a man with a bike. Yeah. Yeah, I don't know why somebody said it was a homeless person at some point. But they, yeah, it's interesting that somebody said that. It was almost like they're saying the person, by the nature of being homeless, they were doing something bad. But they were doing something that they were crossing like an eight lane highway in the middle of the road where they absolutely had no business being, which is what the algorithm's job is. So how does an algorithm take that into account somebody who is blatantly going against all conception of what is normal, like walking across a six or eight lane boulevard Avenue freeway?

Well, I think in this case, this is actually a great example of two things. First, where LIDAR is great. And then second, where... Didn't they have LIDAR on that or no? They have... So the thing is, the models actually detected that person. So in this case, the sort of machine learning and the LIDAR, by the way, the LIDAR also detected the person. It was sort of, it was doing its job.

It was more the sort of like higher level processing that sort of... Ah. That made the unfortunate decision. To not break. Yeah. And to break at whatever, 45 or 60 miles an hour on one of those freeways, that comes with consequences too because there are people behind you. Yes, definitely. How is the computer supposed to make that decision?

Let's put aside this issue where obviously the decision tree didn't make the right decision on the data which was clearly presented. So it wasn't a sensor issue. It wasn't a processing of the data issue. It was a decision issue. it decided not to slam on the brakes. Is that right? I believe if you read the Nitzel report, that is what's happened. Yeah.

So that's a developer who wrote code didn't write the code properly. Well, it's a complex code system that ultimately made the decision it did. Do we even know what's going on with this machine learning? Because my understanding is a lot of times we put a bunch of data in. The answer comes out. And you ask the person who set the whole system up, they don't know how the answer was come to, just that it came to the answer. Well, I do think, so you bring up an interesting topic, which is about explainability.

Like, how do you actually know what these algorithms are doing? And this, again, I don't want to sound like a broken record, but it does come down to the data. And when you actually dig in, usually when the algorithm makes a weird decision, usually you can trace that back to something weird in the data that it was. Do you have any examples of that without mentioning specific customers, but in your test, let's say or in your laboratory or even in the real world without mentioning who, what, when, where. Do you have an example of the data was confusing and the output was then, you know, poor?

Do you bear the responsibility of that? Do we do we don't, we do bear a quality responsibility with our customers. In fact, we sign up for the quality of data we give for our customers. they have the ability to look through all the data, look like audit, et cetera. That's kind of heavy, isn't it? It's a lot of responsibility. Well, it's why I think, I mean,

ultimately, if you think about it, if you really believe in machine learning, a lot of, a lot of the things that you need to do machine learning, there needs to be stable infrastructure just like running water, right? So if you think about, for example, AWS has a pretty tough job.

They have to say that all these machines that they have up and running are going to be up 99.9 or 99.9. 5-9s, yeah. Yeah, yeah, like a crazy high percentage of the time. And that they, like, all your queries will take less than X amount of time, et cetera. So, but that infrastructure lets all these people build on top of it and build these great things, et cetera. So I do think as a general rule, as an infrastructure provider, you are, you need to give infrastructure that's very reliable. that people can depend on. And you know, it's really interesting.

They do such a good job, AWS, that when AWS does go down, which seems to be like some portion of it goes down, you know, northeast, whatever, sometimes it's regional or some section of it goes down, it's almost like people have a funny joking, like it's a snow day response to it for three or four hours. As opposed to five or ten years ago when this would happen,

people would get really bent out of shape that Twitter went down for two days or a day or I don't know what the longest Twitter outage was, but it would be for hours. It could be a half a day of the farewell. Yeah. And then before that, it was like reason to like not trust the internet. So we went from like not trusting it to

being extremely frustrated to now it's kind of like a joke. Like oh, it's down. We know it's coming back. Yeah. It's no big deal. They're going to reboot the servers and figure it out. We've really gotten used to it because it's such a rarity. Yeah. The same thing is going to happen to machine learning and AI over

time. Like right now, you framed it very well. Right now we're in this, this period of, of high distrust of these systems, right? We see them do weird things and we don't really know what's going on and we feel like it just feels foreign and weird. And then eventually everybody will learn more about the technology, learn more about what it's good and bad at. We'll get to a point where when it does something unexpected or does something bad, we'll just get really annoyed, very frustrated because it really impacts people's lives or it impacts people's businesses, et cetera. And then, and then the long term, I think, I think the systems will be extremely reliable. Certainly much more reliable than any human could

ever be. Yeah. I mean, like we work, we interact with lots of machine learn systems already. Google search is a large federated machine learning system today. It's very, very influenced by core deep learning, machine learning, et cetera. And it is extremely reliable. It works like, running water, it's great. Yeah, when's the last time you're like, I couldn't find that answer. Like between Cora, YouTube videos, and Google acting as this glue and fabric between it, like surfacing stuff in the one box, you know, the little box that gives you like a mini scraped answer.

Yeah. But they're not allowed to scrape Cora. Cora doesn't let them index it, right? I think they're still in the standoff. I actually, I don't know. Sometimes I see Cora. Well, maybe this is old. I think you see the link, but they won't let them put it in the one box where they kind

have exposed the answer, they kind of make you click through and they make you log in. This is why Quora is kind of brilliant. They won't let Google take their data. Quora is a is a very brilliant company. DeAngelo? Adam DeAngelo? Adam DeAngelo. You know, I've got to get him on the pod.

Let's make a note. It's been like five years I've been trying to get him. He don't like to talk. He's an introvert, I think. He is, well, he's on Twitter. So tweet him. Yeah, I'll tweet Adam. What would you like to work with?

He was great. very, very smart, very thoughtful, very long-term focused. I think he put 30 million of his own Facebook money into it, right? That's what I heard. He was the first CTO of Facebook. So I think he put 30 million. This is what I heard. Put 30 million of his own money in like the Series B or something and then like let somebody

else put in like 10 or 20. Like talk about skin in the game. Yeah, I have no idea. I was bonkers. I was like, wow, that's a first. But he's thinking 20, 30 years out. Like this is his first and last startup. I think, yeah, he thinks extremely long term.

which I think means that it's a competitive advantage in today's ecosystem where most people in general are very short-sighted. I think they'll look at, oh, where's the quick win or where's like the new hot thing? Right. They don't have any revenue, right? They don't have ads. They do have ads, actually. They put ads on? Yeah.

Oh, I didn't know there's ads on there now. They're so subtle that you can't even notice them. Whoa, I've got to go check those out. does it matter that we don't know how sometimes ML gives an answer when we go look at the algorithm, why did this come up first? We say, well, there's a number of factors, but we don't actually know. Does it actually matter? I think non-explainable systems are already out in the wild.

What's an example of one in the world? Well, Google, I guess. Yeah, before there's machine learning Google or your Facebook feed. or your so Facebook can't explain why a specific post went to the top well they they have they have some information right right and they can give you a little bit they can give you some information but most like this is a problem with code more than anything most code systems are so big and difficult to explain that that it's it's already a big problem so the cost of doing a google search and getting a non-perfect answer it makes it let's say a

makes a huge mistake is very low, right? You just do the search again, or you change the search a little bit, or you pick the second answer. No problem. If Facebook puts something at the top of your feed that's not the most relevant and the second and third one are the most relevant, again, no problem. However, if you do it with a self-driving car and it makes a decision, it could be somebody's life, and if you do it with some kind of system involved in justice, like talking about using

ML for justice. Should a computer be able to give an answer that somebody is guilty without being able to explain it? So when we get back from this break, I want to know, would you trust in the next 10 years ML and AI to make decisions? Because you obviously are fine with them making decisions about driving and people's life and death there. Would you make it work for the justice system versus the justice system in America, which has been proven to be biased against non-white people when we get back on this weekend start. Listen, you're running a small business, you're running a startup, you need money, and it shouldn't take all your time to get money to run your business. The modern way to do that, the simplest way to do that is Cabbage.

They allow you to access up to $250,000 in credit to run your business. Cabbage's application process is online, and it takes just minutes to complete and get a decision. If you qualify, you can access the amount you need right away and withdraw more funds whenever you extra capital. Cabbage has an A-plus rating with the Better Business Bureau and has already provided over 200,000 small businesses with access to funding. Portfolio companies have used it to cover employees over the holidays when a large client missed an invoice. That was an amazing story I heard. So I want you to get the money you need to run your small business today. Go to cabbage.com and use the code twist to get a $100 credit on your first loan statement. That's KABB,

A-G-E. K-A-B-A-G-E.com and use the promo code twist. This is an important disclaimer. You must take a minimum $5,000 loan to qualify. Credit lines are subject to review and change, and this offer ends November 30th. Two days after my birthday, individual requests for capital are separate installment loans issued by Celtic Bank member FDIC. All right, let's get back to this amazing episode. All right, Alexander, we're here coming around the third ad break, which means I'm going to ask you like the really hard questions. I got you warmed up. You're comfortable.

Maybe let your guard down a little bit. PR guys like checking his Slack room. He don't care anymore. He's out of the woods. So let's get into the tough work. I'm joking. This is a PR person in the room was probably freaked out by this question, but or not. I explained that the list of your Twitter and Facebook feeds or Google,

and that we would all agree that doesn't matter. We can argue Facebook maybe matters if it's pushing up stuff. That is fake news. And they're getting some heat about that. But again, nobody's dying, hopefully. But now we get to self-driving cars. Should you have to prove how these things made a decision as opposed to having this non-ability to understand how the decision was made?

Or does it matter if it gets the right answer 10 times better than a human in your mind? So in the situation where your self-driving software or one of your customers is 10 times, 10 times better than a human. It's proven. Should they be able to explain the one in 10 chance they have of an accident compared to a human, should explainability be required? Yes or no? It's a, this is a really important question. Again, that's why I'm asking you, the founder of scale.com, who's raised 100 million to empower all this. This is exactly right. So I think as more and more systems are governed by machine learning, there's, it's very natural to. ask like, okay, if we're trusting our lives to these systems. Yeah, we are. How are we supposed to feel good about that, right?

Am I supposed to just live with the like one in whatever chance that one of these systems will just poop out and then my life will be at risk? And so I do think that ultimately further explainability and a deep understanding of the performance of these machine learning systems is going to be needed. Yeah. Now, that being said, like, again, there are plenty of mission-critical software systems in today's world that we depend on, like, for example, whatever systems that we use to control the power grid or whatever systems we use to control the national missile system, et cetera, those are all software systems that sometimes they will just crap out. Yep. Like every once in a while, your database will go down or every once in a while, since we'll go down.

And whether or not you can explain those phenomena, that tail probability that happens still causes real risk. So in a way, you're saying you're being held to a higher standard. Why do you think you're being held to a higher standard in the systems that came before? Well, yeah, I have a theory, but I'm interesting yours. What I'm saying is there's that we live in a world where randomness is a reality, right? So accept it. So there's as these systems end up launching and end up being more and more important, I think it's important that we realize that the, there is, there are always tail probabilities that bad stuff happens. Now, that being said, I do think it is, it is the responsibility of people who who operate these systems.

Yeah. Yeah. responsibility of the people who make these systems to have a have an understanding of the performance of these systems and also ensure that they are doing everything they can to make sure that these systems are performing as well as they can, which ultimately comes down to, okay, for if I'm training this model and I'm training on so-and-so data set, how do I make sure that the data is unbiased? For example, we talk about the facial recognition of these cases. How do I make sure that it is properly representative?

How do I make sure that there's no weird artifacts in the data that would cause something bad in the model? Yeah. And then how do I trace back these issues, right? Yeah. And I think very much so where the whole machine learning community is understanding these issues and building them in. But it's more about how do you build these systems that are robust, built on large data sets that are very diverse. So the larger and more diverse the data set, the less biased there should be in it. Exactly.

Except if you said, okay, we got a couple of states, let's make a justice system based on it. And then all of a sudden you're like, okay, let's do the whole United States. And then you find out, gosh, the whole United States justice system is biased. And if we were to build on that data set, people with African American names, Latino names, would be convicted more often because we actually use the data set that, had bias in it. But the people, what you're saying is the people who are in this are acutely aware of it and they are good actors who want to get it right or why else would they choose this as their profession?

They would, there's nobody on your team or any ML team you've ever met who said, you know what? Let's put systematic bias into the system as opposed to getting the right answer. Putting bias into the system would mean your business is going to go out of business because you made a poor system. So in fact, machine learning and the people working in AI are so acutely aware of this, their intentionality would be to remove bias and make the world better. So it's, again, this technological phobia, technophobia, and we're holding technologists

and machine learning to a higher standard. I think in part because we as humans are so scared of being replaced that we're going to hold that which replaces us to a standard that is one that we would never hold a human to. Yeah. And by the way, I think this is a, this is a played, there's this played out narrative, right, that AI is this magical thing that will come in and just replace humans at all of these very important tasks, right? And, and that's, I think that's, that's the dominant belief that a lot of people hold.

But, but the reality in the actual nuts and bolt scenarios, it's, it's pretty far from that, right? We have a long way to go before machine learning will fully replace any kind of any jobs, right? Yeah. And there is precedent for this, by the way. Like when the ATMs were originally invented and built and launched, you would sort of believe, one belief you might have is that the number, as the number, as these ATMs were built, the number of bank teller jobs would start dropping pretty considerably. Yeah. But what actually happened is that the number of bank tellers in the United States actually grew pretty considerably. And there's a number of like sort of economic reasons you could think for why this bank started to bank. Yeah. So one is that the ATMs allowed for this huge growth in the banking industry, which means that there's a lot more opportunity. Another is that the bank. You know, all those people who wouldn't get a bank account because they didn't have access to their money. It kept it under their couch. We're like, oh, I can get money anytime. Okay, my number one fear is gone. Exactly. I'll put my money in the bank because I don't have to. go there between 8 a.m. and 3 p.m. Yeah. So that's one. Another is that the bank tellers now can focus on higher value tasks.

For example, I don't know, like mortgages, lines of credit, starting bank accounts, etc. Credit cards, which means that the per, the value of a bank teller goes up, which means it's more valuable to invest in more bank tellers. And so these effects are like, the second and third order effects usually mean that there's, there's way more opportunity and way more growth. Of course. As these sort of like the automation slowly seeps in. Yeah. And when you think about it, we spent all this time creating these phone routing systems.

Remember that? Where you're like, press one to go here, press two to go here. And we did that for what? 30 years. Like everybody was like, oh, just sent it to the phone jail system. They used to call a voice jail instead of voicemail. And we invested for 30 years in voicemail systems. Then at some point we realized, wait a second.

everybody gets to people over messaging or whatever somebody does pick up the phone call they probably have a very acute important problem and that's a chance for us to prove how great our brand is and to get to know our customer better and then beat them with our other customers let's bring back people who pick up the phone and talk to you as a delightful VIP customer type experience and now you have people adding back and they just call it customer success and people look at customer success not as a as a as a cost center anymore. It used to be a cost center. How did we reduce the number of calls? Now people look at customer success as like an investment in them renewing. So the SaaS people are like, yeah, if we get people to call in when they have a problem, maybe they won't churn. Maybe they'll use the product more. Maybe we'll up some. So sometimes we bring the jobs back. We got rid of phone operators and receptionists. And now we're bringing them back. We just call them customer success. Yeah, exactly. These trends are, um, uh, happen over and over. I mean, I think there's like, uh, there are there like helping people focus on higher and higher value work is is real i mean

that's sort of like the core of human progress in some sense but um but it's uh that that very much so is i i strongly believe will be the actual story of a i machine learning and it'll have to happen more and more and more for us to be comfortable with it but a great example is like truck driving so there's there's all these automated truck uh truck driving companies yeah Yeah, lots. We work with a lot of them and bark, Ike, et cetera. And there's sort of the naive view is that, hey, they're just, they're going to automate truck drivers.

And like if you look at the map of the states, like truck driving is a top profession in a lot of states. Sure. So it seems really bad. But actually, if you look, if you kind of like think at the system as a whole, there's a shortage. There's a national shortage of truck drivers. In truck drivers in the United States. And the median age is like 50 or something crazy. Yeah, exactly.

So there's sort of this like, there's this like, millennials and Jan Zs are not becoming truck drivers. So there's this, there's this kind of like instability in the market because of all this stuff, right? And, and, uh, and the, the, the automated truck driving systems, actually what they would do is automate the, the long haul middles of these truck driving trips. The boring parts, which are the boring parts, arduous. That displace people from wherever their homes are, et cetera. Yeah. And allow the current truck drivers to focus on these like higher value trips that are sort of. of like warehouse to a meeting point or whatnot.

Yeah, drayage to, like the drayage to the factory or even the last mile. I mean, who knows? Like, maybe these trucks will change their form factor and be half the size, be automated. And when the truck gets off the road, the same truck. Instead of using 18 wheelers, we might just use smaller mid-sized trucks that'll be electric and solar powered. So you have more of them. When they get off, they become the delivery truck.

Yeah. And they just automatically start delivering. Yeah. It could be a much better model. Yeah. So, so the, um, these, uh, these sort of like the introduction of machine learning to, uh, to improve the efficiency of the economy. It'll, it'll be slow because of how the, how in general free market economics work, it'll, it'll take effect in areas where there's an acute problem today. Right. Right. It'll happen in those places first. And, uh, and it'll allow the, the current jobs that exist to become higher value, more impactful, um, et cetera. So. So the sort of what we believe, the true narrative will be extremely positive, actually, versus the current narrative, which is like AI and H-E-I, et cetera, are going to take over the world. Yeah, it's silly.

I mean, there is a possibility that AI could get out of control at a certain point with exponential computing. That's not far-fetched that it could do something crazy and stupid. You only think that's not far-fetched because you watched a lot of these sci-fi movies. Well, I mean, listen, if you were to train an AI to work on a drug to kill cancer and you didn't program it properly, it could create a drug that was too aggressive because you didn't tell it, well, in the process of killing cancer, please don't make the person blind, right, or all these other things. So you could just forget some edge case and some general AI might think, if you said to the general, AI. You should work on things that make the human species better and goes, okay, yeah, let's kill cancer. And it's like, oh, yeah. Or let's cure this communicable disease. Great. The best way to cure communicable disease is to kill everybody who have the disease currency, so it can't be communicated.

Like, this sounds far-fetched, but there will be instances where they will make the wrong decision, right? Or it will be just too slow of a ramp up for it for us not to catch it in your mind? Yeah, I mean, I think there's like, yeah, there's this, the thought experience always go like, you'll make an errand comment to an AI, and all of a sudden it'll take over the world and do something that you really don't want it to do. But, I mean, in reality, like, there's a lot of oversight over these machine learning systems right now. Like, there's, there's like tens, hundreds of people who look at these models. They look at all the data that comes in and out, and they look, they like analyze everything. And they try to figure out, okay, what is this model doing well?

Was it doing poorly? And how do we adjust to that, et cetera? So I think, I think that could happen in. a world where it's like we believe we we we have low oversight of these systems so oversight is always important in any new technology right it's like when we started having uh airplane autopilot for example it would be crazy to just say okay we have airplane autopilot just let it put any oversights over the facebook and social media companies they they have to be clear they they do they didn't have oversight what do you think the fcc like giving a fine in the review mirrors oversight it's not

oversight they had no oversight and we lost our democracy over it but The Russians came in and spent a ruble doing it. That is one take. They manipulated. They stole the Cambridge Analytica data. They did voter rolls. They tried, whether it actually caused the election to swing, we'll never know, perhaps. But they definitely were able to swing some portion of it.

They definitely were able to manipulate it successfully. So. And what regulation is there of AI right now? There's none. You're acting under zero regulatory environment right now. And China's got a negative. regulatory environment. It's true that, it's true that like...

So you should be regulated to your admission? No, no, no, no. That's not what I'm saying. Well, wait, wait. You just said that you should be regulated so that we don't have problems, so which isn't? I do think that there are a lot of important issues about how we deem what AI systems are appropriate, how we look at what they're supposed to be doing, et cetera. I do think governing bodies, the U.S. government in particular, for example, has to take a deep look, understand the technology, determine what is reasonable, what is not reasonable,

et cetera. And ultimately, they're the... But even in their case, they're looking at the miles driven and the accidents, but they're not looking at the code that you guys are writing. They're not looking at anybody's code. They're not looking at the AI systems. They don't even have anybody on staff who could even write an algorithm, right? Well, that's also changing to be clear. Is it? So the, in general... I think they're looking at any lines of code in any of these systems? I'm not sure about the answer to that, but I do think they look at a large amount of data. So, for example, these...

Okay, they do, yeah. So in Europe, for example, there are all these... There are these ADAS systems, right? So there are these driver assistance programs or driver assistance systems and a lot of, like, high-end vehicles that you buy today, right? Keep you in the lane. Yeah, exactly. Keep you in the lane.

If you have, like, stopping on traffic, you don't need to do anything. Yeah, adaptive, cruise control. Yeah, yeah. And lane change warning. Yep. Standard on BMWs, BMWs at Audi's these days. Yeah, exactly. So there's all these, these systems exist.

People buy these systems. People rely on these systems. And in the EU, for example, where a lot of these car makers are, where BMW, Audi, VW, et cetera, are, they have a responsibility to actually both have a large data set that they have collected themselves that is able to validate that these systems are performing well, as well as past. a series of sort of trials and actual... Oh, really?

Yeah, yeah. Different forms of data that the government, that governing bodies place in front of them. Well, that would be very interesting. Now, I think about it, we do crash tests for cars. You're required to give three cars or something to the government for them to just destroy in their crash tests. Yeah. But we don't require those cars to go into a lab, get taken over by the governing body,

and force them to go into real-world testing environments because there's some real-world testing. environment where you do self-driving up north here, I think, some military base. Do you I'm referring to there's a military base that everybody uses for self-driving? It's like a town that was converted into a like a self-driving town. I forgot the name of it. Yeah, a lot of these it's like a self-driving town. A lot of these companies, they buy cheap real estate. They outfit them into like these mini-town so they can create these funny scenarios. Have you ever been to one of those? I've never been, but I've definitely seen the video from there. I've seen the videos, yeah. Yeah, it's pretty cool.

They have like children come darting out, like little cardboard cutouts of children to see if it hits it. This way they can do that in private. Yeah. But it's interesting that at some point the government's going to have to have people who are developers and coders actually getting into the data and understanding some portion of this, right? At the very least, they'll have to create like the driver's test, for example, the driver's license test for a self-driving car. I mean, that will exist. You believe in general AI that will hit that at some point? AI that is just generally smart can do anything a human can?

I mean, I believe in some sense. I believe in the sense that, like, for most technological things that humans can conceive of, that aren't physically impossible, if humans survive, they'll happen at some point. Like, I think humans are, like, infinitely creative, infinitely ingenious, et cetera. Sure. I think it's very overblown the timelines, people are talking about general AI happening. I think it is,

I can rant about this for a while. There's a lot of things that are wrong about like the common arguments. One of which is people say that like if Moore's Law keeps going, then we'll have all this exponential compute and that's going to like, that's going to be. Unlock it. Yeah, it's going to mean it's only a matter of time before we produce these general AI. Not to mention quantum computing.

Yeah. That gets involved. So, yeah. I mean, Morris Law is going to is going to be dead. Yeah, yeah, exactly. it's going to be dead. And then quantum computing is very far away despite recent press releases, et cetera. And so I think that I think that leg of the argument is not actually that strong. And then I also think it's not even clear that if you have infinite compute, that you'll be able to produce general AI. I think that's very unclear. So infinite compute helps narrow AI because you're doing a number of scenarios and, you know, playing out every scenario and go the game, the stone-based game. Many more permutations. than poker, many more permutations than chess, which is a finite data set.

So yeah, more compute power on those things certainly get you quicker ability, or even just throwing people into a random video game like Open AI is, sure. Definitely. But generally, I taking somebody who've mastered chess and then saying Master Go and then Master, you know, Fortnite or Master Impressionist painting, it's different.

It's very different. It's very, so the argument, one of the arguments goes that once you have enough compute you can basically simulate you can create artificial life by basically simulating evolution

so that's that's one of the more vogue arguments okay let me just see if I understand that you have so much compute power that you can say start with this tiny whatever it is you know piece of bacteria whatever then grow it and grow

and grow an entire evolutionary system to the point at which there is a human-like species and then grow that human-like species in whatever number of scenarios with a big brain into whatever comes after us. Yeah, or even, I mean, even if you just grow the human, like the human-like species that's as intelligent as, then that's kind of good. That would be general AI as read as well, right? Because generally I would normally, most people would define general AI as not even being smarter than us, but being as smart.

Yeah, exactly. Huh. So, so that's an interesting approach. But somebody would have to code that and program that and build the systems to do that. It's not just going to magically happen, right? Yeah, exactly. It's very unclear if that's even possible. But that's an argument.

I honestly think that's the most plausible argument. But it's so it's very much so science fiction in the sense that it is, we're not close to being able to even validate the hypothesis. So I think I, yeah, I don't believe in general AI anytime soon, despite what the pundits will say. Yeah. What's the next big mind-blowing AI project, narrow AI project, let's say, that most people aren't considering right now after self-driving, which is the one that's captured our attention? Well, I think there's, I think there's a bunch of really boring ones. Okay. So the boring ones are like, hey, can you automate form processing really well?

Like paper form processing. Oh my God, that is boring. It's super boring, but it'll be big. Like what? Like I have to fill out a form to get my driver's license and you'll use AI or? Yeah. Well, anyway, we'll move on. But there's a lot of boring examples.

There's a replying to email. That's kind of the dope one in Gmail now. Have you seen that? Yeah, it's great. It's pretty demented how fast it's getting good. And it's personalized, right? Yeah, I would believe it's personalized. I think it's personalized too because it's starting to use my lingo.

Yeah. So I'm like, I would never use that. And then I'm finding myself like, wait a second, it's finishing the sentence in my voice. And then you're like, wait a second, my voice is pretty narrow. I'm a human. Yeah. So giving you suggested replies is actually kind of low-hanging fruit. Yeah, it's pretty.

And that's like, I would say that's like kind of a boring one. Yeah. But I think there's ones, I think there's applications that have like pretty large-scale economic impact. Okay. So, for example, all this automated radiology, an automated like medical imaging work is very impactful. And the technology is like the core technology is good enough, given enough data to actually make that possible. So I get my lung scan because I was a smoker and they're doing lung cancer. now they send those x-rays to India to be reviewed by technicians there who go through it or even heart rate monitors for 24 hours because that's the cheapest labor with the highest ability.

But all that data can just be done by a computer better than a human could ever do it. Yeah, I mean, so for example, so in general, globally, there's a huge shortage of doctors. I believe like, it's ridiculous. I believe, yeah, the world health organization published something is like 10x shortage. in doctors globally. Right. So, so, so, like, there is, there's this massive shortage. And if you can, if you can fulfill some of this demand with automated systems that are

much more scalable, there's a, there's a huge amount of value. And I think it's, it's easy for us to think in the United States that, like, hey, it's not clear what the lift would be. This seems like it'll just automate jobs or whatnot in the U.S. Yeah. But that's because, like, you already have access to the stable infrastructure that, or not everybody. But a lot of people already have access to the stable infrastructure that is health care, right? Yeah, no, as bad as our health care system is here in America or flawed, probably be the

better word, it's not like somebody's not going to be taken to an emergency room, right? Like, and there's other places where they're just never going to have access to a doctor or maybe once a year. And getting an x-ray might be out of the question because of the cost, not just the of the cost of the x-ray, but the cost of actually interpreting the x-ray. So you think x-rays are the big one? Well, it's all. all the forms of medical imaging, right? Like x-rays, ultrasounds, cat scans. Are you working on that yet?

We do work with a bunch of this data. Yeah. And I think this is a... Computers can do it better now than humans, or they can queue it up for a human to review more efficiently? Yeah, a lot of times they do some of the work, and then the humans can do it more efficiently. So like first pass?

Yeah, yeah, yeah. Or even marking out problem zones in an image. Got it. So annotating it. the doctor can then start on second base. Yeah, exactly. So I think a lot of those systems will be very impactful. I think that there's a lot of other boring things that people don't think about.

And then I think that like I think that more and more a lot of the things that, I mean, really, the like sort of the market forces are the things that like there's either incredible demand for or the things that people don't like doing. are going to like are going to be the clearest things to work on. I think education is going to be a huge one, like adaptive learning, where kids can sit in front of a computer and it starts to learn from looking at their facial expressions when they're frustrated or on the cusp of being frustrated. I know this sounds like really like dystopian, but if the computer was watching the child and the child is frustrated at a certain math problem and then it takes them back 20% to an easier math problem and they can tell from the facial expression that they're enjoying it

and that they're feeling confident. And then when they feel not confident, they can push a little bit into that. Hey, I know you're uncomfortable. Let me walk you through this again. Or I get the sense that you might want me to work through this again with you. Can you imagine what Khan Academy with machine learning and adaptive learning and AI could do? Is anybody doing anything in a narrow AI project to teach people how to learn? I haven't heard of a project before, but imagine for literacy, there's still lots of people on the planet who can't read and write.

Yeah, no, I think it's a clear application. There are, I mean, it's funny we're talking about healthcare. There are separate problems where education systems are like are pretty broken and are not ripe places for a lot of economic opportunity. But like a system like you're talking about is really, I mean, It feels easy to make, right? It's only a matter of time. Yeah, yeah. I mean, I think...

It would be easier than self-driving or similar challenge? It's very easy. Easier than self-driving? The challenge of figuring out when someone is frustrated based on their facial, based on a photo of their face. Oh, it's done. It's very easy. That's done, yeah.

Yeah. But combining that with some adaptive learning technology. Well, then it's just about, it's about understanding what's a hard question, what's an easy question. Yeah, that should be super easy. Yeah. Nobody's put that glue together. Isn't it amazing that we don't find it?

I would bet you it probably exists somewhere. We got to find that out. If somebody on the pod is listening and there is an adaptive learning system using AI and facial recognition to kind of understand where the students at. See, that's where I think technology is really interesting when you combine two things. For every dystopian terrorizing thing about facial recognition you can think of, there's 20 you could think of that would actually be amazing. Like if you knew somebody who was walking on the Golden Gate Bridge and you knew. knew they were despondent and considering suicide, you could literally know that a person walking across the bridge was doing so with the potential of jumping off the bridge.

Yeah. I mean, there's a lot of, there's, there's boring machine learning that's really great, right, which is like Apple Watches, for example, or a lot of these like things, wearables. Apple Watches are set up with an algorithm that can, it basically looks at the accelerometer and how fast you're moving, et cetera. Yeah. And it can detect when you have a hard. fall. Yeah. So if you fall with an Apple Watch, it'll detect they have a hard fall. And if you

don't respond to it with some time period, it'll call an ambulance to you, actually. Incredible. Which is like, it's a super... Now that's science fiction. Yeah, it's crazy. It's actually really crazy, right? And that exists today. It exists today. It's amazing. Buy an Apple Watch. It could save your life, right? Yeah. But there's a lot of like boring uses of machine learning that, and this is what, this is like, this is why really, people should view it as like this this crazy incredible thing. I think a lot of people do,

but adding people to the fire, it enables all of these things that you couldn't have done before. And so it just, it enhances the sort of like capability set of the technology that we've built, pretty considerably. Where will all this be in 20 years?

When you're 42, what do you think the world's going to look like an AI machine learning? If you had to describe it, obviously your company, will be a publicly traded trillion dollar company putting that aside. You'll be the richest man on the planet. But putting that aside, what will the world in machine learning and AI look like?

We'll wake up in the morning and AI will interact with us how. It's a good question. I mean, I think it's almost, well, one thing is it's very hard to conceive of, right? Because it'll sort of be like big ideas aren't big ideas from day one. There's sort of like slow ideas that snowball and snowball and snowball. And then eventually it's sort of like this huge thing that everybody thinks. thinks has kind of changed the world. So it's hard to conceive of how these things happen today, but ultimately, I think the sort of the dream and system is sort of as a, as a, I mean, a lot of

people like this idea of the assistant, right, like a machine learning assistant. But ideally, it's, it's some system that you can basically, you interface with it through, through voice or through typing or basically through language. Yeah. And you're able to dictate, like, questions or things that you want done in the world, et cetera. And it's able to understand that reason through it and then understand what the result is. And then a lot of the AI that we're building today, a lot of the machine learning

we're building today, which a lot of it is sort of core perception technology, like just understanding what's happening in general, like knowing that there's a car there or knowing that there's a sofa there or whatnot. A lot of that will sort of be a base layer of intelligence that powers the next layer. And then there will be a base layer of reasoning or whatnot that powers the next layer and so on. Yeah. It's going to be very interesting. You believe in this like brain interface stuff, neuralink?

That is going to work? Well, I don't have any, I don't have any deep inside on it. I know one of the neuralink founders, I think they're working on cool stuff. I don't know all these things the question is like what's the killer app right?

What are you actually going to want to use that thing for and what's feasible and where it's like the intersection of those things? You're ordering food without anything

you just think what you want and a burger shows up so Uber eats plus Neurrelink means like you and I'd be looking at each other and I'd be like cheeseburger and you'd be like

cheeseburger and I'd be like bacon and blue cheese and you'd be like cheddar and turkey bacon and then in 15 minutes would show up But is that that much better than Uber eats?

Yeah, it'd be incredible because you would literally have to not spend the 60 seconds to think about ordering it or pressing the button. Of course it's not. It's not much different, but it's going to be kind of mind-blowing. It is kind of mind-blowing today that you can just take out your phone and order with like three clicks and get food. And it used to be like, I don't know, five minutes on the phone, making four phone calls, seeing who's open. Yeah, I mean, this was the big, by the way, this was like the whole thing with chatbots, right? It was that when chatbots, when there's a chatbot craze, everybody thought like, oh, this is great. It's so much easier.

But then if you think about the actual number of clicks that you have to make, if you click like 60 times to get something done with the chatbot versus like three or four times to get like your hamburger from McDonald's or whatnot. What was the world like before the internet?

You don't know. Well, I can read books. You don't remember a time before the internet. When you were seven years old, it would have been 2005. you would have been on a broadband connection at home. Did you ever use a dial-up modem?

I did use a dial-up modem. Really? The dial-up tone. Yeah, really? I remember that, yeah. For like... Keep in mind, I'm from New Mexico, so it was... Yeah, not a lot of broadband out there.

A backcountry part of town. Really? Yeah. Well, I mean, part-back. Are your parents still there? My parents are still there. Are they still working there? My father's retired.

My mom's still working. Wow. Yeah. Some Breaking Bad County country, right? Yeah, I'm actually watching Breaking Bad for the first. time now. Oh, really? What season are you on? Season five. Yeah. It gets better and better. The first season's so slow. I skipped the first two seasons, but yeah. It gets crazy and I just watched the El Camino, like the, they made a movie that takes place after the end of Breaking Bad. And I actually

enjoyed it very much, but it's, it's definitely top 10. Have you watched the Sopranos yet? That ended before you were born, I think. I, uh, I've watched a good, good chunk of this. Here's what you do. This Christmas break. You're going to be off. Take a break. This is my advice to you as a 22-year-old. Actually take a break over the holidays. Binge watch The Sopranos. It lit, I mean, it was one of the first shows that had a lot of plot lines going at once. Not as many as Game of Thrones introduced or some of these really high-density ones.

But at some point, television writers realized if they made it more dense and harder to follow, you would actually get more out of it. And then because DVDs existed, people could go back and watch the previous seasons on Netflix or buy the DVDs. And it was in the best interest of the people making the TV shows to make them more dense, to create more characters, to create more plot lines, and make it more complex because it drove more DVD sales. Yeah.

Think about that as a system. The DVD was such a popular format and was so profitable that it impacted the art. Because before that, they said every episode needs to stand on its own. If you've never watched an episode of the Brady Bunch, this episode of the Brady Bunch, you don't need any prior knowledge. Yeah. So then you were like, it was almost like Groundhog Day. You're waking up every day.

And you don't even know there's ever been another episode of Gilligan's Island because they're all just singular pieces of work. Yeah. But The Sopranos was, and another TV show called The Shield, were the first to make them very intense, lots of characters, lots of themes, and then having story arcs that would go over seasons. Yeah. And so you'd have like multiple arcs over multiple seasons.

And of course the wire is considered the king of the show. written. I've only gotten five episodes into the wire and I stopped because my wife wants to watch with me. Okay, we'll end here with our favorite new game show. This is the game show where we pull up your tweets and we say like, retweet or block. You're saying it. Well, the audience is going to do it too. Here we go. We're going to pull up your first tweet. Now you're querying your mind going, What tweets have I done? Here's your first tweet. All right.

Many folks believe there's a smooth continuum between good people and great people. In my experience, there's a huge band gap between good and great. Banned you're referring to here as like a band, like as in a... Like an electron band. Yeah. Like, yeah, electrons, there's like these gaps where they won't be. They can't cross. Exactly.

Great people are clever, determined, fight hard, more loyal, and hardship and strategic. It's obvious when somebody's great. And the counter to this is, hey, it is obvious as well when they're just good. So the difference between good and great is exponential in your mind. I would believe that, yeah. To me, this is a like and a retweet. Woo! I like it.

You get both. All right, here we go. Now, here's one that people said might have been a little callous. No, I'm joking. How to build something insanely great. According to Alexander Wang, you can follow him. him. He's Alexander underscore Wang. No E. It's just A-N-D-R-U-S-Wang. Build something you care about. Number two, find users listen to them. Three, improve it every single day. Ah, this is the one people leave out.

Four, find people who inspire you and convince them to work with you. Ah, yes. Five, repeat two to four for years. Correct. Just five simple steps. Yeah, the devil's in the details there. It's really hard to improve every single day, isn't it? Super hard. Super hard. You get more people. It gets harder. Yeah, I mean, there's This is what somebody told me. So this is a secondhand, but they told them their Kung Fu teacher told them this or something. You're either, every day you're either getting better or you're getting worse.

There's no, there's no sideways. No staying the same. Yeah. And so your product's either getting better or getting worse. Right, because if you are not improving it, it's deprecating. And a competitor is improving. And your users are getting used to this average product. Okay, here we go.

Watching YC alumni demo days, if you thought. One, none of these companies are as good as mine. Number two. Oh, sorry. That's not what it says. I'm joking. Alex hit it's like, what have I got myself into? Okay, watch what I'm saying.

What many ideas derivative to the hot financing of today, DoorDash, Compass, Checker, hymns, et cetera. Two, lots of duplicate. 175 greater saturation point for startup ideas. In all likelihood, greater than one of the unhip ideas will be the big company. So what you're saying here is being a follower, it never becomes a big business. And maybe 175 is just too big. It's too many.

That's what, yeah, that's what I was saying. I think there's, there's, did YC. Did you go to YC? We did YC, yeah. Okay. So what do you think? When you went, how many people were there? And what do you think about this ginormous class size?

A hundred companies. Even then there was like, there was duplicates. There were people working on similar stuff. What do they do when there's people working on competitive ideas in the same classes? It's one thing for class to class. That creates enough tension. amongst the loyalty and oh now you've got

you're on the cap table of two competing companies but in the same class I think they're laissez fair about it I think they just they let it go they give a lot of love they give everybody love

and they're like go ahead yeah fight it out yeah yeah interesting what's the right number I don't know I think 100 is fine I mean again it's like

it's your black swan farming so you know takes as many as it takes Yeah, that's what people don't understand. See, systems like Y Combinator or just Silicon Valley in general seem broken to a layperson who's not part of the system because they look at and say there's too many failures here and too many derivative ideas and these people seem unqualified and this company got too much money. And then what they don't realize is that chaos that we talked about at the beginning of the show can lead to people being given permission to try something outlandish that then in. fact changes the world in a way that nobody could have determined, which is the definition of a black son, is that you could not have seen it coming until you've seen it.

Yeah. Because up until the point of the black swan, nobody ever believed there were anything other than white swans. Yeah, exactly. What a great book. Really good. Did you read Anti-Fragile? Yeah.

That's my favorite. Yeah. Developing companies and systems that do better in chaos. Yeah. Oh, what a tremendous idea when you think about it. The world's getting more chaotic and you're doing better. Trump. Captain Chaos.

Yeah. No, he's the, Nassim Taleb is really good. Oh, I thought you're going to say Trump. No, no, no. For your opinion on Trump. No, no, no.

I'm not, no opinion on Trump, but, you know what I love about Nassim's you follow him on Twitter? He is brutal and awesome. He's crazy on Twitter. Yeah, yeah, yeah. He's,

he's gone wild. He's like, this person is a moron. He's like Steve Pinker is in, absolute moron. Yeah. I'm like Steve Pinker's a moron, really. Steve Pinker's brilliant.

What are you talking about? I do think, I think directness is good, but I think I think black and whiteness is bad. So, you know, I, I'm not, I'm not totally going to endorse his Twitter activity. Yeah. But I do think, I think people need to be okay with disagreement. It's so weird that you say that as the 22-year-old because this is. whole generation that you're part of, they're literally going to college, this is why you lasted a year there, and they're protesting when they bring somebody to campus who they don't agree with.

Imagine that. You don't agree with a person. They're coming. You're diametrically opposed to their opinion. And then you protest having them there. It's like, well, you could either go to the lecture and learn about the person who you disagree with and understanding either the enemy or the opposing side or the other side of the argument makes you so much richer because of that. And they're like, no, you're platforming them. Like, platforming. Since when does talking to somebody mean you're platforming them? Like people are like, you talk to Steve Banning, banning, you're platforming them.

It's like, what does that even mean? Yeah, I think the guy put Trump in office. He ran Breitbart. These things are having a major impact on the world. You can't have a conversation with them. Even if you think he's evil. I think there's a big problem when you have like two. I think this is like a

derivative of like too much content out there. So basically people can anyone who has any belief can basically read enough content that reinforces that belief. That like for all these for these people, for example, they believe they have internally a very clear picture and a very high confidence perspective on like who these people are, what they believe, etc.

And that's just how, by the way, this is just how human brains are wired. Human brains are wired to be like, oh, you have a couple data points. Okay, you have to believe that. Yeah, tribalistic, yeah.

Right, because it's like in, in, before now, it was actually, it was difficult to get these data points at all told consistent narratives, et cetera.

But now because like, there's just so much content out there and you can like, you can read all of it and you can develop these very strong opinions. It's hard to think of other people as like nuanced human beings. Right. Versus, uh,

versus these like very, one note kind of like figures. Which is mind-boggling since anybody in their life need only look at their own life, whether it's 22 years in the plan or 48 or 98, and realize how many times they've

changed their mind about it an issue. Yeah. You need only, like, is your favorite ice cream the same for your whole life? What's your favorite ice cream right now? For me, it actually has been the same. Coffee ice cream? We just love that. No, no, no. Roky Road. Mint chocolate chip. Yeah. So you haven't had buttered pecan in a while. You need to just try that butter picketam one time, that might change everything for you.

Well, yeah. Well, I don't know. I mean, mint chocolate chip is kind of close to heart, close to my identity. Have you been to salt and straw yet? Salt and straw. You had the mint chocolate chip there? The mint is so fresh.

It feels like you're chewing on mint leaves. Yeah. We'll go right now. Everybody will see you at Salt and Straw next time on this week's service. Bye-bye.