Y Combinator Startup Podcast - #11 - At The Intersection of AI, Governments, and Google - Tim Hwang

Episode Date: June 16, 2017

Tim Hwang is the Global Public Policy Lead on AI and Machine Learning for Google.Read the transcript on our blog.

Hey, this is Craig Cannon, and you're listening to Why Combinators podcast. So today's episode is with Tim Wong. Tim's the global public policy lead on AI and machine learning for Google. And what that basically means is he interacts with governments to inform Google's opinions on policy. He also helps educate governments on what things like machine learning actually mean and helps them figure out what the implications might be. So in this episode, Tim walks us through how governments are thinking about AI, and he also shares some thoughts on what the future might look like. All right, here we go. And so I think with AI and then policy on AI, you've kind of like nested two obscure things

that people don't really know what you're talking about. So could you just back up a little bit and explain like what doing policy for Google actually means in the context of AI? Sure, definitely. So, you know, I think the really interesting thing about AI is basically that, you know, a lot of the modern techniques in artificial intelligence, if you've even asked people like a decade ago, they would have told you like, this is never going to be a thing. It's a complete dead end.

Why are you doing this research? And it really has kind of exploded in a completely unexpected way in the last few years. And so really a lot of the challenge has been like, okay, everybody's kind of wrapping their heads around even what the business impact of the technology is going to be. But there's increasingly a lot of people trying to figure out like what the social impact of the technology will be. And I would say policy really sits at that interface between these really cool technological capabilities that are coming about and then like what society in general is going to do about it.

And so what would be like a tangible example? Google of like a policy that you guys have worked on to figure out? Sure. So there's a couple of really interesting problems that we've been working on very closely. So one of them is this question about fairness and machine learning systems. And for example, to give you one really concrete challenge we've been thinking a lot about is in order to debias a system, right, once a machine learning system is behaving in a biased way, one way of trying to deal with it is collecting more diverse data.

But one of the big problems is when you do that, you end up collecting lots and lots of data about minorities, which raises all these really interesting questions around privacy and then what have you. And that ends up being a really interesting problem because it's both a technical challenge, right, which is like, can you collect an adequately diverse data set? But on the other hand, like, also this policy question, which is what is society comfortable with you collecting? And like, what are the practices? And that ends up being a really interesting tradeoff that you have to navigate if you're interested in these problems. And so what do you actually have to do? Like, are you going doing like user interviews with people,

or is it just guessing? Yeah, part of its user interviews. Part of it's actually working with like people who know, right? Like it turns out that issues of privacy, particularly minority privacy, are like not new problems. Right. And so a lot of our work is actually like talking with people who are like experts in that space, right? People have worked on, you know, bias and discrimination questions on the past and a lot of

data scientists and trying to get them to talk to one another because I think we're, right now what we're really trying to do is kind of bridge these sort of human values on one hand with like a lot of what's happening on the technological side. And so if I'm a company and I'm like, I can't afford a policy guy like Tim, and I will be dealing with large amounts of data that may or may not discriminate against people. Are there any, like, obvious no-goes that you would tell someone?

Well, I think it's to be sure that you're, like, interrogating the data, right? Like, I think that's, like, one important place to start. Yeah. Now, I think one of the interesting things about machine learning is that there's, like, lots of potential points of failure. And, like, I think every single interesting point of failure is being investigated right now.

But, I mean, one of the most common problems is just that you don't adequately think through your data, And so the machine does what the machine does, right, which is just trying to optimize against your objective function that you give it. And it will often maximize in ways that you don't expect. And that is, in fact, part of the problem, right? So, I mean, one of the examples that I always think about is, you know, we have this project that we released.

It was called Deep Dream. And one of the problems in computer vision is trying to figure out, like, what the computer actually thinks it sees when it looks at an image. And so you go through this process and you basically, the whole process, you show it an image, you ask it like, what do I have to do this image? to make it look more like what you think, for example, a sandwich looks like.

And you edit the image slightly and you keep repeating this process until, you know, kind of you show out, like, what is the ideal thing that the computer thinks it is. And it turns out that when you ask it to see, like, ask it to reveal what, like, it thinks a barbell looks like,

you know, barbells always show up with human arms attached to them. Oh, wow. Yeah. And so, like, that's a really interesting problem, right? Because you've trained barbells on photos that always have someone holding the barbell. And so it ends up learning this completely bad representation.

and what do you got to do? I mean, a big part of it is just like the consciousness around like, oh, that can happen. Right. And like how do you interrogate your data set to make sure it doesn't have those problems? And you guys are doing some interesting stuff around adversarial data, right? Yeah, that's right. So, I mean, I think adversarial examples and generative adversarial networks are like some of the hottest points in the research right now. It's almost become a joke that there's like so many what they call GANS out there right now.

This is like everybody has a GANN. So what does that mean? What does that stand for? So a general adversarial network. So it's a very particular way of. kind of setting up machine learning. But I have a serial examples leads to these really fascinating results where, you know, you can take a picture of a panda, and that's a classic example, and you edit a couple

of the pixels, and it like basically, like, the computer will be like, yep, that's definitely a giraffe, right? And it still looks like a panda to humans, right? It's the really fascinating thing. And so what data are you seeding into that image to make it think it's giraffe? Well, a lot of it, I think, is basically you're editing particular pixels within the image that we know will set off the machine to behave in certain ways. So because it turns out basically that like we always assume that a computer will see the same thing that we do just based on the visuals.

But how we process is actually completely different from machines. This researcher David Weinberger did this awesome article recently, which is basically trying to argue that like, you know, machine learning, it's generating knowledge. But one of the most interesting things about it is that it's generating knowledge and maybe a way that like is completely different from the way our human brains work. And like that ends up being a really interesting challenge is like how do you like understand the knowledge? knowledge that you're getting and how do you understand like the reasoning behind the knowledge that you're getting from machine learning systems. Well, maybe that's a sensible segue into like how people are investigating the impact of AI as it relates to like automation and what humans are good at doing and what computers are

good at doing. Yeah, right. And so when you travel around, you meet with people, you meet with different countries. Sure. How are people gauging the effects of automation and AI right now and in its effects over the next, like, you know, decade? Yeah, I think, so it's an evolving picture. Yeah.

Right. And I think right now, I think everybody is just surprised at all of the things that machines can do that we thought that humans were going to be good at for the foreseeable future. Right. So like Go is the canonical example. But there's all sorts of really interesting kind of like reasoning and other things that like machines are engaging in now. And so one thing I always tell people is basically that everybody always wants to think about AI as if it were like this huge meteor just crashing into the earth where they're like, what do we do when the AI arrives? Right. And it's just like it doesn't just doesn't turns up that it doesn't work like that, right? And in fact, like, what we really need to get to is, like, thinking about, like, how particular, you know, technical capabilities will map onto the economy. And that's what a lot of the work is happening on right now. Okay. And so, yeah, let's go into some examples. Yeah, sure. So, for example, one really interesting question is this adversarial examples, right? Which is basically, like, everybody always assumes that, like, okay, if it can be automated, it definitely will be automated, right? But that's, like, a fallacy because in certain cases, like, you may really worry about the security of your systems, right? So if

someone, for example, can, like, hold up a photo and cause, like, a security camera to be like, oh, it's definitely Tim. Open the door. Right? Like, that ends up being a real reason why you would not necessarily want to implement a machine learning system for, you know, access control, for instance. And so that's actually really interesting because that means that if we don't solve that research problem, that means that we will be limited in the kind of domains that machine

learning enters into. And I think that's what we're really interested in right now is, like, what are these kind of gateway research questions that if we got through, would, like, totally, totally changed the nature of like who when and why someone would implement this stuff. And so are those things collecting the interest in like the momentum of the research community? Because like I can see a certain direction where it becomes incredibly product focused, right? Where I'm like I'm a researcher. I'm incredibly talented.

Like figuring out if the security camera is going to work with an adversarial network like maybe not might not be of highest interest to me. Is that like blocking people or is like the general concept enough? I mean I think right now it's a little bit unevenly. divided, right? It turns out that, like, research interest is not necessarily, like, policy relevant interest, right? And so, in some cases, they're overlapping, right? So I think there's a lot of interest in adversarial examples. There's a lot of interest in, like, what are attacks, essentially, that you can put on these machines to get them to behave

in ways that you don't expect? That seems to be a place where, like, security, which is, like, very much a policy interest, will map on quite nicely to security as, like, a research interest. Okay. But for example, things like fairness, right? Like, I was talking to a machine learning researcher the other day who is basically like, look, I could not, in good faith, advise a grad student to work on machine learning fairness issues. Because it's not just not considered a serious problem in the field, right? And like, that's just like, that has less to do with like the field and more like the norms of the field, right? And then that ends up being a big issue. Right. We don't have coverage on certain types of things. And in practice may actually really limit like where these technologies are

implemented. Well, I think it's a, it's a material issue right now. Like there's a gap between product understanding and like actual deep research. Yeah, that's right. And I say this to a lot of people like, you know, everybody's always like, so what skills do we need to teach people in the future because of machine learning? And I think like one enormous skill will be like domain knowledge. Because like coming up with like a technical capability is just like one part of this huge picture, right, which is just like, okay, so then like how do we actually introduce automation

in a way that like makes sense to people? And like that's like a huge task. And so I know my personal prediction is that like interface and like how we effectively collaborate with machines, particularly with these new types of models. Like how that effectively done is still a big open question and will seem to be increasingly in higher demand as you still only have access to these capabilities. So what I've been wondering then is like does, for example, like, you know, TensorFlow or, you know, any one of like the machine learning APIs, does that become the new AWS for products

or do people have to build their own to create like a defensible company? I mean, I think there's still like, so, I mean, like cloud services will have the same impact on the economy that they always have, right? And I think this is one interesting thing is all these companies are now competing for offering cloud ML services. And the upshot of that basically is that the amount of like you don't need a PhD in machine learning to get all the benefits for machine learning. Right. And I think that will shape the space for sure. Okay, cool. So then what are the other areas like aside from the first one we talked about for automation and work?

Like where are other people interested? Well, so I think the other thing we're really interested in, and I'm really interested in is kind of like, is it possible to pull off machine learning with less and less data? Right? And so, you know, there's a couple examples of that, but one of them is like one shot learning, right? Where people are basically working on the ability to teach machines,

but like a much smaller number of examples. Now, that actually has a really big impact on the game because that means that you can implement machine learning effectively in situations where it's like really expensive to collect lots of data. There's also one really cool interface between VR and AI that's happening right now, where the whole idea is like there's a project called Universe from OpenAI and another project called DeepMind Lab, which basically like imagine you need to teach a robot to get through a maze.

Well, you could have it physically run through that maze millions of times, or you could just have a virtual 3D environment that you cause a computer to run through, and it learns how to do that in virtual space, and then basically you put it into, in practice, in a real robot. And so that's really another exciting way we do it, necessarily need like an expensive physical setup to collect the data that you need in or to accomplish tasks in the real world. Okay. So then what like I guess what I'm curious about then is like how are these countries like preparing for this like you know again not a meteor strike but like

perhaps a gradual shift over 20 years 30 years um to a very different world than what we have right now. Yeah and I think you're right now you're seeing like a bunch of different ideas out in the space. you know, some, for example, like basic income, right? Universal basic income, which would, like, fundamentally reshape, you know, like the social contract and how we think about doing, for example, like, welfare in a whole number of countries. And, like, so you see proposals like that. I think you see a number of proposals that are more, like, focusing on education. So, like, what are skills that people will need in the space? And that ranges everything from, like, everybody needs to be a programmer to, like, oh, well, we need to really encourage, like, computational thinking.

which is like the ability to work effectively with data. Right. And so like there's a couple of different options out there. Some of the more interesting ones that I've heard of that are a little bit more obscure, right? Like so some people have said like, oh, well, maybe we need like automation insurance. So in the future your employer will provide you with like a contract that says if your job turns out to be replaced by AI at some point in the future, we'll pay out at some kind of rate. Right. So people are like experimenting with lots of options right now.

I think what we actually need in the space is like more experimentation. So for even proponents of basic income, a lot of them will tell you, like, Like we actually don't know in practice what this would look like if it were actually rolled out at any level of scale. And so like, I mean, it's cool seeing YCR and a couple other places like experiment with this. And so where is attraction happening then with all these experiments? Like it seems very limited, but is it all like in Northern Europe or I know there's a basic income study in India at this point? Who seems to be focusing most on this area? Yeah, we're seeing a lot of different countries engage in this.

I think Northern Europe is kind of leading. the way in terms of their willingness to kind of experiment with some of these models. And I think they've got a couple things going for them, right? Like on one hand, I think they have a skilled labor force, right, that like is relatively expensive, right? So I think that they are seeking and excited about AI in large part because it's a prospect to bring, for example, manufacturing back to the country, right? Because it allows them to compete on the same footing as like other countries that have

offered like labor for like much lower costs. Right. Right. So like that's one thing that that's good for. for them. I think the other thing that's also encouraging a lot of experiments is that they have a lot more coordination between government industry and labor, which is making it more possible to experiment with these sorts of things. So I think in a really interesting case, it turns out that maybe Northern Europe is actually a little bit ahead in its ability to kind of experiment and understand

some of these programs. And then as a like Google or Alphabet as like this international institution at this point, how are you guys thinking about interacting with different countries as this happens? Yeah. So we're investigating at the moment, right? So the question is, is who on the research side should we be working with? And what are kind of programs that we could support that will help us give a better handle on this picture? Right? Because I think like, look, ultimately like it's a technology company.

Right. And so we know that we don't have all the talents necessary to like evaluate like, you know, what is a proper like social welfare program. But on the other hand, we think it's actually really important that we like encourage a better societal understanding of like how to deal with these technologies.

And so I think we're very much in the mode of like, like how can we support this? Okay. And I think that's partially through potentially resources, but also potentially like expertise as well, right? Like we, if you want to know anything about machine learning, we got people who can tell you about that.

Now we have to marry that up with people who have a good understanding of how this will impact society, either through economics or otherwise. And do you ever feel like the information you're disseminating is like guiding the conversation and guiding the future? Or like people are like playing into the game like it's intentional? Or it's just like opened up. I mean, I think it's very open, right?

I mean, I think like, you know, I think it's easy, particularly in the valley to be like, oh, my God, these big companies. But, like, we're only one part of a much larger, larger picture about what's happening in the economy. I, like, totally think that's the case, right? Like, you know, like, we talk about AI and automation, but we might also want to talk about, like, demographic shifts happening in the economy. Like, what's it mean that we have an aging workforce, right? Or, like, what's it mean that we have, like, falling workforce participation in the United States, right? Like, those are actually trends that, like, there are almost as large as, like, what someone comes up with a lab and presents at a machine learning conference.

And so, like, I think it's actually really important that, like, we look at this all in a bigger perspective. Okay. And so what do you guys do to, like, keep that in mind? I imagine you just have, like, a whole policy team to manage that sort of thing. Yeah, it's kind of what we're responsible for is, like, keeping track of a lot of this stuff. Okay. And, like, getting a better understanding of, like, who is researching in the space?

Because, as I said, you know, like, I think we're still really early on in this technology, right? Like, again, if you had asked someone 10 years ago whether or not neural nuts were going to be a thing, and they'd be like, yeah, I don't know, it probably wouldn't work, right? But like, if we're at a phase right now where, like, suddenly it has become technically real, I think now that understanding is just starting to percolate out to a bunch of other fields who are like, okay, well, I guess we now have to assess what's going on. And so do you see companies and organizations and countries like locking their gates because

they're scared because it feels new? Like, it's obviously massively hyped, but there's also some reality behind it. Has there been a negative reaction? Yeah, I wouldn't say so. I mean, I think by and large, what we're seeing is that a lot of governments are just really curious. They actually want a better understanding of what's going on. So in many cases, I think what we're seeing is people asking, you know, like, what is happening in the technology. Okay. So I think, you know, the phase of what to do about it is still on this way. Right.

So you, like, you like, give them, you know, the PowerPoint deck. And they're like, oh, okay, I kind of get how this works. And then they go home, you know, whatever, to, like, Japan. And they're like, okay, they think about it. Well, yeah, I think so. I mean, this is how government progresses, right? It's like, I think, like, they ask questions, they get information. and then there's like a long process of figuring out what you do around it.

But I mean, that isn't to say like there isn't like laws and other regulations being passed that have relevance for machine learning. So one of the most interesting aspects of the GDPR, which is a new privacy regulation in Europe, is the potential for this, what they call kind of a right to explanation. So the idea is for certain kinds of automated decision making, it might be so significant as to require or give citizens the right for that system to be able to be able to, produce some kind of human understandable explanation for what it's doing. And that raises all sorts of interesting challenges about how you actually pull that off. And so, like, I would say that I don't want to make it sound like no governments are taking action. But I think, like, that's the beginning part of it, right?

And I think, like, by and large, the stance of most governments have been, like, understand what's going on. Do you think someone's doing it particularly well now? Yeah, I mean, I was really excited by some of the stuff happening out of the UK. So last year they actually did a report that was on kind of like giving an account of like the risks and opportunities from artificial intelligence. And I think there was like a really good account to that. So and then last year with the under the Obama administration, there was a really good report that they did as well on the topic. Okay.

And so like can you go specific on that? Yeah, sure. So I mean, I think the what we at least have it in the, what we at least had in the U.S. case, right, was basically a report that like really focused in on like, okay. What are the real concrete risks here? Yeah. And part of the idea was to pivot away from discussions that were just like, okay, the main thing we've got to talk about here is whether or not robots are going to destroy us, right?

Like, or decide to take over, right? Yeah. Which I agree is like kind of an interesting scenario to consider. But like, you're right, like, there's a lot of like core near-term problems that need to be dealt with. And I think that was one thing they did that was very useful. So. Well, aside from the stuff we've talked about, where, what do you find to be particularly exciting, both here at a local Bay Area level as far as like research and then at you know global

international research level moving this stuff forward yeah so I think there's two things that I find really interesting right now one of them is the intersection of machine learning and art right so like largely this is technology we've been using to solve like pretty pragmatic things right which is like how do we ensure that we can adequately recognize like cats and photos but like what's really interesting is a bunch of people who are kind of playing around right now with the intersection between like, oh, could I use this for like artistic purposes? So there's a really fun project. Google has this project called AI experiments,

which is a lot of kind of like small things like this, which kind of demonstrate the kind of artistic possibilities of technology. We have another program called Magenta, which is looking into machine learning and music, and like whether or not there's ways of kind of creating better creative collaboration between humans and machines on that front. Hmm. And have you experimented with it personally?

Yeah, some of it's really fun. There's one project, which is basically like a melody generator. Like you play some notes on a piano And then the computer will play alongside you Like harmonize with you? Yeah, exactly, right, right? And you kind of like improvise with the computer Which is super cool

There's another project called Maroder Cam Which you get on your phone Okay Which is like you take a couple of photos of things in the room And it produces this like bop in like electronic dance hit That has like that uses the words of the objects in the room As like a rhyming, you know, set of lyrics Oh, super cool, yeah

And a great example of like how the technology is becoming like really accessible because again if you wanted to do that like 10 years ago it would have required like a huge amount of money and like you know a bunch of PhDs to try to work on this problem right yeah I've been fascinated with that like how it's become distributed just even in the past like year like I you know I told you about all the speech to tech stuff that I'm working on yeah man like the fidelity of it is shocking yeah that's in like one year right right and so it's gotten like way better which I think is super interesting um I think the other thing is also like trying to figure out like, there's these like really unexpected things that emerge too.

So the other thing that's, I think, is really cool right now is there's a paper that came out from Deep Mind, I think earlier this year that was kind of like, if you get two machines to talk to one another, they will eventually, like, and you can set up another computer to basically say like, oh, I can read what you're saying, I can't read what you're saying. You can basically train these two systems to come up with the rudiments of encryption without even necessarily need to program encryption into the computers, which is also like super cool as well. They learn how to accomplish that task.

And it's not like very good encryption, but like the basics are basically learned by these systems so long as you give them good reinforcement on like, okay, that's still cognizable. I can still understand what you're saying versus like a third party being like, oh, I can't do that. Oh, man. And so do you have thoughts on like how this will become distributed in such a way that any day like we'll be interacting with it in our everyday lives as just like fun projects? Like will it be existing in the art space? will, like, be, you know, like, training new programming languages for folks to work on when they're younger. Yeah, I mean, I think, like, there's, you know, I was talking to Peter Norvig, who is, like, one of the researchers we have is, like, one of the, you know, founding fathers of AI. And he had this really interesting thought, which is basically that, like, we may be approaching the period where we actually have to entirely rethink how we teach computer science.

Because, like, machine learning is such a powerful tool. And also cognitively, it works in a way that's, like, totally, you know, counterintuitive, right? So, like, I do less software than I used to, but, like, definitely when I was in the trenches doing coding work, it was very much like, okay, like, let's get a bunch of smart people in the room, let's come up with a bunch of rules and then, like, get those rules into the machine versus this much different kind of mode of thought, right, which is basically like, let's present the machine with a bunch of examples and then verify whether or not the machine has learned the proper lesson.

And so his idea is, like, actually, we may actually really want to think about how we like think about CS from, like, the very first moment you step into a classroom. which I think it's like a super compelling idea because it was always thought of like, oh, machine learning is just going to become this complement to like how you do programming. But I wonder whether that software in the future will actually look more and more like machine learning focused, right? And like you actually change your entire approach to programming systems. Oh man, that's fascinating. I mean, it's already kind of gone that way in that like many CS programs are so technical.

You actually never build a web app. Yeah, that's right. You can go through Stanford CS and never build a web app. Yeah. And I think it's a very natural trend that like, you know, we're getting to higher and higher level. of abstraction. So like in some ways, machine learning is this like ultimate level abstraction where it's like even if you wanted to understand what's happening in like a neural net, like it might be actually like kind of difficult to do so, right?

Yeah, I mean, I guess so. But I see it becoming like there's just new ways of thinking about how you ought to be programming, right? Like how you structure the code because at a certain point, things will just become abstracted and you won't have to do it anymore. Like I think about it in the context of like, you know, parse creating an API, right? Like that will exist for many things. Like I could see a like a Squarespace type thing, but for like a proper web app, right? And you just drag your database in and you never even think about it. That's right.

Yeah, yeah. And so ironically, like programmers might lose their jobs way sooner than they think. Well, and particularly interesting because like we actually like this emerging research right now, which is using machine learning to train machine learning systems. Yeah. It raises like this meta level where like right now there's a lot of handwork that goes into building a model so it learns the right representation. but like if a machine can do that in the future it gets even more abstracted where you may not

even need to be like a specialist because in some ways the machine kind of like codes itself so so I think one thing that a lot of people are curious about is how you're actually going to build a business around AI so just for like we can start broad and then go more narrow do you think AI will be like dominated by massive companies like Google Facebook or will you know they'll be very successful AI products on that small scale. Yeah. So I actually think that there's actually like a ton of room for competition here. And it'd be interesting to see how all the various companies find their niches in the space. I think there's two really interesting trends right now, right? I think one of them is

the emergence of like cloud platforms, right? Where basically all the companies have said, like, there's a long tail of uses that we would never be able to like take advantage of, but we may be able to like provide the services that like power those services. Right. And so like, for example, Google is offering like cloud amount right now. And I think it's a really interesting development in the space, which I think creates a lot of opportunity because it means that there's all these industries that might not necessarily be like AI industries that might be able to like seize the benefit from the technology. So that seems like a pretty huge thing to me.

I think a second one, which is really interesting, is like some of the one shot learning stuff we talked about earlier, right, which is basically that the amount of data you need to pull off certain types of machine learning applications is going down over time. And what that tells me is that there might not be necessarily a first mover advantage in the space where you may actually have collected a bunch of data, but if it's not the relevant data and also the amount of data you need is going down over time, then the real big challenge is less data and actually more your ability to build like good interfaces and good experiences around the technology. Yeah, I've been wondering about that as I play around with it and build like tiny little web apps and stuff like how much of this is just entirely reliant on the product as like it's all plug and play. And so to a certain extent, like, folks can almost guess which techniques you're implementing, which APIs you're using. And if they're faster with better engineers, and then they have, like, the magic touch of, like, the product person. I don't see any reason why they can't just jump ahead.

Yeah, right, right. And I think we're maybe fooled by, like, the nature of the field right now where it's like, ah, we got to get, like, the most researchers to go and compete on this thing. And, like, that is, like, a big important part of it because they're producing a lot of, like, the breakthrough. threes in the space. But it is, I think, important to consider, too, that, like, there's still, like, this big open question of, like, how this actually becomes, like, effectively part of product. Oh, well, absolutely. I mean, we did an interview at Bidu, and it may or may not come out before years. So we might do, like, a fourth wall jump. But they explicitly are focusing on things for over 100 million people. And you're like, oh, okay, well, I can build plenty of successful

startups or businesses for less than 100 million, maybe even a million. And so, yeah, I think there are just all these fantastic opportunities for people. And yet folks seem to be focusing on very similar implementations, you know, whether it's like chatbot or like, you know, customer service, which I guess is effectively the same thing. Why do you think that is? Is it they just like follow what seems to be like the market leader or these like the most obvious? Yeah, I think people are also still trying to figure it out. Right. Like, um, and I can't, I think we can't avoid like that AI is like a technology, but AI is also like a position. It's a marketing position, right?

Which is I think is actually like a really key part of the picture. Right. It's like, why do we think about like Siri or like the Google assistant as like AIs? But we don't necessarily think about like the Facebook news feed as an AI. Right. Like these are all systems that are all powered by machine learning. Right. But there's something about like it's representation as like, oh, yeah, this is a machine that talks to you.

Right. that makes our brain snap immediately to like pop culture, you know, equals AI, right? And then that ends up being a really big part of it too, is that there's a lot of incentives to like correspond to what we think of as AI, even though like some of the most powerful AI applications may not even come in the form of like a personified, you know, personality. Well, I think that's a, that's a super interesting angle. It's like out here seemingly it makes sense to like raise your money as like an AI business. but like when you look at Facebook right Facebook if you log in doesn't say AI anywhere

and clearly they have a lot of people using it so I wonder if it is like a massive positioning thing that many companies do end up missing because you just have to get like the nerdy people interested in it to sell it to raise the money if you're going to do venture back or whatever but then your end user is like why am I paying all this money for this like chat bot I mean like for example yeah if you want to talk about like one of the most critical applications of machine learning to date, it's like spam filters, right? Spam is like this incredibly huge systemic problem on the internet. It is like largely contended with by machine learning right now. That's like largely the tools that we use to deal with

it. And like that's like an application that we never think about, right? Like with many technologies, the most important applications will be some of the least visible. Hmm. So what, um, what are you excited about? What are you going to build? What are you going to build with AI? space. I got to think about it some more. I mean, you know, I'm really interested in these kind of like small scale machine learning projects. I think we might have talked about it earlier, but like we have this really crazy

story where it turned out that there was this cucumber farm in Japan that was using machine learning to build like a really cheap machine learning like robot. Yeah. That would sort cucumbers. It turns out like cucumber sorting is a really big problem in the machine, in the, in the cucumber farming space. Right. And that was basically just trained using like 3,000 or 4,000 like photos of

cucumbers. And that was sufficient to train a model to do, but like a pretty good job at like sorting cucumbers. And like so like I'm really interested in this kind of like artisanal machine learning. Where like it's like what are these kind of like very specific daily problems that I have? And it's a good way of I think wrapping my head around like

okay, what are actually going to be like the practical uses? Not necessarily like the like Cadillac uses that I think we're being pursuing right now, which are like the demonstration uses of the technology. And then you can open up like Tim's general store online. Yeah, that's right. but like download like Tim's cucumber app. Right.

Yeah, I mean my, I cracked my iPhone earlier and was getting it fixed this morning. And the guy had an entire box of assorted iPhone screws from literally like an iPhone, you know, iPhone 1 to an iPhone 7 now. And these are just like, he's got like a side hustle buying and selling iPhones that are like broken online. And if they're totally damaged, he just like strips all the components. But he spent like half an hour. like trying to figure out what screw would fit.

So like, there you go. You can like use like Tim Screw Identifier. Right, right. Like it's super handy stuff. Yeah, I think it will be like just a lot of small things like that. And what's particularly interesting is like going back to a little bit what we're talking about earlier, like what is the cost of solving a problem through machine learning? Right.

And what is the cost of solving a problem through like traditional coding, right? And that's actually maybe one way of thinking about the problem. Right. Like for example, for computer vision, right? Like now the economies are way in favor of machine. It's just way easier to design an effective machine learning image recognition system with Yeah, with ML than it is with like traditional kind of coding techniques and I think that's actually one really interesting way of thinking about is for a given task how long until machine learning is like the preferred way of solving this problem with the computer It totally makes sense as like new kinds of entrepreneurs pop up in these like very small niche things that are essentially like one developer projects that previously like might have even seemed like way too laborious

to spend your time engineering, like, you're never going to pay someone to do it. You're not going to do it yourself. But, you know, you start plugging into like these cloud ML things and all of a sudden you have this app. Right, right. As far as distribution, I don't know. Like, I've heard more and more people talking about like localizing certain things to the device, which makes them amazing.

Yeah. Have you experimented with that yet? Yeah. So we're actually working on a little bit of research around that. I haven't played around it myself. But for example, there's a couple of papers around what they call federated learning, where which is exactly working on this premise, which is the bet is, okay, well, what happens in a future where the edges of our network, like the phones, like have way more powerful processing

power? Like, is it possible for us to basically do the majority of the training for these systems, like on device? And with, like, basically a lot less data kind of, like, flowing into the cloud. And the idea was basically, like, the local model would update, and it would share its learnings with all the other devices in the network. And it's like a really interesting way of thinking about how you actually do this, because what you ideally want to have is models that are loaded on the device. Right. And can also train on the device as well, right? Because right now, one of the irony is that there's big disparity between like training is computationally intensive, data intensive, and then actual execution, right, which can be

actually like pretty low, low computational. It also creates a giant latency problem with everything that's like in big quotes AI right now. Like, you know, most people, you give them Siri, they're like, oh, that's constantly broken. But if you could communicate with it in a way that's like, eh, you didn't understand, let me go again immediately afterward. All of a sudden, the experience is entirely different. Yeah, and latency ends up being really key, not just for like conversational interfaces, but you think about like, you know, for example, like, how do we deal with, like, using this in medical,

right? Where you may need a response really soon if you're going to use it for, like, diagnosis or whatever. Totally. Like, if this thing turns into a robot surgeon arm and I move it to the Amazon, like, I can't rely on my like you know hot spot yeah that's right yeah yeah and so yeah I think again we're like talking about implementation which ends up being like this really big piece of the AI picture which is still being worked out like we know we can get machines to do these remarkable things the question is like what do people actually want out of it so I guess one of the last questions

I have for you is you know people are interested in AI machine learning across the board or at least people paying attention to this are into it if someone wants to get more into it and they're thinking about like, how do I position myself? Like, what should I pay attention to? Where should I focus? Because like, you know, now tens of thousands of people are checking it out. What would you say? What would you focus on? So I think there's two really interesting problems in the space right now that like desperately need more people to get involved in and more people to kind of like organize events around. Okay. So one of them is I think this like security thing, right? We're like in the traditional computer security space, we've got like events like

capture the flag where people can kind of like share. show their metal in their ability to kind of like secure and compromise systems. I actually think we really need that in the machine learning space. And I really excited to see that, which is like, so imagine a game where like you have to train a machine learning model on a set of data. And then people will take turns trying to like get past your computer vision system. Cool.

Which I think would be super cool to do. And I think like that's one big piece of it that I think would be really cool for people to work on. I think the second thing that's about to be in really strong demand is thinking about the visual dimension of this, right? Which is like, it happens on a couple levels. That's both like the interface of how you work with machine learning systems, but also just like visually how you represent a neural net. Like if you've read the technical papers, one of the things that you'll see is just like,

that's like largely written by machine learning experts. And so like they don't really have a good sense of like how do you visually portray what a neural net is doing. And that stuff ends up being incredibly important for people to like both understand the technology and and also be able to like use it effectively. And so I think that's. another thing that's about to come on the way is like basically a really high demand for people who understand this research and could give it good voice in terms of like representing it visually.

And then if someone isn't into machine learning yet, what would you recommend they read, study, watch, what should they check out? So I mean, I think it's really nice because we're now living in a world where there's a lot more resources for how to learn about machine learning. So I'm a huge fan of Ian Goodfellow's textbook on deep learning. It was really funny. I was in Cambridge picking up a physical copy of this textbook because MIT Press is the publishers. And the guy selling me the book was like, this is like the Harry Potter of technical guides because it had been like flying off shelves so aggressively. So it's really good though. It's reputation is very well deserved. Okay. One of the things I've been thinking a lot about is kind of like

the history of all this, right? Like it's important to recognize that like AI has been through this hype cycle before and there have been long AI winters where this technology is totally oversold itself. And it's like important to understand those dynamics. So two books I'll mention, one of them is John Markov's Machines of Loving Grace, which is all about kind of like the history of AI, and particularly it's competition with the notion of IA, right, intelligence augmentation. Okay. Which I think is a really interesting battle that we're having right now, right, in terms of like what this technology is really about and what should be used for. A second book, this is great,

which is also by MIT Press, is cybernetic revolutionaries. Okay. Which talks about basically the Chilean IAND government. So it's basically the, the socialist government, like, during the mid-20th century. And they tried to basically set up a project called Project Cybersyn, where they're like, let's automate the entire economy. So all factories will have to produce data links that will connect to a single central command center where we will, like, actively control the economy. And it's a great initial, another example of kind of like, oh, like, kind of like the history

of cybernetics, but also it's like implications for like what people try to do back then that I think useful for like, you know, making sure we understand what the limitations of the technology are today. That's very neat. I haven't read that. I will absolutely check it out. Cool, man. So if anyone wants to follow you online, where do they go? Oh, sure. I'm on my website is at Tim Huang, T-I-M-H-W-A-N-G. I'm not the Korean pop star of the same name. And I'm also on Twitter at Tim Huang, so at T-I-M-H-W-A-N-G. Very cool. All right. Thanks, thanks for having me, Craig. All right, thanks for listening. So please remember to subscribe.

and rate the show wherever you find your podcasts. And if you want to watch videos from any of these episodes or read the transcript, you can check out blog.wicommidator.com. All right, see you next time.