Microsoft Research Podcast - 021 - AI, ML and the Reasoning Machine with Dr. Geoff Gordon

Episode Date: April 25, 2018

Teaching computers to read, think and communicate like humans is a daunting task, but it’s one that Dr. Geoff Gordon embraces with enthusiasm and optimism. Moving from an academic role at Carnegie Mellon University, to a new role as Research Director of the Microsoft Research Lab in Montreal, Dr. Gordon embodies the current trend toward the partnership between academia and industry as we enter what many believe will be a new era of progress in machine learning and artificial intelligence. Today, Dr. Gordon gives us a brief history of AI, including his assessment of why we might see a break in the weather-pattern of AI winters, talks about how collaboration is essential to innovation in machine learning, shares his vision of the mindset it takes to tackle the biggest questions in AI, and reveals his life-long quest to make computers less… well, less computer-like.

You cannot know ahead of time exactly what's going to come out because if you knew it wouldn't be research, you don't expect your payoffs to be measured in months or even necessarily a couple of years, but it could be that the things you're doing now pay off 10 years later. And so Microsoft has decided that MSR is in it for the long term. And that changes the type of research that you can do, right? You can afford to make big bets. You're listening to the Microsoft Research Podcast, a show that brings you closer to the cutting edge of technology research and the scientists behind it. I'm your host, Gretchen Huizenga.

Teaching computers to read, think, and communicate like humans is a daunting task, but it's one that Dr. Jeff Gordon embraces with enthusiasm and optimism. Moving from an academic role at Carnegie Mellon University to a new role as Research Director of the Microsoft Research Lab in Montreal, Dr. Gordon embodies the current trend toward the partnership between academia and industry as we enter what many believe will be a new era of progress in machine learning and artificial intelligence. Today, Dr. Gordon gives us a brief history of AI, including his assessment of why we might see a break in the weather pattern of AI winters,

talks about how collaboration is essential to innovation in machine learning, shares his vision of the mindset it takes to tackle the biggest questions in AI, and reveals his lifelong quest to make computers less, well, less computer-like. That and much more on this episode of the Microsoft Research Podcast. Jeff Gordon, thanks for coming all the way from Montreal to join us in the studio today. Welcome to the podcast. Thank you. I'm glad to be here. Montreal has become a global center for AI research, and you've just moved from Carnegie

Mellon University to take on the role of research director at the Microsoft Research Montreal Lab. We'll get to your work a bit later, but right now, what's the deal with Montreal? How has it come to be one of the most exciting places in AI research? Yeah, it's pretty amazing how Montreal has been sort of taking over AI. It started with the deep learning revolution because a lot of the people who were sort of early in on that revolution were from Canada and from this area. Yoshua Bengio is a key figure that everybody sort of credits with bringing so much AI to Montreal. But at this point, it's taken on a life of its own. It's growing by leaps and bounds. Yeah, it's like you get one person there who starts something and then it becomes a magnet. And there's a lot of credit to the Canadian government as well.

They've been trying really hard to make it an attractive place. They sort of understand what research needs to prosper, and they've actively been trying to recruit talent to the area. So that it's more than just industry. Right. I mean, it's government, it's the universities in the area and industry all together making a very great place to do AI work. Let's talk about you for just a quick second here. You told me what you say to people when they ask you what you do for a living. Give our listeners your elevator pitch and then maybe unpack it a bit more, assuming we're interested enough to ask the next question. Sure, sure. So what I always say is, you know how computers are annoying and inflexible. My job is to try and change that. AI is sort of all about making computers less

computer-like. That's a very concise statement packed with technical work. Yeah, no, it turns out it's easier said than done. I like the simplicity of it. So I've already interviewed a couple of, I'll call them rock stars from your band there. Harm Ben-Sayan, who's doing great stuff with reinforcement learning, and Adam Trischler, who's doing machine comprehension. I have to say, it's just fascinating what's going on at the Montreal Lab. What drew you there to be the front man, so to speak? So the Montreal area is one thing. The other thing is just respect for the work that MSR has been doing.

Microsoft Research has developed sort of a reputation for putting together the best of academic and industrial style research. The sort of focus on the long-term big goals, which I like from academia and very few industrial research labs, really try for that. And then there's the ability to build a focused team that's doing something big in academia. Your team is you and however many grad students you can recruit. And how much the grant will pay for it. Well, yes, there is that. So the goal of the MSR Montreal lab is very specific, to teach machines to read, think, and communicate like humans.

That's right. And this kind of learning has traditionally been the domain of humans. So why do you think this is possible for a machine? And where are we on the path to achieving that? Right. Well, we're a long way from fully achieving it, but we're making lots of progress on the way. And the first wave of AI was what I would call very logical. So somebody would put in a bunch of logical facts and syllogisms and that sort of thing. And they would hope that you would wind up with enough facts and enough rules that you would know how to think.

And that didn't work out so well. You were able to do with that some pretty impressive things, but you were completely unable to handle uncertainty. And humans are great at handling uncertainty. They may not like it a lot all of the time, but you're handling uncertainty every second of every day. Even right now we're handling uncertainty. Yeah. I day. Even right now, we're handling uncertainty. Yeah. I don't know what you're going to say next. No, me either.

Isn't that cool, though? Because we can actually formulate what we're going to say next. Right, right. We make it up as we go along. That's what humans are good at. How do I know what I think until I see what I say? Yeah. Hmm. That's profound. I've used it before. So keep going. AI. Right. And so the next sort of big thing was, well, okay, let's try and handle uncertainty. And there were these things like pattern recognition. And that was sort of very short-term thinking, right? It was, you see the picture and you tell it's a face and then you're done, right? And then the next picture comes up. So that's the sort of thing

that's in your cell phone camera now. People made progress on it starting many years ago, but now it's in everybody's cell phone camera, right? It draws a box around everybody's face and focuses there. And it still, to some degree, shocks me. Oh, it amazes me. You look at kids and what they learn. You know, I spend all of my working hours

trying to get computers to learn things. And then I go home and I talk with my kids and I'm like, man, I have a long way to go. So where are we now? So I think now we're sort of getting to the point where we're trying to combine these two capabilities, where there's this sort of reasoning ahead capability with this capability to recognize and react. And I think, you know, language is a good example where you combine the structure. Language is ambiguous. We never realize it, right? But like I saw the man with a telescope. Were you using a telescope to see the man or was the man carrying a telescope? That's a source of our uncertainty. But there's a lot of structure to it, right? You know what the phrases are within a sentence. You know what the relationships are among the words.

Well, okay, so we're making progress. We're a long way off. And yet this is a big commitment of the lab in Montreal is to... Yeah, it's a big bet. See the next thing. Yeah. I mean, the nice thing is that this is the sort of bet that you always win. Research is you go in thinking you're going to accomplish next thing. Yeah. I mean, the nice thing is that this is the sort of bet that you always win. Research is you go in thinking you're going to accomplish one thing.

And, you know, a large fraction of the time you don't accomplish that, but you accomplish something cool anyway. There's a famous quote, research is what I'm doing when I don't know what I'm doing. I love that. Yeah. Kind of feels like a metaphor for life in some ways. Yeah. I must be doing research a lot of the time. We all are, right? It's like trial and error.

So there are a lot of approaches to machine learning that researchers are exploring. And MSR Montreal is really known for its work in at least two of them, deep learning and reinforcement learning. That's right. So give us an overview of the technical side of these approaches and how they've advanced the science of machine learning. That's right. So give us an overview of the technical side of these approaches and how they've advanced the science of machine learning. So people have known how a little bit about how brains work for a long time, right? There's these simple processing elements, but there's, you know, a billion of them and they work together to achieve something that no single one of them could do on their own. And so people have tried to make computers learn that way. So people, when they first saw that you could do this, they were like, oh my God, this is going to be great.

This is going to solve AI. And they sort of overhyped it. And maybe 10 years later, the government stopped sending money into grants for that. The company stopped investing in it. And it was what was called the AI winter. And then people started realizing that it was actually a pretty cool idea. And people had sort of the next wave of neural networks, maybe 20, 30 years later, where they were able to train simple neural networks to do interesting things. And again, everybody's like, oh my God, this is going to change the world and overhyped it. And after a little while, we had the second AI winter. Now, I think at the risk of making a prediction, we're back where people are really excited about neural networks. And I think the change this time is that people have figured out how to work with much larger and more complicated networks and have shown success in training these ones. And they can do sort of qualitatively more than their earlier cousins. And now a student

of history would say, well, we're headed for another AI winter. But I'm a little skeptical of that because now there are so many real shipping products that actually have this technology baked into them, right? Like it's not going to go away. Things might change in the future, but I don't think it's going to play out the same way that it did last time. Right. There are tools in place, compute power, algorithms and massive data sets that weren't there for the neural networks 50 years ago. Right. Absolutely. I mean, the neural networks 50 years ago, I've seen a photo of them. It's like a bank of hardware bigger than this room. And it was there to train a neural network with one unit. And it had potentiometers,

variable resistors, and motors. And it would train by physically turning the knobs with the motors. Like the Alan Turing Enigma machine. It's all the knobs and dials and... Wow. Going off script a little bit, I just had a thing on my Twitter feed that said there's 11 seasons in Washington State. Winter, full spring, second winter, spring of deception, third winter, mud season,

actual spring, summer, false fall, second summer, actual fall. There's a lot of winters in there, though. Yeah. And, you know, you never know. Right, you never know. If it's a psych or... Much like AI.

The one thing we were talking about uncertainty, right? The one thing that you know is that you don't know what's going to happen. Right. And yet you're working on the discovery. And that's the exciting thing about being a researcher. Yeah, it is. Talk a little bit more about reinforcement learning, because that's a big bet of the lab. Harm gave me a good explanation a while ago, but I'd like to hear,

just for the audience now, if they haven't heard Harm's podcast. And they should go back and listen, because he's really a rock star. I love him. So reinforcement learning, it's the problem where you have your AI, the thing that you're trying to train, and it interacts with its environment. So it's usually called the agent and the world. The agent is the AI and the world is the thing it's interacting with. And what happens? You look at the world, so you get observations of the world. Those keep coming in over time. And when each observation comes in, you have to choose actions to affect the world. And this sort of cycle of action and observation with some thought in between, hopefully, is the definition of the reinforcement learning problem. And then there's one more component, which is that based on your entire history of actions and observations, the world can from time to time give you a reward or

a penalty, right? So, you know, you eat a dot and Ms. Pac-Man, right, it gives you 10 points or something like that. Or, you know, you run into the ghost and you lose a life and that's bad. And so your goal in reinforcement learning is to have the AI realize which actions tend to lead to reward and which ones tend to lead to penalty. And it's difficult because you never know, right? Like you might've done something several seconds ago that leads to you being trapped by the ghost. And it's not what you did like right before you got eaten by the ghost, right? It's the thing that you did that got you trapped and you have to go back and figure out what that was. Let's talk about language because that's the epicenter of your work. When I was talking to Adam about how a machine can comprehend and work with in the domain of language, he was saying, you know, you'll give it data sets of

a statement like machine learning is hard or machine learning is difficult. It's like, and then the computer is only recognizing what it's been fed. Why are we kind of poised to see big advances in the domain of language as far as computers are concerned? Right. There's fairly amazing language comprehension baked into products that we use every day, right? Like my phone will do it. And, you know, I think part of the reason that it's been more recent that that's made its way into products is actually not a flaw in the technology, but it's taken this long for that much computing power to become cheap enough to put into consumer products. It keeps getting cheaper. And, you know, we passed

a threshold at some point where now companies can ship, you know, a hundred dollar consumer electronics product that can recognize your voice. One of the other things that has been great for language research is the fact that people put up so much language on the web. And so you can get a billion word corpus of language. You have to be a major company in order to have the resources to process that. Well, you're working for one. Well, yeah, as it turns out. But, you know, you can go and you can say, okay, you know, hard is used in a lot of the same contexts as difficult. And so those are similar words. And actually easy is a similar word to hard because easy and hard are used in a lot of the same contexts.

Whereas, you know, avocado, very few sentences could you substitute avocado for the word difficult, right? And so you can also use hard for like a rock. Right, absolutely. And so you're effectively, I mean, you can cluster the different meanings, right? These are called vector space embeddings of words. So you can come and make a description of a word in terms of a list of numbers such that the list of numbers is similar for words that occur in similar contexts. And when you do that, words that have sort of similar vectors, similar lists of numbers, wind up being often quite close in meaning. Right. So I imagine the algorithms behind all this are pretty complicated.

Oh, but that's the fun part. For you. So let's talk about learning for inference. That's an interesting thread of your research. What goes into developing a machine's skill set for long-term thinking? use that concept. And so we're essentially trying to design algorithms that make that distinction, that give a computer experience at using a learned concept and train it so that it's better at doing that rather than just trying to learn the concepts in isolation. You can start by learning them in isolation, but then you have to try putting them together, actually using them to make a chain of reasoning. Okay. So what's the math look like behind that? I mean, there's a lot of it. The simplest bit is gradient descent, which is ubiquitous in machine

learning. So what that means is that you have your hypothesis described, your AI is described by a whole bunch of numbers, a whole bunch of knobs, that each one of the knobs has a small effect on its behavior. And you go and you have an example and you look at the example and you see, well, did it get it right? And if it didn't, would it have gotten closer to getting it right if you tweak the knobs a little bit? And if you do, you tweak them a little bit in that direction. And then you keep going, see another example and tweak the knobs again. And if you see a million such examples, then all of those little tweaks add up to a fairly well-tuned artificial intelligence hypothesis. But there's a lot more, right? I mean, there's linear algebra, right? There's functional analysis. I mean, I wind up having to learn a lot of tools in order to be able to put them together to design a new algorithm. How as a researcher do you sort through and say, I want to follow that path for a while?

Right. I mean, there's a lot of trial and error. You learn to recognize what promising paths are. And then there's a lot of collaboration. Like one of the key things for research is that you can't do it in a vacuum. You really need to get a good team of people together to make progress. And I don't just mean a good team of people at one lab, although, you know, obviously that's really important. There's like a whole research community, right? And they all build on each other's work. And so it's really important to have that community. So I think we called that

open source research as opposed to open source code. But I mean, there's that necessary collaboration. Right. I mean, you can't do it on your own. The problems are just too hard. And so you have to, you know, one person will have an idea to advance the state of the art just a little bit in some direction. And then somebody else will say, oh, well, now that I know that this thing I was trying to do becomes easier. And now I was trying to do becomes easier. And now I know how to do this other thing. And if you have a thousand people each learning about the other people's contributions, you make more than a thousand times as much progress. So let's circle back to Montreal for a second.

You have big plans for the lab there. What are you looking for? Who are you looking for? Where are you looking for? Who are you looking for? Where are you looking for them? I mean, we are looking for people who are creative about how they decide to attack problems. We're looking for people with great skills. You know, I mentioned that there are all of these mathematical tools that you need to put together to design the algorithms. If you don't know the tools, then you can't be creative about how to use them.

And, you know, there's this desire for exploration, right? That's like the key thing in a researcher. You have to be driven by wanting to know what makes the world tick. Because otherwise you would just never be able to devote so much time and energy into solving a problem. For people like you, researchers with PhDs who have that joy of discovery and the requisite skill set, there have been traditionally two places you could go,

either academia or industry. How is Microsoft Research similar or different? Microsoft has decided that Microsoft Research will attack the big questions with the sort of which implies right that you cannot know ahead of time exactly what's going to come out because if you knew it wouldn't be research and it implies that you don't expect your payoffs to be measured in months or even you know necessarily a couple of years but it could be that the things you're doing now pay off 10 years later. And so Microsoft has decided that MSR is in it for the long term. And that changes the type of research that you can do, right? You can afford to make big bets when you don't have to deliver the result of your work into a product in two months. So very much pure research as opposed to applied

research. Yes. However, you're seeing more and more of the work coming out of MSR. That's right. So you wind up doing this work and then once you've, you know, you don't necessarily know what you're going to get when you start, but once you do it, you look and you see that it's going to wind up being incredibly useful for some software or hardware product that Microsoft is making or is considering making, right? So I've seen researchers who ask a question that I would have said was completely abstract, right? No immediate connection to a product. And then a couple of years later, it turns out being sold to businesses around the world as a new piece of software. So that to me suggests that there is a lifting of the burden of success immediately.

Right. That allows you to ask a bunch of different kinds of questions. That's right. That's right. The freedom from having to know that you will get results in the short term, right, allows you to ask harder questions where you'll get results, but maybe not the ones you were looking for in the longer term. Because you never achieve what you set out to achieve exactly, right? You achieve something good, but never exactly what you thought you would. And so it makes it tough to plan, but, you know, it makes it fun, right? Because you're always seeing something new, something you didn't expect. Yeah. Difficult to plan and also difficult to measure if you have like a performance review.

It's like, well, I really did accomplish quite a bit. You just can't see it yet. Yeah. I mean, there's, you know, people have developed a whole bunch of imperfect metrics. So there's the metric of just counting how many papers you publish, right? But researchers, as they make progress, will write down pieces of that progress. And you can look at them and you can see, oh, you know, that was clever to be able to make that chunk of progress, even if I don't know what it's going to be good for right yet. But I'll bet that something good will come out of that. And that's the sort of thing that you try to learn through experience

and training, how to recognize what types of research outputs are likely to lead to sort of further progress down the road. And again, it's hard, but that's sort of your short-term measure of progress. You look at, you know, what did you figure out today? I love that, though. Yeah. I mean, how rewarding is that? Oh, it's very rewarding. For people who really wanted to, you know, take apart the world and see what makes it tick,

it's really cool to discover something new that you didn't know about how the world works, to add to the sum of human knowledge. Jeff, we're experiencing a kind of AI gold rush. You and I talked about the fact that companies of all stripes are trying to snap up the best talent. Yes. And their forest that they're clear-cutting is often the universities. Yes. And their forest that they're clear-cutting is often the universities. What if we clear-cut all the talent and training the next generation. One of the things that's great about MSR Montreal is we have an explicit goal of working with the local universities, of contributing to training. So a lot of our faculty in universities are interested

in collaborating with the researchers at Microsoft Research. And a lot of the researchers at Microsoft Research are interested in, for example, seeking out adjunct positions at local universities, being able to work with the students there and sometimes teach classes, sometimes teach in the sense of apprenticeship so that it doesn't have to be either or, right? You can be part of a great lab like MSR and still contribute to training the next generation. And there are different motivators for different people. Right. Well, it's the same thing that I said before. In order to be good at research, you have to enjoy the thrill of discovery, right? You have to really want to know how the world works. And I think if you set up a research lab well, it makes a big difference for how fun it is to work there, how interesting, how rewarding it is to work there. I think MSR has really done a great job of setting that balance. I ask this of all the researchers I interview, usually after a long conversation that essentially covers what gets them up in the morning. So given the work you do, even though we don't know all

the future implications, let's say we do succeed. Succeed in teaching machines to interact with humans the way humans interact with humans. Is there anything about that scenario that keeps you up at night? I think if you're going to worry about something with AI, you should worry about people misusing AI, right? So that could be intentional misuse where you design an AI to accomplish some evil task, right? As you sit here and stroke your fluffy white cat, right? But more likely it's going to be accidental. There's all sorts of things where if you don't spend a little bit of thought about how your AI is going to learn, then you can treat people very poorly, very unfairly. And so there's this whole area of AI research called FATE, fairness, accountability, transparency, and ethics.

And that's actually one of the areas that MSR is strong in, very strong. And so we're looking at, for example, how do you train AI algorithms so that they are not biased when they make their decisions? The problem is you train these things on past decisions. And so you learn to copy them, right? You told the algorithm, copy these human decisions. That's your data set. Right. And so if the data set is biased, then the algorithms are going to learn to copy that bias very efficiently and extend it to more people. And so you have to be careful, right? If you do things naively without thinking, you will freeze whatever bias was in the training

set and then ship it out to a much larger number of people. And so there've been a lot of examples where people have accidentally done that. But there are also lots of researchers working on how do you, despite biased training data, train your AI to be fair and to be unbiased. As we finish here, Jeff, what advice would you give to researchers that are in that space now, they're going to make a career decision? What do I do with my life? Where do I go? I've got a lot of options now that I didn't have. Right. I mean, when you have lots of options, you should always think about what you can accomplish with them. Do you just want to go with the gold rush, stick your pan in the water and hopefully find a few nuggets? Or do you want

to solve the world's problems? Or do you want to learn what makes the world work? And when you have the options, you have the luxury of being able to make choices that actually achieve the goals that you have for yourself. And so my advice is to think carefully about what you want. Think about whether you want to learn how the world works or whether you just want to make some money. Right. And there's a big difference in how you treat your options in those two cases. Yeah. Well, Jeff Gordon, you are so fascinating to me. Thanks for coming in. Thank you. To learn more about Dr. Jeff Gordon and the latest innovations in machine learning, visit Microsoft.com slash research.