ACM ByteCast - Henrique Malvar - Episode 71

Episode Date: June 27, 2025

In this episode of ACM ByteCast, Rashmi Mohan hosts Henrique Malvar, a signal processing researcher at Microsoft Research (Emeritus). He spent more than 25 years at Microsoft as a distinguished engineer and chief scientist, leading the Redmond, Washington lab (managing more than 350 researchers). At Microsoft, he contributed to the development of audio coding and digital rights management for the Windows Media Audio, Windows Media Video, and to image compression technologies, such as HD Photo/JPEG XR formats and the RemoteFX bitmap compression, as well as to a variety of tools for signal analysis and synthesis. Henrique is also an Affiliate Professor at the Electrical and Computer Engineering Department at the University of Washington and a member of the National Academy of Engineers. He has published over 180 articles, has been issued over 120 patents, and has been the recipient for countless awards for his service. Henrique explains his early love of electrical engineering, building circuits from an early age growing up in Brazil, and later fulfilling his dream of researching digital signal processing at MIT. He describes his work as Vice President for Research and Advanced Technology at PictureTel, one of the first commercial videoconferencing product companies (later acquired by Polycom) and stresses the importance of working with customers to solve a variety of technical challenges. Henrique also shares his journey at Microsoft, including working on videoconferencing, accessibility, and machine learning products. He also offers advice to aspiring researchers and emphasizes the importance of diversity to research and product teams.

This is ACM Bytcast, a podcast series from the Association for Computing Machinery, the world's largest educational and scientific computing society. We talk to researchers, practitioners, and innovators who are at the intersection of computing research and practice. They share their experiences, the lessons they've learned, and their own visions for the future of computing. I am your host, Rashmi Mohan. Every time you listen to your favorite artist uninterrupted on your preferred online music

service, or when you scroll through the vivid images and videos on your favorite social media platform, I want you to stop and contemplate on the volume of data flying over your network to make that possible. The possibility of streaming content in a smooth and bandwidth efficient manner was brought to life by some of the pioneering work around data compression done by our next guest. Rico Malvar belongs to the very elite group of Microsoft researchers to have achieved a meditator status. He spent over 25 years at Microsoft playing the role of a distinguished engineer, chief scientist, leading the MSR lab at Redmond, championing many exciting projects, and founding the Signal

Processing Group. He is an affiliate professor at the University of Washington in Seattle, a life fellow at IEEE, and a member of the National Academy of Engineers. Amongst his prolific contributions spanning multiple countries and continents, Rico has published over 150 articles, been issued over 120 patents, and been the recipient of countless awards for his service. Rico, it is truly an honor to speak with you. Welcome to ACM Bytecast. Rashmi, thank you very much for the invitation.

It's a pleasure to be here having this conversation with you. Wonderful. I would love to lead with a simple question that I ask all my guests, Rico. If you could please introduce yourself and talk about what you are currently doing, as well as give us some insight into what drew you into this field of work. Thank you.

I'm Rico. My real name is Enrique in Portuguese, but it's a little hard to pronounce. I was born in Brazil, born and raised there in Rio de Janeiro. And then since I was a little kid, I knew I wanted to be an engineer. So I studied to be an engineer. So I studied to be an engineer. And then I had dreams of one day going to the United States and being in a research lab and doing cool things. And sure enough, that happened.

It was a long journey, but a journey that culminated about 40 years ago with my finishing my PhD at MIT. And then since those early days, being involved with startups and other things, always trying to put research and real engineering together. That's amazing, Rico. How does a little kid in Brazil dream of becoming an engineer? What was that early exposure that you had that said to you that, hey, this is the path that I want to choose? Yeah, that's a good question. I always tell this small joke that it took me a long time to decide to be an engineer.

I was already eight years old when I decided I would be an engineer. My dad gave me on my eighth birthday, he gave me one of those electronics kits that you put little pieces together and you make a little amplifier, a little radio, and I just love the stuff. And then I said, Dad, thanks, I'm going to be an electrical engineer. I love electronics and that's what I did in Brasilia, Brazil. We had moved to Brasilia and I did my undergrad there in electrical engineering. And I loved building circuits and doing those things. And then I did a master's over there.

And then I realized that the world was showing signs of becoming digital. And that's when I started thinking, oh, I want to study digital signal processing. And then already in college, I was reading all these papers by researchers here in the US. And then I started having ideas. Maybe one day I'll be there doing some of that. And sure enough, it happened. Yeah, I actually remember the simple circuits that are offered to children.

I have two daughters and I remember we got those as well. And they were very fascinated by it. Interesting that you also decided to sort of jump into research. Was there a sort of an inspiration there as well? Because oftentimes we find, okay, you know, you go to college, you study a certain field, and then you tend to sort of say, okay, where can I apply this? So what was the interest in research in particular? That is a good question. I was lucky. I think about time when I was in high school, I had a couple of teachers that maybe they liked me or they kind of realize the potential in me or something

like that. And they always expose me to new things in the lab or another book. Oh, do you want to read this other book or do you want to know more about this thing or do you want to build there? So they always push me to do more. And I realized I liked to do that. I didn't want to be limited to whatever the textbooks were being covered in the classes.

I wanted to do more and invent more. And I always had in my mind, not only I want to study new things, discover new things, but I want to use that to build new things. And I really always got a kick out of inventing something new, be it a piece of software or a piece of hardware digital or some analog hardware. Every time I built something and that thing worked in some way, I loved it. So that was my inspiration. But thanks to the early professors that helped me since the high school days.

Yeah, no, it's wonderful. I think it's very gratifying to see something that you build work and possibly be used by others But also I think I love that your teachers recognized your sort of innate curiosity and kind of fueled it Because that's so important as well, right? I think finding those pathways and or even those somebody who can show you what is possible Is very defining in kind of how you choose to navigate your education or your career. Oh, what you said is so important. I remember if we fast forward to maybe 10 years ago when I was the managing director of Microsoft Research

or when I was managing my own group, Signal Processing, sometimes I'm having a conversation with a researcher and the researcher is telling me all the stuff they're doing and, oh, I'm fully focused, I'm spending a hundred percent of my time doing this. And I would always tell them, don't do that. Never spend a hundred percent of your time doing anything. You want to focus, but cap it at 70 or 80%. But take one day of the week and go do something else. Go study something else.

Go research something else because you never know. You're going to be able to discover some new thing and invent some new things. So don't get stuck in the same thing 100%. Yeah, I know. I mean, that's great advice. I mean, it seems like it's the foundation of anything interdisciplinary that has to work. Right? The idea to be saying, okay, I have a domain expertise and I'm working in a certain area, but if I explore maybe adjacent or maybe some rank new ideas,

I'd be able to see how the two of them could actually blend and create solutions to problems that maybe haven't even been fully articulated. Exactly, exactly. And one part is that it saves some time to study new things. Another part is save time to talk to other people. I remember when I was the lab director, periodically I would do reviews with research teams and I love doing those because the

researchers would present the work they are doing. And almost to every presentation, I would try to be motivating, asking motivating questions, but near the end of the presentation, I would always say, have, asking motivating questions, but near the end of the presentation, I would always say, have you talked to anybody in Microsoft product team X that could potentially use the results of your research? And some of them would say, oh yes, I'm talking to this team. And others would say, not yet.

And say, okay, so let's talk later and I'll introduce you to a few people. And the more I did that, the more people realized, oh, I better talk to somebody in the product team because Rick was going to ask about that. And that is so important because not only you create potential collaborations, but you also learn about potentially what it means for that technology to work in the real world and how people actually gonna use that technology. It's a long path from something that works in your head or works in a piece of demo software

all the way to be in the hands of somebody, but ultimately that's what we want, right? To be in the hands of somebody and bring benefit to them. Values, absolutely. Yeah, and I think that, you know, that early feedback is also so critical. I definitely want to talk about that. I know I have that down on my list of questions. I want to go back a little bit, Rico, to your early days of research.

So when you decided to do your PhD, you're one of the, obviously, the most critical parts of, I think, defining your PhD is how did you choose your problem space? That is a good one. As I told you, as I mentioned before, I did a little bit of studying on digital signal processing and that was interesting because I was studying telecommunications, I like that.

And I say, wow, but all the telecommunications I studied before in my undergrad work was all done by these analog circuits, so very specific. But if you do digital, if you do via software, you can change the software and then you change very easily what the system can do. That sounds very impressive. And I also started the beginnings of studying in the early 80s about data compression and say, oh, this technology is going to be key because someday maybe things will all be digital. At that time, we didn't have internet or anything, but we could see the path. And I say, I want to study that. How do we do the communication that we do today with radios and things like that?

How do we move all of that to digital? And that was my motivation. And MIT has always been very good in that. And I was lucky enough to be accepted there and it was great. And then I learned a lot about digital signal processing and that became my career. Amazing. Hats off to the insight.

That was one of my follow-up questions, is like hitting upon a research question that you want to pursue, but also anticipating the kind of widespread application that it might have. You are credited with inventing the concept of lap transforms. I'm just wondering if you could maybe delve

into a little bit more detail for some of our more technical listeners on when you defined the problem or what kind of technical challenges did you run into as you were trying to solve some of these problems? Oh, yes, because at that time, the Left Transforms were actually the topic of my PhD thesis at MIT and the core idea there, for example, you've seen digital pictures in the JPEG format, right? It's the most popular, most cameras and phones take pictures in JPEG.

And if you compress the JPEG image too much, you see some little squares, right? In the image, you probably saw, have seen that at some point. They are calling blocking artifacts because the way JPEG works is that it cuts the imaging in small squares, compresses each one and then at the receiving end puts them back all together and then the left transforms, breaks the image into overlapping blocks and then when you put them together things overlap and you don't have as much of those blocking artifacts. That's particularly good

for audio. The audio compressor we're using now uses lapped transforms because this I think is being recorded in mp3. So the mp3 uses a kind of lapped transform which is called the modified discrete cosine transform or MDCT and then with these overlaps, you don't have blocks, so you don't have those distortions. You get a more smooth signal. Was that okay? Yeah, no, absolutely. You might be the first podcast guest who's actually, you know, we're actually using the technology that you worked on. It's quite fascinating in and of itself. Yes, yes. But the interesting thing is that I was studying that in my college mates, my office mates at MIT, and Jeff Bernstein and Brian Heumann, we're always discussing things together.

And then they actually came up with the idea of using those compression technologies to build video conferencing machines. So we are all passing our qualifying examinations and then I have to quote Brian because he told me, okay, Rico, I'm going to stop here at the Masters, I'm going to found a company. You want to continue your PhD, go ahead, continue your PhD, but when you finish, I'm going to hire you and you're going to help our company." And that company was PictureTel. So it happened exactly as he said.

And PictureTel was founded, was an IPO in the mid eighties. It was not that common for that to happen, but it was a very successful IPO. In two days, they raised all the money to fund the company for a couple of years. And then in the 90s came the Gulf War and nobody wanted to travel. Video conferencing took off and PictureTel was the leading company in that. And then a few years after my PhD, there I was at PictureTel as the director of research and then later VP of research. And PictureTel really helped push the technology of video conferencing, which at that time in the 90s was very

difficult. There was early 90s, there was no internet, we used those ISDN lines, the equipment was big and all of that. Today, fast forward to today, every little phone is capable of doing audio and video conferencing using many kinds of applications, Teams and Zoom and many others. But it was fun. So we had our food in the early days of how people communicated with audio and video in real time. Yeah, I mean, that's amazing. I was going to ask you, what happened post-PhD?

You often hear of folks who tend to say, hey, I want to stay on in academia. It's very attractive. You obviously have a cohort of people that you enjoy working with, and you're passionate about the subject. But it sounds like you already had a job lined up post-PhD. It was interesting, because that only happened a few years later.

Because I had actually a scholarship from the Brazilian government to do my PhD at MIT because I didn't have money to pay for MIT tuition. My family could not afford that. So I had that scholarship, but a condition of the scholarship is that I had to go back to Brazil and teach for at least as many years as I spent doing my PhD, which I did that several more years. I spent seven more years in academia as a professor at the university in Brasilia, and then after that, that's when I came back to the US to work for Pictutel.

But it was good that I had my foot in both. But when I was a professor full-time teaching students, I supervised many students doing, for example, master teases on data compression and signal processing. And I always pushed on then the idea, look, try to really understand the math behind this, but also try to make it really work because it is cooler when we build things that actually work. And I always used to tell them, I was always a researcher, but I always see myself first as an engineer and second as a researcher. I really like to build things. Yeah, that sounds amazing. I think that's a great, I would say, attitude to have.

I've previously worked in a research lab because I'm an engineer, I'm not a researcher. And I know that that was something that we collaborated so closely with our scientists. to have. I've previously worked in a research lab, because I'm an engineer, I'm not a researcher. And I know that that was something that we collaborated so closely with our scientists. And one of the things that we would do, I was in consumer internet, but the idea was, you know, go talk to the product teams and say, you know, give us like, you know, 1% of your traffic, let's try this out, let's get some feedback, and then determine where the future of this idea might go. That's really interesting that you mentioned that, Eko. Did you find in, you know, academic,

you're a researcher, you know, at heart, and you're an engineer at heart as well. What kind of skills did you feel that were both transferable between being in academia, being a professor, and then later on being a VP of research at PictureTel? And also, you know, what kind of differences did you see? I think that in my case, the differences, I try to minimize them because as I said before, I always motivated people to try to study this deeply, try to understand how things

are doing. If you're building an adaptive filter and you're using this LMS algorithm to adapt to the coefficients of the filter, whatever it is, try to understand why it works and how it does what it does. Try to understand what is a fast Fourier transform. What does it mean to represent a signal in frequency? So try to have a little bit of mathematical intuition, but don't build this intuition only out of math, because more often than not, when you try to apply that to the real world, then you have real data, real signals, you will see things that say, hmm, that's a bit different. Oh, okay, maybe I should have

another approach here. Or maybe now that I saw the signals and they behave this particular way, that gives me some ideas on maybe how to design a different kind of filter. I try, for example, an algorithm that doesn't quite work. And then I look in practice, the signals, oh, maybe it didn't work because of that. So maybe if I had a new mathematical function. So going back and forth between the math and intuition in the real world and the real signals, I try to do both with a little more emphasis on the theoretical side in academia,

a little more emphasis on the applied side in industry, but always try to have your foot in both. Got it. Yeah, you know, I was listening to one of your older interviews and you were talking about building products while being deeply engaged with your end consumer. Today, we often call that engagement working with say design partners. You have, you identify a potential customer and you kind of almost build a product in conjunction with them, right, as they're providing feedback. But it sounds like you were a pioneer in that paradigm.

Can you talk us through that process a little bit and how and why did you see value in that? Yeah, even in the early days, for example, at PictureTel, I remember before the recording, I was talking to you about echo cancellation technologies, which are being used here so that your voice doesn't come back through my microphone back to you and produces echo, right? And we had that kind of technology since those days 40 years ago in our equipment, but there was one customer. The stuff was always generating echo, the equipment was always generating echo. so we actually visited that customer in Germany and we saw that their main conference room where they put the equipment had all

these walls with concrete walls and glass. So from an acoustic perspective, there were so many reflections that our echo canceler couldn't handle so much sound reflections. So we told them, oh look, there are two solutions here. We're going to improve our algorithm, but please put some drapes in your walls because that will help the acoustic characteristics. So basically that is a simple story that says whatever model we have in our intuition when we design, it never really predicts all the possible cases in the real world, right?

So it's important to work with customers because you always find some cases that say, oh, the model doesn't cover that. And then you go back to the model and try to make your thing more robust. That's a very, very fascinating story. I love the drapes analogy as well, because one of the first few people that I was working with while even launching this podcast gave me a tip and said, you know, when you're recording, if you can record in a closet, because the clothes will absorb some of the, you know, the ambient noise

and you actually get a much clearer recording. So I find that amusing and, you know, I resonated with that. So Rico, then picture tell happened. How did Microsoft happen? Oh, Microsoft was very interesting. The way I got to Microsoft is that one day I was just thinking about it and I was looking at Microsoft research and I realized that it was beginning to grow fast And I realized that it was beginning to grow fast at the time Nathan Merveld was the CTO. And I said, wow, this is interesting, but they don't do much in media. But I think they care about media, but the research division is not doing much about media and signal processing.

So one Sunday afternoon, I just said, oh, what the heck? So one Sunday afternoon I just said, oh, what the heck? I pick up my brain and I write an email to the CTO of Microsoft saying, look, it's wonderful that you're building a research organization. Don't you want to build a team in media and signal processing? Maybe, if you want, maybe I could help you build that. So I showed to my wife first and she said, you're going to send an email directly to the CTO of Microsoft, are you crazy? He's not going to pay any attention to you and say, well, okay, it doesn't hurt. So I'll just

send and see what happens. Comes Monday afternoon, I get an email from Suzanne from the recruiting department of Microsoft and say, Hey, Rico, I'm Suzanne, I'm coordinating your interview trip. And it's like, what? 24 hours later. And then I realized later what happened when I sent my email to CTO, Nathan Merville, he got my email and said, that sounds interesting. We actually want to do that, but who's this guy?

And he sends my email to Rick Rashid and Rick Rashid, who was the head of research at the time said, I don't know this guy. And then he says to the XD Huang, XD, my good friend XD today, he is the CTO of Zoom. And for many, many years, he was the head of speech research at Microsoft. And XD says to Rick, Oh no, yeah, Rico is a good guy in signal processing. Bring him here. Let's talk to him. And it just happened everybody was in the office.

That path was very quick. And then the same day, yeah, okay, bring him in. And then I came here. My first visit to Seattle was to interview. And I loved the place. And then I said, oh, I said to my wife, I called her, honey, you've got to come here with me. This is a nice place.

And then the interview went well, but it's funny. I just sent an email to the CTO and that got me an interview. I love it. I love it. I recently heard somebody saying, which I think has been said in many ways, but you really miss all the shots that you don't take. So, you know, what's the harm in asking, right? Exactly.

You miss the shots you don't take. So, you know, what's the harm in asking, right? Exactly. You miss the shots you don't take. That's amazing. If you're enjoying this episode, please subscribe and leave us a review on your favorite platform. You had a really long career at Microsoft, possibly playing many different roles and maybe even exploring different areas. I mean, you started with signal processing. What else did you do at your career there in Microsoft Research? I started with the signal processing team and then we started doing lots of things. Part of my team was doing machine learning, for example, with John Platt, a wonderful researcher, very accomplished researcher. Today he is a Google Fellow. He was doing wonderful new ideas on machine learning. He invented this algorithm called sequential minimal optimization,

which was very efficient to classify things. So quickly Outlook started the email clients and email systems started using John's technology to classify junk mail, because his algorithm was very efficient to filter what is junk, what is not junk. We did lots of contribution to Windows Media. At that time, Windows had a media division, so we created a new audio format called WMA. We helped design the VC-1 video format

when Microsoft was beginning to get into media, and that was a lot of fun. And a few years later, we actually built a new video conferencing system called Microsoft Roundtable, which was the first hardware product from Microsoft that had a sticker in it saying Microsoft Office. So it was a great collaboration between the product team that really decided to bet on us.

We had built a prototype in research and they said, hey, that sounds cool. Let's make that into a product. And it was the first commercial product for video conferencing with a 360 degree view. I don't know if you ever used Roundtable. And then later we licensed it to Polycom. Polycom was then acquired by Plantronics and Plantronics acquired by Poly.

Now, I think, is their new name. In any ways, from the signal processing to building this prototype of the 360-degree video conferencing so you could see everybody around a conference table, it was very nice. We had to put lots of advances in signal processing, audio processing, computer vision, the electronics of how to capture a 360 degrees thanks to Ross Cutler, who was our key architect, who's still at Microsoft. So, and it was great to be in a company

that gave us an opportunity to do all of that. Indeed. And I have, in fact, to use it, maybe white labeled as Polycom. I have used the 360. Oh, cool. And I was absolutely gobsmacked when I remember going into that room for the first time. I was like, this is so cool. It's really wonderful to hear all the thought, all the hard work that went behind making that happen. I know that you also had a deep interest in accessibility, Rekul. I'm wondering if that was your next sort of chapter at Microsoft Research, building applications for differently abled individuals. I'm wondering if you could talk a little bit more about that work.

individuals. I'm wondering if you could talk a little bit more about that work. That is interesting. I was lucky to have that opportunity because I had a signal processing group. Then I went more a little bit into management. I became an assistant director for the Redmond Lab. Then I became the lab director for the Microsoft Research in Redmond. That was a great experience. It forced me to learn a lot about other areas and how to make decisions and how much to invest in each area and so forth. So that was wonderful. And then at some point Rick Rashid gave me the opportunity to be the chief scientist and maybe have an influence in all the labs across the world, not just Redmond. And at that time, Peter Lee joined us.

And I just saw out of a coincidence that yesterday, in your ByteCast, you published a ByteCast with Peter Lee. And then Peter Lee became the head of MSR Redmond. And then after a while, Rick Rashid retired. Peter Lee became the head of Microsoft Research worldwide. And then as chief scientist, I was reporting to Peter. And at some point, I think back in 2015 or so, around 2015, Peter said, Rico, you, you, you have some interest in accessibility.

What about if we build a group like that? I can put some resources for you. And I say, great. And then we created an accessibility team. And that was a wonderful experience because I came back to being a manager of a team, doing projects in a particular area. And we had to learn a lot about accessibility and it's such a large area. And it's a dimension of diversity that is super important.

Diversity and inclusion to me is very important. I do come from a minority, right? My background is being Hispanic. So I do believe in giving everybody opportunities. And also in the case of disabilities, how can technology help people overcome or compensate for that disability? And then we had a couple of projects that really worked well. people overcome or compensate for that disability. And then we had a couple of projects that really worked well and that was a great opportunity.

And Microsoft really cares a lot about accessibility. Yeah, no, I think using technology to just make the world more inclusive is fascinating, right? I mean, I think technology has really changed and for many of us has been, you know, it's probably the most radical, most amazing inventions in our lifetime. And I think put it to use such that it is actually accessible to many, many, many more people who aren't otherwise thought of in terms of the regular, you know, run of the

mill products that we make is really amazing. It feels like that's the right use of our time and energy as well. Any specific products that you would want to talk about, Rico, that you've had particular, like maybe either, like we were talking about earlier, where you designed it, co-designed it with somebody who had a lot of input and shaped the direction in which it went? Yeah. Thanks for asking, Rashmi.

We actually were a small team, so we couldn't do too many things. So we picked up two core projects. One of them, it became a feature of Windows, of starting in Windows 10, was iControl. Have you heard of iControl? I have. Actually, I also heard a bit of it in your previous conversations,

and I did a little bit more research about it, but I would love to hear more. Yeah, the idea of eye control is that if you're paralyzed, if you cannot move your arms, for example, or if you have difficulty speaking, for example, a person who has ALS or some kind of paralysis, how can you use a computer? In the case of ALS, for example, it affects all the muscles that go through the spine, but for some reason, it does not affect the muscles that control your eye movements. So your eyes still move. So using a camera and some little bit of computer vision, you can actually track the eye movements. And instead of controlling, let's say, the mouse cursor with your hands,

you can control the mouse cursor with your eyes. And once you can control the cursor with your eyes, you can have an actual interface that is not like a regular cursor or something a little bit different, kind of a mixture of a mouse interface and a touch interface, but all of those, instead of being controlled by your finger or by a mouse, they are controlled by a little dot that you control with your eyes. But then that dot can click on boxes, it can click on on-screen keyboards, and you can actually do things in the computer, even though you cannot speak and you cannot move.

So that was a great project and at the end it worked very well. And it became a feature of Windows. That's incredible, Rico. Cause I mean, I, I worked in an area where oftentimes, uh, you know, the eye movement was correlated to user interest. And so of course we were thinking about, you know, how do you show relevant content or how do you show the right ads? I think that it would actually be the most natural way

for somebody to engage even with your product is quite incredible. You're absolutely right. The first uses of eye tracking in computing were exactly as you said, to figure out where is the person looking at and that's an education of interest. But then we can flip that to say, ah, but if needed, you can use that eye interface to actually allow the person to control the computer with their eyes. And that was fun, a lot of fun, lots of technical challenges. But at the end, it worked very well thanks to the wonderful engineers and the team and researchers.

Indeed. Thank you for sharing that story. Very inspirational and very heartwarming as well. I want to go back to something you said earlier, Rico, because you started in the signal processing group and very much deeply embedded in research. And then you took on a management position. And so I'm guessing a long career in research at Microsoft

means that you have all of these varied roles. What was your sort of experience? I mean, when you kind of get into a more of a management role, you don't spend as much time doing the hands-on research, I'm guessing. How did you sort of navigate that? What did you find that was gratifying in that process of like, you know, growing teams? For sure, it was very much gratifying, but I was lucky to be in a company like Microsoft who believes in research all the way from Bill Gates. In fact, I was lucky to have had many meetings with Bill Gates discussing research priorities, and he was always super supportive of research. So a company that is willing to invest in research that we don't know if it's

going to work or not, if we knew it wasn't research, if it is research, it's because you actually don't know where it's going to end. And then I was able to learn lots of things about how to manage innovation. How do you motivate people? For example, telling them, don't do anything 100% of the time, go do some other things, but also allow people to explore things with high risk. I still remember at some point the team, for example, the team doing natural language processing, machine learning,

they said, oh Rico, we're going to start working in these neural networks, but they're going to be very deep. They're not going to have three layers. They're going to have 300 layers or a thousand layers. And we think they can do a lot of good things because now the decision regions in this multi-dimensional manifold is going to probably be able to model very complex PDFs and do wonderful stuff. And then I told them, okay, you got me at manifold. That sounds good, go do it. And then in fact, they had good contributions to what then became deep neural networks,

which today are in the core of a lot of AI technologies. So the idea of doing some different things and also, as I mentioned before, push research, but also push opportunities for that research to go into a product as quickly as possible. And at Microsoft, we had the opportunity because the company is so diverse, has so many different kinds of products. So we had so many possible avenues of impact and that was always very motivating. Yeah, and I think the point you make about taking risks,

I think applicable across the industry, right? Whether it's research or whether you're in an engineering organization, for example, like myself, I think the need for us to constantly maybe think a little bit outside the box, of course, we have deliverables, we have product features, we have customer needs, but I think innovation really comes when you kind of push that boundary just a little bit. And to have a leader or a mentor who's willing to sort of back you on that is actually a real gift. Yeah, and you have to find the right balance,

for example. You cannot have a research that is super applied and super tuned to what the customer wants all the time because the customer doesn't necessarily know what's possible. Right? So you have to invent some free things. You have to have a space where you have free thinking. I'm not thinking yet about how that would be useful to people. I just want to follow my multidimensional manifolds or I want to find an algorithm that's going to run faster to do this thing, because suddenly at some point,

you will find a place where that can actually have an impact. The trick is that don't assume that you just do the research and somebody else is going to do the engineering. Do some of the engineering yourself. And that's what I always try to tell my engineering. Do some of the engineering yourself. And that's what I always try to tell my researchers, do some of the engineering yourself. And we always had engineers who would team up closely with the researchers work together

because that creates a feedback loop where as you're trying to build things, you go back to the research and say, look, these things work, these things don't work and you start thinking differently. So you start thinking the theory a little bit differently because of what you learned on the implementation. But now as you're thinking differently, some new idea comes up, Hey, maybe I

could have this new feature because of this new thing that I thought more theoretically, so going back and forth between the two is really great. And Microsoft research has always had that tradition and that was really a good experience. Yeah, no, that's, that's excellent, excellent advice. I think definitely something that we could all apply in our day jobs as well. Rico, you also spoke about diversity a few times, right? Building a diverse team, catering to a diverse set of customers

to actually be able to build products for different people. And I know that you've also spent some time sponsoring groups of individuals, whether that is gender diversity or other types of diversity to encourage that within your organization. Would you care to talk a little bit more about, I mean, of course, I know you already mentioned

that coming from minority group yourself, that's very clear to you and I completely understand that. But I would love to understand, why did you feel like you doing work in this space was important? I think it is because I truly believe that the more diverse you have, the more diversity you have in your thinking,

the more innovation is going to come. For example, I like to quote an interesting statistic. If you look at computing technology in terms of invention, let's put it in terms of patents, for example. If you look at patents in computing technology, and if you put the patents in two groups, one group where all the inventors are men, and the other group where there's at least one woman among the inventors of that patent,

the patents where you have at least one woman in the team are significantly more referenced than the patents that are built exclusively by men. And you say, why is that? But it's true. It's just numbers and it's a big enough difference. It's not a small difference. It's a big enough difference.

And it comes from the diversity because people with different backgrounds, be it because of ethnic background or gender diversity or ability diversity, they can bring different views that allow you to explore more the ideas. For example, in our team, the accessibility team, one of the key leaders is a blind person and the way he uses technology. We learned, for example, when he was listening to transcripts, he would listen to things at 2x speed or 2.5x speed, because he was able to do that.

We're not. If you try to play an audio clip at 2x speed, you're going to lose track of what the person is saying. But he would not, because then he would listen to his emails very quickly, much more quickly than we can. And then we want to make sure that the technology to speed up audio works well at 2x and 2.5x and you're not going to think about that problem unless you have the use of somebody who really wants to do that every day, every time. So that diversity is super important. Let me add another dimension because sometimes people confuse DEI as giving unfair advantages to minorities.

And that's not it at all. You just want to make sure that the doors are open to everybody. You want to make sure that if there are women among your engineers and they typically tend not to be heard, force the system to give them a voice, ask them questions, and you will see that they have good ideas as well. So it's all about giving more opportunities for people who tend to be, their voices be suppressed because of they are from a minority. And once you give them opportunities, let them shine and they will shine. who tend to be, their voices be suppressed because of they are from a minority.

And once you give them opportunities, let them shine and they will shine. And they will shine as much as anybody else. I love it. Very sound and practical advice. Thank you for sharing the examples as well, Rico. Because I think again, it really articulated the points that you were making and the advantages that we get from actually building a diverse team, something that we can all sort of apply in our real day jobs and our lives as well. Enrico, I hear that you are now retired.

What does retirement look like for you? That's fun. Yes, a couple of years ago, I retired and the timing was by design aligned with my twin grandsons who are now turning four. So I had a chance and I'm lucky enough that my son and his wife, they live nearby so I can drive 25 minutes and be with them. So I do that a few times a week. So I have a chance to play with them and that's just wonderful. And then I still use a little bit of time for technical consulting and other things. But it's a different life. It's a different stage of life.

I've done a lot of things now. What I want to do the most is just enjoy life a little more, spend time with my grandkids, but still do a few technical things here and there. That sounds absolutely delightful. So glad that you have the opportunity to be so close to them. I think that's really amazing. Yes, it would be. For our final bite, Rico, what are you most excited about

what you're seeing in the world of technology? I'm going to sound like everybody else. We've seen in the past few years how AI is having a tremendous impact, right? But in fact, I, like many other people, was surprised. The impact today, we knew there was that path. As I mentioned, the example of one of my teams

working in deep neural networks, that was 15 years ago, and we knew that that would make an impact. But now when you couple those with the large language models, with transformer technologies and other things like that, and you build these new AI systems, it's amazing what they can do. Just last week, I was in San Francisco with my wife and it said, oh, let's take a Waymo taxi. And the idea that a taxi comes by itself with no driver and takes you from A to B. And there are some hiccups in the middle of the way.

There's this car that stopped in the middle of the road so that the taxi had to go around and you did it all of that beautifully. There is a person who stepped appearing that they would cross the road. The thing stopped and the person crossed the road. I was really surprised. I mean, the technology really works well. However, if you look at what's behind, we don't fully understand AI. So I keep reading about it because I'm really curious to see when

So I keep reading about it because I'm really curious to see when my friends and colleagues in AI will take the new inventions that actually start explaining exactly what the AI does. Because parts of it are these neural networks, which are kind of black boxes. The network converges, the weight converges, but then converge to what? It's modeling these very high dimensional probability spaces, but how is it doing that? It's not a hundred percent clear. And the more we try to understand how they actually work or maybe come up with some new invention. So at some point we may end up having AI systems that really work much more reliably with less hallucination, with more precise answers.

And at that point, the world is going to be a place. It's just like internet and digital media. We cannot live without those things. We certainly get into a point that we cannot live without AI. If you are a professional in any profession, you're going to use AI in some way. So it's a wonderful world. I'm not going to see the huge impact maybe because that's going to come after my time, but I'm already seeing some of it. And I was like, wow, am I lucky to be living in these years where we see artificial intelligence doing so much.

So it's really wonderful. I love, love the optimism. Rico, thank you for sharing your rich and accomplished career journey with us and for taking the time to speak with us at ACM Bytecast. Thank you, Rashmi, for the invitation and for your wonderful questions. I really enjoyed our conversation. Likewise. and for your wonderful questions, I really enjoyed our conversation. Likewise.

ACM Bytecast is a production of the Association for Computing Machinery's Practitioners Board. To learn more about ACM and its activities, visit acm.org. For more information about this and other episodes, please visit our website at learning.acm.org slash bitecast. That's learning.acm.org slash b-y-t-e-c-a-s-t.