Microsoft Research Podcast - 044 - The Future is Fusion with Asta Roseway

Episode Date: October 3, 2018

Asta Roseway has a formal title. It’s Principal Research Designer in the HCI group at Microsoft Research. But she’s also been described as a conductor, an alchemist, a millennial in a Gen-Xer’s body and, in her own words, a fusionist. What’s a fusionist, you might ask? Well, you’re about to find out. On today’s podcast, Asta gives an inside look at one of the most unconventional labs at Microsoft Research. Located at the intersection of science, technology and art, it’s a lab that insists that technology, like art, should push boundaries, tell stories and feed our souls. Get ready for the unexpected because when Asta asks “what if?” you’re likely to find yourself immersed in a world of responsive clothing, smart tattoos, talking plants and even environmentally sensitive… makeup!

I crossed paths with somebody who worked at Microsoft Research who was working in virtual reality. And I didn't even know what that was. I didn't even know what email was. And he invited me to come in and see this world. And I remember they had me log in as an avatar and run across the world and be interviewed by other avatars. This was at Microsoft Research in 1997.

And they were like, are you interested in building worlds with us? And I was like, I don't even understand what is happening right now, but I want to. Yes, of course. You're listening to the Microsoft Research Podcast, a show that brings you closer to the cutting edge of technology research and the scientists behind it. I'm your host, Gretchen Huizenga. Asta Roseway has a formal title.

It's Principal Research Designer in the HCI Group at Microsoft Research. But she's also been described as a conductor, an alchemist, a millennial in a Gen Xer's body, and in her own words, a fusionist. What's a fusionist, you might ask? Well, you're about to find out. On today's podcast, Asta gives us an inside look at one of the most unconventional labs at Microsoft Research. Located at the intersection of science, technology, and art, it's a lab that insists that

technology, like art, should push boundaries, tell stories, and feed our souls. Get ready for the unexpected, because when Asta asks what if, you're likely to find yourself immersed in a world of responsive clothing, smart tattoos, talking plants, and even environmentally sensitive makeup. That and much more on this episode of the Microsoft Research Podcast. Asta Roseway, welcome to the podcast. Thank you. Thank you for having me. I'm excited to have you here today. I am too.

And I actually don't like using the word excited. Oh. It's the new amazing. Everything's exciting. Nothing's exciting. Exactly. I really am excited. Yay. So your official title is Principal Research Designer in the HCI group at Microsoft Research,

but you also call yourself a fusionist. So what is a fusionist? What does a fusionist do? And what gets a fusionist up in the morning? All right. So I have always been a bit of an outlier my whole life, and I've never been very comfortable with formal labels. And I have found that some of the roles and the paths that I've been on have been so unconventional that I feel like I needed a title to actually help me explain it to myself. Does that make sense? Totally. And so I find myself working between disciplines and fields. I'm coming at it from a sort of an art brain, but I've been in computer science for almost 15 years. And I'm like, how do I pull all of these things together? And I'm like, fusion,

I fuse things together. That is my process. So I wanted something where I can just say in a quick elevator pitch, hey, I'm a fusionist. That's what I do. And let them suffer with it. Right? Like, let them figure it out. Does anyone ever ask you, what's a fusionist? Yeah, they do all the time. You got that down. But the thing is, is they kind of get it. I'm like, it's art, it's tech, it's science. I'm pulling all of these aspects together to sort of push this next wave. And they go, oh, okay, yeah. And I've had a lot of young folks technology, and science, which is a super interesting intersection as far as holistic picture of who we are and how we function in the world. I feel like without that balance, we can't approach the next wave of problems being 100%

mindful about it. And if you only paint a picture with a certain palette of colors, you're just missing out on the rest of the spectrum that's there to be used. So for me, it's like, this is an opportunity to take on the really hard things, climate change, environmental impacts that we have, even humanities, and how we can integrate that other part of ourselves to actually start tackling these hard problems. How are you finding that to be received by this very computational, STEM-oriented, math-privileged culture that you're in? Well, to me, I'm kind of like a weeble wobble, okay? The harder you push me, the harder I push back.

And I love the challenge. And I actually find inspiration in inspiring my colleagues and people around me to also open up that part of their brain, that side of themselves that's like, oh, you know, if you take it from this perspective, it's so different. And all of a sudden, the traditional obstacles kind of melt away because you're kind of coming at it from a different angle. So to me, I take great pleasure in helping my colleagues join in in that conversation and encouraging them to cross-collaborate and try it from a different

angle. So I'm okay with that. Now I've got Weebles Wobble, but they don't fall down stuck in my head. That's right. The song actually probably dates me. Are you seeing a shift in receptivity to this kind of thinking? Yeah, I am. I would actually say that in the last five years, things have really shifted in a way that has enabled me to continue this work, to grow this platform. I've also seen it in the waves of young people and interns coming in. They kind of come at it from the same mindset. So I feel like I'm finding my kindred spirits. It just feels so reinforcing, you know, just, oh, this feels good. They get it and they want it and they want to know how to do it. So I feel like we're kind of inspiring each other, which is fascinating. And I see more and more of them coming in as the years roll by.

What about the not new people, but the old school, you know? Hey, look, everyone has their place and their function and their purpose. And that is totally fine. For me, I have my own way of coming into a conversation and a project. And I encourage them to get ready for this next generation, because this next generation is a different breed. It's an entirely different breed. So if they come to me, my colleagues say, hey, I've got this hybrid intern, send them to me. I'll talk with them. Let's figure out a way for them to kind of integrate. sort of what you'd call unconventional explorations of human-computer interactions. So what's different about how you approach technology research? What's different about when you ask, what if? Okay, I'll use this analogy. For me, the what if is like me jumping out of an airplane. I have no idea where I'm going to land. I just know I want to get out of the airplane. I want to jump and I want to give it a shot. And that means

that, you know, in traditional research, you know, you work on sort of these iterations of learnings and understandings. For me, it's been more about, okay, I see this trend over here and then this thing is happening over here, but these things have never met in the middle. So what if they do? What happens then? And so that's the driver. I just feel like I want to be a magnet and just kind of bring both sides together. You established Microsoft's first ever artist-in-residence program called Studio 99. And I have to ask, why do we need artists in high-tech research? Okay, so first of all, let's pull this back. Why do we need art that feeds our soul,

right? Art has been with us since the beginning of time. It tells stories and narratives and creates memories and it fuels that side of us. It lives in each one of us. And so why wouldn't it be a part of our tech and our research culture? It has to be. It's a part of that dance that we have to have with that other side of ourselves so that we can ask these questions like, well, how does this impact our society? How does this impact the individual? How does this technology impact our privacy? And art has this way of just shoving it in our faces and really asking the hard questions.

So what kinds of things are happening in the Artist-in-Residence program here? In the past, we did something around, if you think of internet of things and how our environment is sort of moving towards this world where things are talking to each other, whether it's devices like my car to my refrigerator, We wanted to go way beyond that. So we wanted to do human to plants, human to interspecies. And why not? I mean, artists are the ones that are like, of course we should be able to talk to plants and animals. Isn't that what technology should facilitate, right? Wouldn't that enable us to have healthier conversations with our environment? So we did a project called Project Florence,

which was the brainchild of Helena Steiner, who had a background in plant signaling and design. And her approach was, I want to make plants respond to us. I want what we say to impact them in some way. We have done some explorations with computer generated art and sort of playing the way with algorithms can sort of manifest the art. My very favorite is we had a young woman come in with a background in biofabrication. And that is sort of the emergence of new materials from living matter. And so she had a passion around mushrooms and mycelium cultures. And she actually grew her wedding dress out of mycelium. And I was like, yeah, let's do this.

Her name is Erin Smith. You should look her up. She's pretty sweet. So she came in and worked with us on something called Growables. Like, why wouldn't the next wave of the things on our bodies be alive? It was so out there. It was like something out of a David Cronenberg film. But I loved it because it made people just stop for a minute and go, wait a second.

So wearables don't have to be plastic. They don't have to be driven by a battery. They can be alive. They can be on us. I loved it. How did it work out? Did she actually grow a mushroom wedding dress, a fungus dress? Well, she grew, so she, sorry, she grew the dress on her own time.

Like this was before she came and worked with us, but she ended up actually growing these amulets into Petri dishes and we displayed them. And for me, it was so great to see my colleagues, my peers come and look at this stuff and go, oh my gosh, like I hadn't even thought of this. That's why we have these kind of programs. So as soon as you said that she was growing mushrooms to do a dress, I thought, well, she'll grow them and then pick them and make the dress somewhat like Lady Gaga's meat dress. But you're talking about it being a living thing. Yeah. So there's two facets to that. One is her approach was she wanted to create a material from the mycelium itself that would then be decomposed, right? Because wedding dresses are made of polyester and we are just beginning to understand on the micro level,

even on the chemistry level. And these are sort of the next wave of the intelligence and how it permeates into our everyday lives. It's pretty cool. Asta, we can't do justice to the visual nature of the work you do on a podcast, although maybe your group could get to work on that too. Right. Come on in. All kinds of things are going through my head right now. But I'll encourage our listeners to hit up the website and look you up because it's amazing stuff.

Thank you. But let's dig into a few of the projects that typify your work, starting with what's been called your breakthrough design. It was Microsoft's first fashion technology piece. It won best in show at the International Symposium of Wearable Computers. I didn't even know there was one. And was featured at CES, South by Southwest, and New York Fashion Week. Tell us about the printing dress. The printing dress. I love this. You know, I had reached a place in my career

where I had been working as a sort of a user experience designer. And I felt like everything I did was stuck in a square. And I kind of had this weird crisis of like, how do I get out of this box? And so I remember I was doing just some internet browsing and I came across this piece from Phillips Research. It was this mood dress. It was so beautiful and inspiring. There was something deep in me that was like, yes. And I couldn't even articulate to anyone what was going on with that. I was just like, this is great. I want more of this.

How do I do this? And so I decided, you know, I have a background in design. I've done some sewing in the past, like, give it a shot. I partnered up with a colleague of mine, Sheridan Martin-Small, who had a background in engineering. And we got together and we just thought, let's just do this. What's the worst that can happen? We just pulled together extra time. We built this dress. We submitted it to this symposium. And all I wanted to do was just manifest. That was the goal. It wasn't even beyond that. It was like, can we just take something from 2D to 3D? And that's how it started. And it's been a rabbit hole ever since. And I'm grateful for it. So grateful. Well, tell me about it. Because when you say the printing dress, and I've seen some pictures of it in an article, is it a printing? Or is it printed? So it's a dress made entirely of paper. And that's by design. And it looks like something

out of the 1900s. So it has a very retro feel to it, very Victorian. It's got a capacitive keyboard corset that's kind of built into the bodice. And it enables people to actually type and tweet their feelings and thoughts. And as they do, we had a projector set up underneath it that would actually display all the thoughts and animate as memories. So the goal of the dress was to speak to the notion of what happens when accountability and privacy and wearable technology and social networks, they all sort of meet in the middle. Like if you talk trash, you wear trash. You know, that was the message. It's kind of relevant today, don't you think? More so now so now than ever you know every day it gets more and more relevant right it was really supposed to be a thought piece but also it was my wish to

actually bring something like that to microsoft like i wanted a dress to exist inside the building why not another wearable project you did yeah was Lightwear. And this was an exploration in wearable light therapy. Tell us about that project. What was the rationale behind itie Profita. And she and I were really interested in this intersection between wellness and wearables and where is the sweet spot. And so we were both pretty cognizant of the fact that, you know, seasonal affective disorder is really a thing and it really does impact millions of people. And we wanted to explore the light that was coming out of these light boxes and just get rid of the box. Again, my issue was squares. And I was like, can we just get the light out of here and put it in something else? And that began to kind of instantiate as scarves, glasses, or hats. And we used the same light spectrum from the box

into the garments, but we just had a lower intensity. So a standard light box you might sit in front of for 30 minutes because it has a very high intensity. For the scarf, you might wrap it around yourself and you'll get the light from the fiber optics inside the scarf, but at a lower intensity. So you might wear it for two hours or an hour on the bus. But we wanted something that frees you and untethers you from the box. And you can sort of transcend that feeling of, oh, I'm stuck behind a box. So you want to use fashion as sort of this way to escalate and make people feel good while they're getting healed. What a concept. So what happened? So we worked on several instantiations

of it. We did user studies. We got a lot of great feedback from people who actually really loved it. And we were able to sort of do a publication in Pervasive Health and it actually got best paper. So that was amazing. And then from there, you know, it's one of those kind of projects where you might have an expectation for how it should evolve and be received. And yet, it's all about the timing. Are people collectively ready to grab onto a concept like this? I don't know. And it didn't seem so at the time, but I promise you, you will see things like this in the future. I believe it.

Was it the people who suffered from seasonal affective disorder that weren't ready to receive it or the people who were looking at the people wearing a light up scarf? It might have been both. It's really hard to say. You know, that's the challenging thing about trying to ride the wave a little bit ahead of the curve. Like, you just don't know how things will be received. Right. And you don't know if it was, you know, are people just feeling a little bit like that's too much attention drawn on themselves? Right. Because it's not like you see people wearing light around themselves ever. So there's a lot of things to consider with that. But it was a fascinating project. One of the coolest things you're working on is called DuoSkin or Smart Tattoos, which is literally wearable technology that becomes part of your skin.

Tell us about Smart Tattoos, including the history with MIT Media Lab and the very recent mini hackathon you hosted called Hack-A-Tat. So DuoSkin was started at Microsoft Research with my intern Cindy Cao from the Media Lab. It was really sort of an interesting approach to the e-tattoos that we had seen sort of surfacing. And e-tattoos are more medicinal, that are sort of very, very high-end. They're built in laboratories and they might be sort of something that you put on a patient to monitor a pulse. But these kind of technologies are just impossible to get a hold of. And they're very black box in how they're made. So we thought, let's just take that notion of smart, temporary, transitory technologies. And we fuse that together, I'm using that word, with jewelry and accessories

and all of the jewelry tattoos that were starting to come onto the market at the time. And we think back specifically to Beyonce's launch of her jewelry tattoos, and we were completely inspired. So we were like, how do you take something beautiful like this and give it purpose and functionality? And that was the origin of Duo Skin and how that came about. And since then, at Microsoft Research, we've been actually working on the next generation of materials, ways that we can enable people to fabricate and build their own. And so we're looking into ways we can create kits. And that was what sort of brought about the hackathon and the hackitat, where we invited people from across the company to come and build these tattoos, play with them, give

us feedback, and sort of help us along the journey. Because we can't think of every scenario. And so the feedback was just wonderful. It was completely eye-opening. What kinds of people showed up at the Hackitat? I would say millennials, again with the young people. You know, they were just super excited to just think outside of the box. Controllers don't have to be standard controls. I can make anything interactive. I can put a sticker on my water bottle and have it do something

for me. Or I can put something on my clothes and all of a sudden it has a purpose. So I think they loved the creative expressivity of it, but also just the freedom of thinking outside of that norm. So give me an example of what, if I put a smart tattoo on my arm, what's my interface? Do I work the tattoo from my device or do I work my device from the tattoo or is there no device or what? So the tattoo is connected to a small microprocessor that has a small battery. And then we use a Bluetooth radio to connect to the devices. So you can connect it to anything with Bluetooth. And that's where the creativity and the scenarios come in, right?

And that's why we wanted to get it into people's hands to let them dream big and think of the things they wanted to do with it. Because we can't think of it all. We talked about this a little, but I want to go a little deeper. Thinking about the Internet of Things. And obviously, when we think of it, we think of being able to communicate with our devices or having our devices communicate with flora, not fauna. Tell us a little bit more about that out there thinking. And are there any other sort of Internet of Things ideas percolating that you're allowed to talk to us about? Yeah. You know, again, like part of my secret agenda is I'm always excited to make people feel slightly uncomfortable and nervous about a new sort of story that's unfolding in front of them.

And I know we had chatted about this a little earlier about storytelling and how important it is that you have the right voices in the room to actually start to tell a different kind of story about our future. And the drive and the inspiration for Florence was, what if we were able to tell a story where we were interfacing with our plants, our animals, our environment in a way that actually raises awareness about what is going on with it and what we can learn from it? Because nature is this incredible entity that is so

infinitely complex, it makes your head spin. And yet it all kind of works magically. And there's so much to emulate from that. And I want to drive these new narratives for people that don't resonate with the current one. I want people to wake up in the morning and say, oh my gosh, I had never thought the future like this. And that actually resonates with the way I feel about it. And I want to tell my story too. So that's a lot of the driving factor behind this. So we started out talking about technology and fashion in this project's part of the podcast. Let's wrap up on this section with technology and makeup. Right. Tell us about this project called Earth Tones. What is it? Why would we need it? And where might it lead us? one of my near and dear. It's one of those kind of quieter projects that kind of lives under the

radar that has been slowly growing on its own. But the idea is we put things on our body every day, you know, whether it's shampoo, it's cream, whatnot. And why wouldn't we enable these things to act as notifications, as sensors, things that would raise our awareness about what we're being exposed to on a daily basis? And UV is a perfect example. So every day we get exposed to UV levels, whether it's cloudy or not. And what if we had a way that things on our face or our hair could change color, could enable us to actually see the impact of this exposure, to understand that there's a threshold that we shouldn't go beyond? What I loved about the earth tones was it was a very earnest attempt to show the unseen in a way that could manifest in everyday habitual practices by people all over the world. And I loved the analog color-changing nature of it.

It was something that you could do without batteries, and I love that. So tell me what it is. So essentially, Earthtones was the idea of blush powders that could then turn deeper shades of color depending on how much exposure you were getting from carbon monoxide or UV rays. So it was sort of like watching your face change in pigments throughout the course of the day. And a lot of people would say, I don't want my face changing color in front of people. And then a lot of people would say, I'm okay with that, actually. It's kind of punk rock. I love it. I kind of want to like make a statement. And then there's this other part of it that is about the awareness. Like I didn't know that we were being exposed to this

much every day. How is that impacting my health? These are the things we don't think about until we're sick. So how do we give out this sort of slow awakening of things that we just do every day to something that can say, maybe it's time for you to consider how you're going to take care of yourself? Right. So what about future applications and thinkings about what we put on ourselves as sensors? I am excited about programmable materials. I'm excited about the promises that chemistry brings. I'm excited about creating intelligent layers on things that normally would be completely missed by everyday people. And that's that element of shock and surprise. When it's not powered by a battery, it's profound.

So one of my favorite aphorisms is that when scientists make a really important discovery, they don't say eureka, they say, that's strange. And by now I'm kind of like, that must be what Asta says all day, every day. But whether that's true or not, what's the strangest thing that you've discovered as a fusionist here at MSR? The strangest thing. I am going to have to go back to the mushrooms. I mean, seriously, growing wearable amulets in Petri dishes was pretty far on the spectrum for me. It was definitely considering spores from mushrooms, possibly getting out, just ways to sort of manage the life force of these things still remains the most out there thing I've ever dipped my toes into. But I have no regrets at all. Well, I will say this, having seen many of the videos on your page on the Microsoft Research website, there's a lot of out there

stuff that you've seen and been a part of. So that's pretty cool. But my threshold for out there is pretty high. So this is why if you see the other stuff I consider normal, the stuff that really makes my head noodle still remains in a petri dish. I hear you. Because I'm way over on the other side of the spectrum. Like weird is, you to labels for a minute. I think this is really important. Aside from fusionist, some people have called you a conductor. The symphony metaphor, you've referred to yourself as an alchemist.

And one of my favorites is I'm a millennial in a Gen Xer's body. Totally. First of all, unpack what you mean by this a bit and then talk about how self-conceptualization or identity might impact a person's perception, maybe their ability to carry out work in this industry. Let's start with going back to the notion that I've never been able to fit into conventional pockets.

And I've used that as a strength. And that enables me to then transcend labels. For me, it's always been, don't put me in a corner. Don't try to figure it out. Because I'm still trying to figure it out. And I laugh about the millennial thing because I have never been in this line of work to get the big things that people think are important. I live a pretty minimal life and I feel like what moves me, what drives me is to make change and I think the millennials get that. And I think they want that too. And I am not inspired by the big promotions. I'm not inspired by the big titles. I just want change.

And so I'll just do whatever it takes. I will rattle any cage to make that happen. I ask all my guests this question, and I'll ask you too. Given the breadth of what you do and the general landscape of the industry, is there anything that keeps you up at night? Oh, there's so much that keeps me up at night. There's kind of two folds to that question. The first is that feeling that change isn't happening fast enough. You push, push, push, push. You do everything you can.

Is it enough? Is it enough? Like, are we going to be able to dig ourselves out from all these things that present these challenges to us collectively? Those things keep me up at night. And you try to think to yourself today, how can I push the needle a little bit more? So that's one aspect of it. And then the other aspect is that you just hope that whatever it is that you bring and push will actually change something fundamentally in a way that transcends all your hopes, that you really change someone's life in a profound way. And maybe it's just the fact that I'm here

that is inspiring a next wave of young people to consider a path in their career. If that's the legacy I leave and it's not a product or it's not a technology, then I'll take it. So that's a perfect segue into my last question as we wrap up. You're part of a relatively small tribe here at Microsoft Research in that computer science isn't your background or your training. So what was your education path and how did you end up here at Microsoft Research?

What does it say to people who might not assume there's a place for them in high tech? Yeah, I love this. So my path was traditional design. I mean, I grew up in a family of artists and musicians. We didn't even have a traditional working desk in the house. I mean, everybody was just either making music or art. So of course, I found my way into graphic design and I went to Parsons School of Design. And my initial goal was to end up, you know, maybe in a big design company, maybe working in marketing or advertising. And it was the most bizarre thing, another rabbit hole situation where I crossed paths with somebody who worked at Microsoft Research

who was working in virtual reality. And at the time, you have to think of this, this was the mid-90s. Oh, you're kidding. No. So this was the mid-90s. He was like, we're working in virtual reality. I didn't even know what that was. I didn't even know what email was. Okay. Like I was one of the very first classes to come out with, you know, having enough experience working on a computer, making art from a computer, that I could understand that there was some appeal there. And he invited me to come in and see this world. And I remember they had me log in as an avatar and run across the world and be interviewed by other avatars. This was at Microsoft Research in 1997. And they were like, are you interested in building worlds with us? And I was like, I don't even understand what is happening right now, but I want to. Yes, of course. And it's that

desire, that passion, that excitement for these unknown spaces that has been the catalyst in my life, that has pushed me into these holes and they keep going deeper and I've never looked back. And this is why I have always considered myself to be a bit of an outlier because I end up in these situations that I can't even predict or foresee. And that's the wave I ride. So what would you say to this generation, many of whom are in undergraduate or even graduate school, that might not think, hey, high tech is a natural fit for my affinities and my abilities? show them that they can play in this space. Just by the fact that I exist and I'm here doing it, it should be a signal to them that they can too. And I take a lot of pride in showing them, hey, you know, technology and the future can look like this. It doesn't have to look like what you think it does. It can look like something you feel it does. And so I use myself as an example. I'm shameless

about that. And people, you know, ping me on LinkedIn or they send me emails and they ask me, how can I do this? And I tell them two things. First is, these kind of jobs won't initially be there when you get there. You have to do it yourself. You have to build this way of working yourself. You can't expect them to know what you need and want. You have to show them. And the second is be ready for anything because you may think that you don't have a role to play, but something can change fundamentally tomorrow that absolutely makes you critical. So don't underestimate your own skills and your value because we don't know what the future is going to ask for. Asta Roseway, our fusionist future,

visit microsoft.com slash research.