ACM ByteCast - Pat Pataranutaporn - Episode 35

Episode Date: March 27, 2023

In this episode of ACM ByteCast, Bruke Kifle hosts Pat Pataranutaporn, technologist and researcher at the Massachusetts Institute of Technology (MIT). There, he explores the intersection of synthetic virtual humans and synthetic biology, specifically at the interface between biological and digital systems. He is currently a PhD candidate at the Fluid Interfaces Group at the MIT Media Lab and a KBTG Fellow. Pat's research has been published in Nature Machine Intelligence, Nature Biotechnology, IEEE, ACM SIGCHI, ACM SIGGRAPH, ACM ISWC, ACM Augmented Humans, Royal Society of Chemistry, among others. He also serves as a reviewer and editor for IEEE and ACM publications. Pat’s published research is recognized worldwide and has been featured in the United Nations AI for Good forum, Time magazine, Forbes, National Geographic, FastCompany, The Guardian, Disruptive Innovation Festival, and more. In the interview, Pat describes how his early fascination with dinosaurs led him into the scientific realm, and later to the MIT Media Lab, where people are encouraged to think about future challenges rather than just focusing on solving current problems. He explains the research area of fluid interfaces and describes some of the innovative work his group has been doing on human-AI co-reasoning. Pat and Bruke also about the future potential of AI in education and wearable devices, as well as MIT’s recent space exploration initiative. Pat also offers his perspectives on art and innovation, identifies the exciting new directions currently holding his attention, and offers advice for young people interested in the field of computing.

This is ACM ByteCast, a podcast series from the Association for Computing Machinery, the world's largest education and scientific computing society. We talk to researchers, practitioners, and innovators who are at the intersection of computing research and practice. They share their experiences, the lessons they've learned, and their own visions for the future of computing. I am your host, Brooke Kifle. Our next guest is an innovator, futurist, and artist with an interesting love for space dinosaurs

who believes in bringing crazy ideas and moonshot thinking to create future innovation. Pat Patarnataparn is a technologist and researcher at the Massachusetts Institute of Technology, also known as MIT, where he explores the intersection of synthetic virtual humans and synthetic biology, specifically at the interface between biological and digital systems. He is currently a PhD candidate at the Fluid Interfaces Research Group at the MIT Media Lab, and a KPTG fellow, where he works with his collaborators such as NASA, IBM, UCSB, Stanford, and Harvard, amongst many others, to examine the future of human-computer integration. His interdisciplinary research has been published and recognized worldwide, and in addition to his scientific contributions,

Pat's artistic projects have been featured at many museums and exhibits around the world. Pat believes in bringing crazy ideas and moonshot thinking to create future innovation. Pat, welcome to ByteCast. Thank you for having me. Hello. So I'd love to start off with a question that I ask most of my guests. Can you tell us a bit about your personal background and maybe a couple of key experiences or inflection points, as I like to call them, within your personal and academic journey that have led you into the field of computing, but also your field of study today? Yeah, that's a great question. So as you mentioned, I'm a PhD student now at MIT Media Lab.

And one of the reasons I'm here is that when I was in high school, and maybe we should go back a little bit further than that, I grew up really liking dinosaurs. Dinosaurs were a source of inspiration for me. Like many people, I think dinosaurs have been the reason why I study science. Dinosaurs are all extinct, but they're very, very cool and have inspired many of us to think about what is out there, what is in the past and what could be in the future.

I grew up really liking dinosaurs and then I watched many science fiction as a kid. Many science fiction involved dinosaurs like Jurassic Park or there's this anime cartoon called Doraemon, which is about a robot cat that helps a child with different kinds of tasks. And in that cartoon, the Doraamon robot will bring different kinds of gadgets, like time machine, take a trip to the past and see dinosaurs, and also other kinds of gadgets that help improve human life. And what was really interesting to me is that people often associate dinosaurs with the past,

but dinosaurs to me is also a representation of the future, with new technological innovation, bring back the dinosaur, but it could also be dangerous as well. So it's like a cautionary tale of how to usually use technology for the positive application. So after that, I think that science fiction or doing something very cool with science technology would be something that I would love to do in the future, but I often think that it's only in the cartoon or in science fiction until I watched the TED talk of my current professor at MIT Media Lab, Professor Patti Maas. She showed in her, you know, one of the most famous TED talk ever that you can bring technology to the human body and create a real cyborg. And that to me was the moment that changed my life. I say that, wow, there are people that are actually doing really cool scientific work, but also, you know, very fun and very futuristic, something that I only seen in science fiction until that point. And that's how I discovered the field of human

computer interaction. And I think at that time, I was in the last year of high school, and I reached out to Patty, you know, from out of my Gmail, I wrote an email to Professor Patty Myers of MIT Media Lab. At that time, I did not know that MIT professors are, you know, the most busiest people on earth. And I reached out to her and, you know, she was so kind to write me back. And, you know, I sent her a DIY prototype that I made in my high school based on the research that she presented in her TED Talk. And that's how we know each other. And yeah, after that, I stayed in Hushwood Paddy

and then applied to her research lab and become part of her research group. And now I'm continuing that work on human-computer interaction, wearable technology, things that are very futuristic, but also grounded in real scientific research and try to see if we can really use the technology to improve human life. So yeah, to summarize it, it started off as a

scientific, no, it started off as a science fiction, no, not a scientific exploration at all. It started off as a science fiction obsession, a childhood inspiration cartoon, and then that led to real research at MIT. So that's sort of my journey. Yeah. You know, what an amazing life experience. I think many of us in tech, or at least I'll speak for myself, I was also very much interested in the field of science and technology, primarily because of, you know, growing up, watching science fiction and being fascinated by the potential of what could be. So it seems like that passion and excitement ultimately led to a real life career. So it's very exciting to see how that has ultimately manifested in the trajectory that you've chosen.

One thing I want to learn a bit more about, you seem to be a very hands-on individual. I've seen some of your creations. So what ultimately led you to decide that a career in research, or at least going down the path of graduate school was the right decision for you, as opposed to maybe exploring some of your scientific passions as an entrepreneur or as a maker? Why did you feel that, you know, graduate school was the right path for advancing your career i think your your advisor as you mentioned motivated a lot of your interest with that ted talk yes so i'm sure that's a big piece but i'm curious if maybe there are other motivators uh for your

decision oh yes so my mentor definitely is my intellectual hero it's one of the reasons that i pursue my graduate school but i think know, one thing that I really appreciate about doing research here at MIT Media Lab is that we get to sort of imagine the future. During my undergrad, I explored, you know, a startup career and think about, you know, what does it mean to, you know, be a researcher and creating a startup. And then I realized that you need to focus on solving sort of immediate problem. And that's really cool. And, you know, there are a lot of people doing that. But for me, you know, I feel like my strength is to sort of immediate problem. And that's really cool. And there are a lot of people doing that. But for me, I feel like my strength

is to sort of be more imaginative and think in the longer term, explore new possibility that haven't been explored before. And I think that is a lot of fun and a lot of impact in that. I think one of the reason that the Meetup is a really cool place is that we get to work in a very interesting and emerging field.

Sometimes we don't even have a name for that category. I think recently we started to call ourselves human-AI interaction researcher, but that's really new compared to human-computer interaction. So the field is self-evolving, and we bring more and more discipline or more and more diverse background of people to to work in our team and that's how to expand our thinking in terms of you know what that area should be like and um you know in the media we always say that we connect art science design engineering together and that usually not just lead to new answer but also new question question that we never asked before and for me that's really exciting that when you know the research not just generating answer but new question and new possibility for the future for us to think about oh wow that's beautiful i think the mit media lab does a wonderful job of doing what you described

which is imagining the future and blending a wide range of disciplines whether it be architecture and computing whether it be biology and physics. So I noticed that even in your area of research, it seems to draw on many domains, you know, from computing to biology, psychology, neuroscience. So how would you describe your area of study? You know, I think you described it as the intersection of synthetic virtual humans and synthetic biology. As someone who may be new to the field or may not have much of a background, could you describe your area of study for me? Sure. So our group is called Fluid Interface and it's not about water or liquid. It's fluid in the sense of seamless interface between human

and machine. And Professor Padimas, my professor who lead this group, has been working in this area for quite some time now. And the idea of what fluid interfaces mean also evolved over time. It started off as thinking about how human can actively receive information from machines. So not just having passive machine that wait for us to activate it, but thinking about recommendation or agent that can, you know, recommend information to us, that I think is the first wave of fluid interface, create seamless and active information agent.

And then that, you know, it's interesting, Patty had launched many startup because of that, and it has, you know, changed the world. And then Patty was kind of a little bored with the idea that having this kind of active agent behind a computer screen, so it had led to the idea of maybe can we put this kind of a little bored with the idea that having this kind of active agent behind the computer screen. So it had led to the idea of maybe can we put this kind of information agent on the human body? You know, this area of research become what we now know as a wearable technology, right? Technology that are on the human body.

People that have started like Google Glass or VR, AR glasses, you know, were part of that sort of era at the Media Lab. They call themselves, I think, the BOR, where they think about wearable technology on the human body. And then that right now is commercialized. You have smartwatches. Many companies are working on that. And recently we had some excitement about the metaverse or the virtual reality. So many of that was rooted in that second era of thinking about human-machine integration. And now I think we're in the

third wave of what we call fluid interface, which is thinking about not just providing information on the human body, but provide what we call augmentation or enhanced capability, because we are overwhelmed with data, right? We don't just want to wear more glasses or any auto-wearable devices just give us more data. We are already overwhelming with that. So how do we reimagine new devices that not just throw information at us, but rather give us the ability that we want,

like, you know, ability to pay attention, to think critically, to be more creative, going beyond information to augmentation is the new area that we are focusing on. We call it a human augmentation or rather a human to augmentation is the new area that we are focusing on. We call it human augmentation or rather human cognitive augmentation to be more specific. So that is the newest or the third wave of fluid interface research. And that's the area that particularly working on right now. There's a very clear need for the fluid know, the fluid interaction between, you know, machines, technology and humans. And I'll get a bit more into some questions around some of the limitations, but also future directions for, you know, what you described as wearables. But I'll start off, you know, I was reading a bit on some of your research. And I think one of the things that got me pretty excited was the clear practical applications of your work. I'm sure this is probably a result of

the emphasis that MIT Media Lab has on practical applications of some of the research that folks are working on. You've been involved in various projects that use humans and machines combined to improve learning, to improve well-being, to improve decision-making, to detect mental health issues. I'd love to start by learning about one of the projects, which I think is very interesting, which is human AI co-reasoning. What is it? We often think about the success of AI or we optimize the performance of AI as an entity on its own. What is this idea of human AI co-reasoning? That's a really cool question.

And I think that is one of the things that I think our group have been sort of exploring. I think in the past, we have been thinking a lot about how do we optimize AI on itself. Eric Bjornsson, who used to be at MIT, has really coined, I think it's a really cool term called the touring trap. He said that, well, touring is a really cool person and it has created this amazing field of AI, but he also had led the trap for how do we evaluate AI? And for him, it's about how do we replicate human intelligence?

If the AI can trick us into believing it's an AI, then it has shown intelligence. But that by itself is a trap in a way because you are forcing people to think about, okay, how can you use AI to replicate human intelligence? What he argued that I think is really interesting and I think is a vision that our group is exploring now is thinking about how the AI could augment human capability

rather than replicating human ability. So instead of making an AI that are like us, can we create a new kind of AI that are augmenting us or complement our intelligence? And when we talk about intelligence, we are not just talking about one way of intelligence. We think about creativity and rational thinking. Sometimes it's the opposite of one another, right?

Somehow you want to be more creative. Somehow you want to be more logical. And in our group, we work on several kinds of AI interventions that are augmenting our cognitive capability in different domains. The project that you mentioned, wearable reasoning or wearable or human AI co-reasoning, really explored the question of how can we use AI to help us become more critical in the way that we process information. You know, as you know, we're living in the way that we process information. As you know, we're living in the era where we have many misinformation,

we have many fake news, we have a lot of information that are harmful to us. Can we use AI to augment our ability to process this information and teach us critical thinking? So in that project specifically, we explore a way that we can integrate an AI that is trained to pick up logical fallacy or picking up linguistic signal that can be the indicator of misinformation and put that in wearable device. So in the future, when people going around in their life or doing works, they will have this kind of AI integrated into a wearable, we call it like a second brain, that help them process information.

So that AI can give you sort of maybe like a whisper to you, like, oh, this information that you encounter is lacking evidence to support it. Maybe you should think more or do some more research before you believe in it. And this is, you know, micro nudging, we have shown to actually improve human critical thinking in terms of you know the people tend to be able to distinguish between information that is lacking any support from evidence or from argument that have support and we also further demonstrate that you know the AI would work best to help people reason when it can explain its decision like when it can explain to

you why it make that to you why it make that decision or why it make that suggestion. So in that project, we also show that explainability and trying to have an AI that explains itself is very important for human AI co-reasoning. So not just that it needs to help human reason, it also needs to be able to provide the reason for itself. And that's sort of the area that we are working on. And right now we have expanded that to multiple domains. We are exploring that in the form of wearable, in the form of virtual agent.

Maybe the AI doesn't need to be in your glasses or in wearable device. It can maybe show up on the screen. And we're thinking about what kind of personality, what kind of appearance it should show up. Should it show up as Socrates, the Greek philosopher, come and challenge you, make you think more critically, or should it show up as maybe someone that you like or admire? The appearance of AI has also been something that we sort of explore. And thinking about if we were to have more AI in

our daily life, what are these social dimensions that we should explore? You know, the personality of AI, the appearance of AI, what kind of questions it should ask you, right? This is the kind of interdisciplinary research that we are exploring, thinking beyond how do we optimize the AI, but how does the AI interacting with the human sort of change the way that we experience our life and the way that we will

work in the future. So that's sort of the research we're working on. Very exciting. I really like this idea of nudging. I've actually seen that employed across various domains, whether it be nudging on my smartwatch telling me to take a break or to stand up or to, you know, make sure I'm, you know, walking my number of steps for the day. Do you see, so are the primary applications that you're pursuing now for misinformation or fake news detection? Do you see other application areas, whether it be in HR, in health, in, you know, it seems like the applications are very wide here. Yeah, definitely. So I think one area that I'm particularly excited about is the idea of AI for education. You know, right now we have many powerful AI models and they are really exciting and many more researchers are creating more and

more models, you know, last language model, you know, generative image model. These are very cool by itself, but thinking about how they can really improve the way that we learn is also really interesting. Professor Seymour Papert, who is a legend at the Mielep, said that we cannot think about thinking without thinking about thinking about something. That is a really powerful phrase to think about.

But I think going further, we should think about how do we think with the thinking machine? When the AI are starting to have this kind of amazing capability of generating new information, can we use them to make learning, not just about critical thinking, but learning in general, more exciting and more fun? Can we use AI to really personalize the content to make the learning material be more individualized or, you know, targeting the interests of each individual? Or, you know, can we use this AI to allow children

to be able to have, you know, like an expert opinion? Like, you know, one of the projects that we work on on the area of AI-generated virtual human is thinking about can we use this AI to create you know virtual peers maybe you know if a kid want to learn about physics maybe they can have a virtual Einstein that you know provide them with you know state-of-the-art research in a way that they can understand or maybe if they don't like Einstein maybe it could be a virtual superhero or any other character that is personalized and based on the student

interest. And with that, right now, there are a lot of conversations about, oh, should we ban the language model? Should we ban an AI? Because it can answer questions and children no longer need to write essays. For me, and I think in the media in general, we're thinking about, yeah, of course, new technology is going to come in and it's going to challenge the way that we think about learning. Maybe in the future, kids no longer need to write essay because ai can already do that but together can we think about new education paradigm where children with the power of technology can do things that even the teacher today cannot imagine maybe with the power of ai you know virtual assistant or virtual learning assistant, children of the future,

maybe at the age of five or 10 years old, would already achieve something that a PhD student at MIT 20 years or later can do. Can that accelerate the capability of people to learn and make amazing things? Maybe children of the future will be able to cure cancer in middle school or something like that with the power of AI. So that, think is the exciting vision for the future. How do we empower learning with the capability of AI? Yeah, I certainly love this much more optimistic view. I think there's been a mix of response from both the computing community or the AI community, but also the general public. And it seems that most

people are very quick to suggest that banning the technology is the right thing to do. And while I agree that policy and moderation is necessary, I think your viewpoint is certainly much more optimistic, which is how can we think of education of the future or students of the future where perhaps they're not spending their cognitive abilities on writing essays, but now they might be thinking about more critical world issues or thinking about how to solve some of the major problems that we're facing as a society. So it's very exciting to see that, you know, folks at the Media Lab are optimistic on how we can reinvent or think of a new paradigm for education using these foundational models and technologies. Yeah. One vision of the past, I guess, the philosophy of the past that I really appreciate is the vision that was led by Manfred Klein and Nathan Klein. These two people have coined the term cyborg. And it's in contrast to today's perspective that cyborg is about losing humanity and becoming

more like a machine. In their original paper, they say that the idea of cyborg is to use technology as part of a human body. And it's not about becoming more like a robot, it's about becoming more human than ever. So you can use technology to do the things that is mundane and repetitive and like a routine job so that human can you know be more free free to think to create to explore and to feel and the term cyborg was coined in the context of you know the early era of space exploration and at that time they're thinking that you know the space is so big and we by by ourselves you know the biological body can no longer go into the deep space, right? So we need to augment ourselves with machine. But I think that idea of using

machine to make us more human than ever should apply into the way that we think about technology today. Why do we want to be like a robot? I think we should remain human, remain having our agency and freedom to explore things and use technology to sort of expand that capability. I think we take that, we at the Media Lab, we are the optimists. We try to, you know, not just see technology as black and white, but the way that we can really, you know, empower people with new technology. So if we take that perspective and imagine, you know, what learning could be like, the universe is so big we have so many questions that we are curious about where we're from you know what is out there learning

should not be constrained by what we know but what we don't know right if we can really empower you know children to play in the middle that we think a lot about lifelong kindergarten right that's a really awesome group at the lab that really explore the future of learning. We think a lot of playful learning and constructionism, the way that we can construct new knowledge with the power of AI, I think that would really enhance or really accelerate the way that we learn

and think about new challenge for humankind. Certainly. I think any technology has limitations or the advent of any technology has any negative use cases. But I think if technology has limitations or the advent of any technology has any negative use cases. But I think if we are optimistic, the positive impact that technology can have far outweighs any negative impact. So it's very exciting to hear some of the viewpoints that you have. I want to go back on the wearable reasoning project or device that you mentioned.

You know, we're speaking or this is an audio conversation, so I can't physically see or you can't physically show me what this device looks like. But can you describe what this wearable device look like? Yeah, we have, you know, different iteration of this. But one of the most recent iteration that we have is actually now exhibited at MIT Museum in the AI exhibit. So if anyone come to Boston or come to Massachusetts, I would encourage the audience to go and visit that. They have really cool, and they have not just our project, but many amazing projects from MIT

that really highlight the evolution of AI throughout the history. And our project is sort of, you know, one of the recent example of what human AI interaction would look like. Specifically manifest as a screen, as a device that you wear, you know, like augment your vision. I'm curious to learn a bit more about what the device actually looks like.

Yeah. So there are different ways that we can, you know, present an AI to a person. In that project, specifically, we focus on audio interface or voice interface. Particularly, I'm really interested in the way that the AI could become sort of the second inner voice. Like when it whispered to us, can it be sort of the voice of reason or a second brain that, you know,

provide us with second opinion. So in that project is a glasses with audio and microphone and the Bluetooth capabilities can connect with, you know, the computational power in the cloud. This is where the AI exists and processes the information. And we try to, as I mentioned, make an explainable AI so it can not just provide feedback to the user, like, okay, this is reasonable or this is not reasonable,

but provide a meta-analysis, like, why does the AI think that this is reasonable, and see if that could improve human decision-making. And we found that it was really effective in helping people discern misinformation from honest information. So that's what the current wearable look like. Very interesting. Yeah, I've seen a range of responses to wearable devices. I think in sort of the first era of wearables, we've

actually seen many limitations that led to, unfortunately, the failure of many moonshot wearable technology. I think you described some with, you know, Google Glass, and now with the metaverse, you know, there's usually high price, there's the physical inconvenience of the device, it's very obstructive to, you know, your day today activity. It's heavy. There's concerns around privacy and safety. So when working on wearable devices, how do you think about some of these challenges? Is there a future to what wearable devices looks like to help address some of these concerns or pain points?

Yeah, that's a really great question. And I think it changed the social acceptance of wearable devices that have changed and evolved over time as well. i think earlier people think that the glasses you know would be the best form factor to deliver the information to to the person right that's why we have so many glasses in the beginning and now it seems like the form factor that seems to be more widely deployed is the glasses right i think of the things that we think a lot about in terms of when we create new devices

is that can we, you know, piggyback on things that people are already familiar with, right? Like, you know, these two devices are based on objects that we have already encountered in our daily life, like the glasses or the watches. They had less computational capability in the past. Now we have new ability that we add to them. So I think the key to success for these wearables is to sort of think about ways that it

can integrate seamlessly into our daily life. And of course, we're going to see more evolution of that kind of technology in the future. But it seems that what people care a lot about is not just getting more information as i mentioned you know more information doesn't lead to productivity or improvement in terms of our ability to use that information immediately we should think about how the technology really provide the value to the to the people and our group have been sort of looking at the cognitive augmentation but there are many other areas like sensing, can these devices sense information that we never sensed before?

Like right now, one of the trends for deploying wearable devices to use this kind of technology to do things that we used to do in hospital, like measuring high quality heart rate and respiration and skin conductivity and arousal and other things right these used to be things that we do at hospital now they've become integrated into our small wearable that we wear so i think there are many advantages that this technology will provide in the future but thinking about what are the real values that you know that they provide to human i think it's really important you know not just taking for granted that having more information would be useful, thinking about what are the real values of having that information, or what kind of augmentation can come about from that information. I think that's the important part.

So to do that, do you find yourself speaking with potential users of the technology? Or do you have focus groups, studies that you conduct, maybe engagements with community? How do you better understand not just developing technology for the sake of developing technology, but for the purpose of, as you described, providing value and utility to the kinds of people that you're developing the technology for? Right. I think that's an important question. I think in the Media Lab, we tend to be more imaginative. Sometimes we explore needs or challenges that are not a big issue today,

but could be in the future. For example, we have been working on many space projects, projects related to augmenting human capability in space as we become what we call interplanetary species, when humans can travel across different planets. It's become more real than in the past. In the past, we tend to think of this as science fiction, but with the recent space race in the private space industry,

we start to think that this will become more important in the future, especially if we're going to have longer space mission or deep space mission where we go you know into a further planet and you know with human space mission people you know tend to think about like rocket ship and spacecraft but i think one important area is also the space health how do we use wearable to really allow people to have like healthy lifestyle or healthy life in space? I think that's a really huge and important issue. So we have been working with NASA Trish, a specialized unit in NASA

that support the advancement of space health and MIT Space Exploration Initiative, which is an initiative from the Media Lab that really thinks about how do we humanize the space exploration? How do we focus on the human aspect of space exploration? So we're working on wearable systems like wearable lab on the body and wearable bioreactor that allow astronauts to be able to sort of sense new information from the human body

and be able to provide intervention on the astronaut suit itself. Because when you're in space, you don't have the ability to go to hospital, right? There's no Harvard Medical School or MGH around. You need to think about what kind of medicine can you pack into wearable device or on the astronaut space suit so that if you're sick, you can have that healing process or that treatment right away. So that, you know, has been an area that we work on. And that sort of work had led to a community, actually, called Space CHI. We have been hosting a workshop for the last two years at ACM CHI, which is, you know,

the largest conference in the area of human-computer interaction. We have been creating this sort of spatial gathering for people that are working at the intersection of HES-CI, human-computer interaction, and space exploration. And in the past, it's just NASA people. We talk to many people from NASA who say that in the past, it's just us thinking about this question

of how do we augment or allow astronauts to live healthily in space or to be able to, you know, work effectively in space. But now many researchers are thinking about it. Many designers, many artists even are thinking about this kind of challenge because space exploration has become more democratized and more and more people are thinking about how they want to define the next generation of human life in space. So that's, it's really exciting. Yeah. ACM ByteCast is available on Apple Podcasts, Google Podcasts, Podbean, Spotify, Stitcher,

and TuneIn. If you're enjoying this episode, please subscribe and leave us a review on your favorite platform. So it seems like your focus or your research motivations aren't even just thinking about solutions to existing problems, but it's also identifying new problems that have not yet been clearly voiced or clearly formulated. I really love the work that you described with, you know, wearables for health enhancement in space. Do you see this being applied for those who are on earth? I think even beyond applications in space, maybe there are seniors or folks who may be receiving in-home care. Are there applications even here on earth for some of this technology? Right, definitely. So many of these powerful

advanced concepts like the idea of cyborg that has inspired early work on wearable technology, the media people call themselves the board, was in the context of space exploration. And I think space exploration has some kind of magic or some kind of powerful vision that really inspires new innovative ideas right if you're thinking about maybe solution on earth there are more practical things or things that you can deploy with lower cost but when you're thinking about space it really open up your mind or the way that you think about the problem in a new way so that's why many of the technology that we're using today you know from you know gps or even like you know microwave that we're using in our kitchen you know this technology that we're using today, you know, from, you know, GPS or even like, you know, microwave that we're using in our kitchen, you know, these technology that we're taking for granted today, you know, will actually develop in the context of space exploration, right? So as we think further into the future, sometime the research that we create for that sort of extreme use case, but eventually

they will become more democratized and more accessible to other group of people as well but thinking about that right now may not make sense um because as you as i mentioned there are other more practical solution right so i think one of the goal of the media is to sort of always look at the the new frontiers and searching for new possibility things that you know make people question, like, why are you doing that? But then once it reached the level of maturity that we would expect,

it will have tremendous impact on human society. So I think that's our job, to be maybe less practical now, but more impactful in the future. Yeah, it's always crazy until you actually achieve it. So I think pushing the frontier of science is is uh certainly um right the return is certainly there i understand one of your experiments actually recently uh was launched to the iss the international station uh can you tell

me about that that's actually uh pretty awesome yeah as i mentioned mit media has a really cool unit called the space exploration initiative that work across different group in the media lab to explore our project in the media are always sort of crazy and out there and you know at the forefront but many many of them are you know targeting sort of earth-based population or earth-based user the space exploration initiative really pushed the media up to the next level where we're thinking about how can we deploy some of the things that we think about on earth in space and the project that you mentioned is actually uh one of the projects that built on top

of the you know early work that i have done on on wearable technology for astronaut in this project we are looking at how do we create bio digital whereital system that allows us to control the growth of bacteria in space so that we can use these bacteria to do multiple things, help recycle material in space, help recycle material in space, help us produce medicine and do other things. So we have been exploring how do we create this automated bioreactor that is digitally controlled, but allow us to sort of understand and grow genetic engineer bacteria in space for human benefit. And that has been a really fun project to work on.

And it was launched actually before New Year to the International Space Station. And this project is a really fun collaboration among many groups that come together to sort of explore this and now it's just returned from space to us we're gonna you know be busy analyzing the result in the next month or so and you know i think it's gonna really you know show new possibility on how we think about bio-digital interface in the context of space exploration yeah well that's very exciting i'd love to hopefully learn more about

or closely follow some of the results. I don't think many folks can say their experiments have been launched to the ISS. So I think quite an accomplishment and I'm sure the results will be equally interesting. Yeah. When you work on space mission, it's always like you are in some kind of a science fiction a movie you think about

okay you know what's going on you know what challenge do you need to solve like i really love um the phrase from the the movie the merchant right you need to science out of the solution you need to science your way out of the of the problem um i think that's really really true when you work on a space mission there's so many problems so many challenge that you need, really true when you work on a space mission. There's so many problems, so many challenges that you need to face. And when you fix them or when you solve them, your mission, you know, get to go to space. And yeah, and we are very excited about that. Very interesting. You know, we talked about a couple of different interesting things that you've worked on.

And, you know, your research has been published in various journals, you know, featured in various media outlets. You collaborate with stakeholders in academia and industry, with NASA, as you described. What is the impact that you hope your research will have on the future of human-computer integration or human-AI interface? I think of the impact of my work in three ways. So for the first level, I think of my work as showing a new kind of prototype or a new kind of artifacts, things that people would think, oh, why would you do that? Or things that people think that, oh, it would never work. We will make the first prototype, we will make that first attempt to show that this can be done. Maybe it may not be the most effective prototype or may not be the most practical

things that you can use today, but it's the first step to show that it's a possibility so that, you know, future research can improve and work on that. So that's the first level to show new prototype, new artifacts that really bring crazy ideas into something tangible. The second layer of impact, I think, is to, you know is to really show an experiment to show that, okay, the thing that we make have this kind of impact or this kind of improvement on human life. You can see some of this in the experiment we do on human-AI interaction or on some of the space projects. We have real scientific experiments that validate our our work and we have shown that it has improved our human life in terms of our decision, health, creativity, or whatever. To show the experiment,

I think it's really, really cool. And then the third layer is to really deploy this in the wild to see if it really has impact on human life. I've done some project that was deployed during the COVID pandemic, like the chatbot life. I've done some, you know, project that was deployed during the COVID pandemic, you know, like, you know, the chatbot that help people retrieve certain information or design of a project that, you know, look at how we can optimize the flow of people getting the vaccination. These kind of project, you know, maybe less futuristic, but can be deployed and, you know, really save human life.

Like the chatbot project was used by, was used by over millions of users worldwide. And the vaccination site that I co-designed with people in my team have really accelerated the process of vaccination. So we don't focus on just practical things, but if we have a chance to really push that or do something in that area i think that's also really impactful so yeah these are the three sort of way that i think of my on my contribution to you know the research community and and to the world basically to offer a new kind of crazy prototype you know ask question and experiment on things that we never thought of and finally

deploy some of these ideas and and solution into the real world that's that thought of and finally deploy some of these ideas and solution into the real world. That's awesome. And, you know, as I said before, it's only, you know, crazy, or it only seems impossible until you do it. And I think at the end of the day, I really always excited and fascinated by the strong emphasis on practicality on, you know, how does it improve human conditions, I think, you know think many technologists can be motivated purely by scientific pursuit or just the happiness or satisfaction that comes with pushing the frontier of science as we know it. But I think there's also another aspect, which is thinking about how can it be applied to improve society or human life as we know it.

I really love your perspective you know, your perspective or your take on things. And I think one of the things that I found quite interesting is that outside of, you know, your scientific pursuits, you also bring a very strong passion and love for art. And many of your work has actually been featured, I was surprised to see, in museums and, you know, galleries around the world. And from what I observe, as you described, you also have this entrepreneurial spirit or this entrepreneurial drive. You worked on projects before graduate school. You've been working on projects with the COVID chatbot, as you described. You've co-founded various programs and initiatives. Can you tell me a bit more about

these two passions, your love for art and then also this entrepreneurial drive? In the media, we have this full area that we often describe the way that this thing connected. We call it the creativity cycle. And it's about how art can inspire science, how science can inspire engineering, engineering inspire design, and how new design create continue that cycle by inspiring new art right so that's one of the way that we think about um how you know creativity

happened is when new discipline are collide and connected with one another so i was fortunate to work with many um artists that i collaborate with to really, you know, explore the implication or the new question that can come about in terms of, you know, artistic practice from my project. Because oftenly when we work on really advanced technology, whether it's AI or space exploration, it's really beautiful. And, you know, in terms of the aesthetic of it, or in terms of the question that it brings about, and art, I think is a way of asking questions and be imaginative and exploring the hidden question to that. So many of my artistic projects have been exhibiting around the world, and it has really opened my mind in terms of what this technology is really about. Sometimes it's about criticizing the work

and sometimes it's about opening up new way of being expressive. So for example, I've been working with a Thai artist, Kavita, on creating an AI clone of her to have a debate. So she can sort of be in the middle of two AI that is based on her own data,

talking to one another and exploring, you know, what would she actually do when she externalized, you know, her mind into multiple AI that are debating on important issue. And that performance has been, you know, featured in several museums and, you know, has really, you know, not just, you know, it's not just that I enjoy working with her, but it really opened up my mind about new research possibility in her ci as well right so it's not just about making art but how art really inspire new research and you know it's not just artistic community i also engage with you know startup community and industry that's one of the strength of the media lab right that we get to have like you know

member companies and you know they bring new bring new perspective, new challenge for the industry to us. And we have the privilege to be more crazy and say to them that, oh, to your problem, we have these crazy ideas. And, you know, that is beneficial for both of us, right? Because for us, we get to see a new challenge, you know, insight on things that we never thought of. And for the member company, they see new crazy possibility that maybe they never thought of so that's always fun as well to think about how the work that we do can go into multiple channel artistic output or industry output or yeah so i think to me my work is about as i said augmenting human capability and i want to be you know

expressive and and use this work in in many ways um one of my goals to live an expressive life to be able to express my idea and thinking into research you know artistic project and you know solution for um industry and startup and you know social impact and yeah i think that's the kind of fun um research ecosystem that i've really enjoyed the media lab oh that's awesome and so it seems like science and these outside interests or pursuits are not independent or mutually exclusive rather they actually help motivate some of you know your scientific pursuits and contributions absolutely awesome one of the things that I loved in your bio is this idea of, you know, moonshot thinking to, you know, create, you know, a future innovation.

So as someone who actually believes this idea in, you know, crazy ideas, and this idea of thinking about the future, how do you find you are best able to channel your creative or moonshot thinking? Do you have a specific process? Do you have specific frameworks? Do you like to go on walks? Do you think of these ideas when you're in the shower? Where do you find you are able to be at your best when it comes to really thinking about these crazy quote-unquote ideas i think i say it many times but i think that your questions are very cool and yeah for me i always think that innovation is never a straight path you know it's never like you walk into an innovation and it boom and it's become a world-changing idea or things like that it's always like um

it's like a walk in the forest. You go and you see that beautiful tree and then maybe it makes you think of something and you walk and then you see another tree or maybe you see a dinosaur in the forest. And it's always like an adventure. And yeah, if you look at the history of how powerful ideas come about,

it's never happened in the formal context or the context where people are like, okay, I'm going to make something. It's rather happening in the formal context or the context where people are like okay i'm gonna make something it's rather happening in the very unexpected environment or in a very unusual place so for me you know one of the reason that i i love to work in this different domain like you know artistic project and industry project and you know research project is that it really sort of you know make your mind flexible to think across different um possibility and um yeah so i think that's one of the one of the thing that i really appreciate about being in the media is that i get to sort of be flexible and and go into that

different area and the best you know you ask about what is the the best time to to have that kind of crazy idea yeah i think walking definitely can really help it help you be stimulated um as you kind of walk to the environment i love to walk in the park in the afternoon or in the morning to get sort of new ideas and um listen to like music that kind of put you in the in the flow state um and and sometimes you know showering in hot water also really helps um it's it's really helped sort of, yeah. And what I find interesting is that it will never come to the thing that you expect. It always sort of go in a different direction.

But then at the end, if you figure out the connection, it will be something powerful and awesome at the end. So yeah, as I mentioned, it's never a straight path. It's always a complex and malleable kind of thing that you need to sort of keep molding it and keep pushing it and keep thinking about it. And then it will resolve into something interesting, I think. That's awesome.

So there's something to be said about, oftentimes we explore solutions to solve problems, but sometimes we think about how we can apply existing solutions to be, you know, applied to problems that we've probably never explored before. So, you know, you've worked on some cool applications of, you know, wearables and machine learning for learning and well-being and decision-making. So do you often find that, you know, you find yourself identifying problems first, and then thinking of ways where technology could be used to fix these problems, or, you know,

provide a solution to these problems? Or is it the other way around? Or maybe you think about, these are the capabilities that we have today with large scale language models, or wearable devices, and then you think backwards, or you reverse engineering, you say, these are the kinds of problems that now I can solve with this technology technology what's the framework that usually drives more of your research or your new explorations for me i think there's no like one formula sometimes you are inspired by the new advancement in in technology and in ai for example like you know new models always like you know fascinating to me um and sometimes you have like a vision of like oh would it be cool if we have this kind of thing and you know

what's this thing about it like what are the ingredients that we need to put together in order to make that like is it you know new model that doesn't exist or is it new data set that i need to go and search for yeah so i think it happened in in both ways sometimes you see something some interesting ingredient and you want to cook something awesome out of that or sometimes you think of a meal like that that you want to make and then you think okay what are the ingredients that i need to go sort of to go find in order to to make that so i think it happened in many ways um and and sometimes it's about sometimes people criticize you know the the sort of the futuristic work that we do at the media lab at like, oh, you're creating solution in search of

a problem, like you're not solving real problem. But sometimes that's what we need, right? We need to imagine new ways to live a life. And sometimes it's not about using technology to fix the existing problem, but rather to reimagine how do we recreate or reimagine that whole scenario altogether and the lab have shown a really good track record of how do we sort of reinventing that like for example um the early work on touch screen um that you know i i think it's a very fun example to to think about like when the first touch screen came out from the lab it was i think it was an architecture machine it

was the precursor of the media lab but professor Professor Nico Pronte, who was the director of the Media Lab, he said that he was heavily criticized for exploring the work of touchscreen. And the first touchscreen that came out from the lab was criticized that, oh, human hand is so messy. If you touch the screen, you know, it's going to make the screen messy and dirty. No one's going to ever use the touchscreen. Those are valid reason but the thing that we never imagined we could do with touchscreen you know outweighed that small issue so i think sometimes we don't know what the future gonna look like so we should keep exploring and explore new possibility not constrain ourselves in one way of thinking like oh i'm gonna only

solve problem or oh i'm gonna just create things that doesn't exist but you know be flexible like sometimes work on things that are practical sometimes work on things that are imaginative and the two will feed off each other and and make you become a more creative person yeah certainly you know one of my favorite sayings is that good players play where the ball is great players play where the ball is going to be so So I think in pursuit of future problems, sure, they may not be the problems of today, but they are the problems of the future. And if we can proactively think of solutions or explore ways to address those problems, I think it's better for us as a society in general. So quite exciting to hear that, you know, I want to conclude

with open questions for you. One, you know, what are some exciting developments or future directions that you see in your research that you're currently working on or that you're excited about exploring? I think you mentioned some things around education. I think you also mentioned some things around the excitement of, you know, these large models. But are there any exciting developments or future directions that are keeping you up at night, so to say? Well, so I often think of my work as a tool, sort of paradigm. One is about bio becoming digital, and another one is about digital becoming biology.

What I mean is that as some of the projects in space exploration really show that we can really program biological machines, like how we program digital machines, this is about you know synthetic biology how can we program bacteria and things that are on the human body itself to become like a digital mini micro computer on the human body right and the other area is about how do we start to reimagine digital system as a biological system. So start to look at, you know, these large language model, not just as computational system, but, you know, similar to like a biological neural network. Like we need to understand, you know, how the AI work in the same way that we need to understand how the brain work and the complexity of the AI system today, I would argue is kind of getting closer to the level of, you know, human brain complexity.

And we don't have tools to really study either the human brains and AI model at the resolution that we need to really fully understand what's going on. The work on explainable AI is emerging, but we need more work of that kind, especially on large models, models that are very stochastic and models that are very complex in nature. So yeah, we need to really learn from biology, using bio to create new kind of digital and

using new digital to create new kind of biology. I think it's a puzzling idea, but I think it will really open up the way that we think about computing of the future. And of course, for me, I think one of the themes that we discuss is that technology itself is neither good nor bad. It's about how we really deploy it. And the way that we interface technology with humans is not something that we can take for granted.

It's all by design. My advisor said that you cannot just make an awesome AI and expect that when you drop it to a human population, they will be able to use it immediately and use it in the right way, right away. You need to really think about how these two can co-evolve with one another. This is also a very biological concept, like how two systems co-evolve with one another. So the interface itself is really important. The media philosopher really said that the medium is the message, right? So when we work on this awesome or exciting new technology, the interface, how it connects with people is really something that we need to pay

attention and design for. So we talk about large language model and many large AI models. They are really awesome and have many benefits. But that benefit will not happen if we just think about that system alone, without considering the human psychology, without considering how the human would over-rely on it, or how we can really design to promote creative use and critical use of that technology. It's not going to happen because of the system. It needs to happen by design. So we need to pay attention to that.

So I think these two areas, thinking about bio becoming digital, digital becoming biology, and thinking about how we create system or AI system that co-evolve with human in the longer run, in the longer term, are things that I think a lot about. Oh, yeah. As we continue to see technology being so widely adopted in every aspect of our lives, and especially with the emergence, or at least the growing emergence of AI technologies, I think exploring both research

paths, I think is very important. And I think as long as we have folks like you who are thinking hard about these kinds of problems, or hopefully we have more, I think we're all in good hands. So one final question I want to ask, you know, for those in the younger generation who might be interested in the field of computing, what advice would you provide? And one thing I specifically want to ask about is it seems like you've been able to balance, you know, your passion for science with some of your interests in art and entrepreneurship. So how would you advise those who are looking to balance some of their scientific interests with their passions, whether it be art or music or sports, to best channel and fuse those interests together as much as possible? Right. And dinosaurs as well. Yeah.

And dinosaurs, of course. Yeah yeah i think to me what i've learned and seen at the media lab is the the celebration of diversity and diversity in broad a sense possible like you see a lot of interesting and and weird people at the lab and they all really really show that their unique perspective and unique appreciation of things. And I think that would lead to a creative community when people can really bring what they're passionate about to their work. And that passion will spark ideas and imagination in other people. So that kind of energy is really important. So not to kind of constrain yourself to becoming a kind of flat dimension

or a one-dimensional person, but really celebrate who you are as a human being. If you love dinosaurs, then go for it and bring that to your ideas and creativity. So yeah, I think that's really important. And one thing for me that I think is also really important is to sort of think about what kind of impact that you want to have in the future. One of the Media Lab professors, you probably know, Professor Hiroshi Ishii, always asks all his students and other Media Lab students, like,

what kind of impact would you have 200 years after you die? And I, of course, don't have the answer immediately, but it's something to think about, you know, to have like a bigger question or something that is bigger than life to think about, you know, to have like a bigger question or something that is bigger than life to think about. Whether it's a philosophical question or things that, you know, make you not just stuck in the moment, but be more transcended in terms of your thinking. I think that's also really powerful. And I think finally, to embrace all the complexity of the world.

Like, you know, we are living in the era where technology is moving at such a rapid rate and we need thinking and all kinds of thinking, not just scientific thinking, but artistic thinking and all kinds of creative methodology to really help us understand how we're going to live in the future. And it's not just about dystopia, and it's not just about utopia,

it's about the mix of all the good and the bad and the negative and the positive so um exploring all the spectrum with you know embrace all the complexity and all the possibility ways and ways to to understand it i think that's the way of the future people often ask me like oh am i going to be replaced by ai or am i going to be replaced by robot or am i going to be replaced by robot and i think what is really powerful is to think about you know we talked about this earlier about education right to not think about technology just as tool that replace human but things that can really allow us to think of things that we never thought of before the augmentation the idea

of cyborg becoming more human than ever. Children of the future will definitely experience this more. They should not be afraid of it. They should think critically about it. But then at the end, think about what can we invent to really uplift humanity. One of my friends has said that to imagine a dystopia is easy. It's easy to imagine how the world to end in many ways could be alien attack, climate change's easy. It's easy to imagine how the world to end in many ways could be alien attack, climate change, whatever. It's easy to imagine the dystopia, but to really imagine how we're going to make it, how we're going to really turn the scenario around.

That is the challenge that we as a researcher, scientist, artist, designer, engineer, whatever we are as a person of the future need to really think about, be optimistic, but also critical at the same time. With people like you, the future is bright. And I've certainly taken good notes on some of the advice that you've shared. So, you know, I really enjoyed this conversation and I look forward to seeing many of the great contributions you will have and will continue to have in the long future. So thanks so much for joining us on ByteCast and look forward to keeping in touch.

Thank you for having me. ACM ByteCast is a production of the Association for Computing Machinery's Practitioner Board. To learn more about ACM and its activities, visit acm.org. For more information about this and other episodes, please visit our website at learning.acm.org slash b-y-t-e-c-a-s-t. That's learning.acm.org slash ByteCast.