Instant Genius - Changing our behaviour with virtual reality

Episode Date: May 3, 2018

VR can be used for so much more than cheap thrills and casual gaming. Jeremy Bailenson tells us how he is using VR to change the way we perceive racism, highlight the impact of climate change, and help us step into the shoes of our sporting heroes.   Hosted on Acast. See acast.com/privacy for more information. Learn more about your ad choices. Visit podcastchoices.com/adchoices

You said this place was steps from the water. We just haven't found the steps yet. How much did we save? Enough. Enough to get lost. Or you could book a stay with Hilton. Welcome to your ocean front room. Just steps from the water.

The Hilton sale is on now. Book on Hilton.com or the Hilton app and save up to 20% to get the stay you expected. When you want savings, not surprises. It matters where you stay. Hilton, for the stay. When you need to build up your team to handle the growing chaos at work, use Indeed sponsored jobs. It gives your job post the boost it needs to be seen and helps reach people

with the right skills, certifications, and more. Spend less time searching and more time actually interviewing candidates who check all your boxes. Listeners of this shell will get a $75 sponsor job credit at Indeed.com slash podcast. That's Indeed.com slash podcast. Terms and conditions apply. Need a hiring hero? This is a job for Indeed sponsored jobs. Kayak gets my flight, hotel, and rental car right, so I can tune out travel advice that's just plain wrong. Bro, Skycoin, way better than points. Never fly during a Scorpio full moon. Just tell the manager you'll sue.

Instant room upgrade. Stop taking bad travel advice. Start comparing hundreds of sites with kayak and get your trip right. Bad advice. You talking to me? Kayak, got that right. This podcast is sponsored by. by Name, Audio and Focal.

Streaming has made music more accessible than ever, but true listening is about more than ease. It's about quality. British audio experts name Audio, alongside French acoustic specialist Focal, combine handcrafted tradition with cutting-edge innovation and high-end materials, delivering digital precision with analogue warmth,

so you can experience exceptional sound at home. Music just as the artist intended. Visit nameadio.com to learn more. saw the kind of after when they don't do the regulation, it was really powerful for them. And they actually signed into law that day and regulation about how they're going to work with scientists to think about reef policy and affect some of the tourist policies that they have. So that was an instance where using VR actually help lawmakers understand better their climate change adaptation strategy. You're listening to the Science Focus podcast from the BBC Focus magazine team. We're the

UK's best-selling science and technology monthly, available in print and in several digital formats throughout the world. Find out more at sciencefocus.com or look out for us in your app store. Hello and welcome to the Science Focus podcast. I'm Alice Lipscomb Southwell, the production editor of BBC Focus magazine. Virtual reality has officially become mainstream, with top of the range headsets like the HTC V and Oculus Rift available for under £1,000 and cheaper headsets that use smartphones readily available. VR has never been more accessible. But are we really using it to the best of its abilities?

In this week's Science Focus podcast, we talked to Jeremy Balenson, Professor of Communication at Stanford University, and founding the director of the Virtual Human Interaction Lab, who has been studying virtual reality for two decades. In his book, Experience on Demand, he explores the powerful effect that VR can have when it comes to changing people's perceptions and behaviours. He speaks to ScienceFocus.com editor, Alexander Macon

Mara about using VR to change the way we perceive racism to highlight the impact of climate change and to help us step into the shoes of our sporting heroes. So my name is Jeremy Balanson and I've been studying virtual reality for 20 years almost. What I do is I build immersive virtual reality, but what I test is how it affects the mind and what applications work or not. Even though I do build VR, I'm at heart of social scientist and for quite some time I've been running an academic lab over the last few years as virtual reality has become a consumer product. I've been spending time working with companies and governments and nonprofits to help them navigate this very complicated landscape of, you know,

when should I use VR and what is it good for? And so when should we be using VR? My philosophy is virtual reality is not an everything medium. It's not for checking your email. It's not for spending four or five hours a day. If those of listeners who have tried VR, it's heavy on your head. It's hard to wear for a long, long time. It separates us from the physical world.

In my opinion, VR is great for very special experiences, things that if you were to do them in the real world, it would be very expensive or dangerous or even impossible. And I can give examples of those scenarios if you'd like. Oh, that would be great, yes, please. The impossible is one I spend a lot of time on. For example, we do a lot of work with empathy, where let's pretend that you wanted to better understand things like sexual harassment or racism.

you can literally wear the body of someone else and look down and, well, now I'm a person of color, and you can experience discrimination firsthand. And we've done work with companies where we're helping them do their diversity training in a sense where you can actually, instead of making this information abstract, you can feel what it's like to have to go through this prejudice. So how does that work when you're in a VR environment? So we just premiered a film five days ago at the Tribeca Film Festival in New York City. It's a VR experience where it's called. thousand cut journey and in this one you put on the goggles and you become a male named Michael

Sterling you become a black male and when I mean when I say you become him when you move your arm you look down and your arms are moving with you you're wearing these hand controllers and you see your body as a black male and when you take a step forward your legs move forward so you're literally step into his body and walk around what happens is we show over the course of your lifetime you start out as a child and you experience discrimination in the classroom or the teacher treats you differently than the other white kids in the class. And then as a teenager, you get stopped by the police for something that, again, your white friend did, but you are the one that gets stopped and gets frisked for the scene.

And then when you're a 30-year-old, you are in a job interview and you face discrimination there. It's a very powerful piece where it's experiential. You're actually doing these things and feeling reactions from other people. That sounds pretty incredible. So what's happening? What sort of responses are people getting when they're... they're going into this experience. So let's go away for a second from this experience and talk about our academic research.

What we've demonstrated for years now, the first VR empathy study I did was in the year 2004, published in 2005. In general, across our research studies, we'd measure whether you change your behavior later on, whether you're going to be more helpful to someone or whether you change your attitudes towards those people. And in general, across 15 years of research, not every single time, but in general, VR tends to change behavior more than the control conditions compared to say role playing or watching a video or reading about case study. So in this particular experience, Thousand Cut Journey, it's brand new. We just premiered it.

What we're going to do now is go back to the drawing board and start to study this academically to see the effects that it has. What I can tell you at the festival, it's had some pretty intense reactions. And in fact, it's, you know, this experience was designed. to make people who are, you know, who want to be better, want to be more helpful to solving racial injustice, but maybe who aren't moved to action. This is for someone like me, who's a white male, to really, you know, instead of just making this idea abstract, to make it more intense and experiential, to make more likely to act to help others. We are finding when people

of color go through, and my colleague, I should say, Courtney Cogburn, who's a professor at Columbia, she studies black-white racism for a living and this is her. She's the main lead director on this. And what she's been finding is that when people of color go through, this can actually be a bit of a trigger emotionally because some of them are called back to experiences that they've had. So a very different reaction depending on whether you're black or white. It sounds like with your research, a lot of what you do,

reading through the book, there's a similar sort of example, which is very different to the racism thing, but there's the toilet paper example about using trees. It sounds similar to the fact that VR is changing the way how people respond afterwards. Exactly. So VR is great for experiences you couldn't have otherwise. With the tree cutting paper,

becoming someone else falls into the bucket of, quote unquote, impossible. These are things I can't become a different person. The tree cutting study is actually about doing things that if you did them in the real world, it would be counterproductive. And so in this instance, we had people all come into the lab and we brought in people who don't use recycled toilet paper. In other words, they use that soft, fluffy toilet paper that everybody loves that's nice and soft. It turns out that's from non-recycled pulp, which means that you're cutting down a lot of trees to make that toilet paper if you're using it over your lifetime.

So we had subjects come in and half of them read a narrative about what would be like to cut down a tree. and the other half, instead of reading a narrative or some watched a video about cutting down a tree, those were the control conditions. Then the treatment condition people actually put on the VR goggles. They use what's called a haptic device, which is a virtual device that moves your hands. It's for virtual touch to simulate a chainsaw. And they cut down two trees. And they were moving their hands and getting the vibrations and they saw it looked around and saw the forest and all the trees.

and when the tree crashed, the floor boomed and they felt the crash of the tree and the birds all scatter. Very intense experience. What our study showed is that in the control conditions and in the VR condition, everybody said afterwards they wanted to use less paper. However, we found a way to track their actual behavior, their paper use. People in the VR condition used 20% less paper compared to the other conditions and they don't change. The idea is this experience you have, this using your body to do something, it stays with you. And having an experience changes behavior. Why this would be counterproductive to do in the real world is that imagine that I was teaching you about deforestation by force you to cut down trees.

That would be a bad way to do it. It doesn't sound like a great start, really, does it? Right. So when they're in this VR world, how real does it need to be to have an effect? Well, when we talk about realism, there's a few different ways to frame that. One is how crisp the graphics are. The second is how responsive the movements are. In other words, when I'm looking around the scene, it can be a picture perfect, photorealistic image.

On the other hand, it can respond to my body movements perfectly. So if I take a step forward, do all the objects get closer to me in the proper way? Or when I'm using that chainsaw, does it respond to even a touch? tiny millimeter movement of my fingers as I'm going back and forth. And what our research has shown is that behavioral reels and that response to movement is the most important. So graphics are actually not as important as having the scene respond to your body quickly, accurately, and often. So that sort of suggests that the VR equipment that we have, that we're able to purchase now isn't going to be advanced enough for us to really be able to make great changes.

in the future? So here's what's surprising from me as a guy that's been doing this for 20 years. One of the hardest jobs that I've had over time is it's called tracking, which is measuring your body movements as you're moving your hand and your head and your body. The high-end six-figure system that we have installed in my lab, it tracks your head and hands and feet to one-tenth of one-millimeter accuracy, really, really accurate. The commercial systems you can buy now, for example, the HTC V or the Oculus Rift or the Sony PlayStation or the Microsoft HoloLens, any of these systems, they cost a couple hundred dollars,

and they can track down to about half a millimeter of accuracy. So, you know, you're in the ballpark now when you're buying these commercial systems where the movements are accurate enough to simulate these presence. Now, if you go for the phone-only systems, for example, the Google Cardboard or the Samsung on gear, maybe you're not getting enough movements. Those actually track pretty accurately, but they don't track your hands or your feet. So we're surprisingly in the ballpark nowadays where for small money you can get close. So what you're saying is that essentially the work that you're doing to do things sort of like change people's perceptions with the idea of race

or the environment, that we will actually be able to do that with what we have. The software we run in my lab, I can now literally run on commercial systems exactly as is, meaning it's its exact same experience, and there's millions of them around. That must be very different to when you started 20 years ago. It really changes our research philosophy. It used to be the only way we could conduct research in the lab was to have people physically come. Now what we can do is we can go to a festival or a museum and set up a booth and collect data on the road. In addition, I can literally, this is actually a really strange thing for me. The content, I can save it and export it as an EXE file and executable,

and I can just share that EXE file with someone and they just double click it, and it works on their system. So we can actually collect data by sending out the files and receiving the data files coming back. So what's changed as a social scientist is my big. now to look at very large samples and varied samples, people of different ages and in different locations. So from a research strategy, it's almost as if we can do these large-scale surveys, but now in a really realistic immersive VR environment. This new data that you're getting, how has this changed your research and the results that

you're getting? What's changing is the ability to get very large-scale samples. So if you look at the studies we've done in the lab, typically we've got 50 or 100 participants, what we can now do is we can get samples that are thousands. And we're about to publish our first study where we went to all sorts of places, flea markets and senior citizens' homes and museums to set up a permanent booth. And we got thousands of people coming in because a lot of our research shows VR causes empathy in this study where people became homeless. They basically in VR experience to getting thrown out of their apartment and having to live in a car and then try to sleep on a bus because they'd lost their job.

They learned about this journey of situational factors causing you to lose your home. And this is the first study that has quite a large sample so that we can start to understand, forget the laboratory and forget academic work, when you really scale this type of an experience out, does it still change people's behavior? And in this study, what we demonstrated, after the VR experience, when you take the goggles off, if you hand someone a petition and you say, are you willing to have your own personal taxes increase to support affordable housing, when you've had a VR experience compared to control conditions, you're more likely to sign. So this could actually have quite strong implications for things like policies and government policies and fundraising and all of that.

Yeah, so with our climate change research, we do a lot of research where in virtual reality you'll put somebody in, the future of climate change and they'll get to see what it's like in their district after there's been floods and droughts and rising sea levels and we'd go to plow palau is an island nation and micronesia network of islands and we filmed underwater for two weeks and produced you know a four-minute experience that shows palau is a nation with 20,000 citizens they have about 100,000 tourists each year that come their economy is basically based on these tourists coming to see their beautiful amazing coral reefs it's it's just a spectacular place unfortunately climate change is threatening not just the economy of palau because all the tourists are coming to to see that but in the

entire nation given sea level rise and so what we did in palau is we filmed underwater in 360 all these amazing kind of before and after shots of what happens when the reefs are taken care of and what happens when they don't regulate tourism and they don't regulate farming practices and sediments cause the reefs to get destroyed. And we went to a meeting of the lawmakers where we had all 13 of the senators and many of the House delegates and we put the lawmakers inside virtual reality and they got to experience the kind of future of their reefs if they don't regulate. And the reaction, it was just incredible.

They were, you know, a lot of these lawmakers, the culture of plows, they don't go diving. A lot of them are afraid of sharks. So many of them just hadn't seen these reefs. So just seeing the reefs was a different experience for them. When they saw the kind of after, when they don't do the regulation, it was really powerful for them. And they actually signed into law that day and regulation about how they're going to work with scientists

to think about reef policy and affect some of the tourist policies that they have. So that was an instance where using VR actually help lawmakers understand better their climate change adaptation strategy. It just seems like incredible that this technology, which we've been sort of associating for such a long time with just people with headsets on in rooms, just essentially the way how we've seen it a lot in popular media is sort of as games and things like that,

but it's now actually having real impacts on government policy and decisions, decision making. Is this going to, do you see this happening a lot more in the future? I certainly, as the hardware gets even more comfortable and gets cheaper, and as it becomes easier to produce content, I certainly see VR becoming not just a strange technology, but a mainstream medium.

So how accessible do you think VR can be or even how accessible it should be? Well, those are two separate questions, and I like them both. On the Can Be in the United States, conservatively, when you look across all of the different hardware platforms, there's at least 15 million headsets that are around the United States. So, you know, you're at the point.

point where there's a fair number of them. The truth is, though, not all of them and many of them are actually not getting used. There are probably a lot of these guys are sitting around collecting dust. And there's two reasons for that. One is, you know, for some of these, it's hard to maintain them. You've got to set up the tracking systems. The cameras have to be accurately calibrated. Driver updates to your Windows machine can cause some problems in some instances. That's one reason. but I think the true bottleneck is content. The content needs some work. So a lot of people, if you've tried virtual reality, you've done it, you've looked around,

and you said, huh, this is really neat, but you didn't run out and go buy five systems and use it the next day. And the content right now, it's oftentimes just doesn't justify putting on these goggles, separating yourself from the physical world. And it's just not better than other media. when you could just watch the same content on a TV or read about it. So what you're going to see in terms of your question was, you know, should there be more use?

And my answer is it depends on what the content is. And as better content comes in, it will justify, you know, isolating yourself from a room and experiencing something immersively. But it's not for, in my opinion, you know, watching two hour long movies or, you know, checking your email. It's VR is great for these very intense short, special experiences. I think one of the things that I've found with the content that I've used in VR

is that the ones that really work, well, are the ones that are indeed short and very good, but quite shocking content as well. I mean, it's very easy to go into VR and just find lots of horror-based things because they are quite effective. What can we do to make this content better? Well, think about the medium of film. You know, we didn't come up with Star Wars or Citizen Kane. the first decade or even the first few decades of the medium existing.

It's going to take some time for people to figure out all of the storytelling strategies and how to make content engaging. VR is very different than film. You can look around anywhere. You can turn your head up down to the side. You can walk into people and objects. And there's a lot of things that work with film where you can't capture someone's attention in VR because you don't know where they're looking.

So you'll miss a critical detail. Alternatively, there's some experiences, for example, you just mentioned horror. You know, if I were to experience the movie Jaws with the sharks in VR, I would never go in the ocean again. There's certain things that are so intense that you need some separation and you'd have to have that 2D screen as opposed to being inside of it. And does that work with things like games as well? So for instance, you know, like shoot-em-ups are very popular games, but in VR, it sounds like that would be too intense.

So you're seeing a bit of a resistance of the intensely violent video games from users. And I don't have statistics on this, but, you know, for the first video game content in VR to make a million dollars was a game called Raw Data. And while that is a shooter, you're not shooting people, you're shooting robots. And you're seeing a lot of convergence from designers on trying to avoid intense violence to humans. So I'm fairly loud in my hoping and cheering that the video game designers continue strategies like that, not intense violence to humans. The other thing you're seeing here is that when there's just a really intense experience, people just don't gravitate toward it.

Oftentimes when you give somebody the opportunity in VR, in my experience, to experience something really intense in violence, they choose not to. And it's not that they wouldn't watch a movie on television. There's just something about using your body to do this violence that tends to turn people off. And I truly hope that trend continues. Now, with video games, the other thing to think about is that it's an incredibly successful economic market right now. People play video games for hours and hours a day. And in VR, you just can't do that.

So in my opinion, I'm not convinced video games is the best use case for VR simply because VR is great for a couple of minutes. It's not great for spending so much time and it just doesn't feel good. So your book is called Experience on Demand. And you're saying it's good for these sort of short experiences that you're getting as opposed to playing full games. Do you think that we'll get to a point where these experiences can be a bit better than our reality that we're in at the moment?

and that could be somewhat damaging in a way? Certainly, I can't speak for you or for our listens, but I use my smartphone way too often. I think about it too much, and it's become a piece of my life that I wish I would use less of. When every social media experience feels like the best party you've ever been to, when online gambling feels like going to Las Vegas, when pornography feels like actual sex,

the question is, how do people adjust to normal, life and that's certainly something to consider. I don't have an answer on how to avoid that. The one thing I can tell you is moderation and try to limit your use and make sure you go outside every once in a while, more than every once in a while actually. So it's not going to be, you know, we've just seen Ready Player 1 come out. We're not going to end up in a situation like that. I don't think so. However, you know, I'm wrong every day. So it's my sincere hope we don't get there, both from an environmental standpoint and from a media addiction standpoint. But I'll do my best to continue to cheerlead for moderate use and for content that's good.

So coming back to the sort of use, essentially moderate use and the good content, when you're experiencing something in VR, so as a lot of it is used for training, how much of that experience can be done in virtual reality before we have to move on into the real world? do things practically? Training is one of the home run use cases for virtual reality. For decades and decades, since the early flight simulator is built by Edwin Link in the late 1920s, we've used VR to train people. VR is great for things that are too dangerous to do in the real world.

What we've seen in the last few years is a proliferation of using VR for things that are not just dangerous, but are just expensive and rare. So last year, a company I co-founded called Striver, we trained 200,000 Walmart employees to do better at their jobs in retail. Things like look around a room and figure out how to make eye contact with customers, how to survive something called Holiday Rush or Black Friday, where there's people running all over the place and yelling, how to quickly scan a room and find safety violations. We've been working with athletes. So, for example, the German national soccer team spent time with us using VR to train their players to better recognize defensive patterns and for goalies to understand better penalty kick and to try to read body language. Training is an amazing use case. Now, when we work with athletes in particular, it's very important to tell our coaches, we're not saying get rid of physical practice.

We're saying VR is great to give you extra mental repetitions. And it's these repetitions that sort of help build experience to take it into the real world? Absolutely. You're using your body to move around. You're scanning a room. You know, with Walmart in particular, when I mentioned holiday rush. Holiday rush happens once a year in the United States. It happens on the Friday after Thanksgiving.

And it's this day where there's an incredible amount of chaos and concophony and people everywhere. Because of the high turnover rate of Walmart, 50% of the rate of the market, 50% of their managers have never experienced it because they're hired newly. So VR gives them that experience. It gives them this understanding of how intense. You just can't describe how powerful that day is as a manager. It gives you the emotional understanding of, wow, this is even more intense than they described it. It's a really nice tool to help the employees have a visceral understanding and to help them build strategies to be better when that moment hits. And could customers use that as well? to be a little bit more empathetic towards the managers

and the people who are working Walmart on the day? I think that's a brilliant idea. Every once in a while in an interview, I hear a new idea. And I think that's it. The data point I have on that is with US quarterback. So in American football, you've got the quarterback. And I was at a conference where we had an NFL quarterback on stage,

and he was wearing the goggles. And he was looking around going through the plays. And he was showing the audience, you know, what was going through his mind while he was trying to figure out whether or not you change the play while the defense is rushing in. And what you saw collectively in this audience of 500 people was this kind of gasp because we all of a sudden realized how hard it is to be a quarterback and how fast the game moves in a way that we hadn't really understood otherwise. And I do believe that athletes who, granted, they're very well paid, but the death threats and the hate mail that yet is likely not warranted. So I do like the idea of fans having a bit more empathy for their heroes there.

So you see VR as being some sort of like empathy, empathy tool, really? Look, people ask me often, is VR a machine to do empathy? And my answer is VR is a medium. We would never ask, is the written word or does the medium of audio cause empathy? The answer is it depends on what you do with it. and it depends on the content. And what our research has shown, if you read through chapter three of the book,

we give an honest and thorough review of all the research that's been done in the domain of VR and empathy. And in general, it does tend to outperform media like video or the written word and role-playing, but it's not every single time and it's not on every single measure.

And we still have work to do to figure out what exact type of experiences are most likely to cause someone to change behavior. So there's still plenty of work to be done bringing it forward. Plenty of work to be done, yes. But where do you see VR being then? Where do you see it as being a saying,

you've been doing this for 20 years, what about in 10 more years' time, where do you see it being as a medium? In 10 years' time, I think the goggles will be gone, will be using something called light field capture where you project light directly onto the retina. So the goggles will be light and fluffy

and they'll be, you know, the resolution, and the images will be crisper. And I work in Silicon Valley, and I believe the smart money is betting on what we call Lightfield as a technology. In terms of the content, what I hope is that what we can do is remove the commute. So I am all for face-to-face contact and going outside, but I do believe that humans, we drive to work five days a week,

and we fly halfway across the world for an hour-long meeting. And I believe if we can do network social VR to be good enough so that you get this magic social presence, you feel like you're with another person, then we can, if we can solve networked VR, if we can make it so that in VR, you have what's called social presence. You really feel like someone across the table is in the room with you and you can do things like eye contact and nonverbal synchrony and you feel like you're there. Then I believe that we've solved a lot of problems, climate change. Think about not having to fly halfway across the world for an hour-long business meeting. Think about commuting to work two days a week instead of five, so you don't have to drive for an hour each way. And I think I want us still to go to parties and to go outside and to go to important meetings, but so much of our travel is for non-essential tasks.

If we can make VR feel better than video conferencing and really feel like you're there, I think we've solved an important problem, and I hope that's where we're going in the future. So using VR to sort of help our real world life as opposed to taking away from it? Absolutely. Let's give you more time. Let's give you an extra two hours each day to go outside because you're not sitting in your car fighting traffic. That was Jeremy Balenson talking about the power of using virtual reality to affect people's emotions and behaviours. His book, Experience on Demand, is available now. Thanks for listening to the Science Focus podcast. In our May issue, which is on sale on

the 2nd of May, we take a look at human body farms. These facilities could help forensics learn more about how our bodies rot, which could help them in solving crimes. In this issue, we also take a look at how emotions trick your brain, investigate whether psychological profiling can turn Facebook likes into votes, find out the sneaky ways that social networks are built to make you binge, and discover whether pollution could be leading us to a fertility crisis. Did you enjoy this podcast, if you liked what you heard, then why not subscribe and leave us a review? You can find us on iTunes, ACAST, Stitcher and many of your favourite podcast apps. Thank you for listening to the Science Focus podcast from the BBC Focus magazine team. We're the UK's best-selling science

and technology monthly, available in print and in several digital formats throughout the world. Find out more at ScienceFocus.com or look out for us in your app store. This podcast is sponsored by Name Audio and Focal. The texture and emotional depth of music can be lost through digital sources or poor signal. Name Audio believes you can have digital precision with analog warmth. Alongside French acoustic specialist focal,

Name creates high-end audio systems combining innovation with craftsmanship so you can listen to music, just as the artist intended. Discover more at nameadio.com. Ambition comes in all shapes and shapes and sizes. At First Citizens Bank, we roll with your goals because we're built for what you're building fit for your ambition for Citizens Bank. Lots of places can expose you to identity theft.

Oh no. That's why LifeLock monitors hundreds of millions of data points a second for threats to your identity, which is way more than anyone can do on their own. If we find anything suspicious, like new loans or changes to your financial accounts, we alert you right away, all through text, phone, email, or the LifeLock app. Get the alerts that could make all the difference. Save up to 40% your first year at LifeLock.com special offer. Terms apply.