Embedded - 380: Trending Toward Telepathy

Episode Date: July 16, 2021

Adelle Lin (@Adellelin) spoke with us about wearables, art, playfulness, and getting together in virtual reality. Adelle’s website is touchtech.io. For some VR get togethers, Adelle recommends AltSpace (altvr.com) and Mozilla Hubs (hubs.mozilla.com). Some other remote get togethers: Virtual Burning Man (August 29 - September 7, 2021) A. Maze Conference (July 21-24, 2021, remote) We mentioned the Nautilus jigsaw puzzle from Nervous Systems but actually have the smaller Ammonite one.

Welcome to Embedded. I am Elysia White alongside Christopher White. We're going to talk about art and technology today with Adele Lin. Hi, Adele. Thanks for joining us. Hi. How are you? We're good. Could you tell us about yourself as though we met, I don't know, maybe at a Supercon-like thing? Hi, everyone. My name is Adele Lin. I'm someone who is passionate about integrating the digital and the physical. I'm originally from Malaysia. I guess I would call myself a designer and an engineer. I've lived in Australia before moving to the US. Professionally, I work on

software development and machine learning projects at a startup. But with my own work, I try to draw on more personal experience of having negotiated diverse systems to create projects and artwork to help connect people with themselves and the spaces that they occupy. Cool. We're going to do a lightning round where we'll ask you short questions and we want short answers and we may ask you why and how, but we're not supposed to. Are you ready? Yes. And I can pass on some of these, right? Oh, yeah.

Oh, yeah. Always. What's your favorite type of art? I think it's a hard one. Short answers. It doesn't have to be. Nobody enforces the rules here. I like art that's provocative.

I guess making stuff that makes me think about something I haven't thought about before or exposes me to new ideas. Best animal crossing villager. I hope they don't say one of the jocks. You were testing me there. I mean, you could just list like five. We're not going to be mad. I'm just, there's a penguin one, which is my favorite, but I've just forgotten their name all of a sudden.

Frigga. What's your favorite communication method? Face-to-face. What is your favorite wireless communication method for processors? That's also face-to-face. What is your favorite wireless communication method for processors? That's also face-to-face, usually. I like the 900 megahertz radios. Or radios. I'll ask an easier one.

Complete one project or start a dozen? Start a dozen, complete one. or start a dozen start a dozen complete one favorite fictional robot favorite fictional um a few major motoko kusanagi from ghost in the shell um elita the dolls from the dollhouse i think i really like stories that explore the line between humanity and programming. So you've worked in many different industries, investment banking, architecture, and now machine learning. Is that right? How do you choose which industries and how do you go from there to here yeah i often ask myself that question i guess i do think of life as a little bit of a choose your own adventure and um i learn by doing so you know sometimes i get really curious about an industry and I guess I'd take the time to immersedly investigate it.

So I would say I kind of started from going from high level to a low level of working with things and I think initially what I was doing with banking and consulting was definitely from I felt like more from a high level, already kind of working with like businesses that already had been built. And I was very curious about like how things were made. And it was very, I felt like at a young age, it was very hard to give advice to businesses before really even knowing that much about it myself. And so I started getting into design. From design, I got really fascinated about kind of lower level and learning how to actually build things.

And I sort of ended up kind of teaching myself software and electronics and kind of basically professionally found myself there, right here, right now. You got a master's degree a few years ago going back to school after not being in school for a while what was that like um i've um actually gone to school a few times i have an undergrad my undergrad was in actuarial studies so i was math and statistics based and I have another undergrad in design and the master's that I did was at the Tandon School of Engineering and it was kind of a hybrid of art and engineering sort of course and it was a little bit of a I think at the time it was sort of build your own kind of course curriculum and I took subjects in data science programming and kind of when I ended

up doing my thesis and was looking at mixed reality interfaces and kind of designing for I guess sort of like 3D interfaces and sort of virtual slash mixed hybrid worlds and so I felt like that was that was sort of bringing bits of all my different backgrounds together. Mixed reality, like augmented reality or something else? Yeah, I guess mixed reality was a term at the time, but it was using the headsets, which had an augmented reality layer on top of it. So you could still see your environment, but it spatially recognized everything.

So your virtual elements sat in sort of particular spatial coordinates. Or sometimes augmented reality, well, augmented reality at the time was not necessarily tied to 3D spatial coordinates oh okay but it's it they're all i feel like at the moment everything i think has now been called extended reality or xr and it's everything's kind of like hybridized when are they going to start calling it unreality or surreality well when they decide that they need another term so like the microsoft hololens kind of thing would be an example what you're talking about okay yeah in particular i worked with the hololens for my thesis project i've used a few of the headsets

and i was super super excited about them when we got our htc Vive and then we got the Quest and that was really cool. But I don't feel like it's changed much. I had all of these images of how it would change the world and now it's still exercise games and dancing about. Is that just because I've limited myself or is there a lack of progress? Well, I think there's been progress from a technological standpoint, definitely. The processing power has helped bring a lot of realism into these environments. And I think the last few years, it's definitely been that sort of leap from like the first Oculus was almost unusable in a way. Sophie Kravitz and I built a, what do you call it? It was a game called Wobble Wonder.

And we had hooked up a balance board with a headset. And she'd made these fans that would blow at you to try to simulate, you know, being in an environment. And kind of design this sort of Alice in Wonderland like terrain that you would move through but you really couldn't be in that for more than a few minutes and even building stuff in it I couldn't even test the set for for for more than five minutes without feeling super nauseous and I think that was a big hurdle really that needed you know which the last few years have really helped us get through. And I think you're right. Now it's the question is, you know, what happens with it and what can we do with it?

And I think in many ways, especially over COVID and quarantine pandemic, it has changed things for, you you know for a subset of people and even myself i wasn't really so much into virtual reality that's why i was really um focusing on mixed reality and anything that had kind of a real world um tie to it but the uh for virtual, what sort of really got me was the social VR side of it. Last year was really a way to help connect with communities that I hadn't been able to be a part of for those reasons. And I've heard other friends independently have been spending time in social VR spaces and attending dance parties. And it's kind of helped them reconnect with parts of life that they weren't able to. So I think we've made some strides there. But like I said, I love face-to-face connection.

And so I think it's a nice supplement, but I don't necessarily think it's like the main way we need to be interacting. Do you think it will become more part of our toolkit as engineers and designers? Or do you think it will remain, I don't want to say a toy, but... A curiosity. A curiosity. built in the desert of Nevada. And I've been going for quite many years. And when it wasn't happening last year, I started participating in the BRCVR, which is the virtual reality version. And there are multiple platforms that you can engage with. And they're reviving it again this year because it's not officially happening again.

So I'm kind of working on an art piece for that. And part of my motivations for that is that it is kind of a nice speculative design tool in a way too, because, you know, you can kind of build something that you dream of, have people come and interact with it and not necessarily have to worry about how is it going to work yet. And maybe through that process, learn what people enjoy about it and then bring that back to like engineering and how, you know, how would I actually build that in real life for that? That can be a tool. And I think in architecture, people are using, um, VR pretty heavily for client walkthroughs for, um, being in the space, being able to imagine something before you build it. I think it's, it's a very powerful tool or toy in in that sense and i'm curious like what have um what have you experienced with what have you played with or tried or or thought i do like the dance

games um but in the last year i haven't been doing much of those because i need to be in this reality. It's too easy to go there and not want to come back. So that's my own brain quirk. I really liked the Google draw. Yes. Doodle draw. That was pretty magical, but I didn't. It was tool brush.

And it was pretty neat for being able to build things but i couldn't figure out how to build things and then export them so i could 3d print them or i could share them with one person but i couldn't share them with a group that kind of thing it was it was too much me looking at me, and I wanted to go look at other people. I think you stopped doing stuff with it just before the Quest came out. And the Quest really has driven a lot more things because it's so inexpensive. So I think it's moved on a little bit from solitary kind of experiences. And I've seen some cool things that I wanted to try out recently. They had a whole canvas painting app. It wasn't as complicated as Google tilt brush,

tilt brush, um, cause it wasn't 3d, but you had the whole palette and you can mix paints. And I saw somebody demonstrating doing a whole painting and it's like, Oh, this is like painting. Um,

so I think things like that are, are getting more prevalent. I, I have to admit, I don't think either of us have really tried any of the social stuff because we're not inherently social people. I would really encourage you to come to the virtual Burning Man, the BRCVR. I can send a link to you guys. It's really quite incredible. There were some of the camps that had tilt brush art

in it people had actually made last year someone had made this beautiful whale from tilt brush and you could kind of walk inside of it and you know see the sort of like sketch work up close and and then kind of step away and look at it from afar, but that was sitting in the sort of desert scene landscape. So it was very surreal. Surreality? Surreality, what you were saying before. Is it like Gather Town, which is one of the new apps where you, in 2D, have a space and you go and see people and you can identify them and then you end up Zooming with them and then you can walk away. It's kind of parties online, but it's also kind of. Yeah, it's very similar.

It's very similar in terms of kind of, I guess, maybe like the audio quality, like having audio fall off when you move away from someone and kind of like approaching conversations. And I think, yeah, the 2D applications like Gather Town have you know done that pretty well and i think some of i've actually tried quite a few different ones some of them fade in your video as you get closer fade out and that sort of thing um the the vr the virtual reality ones are a little bit different in the sense that you don't have somebody's video and they are an avatar and it depends on the platform or the, you know, the engine as to like how, what these avatars look like and how realistic they are. But if you do have like a handset, you also get the head, you get the head movement and you also get the hand movement. So it's a little bit more expressive in that way. And it does feel a little bit more like real life. And you could, walking up and walking away from a conversation

feels a little bit more realistic in that sense. But it's very much the same concept. And are there servers or platforms I should look for? Yeah. And this brings that you know like it brings up an interesting conversation that i've had with uh my group that you know we've been building these sort of worlds together is that most of the the servers and platforms that work really well they're all corporate owned so the one that the Virtual Burning Man is hosted on Altspace, which is now owned by Microsoft.

There was another one that I was using where our hackerspace, NYC Resistor, a few of us built a virtual version. We called it the NYC Resistor Clubhouse. And that was on Mozilla, Mozilla Hubs. And they had their own sort of proprietary tools, which you could use to upload your models. And they are hosted on AWS. We actually hosted our own, so we didn't use a Mozilla-hosted one. We hosted it on our own AWS server, but they're, I think, reducing support for that. And so now we're trying to figure out how, if we can recreate those servers on our own, but's i think the aw system is a pretty convoluted and and what mozilla did to get that hosted we haven't been able to sort of reverse engineer that a friend of mine tremel hudson is trying to now build his own server to host um another game

that you know that we can play and maybe we'll try to use that model to move our own worlds onto. But what is nice about, I guess, these hosted platforms is the connectedness. So if you're on alt space, for example, you can really move from one world to another. You can take, so there's like, it's like a little portal thing. You can, so if you have been to a previous world, you can choose that world, grab the 10 people maybe around you, and you can sort of teleport to that world. And so very quickly you can move through all these different universes that different creators have built.

And, you know, it's really quite wonderful. And it kind of opens up from that really isolating experience that generally you associate with VR, you know, where you're in that one application that you've, you've selected and you're, you're kind of just playing that until you quit. And sometimes I can't even figure out how to quit games with, without like restarting the headset. But yeah, I would definitely take a look at Altspace and Hubs. Hubs has a lot of interesting world spaces too. And what's really nice about Hubs is you can also connect hyperlink to websites and stuff like that.

So you can click on an object and it could take you to a website and you could almost visit it. Anything that's browser available, you could do on that i know one of your other interests is wearable devices does that fit in with mixed reality or is that separate i think there is a connection there um it's i'm still in a way trying to kind of work out how my interest in these sort of more tactile in-person technologies relate to these virtual ones where you're kind of in one place. And one of the things I really enjoyed about wearables from an or from an early perspective it was more from a self-expression side kind of using wearables to initially as like an add-on to what you're wearing as an additional kind of like

almost in a kind of costumey sort of way adding to um what you're saying to the world about yourself and your personality. And I then started getting more interested in embedding some of the sensor technologies in. And then where I kind of really got stuck into it is the connected side of it. And I think for me, my interest in tech kind of had a lot to do with when I was studying architecture and interior design. I was thinking about technology as an additional layer, as a, technology is something that sort of permeates as this additional layer. And how do we use that to connect people? And how do we draw connections between each other and stuff like that? And I think I'm often an awkward person.

I come across as very sociable, but I'm a little awkward. And, you know, and I think a lot of us in this space are. And so sometimes playful technology can help. And the same with, you know, with wearables. It can help strike up an interesting conversation. And it can also kind of help you feel. And where the wearables got interesting for me was um i started this project called tiny gang and it was this idea of sort of like like

basically like a gang you have your you have your jacket and it kind of means you're part of something um and with the with the sort of radios where anybody could kind of be on that frequency. It was this kind of romantic idea that we could, anybody could, you know, buy this, buy the radios, you know, use our code, put it on something that they were wearing. And if we, you know, met in a space, we may not know each other, but our jackets are suddenly, you know, a connected thread. And we suddenly had that connection. And, you know, I think that's where some of the magic of tech has been for me. This tiny gang, as I look at it on your website, it's jackets that have LEDs.

And I suspect the LEDs light when I'm close to somebody else who has the jacket, as you said. Yes. But wouldn't I know if they had a jacket with a patch on it that said, I'm a huge nerd. Would you wear that? Realistically? Yes.

Yeah, totally. I mean, it's the exact same. It's, it's kind of derived from that concept right that you can kind of wear um the the symbol of your gang or something and the idea would be that people would would contribute content to this project and so the what we do do with this is, um, they light up, but they are, you do send information about your, yourself to the other person. And so, um, at a base level, um, you know, color is a very easy one. So, you know, I pick a color, another person picks a different color. And so when I'm just on my own, my jacket is this one color, let's say pink. I'm often in pink color. And, you know, my friend is in turquoise. And then when

we're together, we start, my jacket starts making turquoise patterns and their jacket starts making pink patterns. So when you're in a group, you become more and more colorful because of the different people that are around. And so that's one of the things that we transmit. I had a version which had an accelerometer and we were using that. We're transmitting accelerometer data across. I've played with the heartbeat one. I Haven't used that in a group yet. I think it's harder to get like a good heartbeat sensor in a comfortable. Yeah, so that was the initial concept. Are you going to productize it or is it a do-it-yourself for other people? It's in a do-it-yourself phase at the moment. But I have found that it is very difficult.

The most successful version we had of this was a friend, Elliot, he had come up with this design to putting LEDs in an Adidas jacket. And he kind of had a really good kind of how-to of how to do that. And there were about 10 of us that were really excited, and we made 10 similar jackets. And me and my partner, Matt Pinner, we made a set of ESP32 pre-programmed boards and sent those out with instructions of how to connect those to the jackets. And that ended up quite successful because everybody had the jacket and then they had the electronics. And I think the tricky part has often been people seeing what we've got and we like often use leather jackets and studs

and things like that often get stuck with the creating of the, the jacket with the studs and that kind of thing. So actually wearable clothing can be quite labor intensive unless you're kind of already doing that. So I think that side of things we're trying to figure out how to make that a little bit easier so that people can kind of um pick that up in a more off-the-shelf kind of way i know you've worked on some wearables i have chris too um we both worked for fitbit um i worked a little bit on Jewelbots, which reminds me of what you're saying so much, because it was friendship bracelets that would glow when you were near another person who had one. And one of the features they wanted was to be able to send secret messages. And I was just like, what if they're in class?

You don't want to make a class note, the class notes feature? You know, I'm such a goody two-shoes. But the idea of being able to bond through the technology and being able to meet people who have similar interests so easily was very nice. They used BLE and a mesh network, which technology-wise at the time was new and tough. Even now, I think it's pretty tough.

Yeah, definitely. I like all of this because it's kind of trending toward telepathy without being telepathy, but it's kind of an unspoken you know, it's a bonding thing that doesn't have to be verbal and I appreciate that. But it's also a discovery mechanism, right? Yeah.

A big part for me was the non-verbal. And often, you can connect with people without having to, especially if you don't speak the same language, what are ways you can have something in common. But I'm actually kind of curious, I guess, because I haven't productized um hardware before i'm really curious like what were the challenges do you guys feel in each of these and especially from the connected perspective um the technology itself uh ble changes quickly and while it is mostly backward compatible it does require you to have a smartphone

and not everybody does not every smartphone works very well with bluetooth and you have to make them all work somehow and the battery situation is always i mean that for any wearable, that's got to be a consideration. So you can put on a jacket, you can put a battery pack in lower back, and it's not that heavy. It probably isn't that hot. But if you're doing something wrist-based, you have to be very conscious of weight, and that is batteries, and then that affects what you can do, and it's all a horrible downward spiral. be and is one button okay if we make it so you have to hold it or tap it multiple times and trying to get things that work for a lot of a diverse group of people that was also a challenge

especially with sensors and just with fit kind of things like some people don't like having things on their wrist and some people don't like certain materials on their wrist which you know okay so do you just ignore those people or do you try to accommodate them and it's just a lot of a lot of issues especially when you're trying to reach a broad a broad customer base i was always mad at fitbit for their charging strategies because it was different every product it was different every product but every was different every product, but every product you had to take it out of its mounting for all the little ones. And so I wanted to build them into jewelry, but I had to take it entirely out of whatever it was I put it in, which meant I couldn't use polymer or anything that wasn't rubberized. Yeah, I think that's just

the kind of thing that happens with real miniaturization where you're trying to make something tiny. It's, okay, well, it has to be tiny. So we're just going to have to force a charging thing in some way and it's going to be off in this corner because that's the only place the electronics fit. So, and that's, yeah, that's another thing with productization, right? It's, we need to make something that's as small as possible. The battery lasts as long as possible.

It's as cheap to make as possible. And then all sorts of things just kind of fly out the window at that point and when you're doing a distributed system like this do you have a single eye in the sky or do they really talk to each other and that's a pretty big difference um having them all talk to each other requires more power because they have to be on more. But having a single thing they talk to, that means they probably have to talk to some sort of router-like things, like a smartphone. Sounds like the blockchain discussion. discussion uh no i think i think you're right um yeah when i was at the when i was at intel i was in the wearables division there and yeah there were a lot of very similar challenges a colleague of mine made a yeah pretty nifty ring actually that did a lot of things but

couldn't get the battery like couldn't get the battery size small enough for a woman's sized hand there are definitely batteries were a huge kind of source of development at the time especially for yeah rings and bracelets and things like that. Yeah, they've had some, like Ringly. Those were cute, but it was clear they were really pushing. It didn't quite do as much as I wanted it to. And technology-wise, I could see why. They were trying to make it so small, it can't do everything. That didn't have a display on the device. So you always had to interact with it through your smartphone. Yeah. Yeah, I had a Ringle too. It looked really nice. It's a balance, you know? It looks really nice, but it's not as functional. And some of the functional ones, they don't look as nice,

but they last for a month, which is pretty cool. It's, yeah, it's all just trade-offs. I mean, that's all the engineering part. Yeah, also user interaction design. I did a few studies where yeah we were trying to teach new gesture interactions to users for some of these prototype devices that we had and and it was almost like you know teaching a new language and the question is like you know how many new languages do people want to learn and um like I tried to create a sort of ring-based input for a head-mounted display and, and kind of had to, you know,

sort of even just like looking at the usual interactions of like scrolling up and scrolling down as a specific example. You know, people got really confused because some people were used to scrolling their mouse a certain way or they would relate it to the way they scrolled their phone. And that took a while to kind of get people used to. And I was kind of surprised at how that was one of the things that I had to spend a lot of time with users between different users. It reminds me of the scene in Independence Day where they have a Post-it note that says,

up, down, left, and right. And then as soon as they get in and they try it, it's backwards. And so they just turn the post-it note upside down. And I was like, there's so many times I want to do that on a UI. And when I switch from using Linux to using Windows, there's something different about scrolling. And I just try all the options until whatever I'm in works. But it is confusing, and if you tried to add another set of choices for things, but people are already being trained surreptitiously. And so you have to kind of think about,

okay, I'm going to do this, but have they been using something else that's similar enough that they're going to have muscle memory that's backwards or wrong? And does that limit you from making kind of more radical choices? Yeah, definitely, right? making kind of more radical choices yeah definitely right like um and the dominant the dominant technology tends to drive these um user interactions that we're we're used to and there's always like the like you've been switching between an android phone and

a ios device the different interactions interactions on this one type of screen, and people have a hard time switching between, but it's these two companies that have kind of now dictated the gestural language of using a smartphone. I don't think it's been that sort of way yet with wearables. I don't think there's kind of like a dominant tech there yet. No, I haven't seen any. And I've always thought that if I was going to try to do a gestural thing,

I would look really hard at ASL. Because it's already a sign language. And if you could get some crossover it would be very nice just because that's already done and it's got a grammar and it's a language language yeah there are

I think there are quite a few projects in machine learning trying to study ASL languages which I thought were quite fascinating yeah there's something to the translation that you can have an avatar

go from English to ASL instead of having a human interpreter I'm fascinated by that I think that's just the coolest yeah yeah I think that's just the coolest. Yeah. Yeah, I think there's some definitely very interesting technology coming out for accessibility,

especially with all the conferences that we've had virtually recently. I've noticed there's been a lot of good closed captioning, auto-captioning, sign language has still, I think, been a human there. But yeah, definitely huge potential for machines to help out with that. When we talk about virtual reality and virtual worlds, do you consider things like animal crossing to be part of that or is it separate yeah i i i kind of have a a loose looser definition in terms of virtual worlds and so i think animal crossing definitely for me falls into that um i guess especially from the sort of social perspective where you can have multiple people in the world itself. I mean, it itself does have like a 3D kind of element to it. And, you know, for all intents

and purposes does sort of fulfill that virtualness from a environment perspective, from a story perspective. Yeah. And I think it's become a very socially relevant platform too. Um, you've done actual things in animal crossing. I mean, beyond dressing up your villagers as they are, uh, trying to fish. Yeah. Um, yeah, last year I, uh sort of like a virtual protest in Animal Crossing, sort of during the start of the Black Lives Matter movement. And when the Black Lives Matter movement started, I think in 2013, but there was a resurgence of it last year and we were all kind of stuck at home. And I was looking for ways to

in a way contribute. And I was spending so much time in Animal Crossing. Initially, I had just wanted to build a memorial and a tribute to, you know, all the black lives lost. And, um, and so I started creating this memorial and in thinking about what else could do with that, I wanted to create a fundraiser and then that sort of like turned into, um, a bit of a protest. And, uh, yeah, I had help from some friends, uh, in terms of like creating the artwork and also kind of like thinking about like how actual protests would would work and we ended up creating a pass that people could walk through when they came onto my island and we did this sort of processional march and we had sat down at the memorial and kind of chanted and it

was very neat in the way that it felt very life. Like I've been to a few protests myself and, and I was kind of surprised at how without the, I guess the loudness and the violence sometimes involved with it, but the spirit of it and the community, it really felt quite realistic. And I thought people would be coming in and leaving really fast, but, you know, we ended up sitting for 45 minutes,

an hour to a time in a group and kind of expressing how we were feeling and all the frustration. And it was super cool that people from, like, all across, like, different parts of America had come. I had a couple people from Australia come. It was a great virtual gathering in that sense. And I know there have been other, like in Hong Kong,

they've been using Animal Crossing for protesting as well, too. People have actually gotten arrested over it. So I think in some places it's actually quite an effective tool. You've also done art, big art. You worked to light the London Bridge with Leo Villareal? Villareal, Leo Villareal Studio. Yeah, I work with Leo on the software side of things. Um, yeah, he's, um, an artist that worked on the Bay lights.

Oh, okay. The Oakland Bay bridge. Is that what we're talking about? Yeah. Oakland Bay bridge. Yeah. That was, I think one of the first kind of, um, large urban scale works that he did. I wasn't with, uh, their studio at the at the time. But yeah, I've, you know, I found it really mesmerizing. And so when an opportunity came up to do some work with them, you know, I was very excited. And he got this commission to work with a nonprofit in London and working with the city and the different boroughs because each bridge there is owned by two boroughs so for each bridge that we worked on there had to be

negotiations and planning approvals from all these different owners so it was a much bigger project from that you know that sort of like bureaucratic. But yeah, I worked on building custom software that he uses to program and light up the bridge, light up the bridges, and we have something kind of basically working, hopefully, for the next 10 to 20 years. So that was definitely a really fun project to be a part of. Those big installations must be, I mean,

people make LED light up things all the time with Arduinos and WS2812s and things like that. What's the challenge in moving from, okay, I can make some lights blink, to I can make giant lights on a bridge blink? Does the technology not really transfer or is it kind of just scaled up i guess the kind of um the programming paradigm is very similar but the um the scale of it definitely from, from the hardware perspective. So you can't really use these, um, you have to use kind of architectural grade, um, outdoor,

um, outdoor sort of, uh, fixtures that can kind of withstand all kinds of weather and stuff like that. And installing those requires a construction team. There were, um um and it even needed to be more specialized the electricians needed to be abseilers as well so they needed two types of licensing to be able to install the lights which was kind of insane kind of watching these abseilers um climb over bridges that were you, some of them hundreds of years old, um, installing these fixtures onto it. Um, and so then it's about, uh, you know, being able to speak the,

the protocol that that light fixture is using. So like Leo studio has, um, different sort of, um, different protocols that, that then connects that we can use the software to then connect to these different fixtures. And every time there's a new project, it's a different protocol. So, you know, so it's always about kind of figuring that out. But we're, you know, from the software perspective, we try to build tools that can, that are more modular. And so that can be kind of used to connect to these sort of different systems. And it needs to be a little bit more robust in that we have to be able to monitor. We have to be able to monitor them.

So we have kind of a bit of a monitoring software that we build and beefier computers. Yeah. It sounds like going from wearables that you do yourself to production, very much the same sort of problems. Yeah, yeah. So a lot of what I've done in a way had kind of touched on production and kind of the, yeah, production work is a whole different beast and skill set. And, um, and I kind of learned a lot of that actually at Intel where we were doing, um, kind of stage scale, uh, kind of like stage demos and, um, had, um, some production people

come work with us. And I learned a lot from them about how to kind of doing everything live. Kind of live data is, in a way, a very different beast to live streaming data, which is also kind of what I work with as my current job, to, you know, kind of more static queries and, and, and the sort of like big data problems. But, um, yeah, they're, they're both really interesting. And I think that has actually helped with the way I think about wearables. And for example, I'm working on a project with Sophie Kravitz right now where we're, um, making our own, own um radio boards and i'm thinking about using that uh you know at a scale of you know 100 people and again kind of back to the what size battery can fit on a necklace and and that sort of thing but then now kind of like thinking about

from a like production perspective um you know what does that kind of need? How does that work with the scene and the environment? And, you know, what is the audience participation and the affect of that? And how can the people be part of the lighting, part of the environmental lighting, for example, are kind of things, you know, that I bring back to it now from working on kind of larger scale productions. So I think it's good to be able to work in both realms. What is a maze fest? Changing subject entirely. Oh, yeah, I'm glad you brought that up. up actually it's uh maze festival is a uh it's a games festival or um yeah i guess it is called it is a festival um in berlin that i got involved with a few years ago actually through my friend phoenix perry we collaborated on and um i guess an interactive musical artwork piece um and we actually like built it in berlin for the festival but it is an independent games festival but it it's a really cool one because it's really pushing,

it works really hard to kind of push the idea of what a game is forward and opening it up to, um, you know, playful media performances, um, you know, software experiments, um, anything sort of like creative that has a bit of, either uses game mechanisms or maybe is built with game engines. And so I got invited back this year to be a judge on the panel and basically have been playing games nonstop for the last two weeks. Oh, no, that sounds awful that's awful uh well not non-stop but like whenever i have free time and that also involved kind of watching

performances there is um one of the pieces is a um this person built their own game so that they could do a hip-hop performance in which was you know really um you know talk about building your own environment and kind of like the virtuality and combining um performance with it and it isn't something that anybody else can play and we you know we um interact with it by watching but that was in one of the game's categories. And yeah, so it's a really interesting festival that's virtual this year as well. And I definitely encourage you to check that out. It's coming up next on the 20... Not the exact dates, but like 22nd to 25th of July.

Something somewhere around there. I can also send you a link to that. We'll put it in the show notes. Yeah. And are these mostly electronic games? That's not the word I'm looking for. Computer games? Digital.

Digital. That's the word. Yeah. I would say maybe 70% of them are fully digital, but a lot of um, also kind of in-person like immersive games. So kind of think, uh, escape room type of thing. Right. So all of those kind of categories come into, come into this, but because like, especially last year specifically, um, all of the physical stuff isn't, um, wasn't really available. So a lot of these sometimes in-person things kind of took on a digital form. Some of the card games that were in there had apps that also used the apps to kind of help do the storytelling mechanism. There was one which was kind of like a Zoom multi-person game where they kind of had actors, it was kind of like a role-playing sort of thing, where they had actors

kind of talk you through different puzzles. It was around the question of urban development, you know, brought people together and, you know, kind of gave them different roles. Like maybe you're the urban developer and you're the resident, like, and had kind of interesting discussions around it, um, with a sort of like game kind of mechanism to move the story along. Um, and that would last for about an hour. So there's, um, yeah, one of the games kind of had a, um, a blinking mechanism. So kind of used, uh, got you to calibrate with your camera and, um, got you to calibrate your

blinking and used a kind of like blinking or closing your eyes or opening your eyes kind of mechanism to propel the story forward. And was done in a very um natural sort of way so there's there are a lot of different kind of analog physical inputs to the sort of like digital kind of virtual space that we know now it's funny how games have gone from keyboard input to joysticks to the incredible revolution of accelerometers and gyroscopes with the Wii U. We don't even have those games anymore. Why don't we have golf anymore? It's a golf game. Yeah, but not like...

You moved on to VR. You just have to do it in VR. Yeah, yeah. And I mean, the mats and, the head movement tracking. Facebook is doing a lot of development. A lot of their machine learning is using their Oculus headsets and tracking everything. Do you have any thoughts for what you would want to have for the future of game inputs? What about you, Christopher? I think I'm becoming too old.

That's a silly answer. I enjoy controller-based games these days. I mean, I do enjoy some of the VR stuff, and I think that's less about creating a new interface and getting rid of interfaces entirely and just making it your natural movements, which i guess is a type of interface but um i mean that's the accelerometers and the gyros in the wii u it went from yeah something totally artificial to something vaguely similar and the blinking sounds like something i I mean, that's something I do anyway. I'm not sure how it would make a story go, but okay, yeah, I could imagine.

Yeah, well, I guess that's sort of like, you know, replace, I replace with clicking. So I guess anything that can replace clicking, you could replace with a different kind of sensor input or, you know, different kind of gestural input. Gosh, it's really hard to think about what hasn't been done. Well, yes. Yes. Let me put you on the spot. Could you please invent a whole new technology right now? I mean, I've seen licking games where you lick something. That is not going

to be the show title. There's a game designer, Kaho Abe, she did a lot for this medium and kind of bringing physical computing to games and had some really interesting game mechanics around that.

I think I've seen ones done with phones. I've seen a toaster am i making it up you know i've seen a game made with a couch um it's going to be hard to product yeah i mean i think well anything anything that any objects or interfaces that you're you're passionate about you know you can kind of think about them like it's more about thinking about different playful ways that you're passionate about, you know, you can kind of think about it. Like, it's more about thinking about different playful ways that you could approach something. And so I think there's, you know, definitely a lot of potential for people to create new mechanisms,

and they may be more towards, like, the artistic side of things and maybe less of a productized kind of game. I think it's definitely a rich landscape to be engaged in. You've said playful. What does that mean? I mean, is it one of those words that everybody knows what it means, but nobody can define it? it oh that's a good question yeah it has kind of become this sort of catch-all in a way of if you don't have like a um you know a pure like win-lose mechanic but it still uses a um you know some come some kind of, I was going to say playful mechanism. No fair using the word to define the word.

Yeah. You should look up the, the dictionary definition of play, but, um, yeah, I think I used for, for me, I use playful when there isn't necessarily a win-lose dynamic, I would kind of say, but still has maybe rules or rewards, different ways of kind of interacting that are parameterized, let's say, but don't necessarily mean you could be a winner or a loser. I'm really glad I asked because that isn't what it means to me at all. And so now I really want to understand more. And I thought about asking what is a game, but I guess I sort of did with the electronic part.

So it's something you do that doesn't have a win or a loss. Is it only when you're intending to play, or can it be playful without rules? Where am I going with this, Christopher? I don't know. I'm pretty confused now. Yeah, I guess. I mean, well, how do you define, like, what feels... I'm trying not to... My brain keeps going to whimsical,

which is not the right thing. Fun. But sometimes playful implies whimsical to me, where there's something either humorous or, yeah, something that makes you smile. And it can be concentrated. I mean, it can be hard and still be play and playful. Yeah. But it requires that happiness for playful.

But also maybe discovery and curiosity. Yeah. Would you consider a puzzle playful? The one we did, the Nautilus one from Nervous Systems, that was so fun. Oh, what's that one? It's cut into really hard shapes, and so it doesn't act like a normal puzzle. Right, right.

Yeah, I know they've been making puzzles. I should play that one. But yeah, so a puzzle could be potentially playful. Sometimes I think you some of the early stuff that Phoenix and I were working on kind of like turning different environments

or different like objects into musical instruments I would call that playful but there isn't necessarily like an outcome that you're trying to get to. If you have three or four people playing musical instruments or playing interactive objects that make music,

those people decide what is the music or the sound that they create together. So those may not have rules, but the rules are kind of built into the input of the object itself. And there's only, you know, maybe so many ways that you could use that. For example, another,

I guess I would say like playful game that I built a couple of years, three or four years ago now, it's called Starcatcher, where we had VR trackers on nets and it was like a canopy of LED, like programmed LED lights. And they kind of look like falling stars and you could go and catch them. And so when you put your wand underneath one of the strips, if the light had hit the bottom of the strip, you would effectively catch the star. And that sort of did have a little bit of a level element to it

where every star you caught filled in a constellation and you could kind of advance levels but i wouldn't necessarily you could totally ignore that mechanism for example and it didn't you didn't need that to kind of advance um in the game so i would still call that more playful than you know than some of the kind of more like, you know, shooting games where you have to like shoot 10 people to finish a level and, you know, you either win or you die. Does playful mean without a goal? Not necessarily to me. No, I don't think so. Because I can imagine getting to a goal.

Yeah, I definitely think you could have goals. But it shouldn't be a goal that makes money or does useful things. I think over-defining it is probably not really. Christopher's bored of my... No, I just don't know that we're going to get to anything satisfying. Etymology of play is... Okay, I'm not going to look it up. You definitely want to bring in a games professor, I think, for that conversation.

Is defining playfulness playful? Well, as long as we're having fun, actually, I think that's the rule. Here. So we could keep going. The listeners may not enjoy that. And actually we probably should close up. We're about out of time.

I'll consider play anyway. You guys can do other things. And Del, do you have any thoughts you'd like to leave us with? I guess maybe reach out to me if you want to have further discussion about any of the things we talked about i'm i love talking about

marbles games play machine learning cameras whatever what kind of machine learning do you like to do? Most of the stuff I'm doing right now is computer vision. And I did work with some musical machine learning projects. Those were really fun. I'd love to do a bit more of that. But that doesn't really pay the bills. Our guest has been Adele Lin, creative engineer and international puppy herder. Thanks, Adele. Thank you so much.

Thank you to Christopher for producing and co-hosting. And thank you to Sophie and Phoenix for suggesting Adele. And thank you for listening. Oh, and Patreons, thank you for Adele's mic. You can always contact us at show at embedded.fm or at the contact link on Embedded FM. We will make sure to have Adele's contact information probably via Twitter and her website. So now a quote, well, of course not a quote to leave you with. So from Middle English, play comes from Old English, which still is kind of play. And it means quick motion, movement, exercise, sport, game, festivity, drama, battle, gear for games, implementation of a game, clapping with the hands and applause. So I kind of understand why we can't define it. Nobody ever has.