Programming Throwdown - Augmented Reality

Episode Date: November 21, 2020

This month we are speaking with Connel Gauld from Zappar about Augmented Reality. Connel is an AR genius and blows our mind with details on how AR works under the hood and how it's easy for anyone to make AR apps and websites. AR is the next great platform, so it's good to get in early. Give it a listen and tell us what you built! Show notes: https://www.programmingthrowdown.com/2020/11/episode-106-augmented-reality-with.html Teamistry: https://link.chtbl.com/teamistry?sid=podcast.throwdown ★ Support this podcast on Patreon ★

Hey everyone, so we have a really awesome episode. I think a lot of folks have tried, and we were just talking about this before we started recording, have tried virtual reality and have even tried augmented reality and may not have even known it. That's what it was called. And so we're going to really dive into this feature. I think a lot of folks have called VR and AR kind of the next smartphone, right, or the next platform. And there's a huge amount of potential here, and it's already being used by tons of people. And so we're going to try and learn as much as we can. And we have Conal here, who is the CTO of Zapar. And yeah, thanks a lot for coming on the show, Conal.

Thanks for having me. It's a pleasure. Cool, cool. How are you doing? How are you making do with these crazy circumstances? Yeah, not bad. We're based in London, and so we had a big lockdown earlier on in the year. And during that process we of course as a business moved to complete working from home and I've actually just taken the decision to continue to have a completely flexible working scenario for people so some of us will be going back to

the office who choose to do so and some of us will be working remotely for the foreseeable so yeah it's been interesting. But, you know, we've not really suffered any loss in productivity as a result, which has been great. And it's obviously not the case for everybody, which is unfortunate. Yeah, that makes sense. So as a company that is centered around augmented reality, were you able to, you know, with this kind of new environment,

like find a new product or were you able to sort of see some area that has been underserved by augmented reality since so many things now have to be done virtually? I think so. But I would say that they're not necessarily sectors that we hadn't seen before, but more that are growing faster as a result. So to take one example, product visualization in the home, augmented reality is a tool you can use to enhance product visualization in the home. And obviously, if people can't be going to stores to see and touch products in real life, then businesses are looking to see how they can help users experience what those

products would be like. And you don't get all the way with something like AR or indeed with VR, but you get better than just seeing it on a page on your phone, for example. So there's a class of things like that. There are specific niche use cases which are kind of interesting. So one of the mechanisms for augmented reality which people might have tried before is face tracking so that's when you take content um and apply it to a user's face in a camera view so you might have tried face filters in snapchat for example

an example of this where you you put some 3d model around the face or head or you'll distort the head in some way um how to fit a mask properly, for example, is a use case where face tracking helps with that. You know, you can show users on their own face that the mask should cover their nose as well as below their mouth and how it should be attached around behind their ears and what have you. So there have been some interesting niche-specific use cases like that. But I'd say that on the whole, augmented reality as an industry has so many sectors, so many use cases that in some sectors that will have been affected by an economic downturn, and in other places there'll be opportunities. So a mixed bag, as is I'm sure the case with lots of industries at the moment. Yeah, that totally makes sense. I remember reading something which said

exactly what you said, which is the coronavirus situation kind of accelerated trends that they were expecting to take 10 years. We're accelerating them so that they happen in 10 weeks. I know personally, I started buying all my clothes online, which is something I'd never really done in the past. And yeah, I think I think a lot of people are kind of changing. And you're starting to realize the like latent advantages. Like one of the advantages I found is, you know, I mean, Patrick and I are both kind of like tall, lanky people. And so I like to buy especially shirts that are

medium tall, which is a very rare size. And now that I'm online, I can always get, you know, get that size and any style. And so that almost kind of trumps being able to try it on in the store because because you get the fit you want. And so, yeah, I think a lot of these trends, it's really going to be interesting when this is all over, like what remains, like what will be popular. Yeah, I was having a similar discussion with one of my colleagues the other day in terms of looking at things that have been huge changes in our history. And, you know, the things like 9-11, which obviously changed how we travel by air, for example, and, you know, mindsets and attitudes in lots of ways. It seems to me like this sort of thing will just eclipse something like 9-11 in terms of the ongoing changes it has for our society. So it's going to be super

fascinating to see what happens in the next like five years when it comes to working and being yeah and i i think that that you know dovetailing back you know i think i think ar and vr are probably like one of the areas that are going to really explode because of this um like actually just tying these two examples i mean it'd be awesome if i could actually see what that shirt looked like on me um before i bought it right and that's something you know, isn't available at at least any of the stores that I checked out online. Yeah, absolutely. Even just being able to see the physical size of something in your environment, even if it's, you know, not quite exactly the right 3D rendering to make it look exactly like your shirt really would, just being able to see, you know, either on a table next to a real one of your shirts oh that's

how it compares size wise it's very valuable yeah it's a good idea yeah totally yeah cool awesome interesting ways yeah so so why don't we explain um you know people have heard the term vr uh probably a lot of people have even heard the term ar but why don't you talk a little bit about, you know, what does virtual reality actually mean? And what is the difference between that and augmented reality? Sure. So virtual reality is a class of technologies that replace the environment that you're in as you experience it. So an example of that would be where you put a headset on, and that headset has a screen in front of your eyes. And as you look around your room, you experience a different environment. So it might be a planet far away or it might be a shopping environment or, you know, it could be anything.

The point, I suppose, with virtual reality is it's just not the one that you're actually in. It's some virtual content um so a classic sci-fi example the holodeck in star trek is a you're almost idealized virtual reality environment where people can walk around and it's almost like they're actually at the place that's being represented yeah yeah augmented reality uh is halfway between virtual reality and our real world environment what augmented reality does is it starts with the room that you're currently in or or you're um in the case the case of face tracking as we discussed it starts with a camera view of your actual real face and then it takes virtual content and adds that into that environment

as if it's actually there present with you and so so, as I say, that could be, you know, a hat on your head that you're previewing with a face filter experience to see what it might look like. It could be pointing at your living room and seeing a virtual chair in that space in order to see what it would look like, where it's actually be in your room. We can also track things. So I should explain, I suppose, tracking is the process of working out where something is in your environment in order to be able to attach content to it. So face tracking, it's an algorithm that finds where in a camera image your face is

so that then when you come to put 3D content on top of it, you know exactly where it should be to line it up on the screen. And so you can track things like faces, you can track things like surfaces or the environment around you, and you can track images and other things. For example, people might have used Connect, Microsoft Connect. It's this camera you could get for Xbox.

And if you stood in front of the camera, the Xbox would be able to know where your arms and legs are in the world. And so there would be gameplay experiences where you move your arms in order to move your character in the game. And so that would be a body tracker or a skeleton tracker, if you like.

And so augmented reality is a huge class of these types of technologies, and there's lots of different ways it's implemented. But that core principle is it's your environment, but some additional virtual content is added. Cool, that makes sense. And so I think virtual reality is, there are definitely some, let's say let's say black magic or something that

requires definitely expertise to understand how, you know, when you turn your head, how it sort of knows, but you can kind of intuit it from things like cell phones where, you know, there's clearly a compass. And if you're on, you know, Google Maps or Apple Maps and you turn the phone, the point on the blue dot, you know, attached to the blue dot turns, right? And so it's more intuitive. Augmented reality is the one that seems like just totally black magic. Like, how does it take this picture and realize that there is this surface that's going in this direction, you know, a table that you can put something on? How does it distinguish that from the wall?

I think that is something that really just blows people's mind. And, yeah, what would be, you know, your answer to somebody who asks, like, just how can something like that work? That's a really good question. And I should say our CEO at Zapper, Casper, he is a fantastic guy, but by his own admission, he's not like super tech. He's definitely tech savvy, but he's not a programmer or anything like that. I had exactly this conversation with him where we've tried to say like uh how how does it go from being like pixels on a screen to being an experience that understands the world you're in and i suppose underlying this it's the same with lots of other really complex systems where if you actually break it down into

a kind of pipeline of of constituent parts where each each part in that pipeline is kind of more easily understandable then together when you build it all together you end up with something that that seems like more than the sum of its parts um so i can give you an example for some of those parts in an ar circumstance uh so the very first lowest level we have for example a stream of camera frames coming from a camera on a mobile phone and so that's just like a stream of camera frames coming from a camera on a mobile phone. And so that's just like a stream of image files that you might get on your computer, like PNGs or JPEGs. At the end of the day, it's a file of data where each little bit of that file is a pixel on the screen of that or in the center of that camera, if you like. And so the very first stage in an AR process is to take that stream.

And I should say different algorithms for different types of tracking work in different ways. But let's say that we want to track the surface of our table. What the first thing that will happen in lots of these algorithms is it will take that all of those pixels and reduce the amount of data that's involved there into a much smaller set of data but with that data has a bit more meaning so the first thing you might do is look for corners in the image or areas that are high contrast little features in the image and it's the sort of thing that your eye would be drawn to if you were looking at a table so there might be a knot in the wood in the table or there might just be kind of grains in the wood. Or if it's a marble table, there would be marbling, you know, just little bits of interesting visual content in that image.

And so we have algorithms that can look through the pixels and say, you know, this little group has more high contrast pixels than other areas. And with some clever choice of how we define those elements, we end up with being able to say, you know, at this point in the image, there's something interesting. At this point in the image, there's something interesting. And indeed, if we look between the frames, those are relatively consistent. So that the grain of, or the knot in the wood in this frame versus the knot of wood in the next frame

looks sufficiently similar that between those frames, when we're calculating what's going on, we can say that particular place in the image has moved from there to there between these frames. And then what you typically do in the next stage up is you take a ton of these matches or correspondences between these frames and you do some effectively geometry to work out if that was there in the last frame and that was there in that frame and it's now here and that one's now here. What has the camera done? How has the camera moved in order to make sure that those places all match up between the two

different frames. And then the complexity is just from there, you know, how you work that mass out, you end up with a set of maths that matches very similar to what you get when you're like rendering 3D content in a computer game. And so we have matrix math or linear algebra. If you studied that at school or university, we end up with matrices that represent the movement of objects and cameras. And those are exactly the sorts of math that we use then to say, now take this 3D model and position it into this scene as if the camera had moved between these frames from here to here. So the idea is like if you have a single picture with a knot in a table,

then there's kind of like a cone coming out of your eye or out of the eye of the camera, and the knot can really be anywhere in that cone. I mean, the knot could be, you know, hundreds and hundreds of feet away if this is some gigantic table, maybe you're at a table conference or something. But, you know, it's somewhere in that cone. And then if you were to move your head, or if you were to, you know, close one eye and open the other eye, if you were to move your head or if you were to you know close one eye and open the other eye if you had stereo vision right but you can kind of accomplish this by moving as well now you have this second cone that sees the knot and so the knot actually exists you know at the intersection of those two cones and so you're kind of doing this over and over again yeah exactly and sometimes um sometimes if you know enough about your environment, you can work things out without having to track things a single frame of a video be able to say, you know, that that image is here and it's positioned and rotated in this way in the camera frame.

But most of the time we use the fact that we have multiple frames coming in from the camera over time to to track an object smoothly in the camera. And I should say that these technologies are the types of technologies that were used in computer generated graphics for movies. You know, when you have the T-Rex in Jurassic Park running along behind your truck, the computation and the processing that the computer, the visual graphics and effects people do is a kind of similar thing for working out the environment, working out the motion of cameras between frames in order to then apply the same, but other way around motion to the dinosaur so that it moves in the frame in the correct way. The difference with modern augmented reality that people might use is that we just have to be able to do that processing very quickly. So it has to be 30 frames every second in order to get smooth video that would come from a

camera image and in some cases if you if you're wearing headsets for example you actually want 60 frames per second if you can you know actually quite a lot and high uh high speed computation has to happen in order to make that work cool Cool. That makes sense. So I think, okay, so we have these points. So we've discovered, you know, here's the corner of the table. Here's some knots in the wood. Here's some marks on the wall that we can track, right? And we're able to do that in real time, which that by itself sounds really, really hard. I have to admit, I don't know how that actually happens. That's pretty remarkable. So then how do we go from that to some 3D geometry to say that this is a surface that has a normal in this direction and so we could put a shirt on it?

We wouldn't want to put a shirt on the wall, for example. Yeah. Well, so the answer is there's different ways that you can do that. Some of that, so surface detection, for example, with surface detection, if you want to say, like, the table is flat in front of you, normally you would also supplement this algorithm with things like gyroscope data or accelerometer data

from your device. So in modern smartphones, they have a very good gyroscope so that the phone can tell very well when small rotations of the device happen. So it can know, oh, the user's turned the device a few degrees this way. They also have not normally quite as good a sensor, but still good enough to kind of get things started, an accelerometer, which knows which way gravity is. So it knows relative to where you're holding the device, gravity is in that direction. And then a combination of that sensor,

a combination of the gyroscope knowing, oh, well, you've now moved the device by this number of degrees from that gravity position that we understood. And you add that into the images that we're getting from the camera. You can build, we call this sensor fusion. So you take all of these sensory inputs, including the camera, and you fuse them together to build up an understanding of that environment. But the process of doing surface detection, so knowing that your table is in front of you and what height it's at and

the, you know, the extents of the table, so how far it goes in each direction, that's quite a complex process that is, you know, a lot of research time, I suppose, has gone into how you solve that problem, especially how you do it real time. Yeah, I mean, that sounds that sounds insanely difficult. I don't know how I have no idea how that works. But yeah, I mean, that's it's definitely so it's called surface detection. And so that's that's that process that takes these set of key points and maybe the raw images as well, and then figures out sort of a set of I guess it's sort of some kind of segmentation and then figures out for each of set of, I guess it's sort of some kind of segmentation

and then figures out for each of those segments which way it's actually facing. Yeah, and lots of the time you don't actually need to know very much about the surface. You can be surprised, I suppose, about how much you can get away with just being able to track a single point in your camera image and then knowing, for example, which way up is in your world. That makes sense. Yeah. Yeah. So if you're happy to be having an experience that takes place on your floor, so on a surface that's not going to move in your environment or on a static table, then being able to track a single point and know which way up is, you can actually do quite a lot of interesting experiences with that sort of thing.

Yeah, that makes a ton of sense, right? Because I mean, most of the things you're interested in are at 90 degrees. Like you're interested in walls, tables, floors, and so you don't have to consider all the possibilities there. Yeah, exactly. And, you know, lots of the time, technology like that we have at Zapper will be used for a creative experience where you might have a character appear and be talking to you, or maybe it's a virtual tennis player and you can play some tennis with them. You don't really need to be understanding too much really about the environment that the user is in in order to facilitate that. That said, there are a ton of use cases where actually knowing that, you know, what the actual surfaces are, where the walls are, et cetera,

that facilitates quite a lot of applications that you can't otherwise do. It's just the more you want to understand about the environment, the more computationally difficult it becomes to work it out. Yeah, that makes sense. Yeah, and devices like the iPhone, for example, have machine learning chips in them specifically to try and process data a lot faster to be able to run in the much more limited computational environment that your web browser is. So in that sort of circumstance, we have a different set of appropriate applications that we can support based on the type of computations we can do. Yeah, that makes sense. And one thing I want to point folks to, if they missed episode 102 with Max Sklar, we talked about Bayesian mathematics and dealing with uncertainty.

And this is a perfect example of that, where you have an accelerometer, it has error. You have a gyroscope with error and you have these images and that algorithm, there's like irreducible error. I mean, there are things that are just ambiguous and there's also error in the algorithm itself, right? And so the way you kind of deal with all of that is you just kind of put all of that uncertainty into this fusion system and it kind of reconciles that. So if your accelerometer is very unhappy, I don't actually know very much about mechanical things, but let's assume you dropped your phone and that made your accelerometer unhappy

for a few minutes or something, the system would understand that and it would compensate with the other devices, right? Or if you were in a very flat room, then maybe that could be compensated by the hardware on your phone. And it all kind of works together in harmony. That's exactly right. And those exact techniques that you're talking about, modeling the error, trying to work out how certain you can be about some

sort of truth in the environment that you're trying to understand versus the sensor input you're getting. We also have a different lag between things like sensors and cameras. So we might, for example, get very high resolution and very fast data back from the gyroscope. I also am not very mechanically minded, so I have no idea why gyroscopes are better than accelerometers. But gyroscopes seem to be really good. Actually, just an aside, I remember reading something, and people should check this out, but the way the gyroscope works in such a small device is it's like an extremely tiny ball. And in that hollow ball, there's some kind of fluid and another ball. And in that hollow ball, there's like some kind of fluid and another ball.

And basically where those two balls contact is how it determines. But it's like macroscopic or microscopic. And it's unbelievable. You should see, you know, you could do a Google search or something and see how the gyroscope in your phone, like what it actually looks like. It's remarkable. I find it fascinating to think about like all of the tech that sits in these devices in our pockets. And, you know, I think if when, you know, we're at the gates of heaven and we were to be shown exactly the complexity

inside the phone in some sort of moment of enlightenment, I think it would be incredibly mind-blowing yeah it would be like taking uh george washington and putting him in a time machine and putting him in in time square or something right i mean it would just blow our mind uh you know and and it's the encapsulation is incredible right that it's it's uh we've taken all of this complexity but we've been able to provide value to just about everybody. Indeed. And I think there's a kind of a comfort in how broad we have been able to use computing, I suppose, as a race, because if you were to take George Washington and bring him to now and then like show him show him some cat video from from reddit he might be like what why why are we using the technology like this um yeah it's so true but the comfort comes in being able to say you know it's exactly this set of technologies that also

facilitates like a surgeon across the world being able to to perform surgery on on somebody in an in a hospital without the same facilities um you know it in that case it probably still comes down to like mpeg encoding a video and so you know there's there's for every flippant use there's there's probably something that's doing a lot of good in the world. Yeah, absolutely. Yeah. I mean, that was really well said. Yeah. I think, um, yeah, it's interesting. There's a lot of energy spent on, um, you know, how can we better use, uh, free time, but then almost serendipitously, we end up, we end up, uh, making better use of people's work time. I mean, what I'm thinking in my mind is the Google Glass, which was an originally represented as a, you know, B2C, business to consumer kind of product

where people would use it, you know, in their free time or maybe as part of their daily activity. But then what ended up happening with Google Glass is the physician, the medical community picked it up. And so, you know, my dermatologist I've been going to for years and years, she's been using this Google Glass for years and as part of her daily routine. And so, yeah, as you said, I mean, it's amazing how you build things, really just entertain yourself. And then in that serendipity comes, you know, real amazing progress that can change a whole industry.

Yeah, and I think augmented reality is a particular industry where this is such an important concept, because there are lots of uses of augmented reality out there, which are entertainment uses, that are marketing uses. So, for example, to take Snapchat face filters where, you know, users get what we would still call short-term but delightful value out of something. So, you know, a user's going to have a go at a Snapchat face filter, and in 10 minutes, they're not going to be using it, and probably will have forgotten about it in that time. But for the time that they were using it, they enjoyed the process of using it. They've had some interesting social interactions as a result. That process has probably been monetized by advertisers. So there's a value chain that goes elsewhere in the industry, I suppose, also. And it's easy to look at a use case like that and be cynical about it and say, you know, it's a short term entertainment experience.

But at the end of the day, it is delivering value for the user and for the, you know, the upward chain in that industry. And also those same technologies are powering other forms of technology elsewhere where it has a different set of value, a different kind of value mechanic. So because this technology is necessarily very content rich, there's just an interesting mix of different use cases where how, I don't want to use the word frivolous, but, you know, how short termterm an experience is for a user versus other

cases where it might be a lot longer term. But in general, it's all valid use. Yeah, and I think the smartphone followed the same trajectory. I think there was Palm Pilot and some of these things, but they didn't have contraction. What really took off was ultimately the iPod, right? So the iPod, which is a way for you to listen to

music on the bus or something, you know, became very, very popular. And then they started putting a screen on the iPod. There was the iPod touch. And then it was pretty obvious that people wanted Wi-Fi because they didn't want to you know get songs through tethering right and then if you have wi-fi then might as well put 3g right and so it's like something that started off as you know you might say well why not just have music on your laptop right so it started off as just kind of like this luxury accessory and it turned into a way for people all over the world to learn something new, right? Yeah, absolutely. I think that particular example is fascinating as well, because

if you look at something like the Palm Pilots from the 90s, you can see already there, you know, the basis of all of these little bits of technology, you know, I'm sure Palm Pilots could play music or MP3s or maybe wav files back in the day and for it's easy for us to in hindsight look at that and say you know it's obvious that basically everybody's going to end up having one of these and it's going to replace their their telephone it's going to replace their music player their alarm clock their like note taking their voice recorder you know like you. The list is huge. And it seems obvious in retrospect.

But I imagine, I suppose at the time, it just doesn't occur to people. And I think you have to make mistakes, too. And I always get this backwards, but there's the capacitive touchscreen and the resistive touchscreen. And one of them basically, the short story is one of them requires a pen. It's just not good enough for you to use your finger on. And so if you were to use like the Apple Newton or one of these Palm devices from the 90s, you'd have to use a stylus. But I don't know if you could have just invented

the iPod Touch touchscreen. I think you had to take those stepping stones. Yeah, absolutely. And I wonder what the process at Apple was in terms of how many aha moments there were in the production of the iPod to be like, oh, you know, this is, there are so many use cases beyond what we're developing here. And eventually you end up with what are very ubiquitous general purpose computing devices that we all carry around in our pockets. Hey guys, I'm going to step in here. This episode actually has a new sponsor for us,

and that is Teamistry. Teamistry sponsored this episode, and they're a podcast that talks about teams that work together, how teams operate, how you can operate in teams using stories from the past, stories from current events, and just really giving you helpful hints about how to be a good team player. Yeah, this is totally awesome. You know, we talk so much about soft skills. And we had an episode of soft skill about soft skills, you know, earlier with I think, john son was, but, you know, it's, it's hard to dedicate, you know, we can't dedicate this whole podcast to that. And so, you know, team history, that really complements ours. You know,

I think these skills are super important. If you want to, you know, talk about how to improve things on your team, or if you just want to hear a bunch of stories of different teams, you know, get practical lessons about how you can kind of take some of those stories and apply them to, you know, the kind of job that you do. I think there's a current season and there's a season two that's coming out. We were fortunate enough to get a sneak peek of the second season. So it's pretty exciting stuff coming up. Yeah. So some of the season two are out now. I guess we don't have seasons. We're just one giant season. Yeah, probably. We're on our what?

Is a season an entire calendar year? Because I think its season is literally like a season. So we would be on our 20th season or something. Okay, I don't know about that. So this is our second season. I believe the first season had about eight episodes. The first couple are out in season two. So if you check them out you can you can see them now and the the first episode of the new

season season two is about uh watch factory seiko so i i think most people sort of know about seiko um if you know about watches they make all kinds of watches everyone often knows about the cheap watches they make the little quartz watches that are know, sell from a few dollars all the way up to, they actually make very high-end watch collector level watches that are, I don't want to say, lots of money. So being a watch enthusiast myself, I knew about their brand. And this talks about the story of how they set up inside of Seiko sort of two competing teams to do research and development. And they were each doing their own thing. And, you know, this strategy is not super surprising. It's actually employed by, you know, other people as well. But I think one of the things that the episode talked about that I thought was really

cool because the first time I heard what they were going to, you know, talk about, and it's like, oh, yeah, of course, it's easy to set teams up to compete but how do you prevent them from you know you know getting at each other and it ultimately being a net negative um and that actually was what they talked about which was how the seiko um organized these two teams as being part of the same family a little bit of a sibling rivalry i don't want to give away the sort of whole podcast um but talking about how they balanced the improvements they got from having parallel R&D efforts without having them be in conflict with each other. And then taking through how Seiko went from, basically, I mean,

they're responsible for making Japanese watchmaking a thing competing with the Swiss. So most people know about Swiss watches. I guess it's a little bit of a, we would call it a meme these days, but you know, switch watch keeping is the sort of pinnacle, but now actually several Japanese brands compete on that level.

And Seiko was one of the ones who really drove that. And then when quartz movements came out, they sort of came into their own and almost destroyed the Swiss industry, but we'll go on forever. This is not a watch podcast. So we'll keep moving. Team History. Patrick is way into watches. Oh, yeah. OK. People who know me know this is not not surprising. But yeah, we're very thankful for Team History sponsorship. Definitely check it out. I think it's super critical, as Jason was saying,

for engineers to not forget about those soft skills. We like to focus on improving programming and being better at that. But whether you want to or not, politics and teamwork are crucial to becoming a well-rounded employee. And you can search for Teamistry anywhere you listen to podcasts. We're going to add a link in the show notes. Thanks to Teamistry for their support. And I'm going to turn it over back to the interview. So most people have seen the Oculus Rift and all of the things along those lines, so the HTC Vive. And so I'm sure people have sort of this mental image of the thing you put over your face. And now it's even wireless. I don't remember.

I forgot which is the one that's wireless. Oh don't remember there's there's uh i forgot which is the one that's wireless uh oh i think there's an oculus quest quest that's what it is thanks so now it's even wire so you can imagine you're just putting this headset on and going for a walk right um but when it comes to augmented reality you know i don't think people uh well first of all i don't even know really what's out there in terms of hardware but but uh but i don't know if people have a good mental model for what that actually looks like. Can you kind of walk us through, you know, what is augmented reality now? I think the example you brought up where you hold up the phone, that's definitely in everyone's hands. But, you know, what is sort of, is that the state of the art and what's, you know, coming in terms of hardware?

Absolutely. the state of the art and what's you know coming in terms of hardware absolutely uh so i think the um certainly at zapper our primary mechanism for our primary distribution mechanism i suppose for augmented reality is through your phone so uh experiences where you um you either have an app or a website on your phone that app or website will open up your camera and then from there it will bring your environment to life either your face or your the the table in front of you so for example that app might be a website or a retailer and they're selling vases and you would be able to with this tool picture what the vases would look like by just holding your phone in front of you over your your coffee table and seeing what it would look like in

place so so that sort of use case for us is where we focus the reason we focus in in mobile handheld distribution mechanisms is because there's a huge market there everybody has a smartphone and and we now have a set of technologies where we can target that device. We have algorithms that run fast enough to facilitate this type of experience on this device. But by no stretch of the imagination is that the only mechanism by which people can have AR experiences. It's just probably the mechanism that those that have will have had it. Yeah, so there are other ways. There are head-mounted pieces of hardware. So Magic Leap, for example, is a company you may have heard of. They make a headset where you put it on your head.

It has some screens that sit in front of your eyes, but those screens are transparent, but can have content on them. So most of the time when you're wearing this headset, you see the environment around you, but inside that space in front of you, there will be 3D content that appears to you on these screens. As you move your head around, that content will appear rooted into your environment in front of you.

So Microsoft HoloLens, another example of the same type of technology where you have some semi-transparent screen in front of your eyes. We do a similar sort of thing with head-mounted display, but because our focus at Zapper is on things that have mass market distribution, we don't focus so much on expensive headsets or anything like that. We have instead a cardboard-based headset where you use your phone. So much like Google Cardboard did for VR,

where there's like a cardboard headset, you put your phone in it it's got some lenses in that headset you wear it on your head and you get a VR experience kind of similar to what you would get with Oculus Quest or Oculus Rift or these types of technologies but probably a little bit lower fidelity but still pretty good and given the fact that it costs like a hundredth of the price, you're probably going to be happy. So we've done the same thing for AR with what we call Zapbox. So basically, it's a cardboard headset with lenses. You put your phone in it and you wear that. The phone shows the camera view that it sees in front of it.

So you see the world around you on the screen of your phone but but close to your head if you like so it looks like you're looking straight through um and then we use our computer vision algorithms rather than any other kind of sensors or camera devices that are attached to the headset we just use the same computer vision algorithms with the sensors that are in your phone to work out where you are in space and to be able to put content into that world in front of you. And then we also have these two cardboard controllers, which they look a bit funny. They've got like black marks all around about them. And we use those marks to work out where those controllers are in space so that as you're holding those controllers and moving them in front of you,

the camera of the phone can see them and recognize where they are. And you use those controllers to interact with the 3D content that's presented in front of you. Wow, that's amazing. So how much does that cost, roughly? So a Snapbox is $30 plus shipping. And so what that gives you is a cardboard headset, the two controllers, and a set of markers that you can put around your environment to help the phone work out what your environment looks like.

We have an interesting constraint with that box, which is because it's on your head, we really do need to be processing frames at 60 FPS. Because if you show the camera to a user on the screen of your phone and it's right up close so that it takes up their whole vision, people are very sensitive to the frame rate and the type of lag you might get from the camera when it's taking up so much of your vision. So we really need to get 60 FPS. And so for that reason, we have a very limited computational budget.

So every frame of the camera needs to be processed really quickly in order to maintain the 60 FPS we want to maintain. And so we have these markers around which help the algorithms do that. So they're little circles with high contrast elements. And basically what it means is we can find those in the environment much more quickly than it would take to, for example, build a huge environment understanding of an arbitrary space. That makes sense. You have some high pass filter on contrast and then these things come right up. Yeah, exactly.

Exactly. And we are moving to using fewer of those markers and using more of the natural environment around the user just as devices improve and as our algorithms improve. But for the moment, the Zapbox kit includes some of those markers so that you can instrument your environment to help the circumstance. Yeah, a random question here.

Why are you showing the camera? So let's assume there's nothing to be shown, nothing to be rendered on the image, right? So like, could you make it so that it's kind of like one of these like mirrors where the camera, you know, if your phone is just a black screen nothing would bounce off this mirror into your eye and you would just look through the mirror to the scene behind you but then if you know the camera if the if the phone was to draw like a white circle on it that would that would show up it's like some way to get the translucent appeal so you because my guess is if you're looking through the camera it's not the same resolution as your eye. No, indeed not. And there is some lag. And it is a bit different to, of course, feels different to actually being in the room and

seeing the real environment. But there are two elements to that. So one is that if you do have virtual content, you want that virtual content to match up very closely with the real environment. And so if you're instead showing a screen with some virtual content on it, and then the real world behind the user, you have to be able to get those pixels and that 3D content onto that screen so quickly in order for it to match up with the real world environment because normally you have like some latency in the in the ability to render content onto the screen and so if you're not careful you'd have a circumstance where you move your head and of course the light is coming to

into people's eyes at the speed of light so they're seeing that instantly but then it's taking the phone maybe like 50 or 100 milliseconds to actually get something into your eye that looks like some content. And during that time, your head will have moved a little bit. So you'd get this kind of effect where things lag a little bit behind. The virtual content lags behind the real world a little bit. Whereas if what we do is we show the camera feed to the user as part of that rendering process, we can make sure that the 3D content is matched up exactly with the frame that is being shown to the user at the time.

So you don't get that lag between content and real world. But of course the cost is, the user sees this lower resolution and slightly laggy version of the camera that instead of the real world environment. I personally find that that view is, if you're focusing on the virtual content, I prefer the content to register correctly

with the background environment. My brain, and I'm sure the brain of others, are happy to deal with the lag for everything than have to deal with a disparate lag or difference in lag between the content and the environment. That said, there are definitely solutions which do exactly what you say and work very well in lots of circumstances. So one example is a headset where you place your phone actually above your and and horizontal um so the phone screen is pointing

down to the ground and below the phone there is uh i think they call it a semi-silvered mirror so it's it's a uh exactly the the thing you describe it's like those mirrors that you get in um the interrogation cells in in police stations oh yeah we're like the police are looking at you being interrogated but you just see a mirror yeah exactly or or the types of glass that sit in front of uh television cameras for the autocues where the where the the um presenter can read what's on the text uh the text on this on this autocue screen but the users who are the users the audience um who are watching the tv show uh are still watching it through this glass but they don't see the the the text themselves it's the same kind of idea and i and i don't understand how it all works

but i think it's got something to do with the polarization of light. Oh, interesting. Yeah. And so you can have this circumstance where if your content is bright enough, then you can show the content on that screen, but still let the environment behind shine through. So you get the combination of virtual with real world environment. There's a few other kind of trade-offs in that example as well, in addition to the lag difference between the environment and the content, which is that a lot of the time that feels kind of ethereal a little bit because the screen can never make the content completely opaque. So everything feels a little bit ghostly.

Oh, that makes sense. You can see the environment a little bit through it. Yeah, that totally makes sense. Everything kind of looks ghostly or would look ghostly um but the different headsets uh use different technologies to try and reduce that effect and and quite a lot of them do quite a good job so things do appear quite um quite uh solid and opaque cool so someone can go for 30 and30 and basically turn their phone into kind of an AR headset using the Zapbox. I'm assuming on the website it lists phones, but does it work on iPhone and Android?

Yeah, exactly. We have a few, I suppose, recommended devices just because particularly on Android, getting 60 FPS from the camera is not always possible. So there are a set of Android devices which we recommend, although the app runs on a ton of Android devices. It's just you might get a kind of more laggy experience on some of them. But most of the iOS devices from two or three generations past till now all work really well. Just maybe not iPads, because getting an iPad to sit in front of your face might not work. But the iPod Touches work well, as do iPhones.

Cool. So let's jump into Zapar. So Zapar is targeted at folks who want to make AR content. So this would be, the app designer behind, let's say, American Eagle, and they want to add an AR experience. The example you gave is great, where you put one of your own t-shirts and they line this one up, right? And so they would use your platform to help build that. Yeah, exactly. We kind of have two sides to our business at Zapper. We have one side, which is a creative content studio. So we make content for people. But the other side is what we call Zapworks, and it's our content development platform. So we have a set of tools for anybody to use to go and make this kind of

content. And they range in complexity from some that are really easy to use. Let's say you want to bring your business card to life with some buttons for like going through your or sending you an email or perhaps linking to your social sites, or perhaps you want to have a video on there that you've recorded that might play as if it's on the surface of the business card. So we've got some simpler tools that help you make that sort of thing really easily. Then we have some IDEs, if you like, so some fully fledged 3D development environments for building content, which feel a bit like a game engine. So if you've ever built something with Unity or that type of tool, then that's what this tool feels like. It gives you a 3D environment.

You can import 3D models, you can script the experience, and then it understands all of these different tracking types. So you could say, I'd like my character, 3D model character to walk around on my table. And so you would have a world tracker and you drag your 3D model character to walk around on my table. And so you would have a world tracker and you'd drag your 3D model into that. Or perhaps you have designed a hat, a 3D model hat.

You could say, I want to create a face-tracked experience and you could drag your hat in and position it so that it would appear on the person's head. And that's all integrated with a publishing platform so you can publish that content straight away, and users can access it either through our app or through the web or through our technology integrated into other apps. Then there's one final element,

which is all of these computer vision algorithms, we also expose as a set of SDKs, so people can use them in their own applications or programs or other creative tools. So we have an SDK for Unity, for example. So you could build with Unity a gameplay type experience using our face tracking or our world tracking or image tracking, etc. So how does this work? Actually, one thing real quick. How does the web version work?

So someone would go to a website on their phone, and it would somehow start this experience, I guess, using HTML5 and WebGL and all of that. Yeah, exactly. And the web for us is a super interesting space. It's very much more convenient for users in a lot of ways than app stores. So, you know, users can get to websites really quickly. The web now has a ton of technologies that help us do this sort of thing.

It's particularly interesting for us at Zapier also because we have, it's a competitive advantage for us, a little bit selfishly, in that our algorithms, we've been kind of on the block for quite a long time at Zapper. So our algorithms are quite heavily optimized. And also lots of them conceived for devices which have less computational resources than our big smartphones do now.

And that means that those algorithms have translated well for us to run them in web browsers. Wow, really cool yeah so what you would do is yeah you'd open your your phone to a website you might have done that by scanning a qr code first or maybe someone's just giving you a url or a link on social and it takes you to a web page like any other uh the app or the website will then prompt to open your camera like as as if you were using Zoom or Google Meet or something on your phone. In some cases, we'll also, on iOS, for example, request sensor data, so accelerometer and gyro data. Yeah, it makes sense.

Motion data on iOS. And websites require permission for that, so we ask for that as well. But then in that web page, you'll see the camera, either the selfie camera or the rear facing camera. And then much like it would in the app, as you look around your environment, we can bring content to that.

Yeah, and there's a ton of interesting constituent web technologies we use there. So you mentioned WebGL, which we use in order to do the 3D rendering. We use Get User Media, which is the API for accessing the camera in the web browser. We use a technology called WebAssembly, which lets us take code that's not written in JavaScript. So I suppose the language of the web is JavaScript, and that's the kind of natural programming language you would use to write content for the web. But WebAssembly lets you take code that's written in other programming languages and use them for the web. And lots of our algorithms are written in C and C++,

and WebAssembly allows us to run those in the web browser very quickly. That makes sense. There used to be something called an Emscripten. Is WebAssembly like an evolution of that? Yeah. So I suppose effectively it is an evolution, but actually what Emscripten serves to do now is provide an environment for WebAssembly. WebAssembly is quite actually a core kind of raw technology in a web browser that lets you run this type of code. But when you're coding in a language like C and C++,

there are lots more elements than just the machine code that the compiler produces that makes things work. So you have what's called a standard library. And that's a set of programs that run along with your program, the program you've written, that let you access files, for example, or the network or the camera. And so the standard library is a set of tools that accompany your program. And the WebAssembly technology doesn't provide any standard library, but Emscripten now serves that purpose.

So if you're building and you're taking a program that you'd previously written in C++, you compile it with Emscripten, Emscripten will compile it into WebAssembly, but will also provide implementations of these extra libraries that you need in order to make your program run. Oh, I see. Yeah. And so Emscripten, I suppose, originally started targeting a technology called Asim.js,

which is the kind of precursor to WebAssembly. So Asim.js is to WebAssembly, I suppose. Those are the equivalent technologies. And then Emscripten serves as a layer on top of those. Got it, I guess. So basically, like, you write a C++ program. In that program, you call Quicksort. And so what happens under the hood is it goes to the STL

or one of these standard libraries and fetches Quicksort. The issues on the browser, you know, you don't have Quicksort. And so Emscripten kind of provides like WebAssembly versions of all of that. Yeah, exactly. And another way is, let's say you've built a 3D program in C++ using OpenGL. So OpenGL is an API for native programs that allow you to use the graphics card on your computer to draw 3D elements to your screen. So, for example, you might, like computer games,

will use OpenGL to render content onto your screen. And if you've written applications in C and C++ using OpenGL, and then you want to use, to port those using Emscripten to run in the web browser. The web browser doesn't have exactly the same set of APIs. They have a slightly different but similar set called WebGL, which has a different set of constraints and slightly different API.

What Emscripten does is it provides to your code a set of APIs that are like the original OpenGL you brought your program for, but then takes those and actually runs the equivalent WebGL behind the scenes on your behalf so that you don't have to modify your program to understand WebGL. Emscripten will translate your OpenGL into WebGL. Cool. Yeah, that's really fascinating. Yeah, and so, yeah, this is really awesome. So if somebody wants to, let's say there's somebody out there who wants to learn this technology, right? So they would get the Zapworks SDK.

Let's say they wanted to, just to use a simple example, make it so when you touched on a surface, a ball popped out of it or something. So they would take this SDK, they would make an app, a shell of an app around that. There would be some way to sort of trigger it, say when a person taps on the surface, there's some sort of logic to

handle that um and then and then um they would say i want this ball to fly out you know uh parallel to the normal on this surface and and under the hood zapar is doing all of that that heavy lifting that we just talked about and more yeah exactly and different different. And different particular ones of our tools, that's not very grammatically correct. Our different tools expose that type of use in a slightly different mechanism. call it but it's a kind of 3d development environment for making content all you need to worry about there is is be like i want to track the environment i've got the 3d model of a ball and then you might have a little bit of script which you attach to like a pointer event so a finger tapping event on that surface and then you'll say okay when the user taps on that surface i want to show the ball and then give it some velocity or take it along an animation, if you like, going upwards. And then you can publish that and then users can access that straight away in the browser or through the app or wherever.

Alternatively, if you're using one of our lower level SDKs, what our lower level SDK will do is tell your code, the surface is here, the surface is here, the surface is here, the surface is here, every frame, and then you can incorporate that into your program however you wish. So let's say you're doing that in Unity, then Unity will have some mechanism for working out how when the user taps on the screen, then it's up to you to work out how to create the mesh in the right place in Unity and give it the velocity. We also have other SDKs that target 3JS, for example. So 3JS is a library for JavaScript that makes it easy to make 3D content.

It's a programming library. So in JavaScript, you would use it to build a 3D experience. And then our SDK for 3GS will just give you a group in 3GS, which is, I suppose, a 3GS concept. And then it would be up to you to work out when the user's tapped there and to, you know, put the ball there and using the 3GS api make the ball render up in the right place cool that makes sense and so then working uh in the other direction at the highest level you have this um you have like a real design studio or if someone wanted um you know maybe for their for their place of employment they wanted some really professional content then you're able to sort of provide more hands-on experience yeah exactly and in, that's the bit of our business that we started with. So back in the day, my business partner and I, we were working on this as a research project,

this type of technology at university. And we had some commercial interest in these algorithms, and so we set up Zapper along with two others. And originally, the purpose of the company was to provide this type of content for businesses and brands who wanted to have these interesting, rich content experiences with their users. But in order to fulfill that work, in order to do that for our clients and

customers, we had to build these tools to make it easy for us to do. And then for the last three or four years or so, those tools have been a mature enough product that we then sell those tools as a product for anyone to use. So it's a true case of dog feeding, like we're eating our own dog food, we made the tools so that we could use them. But it turns out they're actually very useful tools for anybody that wants to make this type of content. Very, very cool. So what's the, so focusing on the tools, what's the sort of pricing, like what's the business model there? And as part of that question, you know, if someone is a student, can they get their hands on a free version and how does all of that work?

Yeah, absolutely. So for personal use, it's free to use. So you can just head to zap.works and get a personal account and you can use pretty much all of our tools with the free account. There are a few things like the kind of analytics and stuff that only really commercial users would be interested in, which are not available for personal, but otherwise a personal account

has access to all of the tools. And so if you're a student that's using it for your studies or something like that, then you can totally use the personal account for that purpose. We have an education tier as well. That's primarily for if an institution is buying our tools

to use as part of a teaching or for promotional purposes. And it's just a significant significant discount uh on our otherwise uh plans and it's the the product is structured a little bit so you can have teachers and students and and you know permissions are all set up so that teachers can see what the students have made but the students can't delete each other's content and that sort of thing yeah that makes sense yeah but then if you're using it for a business, we have a few different tiers. They start from about $100 a month. And you basically pay on a per user basis for access to the tools, much like you would with something like Creative Cloud from Adobe.

Yes, that makes sense. And as part of this tool, we provide a lot of hosting. So if you want to host your content so that people can access it either through the web or through the apps, we can host that all through our district content distribution network. So it's a really easy process to publish and get your content out there. And so we have like a kind of overage on each of the plans so that if you get all of a sudden millions of people scanning your content, then there's like a pair scan fee just so that basically we don't end up with this huge hosting bill. Yeah, I've definitely, definitely been there. I mean, we at one point we tried moving the podcast to from one provider to another. And that second provider wasn't the economy of scale was a little bit different.

And, yeah, you'd easily rack up a very hefty bill if you're not careful. And so, yeah, you definitely need those guardrails in place yeah it's funny it's one of these things that if you think like we don't seem to kind of understand as human beings an exponential increase so if you go from like 10,000 to 100,000 it doesn't really seem to us that much of a difference somehow like but but it is 10 times more or if you go from 100,000 to a million maybe a million is a little bit different because it's so kind of ingrained in our psyche as a number. But, you know, if you have a million people accessing your content, you've had 10 amounts of 100,000 people accessing your content, which doesn't seem like, I mean, it seems like I'm stating the obvious, but 10 times however much you paid for 100,000 is going to, you're going to notice that in your wallet rather than yeah there's this i've been reading um

um there's this uh book i've been reading philosophy for kids i've been i've been reading it with my kids and um there's this one philosopher and i'm not gonna remember the name he's actually an indian philosopher that much i remember but but his thing was basically the king kind of wanted to exile this person. And they said, you know, you can just make a request for me. I'll give you this request and then just leave the kingdom. And the request he made was, he said, take this chessboard and go from left to right, put one grain of rice on one square and then two on the next and four on the next, and then just keep doing that pattern. And then I'll take the whole row and then I'll go. And, or maybe, I don't know if it was a chess board because yeah, two to the eighth is only

200. Maybe it was, it was, it was another game or something, but, but yeah, basically you just, the exponential, you know, the King said, sure. And then you quickly realize that it's more than there are grains of rice in the universe or something right so it's just it's just yeah people have a really hard time grasping grasping that and um what i what i tell i recently did an interview where we talked about um podcasting and i answered a lot of questions about this particular show and that was one of the things i touched on was you know if you, let's say 300 listeners and two of them, you know, have a feature request, if you go to 300,000 listeners, you're going to get 2000 of those, of those requests. And if you're getting two of those a week, now you're getting 2000 emails a week just for feature requests. And yeah, it's a hard

thing to really grasp. I wonder why that is. I wonder what part of our psyche makes it difficult to grasp, you know, exponential or multiplicative things. Yeah, it's super fascinating. And I think we're kind of lucky as programmers that in a lot of ways we are taught to not trust our intuition when it comes to scale and complexity. A lot of the time, you know, lots of these, you know, if you're studying computer science and you're learning about algorithms and all of these things and big O notation,

you know, very quickly in that process, realize that trying to intuitively work out how a particular sort mechanism is going to scale is something that's quite difficult to do. You kind of need to start modeling it or doing the proofs to try and work out how something scales. So I think as a community, we do better than maybe other communities do in terms of understanding that or at least it's just learning not to trust our intuition on it yeah that's right yeah yeah and

it comes down to the or we see the same thing as you say with with trying to understand non-computational concepts as well and how you scale something like support or feedback and these sorts of things is super interesting. And, you know, as a developer of a product, it's also a fascinating process where you see lots of feedback that you get in for a particular product, and you have to discern which elements of those are those that you should prioritize,

that make sense for large numbers of your users and that are feasible. And I find that process fascinating as well. Because inevitably, you're going to get a ton of users that said, but I gave you this amazing idea and you never implemented it. And I think it would be the best idea for your product ever. And what they don't see is that you've also had, you know, 1,999 other similar amazing ideas. And you've had to go with the one that, you know, you thought would push the needle a bit more or would help more users.

Yeah, totally. Yeah, so the product sounds awesome. I think everyone should check this out. Totally free to try this out. And if you do end up making the ball coming out of the surface, send that. I mean, that would be awesome. If there's a website where we could just go and try that out, we'll post it in the show notes if anyone builds that or anything, really. I think it'd be really fascinating to try it out. What's it like to kind of work at Zapper and, you know, what sort of a day in the life of an engineer there? And especially if there's anything kind of that really stands out that people kind of wouldn't expect that's kind of unique, right? And, you know, are there sort of positions available what kind of folks are you looking for

uh yeah i think one of the most uh interesting things for me at zapper is we have a hugely diverse range of skills at zapper and job roles so our uh business because we have this side of business that's a creative uh development studio as well as having our platform side of our business, so the side that makes these tools, we end up with designers, we have 3D artists, we have content developers who are using JavaScript and TypeScript day-to-day to build AR experiences. And then, and I suppose the team that I look after, we have web developers who are working on web-based AR creative tools. We have people looking after our content management system. So Zapworks allows you, as we mentioned, to host content and publish it for users. And so we have a management system for that. So there are people developing and working on that. We have our support team as well who help our users and users of our tools get the most out of those tools and answer any questions that they might have. We have a QA team and sales and business development. So there's a huge different number of roles.

And that's what makes Zapper really interesting. The people that we get to work with are all an incredibly talented bunch and are all really fun to work with. And it means also, maybe this is the unique element, but it's maybe interesting for us, but not necessarily an interesting insight, which is that it changes a lot. We work on lots of different content all the time, and we have lots of interesting new technology that we're working on and new tools that we're building all the time. So it never really stands still very much. Yeah, that makes sense. I mean, you're right at the forefront of something. So, you know, I feel that's one of the challenges with the smartphone is, you know, if know, even, you know, I mean, the plan is just to give it away for free, but it's hard to even do that because, you know, it's like what

keyword hasn't really been taken or how will people be able to find this when there's just so much content out there? And the platform that you're working on is really a green field, you know, and who knows when that device will come out, or even when the environment changes as it has with the COVID-19, that causes that huge explosion of growth, right? So it's like a great time to be in this area. Absolutely.

And I think to your point about apps and trying to kind of outshine other apps and what have you, I think what we'll see is a continuation that the web is the ultimate app delivery platform, if you like. And in the same way, you know, if we're making a website, we will do perhaps SEO optimization to try and, you know, make it so that users find our site. Or we look to social media or, you know, lots of other forms of getting our websites out there for people who are interested to find them. We never kind of think, you know, how am I going to make my website appear near the top of the big list of websites that there is? There's no big list of websites. Yeah, that's a really good point. Yeah. And so I suspect that we will continue to see the web just blossom and more and more the types of

applications that we would be used to having as apps on our phones will actually access through really nice and tightly implemented web applications. Yeah, that makes sense. I mean, also you have a truly neutral platform, right? I mean, I think the app store, you know, I know where I work at now, there's tons of challenges with getting apps approved. It takes forever, even when there aren't issues, right? And you just have kind of like this constant gatekeeper, at least on iOS. On Android, I don't know too much about the Android App Store experience.

My guess is it's similar. Maybe the turnaround time is faster. Oh, no, actually, you can publish without any review, right, on Android? It certainly used to be the case, no, actually, you can publish without any review, right, on Android? It certainly used to be the case, yes, although different apps on Google do get reviewed now, but I think they're kind of triaged and apps which are considered by Google to be more worthy of close inspection will take longer. So I think, for example, if you're part of the Design the designed for families program they will have a closer look at your app or if you request certain sets of permissions and what have you yeah yeah but the web is uh you know it's you don't have to install anything like the the app installs when you go to the website and uninstalls when you leave every

time you know yeah yeah exactly and i think you, there are lots of reasons why I think that having the web is a healthy thing for society in terms of it being, you know, it doesn't have gatekeepers necessarily. Of course, you know, websites are still bound by the laws that exist in our various different countries. countries that doesn't stop some websites yeah right yeah but at the end of the day uh the only thing that stops you publishing a website is your you know creative the point where you decide i'm willing to get this out there and make it and see what happens um yeah and i think that's good that's good for a society um and it's a reason i think why you know as these web browsers get better they're able to expose more of the devices technology-wise like we were speaking about WebAssembly and and get user media the camera API for example I think we'll get to a place where where the web is just the ultimate application platform and it will be the exception rather than rule when you have to be running

native code on on a device as part of a big proper installation. Yeah, totally makes sense. So are you folks hiring? Are there internships? Are there full-time positions? What does that look like right now? Yeah, well, we do have a set of roles up at zapper.com, our careers page there. So definitely worth looking at. Throughout the kind of latter half of this year, we are beginning to think more about not necessarily having to have everybody in London, which is interesting just because we are, you know, thanks to the big COVID word, we are embracing working from wherever you like, working from where you

work best, I suppose. We do have some constraints, of course, like we are a UK-based company, and so we need to hire people who are eligible to work in the UK. But other than that, we are exploring more wide opportunities in terms of where people are, which is super exciting and interesting. Cool. If someone wants to come for a summer, like a co-op or internship or something, I know a lot of companies are just starting, especially when things are changing. They feel like they can't provide like

a consistent sort of stable internship program. And I think, you know, I mean, I think we talked to Rob Zuber for CircleCI and and he kind of echoed that. And I mean, it makes a lot of sense, I think. I mean, even, you know, even bigger companies have a really hard time saying in January, OK, when you come in the summer, this is what you can work on. And so, you know, it makes sense. So is interning something that you're doing right now or not yet? Absolutely. We've in fact just hired an intern role for the next couple of months in one of our

positions. And we have had student level interns over the summer in past years as well, and have had some fantastic contributions from those individuals. And we don't have a specific program. I suppose we're still a relatively small business, but we're always willing to hear from people that are interested and willing to come and learn. And hopefully we all get value from the circumstance and produce some interesting things together. So I think that one of the overriding things at Zephyr is we're just always willing to have a conversation,

be that with potential candidates, be that with interns, be that with our customers or those who are using our tools. We just always want to chat. So yeah, get in touch. Yeah, it sounds like, and please add color to this, but yeah, it sounds like one of the best ways for somebody who might be in college now or in high school now

that's really interested in this is just to start using it. I mean, there's no barrier to entry here. Just start building some stuff and then, you know, they can sort of reach out or they could, you know, have a little portfolio and that can kind of get the ball rolling.

Absolutely. And, you know, it's very similar to, for example, if you're interested in computer game development, any of these technologies that marry creative work with technical work, be that so 3D modeling versus programming or web development versus graphical design and UI and UX. If you jump into a project of your own just because you want to

make something, you end up touching so many of these different areas. And if you're early in your career, that will help you work out the bits that you're interested in, the bits that you're good at, and therefore the bits that you want to take further. So I definitely recommend people to just have a little think about something you'd like to see and then see if you can make it, be that a computer game, be it something AR, be it whatever you like. Cool. Awesome. see if you can make it be that a computer game be it something ar be it whatever you like cool awesome and so yeah this is this thank you so much for coming on the show yeah this is uh absolutely fascinating i mean honestly there's so much uh there's so much content here that

that i'd love to really dive into um we've touched everything from understanding kind of high level geometry, random maths, all the way to building products. So thanks for your time. Absolutely. Oh, go ahead. I was just going to say thank you very much for having me. It's been a pleasure. Cool. Cool. If folks want to reach out, what we'll do is we'll post some links where folks can reach out to you. And also you can check out Zappar.com. That's with an A, correct? Z-A-P-P-A-R.com. That's right.

And Zapp.works is for the SDK. Yep, that's right. Absolutely. Cool. Thank you so much for coming on the show again um thank you listeners out there uh for all of your support on on patreon and checking out our audible uh membership and uh we will catch you folks next month programming throwdown is distributed under a creative commons attribution share alike 2.0 license you're free to share copy distribute transmit the work to remix adapt the work but you must provide an attribution to Patrick and I and

share alike in kind.