Big Technology Podcast - Meta's CTO On Its Augmented Reality Dreams — With Andrew Bosworth

Episode Date: October 2, 2024

Andrew Bosworth is the Chief Technology Officer at Meta. Bosworth joins Big Technology Podcast to discuss Meta's new Orion augmented reality glasses and the future of human-computer interaction. Tune in to hear his insights on the technical capabilities of AR, the potential of artificial intelligence, and how these technologies could reshape the way we interact with computers and each other in the coming years. We also cover the evolution of computing paradigms, Meta's strategy in the AI and AR/VR space, and the philosophical implications of advanced AI. Hit play for a fascinating conversation with one of the top technology executives at the forefront of the AR and AI revolution. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. For weekly updates on the show, sign up for the pod newsletter on LinkedIn: https://www.linkedin.com/newsletters/6901970121829801984/ Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology Questions? Feedback? Write to: [e-mail]

Meta's chief technology officer joins us from Menlo Park to talk about Orion the company's new augmented reality glasses and what they mean for the future of computing. That's coming up right after this. Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. And we're here in Menlo Park with Andrew Bosworth, Boz, as he's known, the chief technology officer of MetaBoss. Great to see you. Welcome to the show. Thanks for having me. Yeah, great to be here. So I am just off a demo of Orion, your new. holographic augmented reality glasses. First of all, they're crazy. That's what we're going for.

First and foremost, they're crazy. I love that in Mark's, the Supercut Mark showed. They're kind of sizzle reel. It was just like people say, like 10 people in a row just saying, oh, this is crazy. It's amazing because you expect it to be an experience you're having in a VR goggles. I'll tell people, you know, just for our listeners. So these are glasses. They don't feel extremely heavy. You wear them just on your face like you would normal glasses, but they're augmented reality. So you can make calls on them, you can watch videos on them, you can play games on them, and you have to like remind yourself like, oh, wow, like these are, these are glasses. That's right.

I'm not wearing goggles. So you've had the, and this is a 10-year project within meta, but it looks like from what I saw with your team that this is something that has really come a long way, even in the past year. That's right. And you've had an opportunity to use them. What have you been using them for? Yeah, the hardware came up last year at this time.

And so the software bring up really only came together a few months ago, and they just made incredible progress since then already. So for us, there's a couple different things I've called out. The first one is actually very relatable, I suspect, for those of us who works during the pandemic, during the pandemic, you're on these, you know, obviously doing a lot of video calling. And one positive silver lining from all of that was the development of the back channel. So every, you know, meeting would have a chat alongside the meeting and somebody could ask a question, hey, I missed that.

Is there a link to that? Okay, cool. and so you're not interrupting the meeting to get this extra context. And what's been fun with these glasses is these glasses kind of serves like the in-meeting back channel. So now I'm in an in-person meeting and a live meeting and somebody can add context to a conversation that's happening in the glasses. So I'm like, oh, okay, somebody references a prototype.

And then the next thing I know, up pops an image of the prototype that they were referencing. So now I'm fully in context again in the meetings. So these things are kind of wild. Like these were before this, it would have been very rude to me in the middle of this meeting with Mark Zuckerberg, take my phone out like. go look up the reference and the thing. So that's been kind of interesting, this real life back channel when you're in a live scenario. But that's also not overthinking. It's also a great way to scroll Instagram and like enjoy the reels and tap on those things and to watch videos.

The surprise hit, I would say, from the demos has been video calling. And there's two parts to that, right? One of which is the experience of having someone video call you while you're just walking around is really cool. And you can see them in full fidelity, but you can also see and navigate the world really effectively. You've got your hands free. And then conversely, What they would see is your Kodak avatar. And we've really fooled some people with the Kodak avatar. When they've seen the Kodak avatar call in, they don't realize it's a Kodak avatar. They think it's a real, an image.

So let's just go right away to like what this does to a real world experience. You mentioned it, right? So like it's so hard to be in the moment now, right? And a lot of times you're speaking with someone. Oh, there's a buzz on their phone. Hey, one second. Let me check this out. And then we went, a lot of people went to smart watches.

I see you're not wearing one. not currently now and that also like you're like sitting on a table with someone and they're like keep them looking at their their watch so does this take people out of the real world in a way that you know might be detrimental i mean what do you think about the real world implications here no a tremendously important question and we we've tried to use a tagline more present and connected the idea of kind of reducing the dichotomy we often feel and actually one of my anti-patterns that i've been telling people about for these glasses is just having to be notification services? Because notification services are an invitation to leave the present moment and instead go

elsewhere. Like that's what a notification is intended to do. So let you know, hey, if you want to, you could leave this physical plane and go dive into this digital surface over there. I want people to be able to do that if that's what they want to do. But it can't just be that. Instead, though, if you think with the examples I've given, it's about adding context. And so the real difference maker between these and a phone in your pocket and all the things that are attached to a phone your pocket is they have the ability to be aware of the world at all times. And this is where AI has been such a tremendous tailwind for our vision for these. You know, we really thought early on we were developing these, even until maybe a year ago, it was going to be holograms and

then at some point later AI. And in fact, it's going to be the opposite. The AI is going to come first. It's already come to the meta-ray ban glasses. And so for me at least, we've got this idea where, okay, what if these glasses are contextually aware? They know I'm talking to someone face-to-face. hey, I don't need that notification about my cat is moving on the camera at home. I don't need that notification right now. We can hold off on that. Oh, someone's calling me. It's my kid's school.

Okay, even though you're talking to someone's face, like we can probably pipe that through and let you know, hey, your kid's school is calling. And so that is the real missing piece for allowing these to both be more present and more connected when you're using them is the contextual awareness that they have the potential to have about the world around you and how you're engaging in it. And there's also just like a cool part of it that I saw. And the demo is, um, imagine you're just like at a restaurant waiting for somebody. Instead of like sitting and scrolling your phone, you can basically have a computer with you. That's right. You just brought in with your glasses.

You can do some work. You can watch a video, do whatever you want. And it's there with you. Yeah. One of the fun demos that we, um, we have in prototyping. It's not even in the demo that you tried yet. Um, is this cool thing where you, you have your whole workstation set up. So you got like a three panel workstation set up. And then you like just get up and walk around and it goes away. And it minimizes. And now you've got like you're walking around UI if you need it. And then if you sit down again, you can just recall it immediately. So like all your context comes immediately back whenever you want it there. So not just a mobile and portable workstation, but one that's literally, you know, at the, you know, twitch of a finger ready for you to use. A lot of computing. It solves a

problem. Sure. And another question I have for you is why? Why do we need this? Like what type of problem does this off? Can't we just get along fine with our phones? And I know that's going to sound stupid because every computing phase they have been like, well, can't we just use desktops? Why do we need this in our pocket? But so what problem do you think this solves? No, it is. It's exactly the right question. And it speaks to how we as a society so quickly normalize everything that's come before as completely normal and everything that's new is like kind of this unnecessary thing. Douglas Adams, a famous quote on this comes to mind. I'll leave that as an exercise to the to the listener. So for me at least that, yeah, when people, I remember when cell phones came out and everyone was like, who is so important that they need a phone everywhere they go? And it's like all of us are. It turns out all of us are that important. And I feel a little bit the same way. And some of these things will sound trivial. But having meta-a-I is actually a really good test point. Having, I've had meta-a-I on my phone for a while as an employee of the company. And it's pretty useful. And I like asking it questions. Now I have meta-a-i on the Rayban meta-glasses. And it's like, oh, I use it 10 times more often. because just the friction of like getting my phone out of my pocket and like pulling it up and like pulling the app is not trivial to me being able to just like ask a question. And now what if instead that AI didn't need to even be asked a question? It's anticipating. It sees me. I'm like checking my pockets.

I'm checking. It's like, oh, your keys are over in the desk where you left them. So these things start to come together where because the glasses are on your face, they're not in your pocket. And they have the ability over time, not immediately. to become aware of the world and to understand what you're trying to do in the world, they can actually be very proactively helpful. Just to you, privately, you're in control. But that is a thing that phones can't do.

And I do think it is a real step forward in our relationship with digital technologies from going very transactional. Like, I have gone to the technology and then I come back with my answer to being very integrated. So is this basically a way to just deliver the AI platform more seamlessly? It's not just the AI platform. I think the AI platform is what changes the scope of how we interact with it from being completely reactive, which is what computers have been from time immemorial to being proactive. I think it also, of course, is a super more convenient way to go ahead and manage communication, manage all the different things that phones do.

Use video calling again as an example. This is just a terrible way that I'm holding my phone out, my hand out in front of my face. My arm is literally tired. I've been doing this for eight seconds, and I'm like, I work out, man. It's like I can't. So there's a few things that are just fundamentally better on this platform. And then there's a few things that I think, yeah, you can do it on the phone or you can do it here. But if this was already on your face, you wouldn't use your phone. Right. I mean, I've definitely had those moments where I'm like video calling in New York and just running into people.

Absolutely. So maybe this will help. I've walked into a post before. Like, I'm not proud. Why is it important for META to do this? So META's mission long has been, all right, let's, you know, connect people. Obviously, it was the name change. now it's Metaverse or meta, why is it important for this company to build this device? Yeah, there's two different answers to this.

One that I think people recognize and understand. Strategically, we have just been at the mercy of platforms for the since the start of the mobile era. And we were a web company, right? We were born of the web. And the web was this amazing, unowned platform that you just like really had a lot of control over what you built. and get a direct relationship with consumers. And we absolutely feel that in our application development, we're stifled by the gatekeepers, especially Apple,

in terms of the policies, what they allow, what they don't allow. And there's a real value that we could have delivered to humans who love our products and would be happy to have that value, but we weren't allowed to. For example. Well, anything that has to do with, you know, start with it. Keep mind on the web, Facebook had a major games platform, you know, which we were not allowed to bring to the, to iOS.

because of the app store fees and app store rules. And gosh, I mean, and they've gone even further than that and kind of, you know, fine, we built a app install business and then Apple came for that with ATT, you know, a tremendous amount of kind of self-preferencing for themselves there. And in doing so, they really put, you know, the brakes on what was a really rapidly growing part of the games industry. And so anyway, so I think like, for us, like, platform control is important to us. That's a strategic reason.

That's what everyone jumps on. That's a fair one. If I could be honest with you, that's like a very nice story, and I'm sure it's a part of it, but it's not the real story. The real story is that we're a technology company, and Mark is an enthusiast who wants to continue to advance what is the state of the art and how we connect people with each other and with technology. And we've known for a long time that these are the next thing. As an industry, I think we've had a sense, AR especially, is like, we don't have to convince people that someday. AR is probably a thing that will exist and will be great. And this is where Michael Abrash, our chief scientist, has this kind of state.

It's like, you know, this is the myth of technological eventualism, this idea that eventually that technology will exist. It's like it's doesn't, that's not how it works. Someone has to stop, sit down, and actually build it. And it's a tremendous credit to Mark. I'm going to sound like a kiss-ass. I don't mean to. I'm very critical of Mark in many, many areas.

But on this, he deserves his credit. It's a tremendous vision and commitment that he's made to this technology in the face of incredible scrutiny the last several years to stay with these investments and deliver Orion, which I think is just a watershed moment for us to say like this is, this technology is coming. It's here in some form. It's not evenly distributed, but it's here. And we, it's like within our lifetime, certainly within the next decade, it's in our view we're going to be able to use and enjoy this technology.

And I think he just wanted to do it. And we should say that these glasses are internal. That's right. Over time, they might be released. They're not released yet. But I do think we should talk a little bit more about the platform side. So, okay, for listeners, for listeners, I'm sorry, I just put on, I put on a pair of clear and transparent ones. The ones that we actually use are, have opaque magnesium shells.

The clear transparent ones are super fun to have. They only work for about half an hour of time before the thermal shut down because this plastic does not dissipate heat fast enough. But the real ones, the real ones, they don't get, they don't thermally exhaust themselves. They go until the batter is exhausted two to three hours. But yeah, so it's pretty fun. It's just, I'm sorry, if I have these cool. demo props, what I can't put them on. But let's talk a little bit about the platform risk and the operating system.

I mean, do you have to build your own operating system for these? Yeah, so a couple different stories to this. I mean, when we talk about what is an operating system, one of my former colleagues, Ficus Patrick had this great saying. It was like, operating systems are a set of promises made to developers and to consumers. So you can think of in iOS, the swipe gesture will get you home. Like no matter what's happening on the app, that's a promise that you're making into a consumer. and the apps can't control it, that's a promise that's being made.

You know, you promise the developers like, this memory is yours. You're allowed to use this memory. This API will cause this thing to happen. And so, you know, we're using Android at the most underlying low level as the operating system. But of course, the operating system isn't Android. The thing that people are interacting with on whether it be in mixed reality or in augmented reality, it doesn't look or feel like Android. And there's a bunch of reasons for that. Number one is the interaction paradigms are different.

And so for augmented reality, to stay there for the moment, you're not going to have a direct manipulation interface. You don't have a keyboard. You don't have a mouse. You don't have a touchscreen. Well, it's worth noting this is really the first computing platform that we've had ever that didn't have a direct manipulation interface. I mean, you're going back to Stanford Research Institute, Xerox Park, the keyboard and mouse. Before that, I see it was all mainframes. The keyboard and mouse.

Then you had this tremendous innovation in touch screens and multi-touch screens, direct manipulation, very akin to the mouse. every single application you've ever used, you've used via a combination of direct manipulation. We don't have any of that because you're not going to walk around with those things. You're not going to have touch controllers with you as you go about your day. So what are we using? We're using hands, which can sometimes be modeled into direct manipulation with a little laser pointers like we do in mixed reality. But ideally you're using voice, which is not always appropriate, and eye tracking an EMG, which is, again, approximating what you had with direct manipulation with your hands. And so, for me, at least, like, when you change the interaction paradigm, that is a half of what an operating system is.

It's like, how do you, the consumer, affect the machine? The other half is how does the machine bring information back to you? And here we're more similar. You've got a display, you've got audio, except for the fact that the display is additive light. So there's light coming through from the world around you. So the display has to be somewhat aware of what is happening in the world around you, ideally, to be effective. And so anytime you're building a new interaction paradigm, time, especially when pairing it with novel display technology, you're basically building a new

platform. There's not really another way to do it. And so then how do you think about like who else might build this? You were just speaking with Ben Thompson. You talked about, I don't know if you were saying the potential for Apple to build something like this, but you talk about, I'm just going to quote from you, you look at what are Orion glasses. They're full AR glasses. You have custom silicon in the puck, but Apple could build all this and just be like, oh, it only works. with us. So does it fully insulate yourself from the platform risk? No, nothing fully insulates us, especially in a world where mobile is the dominant platform. Do you expect them to build something like this as well? I have no idea. They keep it pretty tight down in Cupertino. They're a

pretty tight ship. Instead, I don't have great information on that. I think they're, I think they would love to build something like this. I think we've been investing longer. I think we've believed in it more. And as a consequence of frankly, Mark Zuckerberg's faith and tremendous execution by by a team of thousands of people over a long period of years. I think we have a meaningful lead in a bunch of the technologies here, but you'd be foolish to count out a great technology company, one of the great all-time companies in general at Apple. But my point is, like, I think the fact that they already have this phone that's in your pocket

has some advantages because you've now got this device that has a bunch of compute, it has a bunch of silicon. They can do great custom silicon. They can add more to that. It's also a disadvantage, I think, in a way. are they going to be willing to walk away from, you know, one tremendously outsized profit center in pursuit of something that is fundamentally disruptive to that profit center? And if you're centering more of the experiences on the phone, as opposed to on the glasses,

do you make the leap that you need to make in developer platform, in interaction design? Because a lot of those things that we've discovered, if you really were to center them on just the phone and the way phone apps want to work, they don't work as well. So I think there's like it's going to be a dynamic, it's going to be a dynamic landscape. for a while. And I think we take some risk sharing this with the world early to demonstrate what's possible. But we do want to ignite the passion of the developer community, the engineering community around us. And I think it's also something that we kind of owe to people so they can see what this 10-year journey we've been on in reality labs is all about. Okay. So let's talk a little

bit about the reason to have an app. It's just the normal way that you use a phone. But you're talking about these glasses. They're sensing the world. You know, you now have AI voice that you can speak with. I think Mark yesterday said, or we're talking a week before this is coming out. But he said, I think that voice is going to be a more natural way of interacting with AI than text, which is basically interacting with computing. If you think that's the way this is all moving. Do we need apps anymore? It's a great question. I love this. And actually, it goes back, you know, applications didn't always exist, right? In the beginning, it was just like, the system was the software. It was running at the moment. And you would kind of program it.

And I think often about JCR Licklider sitting down for the first time at a terminal, which is reprogramming itself on the fly. And it's really one of the first times in history that people are doing human in the loop computing as opposed to what would have before that been through mainframes and timeshares. Like you put your job in, you come back later, get the results to your job. If it was wrong, you change your software and try again. This was human in the loop computing. And it was this tremendous watershed moment that ultimately led through, you know, his work at DARPA, the IPT, with Doug Engelbart with Xerox Park to the Alto and that's where you start to see oh there's like a windowing SRI like there's a windowing model and each application is contained and the application

was born as a consequence of us being in a human loop at the computing and what's crazy to me is the analog that I draw to chat GPT in fact which was this human in the loop AI whereas before this you would like create an AI and it would go do a job and come back and if the job wasn't right you'd like reprogram your AI whereas this is like no no like I can just modify my queries on the fly and the AI is like adapting to it. And so one does wonder, like, does this get into a more, if not an app free world, a more web-like world again, where things are deeply linked and interconnected through services to one another. I think it's one of the challenges that the app model is going to face, like things like Apple Intelligence, for

example, will face on the phone is what I really want is for something that actually intermediates the apps and can go to my Amazon app and grab my purchase. history and then go to, you know, a different app and drop that in there and add that to my spreadsheet for my budget, which is in this third app that I have. And that is like, that is not something that Apple wants some other developer able to do because that would disintermediate their position, their privilege position on the phone. And so like if the AI can't work across all the breadth of my applications, well, it's not the most helpful AI it could be. One thing that's really fun about the desktop and the web still is that it can still work that way.

And I love those platforms for that, including OSX. And so I think often about, okay, once I have this AI, I want it to work at basically user level. Like the AI is actually my agent in the machine. I think you probably end up still having applications. There's a lot of really valuable constructs we've built up as an industry over time that have to do with the safety and security of data and the model of how these things interact with both the computer itself, with the network, and with other users.

So there's actually still quite a bit of value in having these up. and not to mention just how literally would we do it and have these systems not constantly walk on each other. So I think the application model doesn't go away, but I do think you end up injecting this potential new layer on top of it, which is an agentic layer. So how do developers then think about building

for this type of technology? It becomes much more about what APIs you're providing and the unique value that you provide through the software below the API. If it's about the API you're providing, does that mean the main interaction layer is going to be through some sort of voice AI? It's hard to say at this point.

I think that's voice is a great modality for some things. It also has a lot of vagueness to it, even amongst humans, a lot of context that's required even among humans that obviously even a really great agent wouldn't have for a long time if it was with you. And so I think voice is probably not the only piece of it. Voice and text are part of it. I'm sure there will still be a lot of very direct manipulation interfaces for a long time to come.

But you can start to imagine, yes, as voice gets more developed, my ability as a developer to create value in how I manage process data or how I, you know, manufacture new artifacts is actually the more important thing I'm doing rather than having to also extend that to the consumer through some kind of interface. So in this world, do the AR glasses replace the phone or do they supersede it in some way? I can't imagine it goes away. It's a good question. We think we think with this a lot. I don't know. It doesn't go away for a long time if it goes away at all. There is a real strong argument that When the AI glasses become super prevalent, and that is your interface to a device, and you have, especially if you have a display, whether it be on AR glasses or be on your phone or your watch or whatever the thing is, that you can kind of, when you need to directly manipulate something or provide very precise entry, you can, you don't need a phone anymore. Like it becomes, it becomes what the phone is today.

We're a long ways from that for a bunch of reasons, not the least of which being the technology is not ready yet, but also because the application ecosystems will take a long time to transfer and user behaviors take a long time. content to change. So I think we have a long timeline there. But like, I do think it's conceivable that for a lot of, at least, you know, high-end capabilities and consumers that AR will replace the phone. So what does, like, computing look like in that world? Hopefully it's a lot more fluid. Hopefully it's a lot more self-aware. I mean, computing today is, like, it's hilariously dumb, actually. Like, it's like, we all, like, computers are amazing and what they can do, and we're impressed because, like, they can do math really fast. But, like, if I like type in my phone that I want to watch

Instagram it like is like sorry there's no app called Instagram it's like I meant Instagram it's like I didn't mean Instagram it's just a typo but like the machine's like no I'm sorry you clearly wrote Instagram here and I therefore will not get you the app that you want like computers are dumb in that way they have no construct of what my intentionality is children human children can intuit the intentionality of adults like very near infancy it's like you know I was just talking to someone recently who told me about these studies that they do with human toddlers or actually I think even maybe infants

who are watching their mother interact with somebody and if the interaction is very kind they start to build a model of the person the mother is interacting with and if the interaction looks transactional then they just ignore the other person exists at all like there's like early on like we're like okay there's some intentionality in the world that's a happening theory of mind of other people that exist and like what are they trying to accomplish and what am I trying to accomplish computers have none of that, and it's the fundamental way that we as people interact with one another. We understand with each other as trying to accomplish, and we then react accordingly. And so I think that's the thing that the thing to look forward to the most about the future of computing is

if you have a device that is instead of being throttled on the bandwidth of how much information you can give it, is actually throttled in the opposite way. It has total context on you, what you've been doing, what you're trying to do right now, what's possible, what tools are available to you, and then has enough intelligence to do something useful to that to give you either visually through a display or audibly to your ears or through haptics,

the hints that you need to then just go be successful at what you're trying to accomplish. It starts to feel, instead of, hey, I'm a human with a great tool, it starts to feel like superpowers. It starts to feel like, oh, like, I just have tremendous more capability than I used to. Okay. I have some follow up to that, but I'm going to ask them on the other side of this break, where I'm going to get a little weird.

So we'll be back right after this. Hey, everyone. Let me tell you about The Hustle Daily Show, a podcast filled with business, tech news, and original stories to keep you in the loop on what's trending. More than 2 million professionals read The Hustle's daily email for its irreverent and informative takes on business and tech news. Now, they have a daily podcast called The Hustle Daily Show, where their team of writers break down the biggest business headlines in 15 minutes or less and explain why you

should care about them. So, search for The Hustle Daily Show and your favorite podcast app, like the one you're using right now. And we're back on Big Technology Podcast here with Boz, Andrew Bosworth, the chief technology officer of meta. Boss, great to see you. So I promised some weirdness before the break and let's do it. Is this, you talked about like humans and computing getting closer and being more context

aware of each other. Is this AR move just a step toward where we eventually go, which is that we just eventually insert computing into our brains through brain computer interfaces? It's a great question. And you already have the neural control for the device. That's right. The neural interface on the wrist, which the, you know, is already very successful. And we think that's just the beginning.

We think we can do a lot more with that over time. Right. And for listeners. So like you have this wristband around your. wrist and you're pinching and it selects you top your middle finger goes home you can scroll by just like putting your your thumb and pushing it on your hand like it's nuts but sorry go ahead and it's just the beginning over time we think we can do that without having any gestures at all just think it just having the intention expressing the intention down your control system so a couple

things you know eight years ago we did an invasive neural implant in partnership with UCSF and it was a patient who was nonverbal due to a traumatic brain injury, and we were able to provide them a 400-word lexicon by virtue of these implant that we gave them. And the reason we did that study was then when the person was using the control system, we put super sensitive EMG sensors on their head to see if we could detect any pattern of the underlying neural activity without having to be invasive. We couldn't. It was actually your skull as an incredible insulator of electrical activity and it's kind of scrambled together. The challenge with neural implants, we've actually been doing this for even longer than that. You know, the first people to be able to control

a computer mass through a mouse through a neural implant goes back into the early 2010s, if not earlier than that. There's a couple challenges, one of which is that the brain myelanates the things that are put there, so they become electrically inert over time. So they have to be removed and replaced. leave them in long enough, they can prove fatal or at least have very severe complications. So you wouldn't be talking about with the technology that we have today, with the materials we have today, and what we know about how the brain works, you're not just talking about a one-time brain surgery. You're talking about annual brain surgery. And annual brain surgery, for me, is like a non-starter as a consumer product. I'm just going to go ahead and assume that's not

going to be the way things are going to go. And so I'd rather wear the glasses than that. Yeah, right. I think the glasses are a popular option relative to that. And so we were tracking that space carefully, certainly, and we're thrilled to see people investing a lot to try to see if they can crack those problems. But one of the things that we keep finding is you don't necessarily need to go to that point. And the glasses are pretty spectacular. And like it turns out, once you get to all day wearability with their lighter and they have full battery life, the ability to take them off. It's actually pretty nice. And the ability to just like have, you know, to kind of be restored to that center. Does it get to implants at some point? Yeah,

maybe. I think that's probably probably light. I don't know if it happens in my lifetime. really um it's like that feels like um even distant by my standards i think we like have in and maybe AI is a key to unlocking this more i mean i studied one of things i studied as an undergraduate was a lot of neurobiology was kind of part of my interest in in AI and in computation um and like the the challenges that we have understanding the brain uh feel like an order of magnitude bigger in terms of just complexity than anything we've managed to actually map out so far So this idea that we're going to merge with AI, you're not buying it? No, I don't see that happening in the near future.

No, I'm not saying... Would you do it if you could? We are getting weird. I appreciate you. Good is delivered. I promised. I think, what I think, I really like, there's a book, Hofstetter's book, I'm a Strange Loop, which has this idea of consciousness as an emergent property of sufficiently self-referential

systems. And it's a beautiful book. and rumination on consciousness from, you know, one of the best to ever do it. And it also is kind of a beautiful pan to his wife, his late wife. And he has this idea that his wife who had passed away unexpectedly, there's like a version of her consciousness running in his brain, right, like it's been uploaded to his brain. Because he can imagine with high fidelity, and this gets back to what I said earlier about, you know, the idea of understanding someone else's intentionality and agency. He has like a pretty reasonable approximation.

of her consciousness, in his consciousness. It's true. Like, if you're close enough with someone, you can effectively, like, speak with them if they're not with you. Absolutely. And we have a model, a version of their consciousness running to it.

So I think if you got to the point where an AI was ever present in your life and helping you, guiding you, you're confiding in it, maybe even your innermost thoughts that you wouldn't normally even share with a close friend or someone else, that it would have in its,

systems, a version of your consciousness, a queryable, like interactable, animated version of your consciousness. Is it you? I'll leave that to the philosophers, but it would pass as you in many contexts. And obviously there's a lot of questions that raises in terms of privacy and security and how you control these things. But assuming we can conquer all those things, yeah, I'd be thrilled. I'd be thrilled for my kids, if something were to happen to me, to have access to even if it's a simulacrum, a construct that approximated what I was for their use. And I'm sure there's going to be a whole new set of challenges that that introduces. But I think that's a version of the future that is likely.

Do you think that the current systems are conscious or approximate consciousness? No, I don't. They're not getting close to it. No, I don't think so. I think it's a category error, to be honest with you. And I'm a huge, I find this is a funny thing that happens to me these AI conversations because I am both one of the most fanatic believers in the power of large language models. I talked about how I think of it as a fundamental computing paradigm shift of going from human in the loop computing to human in the loop AI where the AI is much more informed by modern sensors and availability of information.

So I'm a tremendous believer in it all. And also I do not believe it is even the kind of thing from which consciousness exists. It doesn't exist in any, again, to my point about Doug Hofstadter, it doesn't exist in any self-referential loop. In fact, it doesn't exist at all except when I instantiate it to exist. It's a very, very cool associative database. It reminds me, actually, when I was an undergraduate, the reason I studied this computational neurobiology certificate that I got was this idea of how could I, with a computer, emulate

human associative databases. Human memory is this wonderful associative tool. where when I say the word purple to you, this huge network of ideas pops into your brain. And it's like very rich and complicated. And it's very personal to you and what your history of that language is. And it's a really hard problem for computers to do that. Like it's to have that kind of loose and generative associations. And so I can say to AI today, I can say to it like imagine a purple elephant.

can do it. And that was kind of wild before. There was no way to do it before. So I love it, but I do see it as fundamentally an associative database and an important, maybe a component of a total system. But I tend to think Jan Lacoon's critiques of it are spot on. Are spot on. Okay. So let's rapid fire through the rest of my questions. We have a few minutes left. Let's do it. What happens to VR or these mixed reality devices, the quest and the Rayban Metas if Orion takes off? The analogy I make is to your phone and your laptop. I mean, the truth is there's a, and this is what we found so much. And the reason they have different operating systems, though we have tons of shared technology layers and sensors and hand tracking and

eye tracking and voice and AI, shared technology layers, different operating systems is because the interaction design is different. And the reason that's different is because the use cases are different. Just like, I tell the story all the time internally, but most people who are in information work understand it. Sometimes I'm on my phone and responding to an email. My phone does email, fine. And I'm like, mm-mm, not a phone email. I took my phone down, get my laptop. This is a laptop email. I need to have five browser windows open. I need to have, like, I need to, this is a real, I got to write this thing. And so I do think of these things as like fundamentally different pieces.

I certainly watch quite a bit of short form video on my phone. I also get the big screen at home with my wife and kids and watch a movie. And it's just like they're different things. And I can't describe to you in human terms why they're different, but you and I both know that they are different and they feel different. And so I think there continue to be, even when you have these wonderful AR glasses, things that can be done in virtual and mixed reality that can be done no other way. your augmented reality glasses will never convince you that gravity is off because you can literally see the physical world behind them. When you're in the headset, you can absolutely play as if gravity doesn't exist. And so I think they both serve tremendous value and I think of it as

being a phone and a laptop. Again, speaking with developers, they have lots of options, build on the phone, build for these devices that are starting to get a user base, but aren't. aren't anywhere near as big. It's tough to reach scale. What's the one minute pitch to them if they're thinking about building for these devices, these VR mixed reality or AR devices? Or even AI voice. For sure. The market's super efficient. And so, yep, there's a lot of people on the phone, but it's also the hardest place to get noticed or stand out. It's the hardest place to do new things because there's nothing new that you can do there. It's all been done before. Phones haven't changed in 10 years. Whereas these new devices are risky because they,

audiences are smaller, but you have an enthusiastic audience who's deeply engaged and is absolutely desperate to see new things, to try new experiences. So if you have a great idea, you know, it feels like a gold mine. And walking around Connect the last couple days has been so energizing because what I'm surrounded by the people who have thought of ideas that we never considered when we build these platforms and they're just powering ahead and pioneering those things. So I can't, listen, every developer should kind of probably play to their strengths and understand there's these devices and AI in particular all live in very different contexts. But it's going to be hard to break out on the phone and it's designed to make it hard for you.

Yeah, because, I mean, without the applications from developers, it's going to be tough to find, like, great value in these devices. That's why every time we talk about Quest, we talk about the ecosystem. Right. It's not just the devices. I'm not here to sell devices. I'm here to build an ecosystem. And that's about building, getting more developers to build great experiences to get more consumers and more consumers. which make it attractive for more developers.

Okay. A couple AI questions before we end. We have a debate often on the show. Is it going to be the development of the foundational models or is it going to be the application of them where the real value is found in artificial intelligence? Obviously, you guys are open sourcing the foundational model. We're going to tip our hand on that answer.

You have an answer. So talk a little bit about do you see promising applications being built? And what happens to the big foundational model companies if this stuff is all effectively commoditized. Yeah. So I was one of the loudest voices internally encouraging us to open source Lama 1, Lama 2. Obviously, you see how Bauden Mark is with his announcements around Lama 3. And the reason for that for me is like it's a tremendous benefit to our company.

Whether we make the AI or someone else makes the AI good, I'm the only one who can use it in our products. And that's we have a direct relationship with our consumers. So we can make our products better for consumers, whether we make the AI or someone else makes the AI. So that makes this a compliment to us. And you want to always commoditize your compliments. You always want to make sure that the thing that you depend on is like cheap and getting better by a lot of people, not just by your effort and money. And so strategically, it makes perfect sense. But more than that, we're a technology company and we've always seen AI in particular as the data we bring to it for our own applications.

That's special to us. And the application itself is special to us. But with PyTorch, we've always tried to make sure that the middle piece, the infrastructural pieces, those are for everybody. And that makes our developers have better apps when they're on building for Quest. It makes sure that the partners that we're working with for Instagram are having a better experience that's really good for us to have the entire consumer base, whether they're on our app or somebody else's, have a better experience. I think for us, though, to be honest with you, it's hard to say what happens to these companies. I would have been wrong about AWS if you'd asked me this question a long time ago because we were building a lamp stack.

We were doing it all ourselves and everyone was doing it. It was fine. And then it turns out if you can abstract away at good enough economics, the complexity of managing a bunch of web servers and load balancing and scaling up and scaling down, there's a business there. So I think there probably will be a business there for some of these AI hyperscalers. It's probably, you know, potentially a big business is big business. but they've got to get to that point where the economics make sense and it's sufficiently well abstracted from from them. I think the bulk of the value though. So there will be valuable companies there, but the bulk of the value will be in the application side. So I was driving

down 101 today, which I've been doing for, I was thinking about it nine years coming down to visit you guys. It's an interesting moment, right? We are ahead of November 2024. We're not talking about politics. I was also like thinking about the different types of technology that I saw when I came down here and I mean you know credit to your teams but it was like stickers and messenger and 360 video what does this technology moment look like to you because it I mean seriously we're talking about artificial intelligence that can effectively get to know us glasses I'm looking through earlier glasses that you can like see through that's right and perfectly clear yeah those things are clear that's right no I'm like thinking about the lens yeah totally

just like uh it looks like a VR device pretty much but I mean from the inside So talk a little bit about this moment. And do you think we're going to continue the momentum or the tech industry really will continue the momentum because it seems like progress is nuts right now? One of the things that we talk about all the time as technologists is these overlapping S curves where a new technology is created and then has this period where it struggles and then it gains adoption and then it explodes and it's everywhere. And then there's still a tremendous amount of value at the top, but it starts to like the rate of change starts to slow down. And so, you know, I'll take an easy one. you talked about messenger stickers, and it's easy to make fun of messenger stickers in retrospect, except that they're still super popular, and we wouldn't trivialize emojis.

This idea of expressing yourself through visual form as opposed to expressing yourself explicitly through text was at first a laughable construct that our generation got made fun of a lot by the older generations, and then now is like ubiquitous among all generations and arguably helps us express emotions through a medium that otherwise lacks the ability to do so. so cleanly. And so I don't like to trivialize any of the things that have come before, but I do think it's fair to recognize that 10 years ago, we all felt like we were at the top of an S curve of really what was the kind of tail into the web. We're riding the mobile curve straight up the ramp at that point. So the web is kind of starting to trail off. Mobile is like

still going gangbusters. And it's a tremendous, you know, foresight for Mark to think, what's the one behind that? Like, what's the one? It was crazy. When I was here, I was trying out your AI assistant and messenger called M. that was actually contractors in the WhatsApp building that were approximating what it would look like. And we were all getting this wrong as an industry two years ago, a year ago. You know, I remember when we had a portal,

still one of our most beloved devices, and I'm sorry to all of those for whom we had to shut that down. Yeah, look into the camera until Katie Nautopoulos. Katie, you know how much I applauded. It was, and our, knowing more than our own employees, our employees were devastated. And we did that tremendously popular product,

small market, but tremendously popular product. And we had our assistant on that. and we had an Alexa assistant as well. And when we got into building our own assistant, I could not believe how poorly they scale. Because you have to build it for every language, for every device, and for every use case.

Oh my gosh, that's not software. That's the worst scaling laws of all time. And so even back then, I knew this isn't it. Like, we didn't do it. And it wasn't until you get self-supervised learning

combined with the breakthroughs that Google led in, in how they built these models and then ultimately chat GPT that you see it like ah that's what it is like that's the thing that's the one that we needed all along and we didn't have it yet

you know what they say in the industry in the other way there's no bad ideas just bad timing and M was probably one of those okay Andrew Bosworth great to see you thanks for having me in thanks man thanks everybody for listening and we'll see you next time on big technology podcast