Upgrade - 620: Sweating the Details

Episode Date: June 8, 2026

From Relay, this is Upgrade. Episode 620. Today's show is brought to you by Keeper, Squarespace, and Factor. My name is Mike Hurley, and I'm joined by Jason Snell. Hi, Jason Snell. Hi, Mike Hurley. And we are together recording at Apple Park today. Because we've just, we've left multiple events.

Oh, yeah. We've seen it all. But we've got to talk about. We haven't seen it all. We've seen some of it. There's a lot more. But I have a Snow Talk question for you. Okay.

It comes from me. And what I wanted to ask you today, Jason, It was based on a conversation we had yesterday. Okay. So you were kind of thinking how many WWDCs you have maybe been to consecutively? Yeah, I mean, there was the one where nobody was here, but we were all there. You attended in the way anybody could attend. Exactly, the most, the max.

How many is that now? I actually don't know, but I think this is either my 30th anniversary or it's my 30th. I think it's your 30th. I've decided that because I think that's fun to say this is your... It was 96 or 97. Because, you know, Tim said today it was his 15th. So you've got double on him. I know.

Tim Cook. And he can never catch you now. He can never catch you. He can never catch you. So congratulations to you. I guess I win. So congrats. Tim Cook, who I shook his hand today.

Yeah, we'll talk about that in a minute. Okay. I guess I won't talk about it now. I'm going to talk about it now because first you have to do the draft. Oh, yes, we do. Okay. So this is a hard draft. To score.

This was a very hard job. We did bad things to ourselves. Yeah, I made some picks that in hindsight was very hard to try and work out if it was accurate. And also the main one I'm thinking of is that we do not see a return of all features promised at WWDC24. But we're actually kind of breaking from tradition a little bit this year because me and you were struggling so hard to score it. Yes. Zoe, who puts together Upgrade. They did a scoring.

Yeah, she scored it for us. We're just, yeah, and we're just taking her scoring. I'm going to take her score. And also, I opened two image files here on my computer, and one of them is our draft score from Zoe. And the other one is that slide, that completely deranged slide that had a thousand features on it in the tiniest print. And it was up for like a very small amount of time. Okay. And I had multiple people send me synopsies of everything in, and those were too long.

Yeah. So we're just going to go with this, that we're going to go with Zoe's choices. So I saw Federico take a picture of that and then send. it to Kodax. He did. He did. And somebody emailed me. Somebody emailed me who had done the same thing. People like lists. We love them. Thank you. So let's run through our picks real quick. Thank you. We'll go to see yours and we'll say what. And again, it's like, bearing in mind, we've scored this to the best of our ability. The platform state of the union is happening right now. And I think there may be a couple of things in there that our game is about what's in the

keynote. Exactly. Not what's on the website, not what's in the platform state of the union, not what Craig Federici told us up close and personal shortly thereafter, which happened and we'll get to. So, yes, I said the Photosap gets new AI editing tools. Yep. Got it. More third-party AI support in the OS? Didn't get it. Doesn't seem like it.

Didn't talk about it. We'll talk about that more in a bit. Shortcuts gained support for Apple intelligence-powered creation. Got it. Yep. Specific acknowledgement of the Mac is a popular platform for AI agents. Didn't happen. Craig Federigi mentioned it on stage in the event we went to.

At one point during the presentation. Well, you'll get a point for that in a moment. Oh, yeah. I got camera app, gains an AI mode, image playgrounds, get quality upgrades, Aval Intelligence Wallpaper Creator, at least one feature is labeled as coming later in the year. No new hardware announcements. Mike Rockwell appears. Boy, he appeared.

Here's a good one. You have a problem with that? No, I just, you know, you pick it in the draft, and then he shows that. He was very prominent. He was very prominent. The finder mascot character. appears. Somebody sent us a screen grab of a little finder person. I'm going with the finder mascot.

The finder mascot. I just think it's there because now there is there's a war amongst the naming and there's no official name. Little Findy. Yeah. I like that one too. Behind a like behind a display at one point. I didn't get Vision OS mentioned in passing. No specific features. There were specific features. In fact, maybe we'll get to this later. I felt like this was the revenge of Mike Rockwell. Finally being able to create all the Siri features he couldn't get when he was in charge of Vision OS. Okay. Siri can handle multiple requests at once.

That's absolutely true. And clipboard history added to iPad OS, not in the keynote or the slides. So maybe not at all. How'd you do? So Siri gets a new chatbot-style interface. It did, and there's an app. Got it. Siri interactions moved to the Dynamic Island, which they have.

The camera app gets customizable UI. Now, I didn't get this point. I'm really interested to see if there's something in the... there for this because that was a very heavy rumor that did appear to come to true yeah doesn't seem to have happened spotlight improvements for ipadOS to be more mac like there was no indication of that apple intelligence grammar checking tools i got that syria remembers context i got that improved gemmoji quality we didn't see that no features are labeled as coming later in the year the syria ai is yes john turnus appears which he didn't until later we do not see a return of all the i i features

promised at WWDC 24. Zoe marked this one as I got this one. I don't know. This is one of the hardest things. Yes. We shouldn't have picked this, but yes. Because it was like, that was going to be way up. What were all those things?

Do you remember? I did some Googling. I couldn't do it. Significant or noticeable improvements to liquid glass on macOS is actually everywhere. As well as Mac OS. But they did, they singled out Mac OS. They really did. Someone says a JENTIC, which happened.

Developers get access to more powerful on device models, which they have. and an expansion of satellite features for new parts of the operating system. No. So that was 10 points to 8. Congratulations. Thank you. Thank you.

I am. It's a tightly champion. A tightly fought. 30 years at WWDC teaches you a few things, Mike. We did, but we did say on the last episode that I have won more than you. So you're getting close to tying me at WWDC. Yeah, 30 years.

Details. That's what I say. I remember them all. Details. Before we move on, We have an important update, which is to talk a little bit about the designing California Kickstarter campaign. Yes. So we are running a Kickstarter campaign right now to fund a new show, which is going to be hosted by the two of us, which is an Apple history podcast.

We're going to be digging back through Apple's 50 years and telling stories. This show is going to happen. This show is now going to have 50 episodes a year because we've met the goals that we needed for this. But as it stands right now, I haven't actually looked in the last few hours because we've been somewhat busy. A little bit. I'm loading a page now, but we have very much exceeded every possible goal that we ever could have imagined. Thank you to everyone who's chipped in and supported the campaign. We're very close to have raised $160,000 for this podcast.

So people want it, which is a great thing, and we are so grateful. Please go todesigned.fm, where you can sign up, you can back the campaign. This show will be coming later on in the year. We have a couple of stretch goals. now we're once we get if we get to 175,000 we're going to be doing more art, more music because people have heard in the episode the preview episode that came out on Thursday

which is our first episode in the road to Apple 2 we debuted our theme music from Chris Breen who produced the music for this show as well and if we reach $200,000 we are committing to a live event in both London and San Francisco that we will both be in they may be meetups, they may be live shows we just don't know

We don't know. We're committing that we will do that as part of the show. It'll be somewhere live where you will see us. And throughout the campaign, because there's still a lot of the campaign to go. It unbelievably has only been a week. It feels like it's going for a really long time because it's been very overwhelming in the best possible way. But we have more of the show to come. So coming this Thursday is the second part in our series about The Road to the Apple 2.

And I will say, I got lots of great compliments about the first episode. I'm really pleased people enjoy it. I think as the season goes on, it only gets better. the story becomes more and more exciting and weird and wonderful. So I hope people will enjoy that, but we also have a lot of stuff to show like what our prints are going to look like, what our pins are going to look like, and it's all going to be coming out over the course of the campaign. So please go todesigned.fm, where you can find out more and back the campaign.

Do you have anything you want to add to that? Nope, just it's overwhelming and great, and I'm looking forward to getting started, but we got WWDC-26 first. Yes, we absolutely do. We absolutely do. So I wanted to talk a little bit about what our day has been like so far. So, you know, like I mentioned right now, we're recording this while the State of the Union is going on. So we had the keynote.

We watched the keynote together. It was really nice. It was outside. Short keynote. Yeah. I would have won the tiebreaker. Like 70 minutes? Yeah, 70, 75.

75 with the song at the end. Yes. Which prominently featured friend of the show, Casey Alyssa's app call sheet, which was really good. Very cool. Amazing. We all pop very hard for that. That was very exciting. Then afterwards, we went to something called a tech talk.

Yeah. Now, after WDC. 2024, they did one of these, where Craig Federigi spoke to an audience. Were you in that audience? Yes. About, it seemed to be mostly about private cloud compute, but it was a lot about Apple Intelligence and kind of talking about some of the technical aspects of what was in Apple

intelligence to provide more background. We had another one of those today. So there is actually going to be a lot of the things that we're going to talk about when we talk about Apple intelligence, a lot of it will be informed by having had a presentation and some demos and some question and answer that gives a lot more information and detail. So that's a cool thing that we got to do in advance to do in the show today. And that was Craig Federigo led it and then they had Sebastian from the Intelligence Experience team, Mike Rockwell was there, who was the head of Siri, and Amar, who is the lead on AIML technology at Apple. And they answered some frequently asked questions and then answered a few submitted from the audience.

And yeah, there was a lot of extra detail there about what they're doing. But we were sitting in the second row. We were. They just let us. We just sat in the second. Well, I think Federer Federico was already there. No, you did it. That's true. And so we just go sit down.

So we sit down. They didn't need the second row. They had the first row they needed. We didn't know they needed the first row because all of a sudden in front of us appears. Greg Jawsweak appears. John Ternis. Well, no, no.

Well, there was a break. Greg Jawsweak and John Turner show up. Yeah. They say hi to us. They said, John Gruber makes fun of him and says, oh, you can show up late now. Yeah, you got the new job. Says, I remember the Ternus that showed up on time.

Yes, very funny. And I thought to myself, well, you know who isn't here, though, yet. And there's an empty chair. is Tim Cook. And that's the job where you really get to show up right at the last minute

and just kind of appear stealthily like Batman. And then I'm sitting there talking to a person I've known for a long time from Apple PR. She's right in front of me and then all of a sudden somebody comes up to her and goes,

you got, you gotta go. And that's because Tim was there. And then Tim sat down, plop down right in front of me. Yep. So I had Tim, John Turnus,

Greg Joswiak, all kind of in a row right in front of all of us. So I was sitting next to John Gruber. Yeah. you know, Grub was making funnage on time

as he turns around and he says, hi Mike. Hi, John. And well, you guys shared a special moment and he remembers it forever. I didn't know

that he remembered who I was and I said, congratulations. He said, thank you. He's got to be good with names. He's got to be good with names. Well, he must be

because we met once a long time ago. Maybe, you know, maybe they have like a little, uh, they do like a guess who kind of game. Maybe. Just like lots of,

they took a picture of us while we were waiting for them to come in and then they went over. and said it's who these guys are. There you go. Or maybe there's some kind of secret glasses or something.

Pretty wild. And it meant that when, you know, because Craig Federigi can be a little freewheeling on stage. I get the sense. Every time I see him presenting unrestrained in a live format, I keep thinking of the moments

where I know that he's going to come up with an example or an explanation or a description that's maybe not been approved entirely by Apple marketing. And I can see his brain working. And sometimes I think those come out. I think he's much better at it than he used to be at kind of restraining himself.

But let me tell you, it is a different position to be in when you're sitting right behind his bosses, because then I'm almost like putting myself in the shoes of Tim Cook and John Ternus and Greg Jawswiak and thinking, Craig now, great, easy Craig. Come on now. Come on now. Yeah, it's very funny. Kind of a loose canon. That was, it was quite an experience. And that was a really interesting thing to see. is the same we're going to be talking about that, but like some of our experiences and some of the things that we learned in there,

which can help inform. I did want to say, like, so this year, there's a lot of detail on the keynote, and it was a very condensed keynote. So they packed even more information in there, and I know that there's going to be lots and lots of stuff that we're going to be learning over the coming weeks, as always. Yeah, this will be our first high-level take about what's going on.

But this year was, you know, like it reminded me of 2024. because 2024 was the first Apple Intelligence keynote. And in 2024, I was bewildered. Like, I couldn't put my thoughts together. And I remember that year, you had a very difficult schedule. So we ended up recording the show on Tuesday. Yeah. And I was very happy for that because it gave me time

to kind of collect my thoughts because I couldn't really conceive of some of the things that we were shown. This year, I would say it feels overwhelming, but in a more understandable way. Like, I feel like there's a lot of information, but I'm not, like, blown aside by it. In fact, what I would say is this,

you can see how Apple has thought really hard about how to roll out the information they're rolling out. And what, I think one of the interesting questions is what is the keynote at WWDC for? Because it's not to communicate new OS features to the customers. It's really not, because they don't care. They'll care the moment that they're off,

an update in the fall. But right now, they don't really care. So it's more about general industry messaging to, you know, the industry in Wall Street. It's about communicating to us so that we can start thinking about this and communicating it. And it's about communicating, obviously, primarily to developers. So their first segment was not even, I would say, information overload. It was like a light gloss on some highlights.

and then, you know, stay tuned for more. That's where that slide was that had a zillion features in it that was up for a moment. And then there was the second segment, which was about privacy and safety. And then there was the third segment, which was about AI.

And that was... And it was a short presentation, right? And that AI segment was reasonable. It wasn't information overload. No. It was clear. So there was a lot of information being imparted,

but it was really... really telling that like the first segment felt so packed because they had decided we're not going to spend 45 minutes on this. There are other venues where we'll talk about this. The last part really important, but not like overwhelmingly complicated, especially since you could argue they're restating some of their arguments from two years ago. Yeah, there was like a lot of the big features are things that we've had multiple years to consider.

Right. Like that was the difference last time. It's like, you know, like in 2024, I was really, I was kind of blown away and couldn't understand image playgrounds. And I still have a lot of complex thoughts about it. But these thoughts now are just building on the things that I felt last time. And so, I mean, I guess that's the thing they've kind of gone back and have re-implemented a lot of what they wanted to do. And they've gotten to this point again. Yeah, I was, I was going to say, and we're going to get into it later, but I'll just throw it out here now.

There's one way you could read this, which is Apple saying, actually our strategy is the same as it was two years ago but we needed better models and our partnership with Google has given us better models. Yep. So we can do it now. Yep. It seems like it. This episode of Upgrade is brought to you by Keeper Security. Logging in used to be a hassle with so many accounts to keep track of.

Keeper creates strong and unique passwords, fills in my passwords automatically, remembers logins and keeps everything organized across all your devices, so you can get to what you need instantly wherever you are. Keeper is a password manager that creates those strong, unique password you need for all of your accounts, stores them securely in one place, and logs you in automatically across every device you use, so you never have to remember, guess, or worry about your login credentials again. For businesses, Keeper takes the chaos out of managing access for teams so they can securely share logins, control permissions,

and protect sensitive data without slowing down their day-to-day work. What people really notice is how simple it feels. Security features like autofill and face ID or fingerprint login quietly make things easier, so best-in-class protection doesn't feel like extra work, it just happens in the background. From individuals to large organizations, millions of people worldwide rely on Keeper to keep passwords organized

and accounts protected. Keeper helps you to stop thinking about your passwords altogether and just get on with your day. I really care about my online security, so a password manager is a must. I know how important it is to have strong, unique passwords. There's no way I could remember them all, which is why a password manager like Keeper is an essential tool.

Keeper has everything you need to manage your passwords. They make it really easy to generate them, save them, and access them from wherever you are. And I love that they have support for two-factor codes built in, so it makes their extra layer of security even easier to manage. manage than dealing with emails or text messages. They also have auto-fill taking care of with their Keeper-Fill technology on every platform, which is a must for me. Right now, Keeper is offering listeners of this show 60% of personal and family plans at KeepaSecurity.com slash upgrade pod.

This offer is only for listeners of this show. That is, k-e-peer-security.com slash upgrade pod for 60% of personal and family plans. Make sure you use that link so I know we sent you and you get that 60% off, which is a huge deal at keepa security.com slash upgrade pod. Our thanks to Keepa Security for the support of this show and all

of Relay. All right, so let's start digging in to what's going on. Let's do it here. And we're going to, I want to start by talking actually starting at the start, which was the platform improvements. So this was a, in lieu of doing all of the platforms this year,

Especially because most of the features are Apple intelligence. Instead of going through each platform, they did kind of a big overview, which was a, I would say, an extended way to say, no new features, bug fixes, and improvement, that kind of idea. Yeah, it's the, it's Apple's equivalent of no new features, bug fixes and improvements, which is a slide with hundreds of things on it. But there isn't your big, like, hey, this is, we've redesigned home. screens, we've redesigned lock screens, we have a new design language, like, it's not that.

It's not what they're trying to do. Everything we're currently doing, and we've made a large series of refinements. One of the most interesting things to me was the long, long list of speed improvements that have been made for certain things. You know, like, we're absolutely going to load faster. You're going to end up with, you know, much quicker experiences across the system. AirDrop is faster. Yeah. Files in iPad OS transfers at Mac speeds, which I thought was just like a funny way to think of it. 70% faster getting your photos to show up after you've taken them into the photos library, which is a huge one.

Yeah, that's clearly, I mean, they actually said, we tried to find the places that responsiveness was a problem and fix them. And that's like, that's what people want. And things like on the website, you know, like once having a send indicator, and messages for specific messages. Like, I've had this where, you know, I'm not connected to the hotel Wi-Fi in my hotel. I'm just on cell service, and I'm sending images to Adina or whatever,

and I send her a photo, and then a message explaining the photo. And she'll have to wait for the photo before she'll get the message. And it's like kind of breaking those up and showing independent kind of send indicators. It's little bits like that, where it feels like what they have done is one way or another, collect up a large amount of annoyances that people have with the operating systems. The line that I underlined in my notes was,

Great OSs are built on sweating the details. And, you know, again, it's marketing, and they're not going to say it. In fact, talking, I enjoyed how she struggled with the idea of describing speedups because she had to say, of course, everything's fast already. Yeah, I know. And sweating the details, they're never going to say it. But I think the implication here is, we heard you, you feel like we haven't been sweating the details on a lot of parts of our OS's. And we're going to make good on that this year.

And it's a year where they can. Exactly. Well, yeah, I think this is a great approach to say, look, AI is number one. We're going to focus a lot of our effort on integrating AI wherever we. can. Number two is sweat the details. It's not another, we're not going to do another big initiative. We're not going to do a huge overhaul.

Number two is get things faster, get things more stable, make nice incremental improvements to the stuff that's already there. And from a marketing perspective, they don't need another tent pole feature because the AI stuff is the tent pole. They will stand or fall in the fall based on the AI stuff. So they don't need to gin up five. more bullet points of apps that they're releasing or re-releasing or whatever. And so it's the perfect time to say, okay, let's take some pride in our work and revisit what we've done and make it all

better for our customers. And they should. I mean, honestly, you can't do that. I get you can't do that every year. I mean, you should have a bar of quality. But like, every so often, you need to do this. You need to say we're no longer rushing headlong into another wave of new features. We're going to tighten all the bolts. Well, also just like logically you can't do it every year. Yeah, certainly. Because you would run out with things to do. Right?

You can't keep making the faster thing faster, like year over year over you. Well, that's true. But you can find other things to you. But you want to, after years of breakneck development to take some time and say, you know what, I'm not. Because I think the impression I get is a lot of these software teams, there's always pressure for the next big. big 10-pole feature of the next thing. Which is the issue of year-over-year?

Software upgrades. So to say this year, it's sweating the details is what we're doing. That's the other thing beyond AI. I think, you know, as a user of the platforms, I think that's great. And I like it as them kind of set an example too, like, you know, for all developers in a way. Like this is, you know.

Sweat the details. You're going to sweat the details. But that's an important part of making good software is being that particular over it. So let's talk about liquid glass. Yes. Because they actually spent more time talking about liquid glass than I expected. I wrote some fun things down. Okay.

I wrote down, we continue to iterate on liquid glass. I thought that was fun. I thought, we appreciate your feedback. There were some chuckles in the audience at that. We've made additional refinements. And then also there's a slider so you can just make it whatever you want. What I liked about the slider, though, was some people want it more glassy. Some people want it less.

I love that it was like, there are some people. They just don't look through that. They didn't want to see it. see the apps anymore. Just text floating above nothing. I'm probably going to go all the way. You are. You are.

I'm going all the way, baby. I want to see myself in my phone. Yeah. Yeah, I get it. I get it. I love it. But I think, again, how do you give a marketing message that is, we heard you, without saying, we know you hated it or a bunch of people hated it.

And we're not going to throw it away, but we're going to fix it. Instead, you say, we continue to iterate and we appreciate your feedback, which is what they said. I respect it. Yeah, and that's... I think it's great. And I'm so happy that they didn't retreat. If you go back to us a year ago, you will hear us say the exact same thing, which is they're going to go too far. And then the next year, they're going to iterate and refine and walk some stuff back and fix some stuff.

And that's just how it is. And here we are. Yeah. I think it's great. And I mean, it was particularly interesting how much time they spent talking about the Macs specifically. Like, I don't think they actually spoke about any other platforms specifically. It was just the Mac. It was like we're making changes everywhere.

We've made it better. We've also given you more customization. But here's how the Mac is better. Right. Because there were all those ways where the Mac did not work with liquid glass. And it's like sidebars, corner radiuses. A toolbar.

Yeah. Yeah, exactly. Color back in the icons in the sidebar so you can tell whether it's front most or not. And they've removed the icons from the menus. Stephen texts me. Oh. See?

He sent it to both of us. They've removed icons from the menus, which, you know, it's fine. I think they were fine. I don't want to say, I thought they were fine. I'll say goodbye. Look, well, I don't think they were fine because I think they were the wrong icons. Like, I could see it in another circumstance if the icons were good. But if you can't do good custom icons for all those things, you just got to let it go.

Yeah. Well, because it's like an infinite amount of icons. Yeah. And like you've got to icon, icon, I can't think of a word, with an icon show, complex things. It's very difficult. And then you're reusing icons to mean different things, which is also bad.

Yeah, I think, good. Again, they heard the feedback. Here's my question for you. Did they do this because there's new leadership in the design team? Or would they always have done this? What do you think? I think it was easier because there was new leadership.

Maybe specific things could be changed differently. Sure. I think some things would have done anyway. I honestly think the Mac stuff would have been done anyway. Because the Mac, my whole Tahoe take is that they didn't get to it until too late in the game. And so it was not even finished. And we can argue about it. I mean, I sort of feel like the Mac was spared because they didn't do it more aggressively. But I think that would have been addressed this year because it was broken and they needed to fix it. But you put new leadership in place and it's a lot easier

to change things because you don't have, you know, a new leader can say, let's tweak it, where the old leader will be like, well, I know I approve this thing, but now I'm going to de-approve it. It's like, that's a lot harder for somebody to do with that level. But obviously, with this kind of in the platform improvements, there was like grab bag features that they did. Like, ICloud shared albums will now be full quality images, which I had thought was already the case. Well, there's, I mean, guy who wrote a photos book here, there are different kinds of shared albums, some of which were not at full resolution,

and some of which were. And then also that you can invite your friends on Windows and Android, and you can have them contribute. What do you think that's going to be a web or an app? You are going to be the web? That's nice, though, right? That's nice, though, right? Because then you'd assume they can also download

the images as well, you would assume, right? Like, it's not just uploading, so you'd have a link. I guess you can already do that, right, but you can't contribute. And then, like, AirPods' custom EQ. Sure. Which just feels like a real esoteric feature that you can add in a year like this, I guess.

I really liked, this is why, again, it's not no new features, because there's actually a bunch of pretty great new features on here. The fact that they spent a lot of effort, apparently, looking at what happens in network transitions. So when you walk away from your house, your iPhone is much better at realizing that I can't get on the Wi-Fi anymore, but I'm on the cell network, and not have you have a 15 or 30-second period where there's no network connection. I like in these kinds of presentations.

You know, you're saying, like, oh, they're like, things are already fast. But when they're like, you know what it's like when you have to turn off the Wi-Fi. Who's the guy who did that? Who made this natural process? I'd like to find the guy who did that. We're all here asking. Yeah, exactly. We're all here asking the question. Who did the network transition thing bad?

And then like Vision OS got more features than you would expect. Well, Revenge of Rockwell. Creating your own environments with panoramas. Right. So does this mean other people can make them? Or can you only, so it's like can you only make your own? Like you can't have like, can developers make them? I don't know. We don't know. I don't know. It would be so funny if the thing we've been asking for since VisionOS came around, which was let third parties make environments. They didn't do that, but they let you make your own.

Out of a pan-o. And it's like in a year like this, right? This was the year for third-party watch faces. It's true. Right? Like that feels like it fits in this. It's like a long requested thing. But that is clearly just a thing we're not.

Yeah, I think they're never going to do. They do have a new app grid, which I missed because I was taking a note. Yeah, there's a new app grid. Oh, I'm looking at it now. It's big. Yeah, big, big apps. Okay. Go-to apps.

We have a new dynamic app grid. Yeah, sounds great. I don't really think that the watch got anything else. Oh, and, um, everybody who complained about spotlight will be delighted to know that they rebuilt the foundation of search so it's more stable, efficient, and comprehensive

and this is going to be a theme here too. They did that for Apple and Toddgians, right? Well, they needed to rebuild that so they could index it better. So the classic Apple move and anybody who's had a favorite feature that you wish was better and you waited several years

and then it got better, The classic Apple move is somebody at Apple needs it to be better. So spotlight needed to be better. They had to throw away the old spotlight because they needed the spotlight index is the semantic index. It is the source of what is going to drive

all the personalization in Apple intelligence and Siri. So you've got to have it. Or the other example that Federico and Stephen Robles and I were laughing about at that AI event while we were waiting for it to get started is shortcuts. There are all these improvements in shortcuts,

like if-else statements, and what's the other one? Oh, data storage, which didn't exist for years, and there were third-party apps that tried to do it and all of that. But it's very obvious why those things got to add to shortcuts.

It's because if you're going to build a feature that lets you build shortcuts by typing a text prompt, those people who were building that are like, wait, we can't store data. Wait, there's no if-else statement. and then the problem gets solved because somebody at Apple needs it and Spotlight will benefit from the fact

that Siri needs to use Spotlight and Spotlight wasn't good enough so now it's going to be better which I mean that's great it's just very funny to see it that that's in the end if there's an A tier feature that needs your B tier feature to work better

the B tier feature finally gets the love Yeah do you have anything else you want to touch on the platform in the platform improvements before we move on to the main event because I'm conscious of more and we'll want to spend the majority of our time talking about Apple intelligence. No, let's go to Apple Intelligence.

Okay. This episode of Upgrade is brought to you by Squarespace, the all-in-one website platform that is designed to help you stand out and succeed online. Whether you're just getting started or scaling a business, Squarespace gives you everything that you need to claim your domain, showcase your offerings of a professional website, grow your brand, and get paid all in one place. I've been using Squarespace for 20 years, all than that even,

for the projects that I want to put on. Because when I have a great idea and I want the world to see it, I want to go from that idea to having online as fast as possible and to look as professional as possible. And that's what Squarespace allows me to do time and time again. Their template tools are fantastic. I love how customizable things are. They give me these incredible starting points that I couldn't customize exactly the way that I want to get across what I'm looking for from this project. But Squarespace has so many incredible features. You can get discovered fast of integrated SEO tools because why big?

build this beautiful professional website when no one can find it. That's why every Squarespace website is optimized to be indexed of meta descriptions and auto-generated site maps and more so more people will find your site through search engine results. Websites are so complicated now and they come with so many potential things that you could have. Squarespace gives you the tools so you can offer these types of businesses. For example, you can offer services and get paid through Squarespace. From consultations to events and experiences, you can showcase. You can showcase your offerings of a customizable website that are designed to attract clients

and grow your business. Think built an appointment scheduling, email marketing tools and more. Plus, keep everything cohesive with on-brand invoices and get paid easily with online payments. Go to Squarespace.com slash upgrade for a free trial and when you're ready to launch, use the offer code upgrade to save

10% of your first purchase of a website or domain. That is Squarespace.com slash upgrade, the code upgrade for 10% of your first purchase and to show you support for the show. A thanks to Squarespace for their support of this show and all of Relay. So, they did it again. Here we are.

Apple Intelligence. Yeah. They started. Okay, so, as I said, we have had a, I don't know if you call it a briefing. They called it a tech talk. It was a conversation had in a room. It was, again, what is the keynote for? The keynote was not for an hour of technical detail from Apple's AI leads.

It was on the record, right? Like it was all go ahead and use this, but not in the keynote. And so that message will be mediated through journalists. But it was added technical detail and answering frequently asked questions that they didn't want to add. So all of our questions about the Apple Google relationship, for example, they gave a lot of clear answers about how Apple and Google are working together on these features and how they manifest and where they come from and where they're going and all of that. That was in there, along with some live demos, which. You know, we were, you were sitting right next to John Gruber. And we should talk in general.

I know we're just entering this topic. But they demoed at our thing. They demoed live a couple of Siri interactions. With an iPhone. Someone was doing it on the stage. On the stage with an iPhone. Which, I mean, that was one of John Gruber's big questions from two years ago, is did we ever actually see that running?

And I saw somebody today who referred to the demo of Siri and the keynote as slow Siri. It's like, no, it's not slow Siri. because all AI is that slow. You're used to having sped up, canned, simulated AI demos. It just feels weird to be unedited in a polished video, right? But they knew because of the criticism that it was fake, that they were going to do the demo live, and the split screen is live.

And if you watched one person's hand move in one, it moved the same way in the other. John Gruber was sitting next to me, and he was like, yeah, it's a match cut of, the of the two things side by side. In that presentation, the presenter spelled something correctly in reference that Siri would catch it.

I don't think that that was necessarily intentional. No, it was all basically as live as a pre-camp thing could be on purpose. Now, could they have had it fail and then done it again? Sure. I'm not sure they did. Sure, but what they're trying to say is, this is, we're not faking it, this happened. Yes.

And as a result, the demo, they had little music under it and the demo goes on longer than you want and you're watching it spin. And I will say at our thing that was live, all of the prompts felt like they were faster. I don't know if that's because they were

or because when you're watching something in the context of an Apple product video, it's surprising to make you wait when there's nothing happening. There's a context there. But the whole point of this was to demonstrate that this stuff worked because of what happened in 2024. And, you know,

We're both doing various briefings and things over the next couple of days. I absolutely expect that people are going to be seeing these things in action in close quarters. I have no doubt that we're going to have a serious experience. If they don't do this, if they don't do these kinds of things, every conversation, every podcast, every article, every video is, they showed us this, but can we believe it? Can we trust them? So it seems like they have taken and they're taking every step possible to make. You believe it as much as they can. Because, so this is the thing we don't, I don't understand right now.

The betas are available right now. But Siri AI is coming later this year. So I don't know if it's in the betas or not. I don't know. And like if it's not. So, okay, I just, I open Discord. Zoe says that it is alive in the betas. So I don't, maybe they're going to keep it running through the beta and then it will come out later on in the year.

Well, because they may want to keep testing it. in beta form, even after they ship the point-o, right? They could keep it only turned on. They might keep it only turned on the beta releases. Because the thing is like now, you know, they talk about essentially right now, Siri and Apple Intelligence, they're becoming one and the same. Yeah. Right?

Siri is now called Siri AI. Apple Intelligence, it seems like basically all of the Apple Intelligence features that they mentioned reference Siri in some way. So it kind of feels like they're the same now. Pretty close. I think there are some things that are not Siri, but Siri is the, it's like the medium in which Apple Intelligence infuses their platforms. If it's not like photos, it feels like it really, you know, or like image playgrounds. It's like distinct things. Because even where like the writing tools, it's like write with Siri. It's like all of this kind of stuff. So that they're kind of bringing them together. And in the conversation that we went to, Mike Rockwell said that they, he said a year ago.

ago, we had a kind of a version of this working, like these kinds of features, but it wasn't right. Yeah. So he said, we went back to the beginning and completely rebuilt Siri from the ground up on top of the new models that they've been making. He said, we literally tore Siri to the ground. Yes. That's a quote. We literally tore it to the ground. Which is in line with some of the conversation when he got the role, which is that he was very frustrated about it. Indeed. And so I'm sure he, you know, it's like, I'm done with this for starting over. He got the big reckon ball out there and just took it to the ground. Which I think at this point in series life is something is absolutely needed. It's, it had to happen. That's why it didn't happen for so long. Yes.

It's because they were, they were kind of hesitant to do it. And it failed so spectacularly that they had to do that. I like, the AI section begins with what I wanted, which was a little bit of a declaration of principles. And it's not that different from two years ago. It's don't do AI for the sake of AI. Our mission is to turn the potential of technology into helpful products for people, basically, is what Federiigi said. I'm paraphrasing there. But that's Apple in a nutshell, right? And so they wanted to be catering around you and your needs and your personal context. And so that's the statement of purpose. And then, you know, they introduced their bold new architecture, which is very much like something you'd hear in a Star Trek intro. Sure. These are the voyages of Siri. It's

bold new architecture, to connect apps, to, I don't even know what I'm saying there. So let's talk about the architecture. There are multiple models. Yes. And this is really interesting. So there are multiple models. They are Apple Foundation Models Version 3. And the models are a partnership, they say, between Apple and Google.

Yes. Including models that get trained a certain way, some of them, and then are trained further on Apple Frontier models. There's a lot going on there. Foundation models. No, the Google Frontier models are used to do some training later on. So this is the part that's Apple and Google collab. But what Federici took great pains to point out is it's not using any part of the Gemini infrastructure.

The models run, so there are two models that run on the device, right? there is a, they say, very much improved model. AFM Core Advanced is called. That is the on-device model. And it went from a $3 billion parameter model to a $20 billion parameter model. That is the on-device model. But they're able to call just certain parts of the model.

This is like a new architecture that they've been working on. So they only call what they need. It's a multimodal model, so it can take different types of media. that is what it is by design and that is an on-device model and it's called AFM Core Advanced

I took a picture of a slide that we were shown Yeah there is And and important to note There are actually two on-device models Yes Because if you're using one of I guess it's an iPhone I think it's iPhone Air

and iPhone 17 Pro If you're using an M4 or better iPad and an M3 or better Mac, there's a better on-device model that you get. That includes right now, there will probably be more features that use this model. It's though doing the better speech synthesis.

I wonder if that's AFM-Corp-d-vance. I think that's Core-D-V-V-V-V-Core. But it's a better model, but it's obviously not as good as the new one. Not as much better as CoreVVans. But they have these whole things of like, you know, they have like what they called the System Orchestrator, which works out

and writes a prompt based on what you're giving it. System orchestrator is the thing to decide. am I going to the cloud with this or am I doing it on device? And they made a point of saying they do it based on latency. So the orchestrator is like, okay, I could do this on device, but it will be faster to go out to the cloud. They go to the cloud. Because latency is the priority there.

In the cloud, there are also three models. There is AFM Cloud, AFM Cloud Pro, and ADM Cloud, which is the diffusion model for images. Cloud Pro is where you're going to get, your highest tasked stuff. And again, the idea here is they only send that stuff there if they really need to crunch on something. A lot of stuff just runs to the regular cloud. So AFM cloud is, this is my understanding, AFM cloud runs on Apple's Apple built, Apple Silicon operated, Apple Data Center, Private Cloud Compute. It's what we already know of as Private Cloud

Compute. But Private Cloud Compute got redefined today. Yes. So the new definition of private cloud compute is not is not also whatever Google does for privacy that's actually not quite what it is and it's also not Apple's servers that's not the definition

anymore which it had been Apple servers in Apple's data centers it's not so what AFM Cloud Pro is is they are using hardware in Google data centers it is Google's

hardware, but Apple controls the servers and signs the server ID so that unless Apple is the last one to touch it, your device won't connect to it. And it's got Nvidia with Nvidia privacy tech. And Intel as well. And Intel. And they have multiple paths. But basically what they've done is they've orchestrated. private cloud compute

in Google's data centers but what it isn't is sharing space with Google's privacy thing. It is Apple privacy stuff running on servers in Google's data centers. It feels to me that private cloud compute now is more of like a set of

conditions. It's a brand name. And that some of what Google's doing and some of what NVIDIA's doing and some of what Intel's doing they take care of say two thirds of the requirements and then Apple's like and then we add the other third in. Yeah.

That gets them to where they need it to be. So they took great pains to describe this, and I feel very satisfied by this, the way that they've described it all. Like, it makes perfect sense to me, the way that they're doing it, and that the fact that they are not compromising, even though they have had to get help. Yeah. And the fact that they want to, they know they have. have to use Google's data center for these models because they need the power. And they need the

invidia technology. And they do because they need the power. They don't have it on anything else. They need it. But that's not enough, right? Like they added this other layer of it. So this is, I mean, the way they always framed it is, is it's a Google Apple collaboration that they're very happy with. And that they, I mean, clearly the answer also is that Apple, that Apple's models are much better now that they're Google. models that Apple is is but it's also not a white label of Gemini it's just not no it's Google's model it's not Gemini tuned by Apple and Gemini is a product it's the model that helps Google build Gemini yes exactly because they're not the same they're not the same so like Apple built its technology and then they said they refined it using Google's frontier model which is

like yeah it's out there big boy right but that's not Gemini and when they talked about this multimodal model on devices, say that 10 times fast. You did it. They said the 10 billion parameter model, and that's the moment that Federico, next to me, kind of gasped. He's like, ooh.

Yeah. But it's multimodal. The 20 billion. 20 billion. 20 billion. Yeah. One to four billion used, depending on the request.

Because that's how they focus it. And they kept talking about sparse. I think, Luke, is the name of the architecture. Sparse models. Which Mike Rockler said, technology invented at Apple, we wrote a white paper on it. I was like, okay, okay, I get it. Yeah, we get it. You did something cool. But yeah, and then also there's a developer story, which I'm sure is being spoken about at great length in the state of the union. But that

I'm actually really excited about this. So developers, they have access to give their information to the spotlight index, so their content can be indexed. And the app actions, which I guess is app intense. It's app intense. They talked about the app toolbox as well, which feels like They're just coming up with different names. New phraseology for Appintense, essentially. But, like, you know, they showed the demo that we got on stage, it was like a conversation about an image. Where is this image?

Make me a trip plan to go to this place. And they said, like, it wrote to Apple notes, but it doesn't have to. And what I'm really interested in is, how do I get it to write to a third part? That was going to be one of my questions, is if you know I use, because Craig said, it could be bare notes.

It's like, great. If you know that I use bear, does it know or do I have to tell it? Because they also talk about like, you know, like saying writing tools, something I found really interesting with the writing tools and suggestions, it says it's tailored to the person. So like if I talk to you in a different way that I might talk to someone else, it should give different suggestions based on that. So it's like similar, if I'm saying add a reminder, it should never put that in reminders.

It should always put that in to-doist if the system really not. knows about me because I don't use reminders. I use to doist. And like is, you know, are there going to be preferences you set? Do you tell it? Like what I don't want to happen is like what Siri used to do where you'd be like make a task in to do. You know, you had to spend it. Because it feels like from the things that they've shown is that you don't need to do that kind of stuff of Apple Intelligence now that it should have the context about you. It should know what you're looking at on screen even and that it should be more free flowing. more conversational with the system.

So I'm excited to see what that ends up being. Right, because it's less functional if you can never go beyond stock Apple. Yes. But that, I mean, and that's why Craig Federigi said, specifically, that if you wanted to go to bear, you can do that too. Something I was worried about

was that this kind of reset point would have reset all of that. And so, like, it would have been another year for a third party story. Right. I'm very happy that they've not done that. They've been evangelizing App Intents for so long now. I know.

But I was just, what I was wondering, I mean, we spoke about it in a show, I spoke about Unconnected. It's like, do they just abandon it and go of MCP? Like,

you know, we had no idea what they were going to do. Like, app intents made sense two years ago. Right. Does it now? But does it now? I mean,

the jury's still out on that. Right. Of like whether app intents are the right way to do this. Because it's still going to require third part is, to integrate in the way that Apple needs to get the stuff that you want in the way that we want it.

So it's unknown right now how that's going to go. So I'm about I'm intrigued. I mean, I would say from me kind of like overall, I'm actually really keen on what they've shown today. Like the Apple intelligence features, I think the conversation stuff with Siri seems very good to me. It honestly feels like I will stop using a general chatbot.

Like I can't, if it works, the way it's shown today, and with the promise of it, I don't know why I would use chat GPT anymore. For what you use it for. Yeah, which is, you know, like asking general questions. Like, what do you think of, you know, like more common Google searches.

One of the things that Craig Federigi brought up at the tech talk was, you know, last year you guys said you weren't going to just stick something in the chatbot app. Yeah. Yeah, yeah. Now you have. Now you have. And I think the way that, obviously, you know, self-serving, it's marketing, it's all those things.

But I think it shows you, gives you an idea of what they're thinking here, which is they view Siri as a feature that is available in any context at any time, anywhere in the system. And that you invoke because you want something. whereas a chatbot is, I am going to go to the chatbot and ask this thing. Siri is, I'm looking at an app, I'm working on a project, I ask the system a question, it gives me an answer,

I continue to work, and then they said the existence of the Siri app is largely because we need a place to store the records of your conversation so you can go back to them and refer to them and continue them. And in the context of what happens, does, of course, that would be an app, because what else would it be? And I think that that shows you

that they're viewing Syria as this more global thing. Yeah. That it's deeply integrated in what you're doing and not just something that gets bolted on where there's an app where you can ask questions. Swiping down from the dynamic island, pressing on the button using the wake word. Like you get all of that. And it, you know, I really loved some of the stuff about the on-screen awareness and what that can do for you. Like you just, the iPhone can see what you can see. And so you don't have to structure your queries in such a specific way. You just say, like, tell me more about this. What is that? You know, it feels, and then they've got the voice stuff, which also feels like it adds to the kind of customization, conversation, warmth kind of feeling. I feel pretty excited about it.

Like, the Siri AI to me seems like, it's kind of what I want. Oh, and it sinks across your devices too, that your history with Siri. How? Eye cloud? Yeah, I guess. I'm intrigued to understand that because it's like none of it's being stored in a server. So how is it doing that? Like if they're saying everything you say, we're not storing.

It's the record, and it's not the data that's transferring. It's the question and the result. And those are in a Siri transcript that syncs by iCloud, I think. think. Yeah. That's just my guess. I'm just... So, but not any of the data for processing, it's just your question and how it was answered, and then you can go find it. It's like, that's living somewhere though, right? ICloud, so...

But you guess it's encrypted? These are these weird, like, edge cases. It's not like, I believe what they're saying, but it's like, but then how does that work? But it's the same as health and photos matching stuff, right? Or messages. And I'm pleased, because that was a concern of mine,

which is like, am I going to end up with this disjointed experience between the operating systems where like I'm asking a question here and then I'm not getting it there. The Mac actually got quite a lot of interesting stuff, right? That like right clicking on an image or a file and you can ask a question. Or a set of files. On a set of files. It's a great, I love that demo of like here's a bunch of specs for a bunch of sheds. Sum these up and tell me which is best. I was like, oh yeah. All on a different format. Yes. But you can throw them at the agent and have it look at it. So it's like, you know, like I could imagine downloading a bunch of bank statements, right? And

like here are these bank statements can you just make me a financial dashboard or whatever and i feel so much more comfortable asking apple to do that than anyone else right like sensitive information like that i i i do trust them in a way that i don't trust other companies and i feel like now they've got that information in a way that i feel comfortable not uploading it to open a i or claude or whatever right Like it's like I have full faith in Apple and I do believe what they're saying when it comes to security because it's the whole proposition. Right. Like if this falls down and I like that they, what's something I really like and they mention it again today, they're like, we make it available. People can just check this.

You can just verify it like the private clerk. Yeah, that works the way that we say it does. They've done it for last two years. That's been available. You know, it's been verified and you're still letting people do that. Yeah. And that makes me feel comfortable, you know? Mm-hmm.

Are there any other features that kind of really jumped out to you? Oh, so many. Okay, I know there's a lot, right? And again, this is without us having been able to try it. So. Let's talk about photos. Okay. Unless you wanted to jump to something.

I wanted to jump to Safari first. Do Safari. So we, in advance, we heard this idea like tab groups would be made via AI. But what they've done is they group them as topics, which are not tab groups. They can be saved into tab groups. But it's trying to identify out of your hundreds of tabs what the different topics are and put them together, which I think is really smart. It's all going to be in the details of how it's implemented. But I really like that idea of like, look, you're not going to organize this, but maybe we could organize it for you a little bit.

And those eight things that you're doing while you're shopping for something get rolled together automatically by the system as a tab topic area. It made me laugh because the tabs say things like coffee seven, which is you had seven tabs open about coffee. Or maybe coffee at seven, you know? Maybe. Maybe. So I think that's really interesting. Describe an extension.

Yeah, vibe coding Safari extensions, basically, right? Yeah. I mean, that was interesting to me because, okay, so here is like an Apple vibe coding environment. Only for Safari extensions? Well, I assume there's a bunch of stuff in Xcode that we don't know about. But like as a user-facing feature? No, it's extensions and shortcuts. Shortcuts, yes.

Right? Yeah, yeah, yeah, yeah, good call. And the idea here, I think they're viewing it also as being you're in Safari and you're like, I would like to make it so that this happens and that it could build something for you fairly quickly. That feels way more niche than shortcuts even, to me. Like, I don't know, I can't even think of what I would want to do to a web page that I need a short, that I need to make an extension for me. Just think about it.

You'll come up with something. What kind of? Passwords related... Before we move on from Safari, the fact that you can say, like, let me know when this tab changes. Oh, yeah. And you close it and it will come back.

Yeah. So if you're waiting, you know, like, I made a joke to you. It's like, well, every single person sitting in this audience right now will go develop that apple.com. Let me know when this changes. And so we'll find out

when WWC dates go up, right? Like that is going to be a thing that now everybody just has one of those. But I thought that was fascinating as an idea that you could be waiting for an in-stock drop to appear or maybe a sale to go on a website or whatever.

That's going to be interesting. Sometimes they do these things and I feel like it kind of breaks the dynamic of the web. This is one of those things I feel like it's going to break the way that some people use the web, but I think it's fantastic. Passwords is going to

agentically change your passwords for you. Yeah, so it seems like you would just say go change all of these and then a bunch of servers are going to spin up somewhere and it's going to start clicking around on stuff? I'm unclear, or is it just in the background going to the change password page for those sites

and clicking and filling in a new thing? Yeah, but like it's happening... I don't reckon it's happening on your device. You reckon it's happening on your device? Could be. I guess it could be, right? Yeah, sure. Let's talk about photos.

Okay. Yeah, we can cut back to all of stuff, No, that's fine. Let's talk about photos. Because there's a few things. It's photos and it's image generation. Yeah, this is, you know, we talk about how Apple is really good at taking pieces of their technology and connecting them together in different ways and reusing them. And the photo stuff is fascinating to me. So they may clean up better, of course. Like, of course they could.

There are way better ways to clean up photos. There were two years ago. And there are now. That technology is behind. Absolutely. So, okay. Extend. Again, you can do it in Photoshop. You can do it in all sorts of places.

It's a logical feature that has some real value if you need to just make it a little bit. I do this all the time on my website where, you know, I'm doing a Sixth Color Story and I need a photo that's a little wider. Just a little wider. It doesn't need to be tall. It's too tall. It needs to be a little wider. a little more of the background showing. This solves that problem, right?

Or whatever. Or you're extending the height to get it to be in your iPhone wallpaper. You're posting an Instagram carousel and you want your images to be square, but you have a portrait image that you really like. So you have to crop it awkwardly. Well, I guess now you could just make the image bigger so you could get a better crop out of it or whatever,

stuff like that. That seems really interesting. Right. And this is frame. Also, this is an image model. This is a cloud ball. It's happening in the cloud. Yeah, it's going off.

All the image models are happening in that cloud diffusion model. So, reframe. The reason I brought up Apple reusing its technology is reframe is basically reused technology in a very clever way. Because they built this thing for Vision Pro. You can also do it on other devices where, remember, you know, remember portrait mode. Remember the idea that you could capture a spatial image. Yes. And they threw it away.

because they realized that using an AI model to analyze a photo, any photo ever taken, does a better job of figuring out what the spatial characteristics of it are. And then they brought it to the iPhone last year, right? Where you can do it and you move your phone around. And they did an advanced version of it now that you can do with any photo on VisionPro as well. So the problem with that, so you can, yeah, you can move it around and reframe it, which is what this reframing feature is doing. But the problem is it's happening in real time

because it's just to look at. And the parallax, which is so the background is moving at a different rate than the foreground, that's what parallax is, means that like if somebody's standing in front of something and you move too far to one side, like a wall, the wall, if the wall is a consistent color, it'll look okay. But if there's something like not consistent back there,

it looks really weird because it doesn't match. And if you go off frame a little bit, it's going to just fuzz it out because it's not there. They try to fake it a little bit, but they don't try too hard because I've got to do it in real time. So this is using that exact same technology to let you position the camera a little bit differently, as if you were getting a spatial look at it. But what happens at the end is, once you say, yeah, that's the angle I like. You press reframe and it sends all that data to that cloud model. And it does a good job of generating the background that's missing.

And they said very specifically only that. They don't touch anything else. But they're essentially extending the background and extending the stuff that was uncovered by parallax, which, you know, Joe Rosenstile wrote something for me at some point about this, about how you make 3D movies. And one of the challenges with making a 3D movie is the parallax, because if you only shot it in 2D and then the second eye is over to the right, you've got to paint out,

you got to paint in the background because there's new background there and that's hard. But it's better if you press the button and let it render it than it is if you're just doing it in real time where it looks super fake. So they're just, this is literally just off the shelf Apple technology

that's been around other than that final rendering step, which is in the new model. But they've done it in a clever way where you can say, if I wish this was just over a little and down a little, and then render, and then the photo changes to look like that because they've made this complex

machine learning driven basically texture model, distance model with all the spatial information that it's gleaned from the shot and then it can rebuild that shot. And they said it'll work on anything. So like you could take a slide that was shot in 1964 of your grandparent

and you could do this with it because it's all just using ML data. I have incredibly complicated feelings about this feature. Okay, tell me. I think it's very cool. and I know I'm going to save images, right? Like there will be images that I've taken that I love that will be made better by using this feature.

But there's just something about it wasn't the photo. The image that you end up with was not the photo you took. Yeah. And the things that are being generated to around the edges, they weren't there.

It's not real. It's something. that might have been there, but isn't, we don't know. In someone's imagination. I was talking to somebody about... Because, like, let's imagine, because the demo was in someone's front yard, right?

Like that look or that look, you know, it was these kids, it was a day of school. If he moves it a little bit and it paints something in, and there's a lamp post there usually, but it's not there. Yeah. You're not look... I can imagine looking at it. I mean, like, oh, that doesn't look, right?

Like, will your brain except what it's looking at? The example I got the other day was, this is an old photo of my grandparents, and it's the only one we got, or my grandfather and his brother. It's the only one we've gotten. It's not very good.

And I ran it through an AI engine, and now it looks good. It's like, okay, but were those features there on their faces, or was that an AI approximation of what their features could be like? And are you now representing that? I got one of these.

My mom's family is doing a reunion next year in Pennsylvania, and they sent a photo out, a color photo out with the invitation, and it's of my mom's grandparents. So that doesn't exist. That color photo does not exist. So somebody used AI colorizing, and the quality of the photo suggests very strongly that they also did a massive amount of AI cleanup and refinement.

And the question is, depending on the quality of the original photo, it could look pretty much exactly like them, or it could not. And I had that thought where I, that moment I thought, who's to say how much of this photo is real or not? So I think it's worth pausing about it while also kind of accepting that for personal photos

and you're your own art director, I can see the benefit of it. But also I think about the future and I think about like, People saying, oh, this is the photo I really like. And then you go back and you realize this photo is synthetic. And it didn't actually happen. So let's go all the way into this hellscape.

Let's talk about image playgrounds. I was going to make that segue if you didn't. Mike, I said, while you're negative about things and having feelings about things, I can't believe it. Let's talk about image playground. I can't believe it. I can't believe we're going further. Photorealistic now.

The first time this came around, it was like, it doesn't matter. you're making photos of your friends and family, they're just cartoons. And it's like, now we're going to make it look like they're holding a cake. Yeah, a photo realistic case. As a detective, I don't like it. I don't like it.

It looked pretty good. I don't care. I don't, I, look, I turn to you. And while this was going on, and I think what I said was, it's very impressive. I hate it. What can you say?

I don't, there's nothing that I can say that I haven't already said before. but I am surprised that we're still doing this. I'm just surprised we're still doing this. Image playground felt like a thing that you would add into the list of things you had when you didn't have that many things. I think they feel like

image generation is table stakes, which I think is weird because remember there was that real heyday of Dally and stuff like that. Stable of fusion. Yeah, and I feel like... That's gone now. I feel like people don't talk about it. that anymore, but this is still hanging around. It's more now, edit in your photos. That's what it is

more now. But it's like having a tool to generate imagery, I don't like, but I'll accept it. Where it really gets tough from me is the thing that they tout more, which is are people in your photo library? Yeah, right. I'm synthesizing my friend for her birthday party. Where I feel like now, they have an understanding of the features of image playgrounds, which is that you can do that. But now they've plugged that into a much better model. And it's like, oh, we should have left that part behind if you're going to keep going down this road. I don't like it. Yeah, I get it. I think they think it's table stakes and they need to have something there. And this diffusion model gives it to them along with everything that they're doing in photos. And so they're just, and I think they believe my guess is they believe that some people want this. And so they're just going to give it to them. They do. Like, we know people use these tools. Like,

they, I don't understand it. And there's a question of, like, should Apple be helping in every iPhone come with these technologies? They shouldn't have to. Enable. Because there are other things that these tools do that they don't do, right? Like, they're not doing everything,

but they're still continuing to do this. Good news, Mike. Okay. Syria AI is in beta later this year, whatever that means. And thanks to the good people at Brexit, you might be able to use it because it won't be in the EU. Or China. Or China, but probably in the UK, because they don't care. Whatever. We'll find out, but they didn't, they specifically called out the EU and China.

EU and what they said was we don't have any missing features like the European Union has. The way they phrased it was fascinating because it was like in China because bureaucracy, we're working on it. In the EU because, boy, they're trouble but we're working on it. Interesting, right? Right.

So like in the keynote? Yeah. Like call them out? But not for you. EU. Yeah. Like, because they've done stuff before where they just say where it's going to be.

Right? But this was very much like all those scoundrels in the European Union. Yeah. I mean, the headline of the press release because they did a press release, because they did a press release about it. Oh, okay.

Due to DMA, Siri AI delayed in EU for iOS 27. Oh, okay. Oh, okay. I haven't seen the newsroom. They're very angry. Wow, okay. Yeah. So AI, yeah, overall, um, looks like it could be cool.

And once again, I'll see it before I'll believe it. But I guess I get to see it like this week when I install the betas. So that's fun. Yeah, I'm interested. I am very keen to put this on a device. I may sacrifice my iPad to it. I did say I wanted to wait until I got home, but who knows, maybe I'll get a bit crazy with it.

We'll find out. Don't do it. Someone's going to have done it, and I'll be able to play of it before I leave for home, so maybe that will be enough from me. And we'll see what it's going to have. There was a couple of bits that I just wanted to just mention,

like that I think are interesting. The grammar checking stuff, I think, I'm keen to see how that works. I want to see the UI everywhere. Proof reading as you type. When you call a business, and it will just check things in your email and text messages and put them on screen for you that you might need.

Oh, man. Yeah. Stuff like this. That's the stuff. They really feel, to me now, feels like, oh, actually, I think, I think they might have done it. Like, this feels like these are actual examples, lots of examples of personal context. And what people want. Yes.

Yes. Because it's doing. Not making my friend look like a cake detective. Who doesn't want that, you know? I just. And also I like suggestions in messages, right, for like different, like little UI that pops up so you can add stuff to messages because it's checking what's in your message for you. It's seeing us on screen. Yeah, that's little bits and bulbs like that. There's a context with that and then also for the writing tools, which again, it's generative writing. But again, some people probably need it and wanted. But that the messages would be. Natural language and calendar. There was already sort of natural language and calendar, but I think not at this level. Not at this level. I'm excited about that. I'm just, you may be. until I'm currently just looking at Apple's website and just saying things to you that I'm seeing.

I'm just looking at my little notebook. I also made notes and left them in my bag. Okay. And I didn't want to reach around and get them because I would have made a lot of noise. So now I'm just looking at the website. Oh, Siri Mode and Camera, not sure. Yeah.

So my, I have a, my, I made a prediction on connected that I'll share here. The placement of the Siri Moding camera will be changed before the end of the beta cycle. Okay. Because I think they've, it looks like, right now, it's, you know, you've got like the little wheel. It's the first one over from photo.

So before any of the photo things, portrait mode, I think it's too prominent. I think it should be a button that you press, not in the little mode selector. That's what I think, and I reckon it will get changed. Put it into Siri mode. But this is interesting, right? Like, it's, it is, it kind of feels like on-screen awareness and visual intelligence to like two sides of the same coin a little bit, right? It's like, it's just like, are you looking through your camera at this thing or is this thing already on your screen? Yeah. I just don't know if it needs to be in the camera app.

I just don't know. I don't know. I don't know. Why you open the Siri app and have a camera button in the Siri app? I don't know. Where are these extensions? Where are these A-I extensions we were promised? I don't know about that either. This episode of Upgrade is brought to you by Factor.

It's been a long day and then hunger strikes. There might be something healthy in your fridge, but you could be making, but you want a good meal. and you want it fast. Eating healthily isn't always a willpower problem, it's the setup problem, which is exactly what Factor helps with. With Factor, you can hit your nutrition goals this season about all the planning, grocery runs or cooking. Factor has meals built around your goals, whether that's weight loss, overall nutrition, more protein or GLP1 support. And for strength and workout recovery, check out Factor's Muscle Pro Collection. Factor meals are fresh, never frozen.

There are over 100 rotating weekly meal options, including globally inspired flavours like Mediterranean and Asian, so there's always something new to look forward to. Already in just two minutes, Factor shops, preps, cooks and deliver straight to your door, so you have more time for everything you love this spring. You've heard Jason mention before how convenient he finds Factor so much so that he buys Factor meals for his family members, and he's always talking about the quality of the chicken, for example, Now, great, he thinks the chicken meals taste because the quality is so good. I will say from my own experience as well, you know, having a young kid, especially when not too long after, you know, Sophia came into our life, having options like this, you should throw them in the microwave and get a good, hearty, healthy meal.

It is essential. Like, for people in your life that have young children or for people in your life that are just about to have children, Factor is an incredible way to give them something that they're really going to need. Head to FactorMeals.com slash Upgrade 50 off and use the code Upgrade 50 off to get 50% off and free daily greens per box with new subscription only while supplies last until September 27th, 2026. See website for details that is factamiles.com slash upgrade 50 off and the code upgrade 50 off. A thanks to Factor for the support of this show and all of RELA. So there was one other segment.

Oh, yes. I think as parents, we should talk about it. Yeah. So they called it trust and safety. But it is overall a significant beefing up of parental controls and screen time and all that kind of stuff. And I say as parents because like four or five different presenters used the phrase as parents. As parents or as a parent. Or my kids at home.

Okay, so I thought that was an important thing to do. So here's my thing. We can get into some of the details about this. Yeah. I think a lot of this is going to be seen people leave. It's a lot of smart stuff, right? This is the lightest, this is like a light, fluffy portion of the keynote, right? Like, it was the least consequential and the lowest density.

So I would say it made more of an impact on me than I think you. Sure, because I'm looking down the barrel of this life. And I already been there done that. So like, to me, I was like, I am so happy that they're putting a new foundation in for this. Right, because... Because screen time is terrible. As a user of screen time, it's broken.

It's totally broken. So doing things like having this assistant that can help you set things up and making recommendations, approving websites, putting more checkers for nudity and gore and all this kind of stuff. All of this is interesting, but I want to talk a little bit

about the bigger picture of this. What I find really interesting and it's why I mentioned about the as parents, as parents, and why I thought that was important, is Apple is making a stand. They're making a stand. So this was going to be my question, which is, why now and why at this level of prominence

make this stand? Because this is, I didn't feel like this was the moment where we were like, oh boy, is Apple going to meet the moment in terms of trust and safety for kids? it's been because they've been like you said screen time's been broken for a while now but this they decided this is the moment no i think this is the moment okay social media with young kids is a big problem right right governments around the world want age restrictions

right all these tools meet all of that right so they have to do it because they need to do it at some point they have to do some of this kind of stuff right and they should should stand out in in front. They're the platform vendor. They can't get away with this. So they may as well set an example and try and build tools that people should connect with. And it's up to them to do it. And then also as parents, it is very helpful for them to feel like Apple's taking care of it for them and that they're going to help guide them through. The thing that I find interesting is, I mean, they say they're working with, you know, pediatric society. There's a big bibliography. I wonder about like, okay, are they going to work with different bodies in different

countries? So I think one of the things that we should flag here is, and this is a policy thing for them, is they are citing experts, not governments, not regulations, but experts, which I think says something, right? They're like, we're going to implement this because this is what the science says. This is what experts say. But then also to say, but they're also making recommendations, you don't have to take their recommendations. Right. Every kid is different. Right. Whereas, right, exactly, that the parents know best, different kids are different. What they're not saying is, and in Australia, you have to do this. But it's, it's just interesting to me because no matter what they do or what they say, they are still making

top level decisions about who they are going to listen to and what features they're going to implement. And so I just find it all very interesting where it's like we're deciding to do this. We're deciding to do it in this way and we're going to tell you as the parent of a child what we think they should do.

And I find that interesting and also so messy. Well, I think the approach is basically there's got to be a default. Yeah. And so you set the default. First off, you say it's not us. It's not Apple. Apple's not telling you how to...

It's like we listened to a bunch. This is why they had Simbo Decai do it. I thought that was interesting. Right? When they went to, I was like, health? Dr. Desai, but she's like, we have looked at the research and the literature and all of that. And I think that's a part of why they had her do it. Yeah.

And it means... She was the right person for it. But at last I was surprised. We're not telling you how to raise your kids. Yeah. What we'll set is defaults that you can change. And the defaults will be based on the best research. And they even said, and as research changes, we can change these defaults.

And then step two is, but you know your kid. And you have the ultimate control over what those restrictions are in your kid. So Apple, I think they have to do it that way. Because there have to be defaults. And you have to, it's a service, right? parents. I don't know how to set it. And it's like, well, Apple says one hour. Okay. Let's start there. And if you're like, no, no, no, no, I laugh because they had the thing of like, also, there's this button with an infinity symbol on it. If you just want to let your kid do what the hell they want to do, which, you know, we're at the airport and our flight got delayed. And my kid wants to watch the videos. And that's that moment where you're like, just let them do it. Because I need to live. Because parents know what that's like. Yeah, I certainly laughed at that one. I already know what that's like. Yeah. And it's only going to get better.

So, yeah, I think it was a very carefully constructed idea. They've asked to buy, right? That's been a thing for a long time as part of the family. But now they have like approved new websites. Yeah. They have approved new contacts. Yeah. It seems like they beefed some of this stuff up.

It looks way better, which again is a low bar because the old thing was really not very good. Because I was saying, too, like, what I'm interested to find out is how does this improve screen time from me as an adult as well? Because you'd assume it's gotten better. The difference is screen time is a feature that evolved from a period where people, were saying, hey... Digital well-being. Yeah, digital wellness. Hey, we're using our phones too much.

This is evolving from an era in which there's a lot of literature, and there's governments saying kids shouldn't be on social media. Because what I also like that they said, I don't remember the ages, but they were like, it is recommended that kids under this age do not have a personal device.

13. And I was like... Oh, no, for social media, it's 13. Yeah, but they was like, they said like personal devices. Not even a personal device. And I was like, I'm happy you said that. Because you sell the personal devices. You sell the iPads to make iPad kids.

Like, that's your thing. And it's good because people get used to iPads and then they want iPhone. So it's like, you know, there is a way where you could not say that. But I respected that. I said that. I think this stuff is, like I said, I really do think it is very dependent on who you are. There is a time where I would not have cared about this so much. And I would have just said, yes, but what is this going to do for me?

It's a screen time user. And I do so want that. But I... Now you have a child. I'm, you know, I'm years away from even needing to consider this, like many years away. Yeah. Not that many. It's enough, right? But I'm, so I'm just happy that they've, they have decided that this is an important part of the operating system.

You know, they set up a subdomain for this, right? Where, like, they have a bunch of resources available. And also just for the world, I think that this is a thing that they need to do, especially around social media about trying to help give parents and kids tools to help them use these apps more responsibly. I think it's really, really important. And I'm happy that they're doing it, even though it is odd to have it in a in a keynote like this. A keynote that was all right. But I guess the keynote was so short, or what are they going to put in there. It'd be done in 45 minutes otherwise. Right. But I like it. I like that I spent time on it. I think it's really interesting.

I'm very intrigued to see what the bigger response is to this. What are the articles? is going to be about for this? Like, what are people going to say? Is this going to be considered a good thing, a bad thing? Gone too far, not far enough. It's going to be interesting. And as you say, I do like, you know, for some parents, there might be the thing of like,

sorry, kid, Apple says, no, I don't know what to. I don't know what to tell you. Apple says one hour a day. You'd have to take it up with John Turner. I don't know. So I thought it was cool. Was there anything else you wanted to touch on? I mean, again, this is,

we're going to be unpacking this in the details over the next few weeks, I feel like. Yeah. So we're going to, you know, we're going to go. We're going to find out. We'll have more to say next week because me and you have, like, things in our calendar that we're doing over the next couple of days, experiences that we're going to have that we can talk about.

We'll learn a lot more. We'll know what has been rolled out to developers. There'll be all the sessions. A lot of stuff to cover. On Thursday this week, you can look out for the second episode in our designing California preview on the road. to the apple two. Yep.

I don't remember what this episode was about because I just finished edit in the last one. Is this, they go to Atlantic City. They go to Atlantic City.

Not the good Atlantic City. PC 76. PC76. This is a great episode. USA. USA. Yes, indeed. Don't spoil it.

This is a great episode. This story is amazing. Jason did a lot of great research to find out this story. So this is the Steve's going to PC 76. On a plane, on a red eye to Philadelphia

to go to PC76 in Atlantic City. To show off the Apple one. They have the Apple one. And secretly have a prototype of the Apple II up in the hotel room. And see what their competition is going to be like.

And their competition sees what they are and laughs. This is a great episode. If you want to back our Kickstarter campaign and help make designed in California

be a better and better show because that's what we're doing. We're raising a lot of money, which is amazing. But we're taking that money and turning it in to a better product. We're going to have

a full video version of the show, which is going to be as edited as the audio version. We're going to put imagery and stuff in. So if you know, you want to watch a show by video, you'll have that too. You'll be able to go watch it. Another option. It's another option available to you. But we're taking the incredible support that we're getting and we're going to use it in different ways. We're talking about that over the coming weeks. Go todesigned.com. And you can do that. You can send us your feedback, follow-up and questions by going to upgradefeedback.com. I'd like to thank our members. This will We'll support us with Upgrade Plus.

This week, we're going to decide who the California a bear trophy winner is in Upgrade Plus. You can find us on YouTube. There'll be a lovely YouTube version of this show, which... It may be a little later than usual, but yes, it would be beautiful because Apple has supplied us with just a scene at MLS games and Major League Baseball games in a ray... Shot on iPhones.

Yeah, Final Cut camera, I think. It's great. Shot on iPhone. That's us right now. So I would actually like to give a quick thank you to... There's lots of people here right now. So many people here. And I would like to thank everyone at Apple Park who helped make this episode a reality.

Because otherwise, I do not know how we would have made an episode today. I want to thank our sponsors again, Keep us Squarespace and Factor. Until next time, Jason Snell. Say goodbye. Goodbye, Mike Early.