PurePerformance - 043 101 Series: Visually Complete and Speed Index

Episode Date: August 28, 2017

Visually Complete and Speed Index have been introduced to better measure real end user performance experience. Klaus Enzenhofer @kenzenhofer ( https://twitter.com/kenzenhofer ) gives us a detailed description of these metrics, how they are getting calculated, and which problem they solve. What we also learn in this 101 is why now we finally have these metrics available not just for synthetic monitoring but also for real user monitoring. This can be attributed to the advances in browser technologies as well as to some smart engineering. In our discussion we also cover other recent advances and use cases in Web Performance Optimization – such as the usage of performance markers.If you want to learn more check out the blogs from Google on Speed Index ( https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/metrics/speed-index ) as well as the blog from Klaus on how Speed Index and Visually Complete made it into RUM offerings ( https://www.dynatrace.com/blog/visually-complete-speed-index-for-real-user-monitoring-rum/ ).

It's time for Pure Performance! Get your stopwatches ready, it another episode of Pure Performance. Today we have another show, don't we Andy Grabner? How are you? We do, Brian Wilson. Why do we talk that strangely? Because I have to do something different every show, sort of, maybe. Yeah, that's true. Keeps it unique. Yes.

What's what's new today? I think it's the first status here while in Boston right now. And we have 31 degrees Celsius outside and it is mid of May. And so 31 Celsius is a lot. So that's nice for people that don't do Celsius. I think it's like 90, close to 90. I know people who do centigrade,

but not Celsius. That's a bad joke. Ah, yeah. Yeah. Yeah. Well, let's let's anyway.

So what do we do today? We have a new episode. Yes. Another one-on-one episode. And actually I have the interviewer sitting next to me. The interviewer is another colleague of mine working in the Innovation Lab out of Lindt, but happened to be in Boston today, Klaus Ensenhofer. Hi.

Hi. How are you? I'm good. We're both, hopefully, after this, we can enjoy a little bit of the sunshine. We brought Klaus on the podcast, Brian, because we wanted to talk about two new things that we hear, that we've heard. Right. But before we go there, Klaus, do you remember our first encounter? To put you on the spot, our first date, if you will.

The first time the two of us met was in New York. Well, Jersey City, technically, yes. but then we went for a beer afterward. We had a beer and it was a good evening. Anyhow, that was a long time ago. So anyway. Five years, something like that.

Yeah, I've been here, no, yeah, probably closer to five because next September I think will be my sixth year here. So yeah, quite a long time. No, yeah, probably closer to five because next September, I think, will be my sixth year here. So, yeah, quite a long time. We're getting old. Okay. We're getting old. Speaking of getting old. We're only getting wiser. We're getting wiser, not just old, wiser. Yeah. Well, this intro is getting old, so. Yeah. Go ahead. So we are going to, we asked Klaus to join us today because part of our one-on-one series, we wanted to talk about two new ways to, I think, measure and evaluate performance for end users on websites, mobile applications.

We talk about visually complete and also about speed index. I love these metrics, by the way. That's why I'm clapping. Cool. Yeah, cool. And so for people that have no clue about these new metrics that have always thought about website performance and page load, Klaus, do you want to enlighten us a little bit? Maybe start with the first, visually complete? Yeah, actually, they are not that new.

Visually complete exists for a while already. But with Dynatrace, we brought it into the real user monitoring space. So visually complete is all about getting into the seat of the user, trying to capture what he sees on his device, whether it's a desktop or mobile device. So how long does it take effectively to render everything that is in this visible part of the screen people are talking about above the fold when they speak about it and lately this has become a more and more important metric because the good old response time that we looked at became kind of invalid depending on new technologies like single page applications where the whole loading behavior is so much different that you had this response time eventually triggering really, really early. And you had basically the user, you left the user with an empty screen.

Not really useful. It tears down a little bit the user, you left the user with an empty screen. Not really useful. It tears down a little bit the user experience. Or the other way around, you had really fast, you were done with the visible area, but then you were loading like a gazillion of tracking pixels and some other stuff that is not relevant to the user at all. And it doesn't change the layout, but it kind of, yeah, defers the triggering of the regular response time. It also sounds like that would also make it very difficult to compare your page performance

versus your competitors or maybe even an older version. If you made some changes that that comparison would be kind of tough because there might be a lot of inconsequential things going on that have no impact. And I think that's kind of where we're going with this visually complete, right? How it. Right. Absolutely. So with visually complete, you now are able to compare yourself with your competitor. I mean, everybody knows how his own website is constructed. If not, then try to find it out. constructed, you never know. And this visually complete, you're kind of independent of how this whole thing gets constructed. You're always getting, this is the screen, this is the time it took till the screen was done. And so I think because you mentioned that it has been around for a while, but we brought it to real user monitoring. That means initially it came out of synthetic monitoring.

Right, absolutely. You're absolutely right. So originally Patrick Meenan was bringing this capability up. And, yeah, the thing was in order to get this visually complete metric with synthetic, they really took screenshots real screenshots and compared those screenshots and once the screenshots were no longer changing then basically the metric triggered yeah and we with with our dynamic resolution brought it now to real user monitoring we took in

JavaScript based approach to capture basically our all the elements in the our Dynatrace solution brought it now to real user monitoring. We took a JavaScript-based approach to capture basically are all the elements in the visible area ready for the user, yes or no. It's a much more lightweight solution, so you're now really getting into the real user seat, less of the synthetic one where you're driving a synthetic machine. It's a good first indicator, but you never know what's going on with your users out there. Think of mobile devices. Right. And I think we need to work on our synchronization here.

Just because you mentioned Pat Minan, and I have just, brian we should post a link on this uh pat minan former guest of the show former guest of the show he wrote a blog about this how web page test is actually so web page test the testing tools synthetic and he has a nice explanation of both actually visually complete and speed index um we come to speed index in a second but you can really see here in his time frame where he shows how much percentage of the page was actually loaded after let's say a second two second three seconds and when is then the page visually complete yeah

so so in terms of visually complete right you know we we've had within dynatrace for a while now we've had a time called perceived render time. And I'm sure other tools out there have their own proprietary sort of above the fold rendering measures. What does the switch to something like visually complete mean? Or why is using visually complete as opposed to our own or anybody else's own homegrown version of that important? So perceived render time is something that tried to measure exactly the same thing, but we did it around the year 2006. If you take a look at how browsers evolved from 2006 to today, the insight that you can get via JavaScript APIs into the browser are just

completely different. And yeah, perceived render time was a good metric. It really worked well, but it's now kind of outdated because we have now more visibility we can get more accurate to what the real use of experience is here and getting more accurate timings here cool um so visually complete and if i rephrase this again what for me the way i understand it if you look at the page and let's say the page takes five seconds to load. Let's say two pages both take five seconds to load. But it could mean that they are loading in a different way, right? You said earlier the HTML document might be downloaded in five milliseconds,

but then until everything else comes in, that's why we really want to make sure that we measure these five seconds and not just take the page load time or the DOM load time, which can also be wrong, as the right measure. So really looking at when does the user really see the page above the fold to be fully rendered. That's absolutely correct. But I wouldn't downplay all the other metrics because they have the use cases right

right so if you are into tracking how often somebody clicks on your ad then it's relevant for you that the ad is there and the tracking script is actually loaded in order to be paid at the end so you will probably then take the good old response time again because that makes sure everything actually got loaded. Or if you are into search engine optimization, there is Google is talking about leveraging now this visually complete metric as the metric for performance. That's something they input. Performance is something I have to extend here a little bit

because search engine optimization is not a big it topic but uh there is something about how google ranks their uh yeah results on the website and those results are actually actually influenced by the performance of the website. And they used to look at the time to first byte, which they are at the moment still. But the next step is let's look at how long does it take for the above-the-fold area to load, which is basically visually complete and that's a metric that's one of the few things that are really communicated by google as influencing the search uh rank yeah and that's why for example time to first byte can be equally important as visually complete really really depends and it sounds like that with with visually complete you mentioned like the full page load time which is going to have a meaning for something like the advertising and

keeping up with those things but what it's really sounding like to me is that visually complete is a lot more tied to that i don't know if we'll ever have the magic answer but it seems a lot more closely related to that. What is it my user sees? You know, I go on a lot of engagements and, and when we talk to, um, any of the business owners, they're like, well, what met, what do we, how do we measure, uh, the user's impression of the page? Right. And using something like unload, as you're mentioning, isn't quite really getting there. It sounds to me that something like visually complete is a lot closer to that magical metric of what the user is experiencing. I guess, as you also said, I think Klaus,

it also works beautifully for single page apps, right? Absolutely. So especially the initial load, first of all, but then also when you, when you interact with the app and you click, let's say on the link, and then something changes, we basically measure how long does it take until the fully UI is rendered again of the change.

Right, the visible part of it. So the thing is, single-page application, as you know, it's a single-page load, so W3C navigation timings, you don't have them. They're available. But you have your triggers. You have your available but uh you have to your triggers you have your click actions that you handle in your javascript and then yeah you go off load something from the server side maybe another image um yeah json object that tells you to do whatever um with with the UI, and then you change the UI based on what is coming back.

And we used to be able to see basically the click plus the first request, but this changing of the DOM and the changing of the whole appearance of the website, this is something that now visually complete can capture. This is where we have extended our capability. Again, we want to always be as close as possible to what the user is at the end experiencing. And this really helped a lot. And can we reveal some of the implementation actually?

Because you said you're using JavaScript. Now we are getting in a critical area because we're getting a patent on it. Oh, okay. Well, then, no, I don't want to... But basically, just to confirm, what we do, we use JavaScript for that? We use JavaScript, and I can say that we are using the Mutation Observer

API. But as I said, there is a patent. That's okay. Then we are... But we're doing it obviously in a lightweight way because overhead is always a question. Cool. Well, not to put you on the spot, Klaus, That's okay. But we're doing it, obviously, in a lightweight way because overhead is always a question. Cool. Well, not to put you on the spot, Klaus, because we don't have to dive into it,

but as somebody once said to me, if you have a patent for it, it'll be public. And it'll be public. So anyway, we'll not go into it right now, though. Cool. So that was visually complete, the first metric. Be more accurate in really measuring how long it really takes until the page is fully rendered. Now, the second metric be more accurate and really measuring how long it really takes until the page is fully rendered now the second metric is speed index what is speed index speed index

basically explains how fast the visible area actually gets rendered each part so there is let's say we have two pages both have have visually complete after eight seconds, which is pretty slow, but that's a zero in eight seconds. And the question is how fast did the user really see something and how fast were the bigger parts of the website ready? And speed index is actually looking at the percentage of the visible area. It looks at graphs, basically, is a lot of the visible area really early on done or is it really late? And this really helps them to understand the loading of the page, so the path to the visual. So basically, if I take your eight-second example,

I enter the URL, and seven seconds, nothing happens. And then, boom, everything is here within a second. So loads in eight seconds, but as a user, I'm frustrated for seven. Alternative example, 90% of the page loads within the first second and then the rest loads within the next seven seconds. From the user perspective, it's a much better way of like, oh, this is much faster, even though fully loaded still same to that. Right. And Andy, to counter that with one more use case, which I think is important here, right? Because your first use case was like nothing was happening. And this kind of goes

back to that documentation that we'll have linked on there for visually complete and speed index. The other use case is, let's say, 25 percent loads in the very beginning. And then the other 75 percent doesn't load until the end. That's going to have a lower that that'll be somewhere in between the nothing and the a bunch of front. Because, yes, you get a quick hit on the user that something's happening, but then it stalls and all the main components don't really come in. So it's really, I don't want to say tricky, but it's critical in getting useful and impactful data up front first, as opposed to just maybe painting of backgrounds and box outlines and then pausing for a long time. So people are often asking me, what is a good value for speed index? And what is a good value for speed index or what is a good value for visually complete,

and I always say those two metrics are really working together. Visually complete and speed index should be quite a part because the earlier on you have speed index, let's say our 80-second example, speed index tells you it's 1.2 seconds this is then a good metric because that means that a lot of the stuff is already early on visible whereas 8 seconds example yen if you have a speed index of 7.5 seconds, this is a bad example. This means you have for a very long time nothing visible, and then you get everything, like you explained it.

So they really work together here. And so speed index, so that means, and obviously the reasons to speed these things up, we would go back to our regular web performance optimization. So why is nothing happening for so long? Maybe we had some blocking JavaScript. We have whatever else. Maybe some blocking third-party components that come in.

Right. And something like that. That would be good indicators. Also optimizing the page again to defer loading of content that is not visible or not necessary for the event. These are the regular things. We fall back to that so so speaking of that point andy then um the question i wanted to ask klaus about this is let's say you have an above the fold page or above the fold section that has like your your twitter and your facebook and instagram and i guess i have to say uh what's

what's the one the kids use uh what's happening or whatever it is. Yeah, whatever. It has all those little bits, all those little beacons on them. Now, let's say you did follow the good performance optimization and say we'll only put those in on load, but it's still above the fold. How does that play into visually complete? It depends how you integrate the whole social media, and there are multiple ways how you can integrate it. But does Visually Complete take into account what happens after onload? Yeah, sure it does.

So it doesn't stop with onload. So if you change something in the UI, then it would still capture that. Okay, so that's where Speed Index, I guess, would be a critical counterpart to that even more so than normal now cause so these are basically the two the two key metrics right so speed index and visually complete now earlier you said something about right certain people in the organization maybe marketing they have a different opinion on what is critical for them, right? So, for instance, loading that particular ad. I believe W3C specifies the markers. Yeah, the markers.

So, can you explain this a little bit? Because this kind of helps actually extend these two metrics with custom metrics, right? Right. You can always extend and measure what is critical to you yeah if you have developers that know exactly what is the critical thing for marketing that needs to be loaded or for your business actually then you can place there some markers using JavaScript we will pick them up and you could leverage those and tell Dynatrace, hey, that's my new key metric.

That's my new key performance metric. Another example that I often come across is DOM Interactive. There is one thing to have everything visible in your area, but let's say you're a booking platform. You try to book your travel and you try to enter a date because, you know, when you travel, you have to enter a date. And what happens if your calendar doesn't show up? You cannot pick from the calendar. So that means this critical JavaScript is not there or is still the website itself is still being blocked because of other things going on being loaded so dominant active can be here a really critical

metric as well because it tells you when can I interact with the website really depends on your website you have to pick your key performance metric but they are all important the one or the other way mm-hmm cool and you said so the way we influenced our own product in diameter is UEM we allow people to pick and choose what the key metric is for them and not only or not only I think in general for the whole application we monitor but actually per user action group so you can decide basically let's take the e-commerce example looking at a product detail page versus getting a search result list you might want to

have there a different threshold on how fast that should be or let's say you want to measure how long it takes till autocomplete shows up when you start typing the search term into the search field you would probably wait for the autocomplete maybe a couple of milliseconds to show up whereas for a complete product detail page with video and everything that you want um you will accept maybe like five seconds even because people know there is more stuff coming still they try to get it as fast as possible yeah and i wanted to even mention the going back to the performance marks that came out andy i think as a result of uh of the webinar we did together right and i had no idea these things existed and uh the question

what came up about these user timing performance marks which is where as you all said you can customize any kind of mark in the code that you want. And I just loved when we looked into it, we found out that we actually do capture those already. So I'm just bringing it up because it made me very happy. And I like to talk about happy things. So Klaus is currently pulling up some data here, which reminds me I have to ask him another question. Yeah, yeah. Thanks for the hint. Which one?

Thanks for the hint. So we know that as much as I love my iPhone, Safari has been known as a browser that did not always support the latest and greatest W3C specifications. How about visually complete and speed index? Is this something that I can only get on Chrome or does it also work on other browsers? No, it works actually on all the new browsers because the thing that we are leveraging, as I said, is the mutation observer API. And that one is available besides the Opera Mini

on every modern browser. Okay, cool. That's good. So that's finally good news for for the apple users out there except for all the opera mini users yeah yeah that's right i'm talking to you so different sorry opera mini is working just so fundamentally different from all the other browsers so you cannot have their visual complete yeah and. And by the way, I want to let the audience know, Klaus has been using a website here that's called caniuse.com.

Yeah, it's an awesome site. You can ask questions. Can I use any browser API? Can I use W3C timings? Whatever. It's really great. It tells you basically which browsers are supporting it, gives you a global coverage.

Basically, what does it mean? Like for the mutation observer API, we have global coverage of 93.54%, for example. This means Opera Mini is the biggest part that is missing and some older versions of ie slash uh yeah yeah yeah nice so can i use.com that's a nice page to look at especially i guess for developers as well so that they understand what they can use in a browser wow um brian did we uh so i think we covered the two metrics, also learned about the Right. And not that we need to go in deep or anything like that, cause that's a whole, you know, expertise field, but you know, just the idea of, you know, getting some of those key elements, uh, maybe put it in line. Um, you know,

I know you can do some of your CSS in line. Uh, there was something about what was it? Base 64 images or some kind of weirdness, uh, whatever that, that, that piece is, um, doing whatever you can to get some of that data up and on your page as soon as possible is really going to help, especially for that visual, uh, the, the, um, speed index part. Um, and the, and the only time, you know, I thought about it a little while too, when, when Klaus was talking about, um, the distance between speed index and visually complete, how if you're visually complete as far, you really want, you know, or a good gap between the two is good, right?

Obviously, if your page loads in one second, it's not going to matter too much. But let's face it, this day and age, what page loads in one second anymore? So these things are really, really important. And I just really want to give my hats off to the team at Dynatrace for getting all this stuff in because it's, at least for me, who spent a lot of time on that front-end side with some for integrations, it's really exciting to see these in here.

So thanks to that. And also we have to thank, obviously, the initial creators of that. So Pat Mead and Steve Souders was also a big proponent and was implementing that. And I would like to thank my wife as well, because she always so thankfully, so thoughtfully, leaves the room when I do a podcast to give us some quiet.

Thanks, Megan. Cool. Wow. That was nice. Straight to the point. I think hopefully people understand visually complete and speed index better now. Klaus, we would love to have you back also for other episodes, hopefully people understand visually complete and speed index better now and klaus you would love to have you back also for other episodes because i know you're doing a lot of stuff around user experience monitoring real user monitoring you just came off a webinar with panera bread yeah but they were they talked about how they use real user monitoring to connect all the different

channels that their customers are using right from tablet to kiosk and everything. Happy to come back whenever you have time. Let me know. I wonder if we'll have time ever. Wink, wink. Any last words from anybody then before we wrap up here? This is going to be under 30 minutes

if we got two minutes to keep it under 30 minutes. It's going to be a record. No we if we we got two minutes to keep it it's going to be it's going to be a record uh no the only thing i want to say we we bring out the um the links on the website klaus how can we find you on twitter what's your twitter handle that's a good one put a link on your site it's uh k-e-n-c-e-N-H-O-F-E-R. Perfect. And yeah, I think, Brian, that's it for me. It's great that we have this one-on-one series and we continue recording more of those. And yeah, that's it from my side.

All right. And any suggestions you have for another one-on-one topic, we'd love to hear it. You can tweet us at Pure underscore DT or email us at pure performance at dynatrace.com. And we'd also love to hear how maybe if you are already using visually complete and speed index, how, how you might be using them. Uh, especially if you have any great stories around them, maybe you can be a guest. Um, Klaus, any last words from you? No, I just really appreciate it. I just asked myself, uh, why did it take us so long to get the first one kicked off?

We've asked you, but I don't know. I'm not sure who to blame, but I think you just want to stretch it now so that we actually cross the 30-minute mark. All right, well, let's wrap it up then. All right. Bye. Go for it, Lakers. Bye.