Embedded - 249: It Depends

Episode Date: June 15, 2018

Claire Rowland (@clurr) joined to discuss creating good user experiences for the Internet of Things. Claire is the lead author of Designing Connected Products: UX for the Consumer Internet of Things. You can find more about her on clairerowland.com, from her talks (including Interusability: UX for Connected Products), her book's website, and her guest appearance on the IoT Podcast (episode 21). Her new report about user experience and the IoT will be on Iotuk.org.uk in June of 2018. Elecia was also on the IoT Podcast: episode 158. It was @SwiftOnSecurity who posted the tweet about experts and their typical response.

Hello and welcome to Embedded. I'm Alicia White alongside Christopher White. Our guest this week is Claire Rowland, and we are excited to speak with her about user experiences in the Internet of Things. Hi, Claire. Thanks for joining us. Hi. Thanks for joining us. Hi. Thanks for having me. Could you tell us about yourself as though you were on a panel at a technical conference? So I'm a consultant. I have my own consultancy.

I specialize in product strategy and user experience strategy for the Internet of Things. That includes connected products and hardware enabled services and i've been in ux for about 20 years now spent the last eight of those specializing in in various kinds of iot with particular particular experience in connected home and energy management applications but i've worked on a bunch of things from industrial power tools to health and beauty devices to well-being type applications. So a broad base. And you have authored a book.

I have. I'm the lead author of Designing Connected Products, UX for the Consumer Internet of Things, published by O'Reilly. Excellent. So now, instead of asking you about all of those things that you are clearly an expert in, we are going to ask you short questions and want short answers about completely random things. Okay. What is on the cover of your O'Reilly book? It's a sea pen. It's a kind of apparent, I asked for a colonial organism um i wanted a siphonophore um you know one of those things like a portuguese man o' war um that looks like one animal but it's actually um lots and lots of them working as a as a colony um but uh they they gave me a c-pen

and they told me that was a colonial organism and as i think you know, you don't really get a say in your animal. No. Sad. She's still sad. Do you have a tip everyone should know? I saw that on your list of questions, and I was hoping you wouldn't ask me that. I can't think of anything off the top of my head. Okay, no worries. It sounds like one of those top tips about how to get rid of paint smells or something i feel pressured to come up with something household i don't know how do you get rid of paint smells apparently if you put a

bowl of salt in the room that that gets rid of it but i've never actually tried it well there you go okay uh what's your favorite type of physical button like arcade buttons or keyboard buttons or the big red emergency stop buttons oh that's an interesting one um big red emergency stop buttons are good there's a there's a there's a technique you can do in in user experience research where if you say it kind of say if you had one button what would it do and it was trying to get people to visualize this one big button that does exactly the thing they want so um one big button nice satisfying kind of mashable thing red's good what's your preferred method of connecting a wearable device to the internet i don't know if i i don't know if i have a uh all of the things i've worked on have been Bluetooth to a phone. It depends what it does.

I guess it depends what it does. I mean, certain things you don't necessarily want to be on the internet. You want them to be locally connected to something else. Other things you might want to have a direct connection. I still have an old Apple Watch that doesn't have a direct connection. And I'm looking forward to getting a new model. That's not a great answer, sorry. No, it's fine.

It depends what it is. I'm a big fan of not putting things directly on the internet unless they have to be. I saw a tweet recently about how if an expert has a short answer in something, they're probably not an expert in that experts just really want to like know all the details and be pedantic i think the actual thing was uh if you're asking expert in this don't say it depends to every every question yeah it does it does it does very much depend i think if you're if you're wearing something that's a very personal thing you don't necessarily want that exposed to the internet

unless there's a great benefit to doing that. Okay, one more lightning round question because clearly we have a lot to talk about. When you imagine a generic user of your IoT device, do you have someone you picture? That's a massive it depends um what is it you know i've worked on i've worked on everything from um health devices to beauty devices to industrial power tools as i said earlier you know lots and lots of energy monitoring so there's um there's a current so it totally depends you know in some and lots of energy monitoring. So there's a current, so it totally

depends, you know, in some cases you're thinking about somebody who drills great big holes in reinforced concrete walls for a living. In others, you're thinking about, you know, someone who's worrying about the tone of their skin or something like that. So you would, I would never have a generic, you know, i would never assume that there was going to be a generic user for for for anything i've worked on um i certainly have some common motivations when i'm working on energy management and there's certainly some some sort of common motivations i would draw on there um but yeah i think it's i think you you have to you have to approach each product and each service as it comes.

All right. Okay. So now to the real meat, which we've already started. I have a love-hate relationship with the IoT consumer products, especially with respect to user experience. Can you tell me how it's supposed to work? I mean, what is a good interaction? What is, what is the best case? I don't, can I, I'm going to be contentious and say that I think virtually none of it works the way it's supposed to work. I agree.

No, well, no, probably not. I think the, for me, I mean, I i mean i i got into doing consumer about six years ago i went to work for um a connected home platform company called alert me who were um subsequently acquired by british gas hive which you may you may have heard of um and i had sort of grand plans for how all of the we worked on on some modeling, thinking about how all of the things that you could potentially connect to the platform should be able to kind of, to interact with all of the other things, or at least with the other things that was appropriate. And what do you need to do? What kind of domain modeling do you need to be able to do to think about that? So I suppose I look at it in terms of how lots of things coordinate or should coordinate or perhaps don't need to coordinate. And rather than so much at the kind of user interface layer, I think what's happening,

I see happening at the moment is that there are some nice individual interactions you get. One of my favourites, I think, is, you know, I've never thought Bluetooth pairing could be particularly exciting, but I think with the Apple Watch, you know, Apple have actually done a very nice job of making that kind of, you know, it's beautiful, it's quite pleasurable um but when it comes to the whole coordination experience there's there's tons of things but i think i think certainly at the kind of where we were a few years ago was we thought we'd have this kind of you know all of the things wouldn't it be all the things in the house that that magically talk to each other in the coffee machine that turns on and you know all that all that all that mark weiser stuff that's uh that's that's really not

new um and that's not happening. And I've kind of begun to doubt in recent years really whether that should happen and whether sort of trying to design for that, certainly as things are now, is really not a great idea at all. So I think, I mean, to give you an example um we we have um some and my partner and i both work in both work in iot and you know we have a few things at home we don't have perhaps as many as you would think um but um we have a cut we have some hue bulbs and we have one of the hue bulbs started turning itself on and off by, by largely of its own accord.

I'm laughing. Yeah. And I think there's, and we can't get to the bottom of what's going on. And I think there's, it's, it was connected to it's connected to an echo. It was connected to home kits.

It was connected to, you know, we had some rules set up in the native hue app. I think it was probably connected to, at one point it was connected to home kits it was connected to um you know we had some rules set up in the native hue app i think it was probably connected to at one point it was connected to um uh you know it's been connected to if it's been connected to um some some other things as well and we thought we had some sort of api conflict going on here and you know it's just kind of i've actually kind of it's like we've removed it from all of those things and i'm currently switching it on and off with a physical switch. And it's actually quite refreshing.

Loathe to admit that, but it's, I think there's a real, I think there's a real danger that there's the amount of, you know, the dream of the things that talk to each other and anticipate your needs and all of that is a nice one. But actually the benefit of having that for some consume for a lot of consumer things sort of doesn't outweigh the kind of amount of administrative effort that goes into making sure it's all working properly so um that i think that that's my, that's and, um, yeah, that's, that's part, that's, that's a big part of it. Okay. So from what I heard from that, which you covered a lot of stuff that I'm going to now

gloss over in order in an attempt to be humorous. I asked, can you describe a good interaction? And you talked about flipping on a light switch. Yes. Can you describe a good interaction? And you talked about flipping on a light switch. Yes, that's a good, that's a good interaction. It has an immediacy to it and, and it happened. And then something else happened. I have to go all the way to the light switch by myself. And that's,

that is what we're missing in the internet of things sometimes is the cause and effect you do something you're like okay 20 minutes later you're like i still want the light to be on should i tell you again or are you just going to turn it off when i tell you again i think that that that gets into yeah i mean that gets into um uh there's some really interesting stuff i think about how people's perception of you know when we use um internet sort of services you use something like skype we have a mental model of skype that tells us that occasionally the calls are going to fail right um we know that download you know that sometimes it takes a long time to download things so we have an experience

of the kind of software internet that tells us that that like you know later we we understand some understanding of latency and reliability, even if we don't have a deep technical understanding of those things. When you put those things into the real world, quite apart from stuff like API conflicts and the fact that if I have a connected bulb, I actually can't use the physical light switch anymore. It can feel quite broken. It feels broken when we have millions of years of evolution that tell us that the light's going to come on immediately. And even if it takes three seconds,

that's three seconds of uncertainty as to whether it's broken or not. So, yeah, that wasn't one of the points in my original comment. But, yeah, that's a big problem. Well, I think there are certain classes of things where we accept them working 95% of the time. We're used to it, like you're saying with Skype. But there are certain things where if you took something like a light switch, which works 100% of the time and made it work 98% of the time, that's enough to make it completely useless or extremely frustrating. Untrustworthy, unreliable and you think irritating for an absolute number 98 sounds great but until you actually experience you know one out of one out of every 50 times or whatever it doesn't work that's not acceptable

once a month turning on and off the lights it it doesn't work. And I think that this is, you know, that's annoying enough in that context. When you, in my user experience world, I encounter a lot of people who talk about the idea that there should be no UI and I shouldn't have to get my phone out to turn the lights on. You know, the environment should be smart enough to know that it needs to do these things, but zero UI. And I get really angry with this because it's kind of, you know, imagine having, you know, if you had an algorithm that was able to infer your needs 98% of the time, you'd be doing pretty well. But actually, you know, most of them are not that good. But that's, but to then have no way to know whether it's drawn the right inferences,

if it goes wrong, whether it's drawn the right inferences or not, is even more painful. So, yeah, I think there's clearly a reliability issue, which is often underplayed. As you said, we're replacing in many cases with consumer products, we're replacing things that worked 100% of the time with something that's not quite as good as that. And sometimes the additional benefits you've gained

from that thing being connected make that worthwhile, but sometimes they don't. The underlying technology is not 100%. So I feel like we're never going to win. The IoT is never going to be as good as a light switch for reliability reasons because the internet is not reliable. Well, all right. How do we do better i think what i what i'd like to see is that we use the

perhaps we perhaps we use we we put fewer things on the internet um maybe that's a bit contentious from an iot person i think that things that need to happen if the light if you turn the light on remotely because you want to make your house look occupied and it doesn't come on uh or it doesn't come on for 10 seconds that's fine um if it doesn't come on you need to know about it um you you need to know that you know perhaps there's a problem and you know need to try that again um but if it doesn't come on for 10 seconds that's fine if it doesn't come on for 10 seconds and you're actually in the room you know is that is that does that does that really need to go via a via a server farm on the other side of the world or could that not happen locally

so so i think that i i'd like to see um i'd like to see more i'd like to see connected things i know there's there's a whole i know there's a whole kind of move towards towards edge um edge computing fin for certain things i'd like to see things that can be done at the edge being done at the edge for reasons of user experience, I think, and that will help. And edge computing, if I recall from a previous guest, has more to do with putting a hub in your house and having it not go to the internet but go to the hub

and be dealt with there. Or even be dealt with by the that or or even be that be dealt with by this by the thing itself device to device device to device okay i'm not a i'm not a i'm not a technical expert but i've i've come through the course of my work to understand something about different technical architectures and the the various things that that may that may work or various ways that things can fail as a result of different decisions. And I think it's a really important question to ask

when you're thinking about designing, designing, creating a product is, you know, how does this thing, first of all, how does this thing work? But how can it fail? And can it fail in ways that are worse than the non-connected thing that it might've been replacing? Because sometimes that can happen. Okay. So reliability is one of those big issues. The other thing that is very hard with the internet of things is the consistency. I have 95 apps in order to be able to come home and turn on

the lights. Oh, and turn off the garage. And maybe turn on the sprinklers. I don't even remember the names of all the apps. And it's really embarrassing to like, Christopher, how do I turn on the sprinklers? And are we... I mean, there was HomeKit. I had hope for HomeKit, but it doesn't seem to be doing that well.

Are we going to have a consistent interface someday is it are we doomed i feel like we're doomed i don't i don't think we're doomed i think um i i've come to the you know six years ago i was working on trying to design one interface that that that could be one front app that could control an awful lot of things. And I came to the conclusion that that was not really, that's not really the way to do it. I think we may have, you know, for starters,

if you want to control something, you want to find something, you don't want to go figuring out which app you need to use. You also don't want to go digging around in one app that tries to do all of it because because that's going to be just as much effort just a different kind of effort um i think things like voice systems will work for certain things um you need to be able to say if you want to say turn on the sprinklers and you've got one set of sprinklers, and you have a voice assistant at home, that's a perfectly reasonable thing to do with a voice assistant. I think there'll be certain things, though, that are always going to require a lot of configuration and a lot of setup and a lot of um basically system administration to to

maintain and that might mean things like having individual control of all the lights and appliances in your home um and being able to say things like uh turn on the dining room lights or turn on the, you know, turn the, set the living room to movie mode or, you know, turn off the unnecessary lights, all of those kinds of things require either a lot of configuration or, you know, some algorithms that can figure out what those things are anyway, which isn't going to happen kind of with 100% accuracy. So we're not doomed. But I think rather than trying to kind of impose top-down systems on people, we need to let this stuff happen organically and see what makes sense and always allow for you know or as much as possible allow for something to be locally controlled with control with with controls that

are on the device itself so if you can't remember what the sprinkler app is at least you can go and turn the tap on if only in the old days we had standards bodies and you know when the internet was coming together and everyone worked you know all the companies worked together to to make standards for things and interoperability it seems like everyone just kind of went off and did their own thing for a while uh with this stuff with with iot yeah yeah i'm it's yeah i mean i it's it's a it's a it's a difficult one. I mean, I think part of the reason is everyone's got their different kind of technical, you know, people saying thread, we want thread.

We have ideas about how this should be done. Other people have different ideas about how things should be done. But I think part of the problem at the moment is there's a commercial incentive for companies perceived there to be a commercial incentive to be trying to lock users into ecosystems. So, you know, the top selling devices are voice assistants, which are, you know, Amazon trying to take a cut of every transaction you make. It's about trying to get people to buy more things from the same people or to give you more of their data. So the incentives for that are not perceived to be there, I think. Do you think it's a short-term thing? I mean, this used to be true with AOL and CompuServe

and having all of the... CompuServe. Sorry. All of the service providers were very into having everything you wanted and creating little fortress cities. And is that what's happening with iot in consumer space right now or is this likely to and and that was sort of ephemeral it was um i mean there are i think i think what's i think as if i mean you could argue that that with the web at large if

anything with the rise of mobile and and so many things moving into native apps. So I think more than 50% of internet use now is inside mobile apps. You could argue that openness hasn't really gone the way we'd hoped it would have there either but i think i think what we're seeing is just is that there's no um companies making their own things and then kind of creating limited interoperability with with other products so for example you know q have some open apis you can have lots of other products that talk to q bulbs um but full open standards where everything could potentially talk to everything else is, I think there's just not perceived to be a great commercial interest in that for a lot of companies who are kind of heads down trying to make money out of an individual product.

But you mentioned voice assistants, i think is is a great direction because it is a way to generalize interfaces um if if the companies like you were to write skills apps or whatever the other i don't know all of the different voice assistants but the the skills apps or the interfaces that way then my home IT person can then program everything to do what I want. Of course, that still predicates having a home IT person, which is kind of awful. I mean, I admit, we have an Echo and we've not had great experiences with it. I think from what I understand, most of the interactions with Echoes

are people asking them to play music. Or asking them to tell them jokes. I spend a lot of time asking for jokes in Dinosaur Facts. Yeah, I think I should tell my son about the Dinosaur Facts. I know that there are others which are which are smarter but there is there is still there's still a problem that you have to have that um to do things like the lights or to to you you have to have somehow that knowledge it is it's essentially a kind of ivr system and so you're you're um you have to know it's a nice idea that if I know all the things I can ask, I can ask the echo.

Actually, it's even more like a command line interface in some ways that you have to know what you can ask and you have to ask in a certain way. I've had terrible trouble getting the echo to control the thermostat. Yes. And it changes, right? It changes. I think of it more like knowing wizard spells you know because i i've read maybe too much science fiction but you have to know the proper incantation in order to get the heater set up and you have to say it in just the right way with just the

right inflection and so i totally feel like okay i know this spell and this spell and that spell and if i mess them up then who knows what's going to happen with the house. It might explode. Yeah. I think this is, you know, I sound like I'm being a bit down on voice, but I think, I think it was Benedict Evans wrote a blog post a while ago talking about the

uncanny valley of voice that it's, it's kind of, it's quite a good experience. If you know that there are four or five things you can ask and you will get a response to those and you know how to, then actually that's quite, that can, that's fine. If you have something that can parse any natural language that you throw at it and understand what you mean. So if I were to say, turn off all the unnecessary lights when I go to bed or turn off all the unnecessary appliances or kind of turn on the living room. And it knows what that means. And it doesn't have to know what

that doesn't require somebody to go in and tag all of these things in order to achieve that. Then that's a really nice experience as well. but where we are at the moment is somewhere in between where we kind of the promise is of is of the full natural language system but the reality is actually somewhere along somewhere more like the command line interface um so it's it will get better um but it's it's not it's not quite where it's where we like to think it is at the moment. And sometimes it's just, sometimes, you know, a visual UI is just a better way to do something. I mean, if you walk into a room and say, turn on the light, you know, or turn this room to, you know, 21 degrees,

because Celsius, obviously, then, you know, that's fine. But I think sometimes you do just need to, you want to see what's going on. You want to know what movies might be available that you could watch. You don't want to have to, you want to see a range of, you know, of options that you might have.

So I'm not sure that, I'm not sure that voice is the answer to everything. That's a good point. And definitely, I wish i could do more voice with some of what you're saying but i want it to be interactive like i want to watch a movie okay it shows me all the movies i have and that i can watch i want to watch a comedy no i don't want i want to watch a funny comedy not your stupid sad comedies be able to actually talk to it and you know that's that's an ideal and it's not going to happen for a long time.

And when it does happen, we'll forget how technology works. And I've seen the Star Trek episode. It ends badly. We've been talking a lot about failure and how the things fail and how unreliable they are. How do we as engineers deal with the failure what what are the ways we should tell our users do you have advice for people who are developing imperfect products so i think it's there's um there's some interest there's some very there's some a lot of interesting things in that i think

uh the the very first thing i say is when you come up with an idea for a product you need to understand um you need to think about the value proposition you're offering people but also about the ways that it could go wrong because sometimes the failures you're introducing may be worse than the than the than the problem you're trying to solve. If that's the case, maybe that's not the right product. And there's one, I saw a brilliant example a while ago. There was crowdfunding for a connected stroller, baby buggy thing, as we call it over here, which moved away from you when you walked towards it,

which was trying to solve the problem of having your hands occupied with pushing the stroller. And I think whoever came up with this must have not had kids because, you know, the bigger problem is worrying that the thing might roll away from you down a hill, not having your hand free for your phone or your coffee or whatever. So, you know, I think that was a classic example of, you know, if I walked towards my child and the buggy moved away from me, that's terrifying. That could go wrong in so many ways.

So I think the very first hurdle, and you would think, apparently this is news to some people, is, you know, sometimes the potential for failure is worse than the problem you're trying to solve. But I think once you get past that, there's some interesting things kind of at the UI level for handling inevitable kind of failures and, you know, delays and sort of intermittencies. One of the ones that's quite interesting, and I first came across this, well, thinking about light switches and turning on smart plugs, is just how you communicate the possibility that whether you communicate the possibility that something might fail or whether you pretend that it's just worked. So if you think about how you might design, imagine you've got a smartphone app and you have a button to turn on an appliance. Now, the sort of natural assumption of the way you would design a switch is you'd say,

well, the switch both, if you press a button on an app and it responds, normally that's responding to tell you both to confirm that you pressed the button, but also to tell you that the thing that you wanted to happen has happened. And that's fine when you're dealing with something that's kind of on the phone and in front of you. But when you're dealing with something that's distributed and that response might happen, it happens on a remote device and it might not happen straight away.

You've got a kind of separation between the fact that you have to confirm to the user that they've pressed the button and the fact that you also need to confirm to them whether or not the thing that they wanted to happen, like the light coming on, has actually happened. And so there's kind of various ways you can do that. Talking a lot about the first the first thing the first one of those is basically just to lie and pretend that the thing has happened until it hasn't um if you um we're talking about philip's hue which just makes me think of that as an example with uh with the hue bulbs if you drag the slider in the hue app um up to turn the light on um it just sort of it just stays there um the

lights may or may not have come on um but but the slider is up there um and if it subsequently figures it realizes that that it wasn't able to do that it just slides all the way back down to the bottom and it's kind of comes across like you know yeah but I just can't, sorry, can't be bothered. So, you know, sometimes that kind of thing is, sometimes that kind of thing can be appropriate. I think, you know, the alternative thing that you can do is to separate the confirmation of the user's action from the confirmation of the response. And then you get into things like when you press the button,

but perhaps there's some sort of interstitial state and you get a response, but it doesn't actually change to the on state until it's had confirmation. Or you get into things like the system I worked on years ago that had, it's a very literal engineering interpretation and it would say, it would say, sending changes, please don't exit. And you'd sit there and kind of go, oh, it mustn't touch anything. And then it would say, changes have been applied, which was, it was very, what's the word? It was very accurate and it was very conscientious. And I think the reason, and something like that, perhaps a bit more elegantly done, but something like that is appropriate where it might be, if you might have a life or death situation where somebody really needs to know that the thing has happened, like they've pressed their emergency alarm and they've fallen over and they need help um but if you do that every time somebody turns the lights on you're gonna it's gonna gonna feel really really awkward and it kind of introduces the idea of failure into every interaction um so there are i think it's again it's it's a it's there's a there's a big kind of there's a scale in between those two things um and the appropriate risk the appropriate way to handle it is sort of depends on kind of, is this life-threatening? How important is it that the user is at no point thinks that the thing has happened unless it actually has?

You know, can you go back to them with a message later on that says, sorry, we managed to do that thing, but, you know, but, you know, you might need to try that again. Uh, or can you just keep trying to get that command through to go and, you know, it's, um, you have to, there's a very fine line between, um, glossing over things that might be a problem, um, and not giving people so much information that it just feels unbearable. Is there a process? How do I figure out where I am on the scale? Is this the time we say you have a book and everyone should read it? I do have a book. It's been out for about three years now.

But I think some of the examples in there are, you know, a little dated now, if not even some of them don't, some of them have, don't exist anymore. But I think the fundamental principles that we cover, it's a case of really understanding why people are using this thing, understanding what their level of tolerance is, their level of understanding, understanding what the consequences are of, first of all, of something perhaps not working as expected. Secondly, how much it matters if the user is sort of under the impression that something has worked or not worked, and that turns out to be incorrect. Does it matter if that state persists for a few seconds or not? How do you set how much of an expectation of an instant response is there? So for another, I guess another example of this is a sort of related example, the idea that lots of connected things run off batteries. And so, you know, they don't, when you, as I do,

come from a software services background, you kind of work on the assumption that everything's connected all the time, except sometimes when it isn't, and that's an edge case. But lots of connected devices, because they run on batteries, have to conserve power. So they may only check into the network every every few seconds every few minutes um for instructions so sometimes you don't always know the the exact up-to-date state of of that thing and sometimes it takes time to get instructions through sometimes it takes time to get data back and i think the way that you can the way that you you can design things um as a first of all you have to make the right decisions about

about how frequently a thing should connect but you can set the user's expectations as to how up-to-date things may be how quickly they may get a response um so and then it's and then it's a process of of prototyping of prototyping and testing. And we have ways of doing that that don't necessarily involve having to build things. But you do need to work quite closely with engineers to understand what kinds of response times you're going to be seeing from things and how best to handle that. That's an important point. Because my perspective is as an engineer, and I want to make good designs,

I want to make good products that are useful for long term and make people like them. But there is a point where the experienced designers or even the UI designers do need to talk a little bit to the engineers to understand the limitations. I mean, some of it is this... I have in my outline a question about how do I talk to the interface designers

when they want to use my precious and limited processing power to make rounded corners on their buttons that's just a total waste of time just make square buttons and we can be done and why so many colors two two or three is really i can only see two or three it's all i really need why does anybody need like 16 i don't know i i'm afraid you're talking to the wrong person about this i'm not remotely visual um and uh you know i i do understand i'm i work a lot with visual design at more visual designers and i understand that it's you know there's subtle experience things that can make a big difference to somebody's perception of the quality of a product and their experience but

that that's not my personal um you know, I'm with you on rounded corners, frankly. But I think that there's a really interesting, and I've heard this in engineering, and I think it's in relation to IoT, but I think it's true in relation to product development in general, that to make a connected product or a hardware-enabled service, there's so many skill sets that have to come together to do that. And, you know, we've seen, you know, in recent years, kind of, you know, embedded engineers and web engineers, you know, kind of software services, web software services people, you know, I'm sure have come from different places, but increasingly, you know, working together more and more.

And I think when it comes to the most successful teams I've seen for working on IoT, understand we'll keep each other updated on what they're doing because there will always be decisions that you make. There'll be decisions you make as a designer and i've run up against this uh that you think are kind of not really very much to do with anybody else and then suddenly you realize that somebody's baking that into the firmware over on another on another device and you're like wow i had no idea that that code was needed to be on that smart plug kind of thing um so yeah i think it's it's it's it's impossible for all of the different skill sets involved to know exactly when something

might affect somebody else or what to ask and you learn over time to say okay how quickly is this device going to respond if i press a button here, how quickly do you think this thing might respond? What's the worst case? You learn that from experience and you learn to say, well, I'm currently working on these kinds of thinking about these kinds of issues for how somebody might group the light switches and sockets in their house. You know, does that have anything to do with, you you know is that something that that affects what what you're working on um and you just have to you just have to to kind of have you know keep talking basically um so i think yeah i think that's that's you know that that's probably my best piece of advice is to try to have some understanding of what other people are doing and to to try to and think in terms of how,

what you're doing might, might, might cross over with that. Oh God, communication. What you're saying is we need to communicate with our, our teammates and that we need to like talk to each other about cost benefit trade-offs.

Like, oh yeah, I can respond immediately, but our batteries will only last an hour. Well, yeah, exactly. I mean,

I know it's rocket science is it's kind of, it's a very, it's an obvious thing to say, but it's... But no, it isn't because people forget and they forget that it's often just a hallway conversation of, oh yeah, I'm working on this. And it has that effect that you didn't expect. I think, you know, because I work a lot with, obviously work a lot with designers, and particularly designers who are, and I often work to support designers who are new to working with IoT. And the kinds of, you know,

people bring a lot of assumptions from, most of them have come from, you know, web and mobile services. And they bring a lot of assumptions about that doesn't occur to them to question. And part of what I do is help them kind of question those and say, you know, for example, I've worked on a number of smart energy meter projects. And, you know, the initial assumption from designers and actually from anybody who's not worked with connected devices before will be that there's a constant stream of data and you will always know

exactly what's happening right now. And that's not the case. You might get 15 or 30 minute readings from, if you're lucky, from a gas meter sensor and you might get seven second readings from an electricity sensor which is you know close to live but not live um and that really shapes what you can do with those things so you see people coming up with designs where they've treated electricity in exactly the same way as gas and like well no we don't never we don't we never have a live reading for gas we have a summary of what happened in the last 30 minutes um and that's and the that bubbles up to the kind of value level that you can use the electricity data if you if you um combine it with some other things potentially to tell people if they've left something on when they're leaving the house but with the with the gas uh with those kind of gas readings what you

can do is maybe say to people well your your heating schedule actually natural gas for heating in the uk mostly um your heating schedule doesn't seem to be aligned with the times when you're at home. So, you're probably wasting some energy here and maybe you could sort that out. But they're not the same. They're subtly different and that's informed by just the frequency of data that's being sent. So, knowing to look out for those things, I think is, you know, that's something you, something you, something you learn the hard way. Going back to talking to engineers and UI people and making sure all on the same page, how much, how much outside kind of testing needs to be done? Because we can all sit in our silos and

think about the best, the best interface in in our heads but sometimes when that meets actual people who are maybe not as well versed in how technology works uh things fall apart so is there a big role for okay we haven't finished this product here's some three by five cards that, you know, pretend to be an interface. Let's put it in front of five random people and see where the holes are. I mean, I'm, I'm a user researcher. A lot of my, my sort of previous work was in, was user research and it's, it's testing is an essential part, you know, as well as some upfront research to understand, you know, what kind of thing you need to be making.

But I think testing is an, is a, is a absolutely vital part of that process. Um, and there are lots of techniques that you can use to draw on in parallel with technical feasibility. So you don't necessarily have to, if you wait until you've got something that's working and then you find out that it's either the wrong thing or you've, or you've designed it all wrong, then it's, then it's a, that's a painful time to be, to be learning that. So, um, yeah, I, I would do, I would do a lot of very early, um, I do a lot of very early testing where we might do things like sketch out storyboards, um, which show somebody how something might work. Um, we might do, uh, so that allows you to show both the screens, so on the assumption that there's some kind of app,

and also how a device might respond to that. I've seen people do lovely things with video mock-ups, even quite low-fidelity video mock-ups, you know, bits of filming things on Instagram with bits of cardboard and paper and post-it notes that they've made out of cardboard, paper and post-it notes to try and simulate some of those interactions. So I think trying to, it's much harder than it is

when you're making something that only happens on a screen. But there are ways to try to get try to get at that um and i think you know even sometimes um if you do it in the right way testing something where you have a screen-based prototype and um is you you can you can still get quite a lot from that but what you don't get is those is those those things where you know perhaps that took a little bit longer than you were expecting and how does that expecting? And how did that throw them? So, yeah, testing, yes, hugely important. I think the most successful teams I've worked with who do that view user research as being an ongoing activity throughout the whole project and just test whatever they have at different levels of fidelity throughout.

How much does a good user experience, because that is a much better word when we talk about IoT things, how much more expensive is it? Is there a multiplier? Because we're talking about more revisions in software and hardware, more thought up front, more work for designers. And do you have an idea of like, okay, design costs X and good design costs 2X and great design costs a billion X? How do we quantify this? So the idea should be that good design and good prototyping and evaluation up front should actually save time

because you don't waste time building the wrong things, or you waste as little time as possible building the wrong things. I think sometimes the goal is to get a thing out of the door for one reason or another um and i've struggled with a few of those projects in the past but um i think if it's if the goal is to make a successful product then it has to work for the people who are going to be using it they're not going to they're not going to they're not going to buy it they're not going to continue using it um if it doesn't work for them. So, you know, the end point, it might, sometimes it might cost a bit research, which relates entirely to software around the idea that I think a dollar spent on usability saves $10 in development cost. I'd have to dig that out someone did do that someone did do the stats on that once um but it's it's it's it's very it's it's very it's hard to quantify because well what does

good design what does good design mean what are you what are you actually saving sometimes what happens is you do some research up front and decide that and discover that nobody really needs the product after all saves you a lot of, but it also means that you don't have something to sell. So, yeah, it's a tough one. But it should, doing good discovery, doing good testing up front should save you time because it means you're not getting it wrong. Well, and if...

Too long. For a complicated enough interface, and I've been in situations like this where, okay, we have this UX design, it's great, you implement it and then realize the users really don't like it and somehow that got through your testing and then you have to revamp it and it's like, oh, but we already did a ton of engineering. This is a big infrastructure thing.

It's not just how the screens look, it's everything underneath it. and so you have to refactor the whole thing and it's a major project so yeah it's it can be a yeah undoing it can be a major problem i mean there's this this whole kind of lean model of um you know software development um now there are you get a lot of startups who do do you know may may do very little testing they throw something out they see how it they track how it's doing and they make changes from there but i think you can you know it grates a little bit with me because i think well if you've done a bit of research you've got it righter to start you've got it more right to start with but i think it's it's um you know i find myself quite often having to explain to people that you can't do that with

hardware. You can't throw something out there and then tweak it. You know, you have to, if you're going to put something, you know, if you're going to put a door lock in somebody's house or you're going to have a, you know, a kind of dangerous power tool or, you know, or, you know, a sort of, you know, connected car or something. You can't afford to get these things. You can't change these things so much once you've put them out there. And even if you could, if it's a door lock and you put out something and it isn't perfect, but you affirm we're updated, it's all fine.

You've lost the trust. You've lost the trust. You've lost the reliability. And because it's hardware, it costs real money. It's not software you can give them a taste test for a little while. I mean, maybe you have a few people with prototypes, but your big user testing has to be with users. Well, we come back to that we come back to that thing of about failure that it's it's you know the the consequences of software failure are can be very severe but

the consequences of real world things going wrong aren't are more severe more often um i think there's i think the the update the update thing is interesting i don't know if you saw um i think it was about a week ago there was a an article in the verge about um tesla owners getting upset about uh about about an ota update um and uh i think is that the the model three had been um the braking the stopping distance for braking had uh uh sort of not being that great. And Tesla pushed out an update that dramatically improved the stopping distance of the car. And, you know, Tesla push out updates all the time, but it's usually things like kind of steering wheel haptics or kind of small changes to stuff. And this really freaked people out because it was like, you you know you have that much control over how my car works

you know what what did i buy you know and so there's this kind of idea that that that you know you buy a product and it basically kind of a physical thing and it basically kind of does the same stuff um and then there's and then suddenly there's suddenly it suddenly behaves very differently and i think some people felt that their acceleration wasn't as quick as it had been as as well but just just the degree to which that big physical thing that was a car could be modified by the manufacturer after they had bought it was quite disturbing to some people. I love it when people bring up Tesla and Christopher gets all twitchy. The thing that I actually found disturbing about that update was how fast it happened.

I like my firmware updates to be tested for weeks. We don't know that it wasn't. You know, there was some discussion of how it wasn't good enough, and then poof, there was an update. Like three days later, it was slightly terrifying. I like slow things sometimes. Yeah, I don't know. We don't know we don't we don't we don't know how long we don't know how long they spent testing it that's true but uh going back to your book

uh who is the audience for it so there were two um the first was uh people user experience people from user experience backgrounds who were new to working in IoT. So there's quite a lot of sort of technical primer content in there, which we tried to make as engaging to designers as possible, that's aimed at that audience to help them understand basically what's different about working in IoT from working with pure software services. And the second audience was people from engineering backgrounds, um, who were interested in what user experience, um, you know, what the user experience angle was,

um, of IoT. So, um, and, uh, I had a bunch of, uh, people in mind that I knew, um, in, in each camp. Um, and, uh, I've had, well, yes, we've had feedback from both sides that they found it useful. So hopefully we did a good job. With my book, I had two audiences that were kind of opposed hardware and software engineers. And I felt like if you were in one of those camps,

the book was only going to teach you half, but it was going to cantilever it off the information you already knew. And I liked that. And it sounds like yours is like that as well. And what I read, you know, I kind of skimmed the parts that were about the technologies I understood. But the other parts, because I learned how you talked about the technologies I knew, I felt like I could better understand the new stuff you were presenting. Sorry, because I talked about the technologies you knew. Because the book had covered the technology stuff to start with.

Well, because I had learned to trust your voice. Well, you read a technical book and you're like, okay, how much of what I don't understand is me and how much of what I don't understand is the author doing a bad job. And so if I get to read part where I definitely understand the material and then I can figure out the author's presentation style. Do I trust them on the rest of it? Yeah. Yeah. that's interesting um i mean i think you know we it ended up being a very big book um probably too big but um it was what we were trying to do was to and we realized that no that nobody had done at that point was to try to integrate when people talked about user about ux variety they would tend to talk about sort of parts of the problem so um how do we industrial design of

the things or the kind of you know app design and and um you know and then occasionally they'll be like oh and we have to think about privacy on all business models and this but these were all kind of separate conversations that were going on so what we were trying to do is to sort of bring that together into trying to integrate that into some sort of framework so that people could say, all right, I don't, not all of these things have to be my responsibility, but I understand how the decisions that other people make, you know, whether they're about technical architecture, whether they're about, you know, how to make money out of the thing, I understand how those things shape the user experience. So I would expect that for most people who read that book,

there will be a few things that will go, oh, yeah, yeah, I know all that already, and that's good. So it's nice to hear that. And one of the things you do cover in the book are security and privacy and user experience, which is awesome because so much of the time I spend telling people, look, if you want security, your user experience isn't going to be as good. And it feels like a trade-off. Do you have advice on how to make it less of a trade-off and more of a, of course, we're going to have good security. Of course, we're going to have excellent privacy,

and we're still going to have a great user experience. So I think on the security and privacy front, they are different. We didn't come to, I actually went back and looked at what I wrote about security before I talked to you. And I think with security, there is always going to be a tension between security and usability. Nobody wants to have to log into their light switches every single... We talk about light switches a lot.

I'm not sure where that's come from, but nobody wants to have to log into the light switches every single time um nobody wants to be manually running firmware updates on the on the things all the time um i think some of what we need to do is to to try to reduce the harm that can be done there's i mean there's a lot of basic security in many, many IoT devices that's just not being done right, and that's the first hurdle. I think we also have to try to reduce the harm that can be done if something is compromised. So, you know, it's one thing if your fridge is kind of very, very slowly

and painstakingly mining bitcoin it's it's another thing if you're um it's another thing if you can have a gas leak or a device can set fire to your house or or something so trying to figure out ways that that when things are compromised that they can they can fail in in safer ways um is important with something like a car that's kind of obviously very difficult um but i think that also one of the other i think probably perfect security is is not is not feasible because there is a point beyond which when you make things more complicated for people to do they stop doing them so in some cases it's better to say we will allow you to we will design something which is a little bit less secure but it's easy enough to use that people will use the security that's there rather than kind of

circumvent it so it's like the the kind of thing of requiring people to to use incredibly complicated passwords and then they just write them down and stick them somewhere in plain sight. So, yeah, there's a lot that needs to happen technically, I think. The usability part is maybe not the biggest problem we have with IoT security at the moment. What about privacy? So that's an interesting one. There have been many, many developments on that front since we wrote the book, kind of notably GDPR in the EU. I think, I know I've watched somebody on American News Channel the other day kind of saying how they think GDPR is, there's very, very different cultural attitudes, I think, to privacy in Europe and in parts of the

US. But I think GDPR is a good thing. I think at the very least, it's prompting a conversation about what people should know about the data that's being collected, how much people should know about the data that's being collected about them, how much they should know about what's being done with that, how much control they should have over that. Wait a minute, I'm going to stop you. Sorry. Because all I know about GDPR is that I got a whole bunch of emails that said we're sending this email because of it.

We hope you still like us. Okay. What is it and why should i have read those emails i have to leave it okay so so uh so gdpr uh general data protection regulation this is a system new um eu uh regulation that's um largely designed by germans who are possibly one of the most privacy, pro-privacy kind of nations in the world. And what it says is that users, people must have, it's about privacy, designing privacy into systems. So you have to have a valid reason to collect a piece of data.

You have to be very transparent about what you can do with that. Users have the right to ask a company what they know about them and to force them to correct wrong information or delete it. One of the other things that it also does is put the onus on the provider to obtain informed consent to any use of the data. But it also requires them to prove that the user's consent was informed. So the reason you're getting all of those emails is because a lot of companies who've somehow acquired your email address, but haven't necessarily explicitly asked for permission to send you emails, are now realizing that they have to retrospectively get that permission. There's a lot of misunderstanding. Some of those emails

are unnecessary. There's a lot of misunderstanding about how to apply the law. So maybe not all of those. Some of those might have been emails you had may not necessarily have been relevant. But the reason this is a really interesting one for IoT is that it's really to do with the sheer amount of data that it's possible to collect and the number of things that it's possible to do with that. So, you know, for example, you have if you have a smart energy meter in your home, that data provides a pretty good proxy. You can infer from that just by just by looking at a simple graph of it when somebody's at home and when they're not. And so that permission to collect that data does not necessarily, you know, are you aware that that could be done with that data? Do you believe that that should happen with it?

Do you consent to that use of information? And so all of this stuff has been written up in very long end-user license agreements to date, which people don't read, don't really understand, even if they do attempt to read them, don't really understand. So there's been a lot of kind of uninformed consent to do things, and then subsequently data companies discover that they can use data for new things, or they can draw inferences, they can combine it with data from other sources. And what GDPR is trying to do is make,

incredibly difficult thing to try to apply, is make all of that much more transparent. So, if you have consent, if you're an energy company, and you have consent to collect data on somebody's energy usage throughout the day so that you can provide them with more accurate bills, so you can perhaps provide them with some tips on how they might save energy, that's one thing. If you start inferring when they're at home or not and you start, you know, start many other things you could do with that, then you need to go back to them and get permission for that again. So I think what's an interesting thing UX-wise is going to be how we,

instead of burying all of that stuff in end-user license agreements, we're going to have to get much smarter about designing conversations with users about the data that's being collected and what it's being and what it's being used for. And I, I haven't seen any of that, um, happening in industry at the moment, but there have been some interesting kind of design explorations coming out of academia that show, for example, how, uh, you might say, well, if I have a smart lock, um, I'm going to, uh, I have a trade off. I can perhaps trade off functionality for privacy so I can say I can only unlock this it can only connect and locally so it'll connect to my phone over

bluetooth so I can open it but that's it I can't open it I can't open it remotely or I accept that it will send information to it I accept that that I will share information with the lock company, perhaps gives me remote access. It helps them understand how I'm using it and perhaps make it, you know, perhaps fix problems that might be occurring. Or I can say it's allowed to exchange information with third parties.

So it can, you know, when I unlock the door, I can trigger, you know, the heating comes on or the ac comes on or something else happens uh but i accept that my data is going is going out uh is going out a bit further so it's yeah it's it's a it's it's an area that's ripe for quite a lot of design for a lot of design exploration just to just to have smarter conversations with people about what's happening with their data. I think this is great. I had a potential client contact me and want to do medical devices and didn't really understand why I was saying, no, you can't just collect all the data, store it, pretend to anonymize it and then decide to have grand cross studies. That's not enough.

We have a thing called HIPAA and this GDPR is going to kick your, well, never mind. I do think it's important and I'm glad to see it's getting more conversation. I think it's, I mean, it's just the really challenging stuff is that kind of how, you know, I'm not sure it's actually possible to apply GDPR in all instances. Just the complexity of anticipating and communicating, getting consent to all of the things you might be able to do, you know, and to anticipate all of the things that you might learn from certain data you remember the uh strava thing of a few weeks ago uh a few months ago that um that the data from strava's global heat map was inadvertently revealing the location of u.s military bases in in afghanistan uh and you know and elsewhere um you know nobody nobody saw that coming that's not a thing that you can it's not a thing you can ask permission for. So,

yeah, I think it's an interesting one. It's those emergent kind of uses of data that are hard to anticipate, that even if you do all the right things, somebody may find a way to synthesize meaning from what seems innocuous. Well, and even innocuous things can be interesting, and so you still want to highlight them. When I worked at ShotSpotter doing gunshot location systems, we were doing long-term GPS collection, which makes GPS extraordinarily accurate.

And we could see the difference between Oakland and San Francisco over time. We could see the earth move and it was awesome. And how could we not want to tell everybody about it? And yet it was, I mean, and of course the earth doesn't have a privacy policy, but if it did, the earth might not want to be any, I don't know where this analogy was going. Okay. the earth might not want to be any i don't know where this analogy was going okay uh we should we should let you go about your weekend and uh before that let us ask you mentioned the iot uk report to us before the show started what is that so it's a report uh an organisation called IoT UK. IoT UK is a joint venture between two of what we call the catapults technology generally, and the Future Cities Catapult, who work on smart city applications.

And so IoT UK asked me to write a report about the current state of UX and IoT, focusing specifically on the UK market. So what that is, it's aimed at small, medium-sized businesses, helping them understand what the key issues are if you're developing an IoT application or connected product, and how best to go about uh how best to go about um you know exploring ux in that area and making a well designed product um so it has a bunch of case studies from uh from different uh different mostly startups and some slightly larger organizations in the uk looking at the kind of current state of of how people are doing ux and can people buy it? It'll be free. It's coming out in the next week or two. I'm not sure of the exact date yet,

but I've been promised sort of around about a couple of weeks. And the web address for IoT UK, it should be published on there, is iotuk.org.uk. And I think there's a Twitter account as well, so we're at iotuk.org.uk um and uh i think there's a twitter account as well so at iotuk but i will i will tweet about that when it comes out and uh and my twitter handle is at clurr which is clurr where did your twitter handle come from um it's an old uh it's an old nickname it's an old university nickname it's the way that uh my name my name is pronounced in certain parts of the northern UK. And also, I've been on Twitter since 2006, back when it was important to have a short username.

So hence the not full name. Claire, do you have any thoughts you'd like to leave us with um i think um for me one of the thing i the thing i one of the most important things i learned when i was doing doing ux and in this space um and it's something i i consider i keep having to tell people over and over and over again so it's like i have an audience here. I'm going to say it again. Is that if you're designing a connected product or an IoT application, it's not a case of making some user interfaces,

you know, making a thing, making some user interfaces. It's, you have to think of the whole thing as a system. You have to design how all of those things that people interact with are going to work together. But you also have to think of the whole thing as a system. You have to design how all of those things that people interact with are going to work together. But you also have to understand that the design goes all the way down to how you decide to make money from this thing. Do people consider that to be a fair value exchange?

Is it a product or a service? You know, there's some new challenges in connected products in that sense. You know, and as we've talked about, you know, in connected products in that sense. You know, and as we talked about, you know, the technical architecture, the decisions you make about which bits of code run where and which things may sometimes not work as quickly as you want them to. And I, you know, I think I'd like people, I'd like people to understand, you know, more broadly to understand that, you know, UX is a, UX for IoT is a holistic thing. It's not just about that, you know, UX for IoT is a holistic thing.

It's not just about the, you know, it's not just about the interfaces. Excellent. That's my preaching. No, that's great. And it's so true. Our guest has been Claire Rowland, London-based product and UX strategy consultant specializing in the Internet of Things. She's also the lead author of O'Reilly's Designing Connected Products, UX for the Consumer Internet of Things.

Thank you for being with us, Claire. Thank you very much for having me. It's been a pleasure. Thank you to Roman, a.k.a. NZ Smarty, for asking about UX in embedded systems. I hope we have answered your question much better than we tried to last week. Thank you to Stacey Higginbotham of the IoT podcast for introducing me to Claire. If you want to hear more of Claire, check out her episode on Stacey's IoT podcast. If you want to hear more about me, you can check out my episode too. And finally, thank you to Christopher for producing and co-hosting.

Oh, wait, wait, there's more finally. Finally, finally, thank you for listening. You can always contact us at show at embedded.fm or hit the contact link on embedded.fm. A quote to leave you with this week from Bob Ross. All it takes is just a little change of perspective, and you begin to see a whole new world. Embedded is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show, we did not put them there and do not receive money from them. At this time, our sponsors are Logical Elegance and

listeners like you.