Embedded - 431: Becoming More of a Smurf

Episode Date: October 20, 2022

Jasper van Woudenberg spoke with us about hacking hardware, writing a technical book, and ethics. The Hardware Hacking Handbook was written by Jasper and Colin O’Flynn (ChipWhisperer and episode 286: Twenty Cans of Gas). The site related to the book is hardwarehacking.io, you don’t need the book to play with some of the examples. Jasper (@jzvw) is also the CTO of Riscure North America, a company that specializes in hardware security. They are hiring.   Transcript

Welcome to Embedded. I am Alicia White alongside Christopher White. Have you taken a look at the Hardware Hacking Handbook? I hope so. This week we'll be talking to Jasper von Wutenberg. Hey, Jasper. Welcome. Thank you, Chris.

Thank you, Chris. Thank you, Alicia. Could you tell us about yourself as if we met at an Embedded Systems conference? Sure. So I'm Jasper van Moudenberg, as we say in Dutch, or Jasper van Moedenberg, as we say in the US. I'm originally from Holland, but now living in the Bay Area with two children, one wife and three bicycles. I've been programming since I was analysis and fault injection, and recently more interested in how we use pre-silicon simulations to root cause vulnerabilities on chips early, so they can actually be fixed before they get baked into a chip. I'm generally a curious person, so I like to know how things work on the inside, which is not uncommon for reverse engineers, I guess. I also like to challenge myself to do things that I haven't done before, sometimes maybe things that I'm a little scared of.

And as Alicia already introduced, last year Colin O'Flynn and I released the Hardware Hacking Handbook, which weighs in at two to the ninth pages and took us about seven years to complete. I like to flirt with AI. And in contrast to my former self, I try to avoid caffeine daily and try to get nine hours of sleep. All right, we're going to talk more about the Hardware Hacking Handbook

and hardware hacking, which shouldn't come as a shock. But for now, we want to do lightning round. We'll ask you short questions and we want short answers. And if we're behaving ourselves, we won't say why. And are you sure? Are you ready? Let's do it. When somebody finds out what you do, what question do they always ask you?

What is the latest thing that you hacked? What is the latest thing that you hacked? Yeah. No, no, I'm asking now. Oh, that's the next question. Okay. Gosh. Usually that's confidential.

I've been hacking a lot of things in simulation lately. So I guess the answer would be a simulated AES score. Okay. What's something that a lot of people are missing out on because they don't know about it i think a lot of people are missing out on internals of how systems work just opening up stuff and looking what's on the inside i think it's a wonderful world which sesame street character best represents you? I wasn't a big watcher of Sesame Street. I guess... You can choose any Muppet.

Yeah. It doesn't help. You can pass. I'll pass on this one, yeah. Do you have a favorite video game? Oh, I haven't played video games in a long time. Actually, I used to really suck at video games, which is why I got into hacking.

Let's say there was this game in my mid-teens, which was called, I think, 4D Drive. And I liked playing it so much that I wrote a level editor for it at some point. If you could teach a college course, what would you want to teach? Oh, definitely something on hardware hacking. And do you have a tip everyone should know? Saying, I don't know,? Saying I don't know when you actually don't know is a superpower. It's very hard for some people.

Oh, I know, yeah. So the book, Hardware Hacking Handbook, your co-author was Colin O'Flynn of Chip Whisperer fame. What was it like working together? I think Colin, even for a Canadian, he's friendly. But he's, I mean, he has an academic background,

so he's still critical where needed. So he was a really good sparring partner to think about, you know, what are we going to put in the book? What are we not going to put in the book? Feedback on text. So I think that really helped. Of course, we both are really interested in this topic and passionate about it. So that's always a good binding factor.

He was also, you know, persistent enough to work three years of writing with me. And yeah, I think in that sense, it was a really smooth partnership, if you will. You said it took you seven years to write the book. Did the landscape of hardware hacking change in that time? Oh, that's a more painful question than you realize. There were certain sections of the book that we had to rewrite probably three, four times as things progressed. So the answer is definitely yes. Trying to come up with a good example

of this. One thing that remained mostly stable, at least, were some of the basic techniques. The book is focused on people who are trying

to get into the field of hardware security. So a lot of the basics didn't change. But there were a few things like, for instance, deep learning applications in side channel analysis that all of a sudden came up. I know the Spectrum Meltdown vulnerabilities came up during the writing of the book. So those are all things we were like, oh, we better go back to do some more edits

and make sure that we cover some of that as well. We also realized that if the speed of writing is actually lesser than the speed of the world that moves around you, you may actually end up in an infinite loop of edits and never submit a book. So near the end, we had to push on the gas a little bit to make sure we got the book out and could call it a version 1.0. How did you decide you were done? That's a good question um honestly i think at some point um

you just want to move on and get it out there so i think we we were happy with the content that we had we had a few tough phone calls together where we're like nope we're not going to add this new thing because it's going to you know extend the timeline again we know it's really cool content but you know there can always be a v2 i think that's at some point the thing that started going through our heads this is not a promise by the way but at that point it was a way for us to for us to close things off and call it done. The first section of the book, chapters one through three, were all an introduction to embedded systems. Yeah.

What made you put that in? Why didn't you just start with, okay, let's get to hacking? We assume you know all of this. Do you think we should have left that in? Why didn't you just start with, okay, let's get to hacking? We assume you know all of this. Do you think we should have left that out? No, I actually liked it quite a lot. But it was definitely an introduction to embedded systems as a small course. And I was surprised to find it there. Yeah. Yeah, I think that was, I mean as as most things in the book a very deliberate deliberate placement of us to our decision of us to to to put that there and

we're really aiming this at the beginner um meaning you know somebody at early college level maybe no hardware experience at all or embedded experience at all, and trying to give these people just enough information to understand the more hacky, or hacking, I guess I should say, the more hacking sections of the book.

Yeah, just to make it more of a self-contained book of this journey of, okay, let's first learn a little bit on how these embedded systems work, and then we can start poking at them. accomplished through things like logic analyzers and some are more easily accomplished by things like chip whisperer where you're you're doing differential power analysis or or things like that which one of those is more interesting to write about more interesting to write about um i guess the question is also is it more interesting to look inside the chip or more interesting to look inside the hardware? I think they're both interesting fields. I think they're very different disciplines, though.

When you look at logic analysis, it's much more about reverse engineering, the ones and the zeros that you see, like interpretation of that, which can be a fascinating field in itself. Once you start looking inside of the chip with side channels, et cetera, you have these very noisy signals that come out

and sort of the puzzle that you had with logic simulation all of a sudden has a couple of extra dimensions. Like you need to make sense out of this noise. Like do we need to first average a bunch of traces to see if we can get something? Do we need to do some frequency filtering? What is actually the operation that's going on?

So I think if you look at it from a perspective of somebody who's beginning hacking, definitely start with the logic side, sort of build up those reverse engineering skills, and then you go deeper into the chip where you start dealing with these, you know, 4D chess type problems. It definitely is a 4D chess type of problem.

I agree that it's, you kind of need the basics before you look deeper into those, like trying to find the keys inside of a chip. It's so much harder than trying to find what the chip is saying to the spy flash. Definitely, yeah. Your chapter titles were very amusing um casing the joint identifying components and gathering information how much time did you spend procrastinating choosing titles versus writing the material?

I think, unfortunately, vanishingly little. It would be a nice story if we could talk about it. Now, I'm actually trying to think back on the history of that. I think what a lot of people might not know is I think we started off this book with like seven authors or so. All like at that point in time, like authors with way bigger names and claims to fame and then Colin and me. I think Colin and I were the only ones who were silly enough to make it all the way to the end. One of the other authors was Joe Fitzpatrick, who's also well known in the hardware security field. I think he might have kicked off sort of the trend.

He also donated one of the first chapters on signaling and logic interfaces. I think he may have started the trend, and then Colin and I just kind of rode that way for the other chapters as well. I'd say probably 80% of the titles is Colin, though. He was much better at this than I was. The book goes through different ways of hacking hardware, as to be expected, but there was one chapter about countermeasures. What was in it, and what would you add to it now? Yeah, there's a couple of chapters where, and the countermeasures one is informed by, I guess, Colin and my experience of explaining hardware hacking just to people interested, customers, etc. eventually it's nice that we can find vulnerabilities but the question is for us

always what do we do about them and it turns out that things like side channel and false as well there are kind of fundamental issues in the way the chips are built um It's not a bug that you just fix and then you move on. You actually have to actively sort of counter these physical properties that we're exploiting. So the trigger for the chapter was really like people asking like, okay, now what, right? How do I fix this? So in the chapter, we go through a number of sort of coding patterns it's actually

it's focused more on software developers and with software i mean firmware as well i know there's people who have different implement this is not is not web stuff, right? This is embedded system programming. So this is software that runs on an embedded system. And kind of the design, almost like a design patterns idea of like, what are the countermeasure patterns that you can do? So for instance, for fault injection. So with fault injection, you try to flip some bits in a chip that's actually executing some program and kind of like if you were playing pinball and you bumped the table exactly yes it's funny you bring it up that's one of my favorite go-to examples in some of the presentations I did.

Because the analogy goes even further. Like, why would you bump the pinball table? So it turns out that historically, you had sort of these pinball tables where there was no tilt sensor back in the day. And that was fine. People were just playing with the thing. And then at some point when there was a monetary payout attached to it, like in some gambling halls, I guess, or casinos or wherever they were, then it became interesting to start bumping that table, right, to keep the ball in.

And that's when the tilt sensor was introduced. So it was really this sort of cat and mouse game and or attacker defender where you know all of a sudden there was a monetary incentive to start kicking that table as much as you can and then the countermeasure was oh let's put a tilt sensor in there yeah and then you try to figure out how far you could go exactly and and this applies one-to-one with fault injection on embedded systems, right? Why would you fault a system that doesn't hold any value? Well, you probably won't.

Or maybe you're just doing some, you know. But what's the equivalent of the tilt sensor? So, yeah. So, if you go back to the countermeasures, the equivalent is, well, I guess the most direct equivalent would be there's sometimes in more secure chips, they actually build in sensors to detect faults. So this can range all the way from sensors that monitor the VCC, basically the power line to see if there's glitches coming in from that side. More advanced chips may actually have optical sensors in the die itself. So if you open up the chip, expose it,

and start hitting it with lasers, you accidentally hit the light sensor, then it knows it's under attack. So those, I think, would be sort of the most direct. Yeah. How do you say that? The most direct equivalence. Yeah,

exactly. That's very interesting. I've never heard of the photo sensor inside a chip before. That's, that's, I mean, it's an obvious thing to do, but it's like, okay, that's kind of crude. Yeah. Well, and that's i mean it's an obvious thing to do but it's like okay

that's kind of crude all right yeah well and it's and it's and it's when you design a chip it's actually expensive right yeah it takes away from performance and that's why it's not in all chips right these are chips that are in smart cards that are in you know the bank cards that we are the credit cards with the chip that we carry around nowadays, they will probably have these kind of features because there it makes sense. So you got to do your fault injection in a dark room then? Yeah. Or not use a laser.

Or use an IR laser. Laser, yeah. Or you look at the dye. You try to avoid the positions where the, where the, where the laser sensor is. Amazing. That's another, another way you can do that. I have a listener question from Benny, uh, asking about in many of the risk calculation systems, physical attacks are generally kind of low on the feasibility scale. Is that true or is it a misconception about

what hardware hacking is? Yeah, that's a good question. And the first thing that I always like to say is if you have a system that's attached to the internet, the first thing you got to worry about is software vulnerabilities. Let's be honest, right? It means that anybody from anywhere in the world can probably reach your system, break into it. And with most hardware attacks, and I'm putting a little asterisk on that,

that I'll get to in a second. With most asterisks, hardware attacks, or attacks on hardware um you have to be physically present so that's i guess where where you need to start thinking about your risk calculus if if that is indeed the case i don't know there's mobile phones wi-fi routers there's smart cards these are all um systems where the attacker can potentially get physical access to something. Of course, they still cannot

scale things like they would on the software side, on the internet side of things. But scale isn't always what's necessary. Sometimes there's some valuable data on one particular device, and if you get it off, you win.

Let's say, I don't know, a Bitcoin wallet, right? If I have a hardware Bitcoin wallet and I know there's a million dollars on there and I could steal the seeds off of there, that is not something that I need to scale. I just need to do it on one device and then I'm going to go live in Rio for the rest of my life. Well, maybe not a million dollars, but anyway. So scalable attacks always come first. But yeah, shortly after that, I think hardware attacks come in. And I think what people don't often realize is that if you don't protect the system against hardware attacks, like I mentioned earlier, they will be vulnerable. So, you know, with a chip whisper, you're going to be able to break into it and get some data out that you may not want to get out.

I left a little asterisk earlier. I said most hardware attacks, and I need to actually be precise in what I say, most attacks that exploit hardware vulnerabilities are ones that you have to do locally. So you have to be physically present with an EM probe or with a laser system or a chip whisper or whatever it is um in the last few years there have been um attacks on hardware that

explicit that you know do side channel analysis or do fault injection uh remotely and the way that that works is actually quite interesting. Colin actually had a publication on this. On the SideChannel site, he managed to find a system that had an ADC in it. And that ADC could measure the power consumption of the device itself. So you can actually remotely, let's say, hypothetical situation, you remotely have some sort of shell access to the system. You don't have root access.

But through this ADC, you can measure the power consumption of the device. So you can do, I don't know, you can start. All of my devices measure their power consumption so that I know what the battery is doing. Yeah, but you don't have Shell on all your devices. Seriously. That's true. Yeah.

Well, I think there's, and I don't know if, I don't know about all your devices, obviously. I think the critical part here is that you need to be able to sample relatively often per second. Yeah. Right, if you look at the battery once a second or so, if that's the max sampling rate, then it's not going to help

side-channel analysis much. But if you can do this 10,000 times a second or 100,000 times, you can get really fine readings. You might be able to do side-channel analysis that way. Similarly, there have been attacks where you exploit the fact that modern chips, they dynamically adjust their frequency and their input voltage in order to maintain battery life. if you can simultaneously drop the voltage and increase the frequency,

you may actually be able to push the system outside of its normal operating range. And the effect is false. Bits just start flipping once that happens. And if you get enough control, and that's what these attacks, or at least the publications on these attacks have shown, you may have enough control to actually start injecting faults in a system remotely that way as well. You're effectively overclocking the chip at that point, right?

And causing a burnout. The voltage is too low to maintain its... Yeah. Yeah, okay. Yeah, exactly. There is the class of physical attacks that lead to software attacks. Things like reading out encryption keys that may be used on a firmware update. You read it out on one system and now you have the keys for everybody,

depending on the security of your system, but let's just say that that happens. How many other attacks are like that, where you attack the hardware, but that gives you the ability to attack a larger attack surface? How many times did I say attack there? Was it 12? Yeah, I think in quite a few scenarios this happens. So one of the great examples, and we covered in our book as well accessible and find vulnerabilities in there that then later could be exploited through software means. So that's sort of one typical scenario. Another thing that some of our customers are concerned about is, okay, I dump, I do the firmware. I want to keep my firmware proprietary. I don't want people looking at it. That's their choice. Fine. Once you dump it, yeah, you might be able to use that as a competitor for those products. And we're seeing that more and more also with neural networks nowadays. So there's

large investments into, let's say, a neural network for self-driving. There's billions of dollars being poured into the design of these networks. So there's quite a bit of financial stake for these companies to protect that. If you can dump that through getting access to these systems through a glitch and then just do a memory dump and getting this kind of information out. These are the kind of scenarios these companies are trying to mitigate. What is required to take a physical attack like this from an evaluation board or a lab setting

and put it towards a production system? Is it very different? It is basically the difference of white box versus black box hacking. Like, do I have access to all the internals or don't I? And the difference is usually that with um well you don't have access to the internals much more trial and error is involved um so let's say that i don't know i have a system that i need to um in in the lab we would get a system and it's like okay here's the here's a program to exercise the AES core.

Here you can program the key. Please do your lab analysis and then get back to us on how many side channel traces it takes to get the key out. Somebody who doesn't have that kind of access may actually have to figure out, okay, so how do I even exercise this AES core? What is the protocol like? So these are all things that in security we consider like something that you can overcome. It's not a secret per se, but it takes some effort to get through. So that's the main difference between sort of the lab and reality is just much more guesswork and learn about physical tax without spending a large amount of money on lab equipment?

Yeah, so I think the least expensive option is to look at our books VM. So we actually, that's accessible to anybody, even who doesn't have the book. So if you go to the book's website, which is hardwarehacking.io, you can download the VM and you can sort of in simulation start practicing some of these techniques.

After that, I think Chip Whisperer is probably your best option. Yeah. And yeah, after that, I think Chip Whisperer is probably your best option. And yeah, after that, you start getting into the type of equipment that we sell at Riskure, which is more the professional grade stuff. Although one of the cheap sub $10, sub $20 logic analyzers will do you for a bit. Those are pretty fun. Oh, absolutely. So my brain automatically goes to side channels and faults.

And I know some of my colleagues have also done presentations of like sub $50 attacks, I think. So you cannot do this with a chip whisperer, but there's some other like cheap oscilloscopes you can buy and um um yeah i'll just google that like um i think it's called cheapskate or something where sca is in what is somewhere in the skates of cheapskate in the book there wasn't a chapter about ethics or responsibilities if your book comes up as evidence in a court case where people were harmed, will it bother you? Yeah, definitely. Obviously, when you write things about security and hacking, you know that these can be used for nefarious purposes, but that doesn't make you feel good necessarily i think um but i still would publish the book because i still believe that bottom line this book will do more good than it will do harm um you know people get killed in car accidents every day

we're still driving cars because we think that the risk trade-off is worth it and i think this is something you'll see commonly in in in in security right so the idea is if we don't publish things it doesn't mean that the vulnerabilities aren't there right it's just going to be harder to know about and therefore harder to fix so I think by that reasoning yeah the goal is really to get the information out there so that as humanity we can progress beyond it and have a discussion on, yeah, really, how can we address these things? So in that sense, it's not, hardware security isn't that different from software security

in the sense that we want to be open about it. There's one sort of caveat there, and that is that patching hardware is kind of hard so you know let's say you have a pacemaker you know built into your chest and there's a vulnerability in there now what do you do um i'm not going to pretend that I have the answer here. I think what's important to know is, well, I guess what's really important is to at least be able to patch the firmware, right? Because a pacemaker is built into your chest. So doing side channel analysis or fault injection is going to be hard in the first place. So if you can patch the firmware,

then at least you can sort of protect that remote attack surface that's on there. But I find with hardware, I know software patching is already difficult, but hardware patching is pretty much impossible. So you end up in a lot of these scenarios that can go two ways. I know one of my friends here in the Bay Area,

he runs a company that does cryptocurrency recovery. So what he'll do is he'll, or his company is take a wallet, a hardware wallet, for instance, actually break it and then give the seeds to the person who accidentally forgot their pin and locked themselves out of their retirement fund. And, you know, is this, um, I think it's good that he's able to return this money to the person with the retirement fund. But he's also relying on the fact that he needs to break hardware in order to do so. Should those hacks be published, yes or no? I don't know.

It's a good question um so yeah it i don't see like in software there's sort of this um i wouldn't say industry standard but there's some companies like google project zero they say you know we get a 90 days disclosure window and we inform the vendor and they have 90 days to patch. I don't think there's something like that for hardware. I haven't seen it at least. Probably because of this complexity of patching. It just makes it a harder question to answer. Patching software is probably the largest attack surface that embedded systems have. If you can't update how to patch hardware,

but something that should make it more difficult to attack actually makes it kind of easier to attack? Yeah, that's a good question. Maybe before we get into that, I think still firmer updates are really important. I think that's something that we should have. I wasn't saying it shouldn't, just that it is one of those areas that is dangerous. Yeah, agreed.

You need to get that part right. I think the nice thing is once you get that part right, it's not as bad to get other parts wrong. Yeah, so hardware is not updatable. It's not 100% true. So there's some things called ROM patches,

which are usually hard to apply in the field. But basically what manufacturers do is inside the ROM code, they have a couple of hooks or parts of code that basically say, hey, look up in this table that we can patch later, for instance, in eFuses or something, if there's a patch and then jump into that patch code. So there's some things that you can patch, but it's limited.

And generally, you don't want to mess with ROMs when they're already in the field, so after manufacturing. So let's forget about that case before, because it doesn't always even prevent false injection or side channel analysis attacks. So the question was, is there anything in hardware where a security feature makes it easier to attack? I've seen that where, what I mentioned before is when you look at fault injection, we can have these countermeasures. And one of the countermeasure strategies is redundancy. So instead of doing a password check once,

I'm going to do it 10 times, and then I'm going to make sure that it passes every time. Because now an attacker needs to attack 10 times instead of one, which is much harder. What can happen is that now, because you're doing things multiple times, you actually start to leak more information. So your fault injection countermeasure is actually amplifying your side channel leakage. Um, and that's not something make another technique easier to apply.

Going back to ROM, many chips these days have readout protection, which seems great. You know, nobody can read my code. But as Nathan pointed out, a listener, these always seem to be broken in some way. We had a Jess from Oxide here talking about an ST vulnerability. Why is that hard? That doesn't seem to be hard. Security is hard. We can get into the philosophy of that. But I think, first of all, being broken is a term that I don't like to use in a black and white sense, because in the end, everything can be broken. Just the question is, how much effort do you need to get there?

And once you've learned how to get there, how easy is it to repeat on another device? So with logical vulnerabilities, usually it's fine to take some effort. Maybe it's hard to read out the code in the first place, but you figure out a way around it. Once you've done that, you can just repeat it on all devices.

That's not something that always holds for things like side channel. Let's say that each device has a unique key. Maybe I can sort of tune a system to do side channel analysis, but if it takes a million traces to get the key out, I have to take a million traces for every device. So it doesn't scale as well.

So that's why I always like to think of broken in terms of how broken or how hard is it to get somewhere. Specifically about readout protection, from the logical side, I mean, I agree with you that it shouldn't be hard, right? The bugs are being introduced. That happens. Hopefully, those will sort of filter out over the years. Where it gets harder is things like fault injection. So readout protection is an access control mechanism. Usually somewhere in the chip, it boils down to a single bit that says access yes or no.

And if I can flip that with fault injection, then I can regain access. And that one's harder because now I need to start thinking about, you know, how do I add redundancy to that? It doesn't become a logical, like I have a bug problem anymore, but it's, you know, again, an inherent system property that I need to create countermeasures around. You had some questions for us. Yeah, I'm curious. You wrote a book as well, Alicia, on embedded system design.

So what motivated you to start writing the book? And what motivated you to keep going once you were halfway in there? The motivation to start was partially a level of ridiculousness. Well, I wanted this book. I had been teaching people on a small scale these things. And when I was talking to someone else who was in a similar position, we were kind of talking about what we would want this book, this mythical book, that if only we could find it, we would not have to say the same things over and over again.

And then he turned to me and said, you should write that. And it was so bizarre of an idea that I actually pitched it to O'Reilly and they said yes. As for motivated to go on, that was difficult. It was a tough time in my life. I had gotten very sick and my mom had died. And then I was just kind of homebound and not willing to do anything that didn't involve losing myself in writing or working. And even work was pretty monotonous. So I was motivated because I didn't want to leave the house anyway.

And O'Reilly, my editor, was pretty good about keeping me on task. You went through no starch. They're a little looser with the editing, right? Oh, yeah, definitely. the editing right oh yeah definitely i think um um well after year six i think um bill them the owner of nsp was like well we should maybe get this book out guys and call and i was like yeah yeah i think i think you're right he was the one who introduced us to the concept of there can actually be a v2 right this doesn't have to be perfect. It's good enough. Just get it out. But now as I'm looking at a V2, it seems so daunting. Like, how did I put all of

this information together? So I totally see how it could take me seven years to make a V2. Well, let me know how it goes and we might consider it too. Yeah, maybe a question for Chris then. What's the coolest compliment you've had about work that you've done, either about the podcast or I know you're into music or professionally?

Coolest compliment? Oh, wow. Jeez. See, I just take compliments and I don't file them anywhere. But you give them a criticism. Yeah, maybe where you've had been helped by the device. The company I was at, we did, I've talked about it before, we did imaging that allowed surgeons to repair arteries that had blockages in people's legs was the first application. And it's usually a symptom of diabetes where you get peripheral artery disease and don't want to trigger anyone, but your legs start having problems because they can't get enough blood flow.

And it's hard to fix. So we had a device that made that easier to fix. And, you know, we heard from lots of doctors and patients that, you know, we'd save their legs. So that was always something that made me feel good because in a lot of other companies, it was like, I'm making this thing and it's, you know, very down the line. And who knows if the impact is positive or negative overall. But this was just something that was like, oh, okay. You know, something I did has helped some number of people improve their lives. And that was very gratifying. Yeah, nice. I can recognize that.

Or, I mean, I can understand that, I should say. I think with security, it can be difficult sometimes to, just like what you said, like we find a bunch of vulnerabilities. I'm in the lucky space where at least the riskier customers that take security very seriously. So the things we find, you know, usually get fixed. But it's such a, it's almost like the insurance world right it's it's like we actually didn't change any of the functional properties of the system we just helped some accident from not happening um it's it's a bit more virtual and i know that some people in the security industry struggle with that it's it's tough to find

applications that are you know that you can point to and say directly, this is a net positive good for the world. And I know it, I can see it right now. And it's been rare in my career. And I think Alicia has had, it's been less rare in her career because she actively goes out and seeks those things. But it's still hard. It's still hard because sometimes you go out and seek those things and it turns out to be a bit of of a red herring i've had that with other medical device companies where it's like oh yeah we're gonna help people do this and it turns out it's not really all that cool or it's something else entirely so yeah i have to say and that's one thing i did like about having the book published is like all the positive responses.

Yeah, that's something that when you're doing security projects for professional companies, I mean, they thank you and they're happy with the result, but it's much less tangible than this person who is like, oh oh now i understand how this hardware security stuff works right it's more gratifying feedback i guess yeah that's the educator high too right that that's that's that's one that i've had my i know at least yes hadn't worked you teach somebody something they say oh i understand this now and i i didn't before. Thank you. It's like, oh, okay. Put a little bit of light into the world. Yeah, exactly. I wanted to ask you a couple of kind of forward-looking questions.

When the side channel tech started making a lot of news a couple of years ago, four years ago, three years ago, I don't know. Time is a flat circle. I paid some attention to it, and then I stopped paying attention to it because it didn't really affect me other than as a curiosity. Do you have a sense for what the state of that is now? Have we taken a big performance hit on CPUs and things because we've turned off a lot of speculative kinds of things. The speculative execution? Yes, I'm talking about speculative execution, not side channel attacks.

Yeah, when you gave the timeline, I thought you were talking about speculative execution. They both have an S. Yeah. So have you been following that? And do you have a sense for where we have those things been kind of addressed in future designs? Or is it kind of like, no, we take the 10% performance hit and we just don't do those things anymore. When I, I still remember the first time I was reading about this. The speculative execution vulnerabilities are basically a class of timing side channels.

And, you know, I've been dealing with timing side channels for a long time so that was what sort of my peak my interest and i thought oh crap um what they're actually exploiting is some hardware optimization. And I was thinking, well, this is a hardware optimization. How many hardware optimizations are there in a chip? I mean, that's what chips are. It's just 20, 30, 40 years of optimizations stacked on top of each other. And sort of unfortunately, that's how it panned out right over the last few years it's it's one after the other it's like okay let's look at the uh tlb site okay oh we

got problems there too let's look at the branch buffers oh we got problems there too and it's this is not a surprise right because all of all of these things, they optimize for particular cases. And if, you know, quote unquote, particular case happens to have some information, that's interesting to be extracted. That can happen. So with that said, I have to say I'm not an expert on speculative execution.

So I am, I guess going back to my former comment, I don't know. I do think it is not relevant in all use cases. So in some use cases where you don't have multi-tenancy on a system, it may actually be really hard to exploit some of these things. And more in general, I see that with more security bugs in CPUs. They don't apply in all the situations. And then you can make sort of these straight-off choices of, do we just disable feature X,

or do we only disable feature X in situations Y and Z and sort of keep the performance in the other places? And, yeah, I think that is perhaps a direction we need to go to where, you know, maybe it's safe by default and maybe in an HPC data center where, you know, all the users are trusted, we turn these things off so we can get some extra performance. But yeah, I don't know exactly which direction the mitigations are going. What I do know is, yeah, like I started off, this is kind of a fundamental problem where we're exploiting optimizations and optimizations is what makes our CPUs fast and not too hot.

Is it a losing battle to be a defender? I mean, some of these techniques, if you go back to the most secure thing ever in 1995, it's trivial to break now. Is it always just a matter of I can only secure it for a little while, if that? Yeah, I think. This also could be framed as why do you have the easy job? That's a personality question. Yeah, let's stick to the technology side.

I think, well, I just like hacking stuff. That's the personality side. Though I think in the most recent years, I've been shifting more and more. I'm turning from red into blue, I guess. I'm becoming more and more of a smurf because I do see that, you know, I don't want to be done when I say,

hey, look, this is broken. You know, you know, the, I don't want to be done when I say, Hey, look, this is broken, you know, you go fix it. Um, I want to be part of the, you know, how, how to fix things as well. Um, but yeah, I think what we have to watch out for is sort of the security nihilism of, um, well, it's all broken, you know, let's just give up. It doesn't matter, which I completely disagree with. I mean, if we look outside, my bank account hasn't been plundered over the last years. I mean, there's a little bit of fraud on my card every now and then, but I call the bank that that's solved.

You know, Tesla cars aren't all being hacked and driven to remote areas of the country so if you sort of zoom out it's not quite that bad doesn't my my earlier statement of everything can be hacked that's still true right but there also needs to be an incentive for people to hack things and it needs to be at the cost where they're willing to spend that. We cannot make the risk go to zero. But if we spend sort of our security defense budget in the right areas, I think we can do quite well as a society. And I think you're discounting a little bit, you know, efforts of people like you who are trying to get ahead of some of these things and you know there's a constant cat and mouse kind of thing and as long

as as long as there's a lot of effort on the secure side it's going to mitigate a lot of the stuff oh definitely like like obviously i i believe that even though we're maybe not fundamentally solving problems we're at least incrementally solving some of these security challenges that we have. And the cat and mouse game is an integral part of that, definitely. Reading through your book, I did not get a sense of how I can make my systems more secure. But that is because most of my systems are not worth that level of security. None of them play a part where, you know, they're not controlling nuclear sites.

Yeah. And so I don't, if somebody side channels one of my children's toys, that's totally fine with me. That seems like a great place to try it out. Yeah, definitely go ahead and do that, right? I think we jokingly had in one of the first chapters, we're doing a threat modeling on a toothbrush. And I think threat modeling is actually the key word here, right? Because you're saying it really doesn't matter if somebody does side channel on my on my toys that means you've thought

about it which is you know a fancy way of saying thinking about security is threat modeling which involves thinking about you know what are the things we actually want to protect against who are we protecting this against what kind of attacks and what i said earlier you want to you know you want to spend your security budget there where it makes sense and you know toys side channel you know go ahead there's um there's there's no need to to do that um if you're you know launching nukes then I hope there's more than just a few side channel countermeasures in there. I hope there's layers and layers of mitigations around that. Well, until a few years ago, they were still using floppy disks.

So I think that's a protection in and of itself. Yeah, perfect. Please, please keep it that way. I have one more listener question from James, which kind of fits here. How did we fall into the trap of thinking we could control the hardware or software after we sell it to a customer? Who's we? How did we as engineers believe that we could control the hardware and software after the marketing team sells it to a customer? Sure.

That's almost a question for you two. I never believed that. Okay. I mean... I mean, go ahead. There's like the tractors. Oh, okay. Sure. They don't want people...

They want to be able to charge more money, and they say you can't fix your own tractor. On one hand, I understand why they want to do that, not only for the money part, but also for the security part. On the other hand, once I buy something, it's mine. Well, yes, yes, yes. And you shouldn't be able to tell me I can't open it. Depends on the definition of buy. Well, with the tractors, they tried to change that definition, and that was not great.

But, I mean, as an engineer, I definitely fall into this. It's like, okay, well, if something breaks here, we'll fix it. Or if something isn't good here, we'll fix it. But that's because I'm such a fixer. I don't usually think about, oh, well, my evil company is now going to sell the data to more evil companies. And they're going to use my ability to update firmware to find out more data and badness, badness, badness. I can see both sides of, should we control or should we not control? And how do we tell the customer what they're in charge of versus what we are? They're in charge of updating the firmware because they don't always, and then they wonder why they get hacked. I think it's a complex question because it depends on what control means.

It depends on the definitions of a lot of these things. Oh, yeah. So, I mean, the first time I came across this was we had a dumb medical device with somebody had a dumb idea to sell a consumable. And we put authentication chips in the consumable. And, you know, within two months of selling the thing, somebody had hacked it and was reprogramming the consumables to extend their life and stuff. And we never figured out how they did it. But your consumables were cost hundreds of dollars and were worth dollars. They were worth, they cost us 20 or 30 and we charged to 400, I think. Yeah. Yeah. So there was an incentive.

Oh yeah, there was a huge incentive. And I knew that from the beginning. And as I was implementing, I was like, well, this is going to be a problem because even though I cannot see how to break this, somebody is going to figure it out. And lo and behold, they did. And I never figured, well, this is going to be a problem because even though I cannot see how to break this, somebody's going to figure it out. And lo and behold, they did. And I never figured out how and we kept making countermeasures and stuff. But even in that situation, I was like, well, this is never going to work. Somebody's going to figure it out because they have physical access. And once you have physical access, unless you've got a team of, you know, commandos coming into your customer's office to haul them off.

All bets are off. Yeah, and I think, especially in lower-cost devices, you're not going to protect against physical attacks. I think what you get with, for instance, firmware updates, it really is a double-edged sword, right? On the one hand, you want to make sure that your device is running proper software, especially if it's like safety-critical devices, right? You don't want somebody to just upload another firmware that's going to cause who knows what, right?

But at the same time, you are locking people out of the devices that they're bought like that's that's a total double-edged sword um and i think that really depends on from device device on how you want to make that uh make that choice um going back to the question i mean i talk to a lot of developers, obviously, when I'm doing my work, and I think the majority of them, they do realize that when something is in the field, people can poke at it. I think there is a minority that will sometimes have discussions of like, you know, hey, you have this function sitting in your API

that you're actually not, you know, you say it's deprecated. And then usually the reply is, yeah, we're not using that. And then I'm saying, well, but an attacker can use it. It's there. It's still there. And I think that sort of illustrates a way of thinking about how a device is being used and not always realizing that if you put something in there that may not be the intended use, it can still be used.

And that's maybe the difference between more of a developer mindset of how should this be used versus the attacker mindset of how could this be used versus the attacker mindset of how could this be used. Riskure, that's your company. What do you do? So I'm the CTO for Riskure North America. And that's a fancy way of saying that I do two main things. So one is when our technical teams work with customers and they have some really difficult questions, then they escalate to me every now and then. So I'll join the teams and help them out.

And the second part of it is looking forward. Like what are the, you know, on the innovation side, what are the things we want to put into the market in the next one, two, three years? So those are sort of the two main responsibilities I have. And you're hiring? Always. Yeah, we're always looking for people who are interested in security. Don't necessarily need to have a hardware security background. You've got a book to train them.

We've got a book to train them. We've got courses to train them as well. And there's quite often a software component, firmware component to what we do as well. So a lot of people come in sort of starting on that side and on the job to learn more of the hardware-y stuff. And I have one last question. Well, I have two last questions for you. The first probably is a whole podcast on its own,

but if you had to get an implanted medical device, a pacemaker or a pancreas, how much more would you be willing to spend if you could look at the firmware and possibly patch it yourself? That would be a whole podcast in itself. But to summarize, I would, oh God, I think that would be very valuable to me. Um, I think mostly on the analysis side, I would want to see, you know, what it does and whether it could be attacked in some way. I am not comfortable enough myself to start loading code into my own body and parts that might be harmful to my health. I would like to have trained engineers to do that part. But yeah, I'd pay for that for sure.

I mean, you never see Darth Vader push any of those buttons on his chest thing. You've got to figure he's got people to do that who know what they're doing. Exactly. Well, Jesper, do you have any thoughts you'd like to leave us with? Yeah, I would say it was great to be on the show

thank you for having me and I also really appreciate you guys for doing this podcast engineering is an important part of progress as society, embedded systems are an increasingly large part of that so keep on

keeping on thank you our guest has been Jasper von Wudenberg, CTO of Rescure North America and author, co-author of the Hardware Hacking Handbook. Thanks, Jasper. Yeah, thanks so much. Thank you to Christopher for producing and co-hosting.

Thank you to our Patreon listeners Slack group for questions. And thank you for listening. You can always contact us at showandembedded.fm or hit the contact link on Embedded FM. Now a quote to leave you with. Let's go with Cory Doctorow. Never underestimate the determination of a kid who is time rich and cash poor.