Embedded - 299: Reasonably Foreseeable Misuse

Episode Date: August 15, 2019

Monk Eastman (@MonkFunkster) joined us for an enlightening conversation about hardware compliance engineering. We covered the basics of CE, FCC, UL, and battery certification.  We mentioned that Alan Cohen’s Prototype to Product: A Practical Guide for Getting to Market has a good overview of certification. Alan was on Embedded 269: Ultra-Precise Death Ray. For a deeper view of compliance engineer, Monk suggested this book: Electrical Product Compliance and Safety Engineering. Listener Skippy wrote about his experience with CE certification. Monk plays bass saxophone in the East Bay Brass Band. Details on registering for the Embedded 300 party on Eventbrite.com are in the show itself.

Welcome to Embedded. I am Eliseo White, and I am here with Christopher White. Remember when we got that email asking about UL, CE, and FCC testing, and we had to shrug our shoulders and admit our ignorance? We're fixing that. Monk Eastman is going to help us figure out the maze of certifications. Hi, Monk. Welcome. Hi, thanks for having me.

Could you tell us about yourself as if we met at one of these, oh my God, I have to certify my product one day course trainings? Sure. Monk Eastman is my name, and I've been an engineer of many stripes for many years now. I got my start doing design engineering and then got more and more into the various ancillary bits of actually making a product. So compliance, for example, what we're talking about today, and also I'm deep in the manufacturing and new product introduction supply chain world. So I now solve the various other non-design issues of actually bringing a product to market. What is your background?

I have a degree in electrical engineering and a master's in engineering management. And then I spent a number of years as an electrical designer, starting in medical, did some industrial controls. A long time in electric vehicles and battery management systems. So mostly power, DC power, electronic design, some analog systems, some motor control, handful of odds and ends here and there. Cool. Do you want to do lightning round?

Let's do it. All right. Favorite ISO number and why is it 60601? So 60601 is not my favorite, though it's a good one. I'm actually partial to 1345, which is the standard for quality systems and medical device development. So it's making sure you have all your documentation in a row to actually build medical devices. I remember that one.

Yeah. I'm a little surprised by the answer. Maybe I'll ask more later. Have you ever licked a nine volt battery? Many times. Favorite acronym. Hmm. That's a good question,

actually. I'm just going to say IEC since we're talking standards these days. What does that stand for? IEC is the, it's in Swiss French, it's the Internationale Electrotechnica Commission or something like that. It's the Swiss standards body that writes a lot of standards that gets used for things. I have some true false questions about batteries. Are you ready? Let's do it. Samsung is stealing data through microchips on its mobile phone batteries. I'm going to say false.

Snopes agrees with you. A video shows popcorn being popped via the use of a pickle, a AA battery, and iphone charger cord suppose it's plausible so i'll say true snopes does not agree with you ah well man-made reactive diamond batteries are capable of producing thousands of years of energy without a chart i'm dubious of that one so i'm going to say false. They say iffy. The concept is valid, but they followed up with, at this point, a theoretical idea and one that comes with a myriad of challenges not discussed in viral stories.

I don't know. Snopes has gone downhill. Is there a key command that you can do on a standard cell phone to increase their battery life? Such as star pound four seven two zero pound. I'm going to do that right now. I don't know about that one, but there's usually like a power button on the side that you can hit, and that will definitely extend your battery life. I did it, and my phone is getting very hot. Is that normal? Right. The power button is the way to go.

Okay. When is the Embedded 300 party? I believe that's coming up the weekend after Labor Day weekend. September 7th. Are you going to be there? That's my plan. Cat or hack? Hack. Okay. So for those of you who are scratching their heads wondering what that's about, we are having a party on September 7th in the afternoon in Aptos near Santa Cruz. And we are looking forward to talking to listeners because we've always found them super interesting and thanking them for being listeners and introducing you all to each other because that's really fun because we can only talk so much well that too well i think somebody in the slack challenged me to a real-time turing test at the event so now i have to show up and there will be a very large field there will be building uh food, drinks, all of the super fun things. You should bring a hack

or you should bring something cat themed. You can bring a cat, but I don't recommend it. And if you want to come, embedded300.eventbrite.com. That's embedded300.eventbrite.com. It's not in the show notes, so you have to remember it. Okay. Now let's get down to the good stuff. It all started with an email from Happy Day wanting a discussion to walk through taking hardware through UL testing. And what are the steps? What should it cost? What can we do before submission to

optimize acceptance? And as a software engineer, my answer to that is, will you write some manufacturing test software that puts it in all its modes and makes it squawk? There was a last question, too. Why? Oh, yes. That's an important two. Well, I just assumed it's because most product designers are intrinsically masochistic, but I think there are some other reasons as well. Well, and it's important for hobbyists to know when and why that's important. Absolutely. To, I guess, hit the last question first, the why is to sell your product at a fundamental level or to import it somewhere. There are various agencies in various countries

that want to make sure your product is not going to, for example, have a thermal event, as they like to say in the battery business. Small explosion? Minor explosions, yeah. Thermal event. So the FCC is the one that everyone knows about, of course, and most people know UL as well. Most countries have at the point of importation, a requirement that your product is

meeting various standards and requirements. So in order to be able to even bring your product into a country and sell it there, you have to have all that market access work done. And so that's the core of it. Why do you need it? You need it for access to those markets. That's an odd way to put it. I mean, I guess I would have gone not that applicable. I would have been like to prove that your product is safe,

but it really isn't about proving your product is safe. That's almost an impossible task. It's about proving it is safe enough for these markets to accept. It is. And most of these companies, you know, UL or Intertech or whatnot, will have a market access group. And their specialty is essentially in knowing which standards are needed for which countries and to help you navigate. Especially if you're trying to sell a global product, you need to abide by a bunch of different, not always consistent rules for different countries. Yeah, because there's FCC, but is that the same as CE in Europe and CI in Canada, or is it different? The short answer is no, but they're not dramatically different. I believe there are slight differences in the test setups and the actual requirements of what is not a pass on their testing. So the good news, bad news there. The good news is if you pass one, it's relatively easy to pass the others. The bad news is that you still have to do those separately. How much does it cost? I mean, if I'm just like thinking about this as a entrepreneur,

and I know I need to put this on my radar, how much does it cost like at a minimum or at a usual just to get the testing done? Sure. So the testing itself, well, just for FCC, it's usually about five-ish thousand dollars, depending on how much testing time you need and how many retests you need and things like that. But five-ish thousand is a reasonable ballpark. I should say the answer to all of these is it depends, of course, right? Because costs are all over the place in a lot of these things. Running a battery through the UN38 testing is probably about $7,000, depending on how many times your battery has a thermal event during testing.

The UL testing can get pretty expensive pretty fast. That is almost certainly going to be at least five figures, so probably in the $20,000 to $40 40K-ish range, depending on the complexity of your product and what you need to test. So that's kind of a wide range there. But yeah, all the testing itself usually runs at the low-end $5,000-ish to at the high-end $50,000-ish. There's really not somebody marketing something on Tindy necessarily is going to go through. That would be all their profits for an entire product, potentially. Right. But if your volumes are sufficient that you're selling in other countries and things like that, then that becomes an issue, right?

Yeah, especially at the importation level, because that's where a lot of the paperwork actually gets checked. Well, actually, 10D is a good example because how do people sell on 10D without getting certification? Might be because they're not complete products, like they're modules and things. I don't know. Certainly, if you're selling a kit or something like that, yeah, you can kind of sidestep a lot of this I don't. at the tail end of a data sheet for a DC-to-DC buck converter or other random ICs.

They will warn you heavily that it is not intended for certain applications and that the product itself doesn't come with any certifications or guarantees. But that doesn't mean it can't be incorporated into a later certified product, right? Right, but it just means that all the burden of testing is on you. Gotcha. And that makes sense. I mean, the idea is you test your whole product and you do it as a unit, but because it's so expensive, you really don't want to test it twice. Right. And certain devices and certain parts and things like that that fall in the category of safety critical, for example, are often certified independently.

So if you get a relay for your board that is designed to disconnect AC power, that relay probably has a UL listing and certification to go along with that already. So certain categories of parts do come essentially certified already, and that will help you out a lot later on, too. That makes sense. As a consumer, what should I be looking for? Should I be looking for the CE mark? I know it comes scale products will have it because it covers a pretty large market. There's a whole boatload of marks that often occupy device label real estate, which is usually the designer's worst dream because they hate the aesthetic of those logos. But it's, you know, anything from an FCC mark to a UL mark to a CE mark, CSA if you're going into Canada, CCC if you're selling to China, right? Every country has its own thing. So there's usually somewhere on your product, a label, which includes various bits of information, including these marks. So you should be looking for the FCC mark. You should be looking for the FCC mark. You should

be looking for the CE mark. Those are definitely good indicators that someone has done their homework, done the work necessary to make sure the product is safe. And those are both mostly about frequencies and emissions, aren't they? Or do they also deal with power? So the FCC mark and the FCC standard is all about RF interference and things in that realm. It includes both not transmitting harmful interference and not being receptive to harmful interference. So that's a pretty limited scope. CE covers a lot of territory depending on what product you're making. The essential mark is that you have met all of the applicable directives that the EU has passed related to your product.

Oh, wow. That's the low voltage directive or the EMC directive, or if you're making medical equipment, some of the medical device directives. But the CE mark covers both EMC and product safety. And what about the UL mark? UL is predominantly focused on product safety. They have some EMC-related standards, but it's sort of tangential to the work that the FCC does. So the UL standards are mostly designed to not burn your house down. Yeah. So I see UL marks on lamps. I mean, that was where I was like,

oh, this company is advertising they have a UL mark. That might be something I should look into to see what my other options are. Yeah. And most retail channels, at least mainstream retail channels in the U.S., will not sell your product if it plugs into the wall and does not have a UL mark. So, you know, Home Depot is not going to carry your power tools if it's not UL. But Etsy will. Well, sure. But Target sells forks? I don't think.

Let's go to the wall. Yeah, I mean, not by design, right? Oh, okay. Yeah. Though that actually brings up a good point. A lot of these standards are, you are required to examine a reasonably foreseeable misuse. Eh, that's the show title.

Why are there so many different standards and don't i mean yes that that is that is the xkcd dumb question of the day but for a lot of other things there's there's there have been efforts no matter how successful or unsuccessful to standardize and harmonize um right and you see that across automotive for instance it's very hard's very hard to have a whole bunch of standards for automotive. So there's a couple. But it seems like with UL, CE, FCC, CSA, all the other ones, some of these are not even government bodies. UL is a company, right? Right.

How does that happen? And how do they gain authority just being these people? Are they like consumer reports gone mad? I don't understand how that happens. That's more or less it. The history of UL is kind of a fascinating one in that it was founded by a group of insurance companies well over 100 years ago, who were, they were getting sick of essentially paying out too much for houses that were burning down because of shoddy electrical wiring and shoddy products. So they formed an organization to improve testing, improve product safety and things like that, because they just didn't want to pay out as much money. And so that was the history of UL there. And then it kind of became the de facto

standard, right? If you want to insure a thing, then you have to make sure that thing is UL certified. So while your premium may go down because it's UL certified, the product designer is paying more money to UL to get it certified. So the insurance companies are making their money one way or another. But it's all about essentially managing insurance and managing risk. So that makes sense to me. But also the ones that are government-backed, like FCC. Yeah. I mean, that covers a different range of things. Right. Well, the FCC is there to make sure that your device is not interfering in a negative way with your neighbor's device.

So they want to make sure you're not bursting out a whole bunch of noise in the 50 megahertz band or something like that. That's their objective, is they want everything that emits RF to play nicely together. FCC doesn't want people's reality television interrupted. You know, priorities. And then there's one we haven't really mentioned, UN38. Yeah, that's a fun one. So this is actually a transport standard. And this is what you need if you want to get anybody to ship your lithium ion battery.

It's purely just a transport standard, right? So if you want somebody to throw your product with its lithium battery on their cargo boat or on their cargo plane, you need to have this. That's weird. I mean, my shipper cares what my battery is. I mean, I guess if they don't like, you know, their transport's exploding. Yeah, they're trying to prevent your cargo from catching all of the rest of their cargo on fire, which is, again, probably coming back to an insurance thing, right? Okay. This is, I mean, this is alphabet soup. How do I know, how do I even get an idea of what I need to do? So UL and Intertech and pretty much everyone else in this game usually has a separate division of their company that does compliance services.

How convenient. Sorry. so you have to you know you engage with different sides of the organization for design services versus actual testing and reporting which also has the the bonus of meaning you have to hire them twice now when i did medical we usually used a company called tv and they were they were both the testing body and something called a reporting body yes and if i recall correctly they'd handle like all of this stuff And they were both the testing body and something called a reporting body. And if I recall correctly, they'd handle all of this stuff. They would, yeah.

So yeah, I think it's called a notified body in CE parlance. Yeah, notified body. Yeah. And TUV, actually, TUV has many different organizations now. So TUV North America is an independent organization from TUV Europe. And I think they have a TUV Japan or something like that. I don't know. They have a few other international locations.

But yeah, so for CE, certain directives, I think the medical device directive is one of them and a handful of other safety critical directives require you to do all your testing and documentation through what's called a notified body, TUV being an example of this. And that means that somebody that has been approved by the European Union to vet all of your stuff is doing so and checking all of your paperwork and checking your product and going through the actual requirements for you. It's usually more expensive, but it saves you some headache as well. Because they actually know the T's to cross and the I's to dot. Absolutely.

And they're also the ones who say whether or not you pass, right? I'm getting the idea this is kind of like smog checks, where if you take your car in and you have to get the piece of paper, and then they automatically send it to the state. And if you don't pass, maybe you get to go try again for free. And it's all kind of a game. There's a certain element of that. And a lot of regulatory, right, is making sure you have the right paperwork. If you get into especially to the ROHA side of the European Union regulations, which is the reduction of hazardous substances, that's a paperwork game. That's logistics, right? You have to have all of your components tested and the backing documentation to show they don't contain toxic amounts of lead or something along those lines.

But when you are shipping your product to Europe, you just need the paperwork that goes along with everything that's in your product to show that you have not used any components that are not already compliant. Yeah, that's just a mountain of paperwork. Boring, tedious paperwork. I mean, regulatory certification, this whole world is piles and piles and piles of dead trees. With all of these different tests and having companies that can tell you which ones you need, as a small company, how do I make it cheaper? How do I get in there for as little as I can? Are there ways to do testing at my site before going to a testing center?

How do I just make this simpler? Or do we need to break it up by which one we're going after? There's some of that. So you are, as a manufacturer, you are able to legally find essentially setting up your own research and determining what standards apply to your product. I should probably at this point say, I am not a lawyer and none of this actually essentially setting up your own research and determining what standards apply to your product, I should probably at this point say, I am not a lawyer and none of this actually constitutes legal advice. So this is, this is, you know, generic interpretation of standards and things like that. But yes, you, you are at liberty to, to determine your own products requirements. You know, you, you need to just,

especially for CE, right? Which CE is a self-certification scheme for the most part, some directives accepted. But you are able to determine that on your own if you want. So one of the cheapest things, right, is to just do some research,

understand, you know, maybe you have some people that you can talk to in the industry who have gone through the process with a product like yours before and know the specific things you need. But yeah, you can definitely do that research on your own. You can define then your product requirements based on those standards. The unfortunate part is that you can't, with a few exceptions, you can't just go and get those standards for free.

So that's an interesting game as well. Ah, yes, you have to buy the standards so you know you can pass them. Right. Yeah, like MISRA, where you can't possibly know what errors you're getting in IAR because unless you have the standard, it's just numbers. Right. Similarly, yeah, the IEC, UL, most of these organizations charge you for their standards. The one exception to this I found is actually Etsy, which is the European Telecommunications Something Institute. The S is not something, the S is something else. But yeah, they actually publish their standards online for free, which is pretty cool. Does the FCC have it for free, which is pretty cool. Does the FCC have it for free?

Yeah, actually, because, well, sort of. Depends how much you feel like digging through legal code. The FCC requirements are all outlined in Part 15 of Chapter 47 of the Code of Federal Regulations. So if you feel like digging through all of that, then yes, you can look those up. But because UL is a company, they're going to sell their standards. And then they're going to sell their testing. And then they're going to sell their report. It's a pretty sweet gig, you know? I mean, you're making money a few times over for the same thing.

That's so silly. Okay. So if I want, so I can help a company certify something for cheaper by helping them identify what they need to actually certify that. And one way to do that is to look at the competitors, which I mean, in medical and FAA, that was always a good trick, too. What certification level are they? Because we're about like them. Right. And probably their FDA filings are all eventually public anyways, right? So you can go dig through those. But what about hardware and software? Are there things I should know?

Yeah. So going along with buying the standard, right, is that you should be able to then use those requirements early on in your product development process to make sure you're actually building something that will pass those tests. That's the biggest cost savings you can do right there is to pass the first time, right? So the more you can front load that design work and actually incorporate everything you need early on, the cheaper it's going to be in the long run. You mean we can't just tack this on at the end? I would avoid that. My eyes rolled so hard my head hurt. Right. I play the same game in the manufacturing world as well. You know, it's like, oh, we can just do DFM at the end, right? But the sooner you engage on these topics,

the easier your life will be. That's important because there's a lot of considerations to make there, right? Some of them are designed for test kind of things, like, okay, maybe I'm making a medical device and it's got safety monitoring features and maybe the inspector is going to want to probe the system while it's running in various places if you don't have those test points that makes it more difficult for them and uh that can make it more difficult for you um absolutely well

i think i think on this uh firmware engineers and manufacturing engineers are aligned more test points just everywhere yeah all the test points and then on radiated that seems like it could be a major problem right because you might develop something and it it just it's it's a radio station at the end and unintentionally right it's kind of hard to you know you can tack on shielding in various places but sometimes that impacts industrial design and other things so So how can people kind of monitor that as they go and not get in trouble? Yeah, for FCC and other EMC testing especially, it's often recommended you do various pre-scans. And you can either set up your own facilities to do that, but the gear is kind of spendy. Or you can contract with any of a number of labs that will also happily do the full testing for

you later on. But prior to doing the full testing, you can book essentially unused time in their chamber to test your product. And depending on what hours you're willing to work and what your schedule needs are, that can be reasonably affordable somewhere in the ballpark of like $300 an hour. The trick I found with that especially is if you're willing to work a second shift or a Saturday, you can get much better access to lab time for cheaper. But that's on you, the design engineer. So you can definitely pre-scan and I recommend doing that early and often as soon as you have your first prototypes, right? Get in the lab, spend two to four hours there, run it through some sweeps, see how close you are. Maybe you're way

over the line. Maybe you're not that far off. You're probably not going to pass your first time, especially if it's not boxed up yet, or it's still just kind of a prototype with random test points and test loops hanging off the sides. But you get a feel of how close you actually are and how much noise your product is generating and what your problem frequencies are. So yeah, get in early, get in often, every prototype. Invest in ferrites. Ferrites are great. Capacitors are great. Most of these labs, yeah.

So the other fun thing is that most of these labs will usually have a partnership with worth or somebody you know your local neighborhood uh cable choke rep to add ferrites and cable chokes to your things at the last minute when you're up against a production deadline and need to pass now that's the smog check equivalent of gunning the engine for a few minutes before running right this is why so many cables have those ugly chokes on them still to this day. So when you do a sweep, that is listening on all of the frequencies to hear where my box is shouting. And I can't shout too loud. And I might have problems on things like clock frequencies. Clock frequencies sometimes, the biggest problems that I've run into, I guess, relying on my background as a DC power engineer, is anything with a DC to DC switcher, anything with high current switching,

anything, there's a lot of power involved, tends to get in the way. So those are the big culprits in my experience, but that's mostly what I focused on. Okay. And then one reason to start your certification process as you start your product is because it might lead you to do things like pre-certified modules. Right. Yeah, that's an easy way to give yourself a leg up on the process. You know, there's a cost trade-off, right? You're going to pay more for that module than you would on the bottom cost of rolling your own. But, you know, you're saving the design time and then you're saving the certification time. So, especially for small volumes, that's a very appealing path. It's really nice for the first thousand or ten thousand of a unit

before you are ready to go all the way. But does that mean I don't have to do any product level testing? Ah, that'd be nice, wouldn't it? It'd be so nice. Sounds great. I know. The answer is almost always no, that doesn't get rid of the need to do product-level testing. It makes your product-level testing easier. It means that you've essentially removed one of the culprits from your whack-a-mole hunt when you're doing the frequency searches, but you still need to do a fully integrated product level test. And that's because of things around the module. What copper weight you're using in your PCB,

does the module have its own antenna? If not, you're running an antenna off that module somewhere. That antenna may be doing who knows what. And then you still have all the product level things, your microcontroller or your power supplies that are just blasting away at whatever frequency they're blasting away at. Speaking of power supplies, we mentioned UN38, which, if I recall it correctly, transport protocol to prevent lithium ion batteries from going boom. Yep. Why would anybody make their own battery? Isn't it? I mean, why do they need to be certified?

Isn't it just a bag of sludgy chemicals? Yeah, it's a bag of sludgy chemicals configured in a very specific way to store lots and lots of energy. And when that sneaks out in an unplanned path, you get a thermal event. So the answer to the question, why design your own, is there are a couple of reasons, and this exists on a couple of levels.

If you're just designing your own single-cell battery, usually in a pouch format, it's because of mechanical design constraints. You're trying to pack the most amount of capacity in the smallest area, and most pouch batteries are custom jobs because of this. You have specific form factor requirements, and you need to get a pouch in there that holds as much energy as you can in the space that you have. At a larger level, you'll end up getting into power systems where you have to build out multi-cell packs. And similarly, you may be using standard cells, but you're still building out a custom pack to meet your power and mechanical requirements. So the main reasons are, unless your product is perfectly designed and you

can just fit an 18650 cell in there and not have to worry about it, you're probably doing something custom just to meet various other requirements of your product. I've heard this is one of the more difficult certifications to get, but I don't understand why. It can be, and it's because it's kind of an absurd test if you just look at it on its face. Oh good, tell me more. Yeah. So there's actually eight different tests within UN38. A handful of them are quite trivial. Like there's an altitude simulation. Make sure your battery behaves fine at low pressure so that when it's in the cargo belly of a plane, it doesn't do anything funky.

There's a handful of like thermal tests and short circuit and overcharge tests, things that you would expect to go along with a battery. People tend to fail at the vibration and shock tests. Really? Yeah. I didn't expect the others. Well, okay. I guess that makes sense. I mean, is it just a connector thing?

A lot of it, you can put pretty small protection circuitry into even a single-cell battery so that overcharge, overdischarge, and even over-temper temperature conditions are handled pretty readily. What is often hard, especially for relatively delicate pouch batteries, is that they will subject it to a shock of 150 Gs. That's a bit. I mean, that's... It seems high, but that's not, I mean, that's a drop from, I don't know how many feet. I mean, it's... It's not a drop from 100 feet.

It's a drop from like 10. Yeah, I don't know. I don't remember. 9.8 times. Sorry. Right. It's all good. It's one of those things that sounds very impressive until you realize that, yes, it's not actually that absurd of a drop.

So it's, you know, somebody on the cargo plane has had a bad day and is really throwing around your box of batteries instead of, you know, being gentle with it like they're supposed to. They don't want that thrown around box of batteries to have any problems. So yeah, the shock test tends to be where a lot of pouch batteries especially fail. And do they fail at the connector level or do they fail inside the chemicals? So they usually fail mechanically inside the battery. You figure a pouch battery is this like jelly roll of, you know, an anode, an electrolyte, a cathode and a separator. And they just roll that up onto itself a whole bunch. And these are super thin layers of material.

And then what essentially happens is that mechanically something causes the cathode to bust across one of the barriers and short out to the anode, for example. And now you have a direct short across your battery that's interior to the pouch. So your protection circuitry is useless at this point. That seems bad. That usually is bad, yes. Thermal events are bad. Right. This is how things like your cell phone go poof in your pocket or on the airplane.

Okay, so why is this a transport thing? Why isn't this a safety thing? I mean, because, sure, I understand the transport people need to make sure that they don't transport a thousand of these and end up with a thousand fires, but this seems like a safety thing. Why am I depending on

FedEx for this? It is also a safety thing, and UL has a standard that they'll happily sell you for battery safety testing called UL 1642. So that's just a safety thing, and UL has a standard that they'll happily sell you for battery safety testing called UL 1642. So that's just a different thing. UN38 gets referenced because it tends to be the more stringent of the tests. That said, UL has some really fun tests in there as well for mishandling batteries. One of the UL tests is they set the cell on fire and make sure it doesn't explode

they basically build a cage around the cell uh and then hold it over a torch and then yeah they roast it like a marshmallow and if if anything uh they call it disassembly if anything why is this the second podcast in a row where somebody has used that euphemism for catastrophic explosive failure. I just remember the guy, we had a suite mate who like to like just like blow things up and set things on fire. That was me. No. Oh. And I just, is he the one who wrote these?

Because I could see him like gleefully, maniacally laughing. What are you talking about? Tell me later. I think most standard writers are really just bored tech writers that want to have some fun with that tech writing degree that they got and enjoy coming up with those euphemisms. And I can't blame them. I'd do that too.

So have you ever gotten to light a battery on fire? Many times. See, this should be like, have you ever gotten to light a battery on fire? Many times. See, this should be like, have you ever touched a pangolin? What percentage of those is just for fun? Um, I'd say about half. Wow. Okay. If I want to light a battery on fire,

how would I go about it safely-ish? Well, the first thing is to build a wire mesh cage around it so that if it does have a disassembly event, you're protected. Because a disassembly event, as opposed to a thermal event, may have shrapnel. Right. Yeah, there is potential for flaming projectile. Well, right, not true shrapnel, unless you put it into... Never mind.

Okay, go ahead. Flaming balls of sludgy chemicals. Right. So you essentially build a contraption that safely contains it, and you put a Bunsen burner at the bottom of it, and you somehow mount your battery cell over top of the Bunsen burner and let it burn until it's done burning. I forget what the standard is, but somewhere in UL 1642, and let it burn until it's done burning.

I forget what the standard is, but somewhere in UL 1642, they have a done criteria when you're allowed to pull it off. And I think it's usually when most of the battery has been consumed by flame. Is this the hack you're bringing to the party? Because if so... I haven't decided yet. The interesting thing about that party is it's about six days after I get back from Burning Man, so we'll see what kind of inspiration I get. It's in a forest, no fireplace. It's a big field.

Especially in California. Yeah. Okay, so... See, this is why people get into standards, because it's fun. You get to blow things up. I think that people miss that, because they think it's all paperwork. But really, you can't have the paperwork without things going wrong. That's right.

You have to remember that for every test in a standard somewhere, it's there for a reason. They didn't just invent that. Exactly. Like, somewhere something went wrong to cause that test to be required. Because everything was just reactive. Absolutely. Okay.

So this show is nominally about embedded systems and firmware. What should firmware engineers actually know about this? I mean, we've been doing this for a little while, and we kind of knew a little, like, oh, look, it's part of the alphabet soup. And I know there are a lot of certification things when we talk about FTA and FAA, and Chris and I have read the associated guidelines and written

the documents. But what about these tests, the compliance side tests? What should firmware engineers know about that? Firmware engineers should know how to get hardware engineers to do everything. Yes! And believe me, the hardware engineers will actually be happier if they do it this way as well. If there's anything that firmware is running in your device that is safety critical or anything along those lines, that means that the IEC or whoever is doing your testing has to come in and audit all of your firmware. And as you probably know, that's kind of a nightmare. So if you can put all of your safety stuff as a hardware latch somewhere or

even just like a PLC that is somewhat more verifiable, then you will save yourself a lot of headache and no one will have to audit your firmware. That said, with medical devices, they're going to audit your firmware anyways. You're kind of hosed. But yeah, I think the more you can do in hardware, the happier everyone is going to be. That said, what firmware engineers can do is make the testing easier. And I think we talked about this a little bit already,

but especially for any FCC testing or if you have any number of radios on your device, Bluetooth or ZigBee or any number of other things, there are other intentional transmitter test modes that you'll need in your firmware to make sure that the Bluetooth is behaving as Bluetooth should. So firmware support for FCC testing is, can we put the device into its worst case mode

where it's spitting out as much noise as humanly possible? Because that's what they want to test in the chamber, of course. And then for Bluetooth, it's can we do the device test modes that we want? We want to spit out noise on channel 19 of Bluetooth. So we need a DTM function that does that for us. One of the things that I remember from developing wearables and things is often, you know, not interfering with other people's products is one thing. But as you're developing these tiny, tiny devices which have radios and other things, you might find that one side of your board interferes with the other side of your board.

And that if your screen is on, then Bluetooth doesn't work. And so this isn't just a certification thing. Sometimes you have to do this stuff just to make your device work. Right. Yeah, many radios on a board can create some weird interferences, or if you have a switching inductor too close to your radio, that can mess all sorts of fun things up. So you want to, I mean, just from a product functionality standpoint,

you want to be able to test everything as well. So the same firmware you would use to validate your product is probably what you're going to use for a lot of your testing, maybe with some small modifications. But yeah, you need to be able to test everything and you need the firmware to support that. One of the big modifications that needs to go into FCC testing is no sleeping. I mean, I spend so much time making my device sleep.

And then that's the one thing that in testing, they're like, no, it cannot sleep. Please just delete that part. Because that's not interesting. It isn't radiating then. Yeah, so part of the FCC testing, right, is that your product needs to be in what they call the worst case configuration, which usually means that everything is on

and blasting full tilt. It's up to you, the manufacturer, to decide what worst case configuration is. But if somebody else comes along and says, hey, I found an even worse case, then you have to go and test that as well so you need to get your device into you know the noisiest possible state that it can be in to do the the fcc testing or other emc testing one of our listeners uh nathan jones asked about or hardware best practices that make certification testing easier. We talked about using pre-certified components and getting the document that will tell you how to certify earlier.

Nathan asked about MISRA coding standards, but unless you're doing FDA or FCC, very few people actually want to see your code. Right. Yeah, unless your code is doing something safety critical, people will ignore it. Are there other best practices? I think we've kind means to do so easily. You want to be writing as little firmware as possible to do that, right? And not untangling all of your various sleep mode functions to do so. I should also probably caveat at this point in time, I've spent most of my career avoiding being a firmware engineer.

So I'm not the expert on software best practices necessarily. And that's why I'm just like, shove it off to the hardware team and let them figure it out. That always works for me. Okay, so if I'm a hobbyist, and I don't want to implant, I don't know, weird thoughts into my own head with unintentional radiation. That's a product. Yeah, you can sell that. If you do that, let me know. Are there good ways for people working in their home making their own devices to make sure that their RF devices aren't unsafe?

There are a handful of things you can do depending on what level of home lab you have. Most hobbyists don't have a network analyzer to really chew through the RF and see what's going on out there. But maybe you can beg, borrow, or rent a spectrum analyzer to see what you have going on out there. But maybe you can beg, borrow, or rent a spectrum analyzer to see what you have going on. It kind of depends with the tools.

Could you do that with the STR? You could do a lot of it with STR, depending on your hardware interface and what frequencies it covers. Yeah, absolutely. So you need some measurement, essentially, to be able to see what's going on. It's not going to be the most precise measurement, perhaps, depending on your setup or level of gear, right? But the standards are always conservative. So they're considering the worst case scenario. If you have some hacked together home-built device and you just sweep over it with your spectrum analyzer real quick and make sure there's nothing egregious, you're probably okay.

It's unlikely the FCC is going to come track you down at that point in time. Unless you made a spark gap emitter, in which case they will track you down. The hams will. Yeah, the hams will. The hams are the secret enforcement arm of the FCC. Which actually is one answer to Nathan's question is get a ham license. That will teach you a lot about what you can and can't do with RF devices and introduce you to people who probably have spectrum sweepers. Absolutely. Yeah. As an owner of an ill-used general class license, I can't agree more.

You too, huh? Yeah, I know, right? It's just a memorization and a test. You can get an iPhone app now that will just ask you questions. Well, as I say, the entire question set is published. So if you just have a good memory, you're set. One of our listeners, Skippy, wrote about his efforts getting a CE mark. And when I read his blog post, which I'll put in the show notes, I didn't understand why it went over Rojas and a whole bunch of other things. We, W-E-E-E, LVD.

But I think I'm getting from you that when we say FCC, we mean a couple of small things, intentional radiation and general emissions, sort of. There are only a few things. Right. But with CE, there's a whole bunch of things that it depends on your category. And how do I figure out what my category is?

Is this another way I just try to figure out what everybody else is doing? Yeah, there's some of that. And then for the most part, low voltage DC devices are pretty exempt from a lot of things. So they have the low voltage directive, which is kind of funny because what they define as low voltage is very different from what a lot of embedded engineers define as low voltage. So the EU definition of low voltage is, I think, between 75 and 1000 volts DC and 50 and 700 volts AC or somewhere in that category. So if you're below 75 volts DC or 50 volts AC, your system is actually, I think it's extremely low voltage or some other below category that they

don't care as much about. What? I mean, what? Low voltage in the eyes of the directive is like line voltage level stuff or perhaps small industrial equipment. But, you know, things that would be above that would be heavy generation equipment or large scale transmission equipment. So my 1.8 volt circuit is like extremely stupidly low voltage. Yeah, they pretty much ignore that.

And so the upside is that there are fewer things that you need to worry about with your regulatory compliance there. 1.8 volts, for example, is not likely to shock anybody accidentally. Yeah. Once I put it through a capacitor, though. Well, you know. And then actually Rojas and we are outside of the CE marking, but they're still relevant European directives. So that, again, comes back to, you know, how do you import your product? And to do so, you need the CE mark, at least to do so into Europe, you need the CE mark and you also need the Rojas documentation.

And if you have electronics in your product, then you need a marking that says don't throw this out, which is more or less what the WE directive is all about. You just need something that says, don't throw this out? And that's how you certify your product? I mean, that seems... Yeah, a lot of it's documentation, right? A lot of these standards spend a lot of time telling you how to document your stuff. They'll say what markings need to be on the product, what markings need to be on the packaging, what markings need to be in your user manual, what information needs to be there. So there are large sections of especially safety standards that get into product packaging and product marking. Are these certifications worthwhile? I mean, is it just a big game that generates money for the insurance industry and governments?

I mean, is this making the world better, or is it just making it more complicated? It's doing both. It's able to do both at the same time. It's definitely more complicated, right? And we've covered a lot of that already. But when you get it down to it, many of these safety standards are quite reasonable. You know, it's the, you have a big enough isolation barrier between your AC power and DC power sections of your circuit board, for example, or your battery can survive 150 G shock. So they have some applicability in actually making a safer world and making the

day-to-day products that we use safer to use. So it's kind of a roundabout and convoluted way of going about the problem, but that's what you have to do when you don't necessarily know who you're working with, right? You need to make sure all your manufacturers are on the same page and you don't know how thoughtful they are or how risk analytic they are and how deep they're willing to dive into their own product safety. So you need kind of a harsh and convoluted process just to guarantee that they will actually do this. How do failures get passed?

I mean, Samsung wasn't that long ago before their phones were exploding. Sure. How did that happen? I mean, didn't they have to do all this stuff? And then what happened? Yeah, they did have to do all this stuff. What happened is that standards can only cover so many edge conditions and boundary conditions, right?

There's always that magical combination of 17 different failures that comes together to create a catastrophic event. You see this sometimes in like aviation safety as well, right? There were a handful of failures that comes together to create a catastrophic event. You see this sometimes in aviation safety as well, right? There were a handful of things that came together that kind of messed with the 737 MAX. Any one of those failures on its own was bad enough, but arguably recoverable. However, when you combined a whole bunch of failures in a weird edge case, then you had catastrophe. If you do the same thing with batteries, you combine a whole bunch of weird

failures in an edge case, the probability is low, right? You've basically said, you know, we've combined multiple 0.1% failures into this weird condition, but those things still happen. And, you know, it's sort of, we push the edge of what is and isn't safe to try to make that 0.1%, 0.05% failures or even lower than that. But there's still those weird long end of the statistical probability curve events that can stack up and cause all sorts of things to happen. When you ship millions and millions and millions of something like Samsung does, all it takes is five or ten events to suddenly become a news story.

Right. Well, then you only test one, two, five units. Manufacturing plays a big part on this. How do they retain certification when they're building so many things? I mean, you can say, yeah, the design is good, but with batteries, you know, a little bit too much of the chemical or a little too little of the chemical. We're not talking like 30 off we're we're little bits

and how do how do they keep how do we how do companies keep certification when they're yeah through manufacturing sure so there's a lot of things that are going on there as well uh there's a few sides to this one of which is that this is more or less what ISO 9001 is all about. And this is a standard we've all heard lots and lots about, but most people don't actually think about too much. And it's essentially a standard that guarantees that your manufacturer has documented processes and that they're following them. So that's, you know, when you're working with a manufacturing to make sure they actually have their ISO 9001 certification current. And that means that they are going to be, you know, they have committed and are audited to following certain procedures and certain requirements. On certain products,

especially when you're getting into power electronics, the standards will actually break down three different categories of testing for your product. Type testing, sample testing, and I forget what the third one's called. It's routine. Sorry, routine testing. So you have type testing, which is we have given you a sample that we feel is representative of this product that goes through the whole range of tests. And that's good. And then you have sample testing where a notified body or some other organization will come in and periodically sample your product and make sure it is still conforming to those tests. And then routine testing is something that you have to do

to every single unit. The routine testing is perhaps a high potential test to make sure your isolation barrier can withstand high voltages that you might do to every single unit coming off of the factory line, right? You would have a tester at the end that generates 16 kilovolts that you put across some isolation boundary and you make sure that leakage currents are within acceptable limits. So you do have to do some routine testing as part of some standards, but not as much as one might hope necessarily, especially with batteries. So maybe there's a future in which we do do more routine testing of battery

systems in the manufacturing world, but it's, it's still coming along there. That said, as long as your battery vendor is again, ISO 9001 certified and you should be auditing your your suppliers regularly especially battery suppliers so that you you can then know that they are following their processes and that the the samples you tested are representative of the entire population

of batteries they are producing so aside from batteries everybody has heard about, or I have had happen to me, that the manufacturing line will run out of one type of capacitor, resistor, converter, and they'll swap in another one because it's pin compatible. And when do I have to recertify it? Sure. Well, so first of all, if your manufacturer is doing that without telling you, you need a new manufacturer, they should never be swapping anything out that you haven't approved. So oftentimes when you send a bill of materials to your manufacturer, you'll have an approved parts list, an approved vendors list of like, you know, part B is an allowable substitute for part A and things like that. So that when they do make that swap, you've already approved that swap and

it's all good. To the certification question, there are a couple of things you can go about it. If you're swapping, if it's a safety critical component, you want it to be UL listed already and you want your substitute to be UL listed as well, so that when they swap in one UL listed component for another UL listed component, you're fine. If it's something else, if it has perhaps something to do with the EMC behavior, you'll want to test both configurations, or you'll want to, there is room for essentially drafting an engineering justification of why the substitute is insubstantial to the test result. And you can then submit that to the notified body or whatever

regulatory or agency you're working with as part of the documentation package. And so you do some math, you do some analysis, perhaps some engineering work on the design side just to come up with a justification of why this substitution is okay. And does that work? Often, yes. Okay. I mean, I guess it has to because things have to be flexible enough to be made. Right.

And most components are not critical. You know, the random 47k ohm resistor that's a pull-up somewhere is probably not a critical component unless it's part of a critical circuit or whatever. But there are a lot of parts on your board that are not necessarily critical and can be substituted out pretty readily. at the end of the day, people running standards organizations, doing the testing, doing the notified body work, they're still people. And most of them have engineering training and strong technical skills and will respond to rational technical arguments. Really? No, sorry. That's true. One thing that has never been clear to me is how people learn about this. I only learned about bits and pieces by doing it and watching other people and having to certify things. But I've never seen a book. And it sounds like maybe the only way is through training. But do you have any other suggestions about how to kind of get a handle on all this? If you're going to be somebody working in these

areas? Yeah, I mean, there's there's a number of books on the subject. I think one of them that I was trying to remember the name of a little while ago is just like, you know, an academic style textbook called Electrical Product Compliance and Safety Engineering, which is, I know, a super compelling title, but at least tells you what's in the book. There are a number of conferences in the subject. The IEEE runs something called ISPCE.

Uh, I think it's like the, the international product compliance engineering symposium or something like that. They run that once a year and I know they tried to keep that relatively affordable three day conference. Uh, so you can, you can go and learn from people who have been working in the product compliance space for many decades and, and ask and ask them all the tough questions as well. I've been to that event a couple times, and that's how I learned a lot of what I know about product compliance engineering. So there's a few of those.

There's a handful of good sort of starter-making product books that at least touch on that this is something you need to do. Yeah, I think there's one like making products book from O'Reilly that I forget the name of right now that goes into this at least a high enough level that you know you need to dive more into this. And then there's the worst case way of finding out that you need this, which is when you try to go and import your product and the customs guy says, no, you don't have this standard documentation. So try to avoid that one. I think that a really author was Alan Cohen. We had him on the show a while ago for his prototype to product book. Yeah, that sounds right. It was really good. I mean,

it was very thorough. It's a great book. Absolutely. Okay. So I have other questions, but they're totally not related to compliance or manufacturing. We couldn't record a show yesterday because you had a gig. What kind of band and what kind of gig? Sure. So I play in a brass band, New Orleans funky style brass band, but we play mashups of pop music and other weird charts. So we'll mash up Metallica and disco, but as an arrangement for like four trombones and four saxophones. That's awesome. That is awesome. So we have a lot of fun with that. I play bass saxophone in the band, which is a fun and rather sizable instrument, which is annoying at times, but I still love the sound of it. So, you know, hey, why not?

The gig itself was a wedding gig, which is both lucrative and annoying. So, you know, it was what it was. But, you know, we had some fun. We played some music. We drank some beer. The usual musician shtick. That does sound like fun. I remember when Christopher played out more. I liked sleep. That was the problem.

It is hard to have a side band and still get regular nightly sleep, especially if you're playing bars until two in the morning. That was the problem. Well, do you have any thoughts you would like to leave us with? A handful is consult your manufacturing and compliance team very early in the product development process. You want to get that input in as soon as you possibly can, ideally during the concept definition phase. So consult everybody who actually makes the product happen as early as you possibly can. Was there more than one? More test points. All right. That's a flag I can always stand behind.

Our guest has been Monk Eastman, International Engineer of Mystery. Thanks, Monk. Thanks for having me. Thank you to Happy Day for the question. I hope we have answered at least part of it. Thank you to Skippy for his post on CE experiences. And thank you to Adam, Jakey Poo, Nathan, and Philip for their Patreon Slack questions and their Patreon support. Thank you to Christopher for producing and co-hosting and thank you for listening. You can always contact us at

show at embedded.fm or hit the contact link at embedded.fm on embedded.fm somewhere. Near embedded.fm. Yeah, something. The party link is embedded 300.eventbrite.com. And if you can't remember it, but you can remember the email or the contact link, you can contact me and I'll send it to you. Okay. I was going to try

to read aloud the XKCD for the quote, but I'll just post the XKCD of relevance with standards and go with a quote that is probably from Winston Churchill, but I didn't check it. So if you have 10,000 regulations, you destroy all respect for the law. Embedded is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show, we did not put them there and do not receive money from them. At this time, our sponsors are Logical Elegance and listeners like you.