Embedded - 154: Physics Is a Big Pain

Episode Date: June 2, 2016

Jeff Keyzer (@MightyOhm) joined us to talk about consumer manufacturing, how to solder, and having a full time job and a kit company. Jeff's blog is on MightyOhm.com. The Geiger Counter kit is availab...le atMightyOhm.com/geiger. The really, really useful Soldering Is Easy comic book isMightyOhm.com/soldercomic. At Valve, Jeff worked on the Steam Controller (hardware specs at bottom of the Valve page or for sale on Amazon). There is also a neat video showing the manufacturing automation in action. We mentioned Glowforge, Dan Shapiro was on episode 125 (and if you are going to buy one, please consider using our referral link!) Elecia and Chris have a Hakko FX-888 soldering iron. Jeff suggests Kester 186 flux which you can get in smaller-than-giant containers on eBay. No, not the pen on Amazon. Or maybe the MG Chemicals 835 (which is in little bottles on Amazon). Flux seems like a very personal thing. 

Transcript
Discussion (0)
Starting point is 00:00:00 Welcome to Embedded. I'm Alicia White with Christopher White. This week, our guest is Jeff Kaiser of Mighty Ohm. Hi, Jeff. Welcome to the show. Thanks for having me on the show. Could you tell us about yourself? Sure. Let's see. I'm a hardware engineer. I work at Valve, which is a well-known video game producer, maker of titles like Half-Life and Portal, which I think some people would have heard about. And at Valve, I develop consumer electronics hardware. Probably most notably, I am the lead electrical engineer on the Steam Controller, which was Valve's first hardware product and was released to market last fall. Before working at Valve, I worked as a freelance circuit and PCB designer, and I also spent eight years designing radio frequency integrated circuits
Starting point is 00:00:58 for cell phones and wireless networks. I'm also the sole proprietor of Mighty Ohm. I'm a blogger and I'm a maker of open source electronics kits. Excellent. So now before we ask you more about those things, we have a lightning round where we hope for short answers from you for our questions. And then if we are behaving well, we won't ask you for detail. More likely, we'll just argue. Christopher, you start. Okay. Favorite processor of all time? I think it's probably the Atmel AVR.
Starting point is 00:01:37 That's a pretty big processor there. Yeah. Can you be more specific? Well, I've used the Atmegas, like the 32 328 in projects I've worked on before. And in danger of having a longer answer, I like it because it's fairly simple, but you can still do a lot of interesting stuff. Introvert or extrovert? Oh, I think I'm an introvert. Sometimes I try to be an extrovert.
Starting point is 00:02:00 Yeah, I try that sometimes. And then I take a week to recover. What did you want to be when you grew up, when you weren't grown up? If you are grown up, I don't know. Are you grown up? I think I'm pretty grown up, but maybe not totally grown up. I guess I hope I'm not totally grown up. I always wanted to be an electrical engineer.
Starting point is 00:02:26 And I think a role model for me early on was Forrest Mims. And I really wanted to be able to design circuits. So that was definitely an early inspiration. I've always known I wanted to work on electronic circuits. Okay, I want to come back to that later. But next question. Hacking, making, tinkering, or engineering? Engineering. Contracting or consulting what does that mean I think contracting
Starting point is 00:02:50 consulting is something I picture for people that are later in their career alright should we bring back the dinosaurs no would you rather explain I squared C versus spy or decoupling capacitors? Decoupling capacitors. Is a mobile phone an embedded system?
Starting point is 00:03:15 Yes. Favorite spacecraft? Ooh, probably the space shuttle. Fictional is okay too. Fictional is okay too. Fictional. Probably the spacecraft from 2001. That's pretty awesome.
Starting point is 00:03:31 Cool. Han or Chewie? Han. I think that's it, right? Do you have any more? No, that wasn't very lightning-y. We can go through the whole ring. No, that's fine. i can just do one word answers
Starting point is 00:03:48 we could try to do a speed round grunts binary answers yes or no honor to it yes or no um okay this one in more detail what is a work day like for you uh let's see. I'm not sure that I have a typical workday, but these days, let's see, I usually get into the office. The first thing I usually do is get a cup of coffee. I'll check email, see what's been going on in China overnight, because I work very closely with a factory that's in China. So usually by the time I get in the morning, there's some interesting stuff going on. So I'll take care of that. And then these days, I spend most of my day prototyping. So that could be doing circuit design, quite a bit of PCB layout, building prototypes, testing things. And then I usually round it out. A lot of days, I'll have a conference call with Asia in the evening. And so kind of the cycle
Starting point is 00:04:46 continues. That makes sense. A very traditional sort of day for folks, those of us who work with China sometimes. So that's all for your Valve job. But I want to talk about Mighty Ohm first. Sure. What is Mighty Ohm as a thing? I mean, it's funny, I have to ask that because like Logical Elegance is a consulting company, we don't sell anything physical, but it's also my name on Twitter and what people associate me with. So Mighty Ohm is partially you and partially a company. What is the company? Yes. So the company started out, so Mighty Ohm has kind of evolved over time. But it started as a way for me to document projects that I was working on. So I've been a maker and I've built hobby projects for quite a long time. And I wanted a way to document those things and get some of the stuff
Starting point is 00:05:45 I was working on sort of into the world. Around a similar time, I started doing freelance development work. And so it became the company that I used to sort of sell my services as an electrical engineer. And so for a time, it was primarily my contracting company, and sort of like a personal brand, I guess, like something that people could recognize me by and that I could use to just attach to a bunch of stuff that I was working on. And then later on, it sort of evolved into this open source electronics kit business. And at this point, that's most of what I do under the name Mighty Ohm is I produce and sell open source electronics kits. Did your Mighty Ohm persona help you get your job at Valve?
Starting point is 00:06:41 Absolutely. Yeah, absolutely. That was a, it was a combination of, I think, just sort of being known, like having a entity at the Maker Faire and on Twitter and online to where it was, it was an easy way of sort of making folks that were already at Valve familiar with who I was and what I did. And so when I ran into one of the early members of the hardware group at Valve, which was Ben Krasnow, it's sort of like I didn't need an introduction because the folks that were interviewing me already knew who I was. And it was a huge benefit to sort of have that reputation when I walked in the door. I interviewed at a place and they asked me questions out of my book.
Starting point is 00:07:36 So, yeah, yeah. It gives you a lot to talk about too, because you've got this huge body of information that's online so a prospective employer can sort of look at it and figure out what you're about and it is a great conversation starter too have you had any negative reactions to it um people have mentioned they are concerned that we'll make fun of them on this show and so i wondered if you have any really make fun of anybody on this show like i what kind of negative reactions? Well, like people thinking that you're only willing to work on things you can write about or talk about.
Starting point is 00:08:13 Ooh, that's definitely, I have not had that. Gosh, yeah, I have not ever run into that criticism. I mean, obviously I work on a lot of things outside of Mighty Ohm. So, Mighty Ohm is kind of just one branch of what I do. I've been tempted at times to try to start something else for sort of different projects. Like I work on amateur radio projects and it's like, okay, well, maybe this is something else. But I haven't gotten to the point where I've wanted, once you know how much time it takes to develop something like that, it sort of discourages you to go off and want to do,
Starting point is 00:08:50 you know, three more websites for other things. But no, I found it's been really positive. And both at work, folks know that I do this stuff. They know that I'm on podcasts. They know that I write about stuff. It's never been an issue, not even once. But making kits for other people is non-trivial. I mean, there's a lot of documentation that goes with it. There's a little bit of support. And then there's just shipping the darn things as well as kitting them. Yes, it is a lot of work. How do you have time for this? The truth is that these days, I don't have a lot of time. Most of the work I did at Mighty Ohm, and I was very fortunate, it happened before I joined my current job. So before I came to Valve, I was working on Mighty Ohm pretty much full time. I had moved to Austin, Texas. I was having a
Starting point is 00:09:46 really hard time finding contract work out there. I hadn't lived in the area very long, didn't have many connections, and just sort of things evolved to which I was primarily doing the kit stuff as my full-time job. And that's part of what led me to Valve ultimately was the kits. But at the time, I had a lot of time to work on stuff like that. It was kind of my primary focus. And so I built the business around the kits primarily while I was doing that full time. And then when I went amount of respect for people who can continue. Like if someone were to start a business like that while working at a place like Valve, I think that would be incredibly difficult just because there are only so many hours in the day. And is it, how much of your time now is spent on it? Or is it sort of in the snoozing mode where you do enough to keep it going, but it's sort of resting until you are ready to take up? It's a combination. So I would say that most of the time I spend on Mighty Ohm these days is on fulfillment and keeping up with existing designs. So I'm kitting and I'm putting parts in bags and I'm shipping them to people and I'm providing support. And so I've got released a new kit. And so that part of the company has kind of been sleeping, whereas I'm still spending and the time varies. The kits that
Starting point is 00:11:33 I sell, I spend probably like a weekend a month at this point, building up inventory. And then, you know, it's like an hour a night at most, just putting things in boxes and doing fulfillment. So that's very manageable, a few hours a week. If I was still selling the amount of kits that I was when I first designed, for example, the Geiger counter kit, that would be really, really hard to keep up with because I was spending, you know, probably 40 plus hours a week on kitting and fulfillment. And at this point, it's really sort of tapered down. So let's talk about your Valve job. Four years ago, they didn't have much of a hardware department. No, it was like the Wild West. We should really start with what is Valve?
Starting point is 00:12:26 So Valve is a video game studio. And Valve has been behind some very popular games in the PC gaming industry. And Half-Life is a good example. So is Portal. So is Dota. These are popular titles in PC gaming, some of which are also available on consoles like xbox and playstation but valve is also behind steam and steam is a content distribution network for video games and it's the most popular in the world okay so that's all software right um
Starting point is 00:13:03 but now but about four years, they started having a hardware department. Yes, that's right. So Valve was interested in making hardware and interested in learning about what hardware means for gaming and what hardware might do that's interesting in the gaming space. And so it started as some folks in the company had the idea that maybe hardware would be an interesting area to explore and started prototyping things and things like game controllers and things that are hardware, which interfaces with video games. And that's sort of what led to an expansion of the group. And it was indeed about four years ago, maybe a little more than that, that some folks were brought on to solely build hardware. And it's sort of grown from there.
Starting point is 00:13:54 So Chris, you have joined companies that were primarily software in order to work on hardware. Haven't you? Not really. Oh. What, like what? You'd think I'd know. Well,
Starting point is 00:14:06 but we have both been in spots where you join and they don't understand why you can't do Agile. The reverse. I've joined hardware companies that didn't know anything about software. Well, yes, that too.
Starting point is 00:14:20 But either way, there's a huge learning curve because if it's a hardware company that doesn't know about software, they don't understand the speed. And if it's a software company, they don't understand the slowness of hardware. There's the speed and then there's just what's normal, what development looks like in a discipline that you're not familiar with. But software folks often don't understand that there is physical cost. There are electrons to be shipped. And protons.
Starting point is 00:14:49 Yes. Did Valve hit a learning curve like that? Or did they educate themselves enough? Or did you educate them? at Valve was very much composed of people who, some of which had experience in hardware, but many of which didn't. And, you know, I had not worked in consumer electronics, certainly before joining Valve. I had a hardware background. I'd worked on high volume products, but of a very different nature, like high volume chips. And that's kind of a different world than consumer electronics. So a lot of us were new to it. And I will say that there was a learning curve. And I actually think that there is still a learning curve. It is not like we came out of the past four years being like we are experts at making hardware, but it's really a process of experimentation and evaluation. And I think we're still figuring out what the best model is for us
Starting point is 00:15:46 and trying different ways of developing. I mean, back in the early days, I would say that, speaking for myself, I spent a lot of time where I had no idea what I was doing. And just this feeling that I'm sort of, I think it's like the imposter syndrome. It's like you get to this place and you're expected to do these amazing things. And you're like, I'm not worthy. How am I going to figure this out? And I spent a lot of time in the beginning being like, I have no idea how we are going to accomplish these really ambitious goals, which was to ship mass market high volume consumer electronics hardware. And I think I still feel that way sometimes, but having been through the cycle once, it's a lot just easier. It's a lot less anxiety inducing than it was in the very early
Starting point is 00:16:35 days, for sure. Do you think you've converged on processes and methods that everyone else uses? Or do you think by kind of having a more experimental approach to it that maybe you guys do things a little differently than the rest of the industry is doing? I think we do things quite a bit differently. And I think there are commonalities. But for example, Valve has embraced automation at a scale, which I think is pretty much unheard of in consumer electronics, certainly in the US. And that's very, very different and had some kind of unique requirements, but also some huge benefits for us in terms of allowing us to have a repeatable product that was consistently manufactured at a pretty high quality without having huge teams of
Starting point is 00:17:19 people managing that process, either at the contract manufacturer or at Valve. And I think that's been a huge thing for us. And that was sort of the big experiment that the Steam Controller was embracing, was would automation make sense for us? And that was an amazing process and learning experience to be a part of. Like I said, I think we're still experimenting. And I think that a lot of things are kind of the same. Like we, the way that we manufacture printed circuit boards is the same as I think everybody else manufactures printed circuit boards. Like we're not doing anything that's really significantly different than what's commonly done in the industry. But the final assembly is actually quite different where plastics are combined with PCBs using robots. And that stuff is, I think, very unique to Valve.
Starting point is 00:18:12 If I opened up a Steam controller, what would I find? You'd find a lot of parts. The Steam controller is a very electromechanical device, and that's something that we sort of learned. I'm not sure that we fully understood that when we started, but Steam Controllers are actually quite hard to build. And the reason for that is that there are a lot of fairly tight tolerance assemblies inside, and a lot of moving pieces. Like, if you think of a game controller, like any game controller, there are lots of buttons. Buttons are actually kind of hard to do well, especially hard to do well in a repeatable way when you tried to build a million of something. There are a lot of things that can go wrong with buttons, and I've seen a lot of them. And boy, do gamers care a lot. Yeah, that's very true. The feel, a controller is also something where the tactile, tactility of it matters a lot to the end user, as you said, and that is also something that's really challenging. And so as a result, when you open up the Steam Controller, there's a lot of
Starting point is 00:19:19 parts inside, like there's a lot of moving pieces and hinges and buttons and springs and all sorts of stuff. So you've got a large printed circuit board, which sort of dominates the view when you pop off the back, but there's also a lot of moving mechanical parts that are attached to it. And those are the hard parts, I would say. Do we have a Steam controller? No, we have whatever comes with the Vive, which I think are very similar. So those are the Vives, the handheld controllers. And those are a different design, but there are some commonalities. So the Steam Controller was Valve's first hardware product. We've been working on the Steam Controller since I joined in 2012. At that time, it was mainly hot glue prototypes and trying to figure out what we wanted to build. But some of the stuff that we developed for the Steam Controller is wound up
Starting point is 00:20:10 in the Vives controllers. And probably the most obvious one is the touchpads. How does the touchpad work? Ooh, the touchpad works through capacitive sensing. There's a touch controller that is, well, so the touchpad is a PCB. And on the back of that PCB is a controller IC, which is purpose-built to do touch. It is running firmware, which actually acquires a series of capacitive measurements. This is how all touchpads work, or most touchpads, except for the resistive ones, which are kind of the weird ones. But most capacitive touchpads work in a very similar way. So if you pop off the cover, you'll see rows of electrodes, and those are what actually do the sensing. So it's the job of this controller to basically make sense of the capacitance measurements on these electrodes
Starting point is 00:20:59 in order to determine where your finger is touching. They're circular. Is there anything weird with having a non-rectilinear sensor grid? It's not as weird as you would think. You could think of them as like truncated rectangles. Okay. Yeah. It's kind of like how circular LCDs work. You know, a circular LCD is mostly or oftentimes a square LCD
Starting point is 00:21:24 that someone has cut into the shape of a circle. And so you'll have these kind of phantom pixels, which are still there according to the display memory, but don't actually go anywhere in the hardware. Touch controller, it's not quite exactly the same, but it's kind of a similar thing. Like, you still have rows and columns of electrodes, but you're just paying attention to touch within a circle instead of a rectangle. I'm sad. I was really hoping for a spiderweb pattern and angular. I'm sure that there are other ways of doing it, but this is a way that has worked really well for us. We just got the Vive VR head system about a week ago,
Starting point is 00:22:07 not quite a week ago. And it is pretty awesome. The time between opening the box and sustaining Andrew was less than two hours, most of which was spent lunching. Is that about average? I don't know. Have you gotten to play with it? I've gotten a demo. So I primarily worked on the Steam Controller and I worked on the Vive a little bit, but mostly in stuff that wound up in the Vive than on the product itself. I've gotten a demo and it is amazing. I would love to have one at home, but I have not had time to actually get one and set it up. It's really cool. I didn't think I would care. I really, really didn't think I would care at all.
Starting point is 00:22:53 And yeah, I was nearly late for recording because I was up there playing with it. I think for me, the thing that really, the controllers really make it for me. And I think the ability to do things like paint in the air in 3D space, I found myself like I could do that for hours. It wasn't like five minutes and I was bored. It was just like, even very simple things are really, really engaging. And that's, I think what to me is just the most surprising about it is that I just kind of want to hang out in the virtual environment. And I find that when I come back out again, it's kind of weird to come back to reality. It's just the immersion is pretty, I don't know, to me, it's pretty breathtaking. Yes. And for me, it was sort of like the first
Starting point is 00:23:44 time I used a smartphone, which I think was my sidekick which still was a pretty sad smartphone but I went from not wanting a cell phone at all because it was just a chain and annoying to I can't live my life without this no you may not take it away and I didn't realize at the time that there was the shift. And I'm afraid that VR is going to be that same shift. I didn't expect it, but yes, I want to stay there. And when I leave, I'm like, I made this huge mural. I'm really happy with it.
Starting point is 00:24:19 And I took off the headset and it's gone. Anything that after 15 minutes of using it, you tell me to go build a new computer. I made him move it from his dungeon lair to our living room. It's pretty cool. But I think we may actually talk about VR with one of your coworkers in a couple of months. So I shouldn't use all of my VR questions on you.
Starting point is 00:24:42 Yeah, I don't want to steal his or her thunder. So one of the things you mentioned was ramping up consumer manufacturing. And I know that you hired Phil King, who used to be my double E. And we worked together at Shots, Boughter, and at LeapFrog. So you definitely stole him from me. I'm really sorry, but we're really happy to have him. I actually met Phil at the Maker Faire. I think that the recruiting potential of the Maker Faire is very understated because I ran into him at the Maker Faire, and it wasn't long after that we convinced him to come up and interview.
Starting point is 00:25:25 So, yeah. So if you want to come up and, um, interview. So yeah, it's, so if you want to work at valve, apparently you sign up to do a booth at maker fair. That worked. I'm pretty sure that works with more than just valve. Good to know.
Starting point is 00:25:39 Um, so what are some of the lessons you've learned as a company about making consumer-level hardware? Automation, that's cool. What could somebody who's just starting the process be able to use? You mean in terms of alternatives? In terms of advice, I think that this is probably obvious, but one of the things that I have often reflected back on and wished that I knew when I was starting was the importance of having a clear specification, especially when working with outside vendors. It's really hard because you are early in product development. If you're like me, you've never done this before. You don't really hard because you are early in product development.
Starting point is 00:26:26 If you're like me, you've never done this before. You don't really know what you're doing when it comes to the manufacturing side. And things are moving fast. And most of your focus is on trying to build this product. And it's really not on the manufacturing side. You know, as a design engineer, your first and foremost concern is actually making your thing work and making the user experience good and making it work well and man and manufacturable, but on the design side, you're not really thinking about how you're going to hand this product off to someone else to make it. And I've
Starting point is 00:26:55 definitely there's been times when huge amounts of time have gone by without really clear specs, and it can cause trouble, especially when your stuff doesn't work. And there's sort of nothing to go back to, to say, well, it was supposed to do this. And I'm not talking so much about the product as like component level stuff that goes into it, you know, things like antennas and things like custom components. It's really important that they have specs. And I found it to be really challenging to want to spend time on that early on, but it is so important to do that, to have that written document. And it really needs to be written down. Verbal agreements are not a good substitute for something that says that the return loss of this antenna must be greater than X dB.
Starting point is 00:27:54 That is really useful when you're sitting at the factory and the parts don't meet spec and you need something to say, well, we agreed it was supposed to do X and it does Y. That's incredibly important. That sometimes seems lost almost. I mean, it is incredibly important, not only with hardware and this stuff down and capture it is something that was a big learning experience for us. And something that I think is pretty different from how software groups would typically work. And I'm sure that's not true everywhere. But the ability to sort of freeform prototype and not lock things down early is sort of counter to the realities of volume manufacturing where stuff does need to be locked down pretty early in order to have any kind of a realistic schedule for actually shipping that hardware. It really is important. And I found it annoying,
Starting point is 00:29:00 like having to commit to how a PCB was to be tested early on when I'm not even done with the design yet. It's hard. It's hard to be a design engineer, designing the things for consumers and thinking about what's best for the user, and then switch into this role of manufacturing engineer, where you have to think about how it's being built and how to save pennies. How do you do that? I think that it, fortunately, I was able to do some of those things in series, but there was always the knowledge that from day one, we knew that we wanted, you know, whatever we came up with, we wanted to build it. And so we worked really hard to make sure that even fairly early prototypes were manufacturable. And we were thinking about manufacturability from a very early stage. And so that's sort of counter to the prototyping ethos, which is sort of a hot glue and throwing things together and doing things quick and dirty. And we did that for a while. But I think at some point, it was like, okay, well, a lot of this prototyping is done. And if we really want to build something, we're going to need to make stuff that's closer to what you will eventually buy as a product. And so there was a big push to try to, you know, use components
Starting point is 00:30:20 that we could source in large quantities. And that's something that's really important. And also to think about stuff like test pretty early on. I found it hard to switch, like being a designer and owning the design and then shifting to working on the test systems and how to test that design. It's kind of difficult to, it's like you're zooming up and down in sort of levels of abstraction. And I found that to be somewhat challenging just because there was sort of so much, like every little building block of the design had its own unique requirements for test. And so you're sort of jumping from high and low levels of abstraction around and trying to remember design things that you did earlier. But it's also a huge asset to be the guy who designed the thing. Like one of the things that's interesting is I've spent quite a bit of time in China, and you kind of have a superpower
Starting point is 00:31:17 in China, if you are the guy at the factory supporting a product, and you designed the product, like that is something that is really special because I think the factory guys quickly figure out that you know all the answers. And so you get a really high level of high quality support from engineers. And it makes things so much easier because being in China is you're somewhat disconnected from the guys back home and from the team back home. And so to be able to make decisions because you know the design inside and out when it's, you know, 3am Pacific time and you're in a factory somewhere trying to get things working so you can go home, it's really great to be the guy who has the answers.
Starting point is 00:32:00 And yet it is a different discipline. I mean, there are people who graduate as manufacturing engineers who have a totally different perspective on how to do this right. Did you work with any of them? I did. I worked with manufacturing engineers who taught me the importance of having test specs and the importance of starting test development early in the process. And that was a huge benefit and also taught me a bit about statistical process control, which I'd had experience with previously, you know, as a chip designer, that's also super important in chip design because your yield is sort of one of the paramount things that drives cost. So I was familiar with statistical process control and doing CPK analysis and stuff
Starting point is 00:32:45 like that. But it was sort of a big picture view that I appreciated. It is a different set of things to consider. But I think there's a tremendous value in designers having that experience with manufacturing, at least at some level. I think it's incredibly important to know when you're designing something. It's sort of like having accountability, like your thing has to be testable, and it has to make good yields. Otherwise, it's not a successful design. So it's sort of like being forced to it's like, you Yeah, you did the design, but does it actually work and being able to go all the way from prototyping to volume manufacturing and being held accountable for success at every stage, I think makes you a better designer because you
Starting point is 00:33:30 realize the cost of mistakes. You talked about the controller being complicated and having lots of moving parts and being very concentrated in terms of density and sort of being difficult to build. How do you strike the balance between, or how do you figure out if you've designed something that's maybe a little too far over the edge? Because I've heard of other companies doing that where they release a product and Tesla comes to mind with the Model X. They've had tons of trouble building the Model X because they've jammed everything they could think of into it. And it turns out it's hard to build. But that can be a detriment, right? You can make the most space-aged
Starting point is 00:34:07 thing in the world, but you can't build more than five. Yeah, I think that it's hard to know that stuff up front. I think that with experience, you gain knowledge of what works and what doesn't. A good example is, it's not obvious when you're starting out, but anything, so there's two things that have caused a lot of issues in my experience. And one is anything with regards to labels. Labels are hard. You wouldn't think that labels are hard, but labels are hard because they're sticky. They have artwork that needs to be done correctly. They need to be in the right spot. They need to not come off. They need to not be backwards. There's just all sorts of issues with labels that can cause a problem. And so simple things like that can turn into huge time sucks because of just quality related issues with regards to labels. The other is adhesives for similar reasons. process. So like soldering would be considered a wet process, particularly if it's outside the context of like an SMT line, anything that consumes liquids or just consumables, like where
Starting point is 00:35:12 you're taking solder and you're putting it on a board. Those are the things that are going to be hard. I don't know what the equivalent would be for the automotive industry. But, you know, in terms of like features that just end up being hard to manufacture, it's really hard to anticipate. Like you, you learn that stuff along the way through trial builds. And I think that's a good reason to try to start building larger quantities of a product pretty early. Like you don't want to build one or 10 prototypes and then go from that to a million. You want to build a hundred and then a thousand and then,000. And you want people to use them and kind of beat on them and discover what the bugs are. And along the way, not only do you
Starting point is 00:35:52 get a lot of good data on your product, but you also learn what's hard to manufacture. And I mean, at the end of the day, you need to create a product which is both a good product for the user, but is also manufacturable. And I think that's kind of something that evolves. You know, there are not, it is not without challenges. You learn things that don't work well and have to re just change how you're planning on doing certain sets of assembly. I think it's really hard to sit at the starting line and know what those things are, except if they're labels or adhesives or wet processes. So what should software engineers know about manufacturing?
Starting point is 00:36:32 Embedded software engineers. That's a good question. I think it's that it is slow. Things do move more slowly when it comes to manufacture. And again, I've kind of already talked to this point, but the importance of test. I mean, I think that a good embedded software developer, and I've worked with really good embedded software developers, design test in while they're doing design. And so when you're finally done with the design, you've got a test program or a test framework that you can use. And for a hardware engineer like me, that makes my life so much easier.
Starting point is 00:37:09 And so I guess maybe my advice is if you want your hardware engineers to like you, you should design test into your embedded systems. Because it'll make us hardware guys so happy if we can test your stuff. Oh, yeah. Giving a bring-up test to a hardware engineer and letting them type at it and figure out what's wrong for themselves is so nice. And when you can take that and make it into your manufacturing test with just a few shorter commands,
Starting point is 00:37:34 it's just, yeah, it makes production so much easier. Well, it's not only that, but having efficient methods of programming things too beyond testing because you can have... That's true. You know, you can say, oh, we're going to use Z modem over the UART. Is that all right?
Starting point is 00:37:49 At 9,600 baud to program three megabyte, we're going to do that for every unit. That's great, right? And no, that's not great. Can we do something else, please? No, it's very true. And I think there are just little things that you can do, like expose the ability to reset parts, which
Starting point is 00:38:05 can be very, very useful in manufacturing. If you've got the reset line of a chip brought out, or you have the ability to do in software reset, I think good bootloaders are another thing that can make hardware developers lives much easier. If you've got a good bootloader, the ability to upgrade stuff in the field is incredibly valuable. But even the ability to upgrade things easily on the manufacturing line, you know, you're going to find bugs. It's a certainty that you're going to find bugs in part of the manufacturing process. And the ability to quickly resolve those and have a lot of flexibility in the embedded software is super useful. Okay, so the truth is that when I started,
Starting point is 00:38:46 when I invited you to be on the show, which I think was only about a week ago, because anyway, when I invited you to be on the show, I had no idea you worked at Valve. So you knew me through the Mighty Ohm stuff or you just didn't know me? I knew you through the Mighty Ohm stuff. I knew about your comic book soldering project with a few other folks where you teach how to solder through nice manga style, very simple to follow instructions.
Starting point is 00:39:18 And then that led me to Mighty Ohm and then I stopped there because that was really cool. Can you tell us about this comic book? How did it come about? What is it? So the Soldering is Easy comic book is a comic book that seeks to teach anyone how to solder, even a complete beginner. And it came as a result of a book that I was working on with Mitch Allman. So Mitch is sort of famous within the maker world because he worked on a few projects that were featured in early editions of Make, like the TV Be Gone, the Trip Glasses. And he's also led soldering workshops sort of around the world. Mitch spends a lot of his time every year traveling the world and teaching people how to solder. And so he was sort of a soldering evangelist, right? And Mitch and I were working on a book about how to make cool projects with
Starting point is 00:40:15 AVR microcontrollers. And so Mitch had a one-page version of the comic book that he was using in his workshops, and he wanted to do a larger version that was more complete that we could include in the book. And so we worked with Andy Nordgren, who was the illustrator, and put together What You See. And I think that was in 2011. And it instantly became hugely popular, by far the most popular thing I've ever hosted on my website or ever been a part of in the maker community. And it was so popular that it was translated almost immediately into several other languages. And I think it's up to 17 languages now. So it's, I think one of the things that's really unique about it is just how international it has become. And it's probably one of the things that if I'm some strange place I've never been
Starting point is 00:41:06 before, and there are makers, there will be a copy of this comic book. And it's just amazing to see how far it's gone. Does it come in cycles? I mean, somebody discovers it and there's a whole wave of people talking about it. Yes, definitely. I think it's been picked up by Lifehacker more than once. And so I think sort of it will be popular and then it will sort of drop out of the public consciousness and then Lifehacker will bring it up a Creative Commons licensed comic book. So, anyone can make their own version of it, can print it out for free, can publish it, whatever, as long as they give attribution to the authors. And so, translators, it's there, they can do what they want with it. And so, we've had a number of translators who have volunteered to create translations, and then those get published, and then there will be a wave of publicity. And,
Starting point is 00:42:05 you know, I always try to promote translations, and you'll see it sort of catches on for a bit. And usually when that happens, it'll last for a while, and then it'll sort of like hits will drop down again. But it is very persistent. I'm sort of surprised at how even now it's still super popular and I see it at the Maker Faire and I, it's still something that I think, I think it's still useful, right? There hasn't been anything else quite like it. Well, soldering doesn't change. Yeah. That's true. Yeah. Soldering hasn't changed in a long time except for, I guess, lead-free. Okay. So let's say I am sitting down at my soldering iron and I have something I want to solder, a resistor onto a board, a through-hole part.
Starting point is 00:42:55 So relatively easy solder. I've turned on my temperature-controlled resistor. Yeah, Christopher has turned it all the way up again. And I have a bottle of water to do something it doesn't work. That's because the tip is covered with oxide. And just because the soldering iron is hot, it oxidizes and that causes problems because oxides don't transfer heat. So the first thing to do is to take your sponge and take the tip of the iron and wipe it through the sponge. And the sponge shouldn't be like sopping wet.
Starting point is 00:43:47 It should be sort of damp. And that will typically wipe off the oxides that are contaminating the tip. So now you've hopefully got a shiny tip. And if you don't, that means your tip is bad and it's time for a new one. But assuming the tip is shiny, then you place it so that it's touching the pad on the PCB and the lead of whatever the through-hole component is, like a resistor, and you wait. And you typically want to wait, you know, one to three seconds, and that heats everything up. And then you feed in your wire solder, which hopefully is rosin electronics-grade solder.
Starting point is 00:44:21 And for beginners, it's much easier to use leaded solder and you just need to wash your hands afterward. You feed solder into the joint and remove the solder, wait with the iron still on there for another second or so for everything to kind of flow together. And then you remove the iron and you're basically done. And that's how you make a solder joint. There's a lot more pausing in there than I usually really manage. Once you get really good at it, it's all a smooth process. And it's also something where you tend to adapt to the condition. So, some leads are bigger and they require you to be on there longer and you sort of learn how to recognize what's happening in the process. And there's one thing that's not in the comic book, but something that I do all the time, which is before I tap the tip of the iron to like a big glob. So now you'll have a giant ball of molten solder on the tip and you don't know
Starting point is 00:45:31 what to do with it. It's awkward. And if you put it on the board, it's just going to go everywhere. But it actually does make things a lot easier. And the reason is that that little bit of molten solder, not only is it nice, clean, fresh solder, but it's also liquid and it is very good at transferring heat. So a lot of times when you kind of feel like this tip's on there and nothing's happening, just put a little dab of solder on the tip and it'll make a huge difference. Okay, so now we're done soldering the thing. How do I make sure I don't get oxides? Oxides on the tip of the iron? Well, I mean, we started out having to clean it. So what if I don't get oxides? Oxides on the tip of the iron? Well, I mean, we started out having to clean it.
Starting point is 00:46:08 So what if I don't want to clean? I mean, I guess turn it off promptly is part of it. Right. I think that it is, the key is really to use an iron which is at the right temperature. And a lot of times those cheap soldering irons, you crank them all the way up and they will burn the tip up in no time flat. And it really is a function of how hot the tip is. So you want to use a tip temperature, which is hot, but not too hot. And you don't want to leave the iron on for hours at a time. So as long as you're using the
Starting point is 00:46:34 iron, you know, and every few minutes you make a new solder joint, you're going to be fine. And fancy soldering irons will actually turn themselves off automatically to prevent the tips from burning up. But it really is a matter of not leaving the iron on for days at a time because that really destroys tips. Days at a time. Well, you don't want to turn it on in the morning and try to solder at three o'clock in the afternoon. That would be a bad idea. I've had my soldering iron on a 15 minute timer often enough because i yeah i did go through a period where i'd walk away and forget it and so it just made it was easier to have to bump it every 15 minutes than to to worry about it burning down my garage but at the end um aren't i supposed
Starting point is 00:47:18 to like cover it in solder or tin the tip or you can do that yes i don't think so so i i if you are planning on leaving it it does help if you put some solder on the tip before you put the tip back in the stand and or whatever you've got to hold the the iron uh that does make a big difference um i don't remember to do that commonly but i'm also using irons which sort of turn themselves off pretty quickly and respond to how much I'm using them. And so having a good fresh tip makes a world of a difference. And I think if you want to extend the lifetime of that tip, putting some solder on there when you're done is a really good way to do that. And so if I buy a soldering iron and my budget's like 150 to 200, so I want a good one, but I'm not shelling out for a professional grade. Do you have any advice for what I would look for?
Starting point is 00:48:31 I would probably get a Hakko. Those are probably the best kind of mid-range, lower to mid-range soldering irons on the market. I used to say Weller and I was a Weller fan for a long time. I had a Weller iron on my bench for many years. But these days, you can get a Hakko, you know, they're going to be around 100 bucks, you can get them on, you know, a lot of websites like Amazon. And those are just cheap and easy, much, much better. One of the things that I when I went to college, I had always used those 15 watt RadioShack irons when I was growing up. And I went to college and used a Weller WTCPT, which is kind of the standard soldering iron of the eighties and nineties and made a solder joint and realized,
Starting point is 00:49:18 wow, it is so much easier to make a good solder joint with a nice soldering iron that I highly recommend something of that level. You don't need to get anything super fancy. But yeah, Hakko will work great. You can get lots of tips and they're pretty inexpensive. And you were saying we should change our tips regularly. And the tips are only like four or five bucks. They're not. I mean, if you're going to invest in a soldering iron, the tips seem like you might as well buy a couple up front so you remember to switch them. It's true. And actually, the more expensive soldering irons, the tips will last much longer. So that's one other benefit of having a nicer iron is that with temperature control, you won't burn up tips. There was a white paper, I think, published by Hako, I think,
Starting point is 00:50:03 about tip life versus temperature. And once tip temperature goes above about 700 degrees Fahrenheit, the tip life just goes way down. And so for lead free, that's a problem because for lead free, you're typically up at 700 and maybe even higher if you've got a small iron and you're trying to solder, you know, a big heat sink or something like that. But yeah, the other advice is, you know, if you have an iron and it's got adjustable temperature, you really want to keep it below 700, especially if you're soldering leaded solder. And that was actually going to be my next question is, temperature control was always one of those things that seemed very important.
Starting point is 00:50:39 It sounds like one reason is because it'll help my tips last longer. And the other is that different solders require different temperatures. Yes, lead-free solder is typically going to require a higher temperature than leaded. And I have experienced what happens when you try to use a fixed-temperature soldering iron with lead-free solder, and it's just not good. The solder will behave very kind of plasticky. You don't get a good flow and it can be very, very frustrating. Yeah, but we aren't supposed to use leaded solder anymore. I still use leaded solder. I have used lead-free solder and I can use both. I prefer leaded solder
Starting point is 00:51:22 for prototyping just because it is so easy to work with. And especially when you're using surface mount parts, it's just kind of that much more annoying to have to try to pull a multi-leaded chip off a board if it's got lead-free solder. I can do it and I've done it a lot, but I've got leaded solder on my bench. I've thought about switching, but I haven't done it. The truth is that I think a lot, but I've got leaded solder on my bench. I've thought about switching, but I haven't done it. The truth is that I think a lot of the concern with lead-free solder early on was because the alloys in the beginning were not very good and they required even higher temperatures. These days, I think SAC-305 is the most popular alloy for lead-free solder, and it actually behaves pretty
Starting point is 00:52:04 well. I've used it and it's pretty decent, but of course, I've got to use up all those rolls of leaded solder that I've got first. But I'd actually be curious to try to just switch cold turkey and see what that would be like, because I haven't done it in a while. I haven't tried to do that. I worked at a company that only used lead-free solder a few years ago, and it was not nearly as bad as I thought it would be. That's good to know. So you didn't mention one thing, which I never use, so I assume I must be doing it right. So I guess flux is just something my double Es say they need, but they don't really? I think flux is extremely important.
Starting point is 00:52:48 I mean, the thing, I've often been tempted to give a talk, and I would call that talk soldering is hard, as sort of a counterpoint to the comic book and sort of being my own devil's advocate and saying, okay, so here's how you teach soldering to a beginner. But so say you want to do soldering professionally, like what are the things that you need to worry about then? And it turns out that there are a lot of things that can go wrong, especially when you're working with fine pitched surface mount components. And I have a bottle of, I actually have not one, but three different flavors of flux that I keep on my lab bench, and I use it work all the time. And I use flux a lot. Flux is the difference between a solder joint that looks like a beginner did it and a solder joint that looks like an expert or a
Starting point is 00:53:36 machine did it. You get those nice, beautiful, shiny, smooth solder joints, and flux can allow you to achieve that result. It's probably most useful with surface mount parts because all good solder joints require flux. So typically when you're using wire solder, the flux is in the wire. And so it kind of comes for free, but you can't control how much of it you get per unit solder. So when you start getting advanced, you actually want to have an independent control because say you've got a chip and you've got a board and both of those things already have solder on them. So you don't want to add more solder because then you'll have a big blob, but you do still need flux because the flux is what cleans the oxides off the surface.
Starting point is 00:54:19 And anytime you solder, you need flux. So having some liquid flux, be it in a pen, or I like the little droppers so I can drop on one drop exactly where I want it. You just put a drop on and then you use your hot air gun or your soldering iron to reflow the part and it makes all the difference. I mean, it really will be the difference between success and failure when it comes to components like QFNs that don't have any leads. So yeah, I'm a big proponent of Flux, but the truth is that for a beginner with through-hole parts, you could probably never use Flux and you'd still be okay. I suspect my failure to use Flux is what holds me
Starting point is 00:54:59 back. I totally can do as many header lines as you want. I can do through-hole parts. I can do a little bit of surface mount as long as it's not hard. You know, like the three-pin FETs. It can do those usually, especially under a microscope. But then afterwards, they look like they were sort of smashed in by a giant using railroad ties and nails. They do look horrible. That's when I need flux, that point where I can sort of do it, but not enough. I think that ultimately you want your solder joints to look good from the beginning,
Starting point is 00:55:40 but the truth is that sometimes they don't look good. And so a lot of times just putting a drop of flux on the solder and reflowing it with the iron, it will look perfect when you're done. And a lot of fluxes, you don't even need to clean when you're done. So you are totally done. You'll have beautiful solder joints. A lot of times I get into a situation where I've got solder that's bridging two leads of a component. Like I've got a microcontroller and it's got a lot of leads and I've got this big blob of solder. If that blob is not so big that it needs to come off using solder wick or solder sucker, you can oftentimes just put a blob of or a drop of flux on that solder and then reflow it. And the flux will allow the solder to kind of go where it wants to go, which is not in between the leads, but on the leads, the surface
Starting point is 00:56:25 tension of the lead will actually pull the solder away from where it's bridging, and the bridge will actually go away. So flux is a very useful tool. And I really do think that is kind of the secret weapon. Like if you are having trouble, chances are you've got some kind of an issue with your process, like your iron's not hot enough, or the tip isn't clean enough, or you're not letting it leave on the joint long enough to thoroughly heat it up. But if you want to sort of patch it up and make it look good, a drop of flux will help a lot. How do I choose what kind of flux to get? I mean, I go to Amazon, I search for solder flux, it gives me 4,301 results. How do I choose? I mean, I click prime and it'll give me a few less, but... So, there are a lot of flavors. And one thing I will say is with soldering,
Starting point is 00:57:14 there are a lot of people that have a lot of strong opinions about soldering. And I fully expect there are people that are shaking their heads with regards to what I'm saying. And I'll say that there is somewhat of a personal preference, but there's also one of the issues with flux is that cleanup is a real pain. And if you use a lot of flux that is not a no clean variety, you can have to spend a long time scrubbing with alcohol or flux remover,
Starting point is 00:57:39 which is typically these horrible nasty chemicals to try to get it off the board again. And so you typically want to use the lightest flux. You know, flux is kind of like maple syrup, like there are grades and there are what's called the solids content, which is how much rosin for a rosin flux. It's how much rosin is in the flux. If you use a flux with a really high rosin or solid content, it'll be kind of tarry or sappy because rosin is after all derived from sap. Um, but if you use one that has very little, it will not work as well because there's less of the active ingredient, but it's easier to clean because you don't have as much of this sap on
Starting point is 00:58:16 the board. I like, uh, Kester 186, which is unfortunately only available in gallon containers. But actually, you can buy it on eBay in small vials. And there are folks on eBay that do like Xbox repair and stuff like that, that will sell you Flux that usually you can only get in a gallon. You can actually buy a small amount. And I would totally start with Kester 186. It's my favorite because it's kind of a medium grade. It's like a lightish color.
Starting point is 00:58:45 It's not as dark as there's an MG Chemicals Flux. I think it's $835, which you can buy on Amazon. You can get it at any electronics store. It's a bit heavy duty and sticky. And so that can make it really annoying to clean up. But Kester 186 is much lighter. There is another popular flux which is kester 951 i think it is which is a very clear no clean flux i don't like it but it's popular with the crowd of i think the xbox repair crowd because it requires no cleanup you don't have to clean anything but as a result it has very low activity which means it's not as good at cleaning the stuff that you're trying to solder. So of course you can kind of go down this rabbit hole, but I would say starting out, you know, if you're just going to go to your local electronics store, this MG chemical stuff is certainly better than nothing. And you could
Starting point is 00:59:33 probably thin it with alcohol if you wanted to. But this Kester 186 is my go-to flux that I like a lot. I like rosin fluxes because I understand how they work and they smell nice. I like the smell. It's probably not good for you, but it smells good. No clean fluxes can be kind of nasty. And they also tend to degrade soldering iron tips, which is not good. Well, it looks like your Kester 186 now comes in a pen form. I do not like the pens. People, you know, this is another one of those religious wars, I think, around soldering. There are pen fanatics. I mean, I think the pen is better than nothing, certainly. And for simple stuff, it works well. The problem with the pen is you have to push
Starting point is 01:00:17 on the thing that you're trying to apply flux to. And when you're dealing with like a tiny surface mount part, that's not always a good thing because you push it out of place. Or if you've got like liquid solder paste, it's just going to make a mess. So I usually use it in liquid form, but the pens actually do work. If you've never used flux before, I'm sure that a pen will be amazing. You'll love the pen. Well, the whole, you know, push things, that sounds like something I would do. It's been a long road, but after many, many, many times, I am learning that solder is not a construction material, that I really should have my parts be able to hold themselves in place, either with the clampy third-hand tool or with super glue or something. But I shouldn't just hope that once the solder's on, they won't spring apart because they will always spring apart. That's very true. You have to have stuff where you want it.
Starting point is 01:01:12 The solder's job is just to hold it together. It's not, as you said, to be a structural element. Okay, well, I think we're almost out of time. Do I have time for a few more questions? Okay, Chris is nodding, not as though he has a microphone in front of his face, too. You went to Maker Faire this year. What was your favorite part? My favorite part of the Maker Faire is definitely the people at the Maker Faire. For me, the highlights were the Hackaday meetup on Saturday night and the Osh Park bring a hack dinner on Sunday.
Starting point is 01:01:46 I actually think that in in, if I went to just those two things, I would probably come away pretty satisfied. Which I think says a lot like it's really for me, this is my one chance a year to see a lot of people that I really enjoy being around and talking to. As far as at the fair itself, I was impressed by a couple things. I've sort of become amazed by computer vision. And there were a couple projects where computer vision is being used to make things easier for prototyping. And one of them is Glowforge. So Glowforge is a laser cutter. The company's based in Seattle, which is awesome. And they're using computer vision to help you plan out where to do your laser cuts and also to allow you to register your designs on existing stock. And I think that's awesome. And I wish that the big companies like Epilog would innovate in that way. And I'm really excited to see what Glowforge is. I've never used one of their printers, but I really want to try one out. And there's also another project, Open Pick and Place, by another Seattle local, Jason Von Nieta. And he's using computer
Starting point is 01:02:56 vision to help so that you don't need to put your board in on the pick and place machine in a particular orientation. You just throw the board into the machine and using computer vision, the pick and place machine figures out the orientation and knows where to put the components using fiducials on the board. That stuff is just amazing to me. Um, and I'm particularly excited that the underlying tech is all open source, which is awesome.
Starting point is 01:03:21 And listeners, if you're trying to remember why Glowforge sounds familiar, that was show number 125 when Dan came and told us about the company. Okay. Oh, Christopher, you had a question. Oh, from the beginning. From the very beginning. When I asked you what you wanted to be when you grew up,
Starting point is 01:03:40 you said an electrical engineer. And I was curious when you first kind of knew that or what experience you had, and maybe you can't remember, but that led you to that. Were you interested in ham radio? Did you come across something? I used to play with extension cords. And that's really from a very early age, I found light bulbs and batteries and things like that to be really interesting. And my parents have stories of me basically trying to burn the house down by, at a very young age, plugging as many extension cords together as I could, because I was amazed that electricity could travel so far over a wire. And so I'd get all the extension cords in the
Starting point is 01:04:23 house and I'd put them all together. And then I would light a Christmas light or something on the other end. And this is when I was very young. So I have memories of smoke from when I plugged a frayed cord into an electrical outlet, but that wasn't scary enough to steer me away from continuing to do that. And so literally, I've been interested in electricity all my life, as long as I can remember. And I eventually got turned on to Forrest Mims and the Radio Shack, you know, 40 in one or 101, those little those kits with the little springs, you know, where you put stuff together by sticking wires into springs. And I think, ever since then, I've been interested in electronics, and not just electronics, but circuit design and wanting to be like Forrest Mims and build circuits, because I was amazed that you could just make circuits do what you want, even if it's something as simple as blinking an LED. That was really amazing to me when I was young. I can see that. I really fell in love with embedded systems when I went from writing software to do things to moving a motor. And it was like, oh, this actually
Starting point is 01:05:34 changes things. So yeah. You do RF and that is totally black magic, right? You know, I've, I will say yes and no. So I'm I am an RF engineer by training, I went to school for to be an RF designer, I've worked in the RF industry for a few years, I've done a lot of RF design at valve. And I, there are still aspects of it, which mystifyify me I wish I knew more about how to make good antennas because that is a big thing in consumer electronics and if I knew how to do that it would save me a lot of time
Starting point is 01:06:14 I think that it's there's an element of mystery to it but I think it's once you get to the point where you understand that a lot of the secret is that everything looks like an inductor at sufficiently high frequency, a lot of the black magic goes away. And you realize that the secret is to have good grounds and to understand that while nothing is a perfect ground, everything has some inductance, as I said. And so everything's moving a little bit when you get to a high enough frequency.
Starting point is 01:06:45 And as the frequencies go up, it gets harder to have a good ground. That's where people run into problems. And one of the things that is really important for consumer products is what's called regulatory compliance, which is you want to sell your widget and the FCC won't let you unless you pass a bunch of tests. And there are a lot of products which get to that stage of testing and then fail miserably because of really fundamental aspects of the design. And so that's another thing where you're making large scale consumer devices, you kind of have to do that stuff up front. And we started regulatory compliance testing very early in development, because we wanted to know what the pain points were for us. And that paid off
Starting point is 01:07:25 big time. And this is stuff like ESD. If you shock a product, you don't want it to fail. But there's also other stuff. And Europe is really picky about what products must comply with in order to be sold in Europe. And a lot of that stuff is RF related. So yeah, I could see how it would be considered black magic because your product does the thing it's designed to do. But you can't sell it because it's emitting this weird interference, and you don't know where it's coming from. But the truth is that, you know, there there is a lot of especially physics behind that stuff. And everything is an antenna. That's the other the other rule of RF, I guess, is everything's an inductor, and everything's an antenna,
Starting point is 01:08:04 even things that you don't want to be an antenna are an antenna. I'm always amazed at how antennas used to be really, really important and big. And if you had a GPS, you absolutely had a big antenna. I mean, maybe big is only like four inches in this case but now i i can put in a wi-fi board and a ble board and a gps all in something the size of a wristwatch and they should fight there's actually i think two things there number one is so you you combine all these things into a single package and they should fight and they do fight and it depends on how well they're designed but you can design them to have better isolation so that one doesn't see the other but i think in a very tiny design that's really hard and so i think the reality is that a lot of
Starting point is 01:08:55 wireless protocols are really good at hiding if issues at that level like issues at the physical layer where the radio just didn't hear you because the Bluetooth transmitter was so loud that it couldn't hear the receiving station. You just make sure that within some amount of time, you have another opportunity to get a clear signal. And I think Wi-Fi works that way. Bluetooth works that way. And unless you're really looking for latency down to the millisecond, you can hide a lot of evils and a lot of issues down at that level that won't be apparent until things get really, really bad. I think the other aspect is that antennas at that scale don't work very well, and that wireless systems are really good at compensating for
Starting point is 01:09:40 antennas that are not good antennas. There have been huge advances in design, like cell phone antennas started out as a whip that you literally pulled out of the top of the cell phone. They're not like that anymore. They're much more sophisticated. And they're invisible to the user now. They are, but they're not invisible to the RF system designer that worked on that product. And I've had hands-on experience with the challenges. I was one of the guys that designed the wireless link in the Steam Controller, and it's hard to get right. There are a lot of things that can go wrong, and there's a lot of sources for interference
Starting point is 01:10:20 in modern computers. Computers are interference- interference generating machines, and especially the kinds of computers that you'd find in a typical home that someone has maybe scratch built from components. Those are oftentimes computers that would not pass the regulatory specs that a product like the Steam Controller is required to meet. There are certain exemptions for home-built computer systems that are built of compliant components. So you've got a video card that meets compliance, you've got a motherboard that meets compliance, but there isn't really any
Starting point is 01:10:54 guarantee that when you put them together that they still meet. However, that is how the FCC law is written. It's like a systems integrator exception. And so there's a lot of interference in the modern home. I mean, you've also got microwave ovens and things like that, that are in the same frequency range. And so for a system designer, you have to guarantee that your product works in a variety of conditions. And I think the truth with RF is that you are very rarely in an ideal state. Like there's always something that's compromising the link. And, you know, it's not like it's a satellite where you've got a perfect line of sight and you've got an antenna, which is pointed exactly at the thing. And likewise, the thing that you're
Starting point is 01:11:36 transmitting to is pointed directly back at you. I think consumer electronics design with RF is all about compromises and trying to be innovative about how you work around those compromises or just try to minimize them through better antenna design or better protocols that are insensitive to low amounts of packet loss and stuff like that. You do ham radio, right? Yeah, I do. In fact, I just built an antenna. Well, it was a kit. I put together an antenna kit this weekend. I've been involved in ham radio. Since college, I had a good friend that was into ham radio, and he talked me into it. And I built 10 gigahertz radios when I was
Starting point is 01:12:19 in college, there was a local group in the San Diego area that was into microwave radios. And I think they were mostly ex-military guys, because that's where a lot of this stuff was being used, but also engineers at local companies that were building cell phones and stuff like that. And so they were good at repurposing existing devices to use for amateur purposes. And I thought that was amazing. You could get this essentially trash, like discarded electronics and modify them to use on the amateur bands. And so I've always been into amateur radio since then. What kind of kit did you build?
Starting point is 01:12:53 What was your antenna? It was a dual band, two meter and 440 megahertz VHF UHF antenna. So this is like a base station antenna. I bought a kit on eBay. There's an antenna designer named Ed Fong. I can't remember his call sign, but he's in the Bay Area. And he sells antennas online and you literally take his kit, you get a PVC pipe for two bucks at Lowe's, you chop it in half, you stick the antenna inside and you're done. So it wasn't too
Starting point is 01:13:23 much of a kit. I mean, it wasn't a hard kit to build, but it works well. And I'm actually in the process of designing another antenna, which is a quadrifiler helix antenna to receive weather satellite broadcasts. And there was actually a project at the Maker Faire, which inspired me to want to do that. There are these old NOAA weather satellites, which are still orbiting, are kind of getting obsolete, but still transmitting. And they just transmit essentially faxes from space, which show a down facing camera, show you where you are and what the weather system looks like around you. And I think that's amazing that in this day and age, you can still get these analog radio broadcasts. And I think there's a risk that they're going away. So I
Starting point is 01:14:08 kind of am in a hurry to build this thing. I was trying, now I'm all interested in that, we're almost out of time. I was trying to get to the two meter antenna, which is physically large and yet and and i i understand from the the ham radio perspective and we could talk about the equations and whatever and power and blah blah blah but the capability of that and of the steam controller or a cell phone or any of our little tiny widgets that let us talk to lots of things with these tiny antennas because of redundancy in both code and in the network. Well, it's also frequency. So, you know, this two meter antenna, which is centered around 144 megahertz, is a low frequency, and thus it's a long wavelength. And antennas are efficient as a function of wavelength to make a good antenna it's got to be some multiple of a wavelength and if you
Starting point is 01:15:12 shorten it below that they don't perform as well so the way cell phones work is that they're at higher frequencies in the gigahertz and likewise all this stuff like steam controller and kind of similar devices are in the 2.4 gigahertz band. So you can envision you've got an antenna which is dramatically smaller at 2.4 gigahertz than one at 144 megahertz. That makes a huge difference. And that's why there's a push to use higher and higher frequencies in portable devices.
Starting point is 01:15:40 The challenge with high frequencies, though, is that you can make great antennas because they're small, but those signals bounce off of walls and are attenuated by walls. So for indoor stuff, as long as you're in the same room, it's okay. But if you go a couple of rooms away, your performance drops dramatically. The loss of that signal goes up considerably at higher frequencies. So while your old style, you know, your old analog cell phone, which no one has one of those anymore, those worked pretty well over long distances, your cell site could be pretty far away. Whereas a modern cell phone, which is going to be around a gigahertz to two gigahertz, you need to have a lot more cells, and you need to be a lot closer because the signal doesn't travel as far. Damn you, physics. Physics is a big pain when it comes to RF systems. And RF systems are an interesting way to learn about physics. And I think that's another thing that's really interesting about amateur radio is that if you talk about HF, so HF is the low frequency stuff, like around 30 megahertz and below. Those signals
Starting point is 01:16:46 are influenced heavily by the atmosphere and what's happening on Earth and the Earth's atmosphere and ionosphere. And so by transmitting and receiving on those bands, you actually get a firsthand look at how the ionosphere is varying. And that's all tied back to solar cycles, like where the sun is and the activity level of the sun. So there's a lot of physics just in there, like your ability to transmit a signal from Seattle and have it be received in Boston is a function of how active the sun is at this time of year. And I think that that is kind of amazing, like to have that level of effect on something that's very tangible, like very noticeable, like you can talk to them or you can't. And it's a function of how active the sun is. I think that's the thing that for me, several years
Starting point is 01:17:37 ago made me want to get into the low frequency stuff, the kind of more traditional ham radio, which is kind of at your short wave frequencies. And this is why people think that RF is black magic, because sometimes you can talk to Boston and sometimes you can't, and the sun is responsible? Maybe it's really that physics is black magic. Well, that's long been understood, yes. Okay, so just one more, two more questions. What is your favorite kit that you are selling? So I currently only have two kits. So it's an easy, easy choice. And it's definitely, I sell a Geiger counter kit. And that is by far the most successful kit that I have ever made. And it's thus my favorite because it has made so
Starting point is 01:18:26 many people happy building them. And it's been awesome to be able to be a part of that. Okay, and this is Open Geiger. Is that right? It is the Mighty Ohm Geiger Counter. I think Open Geiger is another project. Oops. This is the Mighty Ohm Geiger Counter is just my Geiger counter, for lack of a clever name. And that's the one that I sell on my website and also through the Makershed and Adafruit. How much does it run? It is, if you buy my site, it's $99. $95, yeah.
Starting point is 01:19:05 $100. Does it come with radioactive material? No. I've steered way clear of collecting. Like, if I were to sell a kit with radioactive material, that means that I would need to have a lot of it at one time, and I'm not into that. I don't know how you should. If you put it all in one place and pack it really tightly, that's really good.
Starting point is 01:19:26 No, actually, I like the fact that it is safe. It is like no questions asked. There's nothing unsafe about it, except I guess the high voltage power supply, which could cause a mild shock. But I thought about that when I first sold it. And I thought it just makes things so much easier if I'm not shipping people radioactive material around the globe. That just sounds like a bad idea. Sure.
Starting point is 01:19:52 It does sound like a bad idea now that you phrase it that way. The truth is you can actually get radioactive stuff everywhere, so it's not that hard to come up with it. I'm just trying to think if there is any in the house, and if there is, I don't really want to think about it. Okay. Smoke detectors. That's true. Yes. Jeff, so we can let you go off to barbecue fun. Any last thoughts you'd like to leave us with? So, I don't have an incredibly insightful last thought, but I do have something which is based on my unique experience. All I can really say is a good design is one that has been reviewed carefully at many stages through manufacturing. And a lot of times it's the really simple stuff that can cause problems. And so I think my sage piece of advice is always check for floating inputs.
Starting point is 01:20:48 That is excellent advice. Thank you for being with us. Oh, thanks for having me on the show. My guest has been Jeff Kaiser, the person behind Mighty Ohm and an electrical engineer at Valve. Thank you also to Christopher for producing and co-hosting, and of course you for listening. Please check out our blog and newsletter. You can find it all on Embedded.fm, along with a contact link if you'd like to say hello.
Starting point is 01:21:17 We'll be here next week. In the meantime, a final thought to tide you over. Wow. Okay, how about this one? Be thou not technical with me, or else thine input valve may swift receive a hearty helping of my golden foot. That was from Ian Dosher, who wrote William Shakespeare's Star Wars, Verily a New Hope. Embedded FM is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show, we did not put them there and do not receive any revenue from them.
Starting point is 01:22:02 At this time, our sole sponsor remains Logical Elegance.

There aren't comments yet for this episode. Click on any sentence in the transcript to leave a comment.