Embedded - 174: It's Not Weird

Episode Date: November 2, 2016

We spoke to Evan Shapiro, CTO and cofounder of Knit Health (@KnitHealth), about baby monitors, IoT security, neural nets, and professional poker. The Knit Health Kickstarter ends November 17, 2016. Evan recommended Google Tensor Flow and Python's Theano for an introduction to machine learning. (If those sound familiar it is because Kat Scott mentioned them as well.) Evan also suggested that if you'd like to know more about the history of neural nets, check out this post by Audrey Korenkov.  If you'd like a gentle introduction, check out a Narwhal's Guide to Bayes' Rule. Evan mentioned some videos he did about poker, they are on Card Runners (NOTE: it is a paid site with free tastes). Final quote was from Neil Gaiman's excellent Graveyard Book.

Welcome to Embedded. I am Alicia White. My co-host is Christopher White. What do you do if you're knee-deep in a Kickstarter and building a Linux camera IoT project? Well, if you're Evan Shapiro, your boss signs you up for podcasting. Hi, Evan. Welcome today. Welcome today. Wow.

Welcome to the show. Hi, Kristen, Alicia. It's great to be talking to you today. Could you tell us about yourself? Sure. So I'm an electrical engineer and software engineer by training and trade. About two years ago, myself and two former colleagues from IDEO left to found a company called Knit Health. And I'm the CTO, and we're building a really smart camera to measure biometrics and health. And that is what I want to talk to you about. But first, we're going to try this thing called lightning round. And that's where we ask you questions, and you hopefully answer short things.

And then if we are behaving properly, we don't ask you why and how and all those questions. And as many of our listeners have pointed out to us, we are terrible at this. Yes. Thank you, Josh. I'm going to be bad at this too. So let's do it. Okay. Worst formal version control system you've used ooh uh subversion there's a lot of asterisks let's go into a more interesting question though how many times should podcasters have to do an introduction before the show starts uh once really if they're professional oh least favorite planet.

Uh, least favorite planet. Really? Um, fast, fast, fast. Uh,

Neptune. Uh, should we bring back the dinosaurs? They're already here. They're flying around us. Our feathered friends. What language should be taught in first year university course for computer science programming uh the premise of that

question is flawed you should learn your first computer science language in elementary school when you're learning french spanish german whatever your first foreign language is and it should be something declarative and graphical, probably. Cool. So like Scratch? Logo? Sure. Okay. Logo's a good one, yep.

BeagleBone Black or Raspberry Pi? I haven't used the new Raspberry Pi, so I'm going to have to say BeagleBone Black, but I am interested in trying out the Pi 3 or whatever they're up to now. Favorite fictional robot? The little robots from Batteries Not Included. Oh, yes.

The second probably. I'll be up there. The other person who I know who's seen that movie. Going pure emotional on that one. Would you rather explain the difference between NPN and PNP transistors or API design? I would much rather explain the detailed fundamentals of how transistors work because I actually know something about that. Oh, sorry. I got out of order.

No, no. Do you? Go ahead. Okay. Tabs or spaces? Spaces. How many spaces are in a tab i can i plead the fifth we may try and hire another developer someday so five okay you mentioned knit health that's where you're at now yep and that's k-n-i-t like knitting socks correct okay i just wanted to make sure because

this is audio only and yeah what is knit health um so the company is a startup in san francisco we're up to nine full-time people now a lot of us have kids we're new. I have a two-month-old. My co-founders have young kids, a three-year-old and a four-month-old. And we are interested in computer vision, in hardware, in health in general. And so we decided to make this company to try and build a camera with some embedded artificial intelligence that uses a lot of novel and new computer vision techniques to measure health in general. And we're starting with things like sleep and respiration. That seems medical-ish, is it? It could be someday. Right now, it's all on the consumer side. So we made a kind of deliberate

decision when we started out to stay on the consumer side of healthcare. It lets us get something out faster, hopefully reach more people with it. But yeah, some of the technology we're building, I think could be medical grade eventually, and we could run clinical trials and get FDA approvals to put it in regulated devices, but that takes a lot of time and a lot of capital to get there. Yes. I'm not a parent, so forgive me for dumb questions about babies and parenting, but what good is it to track babies as they sleep? So, sleep, I think, is becoming a prominent topic of discussion for parents of kids of all ages. How much sleep we get, what kind and quality of sleep we get has a huge impact on how we are during the day, especially for young kids around behavioral issues in school.

Things like ADHD, where there's this huge uptick in diagnoses, a lot of that now is being tied back to relating to sleep somehow where you can have certain issues or certain chronic conditions affecting sleep a lot and you can also have poor sleep manifesting in the day as tired or irritable behavior. There's a lot of developmental milestones around sleep. So it's exciting as a parent the first time your baby sleeps through the night for the first time. You question if they are waking up and putting themselves back to sleep, if they are waking up during the night, how long they're staying up that you don't know about. And you're constantly trying to get them to sleep and to sleep better. And it's hard. And it's, you know, it really is a struggle of trying lots of different things, adjusting bedtimes,

room temperature, what they're wearing, all this stuff. And unless you want to stay up all night with the notebook, there's no way to close the loop on that feedback. And so that's a lot of what we're trying to do is just give parents better tools to do what they're already doing. And a lot of that, if you have a young child that's focused around sleep. And so does your device work on kids as well as babies, or is it mostly babies at this point? It works on humans in general. It works on adults. The constraints right now is that we don't handle more than one person or person and pet per bed. We can do that someday. But if you put it on an adult, which we've done on teenagers, 10-year-olds, young toddlers, young infants, it's smart enough and has a generalized enough understanding of what a person is that it'll work. It's really designed for babies between newborns to

maybe three years old. That's kind of the starting target market. But we definitely have users who have older kids that use it for various reasons, and it'll work on anyone, really. I could see it be very useful for elderly care. Yep. We see that as a potential future vertical as well. So the zero to three market has no expectation of privacy during their sleeping hours. Is that a worry for you, or is that just a parents have prerogative there and that's okay? Yeah, I think it's the parents prerogative and there's also, there's already an expectation set there where parents are already buying cameras and putting them on their kids

as baby monitors. So we're kind of disrupting that familiar purchasing point and starting with something where you don't have to have a privacy discussion because it's already kind of a set societal norm to do that. Okay, so what is different about what you're doing versus the cameras that exist now? The cameras that exist now allow you as a parent to see your child whenever you want. And they might record video during the night that you could go back and look through. But that's about it. It's a camera as an extension of your own eyes. What we're doing is different because we've built into the camera a real level of intelligence. It's got a deep learning neural network in it that's capable of understanding the difference between a child and a stuffed animal. It can recognize the child in the scene, know where they are, know where different parts of their bodies are. And from

there, we can do things like measure very fine motions so it can continuously measure their respiration and by looking at changes in their respiration it can understand if they're entering or exiting different phases of sleep can understand if they're awake or asleep and it can communicate data to you not just in this raw video stream but in human concepts. So the camera itself can share with you a clip that it has labeled as a small video clip of your child waking up in the night. It's recognized that that might be something of interest. It has all the intelligence needed to understand all those concepts and it can surface those to the parent at a much more kind of approachable and consumable level than if you're trying to do the same thing with just a video recorder. Now, most of the sleep monitor kind of technology

that's out there for adults, it's kind of a post facto, you wake up in the morning, you look at your app, and you get some sort of graph that says, you know, you didn't sleep very well, you're going to feel terrible, and then you feel terrible. But this seems like since it's monitoring somebody else on your behalf uh does do you give real-time updates like okay your kid's really restless they might not be making noise but you might want to check on them or is it still kind of a post report okay last night went well here's why yeah there's there's a a whole range from the real-time immediate updates or notifications to your two-month-old or three-month-old just rolled over onto their stomach and that's something you might care

about as a parent, or your three-year-old is making an escape from the crib, or they're awake and you have decided you want a notification if they wake up. So there's the real-time immediate aspects. There's the daily check-in of how was last night. Was it a good night? Was it a bad night? What went on in general? And then there's this kind of long-term trend tracking of understanding how their sleep is changing over time.

You said you have a small one and many of your colleagues do. Has testing these changed your actions? Yeah. So for my wife and I, I don't want to plug the brand of the sleep suit directly, but there are these things called sleep suits that you can buy and put your baby in and they're basically uh like really puffy snow suits that kind of immobilize the kid and they like that it some kids like that it can give them comfort and they can

sleep more soundly because of that this used to be called swaddling it yeah so we tried swaddling to start with which is you know wrapping them up with a blanket, essentially, real tight, and a bundle, a little burrito. And she was doing all right, but she was waking up a lot during the night. Not abnormal for a child her age, but we decided to try one of these sleep suits just in case. And we could immediately see the day after in the data, in looking at how her sleep was in that kind of really high-level summary graphic, that it was way better with the suit than it was with the swaddle. And so we made an immediate decision to use that instead. So it was a real practical kind of small parenting decision. But doesn't, actually, wouldn't either one of those swaddling or the sleep suit affect the camera's ability to see them breathe? No, actually. So, we can see respiration through swaddles, through blankets, through sleep suits. We've had kids bury

themselves in piles of stuffed animals animals and we've been able to detect the respiration signal. So we're, we're pulling out signals that are in the video that are really too small to see with the unaided eye. Um, but with, uh, a bit of signal processing and the context of knowing exactly where the child is and what features in that video are part of the child versus a stuffed animal or part of the crib or the surroundings, we can actually extract that information. So if I was doing this, I would think that you could take a picture and maybe every, I don't know, in minutes or every end seconds, you take a master picture and then you you subtract and add the next one to figure out if you're breathing

in or out i mean okay you take picture a and then some period of time later you take picture b and you difference them yep and that difference is how much you're moving and you can even determine which direction you're moving a little bit yep okay so so that makes sense and and that's sort which direction you're moving a little bit. Yep. Okay. So that makes sense. And that's sort of how you're doing it? At a very high level? Not exactly. But yeah, that's basically what we're looking for are differences in frames and understanding how those differences relate to motion. Gets a lot harder when it's at night and you have a noisy sensor, when the crib itself

might move or the whole scene might vibrate as someone's walking by and shaking the crib with their footsteps. So there's a lot more to it in depth. But yeah, in general, it's looking for very tiny changes in the images that are coming as part of the video. Do you have a specialized camera? Is it IR or something weird that lets you see in the dark? It is IR that lets us see in the dark, but it's not weird. So we just have a normal consumer-grade camera sensor.

It's in a lot of webcams, night vision security cameras. And what we do is we have infrared LEDs that are emitting in near infrared. So nothing used for like thermal imaging or anything like that, but just red enough that you can't see it or can barely see it as a person. And that illuminates the scene

and the camera sensor is sensitive to that wavelength. that you can't see it or can barely see it as a person. And that illuminates the scene, and the camera sensor is sensitive to that wavelength. And so we basically get just the black and white image you might typically see in a security camera at night. So it seems like you've got the classic signal processing problem of a weak signal that's sometimes dominated by noise. Is that accurate? Yep, that is accurate. Okay, so you attach this camera to something,

which runs Linux, and then you're not just connecting it to the internet. Tell me what your camera connects to, and then I want to talk about security, and since you already told me I can, we're going to talk about it a lot. But I want to get the hardware down. I'll try not to share our AWS keys, but yeah, we can dig into security. Yeah, so the camera is, our main processor is a tablet class applications processor.

It's a bottom-of-the the line Snapdragon processor from Qualcomm. We have a external image signal processor that is interfacing with the camera sensor directly, and we're actually accessing data from the main processor on USB streams. And we get things like an H.264 encoded view of the scene or little cropped native pixel resolution views or downscaled views of the whole scene, things like that, that feed into our different pipelines. that it's punched through a router or has some protocol to do some natural reversal and handshaking to send all of its data somewhere else. Because of that small signal and potentially a lot of noise problem, one of the things we have to do is a lot of local processing.

So the algorithms that we're using, the signal itself is really sensitive of local processing. So the algorithms that we're using, the signal itself, is really sensitive to compression artifacts. And so if we encoded the video sufficiently to compress it enough to be able to send it over a home internet bandwidth, we would destroy a lot of that information. And so we need to do a lot of our signal processing

on uncompressed raw video. And so that has to happen locally. As a result of that, we end up with a pretty powerful processor on board. It's not your typical Ambarella just making packets for the internet. It's a processor designed for cheap Android tablets. So we have a lot of overhead there.

And so instead of just continuously sending that video to the internet, the camera does a bulk of the processing, it does a lot of the filtering of the media, making clips, making time lapses, if that's what we want to do. And then most of the communications with the broader network are done in transactional HTTPS calls. And so we can rely on more kind of traditional HTTP levels of security and not have to have ports exposed on your router or continuous streams that could be sniffed or anything like that going on.

Okay. So it is Wi-Fi, okay and it does go up to it does it it takes the data it it makes it much smaller using intelligence and then it goes up to the cloud um yep and what sort of information do you send to the cloud? Like, do you send minute by minute the baby is sleeping, or do you say only alarms that need to be transferred back to the parent somehow? Yeah, so that's a good question. We send, mostly we send state changes. So if anything has changed, that might be meaningful. If they've changed position significantly, changed their sleep state, changed their presence in the crib.

If the lighting in the room has changed, there's a loud noise. We'll capture the state change and send that up. If there's some kind of notification that wants to go to the parent, we can send that in a number of ways as well. But it's not unless someone wants to be looking at a real-time live stream of the camera from outside of their home network, we don't really see a good reason for continuously sending that video to the internet. And so we try and avoid as much as possible sending data to the cloud. We try and push as much as possible to the edge and send things to the cloud when someone might want to view them remotely, when you might want to save it for posterity, or you might want to integrate it with other sensors coming from other places.

And in this particular case, that's not too often. You know, we could send a JPEG once a minute that would give you a good indication of what the current state of the room is. We can send immediate updates if we detect any change of presence of the child or change in their sleeping or anything like that. So there's not really a strong need to have this continuous video stream going anywhere. That's sort of reassuring. I mean, I guess I don't really want continuous data streams going from my house to anywhere. Christopher, don't tell me what's already in place. We don't have anything. Oh, okay. But you can get a video stream. Parents who are on the same Wi-Fi network can check in with their kid. Yeah.

And so that's what we're trying to do is give you at least a baby monitor. If you're considering buying a video baby monitor, everything that that can do, we can do. And so part of that is if you're at home and you're in a different room and you want to have a screen up somewhere that you can just kind of glance over and check in, it's all on the local network, a real-time stream that's very low latency and very high quality because we're not using any of your internet bandwidth to do it. So this seems like a case where you have multiple goals that overlap and you end up with a solution that's favorable for all of them by good design. Because it's not just a privacy concern that, okay okay we're sending as little as possible over the internet we're not sending huge video clips but it's also a scalability thing

right you don't want you don't want to run enough servers to manage all of that um was that a happy accident or was that some sort of convergence of thought that okay how can we align all of these things to come up with something that's that's actually going to benefit us in multiple ways um it was a bit of both so the reality for us as a startup is that we did not take the decision to make custom hardware lightly like that's a really tough sell to investors in general. There's a lot of people who just will not look at a hardware company. There's too much upfront capital investment.

You have to spend a lot before you can demonstrate any kind of traction. And so the real decision for us to do this the way we're doing it, where we're doing a lot of the onboard local processing versus just using a web camera to get off the ground really came down to the necessity of having to do it we we have to be working on the raw video in order to do it at all everything after that is really a happy accident where once we make the commitment to custom hardware, yeah, it's actually a cheaper system overall. If you want to compare cost per CPU cycle of doing this on a camera in someone's home

where they're paying the power and air conditioning bill versus doing it on a big hosted server room somewhere, we're pretty confident that in the long run, this is the better way to design a system like this. We think more people are going to start doing that with the dramatic reduction in cost and embedded processing power. Even the nature of the deep learning models that we use are much better suited to run on lots of distributed small processors

instead of large, powerful clusters. And that's because the model can really be specialized to a given child so it can be tuned to understand particulars of that child what does their pacifier look like what does their pajamas look like and it also wants to keep around a lot of context and so if you imagine having some big neural network that wants to load up hundreds of megabytes of memory per child, that becomes extremely cumbersome if you're trying to do that on a single processor per multiple clients. And so there's a lot of things that just push us towards this nice pattern of having most of the processing happening where the data is being

collected and then sending away what we actually want to send away instead of send away everything for processing i agree with you this is the direction embedded systems will be heading as the prices drop in the power drops everything gets smaller and better and cheaper generally eventually. It does make sense to put the intelligence on the edges where you can do more with the data. But how did you create these models? Did you start out with putting it all in one place and then figuring out, I mean, tuning per child. I'm surprised you went that direction. I would have thought you would have hoped to avoid it, but maybe it's necessary.

Yeah, we do avoid it generally. It's necessary if you want to, I guess the best way to think about it is that if you're a person looking at these videos, there's a lot that you could do without knowing anything about the child, but you'd find there's a lot that you could do if you knew that particular child. And so we think we can build this deep learning system where it can perform about as well as a person in a lot of the tasks we're asking it to do. And so, that might be recognizing that a child is in a crib and where they are. You don't need to know anything about a particular child to do that. But individual children might have certain stuffed animals that they have, and you might be able to recognize if you know something about the child when they have a different stuffed animal

in the crib or when they have a different pacifier or when they do have a pacifier don't have a pacifier so there's all these specialized tasks that you can start to introduce but you really need to understand the history and context to be able to do that that didn't really answer your question though which was how did we start and no actually i want to go on from there i will come back to how did you start so can you really tell whether or not it likes one toy over another a baby particularly prefers something and if so why would you want that information well not where i thought that question was going. Yeah, we don't have anything that will ask the child what they want.

And a lot of kids, they're too young. They won't be able to answer you anyways. But what we do see is how things like comfort objects or pacifiers or objects that children develop sleep associations with will affect their sleep. There's a good story from that with my co-founder, Paul, and his daughter, Emily, where he was one of our earliest trial users. And when he was looking through the video clips that showed Emily waking up during the night, he discovered a pattern, which was that whenever Emily's pacifier fell out of her mouth, which she snuck into the crib, she would wake up and cry, and it would be a whole big ordeal. And the system at the time wasn't recognizing that,

it was recognizing that she was waking up. But if it could recognize that she was wearing a pacifier and waking up when it fell out, Paul wouldn't have had to connect the dots himself on that. It's something that the camera could have discovered and surfaced to them directly. And so is it doing it now? Do parents have to identify things for you to make it work, or does it learn these things? right now and when we send our first kickstarter units it's not going to be at the level of detecting pacifiers or uh individual dolls um that's something that we're going to be adding pretty soon um the question of of how it learns those things yeah someone has to teach it the first thing in general.

There's a toy called the Twilight Turtle, which is pretty popular still, I think. And it's this stuffed fuzzy turtle that has a hard shell that lights up and glows. And the first time that our system saw one of those, it had no idea it was going on. It thought that the turtle's shell was somehow a child's head. It couldn't make any sense of the multi-colored lights coming out of the thing it was really throwing the neural network off and so we had to manually train tell it what this thing is we had to give it the annotations to understand it. But the next time we saw a twilight turtle, the next time a kid had one of those twilight turtles in their crib, it didn't get confused. So it's able to generalize. It doesn't need to learn everything all over again per child. It's able to generalize. We're building in mechanisms where it can tune itself essentially,

where we can do some unsupervised learning so that if it's fairly confident something's a certain stuffed animal or pacifier in a given crib it can change its it can change the way it describes that concept to itself to make itself more confident and that expands when it's able to understand and detect those things. I have a general question. You've mentioned that you use neural networks. I have a general question about neural networks. For most of my career and college, neural networks were always something interesting to play with,

and it wasn't clear what applications they really had. They were hard to use and tune and it was always like well we're gonna have nuclear fusion in 50 years which is you know we're always gonna have nuclear fusion in 50 years something's happened in the last few years that i see more and more things and they're they you know powerful neural networks making you know very subtle decisions you see it in self-driving cars getting deployed and these kind of problems. What happened? Yeah, that's a great point. There was a confluence of things that happened. So yeah, 10 years ago

neural network was a dirty word. You could lose tenure over it. All kinds of horrible things would happen if you wanted to publish papers on neural networks. I mean, they were popular and then they were then they dropped like a rock they've gone through waves i mean they first became popular in the 60s and i think i think part of the problem is just culture and politics so neural networks have this appeal of being elegant in concept, right? Oh, of course, we make something that thinks like a brain. That sounds so nice.

That sounds so elegant. And then they just got hit with this series of devil in the details setbacks where, oh, it turns out it's totally unstable to train because you can't back propagate errors to more than one layer and get it to reliably you know how do you change the design of the first layers based on an error that happens in deep layers and though it turns out that there are classes of problems that are strictly proven to be unsolvable if you don't have more than n layers in the neural network and if you can't train deep neural networks then uh it's a waste of time so it it kept hitting these series of these things don't work because of some very practical technical or theory limitation and the people i think the perception was the people who kept on getting attracted to and again were

the naive people who just like the idea of oh it's going to be great to make something that works like a human brain and so what happened in kind of the past decade is that you had a small cabal of neural network devotees who were willing to continue putting effort into researching and developing them there were technical breakthroughs in the sense of changing some very simple things about the way neural networks work at a operational level so that you work around some of the theoretical limitations that people ran into before. There was a great insight that was driven from neuroscience of the idea of a convolutional neural network. And what that really means is you constrain the neural network

so that when it sees something like an image, any neuron looking at that image is only allowed to look at a small portion of it. If you imagine kind of this pure neural network where you have all these neurons that can connect anywhere and you can connect any input to any output and all these things, it would have to learn things like things in the real world tend to have some locality. If there's an apple in an image, it's going to be in one part of the image.

And if you find apple in one part of the image, you're much more likely to find apple in neighboring regions of the image. And so by imposing the importance of locality into the neural networks design by using convolutional layers, that made it possible to train neural networks stably with much less training data, with much fewer training iterations than it was before. And then a big part of it, a real big part of it, is just that computers got there. There was always this thought that, oh, it's just an intractable problem

where you cannot get enough data and you cannot get enough training iterations to make a neural network really work. And with the advent of the internet and the incredible advances in performance of processors, those things aren't a concern anymore. You can find lots of training data on the internet for general neural networks and you can train them on incredibly powerful machines and there was just this kind of inflection point where um i think the first paper that published really promising results was in 2009 there's another big milestone achievement in 2012 and it was as simple as proving that these things work so it wasn't trying to convince people to put research into them it wasn't we're only 20 years away

from having self-sustaining fusion. It was, it's here now. It just outperformed by orders of magnitude competing architectures. And that was enough to create a monumental shift in the amount of attention and research effort in neural networks. We live in exciting times.

We do. That's an amazing history. I mean, I knew none of that. So thank you for going through that. If somebody out there is interested in neural networks, do you have a resource that's a good introduction? So if you're a developer, I would encourage you to check out a library framework for Python called Theano, T-H-E-A-N-O, or Google's TensorFlow. And there's a lot of examples and bootstrapping guidelines and tutorials on how to start playing with and building these types of models. That's

definitely, I think, the best place to jump in as a developer. I don't have off the top of my head any books or articles I'd recommend, but I could send you guys some if there's a place to put those in the podcast info. Yeah, I can send you. There's a couple of good articles that explain some of the history and some of the details of how exactly they work. Is that history, that kind of contentious history, why I see deep learning being the term instead of neural network sometimes? Yep, it was a rebranding, whether it was deliberate or not. One of the papers that, you know, a lot of the researchers i i think and this is i i'm not you know i've been doing this stuff for two years so this isn't decades of experience talking but um i think

the researchers had to get a little creative in the names of their papers when they wanted to publish papers on neural networks and so there was a paper called Deep Belief Network, and it didn't have the words neural network in it, and it had this new concept of deep learning in it somehow. And so I think that's more of a rebranding than any kind of real fundamental change in what it is. the beginning which is how do you start training your network how do you get the right data the right sorts of data the right quantity of data to train this deep learning thing yeah it's hard when you know it's a bit of a catch-22 where we're trying to build this thing that is going to be appealing to lots of people. And we need data from lots of people to build the thing. And so we did what we learned to do with decades of experience at IDEO, which is we just fake it. And we sat Wizard of Oz-ing the experience. So instead of the neural network performing the task for our trial users, we would be doing it manually.

And so we started back in the spring of last year doing that, where we would find people who are interested in trying out this new experience. And they agreed to share their data with us for our internal science development purposes. And we would manually label sections of video with a sleep or awake child in or out, come up with graphs and video clips and all this kind of stuff to test the experience because we didn't have the data yet

to build the model to do it. So that's how we got started. And that was enough to seed the development where we could get something that kind of worked and then go out to more people. And now we have something that really works that we're ready to share with thousands of people. How many hours did you personally spend watching a baby sleep? Too many. Yeah, that was full days of work for weeks on end. So yeah, I probably can't consider running for office ever now. But yeah, it's a lot of what I did. I understand.

I listened to a lot of gunshots when we were doing gunshot classification at ShotSpotter. And I think I would rather watch babies sleep. Yep. Okay, so you and I talked earlier off the show that you were running Linux. Yes. Lenaro, why don't I let you describe it? Sure. So yeah, we're running Linux on this processor. A lot of it is that we can afford to,

and we're doing a lot of stuff. We have a lot of different services and applications and operation modes and fallbacks, auto-updates, all kinds of stuff. So we have the need and the affordance to have a pretty serious OS. So we're using a... It's a custom image derived from Lenaro, which is an organization that is affiliated

with, I think, a few chip manufacturers. I know they have some relationship with Qualcomm. And they create flavors of Linux targeted at different embedded platforms. And so it's based on Debian. It's a full desktop Linux that we have removed a lot of the desktop stuff from to make it headless and leaner and do the things that we need it to do. But there's nothing too custom in anything we're doing. We're using the standard kind of task management and process supervision stuff.

We've written some custom hardware level drivers for the different IO and the camera and things like that. But it's pretty standard as far as the operating system and applications and using systemd for everything and doing all that kind of good stuff. But one of the problems with running Linux, and maybe this is completely absurd, but it seems like it's more sensitive to security issues. But on our end, it's not really what we're doing. This isn't a home router or an IP camera.

You can't enter its IP address and log into it. You can't even open a console to it. There's no place to plug in to get a serial console open. We can close all of that stuff off at a hardware level. The chip itself is really geared towards Android devices, and there's a great deal of security measures in terms of encrypted partitions and secure boots, all that kind of stuff to prevent someone,

if they did want to hack a camera and get access to things, would make it very difficult. So the way that we control access to the device is, in a sense, custom in that, you know, the device is interacting using HTTPS with our servers securely, exchanging encrypted key pairs and encrypting payloads. And so all of the methods by which the device would change itself or send data is all transactional. There's no exposed login to exploit and so you you have a relatively secure system from your cloud to your device i think so yeah and if i have your device and i am hacking it

does that give me a window into other people's devices? That seems to be one of those common attack points, is once I have a node, I can get to the cloud, and now I control all. Right, admin password type of thing. Yeah, no, we don't have, you know, because we don't have this kind of user-level login or access, we don't have any motivation to make it easy to access or log into a device. If you did successfully hack an individual device,

and you were able to access the encrypted drives and keys, all those keys would be unique to that device. You would be able to post data from that device to our secure system. You would have to hack the user side of it, which is, again, a secure set of keys and token exchanges to be able to access data that you're not supposed to. Of course, it's not impervious. Nothing these days is, but it's the same patterns and protocols

that banking sites and other high-ish security applications use. So the vulnerability that still seems gaping is the Wi-Fi itself, which, of course, you can't secure against. If people's password is 123, three then god help them yep um so there's a there's a couple measures there uh one is that even from your local wi-fi network that your device is connected to you can't actually log on to the camera um there's no SSH daemon running. You can't go to its IP address and get a log on screen. So even from the local network, you can't as a user log on to the system. What you can do is hit a very small server that is only capable of giving metadata.

So it would be things like the child's current sleep state, how long they've been sleeping. That is still, though, using the same keyed and encrypted security protocols to access. We're using modern streaming protocols. So the stream on the local network is encrypted, and there's a keyed security handshake to be able to access the stream from a device on the local network. So even if you get onto someone's, even if someone hacks into your home Wi-Fi, they would still need to go through a lot of work

to actually be able to see your camera or get any data from it. And they wouldn't be able to log on to the device itself and take over the device or anything like that. And one of the companies I worked at, when we talked about security, we always said, my goal is not to have my face on the cover of Wired.

Yes. Is that motivational for you to make sure that you aren't the story that is the baby monitor that got hacked and somebody shouted obscenities at the little baby? Well, nobody wants that, but I don't think that's a good enough motivation. I think we sincerely don't want these things being security concerns it's not a huge amount of effort in this day and age to put in place good security practices. like the baby monitors that were hacked, those were just publicly exposed ports on the internet that you could, if you found it, you had access to the camera. A lot of the exploits in logging on to Linux devices in people's homes is because they have a standard login and password set by the manufacturer and they never change it and those are exposed. And so, you know, we definitely don't want to be on the cover of Wired for some security hole. And we also think that by caring

and putting some attention into it, we become, it's not a question of making it totally secure. You can never be the house that nobody can break into. But we can be one of the least appealing houses on the block to put any effort into breaking into. I think so many people didn't recognize the importance of security because the idea was, well, who cares about my baby monitor but me? Who cares about my TV or my toaster but me? And we've definitely had a lesson recently that, well, if your stupid TV can take down half the internet, then everybody cares about your TV. It's weird and i'm glad that you have a reasonable story thank you for that i'm glad you think our story is reasonable i i mean i did say if i hacked it but honestly i tend to only look at the hardware because i wonder how you

did it all not work at the software because I don't really care. I don't need to see other people's babies. Right. Let's see. Let's go on to other questions for a few minutes. Oh, this should have been back in the lightning round. You used to be a professional internet poker player. Is that right?

That's right. Okay. Do you want the whole story? Yeah, go ahead. Let's just do this as a story. All right. Yeah, so I got into online poker like a lot of people back in the early 2000s as a college student, I would play poker with friends every once in a while and then saw that one of my friends was playing this internet poker and seemed to be making a bunch of money. And I thought, oh, I'm at least as good at poker as he is.

And so I started playing, started reading up on strategy and understanding how the game works. I've always been into games in general. And back in those days, you didn't have to be very good at poker to make okay money, definitely beer money online. And so it was a very appealing thing to do. Out of school, I had a decision between going to pursue a master's or taking some time off to play poker. And by the time I graduated, um, I was making enough at poker that it was a pretty serious decision to consider. And I decided to take some time off. And so I, I played full time for a while. My first startup was actually a data science startup where we designed tools and built subscription data services for other professional internet poker players. One of the products that we built is actually still live in the world today, which is pretty cool.

Then I got a call from a friend who was working at IDEO in their Chicago office and he needed help with a project. And so I came out and started contracting for them. Started with a couple months here and there over a couple years. And then eventually they gave me a full-time offer and I said, sure, but I wanted a yearly three-month sabbatical that I could take off and play poker. And so I would take around three months of work off each year and play internet poker. We went to Thailand for a summer one year, did a lot of traveling. I started producing instructional videos for other online poker players through sites like Card Runners, which are these online video poker training sites that were popular. So, yeah, everything was going great. And then the U.S. government kind of stepped in in 2011 um and they seized you know these poker sites that people were playing on

were foreign companies it's a whole legal gray zone there's a lot of a lot of strange legislation and um political machinations and all kinds of stuff around this gray underground online poker world. It was never illegal to play online poker as a U.S. citizen on these foreign sites, but nonetheless, the government did not like it and eventually enforced it by seizing these foreign domains and really shutting the whole thing down. That was back in 2011, and I haven't really played online since then. But before then, yeah, I've put in millions of logged hands, lots and lots of hours, and did all right at it. How is poker similar to what you do in your non-poker professional life?

I mean, there's a lot of statistics. There's a lot of statistics. And so I've definitely grown my skills in statistics through poker that I've applied to different engineering projects. Really understanding game theory has helped in a lot of my decision making. So I think I think through a lot of decisions in which there's uncertain information and uncertain outcomes in a approach that's pretty similar to how I think through poker. You know, understanding kind of the dynamics of competition at certain levels. So what I really did as a poker player that was different enough to get me a little bit of attention and get picked up by sites to do training things was I really focused on understanding my opponents and tailoring my

strategy to react to weaknesses in their strategy. You know, a lot of effort at the time in kind of the strategy community and among people who are taking this very seriously was to find unexploitable strategies, optimal strategies where, you know, if you were playing rock, paper, scissors, you could say, you could tell your opponent, I'm going to pick rock, paper, and scissor with all exactly one third probability. You can tell them what you're going to do rock, paper, and scissor with all exactly one third probability, you can tell them what you're going to do and there's nothing they can do to beat you. And that sets of those equilibrium strategies

exist in every single game. And people were trying to find poker strategies that would be unexploitable, that would be optimal, where someone else couldn't take advantage of you. But if someone tells you, I'm going to play rock with 40% probability and then paper and scissor with 30% each,

you really want to play paper 100% of the time. You don't want to be playing one third, one third, one third. And so that's where I put a lot of effort into modeling and statistics and figuring out how to apply that general thought to a game like poker, which is enormously complex and hugely noisy. When you think about the amount of time that you have to apply a strategy to get any kind of empirical evidence of the results it's it's uh jaw-dropping um and that's where another part of um i think how poker really affected me was in my tolerance to risk and my growing awareness of how much luck and uh random factors affect everyday life and

affect perceptions of what's good or what's bad or what was the right decision, what was the wrong decision. And so through playing poker, you have to ignore short-term results. You have to make a separation between whether a decision is good or not from whether you had that particular outcome you wanted or not because there's so much randomness that comes into play in each individual decision uh that that thing where you believe it was skill but it was actually just random luck yeah and separating those it's it's so much it feels so much nicer to believe that it's my skill and not just random luck. It just is a nicer mental model for myself. But that is something I struggle with. And it's not, you know, if you're going to have to have a rule of thumb for people to have, right?

For people to have, for chimpanzees to have, for monkeys to have, it's not a bad rule of thumb. If you don't have the time and the capacity to go through and track all this data and try and figure out was it skill or luck, you just kind of go in the direction that seems to be working and cross your fingers. It makes sense that our minds are kind of hardwired to do that. But yeah, it cuts both ways. You're biased to want to believe that when things are going well, it's because of the decisions that you're making, you're making the right decisions. And you could also, when things are going poorly, you have to resist the temptation to blame all the external factors that you don't have control over and really try and assess are

we making the right decisions um and so i've i've learned not to trust my my brain as much uh in in certain times i feel like this is the point where i get to say something about you know that narwhal bayesian introduction I was working on? It is finished. So listeners, if you would like to know more about some of this, try that. It might save you years of poker. I'm sure he had fun. I suspect so. Well, as long as we're advertising, we really should make sure to mention that your Knit is on kickstarter right now yes yeah we've got a couple two or three weeks left on the kickstarter campaign um and yeah it's knit knit health

you could find us on kickstarter just by searching knit monitor uh if you have kids if you know someone who has kids if you have a kid with a sleep issue if you're an adult with a sleep issue and you want to try it out on yourself i would definitely encourage anyone to check it out and uh if you have any more questions on the details of how it works feel free to leave comments or questions on the Kickstarter page. And let's see. It's ending November 17th. Since I know some of you don't listen the week it comes out, you need to know that it might be tomorrow. 17th of what? 2016, November 17.

Some combination of those numbers should get it for you. Let's see. Your early birds are sold out. Your super early birds birds are sold out. Your super early birds are almost sold out. So people are looking at spending between 200 and 230 to get one of these. Is that right? Does that sound right? Okay. And they come in two packs. Why did you do two packs? We added that after we started our Kickstarter because we had a lot of people asking how to buy two of them because they had two kids

and i guess on kickstarter you're only allowed to make one pledge per campaign oh right and and then you get well they can look at the Kickstarter from here. We've talked enough about this. It doesn't look that hard to hang, but you do have to hang it because it has to be able to see the whole crib and the baby or the human or the adult or whoever. So has that been difficult to try to convince people how to set up the system? No, we actually started with something that, you know, was a little easier to put on a shelf. What we found in trials is that hanging it seemed to be preferred for everyone who is using it. It's not a, you know, you can hang it on a picture hook on a one of those command adhesive strips you don't have to screw or do anything to the wall it's it's a pretty easy setup process um and yeah hanging it on the wall gives

you the best view of the crib or the bed as long as the camera can see the whole scene it'll figure out what's going on it doesn't need to be in any kind of special position or directly overhead or anything like that you just want to have a clear view so it so that the child isn't uh off the off the screen ever or being occluded by part of the part of the bed are you doing any clinical style testing with comparisons and gathering evidence from your system as well as some other medicalish monitoring system? Yeah, we're doing some of that on our own right now using some devices designed for clinical settings like CPAP, which is designed for monitoring sleep apnea stuff. We are also actively seeking partners to do clinical and validation trials with using other devices as benchmarks and baselines to compare our data against. So if you're a clinical researcher or a practicing clinician in any kind of medical field that is related to this,

please reach out. We're actively seeking those relationships right now. Christopher, do you have any last questions? I do. Sort of along those lines, when consumer product companies and other companies that start taking a lot of data about something that isn't happening clinically necessarily, because setting up a clinical study with a million people is harder than sometimes selling a million cameras. I guess my question is, do you anticipate or have you already seen behaviors that are kind of

undiscovered or, or unexpected by looking at this many you know being sleeping for this long yeah yeah i think we have seen some interesting stuff already that we you know we do have a kind of panel of clinical advisors researchers pediatricians other doctors that we review data with and we've already shown patterns and and things where they say yeah we don't really know what that is or how many kids it affects it's it's been documented before but there's never been any study or enough sample size to understand really what's going on. And so that could be breathing patterns or sleep patterns or combinations of different things that

once we have these devices in the wild, there's going to be a host of valuable medical data. You know, we take our customers' privacy very seriously, so it's going to be an opt-in to share that anonymously with a medical community when we get there. But we think it's pretty exciting, and it's the type of thing that's never, you've never really been able to collect this much data just because of how cumbersome these studies are when you have to strap a bunch of physical sensors to a kid in a sleep lab so sort of along those lines and i guess we haven't talked that you have audio but you do because of course you do um when i met with paul who i think is your ceo uh coo coo okay when i met with him years ago, when you were starting the company, he mentioned that there are times when babies wake up and you should let them go back to sleep. Yes. I think that's most of the time that babies wake up, as long as they're old enough that they don't need really constant feeding. If they're going to go back to sleep on their own, you really want to let them do that.

You know that's not where I was going, right? Nope, but where are you going? Sorry for the derail. No, no, no. It's just one of those games where I'm trying to get him to say something and I can't figure out how to push the right buttons. So babies sometimes fart exclusively. Oh, didn't quite know you were going there.

We like to have a high class show. Yeah, that might be true. And yes, there's a microphone. And if you want a fart notification, we could make that happen for you. Do you use the audio as part of your machine learning or is that not quite there yet? It's not. We simple sound level and sleep correlations, but it's not there yet in where we want to go, which is getting a real, again, it'll almost certainly be a deep learning neural network unless the next thing is published soon. That is able to understand the differences between cries and coughs and different types of coughs and different types of cries and this really high touch audio intelligence that when you combine it with the sleep data and the breathing data and the gestures and motion tracking paints a richer picture of everything that's going on. See, that's all very, very smart. And it sounds great.

Unlike when Paul explained it to me and I nearly had wasabi out my nose. Okay, Evan, we have kept you for long enough. Do you have any last thoughts you'd like to leave us with? No, I think the final thought is just the marketing plug of imploring listeners to check out the Kickstarter page and let us know what you think. All right. Our guest has been Evan Shapiro, co-founder and CTO of Knit Health. Their Kickstarter does end November 17th. And their website is knithealth.com.

That's K-N-I-T-H-E-A-L-T-H. For the knit part, that's Kilo Nevada India Tango. All right, I'm going to stop spelling it for you because it's just too hard. Thank you, Evan, for being with us. Thank you. It was a pleasure. And thank you to Christopher for producing and co-hosting thank you for listening hit the contact link on Embedded FM

if you'd like to say hello to us or if you'd like to say hello to Evan we can forward things along you may also sign up for the newsletter check out the blog with my completed narwhals and Bayesian stuff and all of the other posts

and all of the other excellent posts about MSP430s and STM discovery boards. And you can also subscribe to the YouTube channel, which has nothing more than the audio from this show. So final thought for you. Oh, oh, I love Neil Gaiman's graveyard books. So let's go with this. Sleep, my baby-o. Sleep until you waken. When you wake, you'll see the world, if I'm not mistaken.

Kiss a lover, dance a measure, find your name in buried treasure. Face your life, its pain, its pleasure, leave no path untaken. Embedded FM is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show, we did not put them there and do not receive any revenue from them. At this time, our sole sponsor remains Logical Elegance.