Embedded - 238: My Brain Is My Toolbelt

Episode Date: March 15, 2018

Chris and Elecia answered some listener questions about dynamic memory and shared code. Then Elecia gave a presentation about ShotSpotter, the gunshot location system she worked on. Elecia enjoyed The Woman Who Smashed Codes: A True Story of Love, Spies, and the Unlikely Heroine Who Outwitted America's Enemies by Jason Fagone. Ben is the editor of HackSpace, a new magazine about making (and hacking). It's produced by Raspberry Pi, but it's technologically agnostic. The first issue is free online. The ShotSpotter presentation was originally given with Sarah Newman at the 2008 Grace Hopper Celebration of women in computing.

Hello and welcome to Embedded. I'm Alicia White here with Christopher White. It's just us this week. We have a lot to talk about. But I want to warn you right now, about halfway through, I start talking about ShotSpotter. And there are some gunshots and some violence, not like actual gunshots. I mean, it's all fine here, but you'll hear gunshots and then you'll hear stories of mayhem. So if you're going to be disturbed by that, either skip the second half when I start talking about ShotSpotter or just go ahead and skip to the next one. Or last week's, or Sunshine's, or Roger's, or Robin's. Actually, we've had a lot of really good shows this year. Do you want to dive into the big things,

or do you want to cover all the little things first? No, let's get the little stuff out of the way first. Okay. The first thing is I was not trying to drive listeners away when I alluded to that in Roger Lynn's episode. We aren't, why not? Well, it wasn't that I don't want people to listen. I'm proud of the show.

I want people to listen. It's just that the more time I spend thinking about the listeners, the less time I spend thinking about what I want to talk to people about. And the shows come out better if I am genuinely enthused for my own amusement. I think that makes sense. I think most people would appreciate that and don't find that to be a problem. I don't quite remember what I said, except that I didn't want him to worry about the listeners.

It was all about me or something. I think we said we didn't care about the listeners. All right. But we mean that in the nicest way. Nicest possible. I mean, we aren't considering, you know,

your wishes and desires. We're not doing fan service all the time. Calling back to the Tim O'Reilly episode. We were talking about waves of technology and it, I was expressing that it isn't new. Having waves of technology. And I was expressing that it isn't new. Having waves of technology isn't new. But they seem to be getting closer together. And this really hit me this week.

I was reading about oxygen in two different books. It was a strange week of all chemistry all the time. And oxygen as a thing didn't exist 200 years ago. I mean, a little bit more than 200 years ago at this point, but we didn't know what oxygen was. We called it something else and we couldn't tell the difference between it and other things. Chemistry is new. Physics is like 250 years old, 275 years old. This is all so new. We have buildings older than science. Well, yeah.

I mean, I would walk that back a little bit. I mean, things about chemistry were known for a long time at different places. And things about physics were certainly known at different times in different places. And things about physics were certainly known at different times and different places. So, in the Western sense, and you know, the post-enlightenment, sure, our current view is new.

Isaac Newton and the periodic table. Those are pretty critical. And then from there, steel and steam and all these things, they're just, they seem old to us, but they're new to the human civilization. And so the fact that the internet is another wave, it's a big wave and it's a fast wave and they keep getting bigger and faster. It's full of trash and plastic. Yeah, I know.

All right. So I wanted to clarify that too because... I knew what you meant. I didn't get the idea. He did, and it just really hit me this week that we talk about everything getting faster, but it's getting faster and faster and faster. And it's not just it happened in the last 10 years or 20 years, whatever. Next thing is I found a fantastic book,

The Woman Who Smashed Codes, A True Story of Love, Spies, and the Unlikely Heroine Who Outwitted America's Enemies. This was about a cryptanalyst, Elizabeth Friedman. And so much of her story hasn't been able to be told because she was confined to the Secrets Act, the United States, World War II. You're not allowed to tell anybody what you did in the war thing. Still?

She wasn't until her death. She was not allowed to talk about it. And so a bunch of Freedom of Information Acts requests and some old papers were found. And Jason Fagan wrote this book and it was, I mean, it was a spy novel.

It was awesome. I mean, it was a biography. It was true, but it was like a spy novel. I greatly enjoyed it was a biography. It was true, but it was like a spy novel. I greatly enjoyed it. Cool. Let's see. Oh, Tool Belts and the Burden of Tools.

Do you have any idea what I was talking about when I wrote that? No. I was watching the electrician install our new kitchen light. And he had all these massive, I mean, he had this amazing tool belt. And he had suspenders to hold up his tool belt, which gave me a new impression of belt and suspenders. And his tools, I mean, he knew where everything was. He was looking up and he could reach down and get whatever he wanted out of it. Clearly something he used a lot. But I was thinking

about code tools and how there are tools that are big and weighty, like make and get. And the less you use them, the more they mentally weigh. Because you don't remember how to use

them? Because each time it's a big effort. Okay. And there are tools that not enough people use because they're too complicated. PC Lint might be one of those. I don't know if it's too complicated. Or people don't see the value in it. There's a burden, right?

That's a tool you have to spend a lot of money on. So, yeah. To get it configured the value in it. There's a burden, right? That's a tool you have to spend a lot of money on. No. Yeah. To get it configured? To buy it. Oh, I didn't think it was that much to buy. Well, anyway, I mean, your point stands. There's some thing that makes them weighty, whether it's cost or perceived complexity.

And the electrician clearly had the tool belt that he needed. And likely there were a lot of tools he didn't have. In fact, his assistant had the cordless drill and he didn't. It made me think about the tools we use for software. Because we aren't very good at pruning our tools. And we aren't good at recognizing that tools have cost. Everybody's like, okay, let's just add one more thing. Let's add a Clang build as well as our C++ build because it has more...

You mean GCC build? Our GCC build, because Clang has better warnings. Well, that's great. And as long as your MIG file doesn't make you babysit it all the time, maybe it's no additional burden until you have to make something new or port something. And I don't know, this idea, it's all a jumble right now. So maybe somebody out there will tell me there's already this book or blog

post about it. This all goes back to tools also need care. Oh, yeah. You have to sharpen your woodworking tools. I have never been happier than when I've worked at places where there's somebody whose job it is to maintain the software tools. Because they think about these things. There's a sense for why the tools are chosen. They're organized in a way that people can use them.

Yeah. And they're pruned. I bet they are pruned because the person who has taken care of it all recognizes when a tool has outlived its usefulness. Yeah, that's tricky though, because there's always somebody who can argue for its persistence, right? Well, I think about the fact that I still have awk in my head. Awk's great. What's wrong with awk?

Awk's a fantastic language for what it is, something that I hope never to do again. And my week of chemistry has been shoving in a bunch of other tools, a lot of Python things. Well, now you're talking about space in your brain. That's a different... My brain is my tool belt. i don't know i there's there's some point in here about tools maybe one of the listeners who hasn't left can email us what your point is yes

um okay let's i there are some more little ones but but now I want to answer a question. A question. A question from Ryan Jones. He's fairly new to the podcast. He wants to know when we would use dynamic memory in Embedded C. Are there reasonable use cases? He knows that dynamic memory should be avoided at all costs due to issues with the stack overflows and dangling pointers. What?

But if you were going to use dynamic memory, when would you use it? Use it all the time. So anytime you have, well, let's define dynamic memory first of all. Because it doesn't have to just be malloc. No, it doesn't. Or new. Dynamic memory is anything where you ask some system, I need this much memory, give me a pointer to it.

Instead of saying, here's this variable, I name it, it need this much memory, give me a pointer to it. Instead of saying, here's this variable, I name it, it's this type. And, you know, it's at the top of the file. And that static memory, it sounds like he's talking about local variables is dynamic memory as well, given the stack over flow quest part. That's kind of sure, sort of dynamic, but I don't usually consider that dynamic memory. Do you? No, I don't consider the stack. No, the heap and the stack are separate. And I think of dynamic memory as mostly affecting the heap, or if not the heap, then a heap. And there are times where you have to use some sort of heap functionality. Screens are a big one. Displays.

Because you're changing what's on the screen a lot. And if you have fonts of different sizes and you don't want to pre-allocate everything. Well, yeah, that's the thing. So taking a step back, remember that we're resource constrained on an embedded system. If your application is airtight and small, and you can define all the things that you will ever use in the memory and allocate them statically, then great, go for it. Do that, absolutely, 100%. The number of times you'll be able to accomplish that and have a meaningful application is very, very small. So the fact that you have a limited amount of memory and you might have different tasks that come and go means you're going to have to be sharing that memory amongst them, which means you're going to be having to

use some form of dynamic memory. For example, I once had an I2C and a spy driver, both of whom had to take in a large amount of memory, but neither one of which could work at the same time. So that shared memory between them is technically dynamic because it isn't a single purpose variable. Now imagine networking. You've got packets coming in, you have buffers, and as they come in, you need a new buffer to do stuff with the packet maybe you've got dma happening you're filling the next one you could statically allocate ping pong buffer or some circular buffer it's much easier sometimes to say here are 30 items of the same size and i have a free list and a and a new list of all those items and when i need one i grab it and when i'm done

i put it back that's not malloc they're fixed size you can consider it kind of a malloc but you can't ask for a different size buffer from that that's definitely dynamic memory you do that all the time it's like a chunk allocator or a however you want to call it there's lots of words for that and your your same size memory there is very important yeah i don't want to call it. There's lots of words for that. And your same size memory there is very important. Yeah. You don't want to lose that detail. One of the reasons malloc and new are so dangerous is due to fragmentation.

So you malloc something that's 10 bytes and then you free it. And then you malloc something that's five bytes and you free it. And then you malloc something that's 5 bytes and you free it. And then eventually your memory gets to the point where the ones that you have kept and the ones you have freed are all mixed together. And it can't...

Your free memory is all little bits. Right. It's one byte at a time. So now you ask for a big chunk and there isn't one because it's all Swiss cheese. And you may have enough memory, you just don't have enough contiguous memory. And there are algorithms that can do that. Right, right. There are absolutely algorithms that can do that.

But again, we're on an embedded system, it's constrained. The chunk allocators prevent that by having them all be the same size. And then you know that when you allocate in free, you're not going to fragment your memory that way. So having said yes, dynamic memory is important and useful. Do you use malloc? Yep. You do? Yep. What cases do you use malloc versus making your own allocator? I'm trying to not talk about the

specifics of what we use malloc for so i'm trying to think of a generic example um do you want me to sing the jeopardy your graphics example right you may not you may be working with a whole bunch of images that you load from disk and And they're going to be different sizes. They're all different sizes, and you want to keep them in memory so you can draw them quickly. Maybe they're fonts, maybe they're icons, whatever. That's one example where you'd use malloc, because you know how big the file is, and you say,

okay, this one's 256 bytes, I need a chunk that big. This one's 128, I need one that big. But you don't want to be wasting a bunch of memory by, you know, saying, well, I know the biggest I ever have is 512. So let me just carve up 10, 512 buffers. Well, you've just wasted a huge amount of memory if, you know, set it aside that you can't use for anything else if there's a bunch of smaller items. So anything where you're loading something from a disk and caching it, anything where you have a document structure, something, a markup that's describing something for a UI or something like that where you don't have a fixed size,

things like that. But it's kind of, in systems that I'm used to, it's kind of where you are loading something from another resource and it's a variable size and you need to store it in RAM. And then you have to be aware of how much heap you have left and the fragmentation issues that come with it. Yeah. And you have to handle them.

We run into those things all the time. And, you know, sometimes you can get away without this sort of stuff if you have a flash file system, or not a flash file system, but a flash storage system that's directly addressable. And so now all your variable size stuff is just baked into flash, and you can just load it without having to set aside a separate RAM buffer. But, you know, it's one of the things you can do with embedded systems. So that's a consideration.

If you have something that's variable size, things you can do with embedded systems. So that's a consideration. If you have something that's variable size, do you need to create memory for it, or is it on memory already that you can just load? And so that's a graphics example. You cheat and you never really load it. Yeah, you might have an 8 megabyte flash

that's programmed with your fonts, for example, and you never really, you know, your bitmap fonts example and you never really you know your bitmap fonts and you never really do anything with them except read them straight from flash and split them to spit them onto the screen um so i hope we've confused everybody i don't know we'll see uh but then you know the dangling pointer issue and garbage collection that's yeah that's life yeah the trade-offs ofoffs of having dynamic memory without paying attention to freeing things yourself are you don't know when your program's going to do garbage collection and that kind of thing. And that would increase your latency and affect your performance.

Yeah. So it's always a trade-off. Yeah. And for the most part in embedded systems, you are trading off ease of programming for speed, size, and cost. Right. Okay. Ben emailed me in December. Sorry, Ben. And said he has a new magazine coming out called Hackspace. And it looks pretty cool. It's produced by Raspberry Pi folks, but they're agnostic as far as technology goes. And he's the editor and thought we'd like it. And truth is, I do like it when I read it, and I should read it more.

And it's interesting. I'll have a link in the show notes. Cool. Did you get to read it? I did not. I wonder if I didn't even find it. I'm not sure you sent it to me. I'm such a terrible person.

What? Okay. So now we're down to a few of the big things. All right, let's do it. Kevin and Sammy came out to California and said they wanted to have coffee with us. And as always, we were baffled, but we had sandwiches with them. Oh, I'm sorry. Was that not right? Hi, Kevin and Sammy. It was really good to meet you. I'm sorry it was so cold at the beach, but it was really lovely to talk to you. And they asked about ShotSpotter and wondered why we had never really talked about them on the

show. So I worked at a company called ShotSpots Butter, and I gave this presentation in 2008. So it's 10 years old and not current, and I can't send you the slides because there are pictures on them you can't have. But I'm hoping it will still be interesting. So are you ready? Yeah, I've pressed play. Okay, now press on the left side of that screen under the 2008. Okay, so part of this presentation is going to be the sound of gunshots. ShotSpotter is a gunshot location system. You sprinkle sensors around a city and automatically call the police when gunshots or fireworks are heard. So we're just going to let that play while I

continue through the first bit. Chris is running the slides and I'm giving the presentation. This should be fine. So gunshot location systems can be used to reduce gunfire due to holiday celebrations, like the one we're listening to now, which is New Year's Eve in Los Angeles, and it's a mix of gunshots and fireworks. And the holiday celebrations and decreasing those is important because that does lead to many injuries and a lot of bullets in roofs. It's more important to discuss the impact on violent crime. Getting officers to the scene of crimes quickly means saving lives and catching criminals. And this is why I worked there, because it was pretty darn cool. I don those cities, there has been a greater reduction in crime than there has been in other cities. Although, as Christopher likes to point out, there's been a pretty great reduction in crime all over. Here you see, which of course you don't because I'm not sharing these slides,

there's a shooter in the center and then there are sensors around the city. Sensors often get put into buildings, the tops of buildings. That's all I'm going to say about that one. Oh, but most of this whole presentation is about stories, because I've always loved stories. And I think it's important that we understand not just the technology, but the effect of it. So early in the ShotSpotter history, there was a serial shooter randomly firing at cars and buildings. In 2003, one person had been killed, others wounded, and more casualties expected, as the Franklin County Sheriff's Department and FBI had no solid leads. ShotSpotter deployed within days once they were contacted. And within

hours of becoming operational, they registered the sound of gunfire and led investigators to pinpoint a location where the shooter had fired. It was the first physical evidence and it allowed them to develop solid leads. The shot spotter data was used at his prosecution when he was arrested. So that's one of the stories. But how did we get there? There are four steps to gunshot location, and each one will get its own story. The first step is to detect. We're going to find gunshots in the form of audio pulses. We're going to locate and triangulate where the shooters are, classify, which is to make sure that it is in fact a shooter and not something more benign, and notify, which is to tell the dispatchers where to find the shooter.

So let's start with how the sensor detects. All right, so that is what the dispatcher would have heard they would have actually gotten a little bit of a siren that said that there was a an alarm and then they could click on where the location was but when they wanted to play the audio that's what they would have heard and there were some other things you can play in that same one. Okay, that's actually the big one. That's a sensor only 100 meters away from where the shooter was located. And it's loud. A football field away, and it was loud. So what about the next slide?

Will you play that one again? Did you hear it? Barely. Yeah, it's a sensor that was almost a mile away. And at that distance, the gunshot sounds like a pretty subtle door knock. Sensors need to find the easy, close pulses and the harder-to-detect ones. You think about the difference in signal quality of those two. It's pretty huge.

In this case, in this story for the sensor detection, there were many sensors reporting the location, so it was straightforward. The response time was really quick. The officers arrived to the location and they found a man who had been fatally shot in the head at a stop sign, really close to the location of the shooter, the shooter position on the map. And I would have been surprised because we put the shooter on top of the building, which is usually not a good thing. But even though it was an odd spot, the officers trusted the system, and they surrounded the building, and they found a man on the roof with a high-powered rifle smoking a cigarette. This was a sniper, a hitman, who didn't expect the police to show up so quickly,

because single shots are nearly impossible for humans to locate. And that's why ShotSpotter had sensors running around the clock, around the city. That sensor does a lot. It captures audio. It uses a GPS for timestamps. It filters it to get rid of spurious pulses and noises. It finds and characterizes these pulses so that they can be combined and compared later. Not just amplitude, but lots of other things. And finally, the pulses are sent back to the central server. So primary function is to detect pulses, but the signal processing is harder than it seems. The buildings distort the sound, giving echoes and reverberations. And the trees eat sound and radio signal.

And kites. And kites. The wind changes the direction of the sound, pushing it around. And then there are the people. You're trying to protect all these people, and they're making all their sounds, like playing basketball and slamming car doors. And we can filter out some of those things, but it only gives us a better shot of hearing shots. Wow.

It even says pun intended in my notes. Good notes. Good notes. So in the slides, there's a graphic representation of those two shots we heard earlier, the super loud one and the door knock. The GPS is used to timestamp the audio to nanosecond precision. And if we were to zoom in, the close one, the really big, boomy one, is full scale. I mean, it clipped the sensor.

The little tiny door knock was 30% of full scale. And the ShotSpotter system could pick out pulses that were only 2% of full scale. And the shot spotter system could pick out pulses that were only 2% of full scale. So the dynamic range was amazing. Other things that they would look for were rise time. Because sometimes fireworks don't explode as fast as gunshots do. And then there's envelope, the duration of the pulse. How long does it take to get back to quiet? And those things help determine how far away a pulse is. If a pulse is very close and very short, that's different than one that is far away and very short. That's just how sound travels. Frequency domain analysis was used as well, mostly for later classification to get rid

of spurious noises. So those various features all went over the wireless radio in summary format. And there's a lot of differences between the close sensor and the far one, between the scale of the amplitude, the frequency, the signal-to-noise ratio, and even the time of arrival, because it's kind of important for later. The close sensor had a much earlier time of arrival than the far sensor did, so that makes sense. And that's a lot about the sensor's pulse capability, and that's where all of the embedded systems are. And some of it's implied. There's also the managing health and settings, the spooling audio, encrypted communication,

and it's all really resource constrained. They're working on larger systems now, I think, but when I worked there, it was a processor that was designed to work in washing machines. It was somewhat limited. So once we have the pulse, what happens next? Okay, so if you were here, you would be able to see this parking lot and a shot spotter red dot and an indication that there was a gunshot. And one night a man fired outside his home and later ran into this apartment building we can see and he took his wife and child hostage and the shot spotter heard these gunshots. Can you play that audio? It's going to be a bit much. Yeah. Shot spotter notified the police and based on the high volume of crime that night,

the state police were brought in to handle the situation. They found bullet casings within the parking space that ShotSpotter put the shooter in. So how do we find the location? The algorithm looks at all of these gunshots from these sensors and uses them together to figure it out. We have to go from these three sensors heard a whole bunch of pulses to these sensors heard 20 gunshots. Here's the location for each one of them. And that's especially useful if the shooter is moving like a drive-by shooting, then you know which direction they're driving. If you could see this, you'd see that we have multiple sensors

all with a huge number of these pulses. They arrive at different times for each sensor, but the relative timing between the gunshots once they start is the same for every sensor. And the ShotSpotter patented method takes advantage of these relative timings. It's a cross correlation where one signal slides across the other to find the best match. And if you're thinking the word convolution, yes, keep thinking that. And then once they do that, the pulses from the different sensors are grouped into a single gunshot. Once you have the single gunshot, you can start to create the location using the same method used in radar and GPS called time difference of arrival. And it's all about the parabolas.

So if we have two sensors that hear a pulse, and we know exactly where those sensors are, because they've been sitting in GPS forever, and we know what time they heard this impulsive sound, this gunshot. The definition of a hyperbola is the set of points where the distance to one foci, like sensor A, minus the distance to the other foci, sensor B, is constant. So this is the definition of a hyperbola, a constant difference in distance. And once you translate that to the speed of sound, then the shooter has to be somewhere on the hyperbola. Adding a third sensor allows you to solve for a specific location. Adding a fourth, fifth, and eighth sensor allows you to get some confidence.

Overdetermining the solution minimizes the error. Given how I just said that more sensors can give better solutions, you might think that stuffing these sensors in high crime areas would be a good idea. But in reality, it doesn't work that way. With ShotSpotter, part of the goal was to keep them far apart. Because if you look at a hammer and a gunshot, a hammer is impulsive, but it's not loud, not really loud, not gunshot loud. So you keep the sensors pretty far away, and the lower density means that you get less false positives. It's a balancing act, and of course, lower density means lower cost. So back to the story. The police dispatcher saw the red dots, found the bullet casings when the state police got there. The server had to group the pulses from the different sensors into individual gunshots

and then calculate the location for each gunshot using time difference of arrival. Because the suspect barricaded himself inside and threatened his family, the state police called in a hostage negotiation team. And the suspect was eventually brought into custody by police. No one was hurt. That's one of my favorite stories, because it starts with the barrage of bullets and ends with no one was hurt. And this low-density, this spatial filtering idea is great for making sure that we only hear very loud events or that shot spotter sensors only hear very loud events.

But it's still very possible to get coincidental events that just happen to all work out. A basketball game here, hammering over there, car door slamming over here. They all just happen to happen at the right time to lead to a bogus location. So we can use acoustic information about the pulse, as well as environment and system data to determine if the sound originated from one place, or is the coincidental location of multiple things. And these same types of information can lead to the difference between fireworks or gunshots. And if that seems so easy, let's see you try it with some acoustics. Okay, so we're going to classify these ourselves.

So we're going to play the first one. And now we're going to play the second one. Okay, so you listener, which one of those did you think was a gunshot? Which one did you think was a firework? And why? I mean, like, really think about it. If you could see the slide, it would just be the audio signature of them. The first one is bigger.

It's longer. It's got two big pulses in it. The second one is big, but it's kind of got a fat tail. And so, which do you think it is? Christopher, you didn't really hear them, did you? No. Never mind. It's all movie magic here.

I've heard nothing. So, the first one was the gunshot. And one of the ways you could have been able to tell is that the second one had the whistle of a firework. And that affects the frequency signature. And of course, that can be used to figure out fireworks versus gunshots. We're going to do two more. And this one is going to be harder. Where those before were taken from sensors about 200 meters from the incident,

these are about 600 meters from the incident. Okay, so we're going to play this first one. And now we'll play the second one. Now, if you're all in a room, I would demand you put your hands up. The top one or the bottom one. But, you know, which one is which? Which is the gunshot? Which is the firework?

And then I would ask if anyone wanted to change their answer. And you know what? It doesn't matter. I mean, the first one is the firework. But the correct answer is that there is no correct answer. The increased distance has attenuated the acoustic information such that it doesn't really matter. These are not useful to figure out gunshot versus firework. So there are some other ways to figure that out, some other classification techniques. Usually I always think of these as a little more plebeian,

but it's still very useful to know the day of the year. For some reason, there's more fireworks on the 4th of July than the 4th of November. And the hour of the day is kind of important. 8 p.m. versus 2 a.m. Yeah, one of those is a lot more likely to be gunshots than the other. And then there's the location. Some locations have more fireworks. Minneapolis had more sleet than fireworks during many parts of the year. So finding these obvious features is not... Wait, sleet?

Yeah. I think you need you explain that? Well, because the sensors would occasionally pick up sleet, but... Okay. When you're looking at figuring out if something is a gunshot, and the weather says it is sleeting out, it's not likely to be a firework. Right.

So weather plays a part. Gotcha. So yeah, finding out what is important and what matters was a huge part of the classification problem. And that's where I learned the Bayes equation. That's where I fell in love with machine learning, although I wasn't quite ready for it. It wasn't called that back then. It was called machine learning back then.

It was called machine kindergarten. Yeah then it was called machine learning back then it was called machine kindergarten yeah it was sort of machine kindergarten and each one of these things has to be the feature has to be identified and you have to have enough data to figure out which is which is it a good separator or just a good separator in certain locations and how do you know when you have a representative sample of each class? And how do you figure out what the truth is? A lot of humans had to listen to this. A lot of me had to listen to this. And it was years of data. And as good as the classification can be, there's always more that humans can do. And officers have told us about things they look for.

Let's listen to this one, the classified as a trained human one. It's an accelerating shot pattern. The officers have said the shooter is unfamiliar with the weapon, and it is sort of getting away from him. Will you play it again? Very often this is a young gang member, possibly an initiation shooting. And if you were here, I could show you the difficult acoustics. It's an apartment building on three sides and the shooter's in the center,

and it's not really important, though. Once they heard the audio, which the assistant presented, the officers knew what to look for, and they arrested a new gang initiate nearby, 14 years old, and they found the homicide victim too. He was 17. The heuristics that the dispatchers use, they're beyond the machine classification that the shot splatter system could use. And sometimes the key is getting the subject matter experts the information they need. So let's go on to how we do notification. Because all that information is no good unless it can get to the right people.

So, when the police dispatcher is using the system, or when it was in 2008, because I think this part's the most different, across the room, there might be a green screen on a black background that shows nothing's happening and then the red red comes up and a sound alerts yeah that's that's the sound alert and the audio notification is there because dispatchers could be a room away they're very busy people so So our imaginary dispatcher walks over to the system and she selects the unacknowledged incident, which displays where it is and adjusts the zoom of the citywide map. She can look at the audio form and wonder if it's a gunshot. She looks at the system confidence. It has a high confidence in this multiple gunshot incident. And she can listen to the incident. I think you can listen to that one.

Dotsbutter, of course, had not 100% accurate classification rate. So they trained the police dispatchers to listen to the incident audio before deciding what to do. In the cases where the dispatcher didn't agree with the system classification, they can reclassify using their personal judgment. And so what was she going to do with that sound? What was she going to do? What was likely the situation on the street? How many and what kind of police officers were needed to keep the situation as safe as possible for everyone? Well, the next thing she could see was exactly where these shots came from, because there were multiple of them. There was one, let's say, at time zero, and then 23 seconds later, and then a minute and 17 later, and then nearly two minutes later. And the first incident was in the back of a gas station.

A drug deal went bad. Someone fired near that station. Now, the police knew the drag races were common around this area, and the drag races are rowdy and troublesome, but not dangerous. Not like this. Once the drug deal went bad, though, everyone started shooting. And the second audio, the drag racers, or in the second shot, the drag racers had to see who had the bigger gun. And that spread into the third and fourth gunshot. Two minutes later, police are showing up.

There's been a massive amount of violence, and the dispatcher called an ambulance. Given this amount of gunfire, only three were wounded and one was killed. Without ShotSpotter in this incident, it's questionable that the incidents would have been called in at all. Less than half of all urban gunfire gets called to 911. We can't locate it. We don't know what it is. It's too far. And the police couldn't have responded so quickly because the shot spotter system would give them locations and not just, I heard a gunshot. So once everything's done, the dispatcher can write down the actions she took and go to about her other work. That data has been used as evidence in court to recreate the crime scene, confirm witness testimony, information like who shot first.

Officers also can use the ShotSpotter tool for short-term planning, looking at concentration of incident over multiple weeks, looking for hot spots, and looking for incidents where they might be able to predict and schedule police officers to be nearby. So that's the planning resource. And then there's a slide about databases that I'm just not going to cover, although I took great glee in it at the time, but now I'm not even admitting that I know SQL. The whole point with the ShotSpotter system is that it can do a lot. And it did a lot to make police more effective, keep them safe and keep others safe. And I hope the stories show that the system was building a better world, which I gather was the theme of the conference that I was giving this talk at. And there's a lot of different components. once the word word gets around that gunshots are not tolerated in the community a lot can change and did uh for the covered cities all right

i i now i'm just in the my area of questions and and additional stuff do you have any questions christopher come on ask me anything so i remember very little the question that everyone always asks for these presentations is these microphones are listening all the time uh what are the privacy implications of having microphones all over a city that's already covered in cameras so you've already given up but what about the camera microphones um well for the most, we didn't want the microphones to hear people. People are annoying in their insistence on making noise. And so you put them on the tops of buildings, and then you don't really hear voices.

There were times, having listened to a lot of audio, there were times I could hear things like kids playing in playgrounds but it wasn't like i could hear people talking and we didn't want to and we didn't store the data for very long and only the data that was around impulsive events so if you stood under a shot spotter sensor and you talk super loud and you clap at the same time, it might catch you, but only if somebody at the right location a mile away is playing basketball and another person is jackhammering at a different mile away location. It wasn't...

I could see how people would feel funky about it. Maybe it was just because I listened to so many birds peeping that I didn't really hear people talking. There are a lot of birds peeping out there. And there's really a bird that sounds like the predator monster. Like, really sounds like the predator monster. Wish I'd kept that audio. It was really good. It just goes by the predator, usually.

Yeah, well. Other questions. Why hasn't this gone everywhere? Are police... Are the departments... Are there downsides? Are they not excited about it?

Was it super expensive, or was it not effective enough? Or was it too effective? There are all kinds of problems and there are all kinds of solutions. System was not cheap. It certainly definitely was not cheap. And it had the downside that suddenly they knew about all of the gunshots. So you put in this expensive system and the expensive system says, wow, you've got a lot more work to do. We'll just turn that off.

As far as software engineers go, we've met that with the aforementioned Clang and PC Lint. Wow, look how much we paid for this. It's going to cost me months to get through the first days of stuff. So that was kind of a downside. And I don't, I mean, it has been used a lot. I mean, they're all over and the company has grown and to some extent they're changing

their focus a little bit. There's been some focus on, I think, some school stuff. But it's pretty amazing. And it went in a lot of places. It went in a lot of places you hear the most for crime. Oakland, some parts of San Francisco, Richmond, Washington, D.C., New York, Boston. I mean, 50 cities when I left in 2010-ish. So this was developed, I mean, most of the I left in 2010-ish. So this was developed, I mean, most of the stuff that was deployed was developed in the 2006-2007 timeframe, something like that, probably, because this presentation was 2008.

In 2000, after this presentation, we did a lot more on classification. I have a question coming. Oh, okay, go ahead. So, now that you don't work there you can speculate given where the technique bring i'm trying to bring us back to embedded systems given where the technology was then what radios you're using and cpus and things how would you redesign this system for free on the radio given what's available now? You have 30 seconds. You can just kind of spitball it. Yeah, well, there are some things that I know that I can't say.

Well, I don't even, I mean, my NDA must be like five years out of date. I would get a single board computer. Okay. Like what, like a Raspberry Pi level thing? Raspberry Pi 3 would be my minimum. Wow. Given what you said about it being a washing machine. Well, I'm going to go into them.

We're going to more. Maybe one of the new BeagleBones. Maybe even the Jetson. I mean, the Jetson would be over overpowered but you could do a lot more with it uh the tx1 maybe and the advantage to having a linux system is we had a lot of care and feeding of our file systems and our firmware updates and while putting up sensors was expensive the most expensive part of putting up sensors was installation. And having a system go down because of bad firmware was just heartbreaking. I mean, knowing that you had to send somebody

into a place where there are a lot of gunshots in order to fix your software bug, it was pretty painful. So I would do Linux because I think that the update opportunities are better. The hard part is the time stamping thing. So you need GPS. And I would put four or even eight sensors, microphones on the system with a very well calibrated 3D printed thing. And then you get the location. You can do the beamforming thing, that Amazon thingy over there that I can't say the name of can do. Yeah.

Yeah, actually, if you do have an Echo, it is like that. It's like where she puts her lights. She's creating the beamforming thing, as Christopher said. Triangulation. And so you have multiple sensors, so you're getting a direction. Now, you still don't have a distance. You need more than one sensor far apart in order to get a location. And that means you need to have some sort of radio form.

If you could convince neighbors all around the city that you were in, although, again, not close neighbors. You still need them pretty far apart. But if you could convince them to put them all on Ethernet and power, that would be awesome. Otherwise, you're going to need a solar panel and something like LoRa, the long-range radio. Or there are some

municipal Wi-Fi networks you can use. Or I suppose you could sniff off a Starbucks. I don't think that's a good business plan. I don't know. It wouldn't be that bad. So, yeah, I would do some higher power Linux, and I would do more features on the device.

I don't... It isn't trivial. I don't want to give that impression because the difficult parts are the distributed sensor network. Right. And you can't do that on your own, by yourself, one person, one sensor. Okay. Okay.

So, Kevin and Sammy, that was ShotSpotter. Hope you enjoyed it. It is not the same now. I have not been involved with the ShotSpotter in many years. Now it's all in the cloud. But I wish them lots of luck. I made that up. I don't know anything.

Yeah, it's in the cloud as they float by. Those clouds, right? Yeah. All right. So that question caused you to give a presentation. What's the next question you're going to do? Bobby asked about the mention I made of LeapFrog. And I mentioned that my toys were canceled one year,

and then it led to an epiphany recently about whatever it led to an epiphany about, because that isn't important. And I mentioned they shared a lot of code. And Bobby asked if I could share a little bit about how I prefer to share code to an epiphany about because that isn't important. And I mentioned they shared a lot of code. And Bobby asked if I could share a little bit about how I prefer to share code between embedded systems projects. And I'm sorry, Bobby. I gave you the wrong impression.

They did share a lot of code, but they were all on the same processor. And I shared code with a few other people who were all on the same processor. And our code, because it was a consumer product, because it was a toy, had a limited lifespan. As soon as we shipped, we did no more development on these toys. So when you think about it. God, that sounds wonderful.

I know, doesn't it? It was kind of magically wonderful. So when you work on sharing code and you think, okay, I'm going to share this code and it's going to be great. And then you realize six months later, you have a dozen processors and two dozen products. And how do you cherry pick from A to B and get the libraries right and not confuse the dependencies there is no good way what i'm sorry to say this there is no canonically great way to unravel this i think the game make is what does it that's what i've heard see make solves all the problems i know christopher's being facetious because I've heard him talk about CMake, and the things he's said about it is not suitable for unexplicit radio. There is no great solution.

If anybody knows one, let me know. And I do keep meaning to write a blog post about this and give a couple of options but you wanted a good cookie cutter solution and I'm sorry I cheated I didn't have all of the constraints that I know you probably have and so I think that's it I have a couple other things to talk about at some point but I haven't tried out my eikologic scopes yet so I'll save that for next time

cool i don't have anything else exciting well then i guess thank you for uh uh doing you know thank you for pushing the buttons thank you for pushing the buttons um yeah i'm sorry about the edit on this one thank you to christ Christopher for co-hosting and for producing this one is going to be tough so I do appreciate how much he let me talk if you would like to contact us you can always do show at embedded.fm or the contact link on embedded.fm we do like hearing from you so feel free to say hello

and now i i usually read winnie the pooh at the end of these so winnie the pooh is uh currently attached to a balloon hoping that the bees will um yeah after a little while he called down to you. Christopher Robin, he said in a loud whisper. Hello. I think the bees suspect something. What sort of thing? I don't know, but something tells me that they're suspicious. Embedded is an independently produced radio show that focuses on the many aspects of engineering. It is a production of Logical Elegance, an embedded software consulting company in California. If there are advertisements in the show, we did not put them there and do not receive money from them.

At this time, our sponsors are Logical Elegance and listeners like you.