Computer Architecture Podcast - Ep 18: Codesign for Industrial Robotics and the Startup Pivot with Dr. Dan Sorin, Duke University

Episode Date: December 17, 2024

Dr. Dan Sorin is a Professor of Electrical and Computer Engineering at Duke University, and a co-founder of Realtime Robotics. Dan is widely known for his pioneering work in memory systems. He has co-authored the seminal Primer on Memory Consistency and Cache Coherence, which has become a foundational resource for students and researchers alike. Dan’s contributions span from developing resilient systems that tolerate hardware faults to innovations in cache coherence protocols, and has been recognized by multiple best paper awards and patents. His work at Realtime Robotics has pushed the boundaries of autonomous motion planning, enabling real-time decision-making in dynamic environments.

Hi, and welcome to the Computer Architecture Podcast, a show that brings you closer to cutting-edge work in computer architecture and the remarkable people behind it. We are your hosts. I'm Suvarnai Subramanian. And I'm Lisa Hsu. Our guest for this episode is Dr. Dan Sorin,

who is a professor of electrical and computer engineering at Duke University and a co-founder of Real-Time Robotics. Dan is widely known for his pioneering work in memory systems and computer architecture. He has co-authored the seminal Primer on Memory Consistency in Cache Coherence, which has become a foundational resource for students and researchers alike. Dan's contributions span from developing resilient systems that tolerate hardware faults to innovations in cache coherence protocols and has been recognized by multiple best paper awards and patents. His work at Real-Time Robotics has pushed the boundaries of autonomous motion planning,

enabling real-time decision-making in dynamic environments. In this episode, Dan discusses his journey from being a university professor in one field to being a co-founder of a startup in another field, and how it all started from one fateful lunch with a colleague. The startup path is never simple nor straight, and he also tells us about the super interesting pivot of how real-time robotics began as a hardware startup, but is now a software company. And with that, let's check out the conversation. A quick disclaimer that all views shared on this show are the opinions of individuals and do not reflect the views of the organizations they work for. Dan, welcome to the podcast.

We're so happy to have you here. Oh, thank you for inviting me. I'm delighted to be here. We're excited to hear from you. Long-time listeners know our first question is always, what is getting you up in the morning these days in broad strokes? Well, I think for most of us in this field, it's research, right? It's finding something new that people haven't found before. It's an exciting job. I am always delighted to work with students to find new things, and it's even more fun when they

work. But even when they don't, it's a great job. I tell people, I can sit outside with students drinking coffee and talking about big ideas. And I can call that work. And that's a great job. It does sound a great job. At first, when you said you like working with students and talking about big ideas, especially when they work, I was like, does he mean the students or the ideas? But I'm sure you have hardworking students, Dan. Indeed, indeed. So part of the reason why we wanted to have you on the podcast

was because you have recently done something kind of interesting and out of the norm for a lot of people in our community. So maybe you could talk us through a little bit what inspired you to transition from, at least from being an academic at all times to being a founder of Real-Time Robotics. And tell us about the company, what it's up to, and what you've been doing for the last few years.

Sure. Perhaps unlike some of my colleagues who have always dreamed of doing a startup, this startup kind of found me. This wasn't on my to-do list as a professor. About 10 years ago now, I met a really interesting professor here at Duke, George Konodaris, does robotics. And like many of the projects I've embarked on over the years, this one was really sort of person-driven rather than immediately based on project. I didn't go in saying, I've got a great idea for robotics. So we had lunch together and I really just wanted

to find out, what do these people do? What do people do in robotics? I think I had this mental image of them sitting around playing with robots all day and that probably wasn't fundable. So I needed to figure out what it was they were doing. And so George was explaining some of the open problems in the field. And one of the ones he mentioned was robot motion planning, which is the process of computing how to get your robot, in this case, a robot arm from where it is to where you want it to be without hitting stuff. And it turns out it's the without hitting stuff part that's hard. It's not conceptually hard, just a bunch of geometry, but it's computationally hard.

And so as an architect, first thought was, well, just use a GPU. You'll be fine. And he responded, well, often it's not fast enough. And most of the time it uses too much power. And I'm like, as an architect thinking, this is the best thing ever. We dream about important computational problems that are too slow. And so my next question was, well, somebody must have built an accelerator for this because that's how computer architects think. And it got really quiet. And then we spent the afternoon Googling. And by the end of the afternoon, we're pretty convinced that computer architects and roboticists had never had lunch before. There was no real evidence that anybody had tried this except for one pair of researchers who went about it in a way that was not going to be successful,

and they dropped it quickly. And so we've got this wide open space. And as an architect, once again, you dream about this, right? This is a big thing. Can we make it go faster? So I brought in a PhD student of mine and an undergraduate who is just a terrific engineer. And we tried to solve this problem. And as I mentioned, it's computational geometry. So we figure we will speed up the computational geometry. Now, you guys are pretty smart. You're probably knowing already that this was a bad idea because we were effectively,

as it turns out, trying to compete with NVIDIA. GPUs are really good at computational geometry. And our ability to do that better than them was, that was not going to happen. But at that point, we took a step back and we thought about it and we're like, well, if we're building our own hardware, we don't have to use the same software either.

And that was a real critical step. When we realized we had the opportunity to co-design the software with the hardware, and that changed everything. And so we ended up using a very different algorithm, as opposed to just trying to make the existing algorithm go faster. And so over the course of about a year or so, we kept working at it, working at it. And we eventually had a prototype on an FPGA, a field programmable gate array, that was solving these problems that had been taking multiple seconds. We were under a millisecond. And it wasn't perfect. It wasn't robust to every possible situation. But we were solving a lot of

problems very quickly, like qualitatively, much more quickly. And considering this had been a longstanding problem in the field, a longstanding problem for industry, I like to joke that the startup found us because at this point we had something that was commercially very important and we were the only ones who could do it. That is amazing. Now I kind of understand as we were sort of prepping for this episode, how you talked about how you became more of a software company than a hardware company, because you started out wanting to build hardware, and then it became a really big co-design thing. Well, that was actually a later step. So we first started off building our own hardware. And when I say hardware, software, co-design, I guess I should have been a little bit more precise. I would say hardware algorithm co-design because the accelerator

that we built was not terribly programmable. Basically, it took in what the cameras were seeing as the current state of the world. And it didn't run arbitrary programs. It was not a general purpose processor. All it did was motion plan. But it was definitely hardware. Now, where things transitioned to software was over the years, as we discovered, people don't want to make, test, buy, distribute, sell hardware. And as a startup, certainly when we even considered using the word ASIC in front of the venture capitalists, for the most part, they ran away screaming.

We like to joke that if you wanted to see how fast a venture capitalist was, mention the word ASIC. It was largely true. They were not excited about that. But at the time, we didn't know how to do it any faster than on an FPGA board. So that was our original product. And we were selling it, but it was not the kind of business that was going to be long-term successful. People want software, if possible. Now, fortunately for us, in the course of trying to solve this problem on the accelerator, it changed the way we thought about the problem. So if we had sat

down originally and said, we're going to write software that goes faster, we wouldn't have found the solution we found. But because we had designed hardware, it forced us to rethink the problem, which led us to then designing software to run on general purpose CPUs that ended up being fast enough. So it was a really interesting process

seeing how being a hardware project led to better software without that hardware. Wow, so your software runs on general purpose CPUs. So you've gone from build, like you started out wanting to build an accelerator and now you're building, now you're using off the shelf hardware. Exactly, yep. What pivot? Well, I mean, from you're using off the shelf hardware. Exactly. Yep.

What pivot? Well, I mean, from a business point of view, it's brilliant. The only thing that was holding us back from that was not being able to do it. If we'd been able to do software from day one, software is a much easier business. It's easier to, you know, you don't have to worry about the boards not working. You don't have to ship stuff physically. It's just everybody prefers software. Yeah, yeah, no kidding. Right.

You mentioned that in the course of co-designing or improving how you solve the problem, you managed to figure out new algorithmic recipes or a new way of casting the problem algorithmically, which allowed you to move to CPUs or other kinds of compute platforms. So the company has been around for a while. So how has the evolution been since the early mindset shift

towards looking for new algorithms or better ways to cast your algorithm for different problems? And typically, one of the things is that algorithmic developments are also harder to come by, like algorithmic insights, because people have been working on these for a long time.

What has been your observation on being able to find these algorithmic insights for new problems versus maybe sometimes it's just applying this central insight to multiple different domains or different use cases with some central insight and how you recast the algorithms? So how have things evolved over the last few years in terms of algorithmic flexibility and the ability to be able to do that? Right. So let me start off towards the beginning and then get back to that if you don't mind. So as I mentioned, the original algorithms were computational geometry. People would effectively take the geometry of the arm, the robot arm, cover it in polygons, maybe triangles. Then they would do the same thing for the current space that was, you know, as viewed by the cameras. And then they would try to see if the triangles covering the robot arm would intersect with the triangles covering the objects and currently in the environment. So it's just geometry, lots, but lots and lots of geometry. What we observed,

though, was that there were a lot of things that we knew ahead of time that we didn't need to recompute at runtime. So, for example, we knew the geometry of the robot. We knew the dynamics of the robot or the kinematics of the robot. We knew that if you moved the robot in a certain way, we knew what volume in three-dimensional space it was going to sweep. And so we didn't need to recompute all of that necessarily at runtime. So what we effectively did was build, if you want to say, perhaps a library of known motions. So you imagine one of these six, seven, eight degree of freedom robot arms, and you move it by changing the joint angles in a certain way. And you simulate that,

and you know what part of the space it would hit if you moved it that way. And you do that for thousands of motions, lots and lots of little motions of the arm, where you're changing the joint angles and seeing what part of the three-dimensional space you would hit. And you remember all of that. Then at runtime, you see what part of your three-dimensional space is currently occupied by objects. And now it's simply, it's a lookup game, right? Because you know what space you're going to sweep and you can remember it as three-dimensional pixels, voxels. You can remember with fancier data structures, but ultimately at the end of the day, we've turned a computational problem

into a lookup problem. And it turns out you can do lookups really fast in hardware, a lot faster than division and subtraction and all of that. And so that was really the key for us in hardware was we turned it into that lookup problem. Well, you can do the same thing in software. And so a lot of pre-computation was the key to both of those. And sort of intuitively, any work that you can do before runtime, you should do before runtime. Why do that at runtime when time is so critical? If you're trying to make decisions quickly, you might as well have done everything you could possibly have done beforehand. And that was really the guiding principle behind all of these algorithms. Your question, though, was more, I think, directed towards now. And so a lot of what's changed over

the years has not been that guiding principle of pre-computation, but rather data structures. How exactly do you represent the objects? Do you represent the robot differently from a static object? Would you represent the robot differently from a dynamic object in your environment? And it turns out different data structures tend to work better for each of these. And over the years, we've gotten better at choosing data structures, cleverer algorithms for deciding what to check first. But it all does boil down to that original insight of pre-computation. Got it. No, I think that's a of pre-computation. Got it.

No, I think that's a great sort of, it's one of the oldest tricks in the book, but surprisingly effective. I mean, it's been known by different forms over the years, you know, lookup tables, memoization, memorization, all various forms of the same trick. Exactly. How do you see this relating to the kinds of problems

that you end up tackling? So you talked about how you can pre-compute a lot of things or you can apply that general mindset to a variety of problems that you end up tackling. So you talked about how you can pre-compute a lot of things or you can apply that general mindset to a variety of problems. I'm guessing this is specific to the kind of problems that your company actually tackles, maybe industrial robotics and so on. So what are the places where it's easy to adopt this?

What are places where it's challenging to adopt this form of problem formulation? It really applies all the time. I mean, you know, when you're using a robot, you know what robot you're going to be using. You might even know something about your environment. You might know, for example, that the robot is mounted on a table that's three feet up. You might know, for example, the robot's never going to do anything behind its metaphorical head. And so any information that you can incorporate beforehand, and you're going to have some, you should do. Anything you

can use from beforehand, use that. And so it applies. I mean, whether or not the robot's being used for inspection or welding or painting, at the end of the day, you still have to move the robot from point A to point B without hitting stuff. And so it's why motion planning is so foundational to robotics, why it's such an important problem, because it's very general. Anything you want to do with a robot, you're going to have to be able to figure out how to get it to where you want it to go. So this sort of begs the question, as Sue and I said, this is an oldest trick in a book. But at the same time, for all these tricks, there's something that you have to do when something changes and you have to react fast enough. So I guess, you know, what kind of rates are we talking about here? Like I can imagine, you know, if you're planning

to move the robot at a certain rate and as long as nothing changes, then you're good. But like maybe a mouse starts across the room or, you know, a kid goes like, you know, and like throws their hand in the way. What happens then? Just as an aside, in the industrial facilities where these types of robots are used, there are no children. That's probably a good thing. We did have a very exciting moment very early on at our facility in Boston, where a news team from one of the local TV stations came by. And one of the newscasters wanted to ask if he could stick his arm in just to see that the robot would move around his arm when the camera saw his arm. And while we knew it would work, it was a little bit disconcerting still to see the

guy stick his arm in. And it did work because otherwise that would have made the news in a very bad way. So typical reaction speeds, you want to be aligned with your cameras, basically. So if you're getting 60 frames per second off your camera, being able to replan 60 times per second sort of makes sense. Being able to replan at a kilohertz is likely to be overkill. And so early on, we were actually planning at a kilohertz and there are no cameras operating at a kilohertz. And there are really no dynamic scenes with arms where you need to plan that quickly. Should this technology later be used in what's called a co-bot, a collaborative robot, what that means is that the robot's working around humans.

And when you work around humans, you make worst case estimates for how fast the human could move, et cetera, et cetera. And so you would need to be able to react quickly enough for that. But at 60 Hertz, we can keep up. We have scaled the technology to the point where we can keep up with one of our customers who has 16 robots in a given workspace. And so we can be controlling all 16 robots in real time if needed. So give us a sense of what that means. Could a robot play ping pong like would it be able to react fast enough to play like you know average garage ping pong say you or me that kind of uh stunt robotics has been done before and in some and by the way it looks really cool but at the same time it's actually not as hard in many ways because unless you're playing doubles or you're worried about hitting the person next to

you, there isn't actually any collision detection to worry about. And that's the hardest part of motion planning. It's not figuring out how to get from point A to point B. It's getting from point A to point B without hitting stuff. And so if you're in an empty environment, like you're playing ping pong, it's actually not as hard in many ways. Okay. So then let me reframe the question, because I guess I'm looking for, what does it mean to be able to react at 60 Hertz? Let's say you've got a robot doing a task where it's got to move bonbons from the conveyor belt into the box and somebody is hitting a ping pong ball across the path. Is that, will it avoid knocking into the ping pong or maybe a paddle, something larger,

throwing the paddle across the path? Like will it hit the paddle? Ooh, interesting question. I have to confess, we've not considered that one. We've yet to find a facility where they're playing ping pong over the robotic work cell. So maybe reframe again. I'm just trying to get a sense of how fast, like a real world scenario where something that 60 hertz is like that it can avoid. Meaning, is there something that's too fast for it to avoid?

Like what is something that would be too fast for it to avoid at 60 hertz? Sure, certainly. I mean, if the robot was working in a work cell and a car comes through just totally facetiously or a drone going way too fast. If a human really wants to get hit, the human could reach out and punch the robot, and the robot's going to have a hard time getting away from it quickly enough. So there are scenarios where 60 hertz wouldn't make sense, but I'm having trouble thinking of practical ones. Sure, it makes sense. Okay, that helps.

It's just like trying to get a frame of reference of like what sort of disaster scenario it would be. Of course, I'm sure that industrial robots are very controlled environments. So yeah, you don't want to over-engineer. Exactly. There was another consideration that you touched upon right at the start,

which was maybe GPUs originally were very power-hungry for GPUs cases. Can you touch a little bit on that aspect as well? Like how important is power in your overall consideration setup for the industrial robots that you have? And how does this change depending on the algorithm, the computational platform that you use? And would there be avenues for further improving that? So let's talk about the power issue first, right? So if your robot is plugged into the wall, okay, so you don't have to worry about running out of battery.

But if you're a large facility with many, many, many robots, and we're familiar with those, 300-watt GPUs add up. You talk about a place like a car maker or an Amazon or one of those, that is a significant cost. For robots that are untethered, boy, oh boy, a 300-watt GPU or something like that, your battery is not going to last particularly long. Our technology, and this is a bit of an aside, applies to all types of robots. And at the end of the day, an autonomous vehicle is just a robot with wheels on the bottom.

And we've been approached in the past by companies that are either in the AV space, I'm sorry, autonomous vehicle space, or companies that want to build the brains for autonomous vehicles. And one of them, this was years ago, mentioned that to do the motion planning they needed to do, they had effectively put a supercomputer of GPUs in their trunk, and they had to cut a hole out of the trunk to avoid overheating them in a way that was dangerous. So power can matter in many different scenarios. What we're doing is just not as power hungry. And if you think about it sort of, once again, intuitively, well, all

that work we did beforehand is work we're not doing at runtime. And work, to some extent, correlates with power, right? Not necessarily linearly, it's not doing at runtime. And work, to some extent, correlates with power, right? Not necessarily linearly, it's not a perfect relationship. But by doing simpler work at runtime, we're using less work at runtime. So you mentioned that the startup found you. Did you mean by that, that you had this cool work with your fellow Duke professor, and someone actually approached you or just that you were like, wow, this just kind of like begs to be made into a startup. And if the latter, why did you start it in another state? So it definitely is the latter. If you're

going to do a robotics startup in the US, you want to be in a place that has a robotics ecosystem, talented people who know the field. And that pretty much left us with Boston and the Bay Area. And uniformly, we preferred to be in Boston for a variety of reasons. My colleague, George Konodaris, moved from Duke to Brown University, which is close to Boston. It was easy for me to get to Boston. And just for a variety of reasons, we chose to be there. And it was a wonderful decision. We started out at a place called Mass Robotics, which is an incubator there for robotic startups, which was just a godsend for us. And it helped us to get up and running. So how long has it been now? A remarkably long time. So I just checked not that long ago. My first lunch with George was

in 2014. We started full-time in Boston in 2017. So it's been a while. I mean, by startup standards, I think that's an eternity. Yeah, no kidding. Kind of like dog years. It's like 50 years. So what are your plans now with respect to real-time robotics? A lot of times startups have some sort of exit plan. They want to be acquired or they want an IPO or they want something. If you feel comfortable discussing, do you have any thoughts on what your steps are? The standard advice given to all startups is you want to operate as if you are going to make it all the way to IPO and beyond. You have to build an enduring business model. And we were warned, any company that sort of tries to

pretty themselves up for an acquisition runs the risk of then not getting acquired and not having anything. And so we're operating this for the long haul. Now, practically, if a company comes through with a godfather offer, we are financially obliged to take it very seriously. I don't think I'm telling you anything you haven't heard from other startups. Yeah, yeah, yeah. So I mean, part of the reason why I asked this is because we had Karu on and he discussed his startup for a while. And one of the most fascinating aspects of that discussion was the sort of broadening of the mindset from the technology to the business. Like here's the technology. It's so cool. We're going to make a company around it. And then there's the, Oh wait,

how do we get somebody to want to pay us to, to, to get this thing, whatever this thing is. So I guess part of my questioning, like the line of questioning is like, okay, so you've discussed how you've gone from this pivot from hardware to

hardware algorithm, co-design to writing exclusively software that can run a general purpose GPUs. And then there's the incubator in Boston and maybe the whole long-term plan of going in it for the long haul. I guess I'm trying to pinpoint at what point along the startup trajectory did these strategic pivots happen and what made you do it and when and that sort of thing. Does that make sense? First things first, I will commiserate greatly with Karu. The gap between really cool tech and really cool product was a lot more difficult than we had anticipated, despite people, of course, telling you it's a lot bigger than you think it will be.

So early on, we had tech where people would come into our place, see a demo and go, I want that. That's cool. And then we show them the product and they're like, I don't want that. And it's been a very long journey going from cool tech to cool product. And in a way that as a professor, that was something that obviously has never been an issue. The joke among academics in startups is, in academia, your thing has to work once, right? Then you get numbers, you get a video, and you publish. And that does not fly in industry.

You know, it has to be usable by mere mortals. It has to have a good user experience. It has to solve the right applications in the right way. And then, of course, all the business stuff that Karu talked about as well. You can even have a great product and you still have to get through to large multinational companies that have ways of doing things. you're told certain things, you can still find them surprising because they just intuitively don't make sense, or at least not to me. But that gap has been intense. So in terms of pivot points, boy, I mean, the big pivot point was going hardware to software. And that was, from a business point of view, very important. We could have kept going with the hardware product, but it was really going to greatly limit what we could do

in a way that scaling up software, there is a really huge potential market here. And so that happened after you had already incorporated as a business and did you have funding yet already? Or was this partially from feedback from VCs? Well, it wasn't the case that we were building hardware because we thought it was a better idea than software. It was that we were building hardware

because we didn't know how to go fast enough in software. Once that changed, it was a really easy decision. Everybody prefers software. And that had never been in question. It was just a matter of, could we do it? Gotcha. So it was really that epiphany of converting from computational geometry to lookups. Well, that epiphany enabled us to get to fast hardware.

Looking at it again enabled us to get from fast hardware to fast software. Gotcha. And more importantly, having really outstanding engineers helps us to get anywhere. I get the fancy title of co-founder and chief architect, but all of the real work has been done by an incredible team of engineers. And a huge tip of the hat to them for being able to do absolutely remarkable things. That sounds great. I'm glad that you were able to sort of make these inflection point changes at the right times, keep the company going and looking forward as well. Yeah. Plus, it enables me to say that I'm chief architect of a company that's not architecting hardware. This space is not static. There's a lot of evolution, new advancements, new developments

that are coming up in the broad robotic space and also like allied technologies. So looking forward, what are some of the challenges on the horizon or opportunities that you're excited about in this particular space? Sivane, this is probably actually not the best question for me. At the end of the day, I'm actually not a roboticist. So I can talk about this, but I would feel a little bit underqualified. I don't know that I'm the right person to answer that. I think though, what I'm taking away as being really amazing about this is that here you are, co-founder and chief architect at a robotics company saying unequivocally, I'm not a roboticist. And it sounds like the company is going great and has made some really cool decisions and is doing well.

And it all started from potentially the first luncheon between a computer architect and a real bona fide roboticist. So one of the themes that I feel like has been sort of come up over and over again throughout our many episodes of this podcast is that everybody we talk to, you know, has, we invite them on the show because they have an interesting story and they're successful in some way, shape or form. And they all say we have to be able to communicate and we have to be able to work with other people. And that sort of like cross-functional, cross-layer, cross-everything is the way to go. And I just feel like this, your story is like really part and parcel with that. Because I remember when I first started seeing you posting on the various social media about this robotics company,

I was like, but Dan is like the wonkiest memory coherence and consistency model guy that I know. Like, what is he doing in robotics? I've always been drawn to interdisciplinary projects. I love working with people who do stuff that's different than me. I spent 10 years working with somebody here who is an expert in information theory. I work with people who do formal verification. I work with circuits people.

It's really fun. And I don't consider myself an expert in any of those fields, but it's a whole lot of fun to work with people who are. And I think robotics just sort of fell out of that same pattern of how I like to work. I like to find people doing interesting things. And sometimes there's an intersection and that could be a lot of fun. So maybe this is a good time to sort of talk about your more academic pursuits and find out a little bit about how you got to where you are, your origin story as a computer architect, in part because you just mentioned you like to work at the intersection of things with lots of different people and that like academia is like the best place to do that. And at the same time, it does seem like the bulk of

your work as if, you know, if I were to close my eyes and imagine Dan Soren, the first thing that comes to my mind is consistency models. So I guess maybe talk about your story and how you ended up perhaps focusing, perhaps not, or at least being most well-known for that particular sub-area? Sure. So the story, as for many of us, starts with an inspirational class and an inspirational instructor. I was an undergraduate at Duke, and I took three classes, including an intro computer architecture class from the same instructor, Professor John Board. And I'm like, this is the best thing ever. I understand how a computer works. This is great. I want to do this. And from there, also looking at what he did, I'm like, that looks like a cool job, but I'm going to need to go to grad school to do that. And so applied to grad school and very

fortunately trusted my gut and went to Wisconsin. I will. And by the way, my gut was very cold because I visited over spring break and it was frigid. I went there anyways, but I had just a really good feeling about the place, about the research that was going on. I talked with David Wood and Mark Hill about cache coherence and memory consistency, which I knew nothing about at the time, but sounded cool. And particularly coherence really appeals to me. I like puzzles and I find coherence protocols to be just puzzles. And I really enjoy that aspect of it. So I got hooked on that in grad school and I had wonderful collaborators. I worked with both Mark and David. I worked with a couple other professors there. Had wonderful grad school colleagues.

Worked very close with Milo Martin, among others. And just thoroughly enjoyed it. There are people who talk about grad school and like, oh, grad school is terrible. I loved it. It was great fun. We did good work. I got to work with great people. The whole thing was just, I had a great time.

And so from there, I was like, yeah, I'm convinced I want to do this. And I went into academia. I was fortunate enough that I was able to get a job here on the faculty of Duke and pretty much been here ever since. And I feel like I found the right job for me. I certainly know there are many people who go into academia and miss writing thousands of lines of code or being able to just work on one thing all the time and they go back to industry. That's not me. I really, this is the right gig for me.

That's awesome. Potentially it's because I can't write code very well anyways, but we'll leave that aside. That sounds great. And you mentioned your students a couple of times. potentially it's because I can't write code very well anyways but we'll leave that aside that sounds great and you mentioned your students a couple of times so I'm sure teaching was maybe another passion of yours how are you thinking about it in the current style do you have a specific style for teaching and of course as I said the industry does not remain static or the entire field does not remain static so how do you think about teaching any philosophies that you have? You have a widely

read book on our primer that you've co-authored on memory consistency and coherence that is like a standard entry point for a lot of people who are entering architecture. So how do you think about teaching your philosophy towards it, how it's evolved over the years? It probably depends on the course. I mean, I teach a large undergraduate computer architecture course, which is kind of similar to the one that got me hooked. And I think because of that, I tried to see if I can hook some of them. And bringing the same energy that my instructor had and the same passion for, this is amazing. Remember that magical box, that computer? We're going to understand how it works. And that's a big deal. The phrase keeping it real is probably not exactly what I meant here. But the fact that this is all

very real world. When I talk about how a processor works, it's not something abstract. It's like, you've got one of these in your phone. I was teaching cache memory today. This isn't some obscure thing that academics talk about and write papers about, what we do, but it's in your phone, it's in your tablet, it's in your laptop. And so I think being able to tie this to the real things that people use and are excited about really helps. It's only at the undergraduate level. i think by the time they're at a graduate level course they're pretty convinced that this stuff is real uh hopefully i think for me one of the keys is just simply being energetic about it i think i respond very well to people who are energetic and seem to care a lot about what they're doing it helps if they're a little bit entertaining

potentially funny too. And I try to bring that to class. I try to keep people interested. I realize people have a relatively short attention span, which is why I'm the only person I know in my department who chooses to teach three days a week for a shorter amount of time rather than twice a week for a longer amount of time. You know, at 50 minutes, even that, I don't think people really have much of attention span beyond that. And by that point, I'm probably boring myself anyways. And so I'm happier to teach. There's the overhead of teaching three days a week, but I think it's better. And so I'm trying to meet the students where they can find something interesting and engaging, and I don't lose them by the end of class, or not too much. So this, first of all, that's amazing

that you sort of self-sacrifice your own personal schedule, teaching three times a week at 50 minutes instead of twice a week for 120. That's big on your part. That's amazing. Kudos to you. But what I sort of wonder about computer architecture, because I taught the high-level undergrad,

grad-level course a couple of years ago via Zoom at Princeton because it was the pandemic and I could. And one thing that was interesting about it is that a lot of it was the same as when I took it, you know, decades ago. And on the one hand, that's kind of amazing. And on the other hand, it's kind of like, it felt like, oh, is it still really the same? And then at the same time, though, there are certain things that you have to know. And yet computer architecture is getting bigger and bigger and bigger and bigger. So, you know, there are many people out there who are probably designing hardware who probably maybe don't need to understand a TLB because it's just not this.

It's not a classical general purpose CPU anymore. It's some sort of accelerator, which maybe doesn't even have virtual memory. So I guess what I wonder is when I think about some of the more basic sciences, like say physics, the field just seems to maybe grow and grow. Now there's a subfield. You can't just be a physicist.

You have to be like a particle or whatever, or a string theory. And like, there's all these subfields. At what point does computer architecture just need to split itself into groups? And this actually is a very natural segue because you are the next chair of ISCA 2025, and you have mentioned have to reassess where we spend time. So, for example, we used to spend a lot more time talking about how to border cores, and that's gotten compressed. I mean, we need to make time to talk about other things. It's affected the way that we teach our two graduate or upper level architecture

courses. Historically, the first one was everything you need about a processor, everything. And the second one was all about multiprocessors. Well, that split no longer makes any sense. I mean, we can't send people off into the world who haven't taken the second course and thus have never known anything about multicores. That doesn't make sense anymore. And so you're right, as the field changes, we need to include certain topics, but also certain things that used to be niche, like multiprocessors, are no longer niche. Everything's a multicore.

So yeah, that part of it has forced us to change things over the years. And once again, coming back to my love of puzzles, I do find it to be a fun puzzle to think about, how do I make this material fit? What's the right order? How does this all fit together such that they're going to get all of it? And I enjoy that part of it. As far as the field itself getting too big, boy, I don't know what to do about that. But I do know that our flagship conference, ISCA, has gotten gigantic. Current projections are like we're

going to have on the order of 100 papers this year, 500 to 600 submissions. I don't know where we're going to end up on this, where this is going to converge. I'm going to use the back in my day line that I probably shouldn't use yet, but we used to be able to have a program committee where we'd sit around with 20 or 30 people, sit around the table and discuss papers. This year, we have a committee of 300. That's just not going to happen. And so it's a fundamentally different experience and a fundamentally different process for selecting papers. I wish I had a solution other than to perhaps whine about the scalability not being great. But I think as a community, you're right. I mean, it's something we need to think about.

You know, there have been communities over the years that have broken into sort of subfields or tracks. I don't know if we're ready for that, though. I mean, that's probably going to be a decision for the community to make as a whole. But I'm not quite sure where we're going to end up with this. But what we're doing right now can't continue to go in this direction. Is this the largest PC ISCA has ever had? I would assume so. Yes. Last year's was quite big, and we got advice from them to go bigger. But at a certain point, I mean, 300 people, yikes. Just even simply mapping papers to people. Do we have the right set of expertise among this group? Wow, I'm not totally sure. You know, when you have a group of 30, you can really sort of curate people the wrong word, but you can't individually pick each person based on how they're going to fit into your expectations for review load. Right. Yeah. So for 300 PC members and like five, 600 submissions, I mean, the community and just in terms of number of people has gotten huge.

And it also seems like that conflict list is going to be very hard to manage. Well, our colleague Boris Grot is championing the use of a new system, ConflictDB. So if that does end up being rolled out for the Cisco, which is the plan, please tell all of your friends to use ConflictDB because it's going to hopefully help us to manage that. Sounds good. Yeah, I'm sure next year's ISCA will be, look forward to a great program next year's ISCA and hopefully all the great submissions as well, you know, very soon.

Maybe actually near the end of the episode, sort of looking back, since you've sort of donned multiple hats, you've been an academic, you've had the journey of, you know, founding a company, co-founding a company and seeing it through for all these years. Any advice that you would have for our listeners? This could either be students or even academics, maybe considering a stint in entrepreneurship.

Anything that comes to your mind that would be useful advice? I don't know if this qualifies as advice, but it's something that has changed in me as a result of doing the company, which is now that I've seen this process of taking technology into the real world, it's a little tougher to get excited about research if I can't see how it might make it to the real world. I don't mean that I have to do research that's immediately applicable to a next generation processor or even the generation after that. But I think there are certain perhaps more abstract or niche things where they're interesting. Other people would enjoy reading them, but I don't see where they're actually going to make it to the real world. I have a harder time getting excited about that now.

That has changed in me through this process of entrepreneurship. And I don't know, we could argue whether it's for good or for bad, but it's certainly, it's affected the way I do research here. Obviously, I mean, the advice I would give to most people, I think is pretty standard. It's do what you love, right? If you can be lucky enough to have a job like the one I have, then you really do have that freedom to do what you like to do. That's really exciting. But I realize I have the privilege of being in a position where I can do that. And that's not standard issue in this world. Yeah. I think, thanks for that. And it's pretty clear to me that one of the reasons why you get to have a cool job that you love is because you do great at it.

You're conscientious about it. You work hard at it and you've done great work in the past. And therefore you get to keep this job, you know? This job, even so, it requires great collaborators. There's just absolutely no way to get to this without having great collaborators throughout. You know, one of my skills has been choosing great people to work with, starting in graduate school and throughout. And that's really critical. You just can't do this kind of work or get to this place without that. Yeah, you're not the first guest to emphasize that. And I hope our frequent listeners, maybe through osmosis or repetition or something, sort of like make sure that that comes through.

Because it may be similar to the whole technology versus product thing. Like you can have a cool idea, you can have a cool technology. But if it can't actually sort of get out there, it's like if a tree falls in a forest did it really fall or something so well dan i think that was a really cool conversation i did not we both see when i and i you know we do a little bit of prep for each of our episodes and we were like we don't really know that much about robotics and it does look like you've published some about it, but we, we, we were able to know that it was mostly about motion planning, but did not have our usual list of like really sort of more technical

questions. So you've been really good at that kind of preparation because we were like, we, I don't know if we're good at qualified to a ask these questions and be B, we don't know exactly what's going on to be able to ask targeted questions. So we were just kind of like, let's let this one flow as it may. And I think it turned out very interesting. I think so too. I can fake being a roboticist for a pretty long while. And I sometimes actually do because one of the jobs I have in my company is I'm basically in charge of finding the people who work there.

And so I've done over 500 interviews now, many of them with people in robotics. And actually, one of my criteria is if I'm interviewing somebody in robotics and I know more about it than they do, they don't get hired. So it's a useful benchmark. But I've learned enough over the years to be dangerous. I can fake it for a while. That's great. I mean, that's all most of us want to ask for. I feel like, personally, I'm a generalist,

and I know enough to be dangerous about a lot of things, but I really don't know that much about anything, honestly. I would disagree, but okay. Thank you. Thank you. But we could have talked about cash coherence for an hour or two, but that might've had a smaller audience. Probably. Although I enjoy that topic myself more than I will say like in our,

in our giant field, I'm not much for programming languages. I'm not much for maybe even, yeah, probably. That's probably the one that i like tend to not read those papers not because it's not great but it's just like that's not my thing but coherency i do like that how about you subine what's your what's your sort of favorite subfield and the subfield that you're like i don't know i don't get it i mean going back to the analogy of puzzles i was actually drawn to coherence and consistency because even like the very first introductory examples or programs that you will see in

consistency, like you'll have two threads and two different programs. What can the registers be after this executes, given a particular consistency model or maybe not? I think that was the hook for me very early on, because I was utterly fascinated by the fact that you could end up with options or values that you would not expect when you think about it in a sequential manner. And I guess the related part was I realized that humans are not very good. We are very good at doing sequential computing that falls very naturally within our paradigm. Even the way we think about algorithms is very sequential in nature. However, a lot of hardware is inherently concurrent or parallel.

And so that leads to a lot of unusual, what might seem unusual because we're not used to thinking in a non-sequential manner. And I think that sort of drew me into coherence and consistency very early on. And maybe that was one of the early hooks for, okay, multi-cores and large-scale computing and so on. By the way, I do that example,

that two-thread example. Love it. You get to blow minds away. It's really just a lot of fun. And even when I was a TA for graduate compute architecture classes, I think I used to assemble a lot of these questions over time, each with different constraints. You would initially start

with a simple two-thread example, and then you would start enforcing constraints on either the interconnection network or some other aspect of it. And then you could you would also gradually expand the example itself, come up with interesting solutions on what is possible versus what is not. So I found that exercise quite fascinating because I also tend to enjoy puzzles quite a bit. Yeah. And it was a really good hook and I hook. And I think students also generally enjoyed it. If you guys both go to ISCA, you should talk then

about your favorite puzzles. Well, I feel like we should close, but I do wonder with this conversation, how good is ChatDPT at consistency models? I've never tried to figure that out. I thought you were going to ask me how good is AI

at motion planning. That was the question I anticipated. Well, how good is it at motion planning? Good, but not good enough. So let's say you're AI and your job is to classify as this thing, cat or dog. If you're right 99.5% of the time, you're good. If you're doing motion planning with it and you're right 99.5% of the time, 0.5% of the time, you've caused a major disaster with a large, scary robot arm, and that's not okay. And the reason we don't use AI is because we don't have to. If you can solve something deterministically, don't use AI, right? I mean, if I ask you to multiply two numbers, you don't plug it into AI. There's a deterministic algorithm, we just go do it, right? AI is for if you have some function that you don't know what it is,

right? There's no closed form solution, you plug it into the box, an answer comes out. But if you can do it deterministically, don't mess around with AI. Yeah, there's a reason why we didn't ask you any sort of AI questions with respect to this, because it didn't seem like it fit or it was a part of the model. And I guess it's good that we didn't, because it turns out that it's not a part of real-time robotics, at least for your particular use case. Because AI is such a hot term, eventually with enough questioning from the VCs, we're like, yeah, fine, it's AI. But don't put that in the podcast. If you're doing task planning,

which is the other part of robot planning. So motion planning is figuring out how to get from point A to point B without hitting stuff. Task planning is deciding you want to go to point B in the first place. So figuring out what it is you want to do. And there's no closed form solution for that. That's obviously going to have to be AI. But motion planning can be done deterministically. Got it. Yeah, I guess a related theme was, you know, when things are very deterministic, as you said,

and you know that you have a closed form solution or something that can be determined very quickly, then you can use that. I was wondering if there are use cases where you do need that fuzziness. You mentioned there are open solutions or open problems where you might need to use AI. I was wondering if there are fuzzy inputs or fuzzy outputs. So task planning, there's no option. Object recognition.

So computer vision's good, and deterministically, you can tell what part of your world is occupied. But if you're trying to tell, you know, is that a box or a mug or just classify objects, that's all done with

machine learning. Grasping. So if you've got an end effector, end of arm tool that's trying to grasp, how do you approach the grasp? That's all done with machine learning because there aren't good closed form solutions for that. So there are parts of what we do that definitely involve, you know, adjacently, there are things that are performing machine learning or AI. But the core of it, the motion planning engine itself is deterministic.

Anyways, that was a question you didn't even need to, you didn't even ask and I'm still answering it. So I should probably let you go. It was on our list, actually. We were trying to think if we should actually ask it,

like, because there's motion planning, control, perception, decision making, right? And we were wondering if we should go down those rabbit holes. But as Lisa mentioned, neither of us are also roboticists. So, like, decision-making, right? And we were wondering if we should go down those rabbit holes, but as Lisa mentioned, neither of us are also roboticists. So we thought we'd just stick to the specific domain that your company works in. We've gone into decision-making.

Perception, of course, is, I think, heavy with AI these days. You know, you can always ask questions. I've had to interview people for my company who have PhDs in robotics. And clearly they know much more about all of that than I do. And yet I'm still trying to interview them. So, you know, don't worry about it. You can wing it. Yeah, sounds good. Well, again, Dan, thank you so much for joining us today. This has been a really fun conversation. And this is emblematic of how I think

our episodes go where it's just kind of like we go around and we ask questions and the conversation goes in some places. There's some things that we definitely want to make sure that we hit on and that there's some things where it just ends up being a great conversation that maybe other people would be interested in being a fly on the wall. So thank you for

joining us today. We're really happy to have you. Thanks for having me. This has been great. Yeah, echoing Lisa, thank you so much for your time today with us. And to our listeners, thank you for being with us on the Computer Architecture Podcast. Till next time, it's goodbye from us.