Computer Architecture Podcast - Ep 12: 50th Anniversary of SIGARCH Special Episode with Dr. David Patterson, Dr. Norm Jouppi and Dr. Natalie Enright-Jerger

Episode Date: June 10, 2023

This is a special episode to commemorate the 50th anniversary of SIGARCH. We have three leaders from our community who have served as SIGARCH chairs -- Dr. David Patterson, Dr. Norm Jouppi and Dr. Natalie Entright-Jerger -- reflect on the evolution of the computer architecture field as well as our community over half a century, and share their perspectives on opportunities and exciting times ahead. David Patterson is a professor emeritus at ​​UC Berkeley, a distinguished engineer at Google, and recipient of the Turing Award. Norm Jouppi, a VP and Engineering Fellow at Google, where he is the chief architect for Google's Tensor Processing Units (TPUs), and a recipient of the Eckert-Mauchly award. Natalie Enright-Jerger is a professor at the University of Toronto, where she is the Canada Research Chair in Computer Architecture, and is a recipient of the Alfred P. Sloan Research Fellowship, and distinguished member of ACM and IEEE.

Hi, and welcome to the Computer Architecture Podcast, a show that brings you closer to cutting-edge work in computer architecture and the remarkable people behind it. We are your hosts. I'm Suvaynai Subramanian. And I'm Lisa Xu. Today we bring you a very special episode. This year marks the 50th anniversary of the International Symposium on Computer Architecture, known as ISCA, to be held in June.

In honor of this milestone we thought it would be a good opportunity to reflect on the evolution of both computer architecture as a field as well as the SIGARC community over the last half century. We are delighted to have three leaders from the community Dave Patterson who was SIGARC chair in the 90s, Norm Jopie who was SIGARC chair in the aughts, and Natalie Enright-Jerger who is SIGARC Chair in the 90s, Norm Jopie, who was SIGARC Chair in the aughts, and Natalie Enright-Jerger, who is SIGARC Chair-elect, to share their thoughts and viewpoints on the past, present, and exciting future for SIGARC and computer architecture. A quick disclaimer that all views

shared on this show are the opinions of individuals and do not reflect the views of the organizations they work for. Our first guest needs no introduction, but let's give him one anyway. Dave Patterson is a professor emeritus at UC Berkeley, a distinguished engineer at Google, vice chair of the board of directors of the RISC-V Foundation, and director of the Rios Lab. His research has spanned a wide range of problems in computing and has had significant impact on the industry, particularly with reduced instruction set computing, known as RISC, as well as redundant arrays of independent disks, known as RAID. He is a co-author, along with John Hennessey, of the canonical computer

architecture textbooks Computer Organization and Design and Computer Architecture, a Quantitative Approach. In addition, he is a recipient of the Turing Award. He is a member of the National Academy of Engineering and a member of the National Academy of Sciences. Dave has also served as one of the early chairs of SIGARC and has been an invaluable shepherd to the SIGARC community and ISCA. Dave, thanks for being with us today. We're so glad to have you. It's a pleasure to be here. So the 50th milestone for ISCA is a significant milestone. So if we can sort of wind the clocks back a bit to the early days of ISCA and SIGAR. What was the community like in those early days when

you were on the board? What are the exciting developments in that time? Well, you know, it was obviously the community was smaller and the conferences, you know, what really striking to me was that the program committees were 20 people in the room talking over a paper and, you know, it was easier to make the case for a paper that had mixed reviews that you thought was very promising you know which is harder to do today they early relatively early we decided to try and do one-third of the conferences internationally that became the norm and that was popular idea so we saw ourselves as a genuine international organization and tried to move the conference around and sort of make it easy for everybody in the world to participate. I think those are the things that really stand out.

I think another thing that I've always liked about it, especially in the early days, is heavy participation with industry. There are fields where the industry and the research communities don't get along very well, but we've always, you know, valued the industry contributions, and there's a lot of the great papers are written from industry, not just from academia, so I think that's what I think of our roots, which I'm, something I'm proud of. If you were to go back to that time, do you feel like you could have predicted the growth of our community to what it is now? No, I don't think there was any sense that the size of the community was going to explode. We probably assumed some slow linear growth, but now with, I don't know, 500 papers being submitted every time that that was

unimaginable right so you mentioned the focus on the industry right from the early days of isca and you've seen a lot of ideas come out of your research make a significant impact on the industry and from many others as well so over the years and fast forwarding to the present day how do you see the current interaction with industry? How have things sort of evolved in that particular landscape? And how do we continue to make an impact where the ideas from our research community are being tailored to the problems that industry faces

and have a path forward towards impacting industrial products and industrial problems? I mean, it's an interesting issue. I think the one thing I was saying was that we welcome participation in industry, that they were welcome. I did feel a couple of years ago, less welcome. And the story was, I just went to ISCA

and kind of instantly was complaining that it felt like the current community, unlike the past, wasn't as appreciative for industry papers where people really built hardware as opposed to, you know, idea papers. And I kind of bemoaned that. And I forgot that Sarita Agbe, who was up table with me, was the SIGAR chair. So she put me in charge of fixing that. So we came up with the industrial track. And so now, and I think we've been doing it for order almost five years now. And I think that's worked well so that there's a different

slot that's available. And so that we set aside some space for people who are really building hardware rather than ideas. So I'm happy with that. And it's important to, speaking from the academic side, to get people to pay attention to your ideas. They have to know about them. So it's important for people to show up from industry at these conferences. And so I think the industry track helps quite a bit. So there's opportunities from the publish and the papers can be, certainly these have been implemented. They're not just ideas. So I think it's a nice complement to what's going on. In terms of trying to get your ideas into influence commercial products, that's always been a challenge in academia. And it's got a lot to do with the

taste of the problems you work on. But I thought one of the benefits for my career at Berkeley was because of our close interaction with Silicon Valley. I could understand what the problems are facing. I didn't particularly, wasn't particularly interested in what they thought the solution was, but I was very interested in what the problems were. So I think what's great about industrial participants at ISCA is it gives people who don't live in Silicon Valley a chance to talk to the people from

industry to understand what the problems they're facing and then maybe what you work on can be a problem that can really make a big difference have have impact and see people use it right that was really helpful perspective and a contrast between how things were in the early days and how things have evolved to the current generation at the same time we have new technology paradigms as Moore's Law is slowing down. We have newer technologies coming on the horizon, including things like quantum computing. We have newer algorithms and applications and problems to solve as well. Taking a very broad-sided view, are there any mindsets that researchers should adopt as we have these new problems coming to the fore? Any mindsets from the past or some foundational attributes of our community

that we should continue to carry forward? For example, you've always advocated a quantitative approach towards studying systems and computer systems in particular. So any foundations that you think we should carry forward, any mindsets that would be helpful to sort of shift? I think the quantitative, you don't wanna only do things that can be measured.

You want to look at qualities or you want to tackle things where you can't yet measure it. But I think, you know, the discipline of measuring, looking at the cost performance eventually is going to be important. From an industry perspective, if they're going to try something new, they want to understand the risks involved and the return on the investment and it's really hard to figure out returns on investments if you can't quantify it and so i don't i don't think that's going away it'd be dangerous though if you only publish research that is quantifiable but if you can't have kind of vision papers, like I think if we do these things, this thing will possibly happen.

You wanna have vision papers to inspire things to do, but at some point, you have to build something. I guess that's the other thing I'd like to mention is I certainly had the feeling at least early in my career because of the Meek-nelly VLSI movement, there was a sense of building chips, of ideas that students can build chips to demonstrate the ideas. I think that's one of the reasons people are excited about hardware, is there's a physical device that gets created. You know, limiting only to simulation, I think, isn't as exciting as if you can build chips.

And increasingly, there's tremendous opportunities to build chips. The OpenEDA tools, there's a bunch of contests that companies are sponsoring. I think even Google is sponsoring to let people fabricate chips and design. So it's even easier today. And then there's, you know, you could start with the open source implementations of RISC-V. If you wanna modify something, you don't have to do everything from scratch.

So I think there's just all kinds of, and then there's FPGAs for emulation and all that stuff. So there's tremendous opportunities today, I would argue it's even easier today to design prototypes of your ideas. And there's a lot to be said for understanding the value of simplicity that's hard to teach if you don't build something if you if you just kind of all you have to do is write the paper and build a simulator

you can do really complicated things because it kind of doesn't have to perfectly work but if you hardware if you have try and start a prototype it kind of works or doesn't right and it And it can be eye-opening to do that. So giving students a chance to build chips is, I think, a really wonderful educational opportunity. And it's part of the excitement of being in this field. And particularly right now, what's really great, it reminds me of my early years, is all of the risk microprocessor work that NEC and I did in the 1980s. We used tools that were developed by academia. All of our chips primarily relied on tools that other academics built. It was popular in computer science circles

for some people to build tools, some of which, amazingly enough, still exist today. But there's this open eda tooling movement that's going on right now so that people all over the world can build chips and they don't have to they don't have to be friends with one of the ecad companies so that's that that's a big benefit and then there's open source hardware you know the risk 5 movement has helped start but there's a lot of things so there's a lot of opportunities now that you could argue makes it easier to do research that's easier to demonstrate to industry that has big impact.

Now we've talked a little bit about how the technical landscape has evolved and like as Souvenet was just talking about and you were addressing, you know, how the mentality of trying to do things quantitatively and trying to actually build systems and build real chips, that evolution throughout the last 50 years. Now, what about the community itself? So, we talked a little bit about that in the beginning where it was very small and now it's much larger.

Tell us a little bit about your perspective on how the community of SIGARC and ISCA has changed over the last 50 years. Well, it's really important to have the opportunity to see people face to face, just to make friends and reconnect. All of us are worried about global climate change and airplane travel is one of the worst things. But somehow, how can we strike the balance of people still getting a chance to meet together and have the spark of ideas of mind you know people talking and discussing that's really an exciting the hallway conversations as well as the talks so can we figure out a way to do that while not you know burning down the planet so I think it's been great the virtual access to talks so that more more people

can participate more people can participate, more students can participate. That's a plus, but can we figure out a way to keep having events where people go to and have this interaction in this modern world will be an important challenge going forward. Right, the next ISCA we're going to is FCRC, the Federated Research Computer Conference.

Back when I was, I mean, this was something we invented, right? There didn't used to be a, you know, there used to be long ago, early in my career, there were these national conferences that people went to, but those kind of died off. And people from all fields would go to. So this was inspired for the Federated Computing Research Conference that was started by the Computing Research Associations, trying to recreate that community building and bring inspired for the research conference that was started by the research associations trying to recreate that community building together and now we're doing it every four years. We could do that something like that on our own in the architecture related things. We could have on the non FCRC years, we could have a cluster of conferences at the same location.

And it's because it's really fantastic when you have that, you can say, wow, there's something else going on in this other conference at this time slot. I'd love to go or there's people I'd love to talk to. I think that would take some combination. I also see the benefits of, you know, reducing some of the paper production. If we had common deadlines across a set of conferences, you know, the people who are going to write a lot of papers, you know, that's going to be harder if you have to do it to the same deadline. And I don't think that would be too bad if we had not quite as many papers being written and we could have more community

opportunities to get together. It's not an easy problem to solve, but it would be a great problem to solve if we could do that, co-locate many architecture conferences, have a common deadline, and have plenty of opportunity to interact. That sounds interesting because right now there's actually a lot of discussion about how to deal with the volume of papers and the PC meetings. As you said in the beginning, we have 20 we have 20 person PC or we had 20 person PCs in the early days. And now it's like, I don't even know what the PC size was for the last

ISCA. I mean, it was close to a hundred and it was like spread over a couple of days and that's not including the external review committee. So, yeah, it seems like there's a lot of push pull about how to solve this problem i feel like i haven't heard the notion of this kind of mini federation architecture conferences with a common deadline which is which sounds really interesting actually because then you know potentially students don't have like i remember when i was a student it was just like okay if you make this deadline or you miss this deadline and then there's another one a couple months later there's another one a couple months later and just pretty much like you have deadline deadline deadline you're just trying for

the next one but yeah i should give you the historical perspective here is there was kind of one conference the architecture conference and then as it got bigger there was an argument was we needed to have uh opportunities for more people to write papers, which only one conference would limit. And so that was the original argument. And that led to AskLoss and these other conferences. We did not predict that what would happen now at some universities, graduate students were expected to write paper. Oh, there's another deadline. Well, we have to write a paper for that conference too. So every time we added a conference, it added another deadline that people had to write papers to. So it didn't end up accommodating, didn't so much accommodate more people as it created more opportunities for the same people

to publish. And that was not, I don't know if we'd go back in time and not have those conferences, but it didn't solve the problem. It made it worse. I guess probably more people are participating, but we're certainly being flooded by papers. And then, as it's well known, just because a paper is rejected by one conference, it's zero cost to submit that revised paper to the next conference deadline. So the conference model is not working well, and we need to find some better way to handle this. Having a common deadline does reduce the temptation of writing in papers per year about the same idea.

And I gotta say one of the things I thought, cause I thought it was visual. I looked, I tried to figure out what's the most cited papers in the architecture conference. And what you want to have, you'd like to be on that list of having a highly cited paper that lots of people are referencing, right? So that's kind of the goal. It's not to have the longest resume, it's to have the most high impact papers. And I think taking more time to take on a more sizable paper is certainly the career advice I would give swinging for the fences here.

And there's a negative side effect of doing the least publishable unit kind of approach, where you write many, many papers. It puts a huge burden on the community. So we have to figure out some mechanism to be able to address this issue. And maybe a common deadline would help with two, kill two birds with one stone.

We get more people participating at the same time, more interactions with people and less opportunity to publish as frequently as possible. Right. I think that's certainly advice that's well taken. Taking the time to write substantive and well-thought-out papers improves the quality, improves the reading experience for people who are attending the conference, reading the papers as well. So on that note, do you have any favorite paper or favorite keynote or talk or just a favorite memory from attending a school these years? When the architecture conference was in Philadelphia, Eckert and Malky attended.

You got to meet Eckert and Malky. And that was pretty amazing. And in these early days, and Maurice Wilkes, right? The person who probably built the first general purpose store programing computer and the second Turing Award winner would come to these conferences. And, you know, he lived almost to a hundred, but he was sharp all the way along.

And it was just really an amazing opportunity to actually talk to the people who were there at the very beginning. That's certainly some of my favorite memories. And then, yeah, just the companionship. And I enjoyed the sporting stuff that was done, right? At least for ASPLOS, we played soccer for a while. And I think there's hockey games

for people who know how to play hockey. So I think those kind of social activities stand out over the years and I'm hoping they continue. Yeah, definitely. Our community is certainly known to be one of the more fun and social communities for sure compared to some of the other SIGs. I remember actually one of the more recent ISKAs that I went to, you know, there's always people, I think, well, I can't remember where it was, but there's always people who are

like willing to go and take younger students out to go have a good time and I think at this one there was a it was I think it's Joshua San Miguel was taking a bunch of people karaoke and afterwards I remember hearing some grad students like I love computer architecture the reason was because they went karaoke but like they really enjoyed the companionship and the fun that we had at this conference and so you know I have great memories of of karaoke at the Madison one from, I don't know how many years back now, 10? I hope it's not that long, but I think it is.

So maybe we just close with hearing from you on what you're most excited about with respect to the community and ISCA going forward. As you look ahead, what do you think is going to happen with our field in the next 50 years, let's say? Well, I try not to predict 50 years in this fast-moving field. But, you know, John and I, when we did our Turing lecture at ISCA, we called it a new golden age. And I think we said that almost five years ago. And it's proved to be true.

This is a golden age. With the slowing of Moore's Law and the end of Dernard scaling and these tremendous opportunities to have huge impact on these increasingly important fields, we're seeing this explosion of cornucopia and architectures out there. Already, the machine learning stuff has led to lots of exciting papers. And it looks like there's a danger of artificial genital intelligence being right around the corner. The only way that's happening is with innovation in computer architecture. We cannot just buy x86 servers and make the advances in artificial intelligence that we need. It's taking, it's putting tremendous pressure. I mean, this is exactly what you want in computer architecture.

You want a community that is desperate for faster computers and then they want desperate for faster computers without and Moore's law is failing us. So what does that leave us? That leaves architects that come through with really great ideas. And so, you know, maybe our ML colleagues are pushing the state of the art, but right behind that is the hardware that's empowering them to do that.

And it's at the highest priority in these companies to figure out how to invent new hardware, new architectures that can run machine learning. And it's probably not only going to be machine learning. And then all these opportunities that are in front of you the open eda tools the open architectures the open source software cloud computing it you know it it's you're only limited by your imagination and willingness to do hard work there's not a lot of other obstacles there and uh you know there's been times in computer architecture where it hasn't been so exciting. There was times like the only thing that mattered was the x86 instruction set.

How do you make it run a little faster? That was the only practical thing to work on. We're in the opposite era now. There's many instruction sets. There's many opportunities to impact. It's a particularly great time in computer architecture. This next decade should be full of high-impact projects and opportunities. So it's certainly, and I'm not stopping, by the way.

I'm going to keep participating in this because it's a very intellectually stimulating and high-impact time. So it's a very great time to be a computer architect. That's great. Yeah. And on that really upbeat note and message to our community, let me thank you once again for taking time to be on this podcast. It's been an absolute delight talking to you and hope to see you at ISCA. I'm going to be there. All right, listeners, that was a great opening conversation with Dave Patterson. And before we get into our next guest, I want to say a little something about pronunciation. I've noticed that for myself, I always say SIGARC because in my mind it's architecture and I just can't seem to say it any other way. But there are others, Dave and Souvenay included, that say SIGARCH, presumably because that's how it's spelled.

At the same time, when I was at Microsoft, very high-level people who had been around for years, some would say Azure, and some would say Azure. So you could argue that there are potentially different ways to say both of these words. But with our next guest, we discovered straight from the source that we have all been saying his name wrong the entire time. We all say Joopie, but apparently the proper Americanization of his name is Jopie. And so breaking news folks, you heard it here first. It's Norm Jopie. And so with that, I'd like to introduce our next guest, the venerable Norm Jopie, a VP and engineering fellow at Google, where he is the chief architect for Google's ML accelerators, known as Tensor Processing Units, or TPUs.

He's known for his innovations in computer memory systems, including stream prefetch buffers and victim caching, as well as the development of the Cacti tool for modeling memory timing, area, and power. He has been the lead architect for several microprocessors and high-performance computing systems. He is a fellow of the ACM and the IEEE, a recipient of the Eckert-Mockley Award, and is a member of the National Academy of Engineering. Norm also served as SIGARC chair in the early 2000s and received the SIGARC Distinguished Service Award in 2013. We are so glad to have you here, Norm. Thank you so much for being with us, and thank you for correcting us on the pronunciation of your name. Thank you. I'm happy to be here.

So winding the clocks back to when you were the CIGAR chair, what is the community like? What is the exciting developments of the day? And what were people mostly working on or talking about back then? Yeah, there's been various phases on the technical side, but the key thing is the community was much smaller I remember my first business meeting at the first is good in Israel and we all fit in this maybe 15 person conference room and we have people standing for the business meeting but it was priests and people sitting on the floor as well. But it was a much smaller group than it is today. So what are some of the key technical themes of the day? Of course, today, the tapering of Moore's law, we have a variety of exciting ideas. But back then, what is the exciting technical themes of the day? And what is the overall environment like?

I think it's always been a fairly friendly and collegial group. And the exciting technical topics were instruction level parallelism that was, you know, in the late, like, 89 through early 90s. There was the Yale paper, instruction parallelism is greater than two and stuff. So there were debates like that going on. But one of the things that differentiates it from today is that there were a few key ideas, and a lot of them were microarchitectural-based there'd be like several sessions on them and debates, technical debates going on.

But nowadays I see a lot more breadth and diversity in the technical topics than back then, which is exciting. There's so many different things going on. Do you think that at that time, you know, in the early 2000s when you were chair, you could have foreseen the growth of the field as we have today? And do you feel like at a certain point, you know, at least like in the business world, the mantra seems to be, you know, grow or die, right? And so our field has definitely grown quite a bit. Is there a point where you feel like we should stop growing or have we grown too big or should we keep going? You know, what are your thoughts there? I think it's good. I think the research is high quality by and large and interesting new areas. It's having a bigger impact all around. so when i first started as a vice chair and chair

i think we had like four conferences so there was and they weren't all a cigars one so there was isca micro asplos and then hpCA starting in the 1990s. And so that was a fairly regular schedule where you could, if you submitted a paper and it didn't get in, you could revise it and resubmit it to the next thing. But now, well, by the time I was, the end of my term, there was a conference every month. So, and now we seem to have more.

So I think that's good. So Alan J. Smith, who goes way back in terms of his SIGAR participation, said that the number of conferences should be papers written per year, per graduate student divided by two. So I think what we've seen is just that playing out as the field has grown. Yeah, so during your time as the SIGGARD chair and vice chair, what were some of the exciting technical developments and or even community developments at the time? In conjunction with more people joining the community, we developed a set of awards.

There was a test of time award, the service award, which after his passing was named after Alan Berenbaum. Alan was the SIGARCH chair when I was vice chair and he was just a wonderful mentor. And so I think it's truly appropriate for the service award to be named after him. Some of the other exciting things were, we've just lived through a pandemic, but there was also a pandemic in the early aughts. There was the SARS pandemic, and that closed down conferences.

So for example, at that time, supercomputing didn't have its own SIG HPC. It was part of SIGARCH. And it was kind of the elephant in the room compared to all the smaller academic conferences. And we always relied on them to make money for SIGARCH, and we'd use that money, thankfully, to do lots of services and subsidize the membership cost and other things. And when they had a big loss, that really threw us for a loop. And so that was some of the excitement on the budget side.

No, I mean, certainly sounds challenging. And I mean, getting and shepherding the community through those times takes a lot of learnings, wisdom and vision also to see how do we make the right set of trade-offs while staying true to the values of the community itself. Yeah. Yeah and since then we've decoupled from super computing haven't we? And so was this sort of the first seed to do that? Well it was more driven from the super computing side. They said you know if we're the elephant in the, we should have our own room kind of thing. So we had not really, we had let them do whatever they wanted, basically. So it's not like we were making demands on them, but as part of the sponsorship arrangement, we did get a percentage of the

conference profits or loss. So they wanted to use their profits more specifically for the supercomputing community as opposed to the overall architecture community. We did, you know, things like sponsor student travel to all conferences based on money from supercomputing. We kind of took a socialist or whatever shared approach and they wanted to direct that more to their own community. So there was a negotiation process that did split over several years, so it wouldn't be so big of an impact for us. And we could ramp it down gradually over a couple of years.

But it was an interesting negotiation. It went on for almost a year. Yeah, I remember hearing that that was a fraught and delicate and long drawn out process. Yeah, I remember hearing that that was a fraught and delicate and long drawn out process. Yeah. As you look at, you know, the expanding diversity of topics in terms of themes and technical themes, but also in terms of the people that we're sort of attacking to the field, like any particular things that you think we should sort of maintain as the essence of our community, things that you think we should expand into and change our mindset on in terms of the way we do research,

the kinds of problems that we try to tackle or the way we approach problems, especially given the variety of forces right now from, you know, reducing like increased demand on the application side, diversity of different applications, new technologies, as well as, you know, tapering off of Moore's law and things like that. Yeah, maybe I'll use an analogy from physics. I don't know how familiar people are with physics, but you've heard of the Large Hadron Collider, which, you know, is 20 some miles long or longer. And when they write papers, they have 200 authors. And it's big science. I think in our field, big science needs to be done in industry

because, you know, you don't have 10,000 GPUs at your disposal as a grad student. But the interesting thing about physics that I find is that there's all these smaller experiments that are still going on, like bounding the mass of the neutrino with just one professor and some grad students. And so I think there's a lot of interesting research, both on the big science,

as well as things that can be done at a smaller scale. Yeah, that's interesting. I think we had some chat a little bit with Dave too, about industry and academic sort of interaction and participation. I feel like I haven't heard that notion before of like leaving the big science to industry. Although for sure, we're at a stage now in the field where there's certainly kind of resources

with respect to data and computational resources that are really only available to massive companies. But maybe you can talk a little bit more about what you mean here by the big science versus the little science, the kind of things that should be left to industry and that left to academia so I thought I heard your example about the physics stuff but maybe a computer architecture example would be instructive sure a big science is like what you mentioned for industry in computer systems.

It's hard to do research in large-scale systems without a large-scale system. There's a lot of effects like tail latency of networking, OS interactions, access to huge data sets, those kinds of things. You really can't do them that well in academia I don't think anymore which is kind of sad but the good news is there's a lot of these other smaller things and I reviews some number of papers each year less

than I used to but some of them are in areas, you know, like a microarchitecture idea, for example, that, you know, someone had an area that had been being worked in for 10 years. I tend to get those kind of papers since I've been around longer and not so fluent in the new areas. And you read it and you say, why didn't anyone think of this before? This is such an obviously good idea.

So I think that's the example of the smaller science, for example. Right. I think there's certainly a lot of value in picking fairly well-defined problems and analyzing them very carefully and rigorously and coming up with simple but obvious ideas, right? Things that just followed from the experimental setup and so on. And yeah, that's certainly a style of research that even I'm a fan of if you can define the problem fairly well. Just expanding on the theme very broadly, like

there's both big science and the little science, but still very impactful science. So are there any grand challenges that you see on both of these ends of the spectrum that you would like to see the community tackle? Things that have been sort of at the back of your mind? Yeah, this is more of a systems kind of research area. There was that paper. There's a lot of room at the top, which is something that I've thought about writing a paper about for a long time. I never got around to it. So congrats to them for writing it up.

And I agree, there's a lot of room at the top, which means room for software optimizations. And I think as we get these tough problems, it's gonna be more and more important for co-design and people to have an interdisciplinary background, which I know it's hard for a graduate student because that tends to make it a longer time to graduate if you have to get up to speed in multiple fields. But in the real world, having experience in multiple fields is

way, way more impactful and important than just being an expert in one field. I think that's a very good piece of advice in the current times, the ability to go up and down the stack, peer into the adjacent layers. For example, if you're an architect, understanding either the compiler side of things or the operating system, or maybe the algorithms for new applications, or on the other side of the stack, maybe peering down into these circuits, devices, new technologies that are coming up and sort of crossing that boundary and having a reasonable mental model allow you to sort of broach more interesting problems and maybe come up with more interesting

co-design solutions as well. Right. I think the largest problems going forward are the end of Denard scaling. And also Dave probably talked about this because he's very excited about it. And I think it's very important. It's the embedded carbon in our

infrastructure. So given the climate crisis, I think we have to work on things that provide the most value with the least harm to the planet. So Norm, you know, during your time in the community, do you have like a favorite memory or a favorite keynote or paper from ISCA? It's hard to pick a single favorite. I'd say in general, my favorite part of the conference over the years has been the Eckhart Mockley Award lecture. lecture because the people who get that award tend to be have a lot more experience than the majority of the audience

and so they had decades of wisdom that they communicated in the lecture and I learned a lot from that when I was younger and I still learn from the lectures. One of the most recent ones, for example, was Louise Barroso's. It was just

a stunning amount of good advice throughout the whole lecture. So that's the part that I like the best. The second best thing i like is the cigar business meetings when i first heard about a business meeting i said like this has got to be really boring why would i go to that why don't i go like to the beach or something and that was in hawaii but Hawaii. But it's really fun to see the different people interact and they're passionate about moving the community forward. They may have different ideas about what's best for the community and so there's lively discussions and it's always kind of fun to see the people who are highly respected and famous disagreeing with each other in public but then coming to a conclusion right so that's how we

got the industrial track you know like dave stood up at a business meeting so yeah i think healthy debate is essential for like meaningful forward progress in any community. I think it's most welcome. Yeah. Agreed. So as we get towards the end of our conversation, what do you think now, as you're presumably in the apex of your career, the middle of your career, you have a long bit behind you and a long bit in front of you, right, Norm?

Don't tell my wife that. She wants to go on vacation. So as you look ahead, like what makes you most excited about the SIGAR community going forward? And, you know, we just had our 50th ISCA, you know, the next 10, 20, 30 ISCAs. What do you hope to see? I think one of the most important things is culture. And SIGARCH, I think, always had a pretty good culture, but there were some issues, you know, addressed in the community recently. And I think it's good that those were addressed. Just going back to the historical things, SIGARCH was the first person, first SIG to have care support. Either, for example, if there was a nursing mother or someone with a physical handicap

or something like that. And so I think it's really important to support all members of the community and make sure we have a culture kind of like the kindergarten thing, you know, of respect and helpfulness and all those things like that you're supposed to learn in kindergarten. So I think that's what I'd hope to see going forward. And I know there'll be a lot of really creative people. And it's hard for me to imagine all the things that will come from it. But I think it's very important for society. We've seen the impact just continue to increase.

Computers are... we've seen the impact just continue to increase computers are you know there was this notion that you could buy computers to do the jobs and then as computers got cheaper then the market would go away or something like that but in reality it's exploding even now with chat GPT and things like that. So it's a very exciting time. It's a lot harder to design systems, but it's more challenging for engineers. So I think it's a great time to be an engineer in the computer architecture community. I think that's a great note to sort of end our conversation on two

themes, right? Like one on the technical side, there are a lot of exciting challenges, a lot of new ideas. And on the community side, improving our inclusivity, making sure everyone is supported, and the right environment that you said culturally fosters innovation and better ideas, and more people to work on interesting problems and come up with interesting solutions. I think that's a timeless message. Thank you so much, Norm, for talking to us. It's been a delight hearing your perspectives. My pleasure. Thank you.

Yes, thank you, Norm. We're so glad to have you here today. Thanks. Well, listeners, that was another pretty interesting conversation with Norm. And in listening to these tracks to put this episode together, it's pretty notable that Dave and Norm have distinct opinions on how to deal with conferences and the volume of papers in our community. With Norm commenting that by the end of his tenure as chair, there was a conference every month and it being a good thing. But with Dave positing that co-locating multiple conferences and having a shared deadline would be good to pull in the tail of prolific write-a-paper-every-deadliners, as well as to increase the likelihood of everyone in the

community attending these fewer shared events. Our next guest is also an innovator with respect to these issues facing our community, having just co-chaired the AskPlus PC with a novel reviewing system. And so I hope that as folks head to ISCA, there are hearty and fruitful discussions on these topics, particularly at the business meeting, since hearty discussions and business meetings have come up as a source of vibrancy in our community. With that, I'd like to introduce our next guest, Natalie Enright-Jerger. She's a professor in the Department of Electrical and Computer Engineering at the University of Toronto, where she is the Canada Research Chair in Computer Architecture. She's

made wide-ranging contributions to multi-core architectures, interconnection networks, cache coherence protocols, and approximate computing. She's a recipient of the Alfred P. Sloan Research Fellowship, the Borg Early Career Award, and she's a distinguished member of the ACM as well as an IEEE fellow. She's been an active leader in the SIGARC community, advancing outreach activities, and has served as the chair of Women in Computer Architecture, and she's the incoming chair of SIGARC. Welcome, Natalie. We're so glad to have you here. Thanks for joining us. Hi, it's great to be here. So you are the incoming chair of SIGARC, so you are going to be the future of the community. So some of the questions that are kind of

geared towards the past that we gave to Dave and Norm, we want to skew a little bit more towards the future for you. So maybe we'd say, would you describe the Seagate community today and what do you think is the most exciting about the the state of the community right now? Absolutely, so I attended my first ISCA in 2005, so I've been a member of this community for around 20 years and so what I think is really exciting today is the the growth and the diversity in the community. I think we've seen tremendous growth in terms of the number of people engaged in ISCA and in SIGARCH and I think it's really exciting to see how that's that's changed in the the 20 years I've been involved and certainly probably even more so over the 50 years

of ISCA. Yeah I will reiterate that we're so I've been in the community over the 50 years of ISCA. Yeah, I will reiterate that. So I've been in the community around the same amount of time as you. I think my first ISCA was approximately in the early aughts, whenever I don't really know which one it was. But I saw a Facebook history post of mine that said, like, I'm really excited. There are 30 women at ISCA and there were like 500 people there. And I remember going to you know the the

dinner and Sarita was there and I felt so in awe of Sarita and now you know 30 would be paltry like really really paltry and so it's kind of amazing that that has changed in just our lifetime right and I bet you know probably for you Souvene like the days where there were five women at ISCA, you've probably never seen that, right? Most certainly not, yeah. Yeah, but that was a reality for when we were young. We're still young, right? We are. So what do you think about how that came about?

Was that like kind of in your mind like a natural evolution because that kind of went along with the state of computing in general? Or was there a push from the inside or push from the outside? You know, talk a little bit about how you saw that happening. Yeah. So I remember in in the first disc I attended 2005, Anne Bracey, who was then a graduate student, had this idea to get the women together and we all went out for drinks, you know, one of the evenings of ISCA and it was a small number of women that she got together and so that was the beginnings of the Women in Computer Architecture group and so I think the idea that we would gather and we would support and network with each other was sort of a new idea. As you mentioned, leaders like Sarita, Margaret, Sandhya, the number of women that you could kind of point

to as leaders, Catherine McKinley, was relatively small, but there were more, you know, female graduate students. Lisa, you and I were grad students around the same time. And so, you know, I think it sort of grew internally from, you know, kind of a student-led need to see more people who look like us. And, you know, now the Women in Computer Architecture group has over 250 members. And so it really has ballooned from maybe the eight or 10 of us who went out at that first gathering to you know hundreds of members and it's really inspiring to see you know the the young women involved as well as to see the the fall of the careers of women who you know are now you know on the tenure track have recently become tenured and just to see those numbers grow yeah definitely you being one of them miss Miss Chair-elect and fully tenured professor.

It's been really, really exciting to watch your career, Natalie. I remember actually going to a PC meeting once where, you know, how those can go and you were very emphatically talking about a paper. I remember the key phrases, I agree with Natalie. And the thing was like five or six people just said, I agree with Natalie. And I was like, this is amazing. I love this. I just agree with Natalie.

So that was a really good PC meeting. And that was years ago. So it's really been cool to watch your career develop. Yeah, absolutely. I think it's been fascinating to see the community sort of evolve and grow and become more diverse as well. And I think there are ever more initiatives initiated by folks even younger in the pipeline you know you have the computer architecture student association which grew out in the last

few years and they've been doing some fantastic work in this particular space just wanted to pick your brain on that like how do you see the community evolving what can we be doing better what are some additional opportunities that we have moving forward? Absolutely. So you mentioned the Computer Architecture Student Association, and I think that's a fantastic development. I love to see the students advocating for themselves. I love to see them taking on leadership opportunities

to sort of prepare them to be, you know, eventually senior members of the community. I think it's great. I think it's really important that we're sort of centering the student experience in ISCA and SIGARCH more. I think that the students are the ones who are doing the hacking and the writing and all the work behind some of these incredible ideas.

And we have to make sure that the space is inclusive and supportive and helping them to thrive. So I love these initiatives and I think it's really fantastic to see students you know take on these roles. I think things like the meet a senior architect and meet a senior student are also really great initiatives. They weren't things that existed when I was a grad student, but as a senior architect, I love having a chance to sit down and chat with a few grad students. Sometimes ISCA has gotten, I think, so big and sometimes feels so hectic that it can be challenging to meet people or have sort of a more quiet and focused conversation. And so I love the opportunity to sit down with two students and get two students and get to know, you know, them and their research.

And I've kept in touch with some of my mentees over the years. And that's been a lot of fun. So I think, you know, continuing to grow those programs and to, you know, make sure that, you know, I think all students attending ISCA should have a mentor and that we should really be, you know, making sure those programs are sustainable and, you know, well run and that we're really be you know making sure those programs are sustainable and you know well run and that we're able to keep keep the momentum that's been built over the last um several years and i understand there's also you know a long-term mentoring initiative that is smaller in scope right now but i think continuing to support that and making sure that you know

students are getting support and advice not only only from their advisor, but from a variety of perspectives. It's just really, really valuable. I was really fortunate to have a lot of mentors throughout my graduate school and throughout my pre-tenure days. And I think some people maybe need more help identifying mentors or connecting with them. And so I think these formal programs are really valuable. That's a great point. I think mentorship is such a valuable part of the experience of both students and as you move in your career as well. And as you said, like having different perspectives outside of your advisor's own can lend a really more wholesome experience for students' development. Swipping gears a little bit onto the technical side, like how do you think the technical landscape of the community sort of evolved over the years? You've also been

you know vice chair, you've started on the PCE and so on and so forth. So yeah, I think that when I you know first joined, started attending ISCA, you know we were sort of at the dawn of the multi-core era and so we saw this you know maybe pretty big switch from focusing on single threaded performance to multi-core we also started caring a lot more about power and energy efficiency and so i think some of the things that we sort of just take for granted now i was really it was an exciting time to come into the field because those were all new and exciting you know the area that i did my own PhD in, in networks on chip, was a very new idea, started around 2002. So the field is really,

it's been an exciting time to see the shifting landscape. Also to see ISCA as sort of a big tent community that we're taking in a variety of papers on different topics. We've seen the rise of accelerators, we've seen machine learning become really popular within the architecture community. We've seen maybe more niche topics like quantum computing and DNA based or molecular computing. And I think it's just really exciting that ISCA

is willing to take these ideas and highlight them. And, you know, it's still some of these things, you know, some of the things ISCA takes are not proven out technologies. But I think that's what's exciting. We get to see lots of interesting and diverse ideas and then, you know, the community can decide how to how to run with those different ideas. Yeah, that's really interesting because I do remember when we first started coming to these conferences and it was like shared caches was a thing. And then that's where this rash of replacement policy papers came out. And now, like you said, it's such a big tent where sometimes I feel like I would go and I'm like,

I don't know anything about this topic, like literally at all. I don't know about this brain interface, or I don't know about this particular biological algorithm that we're trying to accelerate. So, you know, do you see any potential worries about becoming too big of a tent where it's like chaotic or like how do you think we kind of protect the not protect that seems too defensive but kind of guard the the thing that makes computer architecture computer architecture while still being an open space for new and innovative ideas yeah I think that's a really good question and I think it's really challenging because I think you know we want to make sure that you know we're maintaining you know a particular standard and that we're you

know being very rigorous in our evaluation of papers and it can be more challenging to find a set of experts when you have a new and emerging topic and it can potentially feel maybe more you know risky to to take some of those papers and you know on one hand i think it's it's okay if we you know get it wrong you know not every paper is going to go on to to win the test of time but i think it is important that we try you know we have to do due diligence and make sure we have experts who can you know evaluate you know the papers and that the papers are still connected to the architecture community that they you know that they have that they have some tie that makes them interesting and exciting to architects.

We don't wanna take, say, pure machine learning papers. They have to be machine learning as applied to systems or systems as applied to machine learning. There has to be some architectural take on it. So the tent, I don't think the tent can become too big and we don't wanna lose the core, some of the core values of ISCA, but I think being able to explore new avenues and be open to new ideas is a really important part of evolving the community. Soni, that's a fascinating take. And, you know, as we think about expanding the tent,

we are increasingly noticing that a lot of interesting problems span multiple boundaries. For example, if you talk about machine learning, we have had to understand the. But this might be further up the stack. For example, the compute required for machine learning is growing manifold and that has impacts on the carbon emissions and so on. So how do you think about crafting collaborations with researchers in adjoining fields as well as up and down the stack? Absolutely. So a few years ago, maybe more than a few years ago,

SIGARCH had a series of visioning workshops. And I think the first visioning workshop may have been in the area of machine learning acceleration. And this was an initiative, I think, first visioning workshop may have been in the area of machine learning acceleration. And this was an initiative I think launched under Sarita's term as chair of SIGARCH. And I think that this was a really exciting opportunity to try and foster these interdisciplinary conversations and to get people in a room, making sure you got people up and down the stack. And so I think there were a series of visioning workshops that

were modeled after the CCC visioning workshops that were really very exciting in terms of these different areas. I think that that might need a bit of a reboot post-COVID, but I think you mentioned things like sustainability, carbon footprint, policy. And I think these are exciting questions that I'm interested in learning more about. So I'm hoping we can get people from multiple communities to want to come to ISCA to participate in some of these bigger picture discussions. I think also thinking this year is going to be an FCRC year, and I think FCRC is an opportunity we co-locate with a number of conferences.

And I think that more broadly, as we think about multidisciplinary and interdisciplinary work, there's an opportunity for potentially other co-locations that we could do between ISCA and other conferences or between ASPLOS and other conferences as a way to just get people in the same physical space to have those conversations

and to learn whether it's about algorithms, policy, you know, what have you, circuits. I think that there are some opportunities there to broaden, you know, who's coming to ISCA, which I think would be really exciting. Well, Natalie, I feel like we have so much more to talk about with you. Maybe we'll have you on as a full guest for one of our normal episodes. But so to sort of close out our little chat, what makes you the most excited about with respect to the SIGARC community going forward? What are you looking forward to for your leadership term? Yeah, so being on the SIGARC board,

I've been on the executive committee now for eight years. I served a term as a director and now as vice chair and it's been a really rewarding service that you know it's a set of really dedicated and hard-working you know volunteers and the initiatives that have been launched and maintained are just really exciting. So I feel like as a community we're in a really great place to continue what we've been working on as well as, you know, look forward to new opportunities and initiatives. I think it would be an exciting time to, you know, think more broadly about, you know, how we can continue to, you know, connect with

computer architecture researchers around, you know, the globe. We saw tremendous international participation in things like ISCA during the pandemic because as much as we maybe didn't like virtual events, they were tremendously accessible and affordable for people to participate. And I think, you know, thinking about ways we can continue to make ISCA accessible to, you know, a very broad set of researchers is a great opportunity. I think there's a lot we can learn from during our experiences throughout the pandemic to make that happen.

I think continuing to support and mentor students is tremendously important and I'm excited about that. And I think there's just a really, there's a really, I think there's a lot of exciting research opportunities, exciting opportunities to network and vision. We lost a lot of networking time over the pandemic. And so, you know, sort of restoring people's ability to network and drive these interdisciplinary collaborations will just be a I think we'll have to have you back for a full episode. But we're really excited for your upcoming term as chair of SIGARCH, and we look forward to seeing what the future brings to our community and to another 50 years.

Thanks for joining us. Thank you so much. This was great. To our listeners, we hope you enjoyed this special episode of the Computer Architecture Podcast commemorating the 50th anniversary of the International Symposium on Computer Architecture or ISCA. We heard from three leaders from our community on how the computer architecture field and community has evolved over the last 50 years, as well as the opportunities and exciting times ahead.

We hope to see some of you at ISCA this June and look forward to an ever-interesting slate of ideas that push the frontiers of our field in the coming years. Thank you for being with us on the Computer Architecture Podcast. Till next time, it's goodbye from us.