SemiWiki.com - Podcast EP359: The Benefits of the SiFive Smart Flow™ Technology with David Kravitz

Episode Date: August 7, 2026

Daniel is joined by David Kravitz, Senior Principal Engineer at SiFive, the pioneer of the RISC- V computing revolution. A veteran silicon architect with decades of experience designing high-performance processors ranging from VAX and Alpha to MIPS and ARM, David led the team behind some of the most critical breakthroughs in… Read More

Hello, my name is Daniel Nenny, founder of Semaywiki, the Open Forum for Semiconductor Professionals. Welcome to the Semiconductor Insiders podcast series. My guest today is David Kravitz, Senior Principal Engineer at Sci-5, the pioneer of the Risk 5 computing revolution. A veteran silicon architect with decades of experience designing high-performance processors ranging from VACS and Alpha to MIPS and Arm, David led the team behind some of the most critical breakthroughs in modern processor design. He is the co-inventor on several patents powering the SciFive Smartflow technology inside CyFive's performance family of products. As recently featured in their Technology Explaner video series, which is a great series, by the way, his pioneering work on hardware-level

dependency chaining and specialized transfer buffers solves one of the biggest challenges in compute history today, eliminating data traffic jams between traditional processing and heavy lifting AI vector workloads. By rewriting the rules of out-of-order execution, these architectural advancements allow next generation chips to deliver massive server class AI performance in a tiny, ultra-efficient physical footprint. Welcome to the podcast, David. It's an honor to have you. Oh, thank you, Daniel. I'm glad to be here. So first, let me ask, what brought you to SciFive? Tell us a little bit about your journey. Okay, wow. So in college, I read Soul of a New Machine and thought, this is the coolest thing. I really want to do this with my life and was, uh,

able to get a job at digital in Hudson Mass after graduation, where I worked on the Vaxes and alphas and learned so much from the team they had there. You know, many of those folks went on to do other great things. And I spent a little bit of time after digital, doing digital signal processing stuff, which none of those panned out. And then a bunch of people who had been at deck and had been early at Cavium persuaded me to join them. So I did. did that and worked on some MIPS cores and we did our own arm cores at cavium, first in order core and then an out of order core. And I led the neon unit development there. At some point, Marvell bought cavium and turned away from doing in-house cores. And I started looking for a place

where I could still work on a core, which led me to sci-five the chance to work on a fairly new I saw and especially the very new risk five vector I saw at the company that pretty much, you know, the founders created the risk five I saw was was really attractive to me to go in and contribute to something that was basically at the cutting edge of processor design. That's a great story. You know, we're of the same generation. I read Sol of a New Machine in college and I actually went to work for data general based on that. I spent the first four years of my career. in Data General, they had a FAB here in Silicon Valley. And I went from the FAB and then I went to one of the field offices to be an application

engineer and then, you know, working with customers and stuff. And it was the most wonderful experience. And so we're the same generation. So let me ask a little bit more detailed questions. Along with the RVA 23 support, the big news with your most recent performance core announcement is a feature called SciFive Smartflow technology. which is based on your recent patents. If you were explaining Smartflow to a non-Semiconductor professional,

what everyday analogy would you use to describe what it does and what problem it solves? I'll do my best. It might be tough because to understand the motivation behind SmartFlow, you really need to have some grasp on how out-of-order processors work in the first place. So I'll try. So in an in-order processor, you execute the instructions in the order that they were produced by the compiler, and you need just one physical register for any architectural register in the ISA. So if you had an ISA with 32 architectural registers, you really only need 32 physical registers to handle those operations. An out-of-order processor tries to do work as soon as possible, so not in the program order,

and then there's a lot of work at the end to make it look like it happened in program order. But in the middle of all that, the processors usually have a large number of physical registers because they can have multiple architectural registers in flight. You could have X-7, for instance, be in flight in three different places, depending on branches, or what have you. So they're usually a high number of these physical registers. The problem when trying to do this with a vector unit that's attached to a scalar core that's doing the integer and floating point is that we looked at several places where

you could diverge the execution path between the scalar unit and the vector unit. And if you try to do it too early, there's a problem. And if you try to do it too late, there's a problem with you're waiting a long time to figure out where you really should be doing the work. And we chose a path in the middle that made our lives complicated in a way because now we have a scalar out of order unit and a vector out of order unit and they don't really know a whole lot about what the other one is doing. And in order to get the right result in the end, we need some way to have them figure out, well, here's the work that I did that you need later, or I'm doing this work early,

and you can take it whenever you need it. So that's one of these smart flow techniques is these buffers, the transfer buffers that we use to move operands or results between the units at the right time without delaying the calculation too much and without making one side or the other wait or get blocked because data isn't available. or space isn't available for a result. Is this getting to a point that might be clear enough or do I need to try again? Get it.

No, you absolutely covered it. So we can talk a little bit more about it. So David, you co-authored a patent specifically addressing a transfer buffer between scalar and vector pipelines. For software developers used to writing unified code, why does hardware separate scalar and vector task in the first place? Why can't one processing engine just do it all? Right. So in fact, there are processors that do try to do it all. The arm vector architecture utilizes the same architectural registers for floating point and the SVE operations. And they did the same thing for floating point and neon in earlier generations. But that limits your flexibility and it limits the amount of work you can do out of order. Also, there are cases where you want to,

put the complexity or the extra hardware only in the places they need it. So you can have a scalar unit with integer and floating point and those registers in the risk five or 64 bits wide. Whereas the vector unit is the width is flexible. You could do it in 32 bits. The P570 that we're using it or that I led the work on is a 128 bit wide vector registers. And you can go up to really quite large, but it's feasible and people have built machines with a 1K and 2K bit vector registers. Once you get to that size, though,

you don't want to have this out of order issue where you have multiple physical registers for each architectural register. It gets really expensive at that point. So in traditional chip architectures, what actually happens when a scalar pipeline needs to pass a data address or an instructional over to the vector pipeline.

Why is this handoff historically such a slow and painful bottleneck for performance? Right. So this goes back to what I hinted at before, which is these two sides of the machine. If you want them operating at maximum efficiency, you can't couple them too early or too late. If you couple them very early in the vector side of the process, then you've waited a long time to find out where in the scalar unit your data is kind of coming from or going to. And if you try to split it too early, you don't know anything about where they should go

across the boundary. So when we hit on this middle path, we needed to come up with a way to have the two sides sort of handshake on where something should go or how to identify where a particular data transfer belongs. And we did this by using a, what we call a global ID that is unique for the lifetime of an instruction. And the two sides can both refer to this common instruction ID, but they don't need to know about the other register files. They don't need to know that this instruction ID corresponds to scalar register X7 or vector register V22. They can

just use this instruction ID that's available at a good midpoint in the pipelines. So how does your patented transfer buffer solve this? Is it like a valet service or a smart waiting room that keeps the scalar side from freezing up while the heavy lifting vector side gets ready? This is more like a smart waiting room where each piece of data that needs to cross between the scalar and vector unit is given a tag based on a unique instruction ID that's unique for the lifetime of that instruction. And when data is available to be transferred, it's put into a buffer and tagged with this instruction ID. And when the other side is ready to make use of it, the other side looks it up by using the instruction ID and consumes the data at the time that the consumer is ready to consume it. You know, it sounds like keeping those pieces separate, but forcing them to share the same data highway would normally cause a mass. traffic jam, how do your specific patents, you know, the transfer buffer we just talked about and your dependency tracking, act as kind of a magic trick that makes this unique, you know,

splitting split units set up actually work with, without choking on data? I mean, that's a big problem. Right. Yeah. If we didn't have this, then we'd have a lot more pipeline stalls where data has been produced in one pipeline, maybe, and the other pipeline isn't ready to consume it, or vice versa, you know, one pipeline is sitting there waiting for its data to be consumed, and that hasn't been produced by the other pipeline yet. When we use these buffers, that lets the main pipelines do what they're designed to do with any instruction that it could be executed, and the instructions that need to wait for the data can do that.

Okay. So, you know, just a follow-up question. What application areas is this most suited for? I mean, to me, data center, data center keeps coming up in my mind. But, you know, what specific applications are you seeing? Right. So the P570 here and the P870D, which has a similar microarchitecture of the vector unit, are designed as application processors that are going to be in a Linux environment.

So there might be a wide range of applications. These are not really aimed at the highest arithmetic throughput that, might be desired in some applications. We have other processors in our intelligence line that do things like cell phone camera image processing because there you have much wider data, a lot of data, and the application is really well understood, and you can handcraft the code if you want to.

In our case, we need something that will work flexibly with compiled applications that could be anything. When we were developing this, we spent time looking at, of course, the various spec benchmarks, things like Geekbench, but we also saw that libraries like the GlibC string library want to use vectors, and that's going to be used all over the place. And so we made sure that our design would be reasonably responsive, reasonably performant for a wide range of uses. Yeah, yeah, I agree with that. You know, the final question, I ask is, you know, how do customers normally engage with SciFive? But I tell you, I spent a lot of time on your website. It is a really, really good website. There's huge amounts of data on there, a lot of excellent content. So I'm guessing sci5.com is the first place that customers would go. Yes, that would be my first stop. If you, you know, if you're looking to do business with SciFive, if you are interested in, in more some technical information about Risk Five in general, one of our engineers, Nicola

has a series of blogs where he writes about the Risk VISA and its benefits. And there's a lot of Risk V interest in general. But as the, I think the leaders in the Risk V space, the SciFive.com website is going to be where you should go to begin getting more information. I think you're one of the foundation companies of the Risk Five movement, right? Yes, absolutely. I mean, the creators of the ISA, Kirsta Arsenevich and Andrew Waterman, some of the founders of the risk five are founders of the company. And we do participate very actively in many efforts around the ecosystem, clearly in the Risk Five International Association,

the RVIA, but also efforts like Rise, which is about common software and applications, which is part of the RVA 23 platform to make it easy for customers to adopt Risk Five. And not be tied down to any one vendor, though we hope that they like SciFive. Yeah, well we've been working with SciFive for about 10 years, maybe over 10 years, and the transformation of your company is just amazing. So thank you very much for your time, and I hope to have you back. There's so much more to talk about. Thank you, Daniel. It's been a pleasure, and I'd be happy to come back. That concludes our podcast. Thank you all for listening, and have a great day.