CppCast - Realtime Sanitizer and the Performance Constraints attributes

Episode Date: October 29, 2024

Christoper Apple joins Timur and Phil. Chris talks to us about his work on the new Realtime Sanitizer in the Clang20 release, as well as the associated Performance Constraints attributes, how they differ, and how they work together. News The C++23 Standard has finally been released by ISO "Why Safety Profiles Failed" - draft of new paper from Sean Baxter "if constexpr requires requires { requires }" - Jonathan Müller Links RealtimeSanitizer docs Performance Constraints docs RealtimeSanitizer helper repo (including how to run it "standalone" and find the team on discord) slides from Chris and David Trevelyan's CppCon 2024 talk

Episode 393 of CppCast with guest Christopher Appel, recorded 28th of October 2024. In this episode, we talk about safety profiles, and about safety profiles. And about requires, requires. Then we are joined by Christopher Apple. Chris talks to us about Klang's new real-time sanitizer and performance constraints attributes. Welcome to episode 393 of CppCast, the first podcast for C++ developers by C++ developers. I'm your host, Timo Dummler, joined by my co-host, Phil Nash. Phil, how are you doing today? I'm all right, Timo. How are you doing?

Yeah, I'm good yeah i'm good i'm good um i don't think i really have anything particularly new to say this time it's like yeah things are going along work is chugging along uh you know things are good how are you phil that's good sometimes just not too much going on and i'm actually at home at the moment and will be for a few weeks so just for for a change, not too much traveling. However, just getting used to that. Actually, on that topic, hopefully my audio is a bit better this time. When I was editing last time, I noticed my audio was pretty bad when I was recording in that hotel room where we didn't have a decent mic as well.

So apologies for that. Hope you're able to keep up. But we are back on form today. We are indeed. So at the top of every episode, we'd like to read a piece of feedback. And this week we got an email from Gregory who says, thanks for the show. Just noticed the Mastodon poster isn't showing episodes since May.

Phil, do you want to talk about that briefly? Yeah, I suppose I should confess that I've been a bit remiss in sort of updating the social media feeds when releasing episodes because I have to push out on multiple channels and it means logging into, I've got about, I don't know, four or five different accounts on every social media platform at the moment. And I've just been a bit burnt out on it. So I've sort of not really been keeping up to date on that for a while. I've been looking into ways to automate that a bit better. So hopefully we'll get that back up and running soon.

So apologies for that if you are relying on Mastodon or some of the other feeds to keep up to date. So I actually have, I think it's a good moment for me to make an even bigger confession. I actually haven't really used social media in quite a while. So I haven't had a Facebook account or an Instagram account or a LinkedIn account, which people apparently find shocking that I'm not on LinkedIn since years. And the last two that I kind of had active were Mastodon and Twitter. And Mastodon, I never kind of got really into it very much. Twitter, I was quite active for a while, but I kind of just noticed that it's just not very good for my mental health. Like basically how I felt like in terms of just anxiety levels

and just overall mental health or how much time just disappeared in a black hole was somehow directly proportional to like how much time I spent on Twitter. So I kind of gradually faded out my usage of social media altogether. And I just checked, I just logged in again into Twitter for the first time in months. And I said that I saw that my last like meaningful tweet, which wasn't just a retweet of something was like back in February. And yeah, my last login was probably two months ago. And I have to say, like, I feel a lot better since I'm just not using any of those things.

And I know that they work great for other people, but I'm kind of convinced at this point that I don't really want to go back there. I think my life quality has improved so much since I've stayed away from all forms of social media. So I know it's weird for somebody to say this, for somebody who's hosting a podcast and giving public talks and things like that but i think i'm going to just entirely quit social media at this point so just uh giving you a heads up that this is this is happening i don't expect me to like respond

really if if if you tag me on twitter or anything like this i will probably push out some kind of official thing that i'm actually going to close that account um haven't yet figured out how to exactly do that but just as a heads up that that's probably going to happen quite soon yeah and i'm not far behind you to be honest i've had a very similar experience and i know a lot of people have so sure there's people listening to this saying yeah i can relate to that as well i think the uh the peak of social media when everybody was was all into it is is past now so a lot of us on the other side so um yeah most my account is mostly just been reposting my sort of more official accounts for cpp cast and conferences and things so yeah yeah

so the official accounts we obviously keep open uh so i was just talking about my personal account yeah just just to make sure that people don't misunderstand like yeah we will try and monitor the the official accounts don't worry about that okay so we'd like to hear your thoughts about the show and you can always reach out to us on x master on linkedin or email us at feedback at cppcast.com and we will listen and we will listen uh and i am uh yeah i'm not going to be on any of those except the email personally, but we will still be reachable through all of those as CPP cast going forward. So that's not going anywhere.

All right. So having said that, joining us today is Christopher Apple. Chris Apple is a software engineer with nearly a decade of experience in the audio industry. Everything from optimizing real-time audio playback engines through 3am speaker installation in Tokyo nightclubs. He has worked at companies like Dolby, Roblox and Spatial Inc. and is currently on a sabbatical as his wife finishes her degree. More recently, Chris has focused on real-time safety

and rendering performance in audio playback systems written in C++. This focus has led to him co-authoring Realtime Sanitizer, a new real-time safety checker coming to LLVM 20. Chris, welcome to the show. Hi, everybody. Good to be here. Big fan of CppCast, and it's nice to be talking with you today. Great to hear it. There's a few things I could pick on in your bio there, but the one bit that stands out for me is this 3am speaker installation in installation in tokyo nightclubs so does that mean you're actually installing speakers at 3 a.m or yeah so uh maybe the most interesting fun job i've ever had i worked on the dolby dj app

which was a dj app uh at dolby as you could probably guess, that is instead of stereo files being played in stereo in and out, it was multi-channel using the Dolby Atmos immersive audio system. So we went into a nightclub in Tokyo. We individually addressed each of their speakers and were able to play Dolby DJ out. And, you know, instead of all the sounds coming from all directions, maybe a synthesizer was flying over your head or anything like that. So I was the lucky person to go out there and actually connect all the dots, one of a team of two. So it was quite fun. So that's really cool.

Like I did grow up in Berlin, so I, you know, I had my fair share of going to these kinds of nightclubs, places like Berghain, which also has an amazing sound system. But I don't remember ever having been to a place where proper surround or immersive audio, like spatial audio, was really used by the DJ in a place like this. I don't think that's very common. I've certainly never seen it live. Yeah, it was really something that Dolby was trying to do uniquely. I mean, a lot of clubs are even mono. You'd be surprised. I mean, in such a loud volume in such a small box, it really doesn't make a difference. But Dolby went in and installed speakers above. It was only two or three nightclubs, and this project is long closed, unfortunately, for everybody, but, uh, yeah, it was really nice. And we got, we got good feedback from fans that saying, Oh, wow. I did hear the vocal flying over the crowd. It does make a big

difference. So I'm hoping that a reincarnation of this project exists somewhere in the future. That would be so cool. Just like the clubbing experience. I'm just imagining what that would be like. Like that sounds pretty awesome. Yeah. The DJs loved it. The fans loved it. So I think there's, there's something there for sure but we weren't quite able to crack the nut of how to uh make make it make money you know all right so chris we'll get more into your work in just a few minutes but first we have a couple of news articles to talk about so feel free to comment on any of these okay okay so the first one is that's an interesting one so the c++23 standard has actually now, as of last week, been officially released.

Finally. Finally, by ISO, the International Organization for Standardization. It took them only, what, one and a half years? Yeah, about that. Yes, it's February 23 is when we delivered the committee final version of it. And since then, there's been a lot of wrangling over like editorial issues and formatting issues and like which fonts you're allowed or not allowed to use in that document or whatever and it it took until now to like sort that out and you know

have great respect for the editor editors of the of the actual document for their patients to sort out all of these annoying formal things. And actually, so now the C++ standard is officially available on the ISO website. Its official ISO designation is ISO IEC 14882 colon 2024. Just rolls off the tongue. But everyone will call it C++ 23 anyway, even though it came out in 24, I believe. You can buy your own copy of the new standard on the ISO page as a PDF for the price of 216 Swiss francs.

Yeah, I'm going to call it C++24 from now on, just to control the committee. It's not our fault. It's the people in Geneva. It's not us. We it's you you should like it's it's the guy the people in geneva it's not us we have to live out on time yeah but this this was a particularly long delay this time but it always takes quite a few months i should look back and i think since c++ 14 it's always been around december that iso finally gets to publish so it's about sort of six to nine months that it sort of gestates through the system. So this was a bit longer than that, but potentially this could have happened at any time that it just slips into the next year.

That's happened eventually. Well, now we're in October next year. So, yeah. Well, anyway. So we have now officially a new C++ standard, which is great. Speaking of standardization, there was an interesting new paper draft by Sean Baxter, which I thought was interesting, which kind of dovetails

into his other recent paper about safe C++, about adding a borrow check out of C++, which we mentioned on the show. And so he is writing about the other effort in the space that has been going on

since I believe 2015, spearheaded by Bjarne Stroustrup and other people called Safety Profiles, which is a different approach to get memory safety into C++. Basically, Bjarne is claiming that you can detect many, most, all, I'm not sure,

memory safety issues in C++ with just local static analysis without any extra annotations. And so he has been pushing for that, and this work has been progressing. It's called Safety Profiles. I don't think there has actually been a specification produced in all of this time,

so it's kind of more conceptual work at this point, but the committee has started to look at that material. And what Sean basically says in his paper here is that what the safety profiles a project tries to do is impossible and that if you really want to uh you know get memory safety and see what's else you need to do something very similar to what he's doing with his borrow checker and kind of you can't really get away without making a lifetime information part of the type system or some other form of like explicit part of the type system

or some other form of explicit part of the language because otherwise the compiler just doesn't have the information it needs to deduce whether a pointer points to a valid object or things like that. So I think that was quite an interesting entry into this never-ending debate about what to do about memory safety in C++. But it's quite a well-written, I think, paper, well-written paper.

So I think it's worth a read. You might not agree with everything he says there. In fact, just earlier today, I heard a senior committee member, who was not Bjarne, disagreeing with Sean's analysis in that paper. But, you know, your takeaway from this can be different but um i think the point is that it's a very well written paper and i think it's a very valuable contribution yeah and i particularly like the way that he used the the term carcinization which i think really sums up the the whole feeling of the paper which is a term borrowed from uh

evolutionary biology the the idea that idea that any species eventually evolves into the shape of a crab, which I think is quite appropriate that all attempts to do memory safety in any programming language end up looking like rust. Okay, so can we dig into this for just a moment? Because I think it's a very interesting topic. So what you end up with is essentially

a point on a spectrum and on one end of the spectrum you have functional languages where there's just no point as a reference of any kind and it's all just values yeah and on the other end of the spectrum is rust basically and then you can be somewhere in between like high low is like an in-between solution it has these like mutable value semantics where you have mutation which you don't have in functional languages but you don't really have kind of references that can be that where like multiple places can have mutable references the same thing that's kind of somewhere in between but you inevitably end up somewhere on that spectrum i think i mean i can't claim that comprehensively because i'm not like a language designer that has you know spent years and years of his life in this field.

So there are certainly people who know a lot more about this than I do, but from staring at it, I get the strong impression that this is the case. So I'm, I'm very curious how, like where this all is going to go. Yeah. All right. So last news item for today is actually about today's C++. It's a blog post by Jonathan Müller, who has been working at ThinkCell for a while as a German company,

doing lots of good C++ work. And he has been writing blog posts for them for like, with quite an interesting frequency now. He churns out good blog posts lately. I like that. So the latest one I really liked, it's about requires requires. So he says that, you know,

there's two features in C++. One of them is requires and the other one is requires. And what requires does is it can, you can put it, you know, into a template head, like template type empty requires, and then write a concept like requires

that something is an integral type, right? Requires the integral T or something. And then you can do basically Sphenae on like, you know, whether your template argument satisfies that concept. And if not, then the overload is just going to get thrown out. So that's kind of a better way

to do what we did with enable if before. And then there's another requires, which is you can use it as an expression. You can say requires basically brace, and then you can write arbitrary expressions in there. And then that's going to return true or false, depending on whether that expression is well formed. Right. And so both of those you can use as like Boolean expressions, which I think not everybody knows in any context. Right. And so both of those you can use as like Boolean expressions, which I think not everybody knows in any context, right? You can put it into an if constexpr and you can say if constexpr requires whatever, you can use it as a compile time like Boolean function that returns true or false, basically.

It doesn't have to be when you're like declaring a template. And so you can combine the two requires also. You can, for example, write, you know, template-technum-t and then requires, requires. And the first requires says this is the requirement for this template. And the second requires is the thing where you have like, and now you get like a pair of braces with some expressions in between and you're going to see if they were formed. So that's when you write requires, requires. But the blog post is about his observation that you can also do the requires. But the blog post is about his observation that you can also do the requires requires

slightly differently in other contexts, like if constexpr. So there's a particular problem, apparently, if you write if constexpr. Okay, so if you write template type in p requires, you know, a and b, you can like chain them. You can do and and or, and there's like certain rules how that works.

And if any of those don't compile or are not valid or if the whole thing about it's too false then then then the whole thing is just going to spin out but if you do that in an if const expert and you write requires a and and b i believe if the b is ill-formed then the whole thing is going to be ill-formed yeah but you can get around that by then writing requires brace, requires whatever. And the brace, so it's the other way around, right? You have the brace on the outside, and then you have requires A and B on the inside. And the requires brace around it makes it into one of those contexts where it's fine if it's informed.

And then the requires inside is where you can then chain your concepts and do and and or and stuff like that. It's kind of a neat trick that I haven't seen before, so I thought that was interesting. Yeah, and very musingly written as well, just playing to the ambiguity of which requires you're talking about. And I think in the end, it turns out that if you want if constexpr to work this way, then it requires, requires, requires.

Right. Okay. Yeah, I was going say uh some of the subheadings uh read like something that mad hatter would say in alice in wonderland or something like that and i'm glad the recursion stopped it just requires requires and not requires requires requires but yeah really well written article and uh going over the basics you know i would say i'm a casual template user and i feel like it was a good lineup of just kind of the basics of using requires expressions and then going into a little bit more advanced yeah all right i actually think he has an example where you can write requires

requires requires but it's not really something you would do or is useful but it compiles so you could say it will work he actually says at the end that yeah you can do it but it doesn't actually achieve anything so you i guess you could say that you know there are situations that require requires requires but you don't usually require requires requires requires anyway so it doesn't require require require yeah yeah anyway i don't think this conversation is going anywhere further at this point. So let's transition to our main topic for today, which is very exciting. We have Chris Apple here today. Hello again, Chris.

Hello. And we want to talk to you about the stuff you've been working on, in particular, the real-time sanitizer, which is a new sanitizer which is coming in Clang 20. And I think it's really exciting because it's not every day that something like this happens. We have UBSan, AddressSan, UBSan, ThreatSanitizer. which is coming in Clang 20. And I think it's really exciting because it's not every day that something like this happens. You know, we have UBSan,

AddressSan, UBSan, ThreatSanitizer. And I think people, those are very beloved tools, I think by now, which many people use in their CI. But like to have a new one in there is quite interesting and special. So do you want to talk to us maybe about how you got involved,

how this uh kind of happened and and like what what is the problem that you're trying to solve there yeah i think that's a lot of questions at once but let's dig in and absolutely absolutely so yeah uh i'll get a little bit more into the problem it's trying to solve but i've worked for a long time in the audio industry which is um you is an industry that's worked with real-time constraints. I have to make sure that I'm filling audio buffers in a certain call back or otherwise there's some consequences. But historically, this has been very hard to figure out if your code is real-time safe or not, if you're using mallocs in certain contexts, etc.

So anyways, I've had this problem for a long, long time. And the historical tradition is to just look at code reviews and say, oh, I see a malloc in this context. That's illegal. Obviously, that gets very fraught when you're using third party libraries, when you're using std vector pushback. Sometimes it's safe, sometimes it's not. So anyways, last year, in October timeframe, I left my job at Spatial Inc and I attended Audio DevCon virtually and I saw this incredible talk presenting this prototype version of real-time sanitizer and I instantly knew this is a big deal.

Like this would change so many developers' game plans for detecting these errors. So David Trevelyan, who's my other co-author, and Allie Barker, who worked on the prototype, presented this first version of it.

And instantly after ADC, I'm like, I'm free. I got a lot of time. Help me. I want to get involved in this. So over the last year or so, I've kind of headed up the upstreaming process into LLVM. We decided this was a tool that would benefit a lot of people. So we went through the RFC process, started putting PRs in, started improving it, fixing bugs that were in the prototype and things like that. And yeah, we got all the thumbs up that we need and it should be shipping in

January in LLVM 20 to have real-time sanitizer in there. That is really exciting. I have maybe a little bit of a personal story about this as well so first of all i also worked about a decade in the audio industry and and how to you know not allocate memory on a real-time thread where you can't afford to do those things has been a problem uh this entire time i have seen people doing things like replacing global operator new, which obviously doesn't catch like just naked malloc. I have people seen like doing this like LD preload thing, which I think is a Linux only thing to like just put their own malloc and free like into the runtime that kind of does something else

and warns them if that happens or whatever. But like none of the stuff really works properly right it might catch some of the cases but it's not really a good tool and also it's not like portable or usable on right like basically if you write commercial all your stuff you care about mac and windows you don't care maybe a little quite as much so anyway so i got really excited when i when i first heard about this effort i remember chris you actually pinged me uh what was it a year it was a year ago right yeah you're thinking like hey you could really use like some help here this might be something that's right up your street and

i was like yes normally it would be but like a year ago my son was born so i was like yeah like sorry this is just the worst possible timing i have like zero time for this right now. So that was unfortunate. But like, I could get involved. But I now see that it turned out the attribute, how you name the tool, things like that. So you and I had some good exchanges on Discord, and I think it helped David and I kind of shape the thing that we were going for and also just clarify our ideas about it. So I think even passively, your imprint is on real-time sanitizer, even if it's not directly in code. All right, thank you.

So let's talk a little bit about what the problem here is. So we have any code that's time-sensitive. So we don't want to do things like allocate memory because we don't know how long that takes, right? There's no deterministic limit for how long that takes. So you don't want to do that when you have a deadline. But is that just about allocating memory or are there other things So you don't want to do that when you have a deadline. But is that just about allocating memory or are there other things that you might not want to do or that this

tool might want to catch or will catch? Right. So yeah, the definition that I really love of real-time programming is you have to provide the right answer in the right amount of time. All programming you hopefully want to give the right answer in your calculator app, providing something other than 2 plus 2 equals 4 is incorrect. But in real-time programming, you hopefully want to give the right answer in your calculator app, providing something other than two plus two equals four is incorrect. But in real time programming, you have to do some calculation and do it in the right amount of time. So on a very high level, this is something like self driving cars, if your perception subsystem, if you have your LIDAR coming in, but it's too late saying there's a red light coming up, you might T-bone a car, right? So it's not enough to just say that I detected a red light. I detected a red light within three milliseconds,

which is how long it takes to apply the brakes, for instance. Something like aerospace, when you have a navigation system, you are shooting your rocket into space, you're doing some calculations modeling the world. If you are perpetually late, providing that update to that system, your reality is going to diverge from your projected reality. And I think more importantly, your rocket might converge with the ocean or the ground some sometimes. So that would be bad. And of course, I came into this from the audio side of things. So audio, you have a real time thread, say on your Mac, you're recording some music or something. And every 10 milliseconds or so your operating

system requests a buffer to be filled. So you fill it with a sine wave, you fill it with another sine wave and over and over. If you are too slow, if it takes you 11 milliseconds to fill that buffer, you get a click or a pop and everybody's experienced this. If you're milliseconds to fill that buffer, you get a click or a pop. And everybody's experienced this. If you're listening to Spotify or too many YouTube channels at a time or something like that, it'll get crackly and poppy and things like that. And that's a dropout. So how do you make sure that you hit these deadlines? So these are all the things that can happen if you don't hit your deadlines. Just like you said, Timur, it comes down to non-deterministic time operations. Adding numbers, multiplying numbers, things like that, they boil down to

a succinct list of instructions that your CPU executes. However, when you do something like a malloc, or you take a walk, or you do an IO operation, and you ask your operating system, hey, I'd like to use the read system call. It traps in the operating system. Maybe your operating system is doing something for a little bit and it comes back after a certain amount of time. And you can't account for how long that is. If on 99% of cases, it takes 10 nanoseconds to do a malloc, but one time out of every 10,000 or whatever, it takes one millisecond, you might have this missed deadline and have to suffer the consequences. So typically, these have been really

difficult to detect. You do them through code reviews or something like that, just looking at the code, other tools that, Timur, you mentioned before. But real-time sanitizer comes along and attempts to detect these issues easily at runtime just by flipping a compiler flag. So how does it work? I just compile my code with some flag and magically I get like a runtime error when I hit one of those allocations? Yeah, so there's two requirements. The first one is that any callback constrained by this or subject to these deadline constraints is marked with Clang non-blocking. So this is a vendor specific attribute that you put kind of at the back part of the signature. So Clang non-blocking. No, it's a double square bracket attribute. Double square bracket, Clang colon colon non-blocking okay exactly right and so you attribute your top layer callback uh with that and then you compile and link with the f sanitize equals real time

just like you would do f sanitize equals address or undefined and uh then next time you run you will be running under the real-time sanitizer runtime library. And so what happens if I allocate memory or I accidentally do like a pushback into a vector and I didn't reserve enough memory beforehand and like it results in a cold or malloc and somewhere under the hood in one of those functions marked with one of those attributes? What happens exactly? Yep. So what happens exactly is we detect it in our library and we print a stack and abort the program. So there are ways to configure this. You can, for instance, print the stack and continue. You can suppress errors pointed exactly to, hey, somewhere under the covers, under 20 layers of STL vector code,

this is where you allocate it. And here's in your user code where it actually started. So I wouldn't ship this in production, but I would like run that on my nightly builds or something. Exactly. Nightly builds or QA builds. One of our guiding principles for writing this thing is make it so performant enough to still listen to audio and still kind of run and maybe a qa setting so yeah your nightly builds okay so what is the what is the runtime impact of those like checks

yeah so uh very very little so first off i have to get a kind of a koan, you know, a one hand clapping kind of strangeness out of the way. So well behaved, non blocking contexts have no overhead. Right. So these contexts in which you don't allocate in a real time thread, there's nothing to check. Right. There's nothing to actually detect or anything like that. So it boils down to incrementing a number when you enter this function and decrementing a number afterwards. So if you're well behaved, if you have nothing to hide, you have nothing to fear. Now on your non-attributed functions, there's some checks

put in between you and your mallet call. So we need to make sure that we're initialized. We need to make sure we're not in a real-time context, And then we call the real malloc. So what we've been saying this in a very hand wavy way is there's a few additional predicates that check a few like atomic variables and things like that. So it'll slow down your non-real-time thread by a little bit, nothing too crazy. Address sanitizer says it slows down on the order of 2x. We're saying it barely slows things down. And for most of our users, the real-time thread is really the king, really the thing that they care the most about. So slowing down the non-real-time threads by some incremental few predicates doesn't concern us in this kind of testing situation. so you said it's a new attribute that you add to a

function declaration is that exactly so sorry i the language designer in me is not curious is that part of the type system like no except now or not like can i overload on whether something is non-blocking like how does that fit into the language i don't know about overloading it is part of the type system so one of the interesting kind of like fallouts of this is kind of casting between function pointers to different types and things like that. One of the most functional like high level uses of this is this attribute is inheritable.

So say you have some pure virtual base class that has a process function defined, a method that you can overload all of your child classes that over overload or override that method, uh, actually get that attribute by, by default. So yes, it is part of the type function that work kind of came in with our sister project, the performance constraints attributes, which we'll talk about a little bit later on. So I'm far from an expert in all of the little nuances there but uh yes it is part of the type system for those for those functions

interesting so you mentioned the performance characteristics or performance constraints which i believe is compile time checked and what you've been talking about is runtime checked so are these two completely different systems or is there some relationship between them yeah so this is another cool thing again clang clang and lvm20 a lot of new changes to help you know the real-time programmer uh uh our approach that we took was a runtime approach with a real-time sanitizer but at the same same time, another author proposed this approach called performance constraints attributes. And the idea is, again, you mark your code as Clang non-blocking and or Clang non-allocating is the other attribute that it uses. You compile with the WPerf constraints

warning flag, and it warns you under some certain circumstances. So let's say you have a function that you marked Clang non-allocating and you try to use operator new, you'll get a nice warning that says you tried to allocate in a non-allocating setting. And it also constrains the functions that you can call. You know, if you call a function that calls a function that allocates,

you want that also to be warned about. So this performance constraints attributes being at compile time and real-time sanitizer being at runtime kind of complement each other and fill in the gaps of one another. So the compile time thing is going to basically generate warnings. So I cannot allocate in a non-allocating or non-blocking functions and I can't call a function that's not allocating or non-blocking? Yep, exactly right. So to be really precise with the wording, so from a non-blocking context,

you can't allocate memory. You can't call like static local variables because a compiler might insert a lock there to make sure that it's initialized properly. And you can't call any function that is also not constrained in the same way. So all your functions that you call need to also be marked non-blocking. And all the functions that they call also must be marked non-blocking and so on and so forth. Okay, but is there like an escape hatch or something?

Because, for example, if I know that my vector, I just reserve memory for my vector, that's actually quite common, right? Then you know that pushback is not going to allocate, but the compiler doesn't. Is there like an escape hatch to tell it, no, no, no, this is fine. Like I know what I'm doing. Obviously the runtime thing is going to just catch it or not catch it, right? Yep, exactly. And I think that's good to bring up that again, these tools compliment each other, right? So even if you put the escape hatch in the compile time approach, real-time sanitizer can look at it and verify that right like this this is why both these tools went on for a little bit it was we were debating is one better than the other should we focus like meaning in the LLVM community and ultimately I think we decided that

they both complement each other so to get back to your escape hatches question so just like you can disable other warnings on a call site you can do the pragma push pop kind of thing. In the original proposal, there was a really nice macro magic. He marked it non-blocking unsafe. So you could put this in your code, but every time you watch by it, maybe you'd have some psychic damage seeing it and saying, like, I should really fix that. Or, you know, or maybe this is a true opt out. Maybe this is the pushback case. And this is safe at this case.

And then there is the inverse case. For example, I write something that I know is going to be blocking, but the compiler doesn't because I'm not, you know, I'm not using, I'm not calling malloc or like waiting on a mutex, but I have like my own spin mutex or something where like i'm just spinning on a on an atomic flag or something and i don't think the compiler knows you know that whether that's bounded or unbounded right so it's not going to be able to reason about whether that's effectively a blocking operation or not is there like an opt-in i can use for that absolutely it

is called clang blocking so just like you would mark something as non-blocking you can document to your user that this is clang blocking and shouldn't be used in this situation so here's a good point to kind of like point at the two tools different approaches for real-time sanitizer you have to actually mark things as clang blocking for it to be picked up and put in our interception system. But the performance constraints is kind of a pessimist. It says like, if anything is unattributed, I'm going to be skeptical of it. And I'm not going to allow you to call it, right?

So this is kind of like a pessimist optimist approach. Like real-time sanitizer will run until there's actually something you intercept while performance constraints will say, I don't know what you're about. I don't know if you're safe or not, so I'm going to disallow it. But clang blocking is the attribute you're looking for. And I'd highly recommend if you know something that shouldn't be called in the real-time thread. I've seen comments in audio code base, I'm sure, Timur, you've seen this as well.

There's some big all caps comment that says, do not call this in the real-time thread. I swear to God, please do not. And that the real-time thread i swear to god please do not and that that only works as far as any comment ever works which is to say not at all so so this attributing your thing clang blocking actually provides some compile time or runtime approach to preventing this so the way you explain this makes me really think i wish we had this tool 10 years ago or you know potentially 20 years ago uh i think that would have been so so so valuable to have yeah i i mean again that's that's why i joined this like i i i think if you went back and looked at whenever david and ally presented this talk

to me sending the first comment to say i want to be involved in this this is really important work was probably like five seconds like it was so obvious to me like if you've worked in these environments that's not a real-time operating system but you need real-time ish constraints to this thing things like audio etc it's such a no-brainer it's such an obvious improvement to the workflows that i couldn't i couldn't get involved fast enough but barely longer than a typical blocking call yes exactly right so okay i have one question are you aware of any particular foot guns with these tools that you know people should know like is there any way i can shoot myself in the foot as we like to say in t++ with this stuff yes absolutely so there are things that are definitely uh good to be aware of uh with these

tools so uh neither of them can provide full real-time guarantees and what i mean by that is you know they can't guarantee slots on the processor they can't you know like there's no way to fully uh say if you pass these tools, you're 100% compliant. So the first off, you know, a little mealy mouth maybe, but good to be aware of. So we're still in user land, right? We don't have any way of actually telling the kernel, like the thread scheduler, like what to do, right?

So we're still at the mercy of the operating system with all of this stuff, yeah. Exactly. And just like any sanitizer, so I'll start with the real-time sanitizer. Real-time sanitizer only shows you issues that it hits during runtime. So to make that concrete with an example,

if you have some if statement that says if leap year on February 29th, and in there you allocate an extra day in your calendar and you're doing this in a real-time thread for some reason, if you don't actually hit that during testing, you'll never see that stack trace, right? So just like address sanitizer, thread sanitizer, when you hit in a thread sanitizer, a data race, it will print it out. Real time sanitizer, you actually have to hit the issue to see the issue. So that's one thing to be aware of. On the performance constraints attributes,

one thing that you can see as a possible problem is third-party libraries. Let's say you have a third-party pre-built dependency. They haven't put in the performance constraints attributes anywhere. What do you do with this? Well, one way to get around this is to redeclare. Basically, it uses the most recent declaration to determine if it's non-blocking or non-allocating. So you can redeclare. Basically, it uses the most recent declaration to determine if it's non-blocking or non-allocating. So you can redeclare the function as non-blocking or non-allocating, but you have

to pinky promise that you actually looked at the code and said, this doesn't allocate or block, right? And if you lied to the compiler or you misconstrued it or it changed in a future PR, the compiler will be silent in that case. And and again this is one of the ways that these tools really complement one another is you could run real-time sanitizer on that section of code to verify that it actually was safe in that circumstance so these are the kind of foot guns to be aware of so i i am actually curious about the redeclaration thing because i think what we do in the c++ standard with the standard attributes is that if you have like one translation unit where a function is declared with an attribute and another one where it's declared with another without the attribute then that's like it'll form no diagnostic required so like basically

you have no guarantees that what what you get or don't get, and it's technically not allowed. But I guess because that's a compiler thing that the compiler is aware of, it can probably do something slightly better than that. Yeah, I think it's the latter case. Again, I'm less of an expert on the specifics of the attribute, but I do know you can redeclare a function as a stricter constraint,

and you can't redeclare a function as a stricter constraint, and you can't redeclare a function as a less strict constraint. So if the compiler, if you include the header and you try to declare away a non-blocking function, you will get a warning in that case as well. So I think because the compiler is kind of involved in this, it's a little bit more specialized maybe than the standard but yeah i guess i guess it's a problem because some third-party things you get like a library or framework where you have the whole code and you can like for example something like the juice framework which is what like lots of audio plugins are built with i expect that they're going to add this stuff to their their framework but if you have like some some dsp library from some dude

somewhere in this like small german town who figured out the best like i don't know pitch shifting algorithm in the world and just sells it now you're not going to get his code right you just get a binary and so with the header but then i guess the redeclaration method works. Yeah. But not for anything that that library uses internally, right? But I guess you don't need that. Yeah, you shouldn't need it for the internal ones. It kind of stops at that layer. It cascades all the way, right? So if the compiler sees, oh, this is a non-blocking function,

if it doesn't see its definition, if it only sees the declaration it will still yeah i think if it doesn't see the definition it's okay with just the declaration and i think again this is an interesting uh combined use case of real-time sanitizer with the performance constraints attributes because you can kind of empirically test one of the major benefits of real-time sanitizers. You can go through and empirically test like a standard library or a third-party library and see, did I actually allocate in this case?

Like, is pushback safe in my context? Or did this library from this small German town actually allocate in this case? Right. Unless they have their own allocator. But even in that case, they will eventually call malloc to request underlying memory from the operating system.

And that's when you get them. That's when you get them. You point to the stack trace and you file a bug on GitHub saying, I knew it all along. You weren't playing by the same rules that I was. Right. I think I saw also that for the trivial inline functions

it can sort of infer it if you prove some heuristics if it's not attributed like yeah imagine if you call like make unique for example it can say what that ultimately calls new exactly so that's one of the really interesting parts of the performance constraints attributes again you can imagine this very laborious process. There's no way around it that the performance constraints attributes, if you have a 10,000 line code base, it's going to take a little while for you to propagate these things around. with the compiler are smart enough to go look at function bodies in the same translation unit in which it's used in a non-blocking context and deduce if it's non-blocking as well so this saves you a lot of time you know if you have some function that's trivial you know definitely is is safe it will be able to look at that and make a make a good guess of if it's a non-blocking or not yeah that sounds like it should make it a lot

a lot nicer to work with i think absolutely right so one question so we're talking a lot about allocations and and locks uh which are kind of the two classic examples of like non-deterministically timed things that you shouldn't be doing uh if you care about how long a piece of code executes but are there i mean there's a bunch of other stuff right there are you shouldn't probably do it i mean i've done talks myself about this topic right so so it's kind of like you shouldn't do any io you shouldn't like probably do any other syscall uh like anything that crosses this kind of like user space kernel barrier like how do you determine like which things do the tools actually catch? Like in particular, the real-time sanitizer,

like does it catch any syscall or does it catch like, is that particular whitelist or blacklist or what, like which things apart from like mute access and allocations, does it actually trigger on? And how does it work? How does it know? How is this implemented? I'm actually quite curious about that too.

That's another question. Yeah, absolutely. So the basic idea is right now there is a blacklist or a disallow list that we have come up with to say like, okay, Malik for sure is unsafe. Okay. Something like read, write, open your IO operations. We were talking about any interaction with sockets. So these are all things that live at the lib C level of, you know, things that we have a pretty good educated guests call a syscall under the hood. So we

unfortunately can't intercept is a syscall happening but we can make an educated guess saying this libc function is named the same exact thing as a syscall so we're pretty sure that it's syscalls under the hood and we can empirically test that using something like strace or instruments or things like that so if i just manually do like a direct like syscall into the kernel, it's not going to... Yeah, that's another one of these kind of foot guns, you know, things to be aware of. Something like if you do an assembly block that makes a syscall directly, we can't help you. What if I create like a manual like pthread instead of using stdthread or something like that? Yeah, so that's one of the benefits of intercepting at this libc level is it lives below the C++ runtime. So in C++ runtime, there's a bunch of functions, CXANewException, or however it's called, OperatorNew.

At the higher level in the STL, there's vector pushback. There's many ways to allocate, but it all comes through this nice funnel that eventually ends up at malloc or if you have some scoped lock i think you know on posix systems i think we can all agree that somewhere under the hood it lands on a p thread mutex lock so that's the beauty of kind of intercepting at this libc level okay uh what happens if you're not on a posix system like does this work on windows so we are in early stages of exploring this so i saw the first version of it uh you know running maybe the tests weren't passing all the way but uh we're exploring that right now so there is an interception mechanism on windows and we're optimistic that we

can uh you know at least uh figure out what what the pros and cons are i think the only other sanitizer that works on windows right now is asan but they do have some interception of malloc and things like that. So we're in an exploratory phase. I really would love to see this working on Windows. And thanks to a couple of people on our Discord server that are helping us kind of like explore that. So if I want to do this, if I want to use this today uh where do i where do i get these tools so it works on linux and it works on apple platforms is that right yep exactly right do i get like clang trunk or something where do i get these tools from if i want to use them today

and are they actually uh like now safe for production or is it still like a typey kind of thing? Great question. So there are kind of two approaches. One is the most official supported method that I want to encourage people to do if they can. That is get Clang Tronc, build it. We have a helper repo as well as official documentation that I'll send to you all. If you would be so kind to put it in the show notes, would be very helpful to us so that's the most official version you build the compiler with the tool in it etc etc we're also realists that we've worked at big companies that you can't use the most you know what I'm excited about C++ 26 but it'll be 2035 before I see it, right? We're aware of these limitations, I guess, in the C++.

Yes, yeah, right, exactly. So there is a way to build the standalone library, and as long as you're willing to put in a few objects and function calls into your code, you can actually use RTSan today basically as a standalone library. And again, in these links that we'll provide, you can do this in your code today. And I would say, yes, it's production ready. We're very satisfied with the feature list as it is now.

Between now and January, we want to add even more interceptors, make sure we round out all of the things. I know there's a few like socket libc calls that we're not intercepting. There's a handful of things that we want to do to just clean it up but even today I think you would get great benefit out of it as a as a developer in this space right we mentioned earlier a few of the application spaces you might use this obviously audio both your backgrounds are in we also mentioned you know rockets and and some others but but are there any other use cases

that people might want to use a sanitizer like this? Yeah, I mean, just to be clear, because we've gotten this question before, this is not as ubiquitous as an ASAN and as a UBSAN. And that was one of the hurdles we actually had to come over is kind of convince people that the user set was big enough for this thing, but it's not ubiquitous. You know, all of our code bases in C++ have to contend with undefined behavior, memory issues, etc. This is a very specialized set. That specialized set includes things like the video game industry, you know, you're rendering a frame, you don't want to have frame drops, things like that, audio, autonomous vehicles, robotics, we've seen a lot of interest from the embedded space so like you know imagine you have some little embedded device that you have to

provide some feedback on a on a given loop things like that aerospace maybe even more constrained environment is like low latency so like i know i know for a fact in like bloomberg trading systems they definitely are not allocating for the same exact reasons that we don't allocate. So maybe they would be interested in FinTech, high-frequency trading side of the world. Yeah, which is strictly not real-time, but it's subject to many of the same constraints, yeah.

Right, exactly. It's kind of, in a way, more brutal because it's like, in real-time, you at least know your deadline, right? It's like you have, you know, 15 milliseconds to render the next frame and you have like three milliseconds to render the next audio buffer and you have like one millisecond

to like trigger the brakes in a self-driving car or whatever, right? I think I've never actually worked as a developer in the fintech industry, but I have friends who do work there and from what I've heard like you don't actually know what your deadline is because you just have to be faster than all your competitors

it's always too slow so it's in a way it's actually more brutal Phil you've done this kind of stuff haven't you yeah I've not really worked strictly in high frequency trading but I've been very near it so I've definitely encountered those sort of mindsets yep if it's running at all it's too slow yeah i i have uh you know it's like if

if i have a 10 millisecond audio callback i don't get a cookie for finishing in one millisecond but you know like they they might get reprimands for finishing as slow as one millisecond you know it's as fast as possible as opposed to finish before this deadline i guess yeah so so it's real time or faster exactly right so before we get to the end i'm curious if you can say a little bit about how it works under the hood you haven't talked about that very much like i think maybe some of our listeners might be curious like because you said that there is like a counter going on like how do you yeah how does it work what does it actually do like the real-time sanitizer obviously like the other thing is like a thing in your compiler, but the real-time sanitizer seems a bit more kind of interesting.

Yeah, so again, to kind of restate the very high level, we are going to catch real-time safe calls like malloc and pthread mutex locks in any function marked clang non-blocking. So that kind of implies two different things. One, we have to track the context of what is non-blocking. So the way this works is when you compile with the fsanitize real-time flag, in the LLVM IR, the intermediate step, it just puts a call, a kind of scoped call,

you can think of this as RAII or something like that, saying I'm entering a real-time context, and I'm exiting a real-time context. So this is the counter we're talking about. You can imagine it being a simple stack, basically. Increment by one, decrement by one at the end. So it has to be atomic, right? Because it needs to be threat-safe. Yes, exactly right. And yeah, we use some...

Yeah, exactly right. And when it is in that context, the other most interesting part is the interception or interposing system, which basically says, when you request Malik, instead of going to LibC Malik, go to our RT-SAN specialized Malik. undergirding part of all sanitizers, it looks and it captures this call and then changes the call in some way. So you could print every time you call malloc or you could log a timestamp or something like that. What our intercepted functions do is we check if our thread is in a real-time context. If it is, we print the stack and die. we give back the real malik so we point them onto the real malik so we're tracking the context via this counter and then when we get a call that is suspect one of our intercepted calls we check if we're in a real-time context and if we do we die otherwise we just hand back normal malik and you behave like normal and you mentioned you got

this sort of raii like context is there a way to sort of opt out of that as well so if you're in a non-blocking contact can you say it's actually okay here if we're blocking again for a while yeah really good question so there are multiple ways to kind of opt out of this these uh kind of checks like say you have something that you know triggers the sanitizer but you don't want to deal with it right now. So at compile time, you can specify this scoped disabler object. So LLVM20 is going to ship with this header that defines a scoped disabler, which is exactly what you're describing. If you have something like a third-party library that's particularly noisy and you don't have access to the source code, for instance, you can define a suppressions file at runtime and pass it through this environment variable and it

will say oh did uh my sketchy library malik come up again i can't really fix that i'm going to suppress that problem for now and again i would i would encourage you to fix that and you know the suppression suppressing is not fixing but uh it's good to have these escape patches. Yeah. So one meta comment here, I mean, I'm impressed how, since the first time I heard about this tool and, you know, you asked me, Hey, do you want to have a look at this? I'm very impressed how far it came and how short of a time, like, it seems like you've basically thought of all the, all the use cases, all the corner cases, all the things that you might want to have there. Uh, it seems very

comprehensive. I'm very impressed by that. Well, thank you for both me and David say, say thank you for that. That means a lot. Um, yeah, big, big shout out to, uh, the other co-author and the guy who kind of came up with it to begin with David Trevelyan. I think we've had a really good working relationship and, um, just kind of, again, thinking about all these issues when you're, when you're going to propose something to LLVM, like you're going to provide, propose something to the C++ standards committee. You have to kind of think about all of these things. How is somebody going to opt into it? How is somebody going to opt out of it? How, what if people don't like the defaults, et cetera, et cetera etc so a lot of you know of the

last year that we were implementing and upstreaming this very small portion of that was coding things but very large proportion of that was wrangling all the cats and like making sure we what we were coding was actually correct so um yeah it's it's a tool we're both really excited about, and I think it will provide a lot of good use for the users. Good stuff. We are at our real-time deadline, so we've got our final question that we usually like to wrap up with, which is, is there anything else in the world of C++ right now? So particularly we're talking about another latest language features um that you find particularly interesting or exciting yeah so a couple things that kind of come to mind first on the lvm uh approach highly recommend people get in there and learn more about their compilers and

tools you know like this is something i kind of took address sanitizer and undefined behavior sanitizer for granted learning about how they work and actually reading the darn manual a little bit taught me a ton about these tools and how they work and stuff like that. So a lot of really cool things going on. Timur, you said there's not every day a new sanitizer comes in, but alongside us in LLVM20 is numerical sanitizer, which I think tries to detect kind of like round tripping of floats and things like that.

Like, did you get converted between a float and double? And I'm far from an expert here, but all I'm encouraging people to do is check these out and be kind of like on mailing lists and learn about these tools. Right. Yeah, that sounds fascinating. That's actually something we should also probably have an episode about, Phil, at some point. Yeah, I'll try to link you guys to the documentation it's it's come again a long way like very quickly and i think a lot of

companies are very interested in that so yeah uh nsan is what this new sanitizer is going to be so anyways there's there's a lot of things out there in your compiler tools like clang is constantly being developed um i know i'm cherry picking here, but contracts, I sat through your contracts talk and CPP con teamer, and I find that really interesting. As a heavy assert user, I've always wanted asserts to be substantially better than they are. They have a lot of pain points, as you well know. So I'm really crossing my fingers for 26 there.

And yeah, of course, reflection is the word on everybody's mind. I think C++ a decade from now is going to look a lot different than it is today so lots of cool things going on in the world all right well um we're gonna have to wrap up so uh thank you for coming on the show and telling us all about real-time sanitizer and the the performance constraints stuff as well or anything else you want to tell us or where people can find you if they want to find out more how they can reach you yeah so i'll send you some um links to put in the show notes if you'd be so kind but yeah we're we're on discord we have a little kind of spin-off

discord that we were using for development for a little bit um you can reach david or i there um the documentation page for real-time sanitizer is up, and we have this little helper repo kind of helping people get through initial implementation. But yeah, please reach out if you enjoyed the tool. Like, you know, the curse of software engineering is you only hear about when your tool fails in some way. So if you have some success story,

like somebody caught an allocation where they didn't expect an FFTW, which is the big FFT library that almost everybody uses. So we were like, oh my God, this is like the coolest thing I've ever heard. So please reach out if you have a success story. We'd love to hear that. Yeah, absolutely. All right. Well, that's been a great episode. Thank you again, Chris, to be our guest today. And we will be checking out those tools. I'm very excited about them.

Thanks so much for listening in as we chat about C++. We'd love to hear what you think of the podcast. Please let us know if we're discussing the stuff that you're interested in, or if you have a suggestion for a topic, we'd love to hear about that too. You can email all your thoughts

to feedback at cppcast.com. We'd also appreciate it if you can follow CppCast at cppcast on X or at mastodon at cppcast.com on Mastodon and leave us a review on iTunes. You can find all of that info and the show notes on the podcast website at cppcast.com. The theme music for this episode was provided by podcastthemes.com.