CppCast - Software Transactional Memory

Episode Date: September 2, 2015

Rob and Jason are joined by Brett Hall to discuss Software Transactional Memory. Brett Hall is the lead engineer on Dynamics, a desktop application that collects and analyzes data from the light scattering instruments built by Wyatt technology. Prior to joining Wyatt, Brett worked in web application development, remote sensing, and spent a summer in the games industry. He holds a PhD in physics from the University of California, Santa Barbara. Part of his research work involved using C++ to solve the PDE systems generated by the rest of the research. All told he’s been using C++ for around 20 years now. These days the bulk of his programming interest is in concurrency and parallelism. When not programming he’s usually hanging out with his family and/or mountain biking. News CppCon call for additional content Served: A C++11 RESTful web server library Modern C++ for the Windows Runtime now available Brett Hall @bretthall Backwards Incompatibilities Links CppCon 2015 - Transactional Memory in Practice CppCon 2014 - Software Transaction Memory, For Reals ISO C++ Paper - Industrial Experience with Transactional Memory at Wyatt Technology

This episode of CppCast is sponsored by JetBrains, maker of excellent C++ developer tools including CLion, ReSharper for C++, and AppCode. Start your free evaluation today at jetbrains.com slash cppcast dash cpp. And by CppCon, the annual week-long face-to-face gathering for the entire C++ community. The 2015 program is now online. Get your tickets today. Episode 25 of CppCast with guest Brett Hall recorded September 2, 2015. In this episode, we discuss a RESTful web service in C++. And we'll interview Brett Hall from Wyatt Technology. Brett will tell us about his experience implementing a software transactional memory library. Welcome to episode 25 of CppCast, the only podcast for C++ developers by C++ developers.

I'm your host, Rob Irving, joined by my co-host, Jason Turner. Jason, how are you doing tonight? Doing good, Rob. Spending lots of time working on my presentation for CppCon now. Yeah, it's coming up soon, isn't it? Yeah, it sure feels like it. Yeah, and it's two soon, isn't it? Yeah, it sure feels like it. Yeah. And it's two presentations this year, right? No, just one presentation, but I did sign up maybe to do some open content about ChaiScript. Right, right. Okay. That'll be exciting. So are your, all your slides ready and

everything? Oh no. Oh no. Okay. Well, that's good. Still got a lot to do, but it'll be exciting. I really wish I could make it out there this year. It looks like it'll be good. Yeah. Well, at the top of every episode, I like to read a piece of feedback. This week I got an iTunes review. It comes from Keith, and he wrote in that it's great to have a podcast specifically devoted to C++ topics.

I listen to a couple of other software development podcasts that, while interesting, usually end up discussing technologies that are not directly relevant to me personally. This podcast is excellent, covers a broad range of topics, and frequently is an interesting guest. Keep up the great work, guys. Keith, thanks so much for the kind words. We're glad that you find the podcast interesting and, uh, yeah,

we'll keep it up and definitely keep bringing you a interesting guest to listen to. This show would be nothing without guests. Yeah, that is definitely true. So if you have any desire to come up on CPCast and talk about what you're working on, we're always looking for new guests and,

uh, we'd love to hear your thoughts about the show. So you can always email us at feedback at cppcast.com, follow us on Twitter at twitter.com cppcast, and like us on Facebook, and always review us on iTunes as well.

So, joining us tonight is Brett Hall. Brett is the lead engineer on Dynamics, a desktop application that collects and analyzes data from the light scattering instruments built by Wyatt Technology. Prior to joining Wyatt, Brett worked in web application development, remote sensing, and spent a summer in the games industry. He holds a PhD in physics from the University of California, Santa Barbara. Part of his research work involved using C++ to solve the PDE systems generated by the rest of the research.

All told, he's been using C++ for around 20 years now. These days, the bulk of his programming interest is in concurrency and parallelism. While not programming, he's usually hanging out with his family and or mountain biking. Brett, welcome to the show. Thanks. Good to be here. So, we have a couple news items to talk about before we get to you, Brett. The first one, as Jason just mentioned, is the CppCon open content sessions.

Jason, do you want to talk about this? Yeah, they've got open content and lightning talks that they are looking for submissions for. So the open content sessions are in the morning, lunchtime, or in the evening, I believe. And they've got, it looks like, a lot of opportunity for people to sign up. And you can propose a talk on basically any topic. It doesn't need to be general interest. It can be something specific or some project you've been working on. You just want to get other people involved. It's a good opportunity if you've got, I don't even know, some way that you're trying to connect with users of your project or something,

which is what I'm hoping to do with my ChaiScript open content session. And the lightning talks, if you've never been to lightning talks, are incredibly awesome, fun times. Absolutely. And Brett, you did a lightning talk on what we're going to be discussing tonight, and that's now evolved into a full talk at this year's CPPCon. Is that right? Yeah. Yeah. And definitely the lightning talks are a great way to kind of get used to doing talks since you don't have to prepare as much, five or 15 minutes, though it's actually kind

of tough to fit a topic into that amount of time. I was taught, if you watch the video of my talk from last year, I was talking very quickly. Right. Well, all you need to do if you want to get into an open content session is email open-content at cppcon.org. So if you're planning on going to CppCon, it's definitely something worth doing or, you know, look for the open content sessions that you might want to attend. So the next article is Served, which is a C++11 RESTful web server library. And this seems pretty interesting. It's definitely not something I've seen before from C++,

but they have a pretty basic hello world example here on GitHub, and it's just like a dozen lines of code to start up a RESTful web service that you could ping using curl to test it out. Looks pretty neat. Jason, do you have any chance to look at this at all? I did. That's pretty awesome. It's the kind of thing that every language except for C++ and C has had for a while, so it's awesome to see people working on this kind of thing. My only disappointment is that it doesn't say that Windows is supported. Did they list the compiler support? Yeah, at the bottom of the readme system compatibility.

Is it just Visual Studio they don't support? Well, I guess that's technically true. Well, they say Linux and OS X specifically. But there's no reason why they couldn't add AppVeyor and have automated builds for Windows just to see what happens. I thought I'd throw that out there. I'm a huge proponent of AppBear. Yeah, I wonder what they might be missing.

Or if anyone's even tried to support Windows using this project. Yeah, I don't know. Might have to look into that. It's all based on Boost, so it should work. Right. It doesn't say anything about being C++ 14 dependent, does it? I don't think I saw that anywhere in there. Okay. And the last article

I had, this actually came out a couple weeks ago, but we had Kenny Kerr on maybe a month or two ago, and while he was on, he talked about his library, Modern C++ for the Windows runtime. And it's basically, the goal of it is to

avoid using the C++ CX language extension when writing Windows apps and be able to do everything in pure idiomatic C++. And at the time that we were talking to Kenny, he was still kind of keeping the project to himself, but he has now put it up on GitHub, and anyone can go and create an application using Modern. So if you're interested in making your own Windows apps using C++, it's definitely something to look at. Did either of you have anything to add to this?

Well, just that the Modern, it looks looks since the project I work on at work is pretty much entirely Windows it definitely looks nice since the Win32 API is kind of a pain to use from C++ yeah are you making a Windows desktop application I'm guessing

yeah so it might not help you too much there but if you were interested in porting it to, you know, the modern windows, is it mostly the, the universal? Yeah, exactly.

That's what they were calling Metro. Right. Right. Yeah. Well, yeah. So if you're probably half, we'll probably have to switch to that at some point.

So it'll help at some point probably. Okay. And so let's start talking to you, Brett. Um, as we've mentioned, you did a lightning start talking to you, Brett. As we've mentioned, you did a lightning talk at CppCon 2014. Can you give us an overview about that talk and what exactly transactional memory is? Sure.

So the talk was basically about our experience using transactional memory in the Dynamics application, which is the main project I work on at Wyatt technology. Um, so yeah, I guess we should probably start with what transactional memory is since whenever I asked someone, like if I'm interviewing someone for a job at Wyatt, I asked him if you heard of transactional memory, it's always blank stares and crickets. So, uh, basically it's an alternative to using mutexes when you're doing concurrency.

And basically the gist of it is in some way you start a transaction, whether it's someplace in our system you call a function, which then calls a function you pass to it with the transaction running. Other systems have other ways of starting it. But once you're in the transaction, your reads and your writes are journaled,

so it keeps track of all your reads, and if you read the same variable twice, you see a consistent value, even if another thread changes the value between the reads. And the writes don't become visible to the other threads in the system until you commit the transaction. So it's sort of similar to like a database transaction. And so when you get to the end of the transaction,

all of your reads that have been journaled are checked to make sure that they're consistent. No one has changed any of the values that you saw. And if the values are all the same, then it commits and your rights become visible to the rest of the system. If there were changes, then you have to throw everything away and go back to the beginning and run the transaction again. That's generally how it works. You can get a bit more technical about how it's specified to make it a bit more flexible implementation-wise. But generally, that's what's meant by transactional memory. So you said it can be thought of like database transactions. Is there any notable differences that you could point out between them?

Well, I mean, usually with database transactions, there's also notions of durability and when things get written to disk. And obviously, since this is all just memory stuff, that's kind of out the window. Right, okay. I think that's probably really the only real difference that I can think of. I mean, also, with database transactions, you can implement them lots of different ways. You can implement a transactional memory system lots of different ways as well. So, yeah, if you're used to database transactions, it's kind of similar.

Okay. That's a good starting point for, I think, a lot of developers. Yeah. So how do you go about using transactional memory in Dynamics? So we started, it was going to be basically for the data store. So maybe I should start off and describe Dynamics in a little more detail. So basically our light scattering instruments, they're mostly used by biotech and pharmaceutical companies

doing r&d and maybe quality control um so what is a light okay sorry go ahead yeah i'm about to yeah i'm about to describe that um basically you have some sample of like um some compound you want to use as a drug or something um and you dissolve it in a solution you put it in our instrument we shoot a laser into it and we look at how the light comes out and from that we can tell how big the particles are, how heavy they are, things like that. So Dynamics controls the instrument,

takes the data and analyzes it. And so the data is basically broken down into measurements where like you say okay measure this sample now measure this sample measure this sample and so each measurement was something we were going to transact it's actually broken down a bit finer grain than that but originally we were just going to use it in our data store so that you could tell if like when you're doing calculations if like the parameter you can modify things about the measurement, like the parameters, like you can say what kind of solvent you use, things like that. And let's see, where was I going with that?

Oh, so we were just going to transact the data in the data store when you're doing calculations. And so, like, the GUI could safely get the data out of the data in the data store when you're doing calculations. And so, like, the GUI could safely get the data out of the data store and then be informed if there were changes. And it kind of sort of grew and engulfed the whole program. I think the only place we don't use the transactional memory is at the lowest level of instrument communication where we're getting data off the socket,

and we have to do that um sort of deterministically um and quickly so that the instrument doesn't hang up with us if we don't keep up with the instruments what the instrument is streaming us over the um streaming us over the network then it'll hang up on us um and obviously the gui is just written in mfc so that's not transactional. It interacts with the transactional system. But basically it's allowed us to eliminate locks. We don't have to worry about deadlock at all, which is really nice. Let's see. Hopefully that answered your question.

Well, it's made sense to me. I guess Rob's actually the one who answered the question, but asked your question. Well, it's made sense to me. I guess Rob's actually the one who answered the question, but asked the question. But it led to another question for me. So you've eliminated mutexes, which is super awesome, but you do have to deal with the state, the possibility that you have a conflict, right?

That two different processes have written to the same data at the same time, and now you have a conflict and have to deal with it? Yeah, well, the system just handles that for you automatically. That's the basically it rolls back if whichever one, which so if you have two transactions, and they both say one reads from a from a certain variable, and one writes to it, if the one that wrote to it commits first then the the second transaction that did the read will have to go back and start over and try and make it through a second time without something uh without another thread um giving it a conflict so is there like an automated retry process uh yeah yeah that's why so in our system when you basically there's a function called atomically you pass it some sort of function object it's pretty much always lambdas these days um and it runs the transaction

for you and when it gets to the end it does the validation where it checks to make sure that you reads no one's written to something that you read and if it does it basically just throws away everything you did and calls and calls your function object again. So it's basically all handled automatically. Do you ever end up in a race or, I don't know, some situation where they get hung up, everyone trying to read and write from the same variable at different times? No. Well, so the whole thing is you can actually show,

there's academic papers on it, that there's always at least one thread can make progress. There's always one thread that will commit first, and it has made progress. You can end up with what's called starvation, where if you have a really long, like if you have one thread that's running a really long transaction, you have another thread that's running a bunch of short transactions,

and the short transactions keep writing to the thing that the long transaction is trying to read from, and it can never finish because other transactions keep running and giving it conflicts, that can be an issue. It's only actually... We've been doing this for five years, actually. The other day I went and looked in our source code control system

because I wanted to figure out how long we'd actually been using this i'd forgotten and the first commit was like in january of 2010 i believe so we've been doing this over five years and i've only run into that once and it was like a few months ago um and it wasn't even it wasn't horrible it was just that it caused something to to wait for it for a really long time while there was another process going on once that process, the first process could then do its thing. It was more of just, it would be an annoyance for users. It didn't cause the whole program to lock up. But yeah, so there isn't

really a danger. There's no deadlock danger. The system can always make progress. It's, you can run into starvation and we have ways of mitigating that. Um, in our case, we just basically, if you, if you know that a transaction is going to take a long time, you can put a limit on how many times it can get conflicts. And if it gets, if it hits that limit, it will go into what we call running locked, where basically no other transaction can commit until this one does. It's kind of a heavy-handed way of dealing with it, but it works for us. Other systems do things like you could prioritize transactions. If a transaction keeps getting restarted, you give it a higher priority

and sort of exponentially back off the the other transactions to give the other one to give the long one a chance to finish if in the literature there's lots of things you can try but we just kind of did the least complicated thing first that seems to be working okay for us isn't there something like that in tcp if there's a conflict. I forget. Yeah, I think if you're having errors sending packets, like they're not arriving, instead of sending it and just compounding the problem, it backs off and it waits longer and longer between retransmissions or something like that.

Right. So it is kind of similar. For over a decade, ReSharper has been helping.NET developers be more productive and write better code in Microsoft's Visual Studio. Starting this year, it expands the list of languages it supports by adding a dedicated edition for C and C++ developers. To put it simply, ReSharper C++ makes Visual Studio a much better IDE for C++

developers. It provides on-the-fly code analysis, quick fixes, powerful search and navigation, smart code completion, automated refactorings, a wide variety of code generation options, and a host of other features to help increase your everyday productivity. The list of things that ReSharper C++ can generate for you include definitions, missing and overriding members, equality and relational operators, hash and swap functions, add dozens of customizable templates, and you'll deal with boilerplate code faster than ever. Code refactoring for C++ help change your code safely, while context actions let you switch between alternative syntax constructs and serve as shortcuts to code generation actions. With ReSharper C++, you can instantly jump to any file, type, or type member in a solution. You can search for usages

of any code and get a clear view of all found usages with grouping and preview options. Visit jetbrains.com slash cppcast dash resharper dash cpp to learn more and download your free 30-day evaluation. And if you're a student or an open source project, get your license for free courtesy of JetBrains. So what motivated you to go about implementing transactional memory with Dynamics? Well, so originally I had been playing with Haskell, and Haskell is one of the places where transactional memory is used quite a bit. I've seen, like, forum posts on the Internet where people say,

eh, it's just kind of a solved problem, we just use transactional memory. And I had been playing with that, and I'm like,'m like i can probably implement this in c++ it's um there are some differences but um it was more just kind of like a hey i think i can do this let's see if i can do this um and then at the same time at why i think i've been there a couple of years at that point and dynamics is a pretty old program. And we actually acquired it out of one of our company we used to partner with. They built some instruments that complemented the ones

that we built. This is all ancient history before I was at Wyatt. But that company went out of business and we bought their intellectual property out of bankruptcy in their instrument designs and the dynamic software. And it had, at the end they had, let's just say the people who had been working on it weren't so concerned with maintainability of the software. So it was kind of a mess.

And it also, when it was designed dynamics, the, the experiment files could only hold one measurement. And the way that they did thread safety was basically use the do it in the GUI pattern, where basically once the, there were other threads that were used for instrument communication, but once the data was collected and put into like the data store, you could only do, you could only use it on the GUI thread,

which meant the calculations had to be done on the GUI thread. And the calculations, I mean, to calculate one measurement probably takes maybe a couple hundred milliseconds. But when you have a thousand measurements, that adds up fast. So if you change something where you have to recalculate all the measurements, and then you go and view the main data table in the application, it would basically calculate on demand. Like when you change something, it would mark,

oh, okay, so this measurement is going to need to be recalculated. When you went to look at something for that calculation, it would then recalculate it. If you went to look at something that looked at all the measurements, then the program would lock up for two or three minutes. Wow. It was really embarrassing whenever I had to talk to customers. Kind of takes the fun out of

working with lasers. Yeah, yeah. But unfortunately, there was competitive pressure to have certain features in the program, so we kind of had to... Well, so before that, I started investigating, well, how can we make this stuff

thread safe and deal with this problem and push the calculations into the background and at the same time also parallelize the calculations. All the measurements can be calculated independently. There's a lot of just embarrassingly easy parallelism to exploit. But the problem was when I started looking at that, I just found this cascading series of changes which would have to be done, which we'd effectively be rewriting the core of the program. So we had to put that off for a couple of years and I was playing with my own STM system

and I was also, as I was working on other things in Dynamics, I'd look at it, oh, this place would be perfect for transactional memory. This place would be perfect for transactional memory and eventually I convinced my boss we should try this. And originally, like I said, it was supposed to just be limited to this one area of the program where if it didn't work out, we could replace it relatively easily.

But so eventually, I think, so yeah, five years ago was when we started the, basically rewriting the core of the program, which obviously, which then also meant rewriting a good chunk of the GUI because that had to interact with the core of the program, which then also meant rewriting a good chunk of the GUI because that had to interact with the core of the program. And the transactional memory just fit, and then we kept finding more areas where it fit, and it just seemed to be working really well, so we kind of ran with it.

So that's how we ended up using it in dynamics um i don't know it just it just fit so you wrote your transactional memory library from scratch uh yeah yeah it's modeled pretty heavily after the haskell system um our it's it sounds more impressive than it is it's it's a really boneheaded implementation that kind of just uses one shared mutex that you take a shared lock when you're reading from a variable, and when you're committing, you get a unique lock. So no one can be reading while you're committing,

and only one thread can be committing at a time. Currently, most of our customers and most have four-core machines, and from my performance testing, the contention for that lock isn't bad enough in four-cores to motivate us to change that yet. I have lots of ideas. I've been reading lots of papers on how to fix that. I looked at a few other systems, like Intel had an experimental, like in their compiler, there was an experimental version of it that had a transactional memory system, but it was someone else's experiment that I wasn't going to have source code for, so

that's kind of a big gamble. And there were some academic systems that kind of looked interesting, but they were trying to transact it really at the bits and bytes level instead of the object level. And that wasn't quite what we were looking for.

And just looking at the system I wrote, it seemed to work fine for us. And we had all the source code. We had the guy who wrote it. So, okay, let's use that. You may have just answered my next question. I'm assuming that the STM system you wrote is all proprietary software. Are you aware of any open source STM projects?

There are. Well, so, like I said, there's a few academic ones. I don't remember them off the top of my head. And actually GCC as a version 4.7 has a system in it. It's more C based and it involves like calling when you want to read something transactionally, you have to like call a function to read it. And actually, just last week, we got the go ahead from the from the higher-ups at Wyatt to open-source our system. Oh, great. Currently, I'm trying to get a big release out, and I'm trying to get my talk together for CppCon. So nothing's happened beyond we've spitballed some ideas for what license to use. Hopefully in the next month or so, there will be something more. And I know the plugs don't come until the end, but I think there was,

I gave you a link to my blog. If people watch that, I'll definitely post something there when it happens, if people want to play with it. That's pretty exciting. Yeah, yeah. I'm kind of surprised.

I wasn't sure when I mentioned it to my boss a few months ago. I said, hey, what do you think about open sourcing some stuff? He was into it, but we weren't sure if we could convince the higher-ups or not. Yeah, 15 years ago.

Yeah. It definitely seems like a lot more companies are buying into the idea these days of open sourcing parts of their system. Yeah. The main concern was, you know, don't

open source any of our analysis algorithms or the instrument communication stuff. But everything else, I mean, is kind of fair game, I guess, to a certain degree. I was talking to my boss. I'm like, yeah, well, now I have to go in and improve the code, all the embarrassing things that I've just been living with. Because now everybody's going to see it. So it's already having that side effect of improving the code. So you've told us about all the awesome things that transactional memory can do.

Have you had any notable failures or problems? Well, I mean, one hard thing is you can't hire anyone who's used it before. There's just, like I said at the beginning, when you ask someone in an interview, have you heard of transactional memory? No one has. That might slowly change, but luckily it takes about a month before people stop making rookie mistakes with it. Part of it is you can't have side effects in your transactions. If you say you pop up a dialog, you have a function that you're running in a transaction. If that pops up a dialog, that dialog can pop up anywhere from one to a thousand times, depending on how many conflicts you get. That's usually the kind of error you see, and that's immediately you're like, okay, this thing's popping up.

Something is happening in the GUI a non-deterministic amount of times, okay, someone put a side effect in a transaction. Every once in a while, there's something like, I think there's been three or four times where it's taken me a couple of days in the debugger to figure out where the side effect was or what the side effect was that was causing a really subtle problem.

And other pitfalls we've run into, um, there's a side of it, a sort of related, um, there's a thing where you can get inconsistent values. Um, and this, this may be peculiar to our system and the way, way we do things, but say you've got two, two, say you've got two integers, A and B that you're going to read, and then you're going to, say, do A minus B and allocate that many bytes. And normally A should be bigger than B. But you could do something where you read A, and then another thread changes both A and B, and the other thread changes them consistently. A is still bigger than B. But when you go to read B, B has changed, and maybe now B is bigger than A. And then you're allocating, you know, 4 B, B has changed and maybe now B is bigger than A and then you're allocating 4 billion

you're trying to allocate an array of 4 billion bytes or something and that's not going to work that comes up every once in a while that may be peculiar to our system, I'm trying to come up with ways to deal with this that don't completely kill performance, at the moment it's only come up twice

and the solution is just to do a manual. We have a way of manually validating, basically running the same validation that gets run at the end of the transaction. You can run that whenever. Originally, we put it in because if you're doing a long calculation or something, you might want to validate partway through to make sure you're not doing work that's going to be wasted because you're going to have to restart anyway. Um, and in this case you, you read A and B and then you validate before you do the memory allocation so that you know, the A and B are consistent. Um, I'm trying to think there's some others. I, um, there's actually been a bunch of, I,

I do a little bit of work with, there's a study group for the C++ Standards Committee that's working on a transactional memory technical specification. Right. And eventually, hopefully, adding that to the language. And there's been a, so right now they've had their first version of it voted out of committee.

That was at the last standards meeting. And voting out of course means voting in, in the standards committee. In the standardese, it means yes. They've decided they want this. And so the version 1 just has the basic transaction stuff.

But now, so I wrote a paper for them on our experience. Basically the same thing I'll be talking about at CPPCon and what I talked about last year. Our experience using it and what features in our system we found useful because they're looking at a couple of newer features like... Well, excuse me. We have what we call...

Well, they're calling them on-commit actions. We call it after actions. But if you need to do like a side effect and you're in a transaction and you need to do a side effect based on what you've read in that transaction, obviously you can't do it in the transaction. And so you need some mechanism to say,

okay, when the transaction commits, do this. So we have a way you just register a function object and it gets called when the transaction successfully commits. So they've been working through things related to that. Also, there's another thing we have called retry, which is if, say, you read a variable, like you're waiting for some counter to reach 1,000 or something. So you read that variable, you check to see if it's greater than 1,000.

If it's not 1,000 yet, then you call this function retry, and what that does is it puts your thread to sleep until another thread modifies one of the variables that you've read. It's basically a replacement for condition variables. Right. And they're also looking at adding that, but there's actually a lot of pitfalls

related to retry that they're working through in the committee. We haven't been, I think we've been bitten by one of them, having to do with, if you say you have a transaction, the beginning of the transaction, you write to some queue that some other thread is going to read, and when it reads it, it's going to write to some queue.

And in your transaction, so you've written to the first queue, and then you go to check to see if there's something on the other queue. And if there isn't, then you retry. The problem is when you retry, your writing to the first queue technically never happens. So the other thread's never going to write and never trigger your retry. We're working through some, they're working through some issues like that. So how involved have you been with the committee exactly um really so i wrote the paper for them it was about three months ago i think that was the first thing first of all my other than at last year's cpp con i gave my

lightning talk um and then michael wong who's um who's in charge of the study group, gave a talk on transactional memory, what they were working on, and I went to that. I started talking to him afterwards, and he asked me to write this paper for them. And since then, I've been sitting in on their conference calls and their mailing list and just kind of shouting from the peanut gallery, oh, here's what we did when that happened. So I don't know if I've moved past the annoyance level yet

but um yeah so it's only it's only been a few months they've been around i think that's i think that effort's been ongoing for years now um and they are so so like i said they have voted out their first version of the specification of their technical specification and i there aren't any reference implementations yet. I know they're working on, one's being worked on in GCC, and there's talk of one happening in Clang. But none of the Microsoft or Intel or any of the big companies that build compilers have said anything about it yet.

So it looks like transactional memory is, you know, it's been a topic people have been talking about for a long time do you have any idea why it hasn't been thoroughly explored in c++ until like right now um well i mean there it's just we're still kind of you know feeling our way through into the multi-core era so i mean before there wasn't enough concurrency in most cases to need it. There's also concerns about performance. Just, you know, the transactional machinery has some costs and the non-deterministic nature has some costs compared with, like, locks. You know, locks, you're either stuck waiting on a lock

or you get through with the transactional memory. You might have to do a bunch of work and then do it again if you had a conflict because of the optimistic, basically it's optimistic locking. So I don't know. I don't have a really good idea why it's the case. I just know it worked for us and maybe part of it

is just mainstream the main non-academic people discovering it i guess might be part of it it never seemed to make the jump from academia into like real products well so then on the topic of performance it looks like on the wik Wikipedia page that there's some some of the modern CPU architectures have extensions that support transactional memory. Is that anything you guys are taking advantage of? We aren't, because

I don't, well, so what I know about it, so with Intel, there's the TS, I think it's TSX. Yeah, TSX on their newest processors. I actually just learned about it last year. Michael Wong told me about it. I had no idea it even existed. Intel hasn't been making a big marketing push for it, it seems.

But I know, like, originally it was on the really high-end Haswell processors, but it had a bug in it they discovered six months after they first shipped it. Oh. And they basically did a microcode patch and disabled it on all the processors. And I think about halfway through the Broadwell processor rollout, they fixed it and enabled it again.

For us, it's... So I guess I should get into a little bit. So our system, to use some buzzwords, is explicit and unbounded. It's explicit in that the only stuff that's transacted is stuff that's declared to be transacted, and you declare it to be transacted by, we have a templated class called var, which is the transactional variable. The template type is like integer or whatever type you want to be storing in that variable.

And that's the only stuff that gets transacted. With a library-based system like ours, that's about all you can do. The other option is to do an implicit system, which is what the standards committee technical specification does. And to do that, you have to modify the compiler because basically you start a transaction, everything inside the transaction gets, any memory operations you do inside the transaction

get transacted, so the compiler is instrumenting all that to make all the transactional stuff happen. The hardware systems are the same way. They're implicit. They basically, so I'll preface this with the only hardware system I've looked at is Intel's. And this is all just based on reading stuff. I haven't actually had a chance to sit down and play with it. But basically you issue, there's an assembly instruction you issue that starts a transaction.

And then everything that happens within that until you either commit or abort the transaction, which there's two other assembly instructions, gets transacted. So the other buzzword I mentioned, unbounded means you can do as much as you want in the transaction. There's no limits on how many transactional operations you do, obviously up to running out of memory

or address space or whatever. And the technical specification is also unbounded. The hardware systems are bounded by necessity because, at least for the Intel system, they transact cache lines, and they have two memory buffers, one for reads and one for writes. So when you transactionally read a variable or a memory location, that cache line goes into

the read set, and when you write to something in a transaction, it goes into the write set. If you read or write to too many cache lines so that you basically spill one of those buffers, your transaction's over. You're just, you can't commit it.

And then you have to somehow switch over to a software system. And the only way I can think of to do that is kind of, it's going to be a major problem. But really, so the bounded-unbounded is a problem. But really for us, we really rely on our system being explicit like when we do calculations basically we grab the the parameters we grab the data those are both

transaction transactional things um then we do a bunch of horrible math which is in in the part you know in like functional programming parlance that's all a pure function it doesn't have any side effects it just calculates something and then we have some results, which we then write, and that's transactional. So we rely, in order to get good performance, we rely on all that stuff in the middle, all the horrible math, to be non-transactional. With an implicit system, we would need to, basically, we need to read the stuff at the beginning, commit our transaction, do all the math, and then at the end, start another transaction, do the reads again to make sure nothing changed while we were doing our calculations, and then do the writes.

I mean, that's not horrible, but we'd have to restructure a lot of the program in order to do it like that. And I'm not sure. I think the new Skylake processors that are coming out, I think the TSX is kind of uniformly distributed through all of them. It used to be only like the high-end Xeon processors were getting the TSX extensions.

So it might be something we have to look at in the future as it's something our customers will actually have. But for now, the hardware hasn't been a big deal for us, and really we're getting plenty of performance out of our software system.

Just to dig into the explicit versus implicit difference for a moment, if you were starting today and the standards committee had already implemented transactional memory, do you think you'd be better off with implicit or do you prefer explicit um there are there's pluses and minuses to both um and actually so the standards people are also considering um something called escape actions which is basically you can delimit parts like when you're in a transaction you can delimit parts of it to be like there's nothing in here that needs to be transacted right um which would basically fix the problem we would have with our calculations we would just you know do the reads at the beginning and then the stuff in the middle would all be in an escape action and then the part at the end would be transacted

um i mean if if if the ts was here and supported and all that, I would probably just run with that. While it's fun working on our transactional memory system, it would probably be hard to come up with a business case for me spending time on it when there's already a perfectly good implementation that someone's probably way smarter than me is already working on. But as far as...

So the other thing is... One thing I like about our explicit system is when you mark something, when you use this var type to mark something as being transactional, you can only use it in a transaction. In the TS, it's just whatever variables you access in a transaction get transacted. But if you access something in a transaction and outside of a transaction

at the same time, so on two different threads, you have a data race. So you have to keep track of that. It's not any worse than, you know, using mutexes where you have to be sure that every time you access this variable, you hold the right mutex. But it is one thing I like a little better about our system. But it does complicate things because you have to think about ahead of time, is this something I'm going to need to transact? Okay, then it affects the type. And the type obviously has its own interface that you have to use to access the value. So, I don't know.

It's six of one, half dozen of another, I guess. Let's be wishy-washy about it. Okay. Correct me if I'm wrong, but I'm pretty sure reader-writer mutexes or shared mutexes did not make it into C++11. They came with C++14. Yeah, I'm using the boost.

That's what I was wondering. Yeah, actually a lot of our system is implemented on top of the boost stuff because, well, we're still actually on Visual Studio 2010, which is pretty bad. We're going to go to 2015 as soon as we get this

version we're working on now shipped. If for no other reason, I will want to have variadic templates to use. I keep finding places where they would be extremely useful and cursing that I don't have them yet. I'm having to do something else that's way uglier. Yeah, when I ported my library from Boost to C++11, I basically type-deft all my shared mutexes to exclusive mutexes so that I could go back once C++14 had been released.

Ah, okay. Yeah, and we actually... So I'm not sure what... Did they do it entirely? I haven't actually looked at the shared mutex in the standard library yet. I haven't yet, actually. Oh, you haven't either? Okay, I'm wondering, because technically't either. Okay. I wonder, because we actually, I mean, technically we use

the boost upgrade mutex because it's basically a shared mutex except for, so you have the shared lock. You can also get an upgrade lock, which is a shared lock, but only one person can hold an upgrade lock

at the same time. Like the upgrade lock can coexist with the shared locks, but it can't coexist with a unique lock. And while we're doing our validation, we hold the upgrade lock, and then we can upgrade that to a shared lock, or to a,

sorry, upgrade the upgrade lock to a unique lock once we're actually ready to publish the changes after we've done the validation. And I'm not sure if that much made it into the C++14 shared mutex or not. Yeah, and some of looking at CBP reference, it looks like some of it is C++14, and some of it didn't get in until C++17, maybe. Yeah. And I'm looking at replacing that shared mutex with a bunch of lock-free stuff. And that's always fun to work on until you actually have to ship the code and then you have to, you know, pull your hair out and hope there's no bugs hiding in there

that are going to bite you six months down the road at a customer site. Well, is there anything else you wanted to go over before we let you go brett um no i mean just i guess generally i'll say uh that you know our experience with uh transactional memory has been great um it's definitely it's much easier to think about than than locks when you're you know programming the large locks're not too bad when you only have one or two and you're working with a small subsystem. But when you have to talk about synchronizing big parts of the program,

the transaction memory is just a lot easier to deal with. We've had minimal problems. That being said, we don't have big performance constraints. Our main performance constraint, I mentioned earlier,

when we're taking data, we have to be able to keep up with the data stream from the instrument or it hangs up on us. And we don't actually use transactional memory there. Stuff comes in off the socket and it goes into a lock-free queue. But once it comes out of that queue, it goes into

transactional stuff and it's transacted from there. And we're not a game or a trading program that has low latency requirements. If our calculations take a second or two longer, the scientist isn't really going to care. They're probably not even there. They're off getting coffee or something because they know things are going to take an hour. So, yeah. So it works great for us. And, yeah, the open source library will hopefully be out there soon

and people can play with it if they want. Okay. Well, you have to send us a link once the library does go open source so we can share it with the listeners. And where can people find you online? So my blog is backwardsincompatibilities.wordpress.com. And my Twitter handle is bretthall, B-R-E-T-T-H-A-L-L. Very unimaginative handle.

And my blog, there's contact info if people want to email me or something. Yeah, and I think I sent you a few links to, there was the paper I wrote for the committee and their papers, which we're taking a look at if you're interested in this stuff. Yeah, I will make sure to put all those links in the show notes. Okay. Okay, thank you so much for your time, Brett. Yeah, thanks for Brett. Yeah. Thanks for having me on. Thanks for joining us. Okay. Another great episode, Jason. Yeah, it was. And, uh, you know, before we end this one, I just wanted to make a quick announcement that

we have a pretty exciting guest coming up next week. We're going to be talking to Scott Myers, obviously the author of the Effective C++ series. So I just want to let people know that he'll be here next week. And if you have any questions you'd like us to ask Scott, you could email us at feedback at cppcast.com or just ping us on Twitter at cppcast and let us know what you would like us to talk to Scott about. I'm looking forward to this. It should be a great interview. Yeah, I'm really looking forward to it. It should be great.

Okay, well, I'll see you next week, Jason. Bye. Thanks so much for listening as we chat about C++. I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in or if you have a suggestion for a++. I'd love to hear what you think of the podcast. Please let me know if we're discussing the stuff you're interested in, or if you have a suggestion for a topic. I'd love to hear that also. You can email all your thoughts to feedback at cppcast.com.

I'd also appreciate it if you can follow CppCast on Twitter and like CppCast on Facebook. And, of course, you can find all that info and the show notes on the podcast website at cppcast.com